Advanced properties

This page provides additional information about advanced property types available in the Software Catalog. For a general overview of properties, see the Software Catalog overview.

File matching rules

Overview

File matching rules make it possible to read file contents in each repository and extract pattern matches. This allows you to track facts about your services that live directly in source code.

Examples of questions that file matching rules can answer:

• Does this service have a “Local development” section in its README?
• Does this service run unit tests in a GitHub Actions workflow?
• Does this service have an up-to-date version of a critical internal library dependency?
• Does this service use an internally-managed Docker base image?
• Does this service still have any function calls out to a vendor we’re migrating away from?

How to set up a rule

Prerequisites

• Create a Repository Integration for your source code management tool.
• Ensure your catalog entities have aliases set up so DX can match them to repositories.

Steps

1. Navigate to Catalog Settings and create a property of type File matching rule.
2. Click Define rule to open the rule editor.
3. Choose a rule type: File exists, Substring or Regular expression.
4. Enter a file path — this must be a static path (glob patterns and recursive searches are not currently supported).
5. Enter a match expression — the substring or regular expression to look for in the file.
6. Use the sample file contents field to paste in example file content and test your configuration.
7. Use the sample entity selector to quickly test configuration with live data in the catalog.
8. Review the results and confirm the rule behaves as expected.
9. Click Save changes.

How to view results

File matching rule results are visible in the following places:

• Entity detail page — displayed alongside other property values.
• Entities list page — available as a column in the entities table.
• Scorecard checks — use the related_properties column to surface the property in the check result sheet.

The results are displayed in a list, specifying the number of matches (or if the file was not found):

If the result contains one or more matches, clicking on it will open a dialog showing a table with each match:

Rule execution

File matching rules are re-evaluated every 4 hours. Additionally, individual rules are ran immediately after being created or updated.

Scorecard checks with file matching rules

The status field in the entity property value will be one of three strings:

• FOUND_MATCHES
• NO_MATCHES
• FILE_NOT_FOUND

If an entity does not have a repo alias covered by a repository integration, there will be no corresponding record in dx_catalog_entity_properties.

To write a scorecard check based on file matching rules, you can leverage the status field in one of the following two ways:

Identify Good matches — if you want to check that the pattern specified was found (e.g. “README contains a Local development section”):

WITH result_record AS (
  SELECT (ep.value->>'status')::text AS STATUS
  FROM dx_catalog_entity_properties ep
    JOIN dx_catalog_entities e ON e.id = ep.entity_id
    AND e.identifier = $entity_identifier
    JOIN dx_catalog_properties p ON p.id = ep.property_id
    AND p.identifier = 'readme-local-dev-section'
)
SELECT CASE
    WHEN (
      SELECT coalesce(status, 'FILE_NOT_FOUND')
      FROM result_record
    ) = 'FOUND_MATCHES' THEN 'PASS'
    ELSE 'FAIL'
  END AS status,
  'readme-local-dev-section' AS related_properties;

Identify Bad matches — if you want to check that the pattern specified was not found (e.g. “package.json does not contain legacy-vendor-xyz”):

WITH result_record AS (
  SELECT (ep.value->>'status')::text AS STATUS
  FROM dx_catalog_entity_properties ep
    JOIN dx_catalog_entities e ON e.id = ep.entity_id
    AND e.identifier = $entity_identifier
    JOIN dx_catalog_properties p ON p.id = ep.property_id
    AND p.identifier = 'package-references-legacy-vendor-xyz'
)
SELECT CASE
    WHEN (
      SELECT coalesce(status, 'FILE_NOT_FOUND')
      FROM result_record
    ) = 'FOUND_MATCHES' THEN 'FAIL'
    ELSE 'PASS'
  END AS status,
  'package-references-legacy-vendor-xyz' AS related_properties;

System properties

The following property types are system properties, meaning they cannot be edited via the UI or the web API:

• File matching rule — values are computed from repository file contents
• Computed — values are computed from a SQL query