Skip to content

dbt Freshness Test Results Not Displayed in DataHub UI Despite Successful Ingestion #15292

@askumar27

Description

@askumar27

dbt freshness test results are successfully ingested into DataHub and stored in the schemaMetadata.lastModified field, but they are not visible in the DataHub UI. This is due to a missing implementation in the GraphQL schema and UI layers, even though the backend data model contains the necessary field.

The ingestion pipeline correctly maps dbt.max_loaded_at values to SchemaMetadata.lastModified, but this information is never exposed to the frontend because:

  • The GraphQL schema (entity.graphql) does not include a lastModified field in the SchemaMetadata type
  • The GraphQL mapper (SchemaMetadataMapper.java) does not map this field
  • The UI cannot access data that isn't exposed through GraphQL

To Reproduce
Steps to reproduce the behavior:

  1. Configure a dbt project with freshness tests on source tables (with loaded_at_field and freshness rules defined)
  2. Run dbt source freshness to generate freshness test results
  3. Ingest dbt metadata into DataHub using the dbt ingestion source
  4. Navigate to a dataset in the DataHub UI that has freshness test results
  5. Check the Schema tab, Properties, and Validations tab
  6. Observe that the max_loaded_at timestamp from dbt freshness tests is not displayed anywhere in the UI

Expected behavior
dbt freshness test results should be visible in the DataHub UI. Expected implementations could include:

  • Display in Schema tab: Show the last modified timestamp (from max_loaded_at) in the Schema tab header or metadata section

Affected Version: v1.2.0

Impact
Users running dbt freshness tests lose visibility into their data freshness monitoring when viewing datasets in DataHub, despite this information being successfully ingested and stored in the backend.

Issue reported on Community Slack: https://datahub.slack.com/archives/CUMUWQU66/p1762404839124209

Metadata

Metadata

Labels

bugBug report

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions