Skip to content

[Bug] When adding a new version of model foo - partial parsing runs into a Compilation error 'model.my_dbt_project.bar' depends on 'model.my_dbt_project.foo' which is not in the graph! #11869

@jeremyyeo

Description

@jeremyyeo

Is this a new bug in dbt-core?

  • I believe this is a new bug in dbt-core
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

This is kind of a extension of #8872 but in filed to capture the general behaviour of:

  1. dbt project is parsed / fully parsed.
  2. Users add a new version of an existing model (there's many different patterns that can be taken).
  3. dbt does a partial parse and runs into a compilation error.

Expected Behavior

dbt can partial parse these scenarios without issues.

Steps To Reproduce

All of the scenarios that follow will have a dbt_project.yml setup like so:

# dbt_project.yml
name: analytics
profile: all
version: "1.0.0"

models:
  analytics:
    +materialized: table

Scenario 1

-- models/foo.sql
select 1 c

-- models/bar.sql
select * from {{ ref('foo') }}
$ dbt compile --no-partial-parse
23:56:23  Running with dbt=1.10.5
23:56:23  Registered adapter: postgres=1.9.0
23:56:24  Found 2 models, 549 macros
23:56:24  
23:56:24  Concurrency: 1 threads (target='pg')
23:56:24 

Changes made

  1. Add new model foo_v2.sql.
  2. Add new yaml file - within it specify 2 versions of foo.
-- models/foo.sql
select 1 c

-- models/bar.sql
select * from {{ ref('foo') }}

-- models/foo_v2.sql
select 1 c
# models/schema.yml
models:
  - name: foo
    latest_version: 1
    versions:
      - v: 1
      - v: 2
$ dbt compile
23:56:34  Running with dbt=1.10.5
23:56:34  Registered adapter: postgres=1.9.0
23:56:34  Encountered an error:
Compilation Error
  'model.analytics.bar' depends on 'model.analytics.foo' which is not in the graph!

$ dbt compile --no-partial-parse
00:00:20  Running with dbt=1.10.5
00:00:20  Registered adapter: postgres=1.9.0
00:00:21  Found 3 models, 549 macros
00:00:21  
00:00:21  Concurrency: 1 threads (target='pg')
00:00:21  
00:00:21  While compiling 'bar':
Found an unpinned reference to versioned model 'foo' in project 'analytics'.
Resolving to latest version: foo.v1
A prerelease version 2 is available. It has not yet been marked 'latest' by its maintainer.
When that happens, this reference will resolve to foo.v2 instead.

  Try out v2: {{ ref('analytics', 'foo', v='2') }}
  Pin to  v1: {{ ref('analytics', 'foo', v='1') }}

Scenario 2

-- models/foo.sql
select 1 c

-- models/bar.sql
select 1 c
# models/foo.yml
models:
  - name: foo
 
# models/bar.yml
models:
  - name: bar
    columns:
      - name: c
        tests:
          - relationships:
              to: ref('foo')
              field: c
$ dbt compile --no-partial-parse
01:10:16  Running with dbt=1.10.5
01:10:16  Registered adapter: postgres=1.9.0
01:10:17  Found 2 models, 1 test, 549 macros
01:10:17  
01:10:17  Concurrency: 1 threads (target='pg')
01:10:17

Changes made

  1. foo.sql renamed to foo_v1.sql.
  2. foo_v2.sql created - duplicate of foo_v1.sql.
  3. models/foo.yml modified to declare 2 separate versions of foo.
-- models/foo_v1.sql
select 1 c

-- models/foo_v2.sql
select 1 c

-- models/bar.sql
select 1 c
# models/foo.yml
models:
  - name: foo
    latest_version: 1
    versions:
      - v: 1
      - v: 2

 
# models/bar.yml
models:
  - name: bar
    columns:
      - name: c
        tests:
          - relationships:
              to: ref('foo')
              field: c
$ dbt compile
01:13:41  Running with dbt=1.10.5
01:13:41  Registered adapter: postgres=1.9.0
01:13:41  Encountered an error:
Compilation Error
  'test.analytics.relationships_bar_c__c__ref_foo_.c2bbacad87' depends on 'model.analytics.foo' which is not in the graph

$ dbt compile --no-partial-parse
01:13:53  Running with dbt=1.10.5
01:13:53  Registered adapter: postgres=1.9.0
01:13:54  Found 3 models, 1 test, 549 macros
01:13:54  
01:13:54  Concurrency: 1 threads (target='pg')
01:13:54  
01:13:54  While compiling 'relationships_bar_c__c__ref_foo_':
Found an unpinned reference to versioned model 'foo' in project 'analytics'.
Resolving to latest version: foo.v1
A prerelease version 2 is available. It has not yet been marked 'latest' by its maintainer.
When that happens, this reference will resolve to foo.v2 instead.

  Try out v2: {{ ref('analytics', 'foo', v='2') }}
  Pin to  v1: {{ ref('analytics', 'foo', v='1') }}

Relevant log output

NA

Environment

- OS: macOS
- Python: 3.11.9
- dbt: 1.10.5

Which database adapter are you using with dbt?

postgres

Additional Context

The current workaround is to essentially do a full parse via --no-partial-parse as demonstrated.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions