-
Notifications
You must be signed in to change notification settings - Fork 2.1k
Labels
Description
Is this a new bug in dbt-core?
- I believe this is a new bug in dbt-core
- I have searched the existing issues, and I could not find an existing issue for this bug
Current Behavior
This is kind of a extension of #8872 but in filed to capture the general behaviour of:
- dbt project is parsed / fully parsed.
- Users add a new version of an existing model (there's many different patterns that can be taken).
- dbt does a partial parse and runs into a compilation error.
Expected Behavior
dbt can partial parse these scenarios without issues.
Steps To Reproduce
All of the scenarios that follow will have a dbt_project.yml
setup like so:
# dbt_project.yml
name: analytics
profile: all
version: "1.0.0"
models:
analytics:
+materialized: table
Scenario 1
-- models/foo.sql
select 1 c
-- models/bar.sql
select * from {{ ref('foo') }}
$ dbt compile --no-partial-parse
23:56:23 Running with dbt=1.10.5
23:56:23 Registered adapter: postgres=1.9.0
23:56:24 Found 2 models, 549 macros
23:56:24
23:56:24 Concurrency: 1 threads (target='pg')
23:56:24
Changes made
- Add new model
foo_v2.sql
. - Add new yaml file - within it specify 2 versions of foo.
-- models/foo.sql
select 1 c
-- models/bar.sql
select * from {{ ref('foo') }}
-- models/foo_v2.sql
select 1 c
# models/schema.yml
models:
- name: foo
latest_version: 1
versions:
- v: 1
- v: 2
$ dbt compile
23:56:34 Running with dbt=1.10.5
23:56:34 Registered adapter: postgres=1.9.0
23:56:34 Encountered an error:
Compilation Error
'model.analytics.bar' depends on 'model.analytics.foo' which is not in the graph!
$ dbt compile --no-partial-parse
00:00:20 Running with dbt=1.10.5
00:00:20 Registered adapter: postgres=1.9.0
00:00:21 Found 3 models, 549 macros
00:00:21
00:00:21 Concurrency: 1 threads (target='pg')
00:00:21
00:00:21 While compiling 'bar':
Found an unpinned reference to versioned model 'foo' in project 'analytics'.
Resolving to latest version: foo.v1
A prerelease version 2 is available. It has not yet been marked 'latest' by its maintainer.
When that happens, this reference will resolve to foo.v2 instead.
Try out v2: {{ ref('analytics', 'foo', v='2') }}
Pin to v1: {{ ref('analytics', 'foo', v='1') }}
Scenario 2
-- models/foo.sql
select 1 c
-- models/bar.sql
select 1 c
# models/foo.yml
models:
- name: foo
# models/bar.yml
models:
- name: bar
columns:
- name: c
tests:
- relationships:
to: ref('foo')
field: c
$ dbt compile --no-partial-parse
01:10:16 Running with dbt=1.10.5
01:10:16 Registered adapter: postgres=1.9.0
01:10:17 Found 2 models, 1 test, 549 macros
01:10:17
01:10:17 Concurrency: 1 threads (target='pg')
01:10:17
Changes made
foo.sql
renamed tofoo_v1.sql
.foo_v2.sql
created - duplicate offoo_v1.sql
.models/foo.yml
modified to declare 2 separate versions offoo
.
-- models/foo_v1.sql
select 1 c
-- models/foo_v2.sql
select 1 c
-- models/bar.sql
select 1 c
# models/foo.yml
models:
- name: foo
latest_version: 1
versions:
- v: 1
- v: 2
# models/bar.yml
models:
- name: bar
columns:
- name: c
tests:
- relationships:
to: ref('foo')
field: c
$ dbt compile
01:13:41 Running with dbt=1.10.5
01:13:41 Registered adapter: postgres=1.9.0
01:13:41 Encountered an error:
Compilation Error
'test.analytics.relationships_bar_c__c__ref_foo_.c2bbacad87' depends on 'model.analytics.foo' which is not in the graph
$ dbt compile --no-partial-parse
01:13:53 Running with dbt=1.10.5
01:13:53 Registered adapter: postgres=1.9.0
01:13:54 Found 3 models, 1 test, 549 macros
01:13:54
01:13:54 Concurrency: 1 threads (target='pg')
01:13:54
01:13:54 While compiling 'relationships_bar_c__c__ref_foo_':
Found an unpinned reference to versioned model 'foo' in project 'analytics'.
Resolving to latest version: foo.v1
A prerelease version 2 is available. It has not yet been marked 'latest' by its maintainer.
When that happens, this reference will resolve to foo.v2 instead.
Try out v2: {{ ref('analytics', 'foo', v='2') }}
Pin to v1: {{ ref('analytics', 'foo', v='1') }}
Relevant log output
NA
Environment
- OS: macOS
- Python: 3.11.9
- dbt: 1.10.5
Which database adapter are you using with dbt?
postgres
Additional Context
The current workaround is to essentially do a full parse via --no-partial-parse
as demonstrated.
josescuderoh and shm-alma