-
Notifications
You must be signed in to change notification settings - Fork 188
Description
Hello, this is a question rather than an issue. I'm becoming more familiar with the code base and have been testing changes that I've been making locally. I've been having trouble figuring out how I can test compile step code changes locally, though.
I've completed the setup as described in the contributing documentation and generally things are working well. However, I haven't managed to figure out how to test changes regarding the compile step.
Here's what I've done:
- Create a new project with
./scripts/run init bigquery my-project --default-database my-project --default-location EU - Create credentials file with
./scripts/run init-creds bigquery my-project - Add the following
test.sqlxfile tomy-project/definitions:
config {
type: "table",
testProperty: "test"
}
SELECT
CURRENT_TIMESTAMP()
- Make some changes to
table.tsthat I would expect to be reflected when compiling this file (console.logs, updating strings, etc.) - Run
./scripts/run compile my-project - Instead of seeing the changes that I would expect, it appears that the compile command is using code from
/my-project/node_modules/@dataform/core/bundle.js(so the published version of Dataform), rather than using the code that I modified. Below is the output that I see after running the command:
Compiling...
Compiled 1 action(s).
1 dataset(s):
dataform.test [table]
Compilation errors:
definitions/test.sqlx: Error: Unexpected property "testProperty" in table config. Supported properties are: ["type","disabled","protected","name","redshift","bigquery","snowflake","sqldatawarehouse","presto","tags","uniqueKey","dependencies","hermetic","schema","assertions","database","columns","description","materialized"]
at Object.t.checkExcessProperties (/Users/matthewkocak/Documents/GitHub/dataform/my-project/node_modules/@dataform/core/bundle.js:1:209544)
at a.config (/Users/matthewkocak/Documents/GitHub/dataform/my-project/node_modules/@dataform/core/bundle.js:1:252775)
at t.Session.sqlxAction (/Users/matthewkocak/Documents/GitHub/dataform/my-project/node_modules/@dataform/core/bundle.js:1:383183)
at /Users/matthewkocak/Documents/GitHub/dataform/my-project/definitions/test.sqlx:1:24
at VM2 Wrapper.apply (/Users/matthewkocak/Documents/GitHub/dataform/bazel-bin/packages/@dataform/cli/bin.sh.runfiles/npm/node_modules/vm2/lib/bridge.js:485:11)
at NodeVM.run (/Users/matthewkocak/Documents/GitHub/dataform/bazel-bin/packages/@dataform/cli/bin.sh.runfiles/npm/node_modules/vm2/lib/nodevm.js:497:23)
at CustomResolver.loadJS (/Users/matthewkocak/Documents/GitHub/dataform/bazel-bin/packages/@dataform/cli/bin.sh.runfiles/npm/node_modules/vm2/lib/resolver.js:210:6)
at CustomResolver.loadJS (/Users/matthewkocak/Documents/GitHub/dataform/bazel-bin/packages/@dataform/cli/bin.sh.runfiles/npm/node_modules/vm2/lib/resolver-compat.js:63:65)
at /Users/matthewkocak/Documents/GitHub/dataform/bazel-bin/packages/@dataform/cli/bin.sh.runfiles/npm/node_modules/vm2/lib/resolver.js:84:14
at VM2 Wrapper.apply (/Users/matthewkocak/Documents/GitHub/dataform/bazel-bin/packages/@dataform/cli/bin.sh.runfiles/npm/node_modules/vm2/lib/bridge.js:485:11)
I expected the error, since I added the testProperty to test.sqlx, but I expected the output text to reflect string changes that I had made in the table.ts. In the call stack, you can see that /my-project/node_modules/@dataform/core/bundle.js is being used to complete the compile step, rather than my local files.
When I run the run command using ./scripts/run run my-project, it appears that the compile step in this command still uses the published version of Dataform, but then the run/execution step of this command uses my local files. So I'm able to see the expected changes that I made locally for this step in the output.
Long post where I'm sure I'm just testing things incorrectly in my local development environment. How can I run the compile command in my local environment and have it reflect changes that I had made to the code?