Tests using --update arg cannot be called more than once

[kirecek]

Hello,

I tried to test conftest with remote policies using --update flag. However, it seems update cannot be called multiple times because it conflicts with previously downloaded tests.

First run works:

conftest test --update https://raw.githubusercontent.com/kirecek/policy-test/refs/heads/main/policy/main.rego main.tf

3 tests, 3 passed, 0 warnings, 0 failures, 0 exceptions

Second run fails as policy is already there.

conftest test --update https://raw.githubusercontent.com/kirecek/policy-test/refs/heads/main/policy/main.rego main.tf

Error: running test: update policies: policy file already exists at policy/policy-test, refusing to overwrite

Which is set by downloader code:

conftest/downloader/downloader.go

Line 47 in 609490f

return fmt.Errorf("policy file already exists at %s, refusing to overwrite", targetPath)

I'm wondering if it would make sense to keep those policies in separate (tmp) directory or introduce some flag to clean downloaded policies?

Thanks!

[jpreese]

It may make more sense to just have the --update flag overwrite the contents that exist, if any. I assume your fix for this is just to delete the folder then run update again?

I don't think creating multiple folders would be a good idea, and I'm generally hesitant to add more flags if we can get away without doing so. Especially in this case, where it feels natural to think that --update would give you the most up to date contents at the target destination.

[kirecek]

yes, deleting the folder/file is possible workaround.

I assumed that this prevention is here for cases when content of repository would conflict with content of policies fetched from --update so didn't want to suggest to break this.

But I agree with your last point since it is something that I also expected.

[jalseth]

Reopening due to #1138

[kirecek]

Thinking if it makes sense to keep implementation with temp directory and just fix mentioned use-case (#1136 (comment)). Or was the proposed solution in the PR completely wrong touch? 🤔

And I'm sorry for not testing this "fix" properly for more usage patterns.

[ericsonrumuy7]

Im currently using atlantis and seems like its effecting policy check in atlantis when using new version of conftest, and work around to delete directory does not work as well because atlantis can run in parallel. is there any other work around ?