Skip to content

Conversation

@emmanuelmathot
Copy link
Contributor

Description

This PR adds a new load_queryables function to pypgstac that allows loading queryables from a JSON file. Queryables are a mechanism that allows clients to discover what terms are available for use when writing filter expressions in a STAC API.

The new function:

  • Takes a JSON file path and optional collection IDs as input
  • Parses the queryables definition from the JSON file
  • Determines the appropriate property wrapper based on the type of each property
  • Inserts the queryables into the database
  • Handles conflicts with existing queryables by deleting and reinserting them to ensure triggers are executed

Changes

  • Added load_queryables method to the PgstacCLI class in src/pypgstac/src/pypgstac/pypgstac.py
  • Updated documentation in docs/src/pypgstac.md to include information about the new command
  • Created example files to demonstrate usage:
    • src/pypgstac/examples/sample_queryables.json: A sample queryables definition file
    • src/pypgstac/examples/load_queryables_example.py: An example script showing how to use the function programmatically
  • Added comprehensive unit tests in src/pypgstac/tests/test_queryables.py
  • Updated CHANGELOG.md to include the new feature

Usage

The new command can be used as follows:

# Load queryables for all collections
pypgstac load_queryables queryables.json

# Load queryables for specific collections
pypgstac load_queryables queryables.json --collection_ids collection1,collection2

Or programmatically:

from pypgstac.pypgstac import PgstacCLI

cli = PgstacCLI()
cli.load_queryables("queryables.json", collection_ids="collection1,collection2")

Testing

The implementation has been tested with various scenarios:

  • Loading queryables for all collections
  • Loading queryables for specific collections
  • Updating existing queryables
  • Handling invalid JSON files
  • Handling files with no properties

All tests pass successfully.

@vincentsarago vincentsarago requested a review from bitner March 25, 2025 07:42
Copy link
Collaborator

@bitner bitner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As per my comment above, I think that we should not create an index for every possible queryable. That is likely to degrade performance more than it would help. Indexes should only be created on the most used or most selective queryables. I think adding a index_fields argument that allows for giving a list of fields which indexes should be created for (Or possibly a second command just for setting indexing???)

@emmanuelmathot emmanuelmathot requested a review from bitner March 25, 2025 16:57
@bitner bitner merged commit 67a5d9a into stac-utils:main Mar 27, 2025
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants