Skip to content

Conversation

@andy-k-improving
Copy link
Contributor

@andy-k-improving andy-k-improving commented Jan 1, 2025

Description

Introduce a new PPL command expression geoip, to perform geo-spatial information lookup with the provided IPv4 || IPv6 addresses, result of the lookup is formatted into a tuple with attribute as key and location detail as value.

In this particular setting, SQL plugin will act as a thin client, by relaying the IPEnrichment request to OpenSearch Geo-Spatial plugin, WITHIN the same cluster.
Detail implementation and interface that exposed on Geo-Spatial side can be found:
opensearch-project/geospatial#700

Internally this functionality is achieved by:

  • Adding an no-op OpenSearchFunctionExpression marker to identify this is an expression has no default implement on other runtime (Ex: Prometheus)
  • Update OpenSearchIndex in order to provide an OpenSearch specific handler for eval operator and its expressions, when OS being used as the storage engine.

During runtime, all eval expressions, will being passed to OpenSearchIndex.visitEval( ), then OpenSearchEvalOperator class will pick up the call, by evaluating all eval expression as it is, and then handle all occasion of OpenSearchFunctionExpression separately, by reading the function name and argument, and execute the appropriate business logic.

Marker class OpenSearchFunctionExpression is being used in this case because the actual implementation require runtime OpenSearch client connectivity, however core module is mean to be generic, hence this workaround is being deployed, by tagging it as OpenSearchFunctionExpression on core and only handle it on the opensearch Cradle module .

Related Issues

Resolves: #3037

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • New functionality has javadoc added.
  • New functionality has a user manual doc added.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@andy-k-improving
Copy link
Contributor Author

As per the offline discussion, I have separated out the integration related changes into #3244, in order to minimise the diff.

Signed-off-by: Andy Kwok <[email protected]>
Signed-off-by: Andy Kwok <[email protected]>
Signed-off-by: Andy Kwok <[email protected]>
Signed-off-by: Andy Kwok <[email protected]>
Signed-off-by: Andy Kwok <[email protected]>
Signed-off-by: Andy Kwok <[email protected]>
Copy link
Contributor

@normanj-bitquill normanj-bitquill left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

andy-k-improving and others added 3 commits January 28, 2025 15:14
Signed-off-by: Andy Kwok <[email protected]>
Co-authored-by: Andrew Carbonetto <[email protected]>
Signed-off-by: Andy Kwok <[email protected]>
…/physical/OpenSearchEvalOperator.java

Co-authored-by: Andrew Carbonetto <[email protected]>
Signed-off-by: Andy Kwok <[email protected]>
Signed-off-by: Andy Kwok <[email protected]>
Signed-off-by: Andy Kwok <[email protected]>
Signed-off-by: Andy Kwok <[email protected]>
Signed-off-by: Andy Kwok <[email protected]>
Copy link
Collaborator

@acarbonetto acarbonetto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@acarbonetto acarbonetto merged commit 3bf19ef into opensearch-project:main Jan 30, 2025
15 checks passed
@acarbonetto acarbonetto deleted the ft-ak-iplocation-cmd branch January 30, 2025 22:49
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/sql/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/sql/backport-2.x
# Create a new branch
git switch --create backport/backport-3228-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 3bf19ef6d5fc931d89b0b9ec08840ec643f8e790
# Push it to GitHub
git push --set-upstream origin backport/backport-3228-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/sql/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-3228-to-2.x.

@yuancu yuancu mentioned this pull request May 6, 2025
7 tasks
penghuo pushed a commit that referenced this pull request Jun 16, 2025
Signed-off-by: Andy Kwok <[email protected]>
Co-authored-by: Andrew Carbonetto <[email protected]>
Signed-off-by: xinyual <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE]Add iplocation function to PPL for IP address geolocation

5 participants