-
Notifications
You must be signed in to change notification settings - Fork 9
Implement matches override #319
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
1f62639
Fix flags
3bb71cb
Tests
1171740
Simulate re2
63a2a59
Remove comment for now
72149b9
Merge branch 'main' into sayers/re2_workaround
smaye81 e923050
Validate `MessageOneofRule` (#320)
smaye81 9fc2b6f
Add implicit `IGNORE_IF_UNPOPULATED` for `MessageOneofRule` fields (#…
srikrsna-buf 9d5eb7b
Remove flag parsing
40b66d5
Merge branch 'main' into sayers/re2_workaround
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| # Copyright 2023-2025 Buf Technologies, Inc. | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| import re | ||
|
|
||
| import celpy | ||
| from celpy import celtypes | ||
|
|
||
| # Patterns that are supported in Python's re package and not in re2. | ||
| # RE2: https://github.com/google/re2/wiki/syntax | ||
| invalid_patterns = [ | ||
| r"\\[1-9]", # backreference | ||
| r"\\k<\w+>", # backreference | ||
| r"\(\?\=", # lookahead | ||
| r"\(\?\!", # negative lookahead | ||
| r"\(\?\<\=", # lookbehind | ||
| r"\(\?\<\!", # negative lookbehind | ||
| r"\\c[A-Z]", # control character | ||
| r"\\u[0-9a-fA-F]{4}", # UTF-16 code-unit | ||
| r"\\0(?!\d)", # NUL | ||
| r"\[\\b.*\]", # Backspace eg: [\b] | ||
| ] | ||
|
|
||
|
|
||
| def cel_matches(text: celtypes.Value, pattern: celtypes.Value) -> celpy.Result: | ||
| """Return True if the given pattern matches text. False otherwise. | ||
|
|
||
| CEL uses RE2 syntax which diverges from Python re in various ways. Ideally, we | ||
| would use the google-re2 package, which is an extra dep in celpy, but at press | ||
| time it does not provide a pre-built binary for the latest version of Python (3.13) | ||
| which means those using this version will run into many issues. | ||
|
|
||
| Instead of foisting this issue on users, we instead mimic re2 syntax by failing | ||
| to compile the regex for patterns not compatible with re2. | ||
| """ | ||
| if not isinstance(text, celtypes.StringType): | ||
| msg = "invalid argument for text, expected string" | ||
| raise celpy.CELEvalError(msg) | ||
| if not isinstance(pattern, celtypes.StringType): | ||
| msg = "invalid argument for pattern, expected string" | ||
| raise celpy.CELEvalError(msg) | ||
|
|
||
| # Simulate re2 by failing on any patterns not compatible with re2 syntax | ||
| for invalid_pattern in invalid_patterns: | ||
| r = re.search(invalid_pattern, pattern) | ||
| if r is not None: | ||
| msg = f"error evaluating pattern {pattern}, invalid RE2 syntax" | ||
| raise celpy.CELEvalError(msg) | ||
|
|
||
| try: | ||
| m = re.search(pattern, text) | ||
| except re.error as ex: | ||
| return celpy.CELEvalError("match error", ex.__class__, ex.args) | ||
|
|
||
| return celtypes.BoolType(m is not None) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| # Copyright 2023-2025 Buf Technologies, Inc. | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| import unittest | ||
|
|
||
| import celpy | ||
| from celpy import celtypes | ||
|
|
||
| from protovalidate.internal import extra_func | ||
|
|
||
| invalid_patterns = [ | ||
| r"\1", | ||
| r"\k<name>", | ||
| r"Jack(?=Sprat)", | ||
| "Jack(?!Sprat)", | ||
| "(?<=Sprat)Jack", | ||
| "(?<!Sprat)Jack", | ||
| r"\cM\cJ", | ||
| r"\u0041", | ||
| r"\0 \01 \0a \012", | ||
| r"[\b]", | ||
| ] | ||
|
|
||
|
|
||
| class TestMatches(unittest.TestCase): | ||
| def test_invalid_re2_syntax(self): | ||
| for pattern in invalid_patterns: | ||
| cel_pattern = celtypes.StringType(pattern) | ||
| try: | ||
| extra_func.cel_matches(celtypes.StringType("test"), cel_pattern) | ||
| self.fail(f"expected an error on pattern {cel_pattern}") | ||
| except celpy.CELEvalError as e: | ||
| self.assertEqual(str(e), f"error evaluating pattern {cel_pattern}, invalid RE2 syntax") |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
does this match handling in
celpy? or should we pull it out in an f-string + msg like the others?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried to match what celpy was doing here yeah: See code here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ah, gotcha, my searching was not finding that code. works for me!