Introduce OpSignature to IR #1838

justinchuby · 2024-08-30T22:56:10Z

Introduce OpSignature accessible from the .op_signature property of all OpLike objects (traced function, onnx function and op). The OpSignature class leverages the IR to represent the signature of an operator, preserving ordering of all inputs and provides easy to work with type representations.

The PR also deprecates the ParamSchema class and properties.

Fixes #1697

The next PR will replace param_schemas usage.

onnxscript/ir/_schemas.py

onnxscript/values.py

codecov · 2024-08-30T23:02:49Z

❌ 19 Tests Failed:

Tests completed	Failed	Passed	Skipped
14911	19	14892	2049

View the full list of 3 ❄️ flaky tests

tests.eager_mode_test.TestEagerModeArguments_0_reference_runtime test_function_all_input_by_kwargs

Flake rate in main: 35.66% (Passed 4270 times, Failed 2367 times)

Stack Traces | 0.002s run time

..../test_torch_nightly/lib/python3.12.../reference/ops/_op.py:91: in run
    res = self._run(x, y)
..../test_torch_nightly/lib/python3.12.../reference/ops/_op.py:139: in _run
    res = (convert_from_ml_dtypes(res[0]),)
..../test_torch_nightly/lib/python3.12.../onnx/reference/custom_element_types.py:50: in convert_from_ml_dtypes
    return array.view(dtype=dtype)
E   ValueError: Changing the dtype of a 0d array is only supported if the itemsize is unchanged

The above exception was the direct cause of the following exception:
tests/eager_mode_test.py:109: in test_function_all_input_by_kwargs
    self.assertEqual(add_with_alpha(this=1.0, other=2.0), 3.0)
onnxscript/values.py:576: in __call__
    return evaluator.default().eval_function(self, args, kwargs)
onnxscript/evaluator.py:307: in eval_function
    result = function.function(*adapted_args, **adapted_kwargs)
tests/eager_mode_test.py:59: in add_with_alpha
    other = op.Mul(other, alpha)
.../onnx_opset/_impl/opset14.py:696: in Mul
    return op(*self._prepare_inputs(schema, A, B))
onnxscript/values.py:304: in __call__
    return evaluator.default().eval(schema, args, kwargs)
onnxscript/evaluator.py:194: in eval
    outputs = self._eval(schema, inputs, attributes, closure)
onnxscript/evaluator.py:524: in _eval
    result = session.run(None, session_run_input)
..../test_torch_nightly/lib/python3.12.../onnx/reference/reference_evaluator.py:599: in run
    outputs = node.run(*inputs, **linked_attributes)
..../test_torch_nightly/lib/python3.12.../reference/ops/_op.py:114: in run
    res = OpRunBinary.run(self, x, y)
..../test_torch_nightly/lib/python3.12.../reference/ops/_op.py:93: in run
    raise TypeError(
E   TypeError: Issues with types &lt;class 'numpy.ndarray'&gt;, &lt;class 'numpy.ndarray'&gt; (binary operator 'Mul').

tests.eager_mode_test.TestEagerModeArguments_0_reference_runtime test_function_attribute_by_positional_args

Flake rate in main: 35.66% (Passed 4270 times, Failed 2367 times)

Stack Traces | 0.002s run time

..../test_torch_nightly/lib/python3.12.../reference/ops/_op.py:91: in run
    res = self._run(x, y)
..../test_torch_nightly/lib/python3.12.../reference/ops/_op.py:139: in _run
    res = (convert_from_ml_dtypes(res[0]),)
..../test_torch_nightly/lib/python3.12.../onnx/reference/custom_element_types.py:50: in convert_from_ml_dtypes
    return array.view(dtype=dtype)
E   ValueError: Changing the dtype of a 0d array is only supported if the itemsize is unchanged

The above exception was the direct cause of the following exception:
tests/eager_mode_test.py:112: in test_function_attribute_by_positional_args
    self.assertEqual(add_with_alpha(1.0, 2.0, 3.0), 7.0)
onnxscript/values.py:576: in __call__
    return evaluator.default().eval_function(self, args, kwargs)
onnxscript/evaluator.py:307: in eval_function
    result = function.function(*adapted_args, **adapted_kwargs)
tests/eager_mode_test.py:59: in add_with_alpha
    other = op.Mul(other, alpha)
.../onnx_opset/_impl/opset14.py:696: in Mul
    return op(*self._prepare_inputs(schema, A, B))
onnxscript/values.py:304: in __call__
    return evaluator.default().eval(schema, args, kwargs)
onnxscript/evaluator.py:194: in eval
    outputs = self._eval(schema, inputs, attributes, closure)
onnxscript/evaluator.py:524: in _eval
    result = session.run(None, session_run_input)
..../test_torch_nightly/lib/python3.12.../onnx/reference/reference_evaluator.py:599: in run
    outputs = node.run(*inputs, **linked_attributes)
..../test_torch_nightly/lib/python3.12.../reference/ops/_op.py:114: in run
    res = OpRunBinary.run(self, x, y)
..../test_torch_nightly/lib/python3.12.../reference/ops/_op.py:93: in run
    raise TypeError(
E   TypeError: Issues with types &lt;class 'numpy.ndarray'&gt;, &lt;class 'numpy.ndarray'&gt; (binary operator 'Mul').

tests.eager_mode_test.TestEagerModeArguments_0_reference_runtime test_function_input_and_attribute_by_kwargs_out_of_order

Flake rate in main: 35.66% (Passed 4270 times, Failed 2367 times)

Stack Traces | 0.003s run time

..../test_torch_nightly/lib/python3.12.../reference/ops/_op.py:91: in run
    res = self._run(x, y)
..../test_torch_nightly/lib/python3.12.../reference/ops/_op.py:139: in _run
    res = (convert_from_ml_dtypes(res[0]),)
..../test_torch_nightly/lib/python3.12.../onnx/reference/custom_element_types.py:50: in convert_from_ml_dtypes
    return array.view(dtype=dtype)
E   ValueError: Changing the dtype of a 0d array is only supported if the itemsize is unchanged

The above exception was the direct cause of the following exception:
tests/eager_mode_test.py:115: in test_function_input_and_attribute_by_kwargs_out_of_order
    self.assertEqual(add_with_alpha(alpha=3.0, other=2.0, this=1.0), 7.0)
onnxscript/values.py:576: in __call__
    return evaluator.default().eval_function(self, args, kwargs)
onnxscript/evaluator.py:307: in eval_function
    result = function.function(*adapted_args, **adapted_kwargs)
tests/eager_mode_test.py:59: in add_with_alpha
    other = op.Mul(other, alpha)
.../onnx_opset/_impl/opset14.py:696: in Mul
    return op(*self._prepare_inputs(schema, A, B))
onnxscript/values.py:304: in __call__
    return evaluator.default().eval(schema, args, kwargs)
onnxscript/evaluator.py:194: in eval
    outputs = self._eval(schema, inputs, attributes, closure)
onnxscript/evaluator.py:524: in _eval
    result = session.run(None, session_run_input)
..../test_torch_nightly/lib/python3.12.../onnx/reference/reference_evaluator.py:599: in run
    outputs = node.run(*inputs, **linked_attributes)
..../test_torch_nightly/lib/python3.12.../reference/ops/_op.py:114: in run
    res = OpRunBinary.run(self, x, y)
..../test_torch_nightly/lib/python3.12.../reference/ops/_op.py:93: in run
    raise TypeError(
E   TypeError: Issues with types &lt;class 'numpy.ndarray'&gt;, &lt;class 'numpy.ndarray'&gt; (binary operator 'Mul').

To view individual test run time comparison to the main branch, go to the Test Analytics Dashboard

onnxscript/ir/_schemas_test.py

onnxscript/values.py

github-actions · 2024-08-30T23:23:05Z

Test Results

24 files ± 0 24 suites ±0 2h 44m 6s ⏱️ + 4m 1s
14 152 tests + 34 11 778 ✅ + 36 2 336 💤 ±0 32 ❌ - 2 6 🔥 ±0
355 111 runs +238 81 240 ✅ +240 273 617 💤 ±0 212 ❌ - 2 42 🔥 ±0

For more details on these failures and errors, see this check.

Results for commit ff1aaa5. ± Comparison against base commit fac4825.

♻️ This comment has been updated with latest results.

onnxscript/ir/_schemas.py

justinchuby · 2024-10-23T23:36:42Z

When the signature property is defined, PyTorch onnx exporter will not generate a new signature, and instead use what is here. However, since it is doing isinstance checks on the signature the exporter itself defines, we get isinstance errors there.

onnxscript/values.py


-    def param_schemas(self) -> Optional[tuple[ParamSchema, ...]]: ...
+    @property
+    def op_signature(self) -> Optional[_schemas.OpSignature]: ...


justinchuby added 2 commits August 30, 2024 22:52

Introduce OpSignature and deprecate ParamSchema

e978e4d

Format

ea90e0e

justinchuby requested review from gramalingam, titaiwangms and xadupre August 30, 2024 22:56

Move

56c1bba

justinchuby added topic: api module: IR Intermediate representation labels Aug 30, 2024

justinchuby commented Aug 30, 2024

View reviewed changes

onnxscript/ir/_schemas.py Outdated Show resolved Hide resolved

Update onnxscript/ir/_schemas.py

0ca7604

justinchuby marked this pull request as draft August 30, 2024 22:59

github-advanced-security bot found potential problems Aug 30, 2024

View reviewed changes

onnxscript/values.py Fixed Show fixed Hide fixed

github-advanced-security bot found potential problems Aug 30, 2024

View reviewed changes

onnxscript/ir/_schemas_test.py Fixed Show fixed Hide fixed

onnxscript/ir/_schemas_test.py Fixed Show fixed Hide fixed

onnxscript/values.py Fixed Show fixed Hide fixed

onnxscript/values.py Fixed Show fixed Hide fixed

justinchuby and others added 2 commits August 31, 2024 15:40

setter

ff1aaa5

Merge branch 'main' into justinchu/op-signature

b88ab17

justinchuby changed the title ~~Introduce OpSignature and deprecate ParamSchema~~ Introduce OpSignature Oct 23, 2024

justinchuby changed the title ~~Introduce OpSignature~~ Introduce OpSignature to IR Oct 23, 2024

justinchuby marked this pull request as ready for review October 23, 2024 23:04

titaiwangms self-assigned this Oct 23, 2024

justinchuby added 3 commits October 23, 2024 23:15

update with the latest pytorch change

c10f5f5

lint

e6c36dd

Warn once

ce95d91

titaiwangms reviewed Oct 23, 2024

View reviewed changes

onnxscript/ir/_schemas.py Show resolved Hide resolved

titaiwangms approved these changes Oct 23, 2024

View reviewed changes

justinchuby added the hold on merging label Oct 23, 2024

titaiwangms removed their assignment Oct 23, 2024

titaiwangms mentioned this pull request Oct 29, 2024

Set 0d shape to python constant tensors #1918

Closed

justinchuby and others added 2 commits October 29, 2024 18:53

Rename to op_signature

1f360c8

Merge branch 'main' into justinchu/op-signature

508fb62

justinchuby removed the hold on merging label Oct 29, 2024

github-advanced-security bot found potential problems Oct 29, 2024

View reviewed changes

onnxscript/values.py

def param_schemas(self) -> Optional[tuple[ParamSchema, ...]]: ...

@property

def op_signature(self) -> Optional[_schemas.OpSignature]: ...

Check notice

Code scanning / CodeQL

Statement has no effect Note

This statement has no effect.

message

24b0760

justinchuby enabled auto-merge (squash) October 29, 2024 19:18

justinchuby merged commit 3e795f2 into main Oct 29, 2024
24 of 41 checks passed

justinchuby deleted the justinchu/op-signature branch October 29, 2024 19:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Introduce OpSignature to IR #1838

Introduce OpSignature to IR #1838

Uh oh!

justinchuby commented Aug 30, 2024 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Aug 30, 2024 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Aug 30, 2024 •

edited

Loading

Uh oh!

Uh oh!

justinchuby commented Oct 23, 2024

Uh oh!

Check notice

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Introduce OpSignature to IR #1838

Introduce OpSignature to IR #1838

Uh oh!

Conversation

justinchuby commented Aug 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Aug 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❌ 19 Tests Failed:

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Aug 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results

Uh oh!

Uh oh!

justinchuby commented Oct 23, 2024

Uh oh!

Check notice

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

justinchuby commented Aug 30, 2024 •

edited

Loading

codecov bot commented Aug 30, 2024 •

edited

Loading

github-actions bot commented Aug 30, 2024 •

edited

Loading