Skip to content

Conversation

@rivkastroh
Copy link

Description

This change extends the ConvInteger implementation to match the ONNX operator spec, which allows both int8 and uint8 for the input tensors:

  • The ONNX ConvInteger schema defines:
    • T1: tensor(int8) or tensor(uint8)
    • T2: tensor(int8) or tensor(uint8)
    • T3: tensor(int32)
  • Previously, only the uint8 × uint8 combination was supported.
  • This PR adds support for all 8-bit combinations:
    • uint8 × uint8 (existing behavior)
    • uint8 × int8
    • int8 × uint8
    • int8 × int8

Motivation and Context

Fixes #24183
Fixes #15888
Fixes #12558
Fixes #3130
Fixes #12362

The ONNX ConvInteger operator schema allows both int8 and uint8 element types for its inputs, but the current implementation only supports uint8 × uint8. This leads to a gap where valid ONNX models using ConvInteger with int8 tensors cannot be executed.
This PR closes that gap by:
Aligning the implementation with the official ConvInteger type constraints.
Enabling models that use int8 (or mixed int8/uint8) for X and W to run without needing operator rewrites or additional custom kernels.
Keeping existing uint8 behavior unchanged, so the change is backwards compatible for current users.

Implementation details

  1. Templated core implementation (ComputeInner)
    The core logic of ConvInteger::Compute is moved into a templated helper:
class ConvInteger : public OpKernel {
 public:
       ...
 private:
  template <typename XT, typename WT>
  Status ComputeInner(OpKernelContext* context) const
};

XT is the element type of X (uint8_t or int8_t).
WT is the element type of W (uint8_t or int8_t).

  1. Zero-point handling
    Zero points are still treated as per-tensor scalar values, with the same validation,
    The values are read via DataRaw() and stored as 8-bit scalars, preserving the previous behavior.
    Interpretation of these raw bytes as signed or unsigned is delegated to the GEMM implementation via explicit signedness flags (see below).

  2. Im2col templated on XT
    The Im2col call now uses the runtime input type XT.

  3. Quantized GEMM with signedness flags:

gemm_shape.AIsSigned = W->IsDataType<int8_t>();
gemm_shape.BIsSigned = X->IsDataType<int8_t>();

AIsSigned and BIsSigned are derived from the runtime types of W and X.
Data for A and B is passed as raw bytes, the GEMM implementation uses the signedness flags to interpret them correctly (In a manner similar to the implementation in MatMulInteger).

  1. Runtime dispatch in Compute()
    The public Compute method becomes a thin dispatcher that selects the appropriate ComputeInner<XT, WT> instantiation based on the actual input types.

In addition, a small set of unit tests is added on top of the existing ConvInteger tests to cover the new type combinations, including cases where the first input tensor contains negative values (for the int8 × int8 path).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment