[Bug]: Exception from inference\src\dev\plugin.cpp:53: invalid broadcast

[Byte247]

OpenVINO Version

2024.6.0

Operating System

Windows System

Device used for inference

CPU

Framework

ONNX

Model used

Dummy - 2 conv2d layers

Issue description

Hello,

I try to use openVino in C++ for model inference.
I build a very simple dummy network (just 2 conv layers) and exported to .onnx
When I try to compile the model using openVino, I get the following error:

"Exception from inference\src\dev\plugin.cpp:53:
invalid broadcast "

I tried everything I could think of: With preprocessing like in https://github.com/openvinotoolkit/openvino/blob/master/samples/cpp/hello_reshape_ssd/main.cpp

and without any. With dynamic output dim and without (this dummy model is fixed anyway).
In the simplest version I just call:

std::shared_ptr<ov::Model> ov_model = core.read_model(path);
ov::CompiledModel compiled_model = core.compile_model(ov_model, device_name); <-- crashes

Can someone help?

Step-by-step reproduction

Dummy net:
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.onnx
import os

-----------------------------

1. Define a simple CNN model

-----------------------------

class SimpleCNN(nn.Module):
def init(self):
super(SimpleCNN, self).init()
# First convolution: 1 input channel (e.g., grayscale), 8 output channels
self.conv1 = nn.Conv2d(in_channels=3, out_channels=8, kernel_size=3, padding=1)
# Second convolution: 8 input channels, 16 output channels
self.conv2 = nn.Conv2d(in_channels=8, out_channels=16, kernel_size=3, padding=1)

def forward(self, x):
    # Conv1 -> ReLU -> MaxPool
    x = F.relu(self.conv1(x))
    x = F.max_pool2d(x, 2)  # halves spatial size
    # Conv2 -> ReLU
    x = F.relu(self.conv2(x))
   
    return x

-----------------------------

2. Instantiate and test model

-----------------------------

model = SimpleCNN()
model.eval() # Set to evaluation mode

Create dummy input (batch_size=1, channels=3, height=256, width=256)

dummy_input = torch.randn(1, 3, 256, 256)

Forward pass (just to verify)

with torch.no_grad():
output = model(dummy_input)
print("Output shape:", output.shape) # Should be [1, 10]

-----------------------------

3. Export to ONNX

-----------------------------

onnx_filename = "dummy_cnn.onnx"

Ensure directory exists

os.makedirs(os.path.dirname(onnx_filename) or ".", exist_ok=True)

torch.onnx.export(
model, # model being run
dummy_input, # model input (or a tuple for multiple inputs)
onnx_filename, # where to save the model
export_params=True, # store trained parameter weights inside the model file
opset_version=16, # ONNX version to export to
do_constant_folding=True, # optimize constant expressions
input_names=['input'], # input tensor name
output_names=['output'], # output tensor name
)

print(f"Model has been exported to {onnx_filename}")

Relevant log output

Issue submission checklist

• I'm reporting an issue. It's not a question.
• I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
• There is reproducer code and related data files such as images, videos, models, etc.

[YuChern-Intel]

I didn't encountered issue when running inference with your created ONNX file.

Sharing the steps here.

1. Generate ONNX model.

dummy_cnn.zip

2. Download OpenVINO 2024.6.0.

3. Extract the files from the zip.

4. Run setupvars.bat to set OpenVINO environment.

5. Run samples\cpp\build_samples_msvc.bat to create C++ sample application.

6. Run the generated benchmark_app.exe with the ONNX file.