deepseek-vl2 RuntimeError: input shape dimension cannot be broadcasted

[cc0708]

Hi,

I'm following the steps to run deepseek-vl2 example, but got the following error. Would anyone be able to help?

Environment: Python 3.11; BMG
Steps to reproduce: https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/deepseek-vl2/deepseek-vl2.ipynb

[Wan-Intel]

I have installed the latest OpenVINO™ Notebook, and I have validated the Visual-language assistant using DeepSeek-VL2 and OpenVINO notebook.

Please remove existing OpenVINO™ Notebook and re-install OpenVINO™ Notebook. Please refer to OpenVINO™ Notebook Installation Guide here.

If the issue persists, could you please tell us which cell did you encountered the runtime error?

[cc0708]

Thank you for your response! I've verified that I'm using the latest OpenVINO™ Notebook (commit ID: 21cca8b). And I've tried both deepseek-vl2-small and deepseek-vl2-tiny models. The runtime error occurs in cell 12: Visual Grounding.

Would you be able to verify whether this issue reproduces on your end? @Wan-Intel

pip list info:

Package                   Version        Build
------------------------- -------------- -----
about-time                4.2.1
accelerate                1.7.0
aiofiles                  24.1.0
alive-progress            3.2.0
annotated-types           0.7.0
anyio                     4.9.0
argon2-cffi               25.1.0
argon2-cffi-bindings      21.2.0
arrow                     1.3.0
asttokens                 3.0.0
async-lru                 2.0.5
attrdict                  2.0.1
attrs                     25.3.0
autograd                  1.8.0
babel                     2.17.0
beautifulsoup4            4.13.4
bleach                    6.2.0
certifi                   2025.4.26
cffi                      1.17.1
charset-normalizer        3.4.2
click                     8.2.1
cma                       4.2.0
colorama                  0.4.6
comm                      0.2.2
contourpy                 1.3.2
cycler                    0.12.1
debugpy                   1.8.14
decorator                 5.2.1
defusedxml                0.7.1
Deprecated                1.2.18
dill                      0.4.0
einops                    0.8.1
executing                 2.2.0
fastapi                   0.115.12
fastjsonschema            2.21.1
ffmpy                     0.6.0
filelock                  3.18.0
fonttools                 4.58.2
fqdn                      1.5.1
fsspec                    2025.5.1
gradio                    5.33.0
gradio_client             1.10.2
grapheme                  0.6.0
groovy                    0.1.2
h11                       0.16.0
hf-xet                    1.1.3
httpcore                  1.0.9
httpx                     0.28.1
huggingface-hub           0.32.4
idna                      3.10
ipykernel                 6.29.5
ipython                   9.3.0
ipython_pygments_lexers   1.1.1
ipywidgets                8.1.7
isoduration               20.11.0
jedi                      0.19.2
Jinja2                    3.1.6
joblib                    1.5.1
json5                     0.12.0
jsonpointer               3.0.0
jsonschema                4.24.0
jsonschema-specifications 2025.4.1
jstyleson                 0.0.2
jupyter_client            8.6.3
jupyter_core              5.8.1
jupyter-events            0.12.0
jupyter-lsp               2.2.5
jupyter_server            2.16.0
jupyter_server_terminals  0.5.3
jupyterlab                4.4.3
jupyterlab_pygments       0.3.0
jupyterlab_server         2.27.3
jupyterlab_widgets        3.0.15
kiwisolver                1.4.8
latex2mathml              3.78.0
Markdown                  3.8
markdown-it-py            3.0.0
MarkupSafe                3.0.2
matplotlib                3.10.3
matplotlib-inline         0.1.7
mdtex2html                1.3.1
mdurl                     0.1.2
mistune                   3.1.3
mpmath                    1.3.0
natsort                   8.4.0
nbclient                  0.10.2
nbconvert                 7.16.6
nbformat                  5.10.4
nest-asyncio              1.6.0
networkx                  3.4.2
ninja                     1.11.1.4
nncf                      2.16.0
notebook_shim             0.2.4
numpy                     2.2.6
openvino                  2025.1.0       18503
openvino-telemetry        2025.1.0
orjson                    3.10.18
overrides                 7.7.0
packaging                 25.0
pandas                    2.2.3
pandocfilters             1.5.1
parso                     0.8.4
pexpect                   4.9.0
pillow                    11.2.1
pip                       25.1.1
platformdirs              4.3.8
prometheus_client         0.22.1
prompt_toolkit            3.0.51
psutil                    7.0.0
ptyprocess                0.7.0
pure_eval                 0.2.3
pycparser                 2.22
pydantic                  2.11.5
pydantic_core             2.33.2
pydot                     3.0.4
pydub                     0.25.1
Pygments                  2.19.1
pymoo                     0.6.1.5
pyparsing                 3.2.3
pypinyin                  0.54.0
python-dateutil           2.9.0.post0
python-json-logger        3.3.0
python-multipart          0.0.20
pytz                      2025.2
PyYAML                    6.0.2
pyzmq                     26.4.0
referencing               0.36.2
regex                     2024.11.6
requests                  2.32.3
rfc3339-validator         0.1.4
rfc3986-validator         0.1.1
rich                      14.0.0
rpds-py                   0.25.1
ruff                      0.11.13
safehttpx                 0.1.6
safetensors               0.5.3
scikit-learn              1.7.0
scipy                     1.15.3
semantic-version          2.10.0
Send2Trash                1.8.3
sentencepiece             0.2.0
setuptools                80.9.0
shellingham               1.5.4
six                       1.17.0
sniffio                   1.3.1
soupsieve                 2.7
stack-data                0.6.3
starlette                 0.46.2
sympy                     1.14.0
tabulate                  0.9.0
terminado                 0.18.1
threadpoolctl             3.6.0
tiktoken                  0.9.0
timm                      1.0.15
tinycss2                  1.4.0
tokenizers                0.21.1
tomlkit                   0.13.3
torch                     2.7.1+cpu
torchvision               0.22.1+cpu
tornado                   6.5.1
tqdm                      4.67.1
traitlets                 5.14.3
transformers              4.52.4
typer                     0.16.0
types-python-dateutil     2.9.0.20250516
typing_extensions         4.14.0
typing-inspection         0.4.1
tzdata                    2025.2
uri-template              1.3.0
urllib3                   2.4.0
uvicorn                   0.34.3
wcwidth                   0.2.13
webcolors                 24.11.1
webencodings              0.5.1
websocket-client          1.8.0
websockets                15.0.1
wheel                     0.45.1
widgetsnbextension        4.0.14
wrapt                     1.17.2

[Wan-Intel]

Results:

Could you please provide the following information with us to further investigate the issue?

• Hardware specifications
• Host Operating System

[cc0708]

CPU model name: Intel(R) Core(TM) Ultra 9 285K
GPU: B580(DT6)
OS: Ubuntu 24.04.1 LTS
Kernel: 6.12.0-061200-generic

The broadcast issue happened when I set the inference device value to GPU, but it works with CPU.

[Wan-Intel]

I also encountered the following error when running the deepseek-vl2 notebook on GPU plugin:

RuntimeError: While validating node 'opset3::Broadcast Broadcast_455309 () -> ()' with friendly_name 'Broadcast_455309':
Input shape dimension equal 8673 cannot be broadcasted (numpy mode) to 8676. Allowed input dimension value would be 1 or 8676

I will check with relevant team and we will provide an update here as soon as possible.