Skip to content

fix: change PP-OCRv5_server_det HPI backend priority to openvino-first#5032

Open
SankaraVenkatRam wants to merge 1 commit intoPaddlePaddle:developfrom
SankaraVenkatRam:fix/server-det-hpi-backend-priority
Open

fix: change PP-OCRv5_server_det HPI backend priority to openvino-first#5032
SankaraVenkatRam wants to merge 1 commit intoPaddlePaddle:developfrom
SankaraVenkatRam:fix/server-det-hpi-backend-priority

Conversation

@SankaraVenkatRam
Copy link
Copy Markdown

Problem

PP-OCRv5_server_det defaults to paddle_mkldnn backend when HPI is enabled on cpu_x64,
which causes std::exception crashes in multi-threaded serving environments (gRPC, FastAPI
with workers, etc.) due to MKL-DNN thread-safety issues.

Root Cause

The backend priority list in hpi_model_info_collection.json places paddle_mkldnn before
openvino for PP-OCRv5_server_det in cpu_x64 sections.

Evidence

PP-OCRv5_mobile_det already correctly prioritizes openvino in the paddle30 section
and works in the same multi-threaded setup without crashes.

Fix

Swap openvino before paddle_mkldnn for PP-OCRv5_server_det in all 3 cpu_x64 sections
(paddle30, paddle31, paddle311). GPU sections are unaffected.

Reproduction

  1. Initialize PaddleOCR with PP-OCRv5_server_det + enable_hpi=True
  2. Serve via multi-threaded gRPC or FastAPI
  3. Send concurrent image requests → crashes with std::exception

PP-OCRv5_server_det defaults to paddle_mkldnn backend when HPI is enabled
on cpu_x64, which causes std::exception crashes in multi-threaded serving
environments (gRPC, FastAPI workers, etc.) due to MKL-DNN thread-safety
issues.

This change swaps openvino before paddle_mkldnn in all cpu_x64 sections
(paddle30, paddle31, paddle311). GPU sections are unaffected.
@paddle-bot
Copy link
Copy Markdown

paddle-bot bot commented Mar 3, 2026

Thanks for your contribution!

@paddle-bot paddle-bot bot added the contributor External developers label Mar 3, 2026
@SankaraVenkatRam
Copy link
Copy Markdown
Author

It looks like the CI jobs PaddleX-CI-MLU and PaddleX-CI-XPU were cancelled during dependency installation (Error: The operation was canceled.).
Could you please help re-run the CI checks if needed?

@luotao1 luotao1 closed this Mar 9, 2026
@luotao1 luotao1 reopened this Mar 9, 2026
@Bobholamovic
Copy link
Copy Markdown
Member

PaddleX determines the priority of HPI backends according to a unified rule. In our testing environment, backends with better inference performance (i.e., higher speed) are assigned higher priority. As a result, the default backend is typically the one with the best performance, which helps maximize the acceleration effect of HPI. Although, as you mentioned, some backends such as paddle_mkldnn do have thread-safety issues, they are still placed at a higher priority because they provide faster inference according to the rule above. In addition, HPI itself allows users to switch backends by modifying the configuration, and the builtin pipeline serving solution in PaddleX already takes thread-safety considerations into account. Therefore, we do not believe there is a sufficiently strong reason to change the default backend of these models to openvino at this time. Nevertheless, we truly appreciate your suggestion and contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor External developers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants