paddle_ocr example times out

[standuprey]

Hi,

When I run the example with more than a few pictures (>3) I get the following error:

[HailoRT] [error] CHECK failed - Waiting for async job to finish has failed with timeout (10000ms)
[HailoRT] [error] CHECK_SUCCESS failed with status=HAILO_TIMEOUT(4)
[HailoRT] [warning] Failed to acquire buffer because the buffer pool is empty. This could be caused by uneven reading and writing speeds, with a short user-defined timeout. (timeout=10000ms)
[HailoRT] [error] Non-recoverable Async Infer Pipeline error. status error code: HAILO_TIMEOUT(4)
[HailoRT] [error] Shutting down the pipeline with status HAILO_TIMEOUT(4)

Here is the command I used:
./paddle_ocr.py -n ocr_det.hef ocr.hef -i input/ --show-fp
(tried to use -b 2 or -b 5 but it didn't help)

Here is the content of the input:
input.zip

I am using a Pi5 with Hailo hat, and I PCIe 3 activated

Here is the output of hailortcli fw-control identify:

Executing on device: 0001:01:00.0
Identifying board
Control Protocol Version: 2
Firmware Version: 4.20.0 (release,app,extended context switch buffer)
Logger Version: 0
Board Name: Hailo-8
Device Architecture: HAILO8
Serial Number: <N/A>
Part Number: <N/A>
Product Name: <N/A>

[majedabu]

Hi Stan,
I tested locally with your input files and didn’t encounter the timeout issue. Could you please try running with the latest firmware version (4.22)?

[standuprey]

Ok, thank you, it seems like upgrading to 4.22 does solve this issue, but creates another one. We are running an object detection pipeline before running the ocr and this is now broken. What previously worked (with 4.20) now returns:

Aug 21 15:15:17 raspberrypi bash[12695]: terminate called after throwing an instance of 'std::invalid_argument'
Aug 21 15:15:17 raspberrypi bash[12695]:   what():  Output tensor best/yolov8_nms_postprocess is not an NMS type
Aug 21 15:15:17 raspberrypi bash[12695]: --------------------------------------
Aug 21 15:15:17 raspberrypi bash[12695]: C++ Traceback (most recent call last):
Aug 21 15:15:17 raspberrypi bash[12695]: --------------------------------------
Aug 21 15:15:17 raspberrypi bash[12695]: 0   filter_letterbox
Aug 21 15:15:17 raspberrypi bash[12695]: 1   filter
Aug 21 15:15:17 raspberrypi bash[12695]: 2   HailoNMSDecode::HailoNMSDecode(std::shared_ptr<HailoTensor>, std::map<unsigned char, std::string, std::less<unsigned char>, std::allocator<std::pair<unsigned char const, std::string > > >&, float, unsigned int, bool)

the model was compiled with NMS, here is the log of the onnx to hef conversion:

xxx:~$ hailo parser onnx best.onnx --hw-arch hailo8
[info] No GPU chosen, Selected GPU 0
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1755811062.500358    9861 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1755811062.507213    9861 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
[info] Current Time: 21:17:49, 08/21/25
[info] CPU: Architecture: x86_64, Model: Intel(R) Xeon(R) CPU @ 2.20GHz, Number Of Cores: 12, Utilization: 0.0%
[info] Memory: Total: 83GB, Available: 81GB
[info] System info: OS: Linux, Kernel: 6.8.0-1034-gcp
[info] Hailo DFC Version: 3.32.0
[info] HailoRT Version: Not Installed
[info] PCIe: No Hailo PCIe device was found
[info] Running `hailo parser onnx best.onnx --hw-arch hailo8`
[info] Translation started on ONNX model best
[info] Restored ONNX model best (completion time: 00:00:00.25)
[info] Extracted ONNXRuntime meta-data for Hailo model (completion time: 00:00:00.93)
[info] Simplified ONNX model for a parsing retry attempt (completion time: 00:00:02.17)
Parsing failed with recommendations for end node names: ['/model.23/Concat_3'].
Would you like to parse again with the recommendation? (y/n)
y
[info] According to recommendations, retrying parsing with end node names: ['/model.23/Concat_3'].
[info] Translation started on ONNX model best
[info] Restored ONNX model best (completion time: 00:00:00.18)
[info] Extracted ONNXRuntime meta-data for Hailo model (completion time: 00:00:00.84)
[info] NMS structure of yolov8 (or equivalent architecture) was detected.
[info] In order to use HailoRT post-processing capabilities, these end node names should be used: /model.23/cv2.0/cv2.0.2/Conv /model.23/cv3.0/cv3.0.2/Conv /model.23/cv2.1/cv2.1.2/Conv /model.23/cv3.1/cv3.1.2/Conv /model.23/cv2.2/cv2.2.2/Conv /model.23/cv3.2/cv3.2.2/Conv.
[info] Start nodes mapped from original model: 'images': 'best/input_layer1'.
[info] End nodes mapped from original model: '/model.23/Concat_3'.
[info] Translation completed on ONNX model best (completion time: 00:00:02.12)
Would you like to parse the model again with the mentioned end nodes and add nms postprocess command to the model script? (y/n)
y
[info] Translation started on ONNX model best
[info] Restored ONNX model best (completion time: 00:00:00.19)
[info] Extracted ONNXRuntime meta-data for Hailo model (completion time: 00:00:00.89)
[info] NMS structure of yolov8 (or equivalent architecture) was detected.
[info] In order to use HailoRT post-processing capabilities, these end node names should be used: /model.23/cv2.0/cv2.0.2/Conv /model.23/cv3.0/cv3.0.2/Conv /model.23/cv2.1/cv2.1.2/Conv /model.23/cv3.1/cv3.1.2/Conv /model.23/cv3.2/cv3.2.2/Conv /model.23/cv2.2/cv2.2.2/Conv.
[info] Start nodes mapped from original model: 'images': 'best/input_layer1'.
[info] End nodes mapped from original model: '/model.23/cv2.0/cv2.0.2/Conv', '/model.23/cv3.0/cv3.0.2/Conv', '/model.23/cv2.1/cv2.1.2/Conv', '/model.23/cv3.1/cv3.1.2/Conv', '/model.23/cv2.2/cv2.2.2/Conv', '/model.23/cv3.2/cv3.2.2/Conv'.
[info] Translation completed on ONNX model best (completion time: 00:00:02.40)
[info] Appending model script commands to best from string
[info] Added nms postprocess command to model script.
[info] Saved HAR to: /home/stanislas.duprey/tmp/best.har
xxx:~$ hailo optimize best.har --calib-set-path calib_set.npy --hw-arch hailo8
[info] No GPU chosen, Selected GPU 0
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1755811216.994722   10446 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1755811217.001659   10446 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
[info] Current Time: 21:20:22, 08/21/25
[info] CPU: Architecture: x86_64, Model: Intel(R) Xeon(R) CPU @ 2.20GHz, Number Of Cores: 12, Utilization: 0.1%
[info] Memory: Total: 83GB, Available: 81GB
[info] System info: OS: Linux, Kernel: 6.8.0-1034-gcp
[info] Hailo DFC Version: 3.32.0
[info] HailoRT Version: Not Installed
[info] PCIe: No Hailo PCIe device was found
[info] Running `hailo optimize best.har --calib-set-path calib_set.npy --hw-arch hailo8`
[info] For NMS architecture yolov8 the default engine is cpu. For other engine please use the 'engine' flag in the nms_postprocess model script command. If the NMS has been added during parsing, please parse the model again without confirming the addition of the NMS, and add the command manually with the desired engine.
[info] The layer best/conv51 was detected as reg_layer.
[info] The layer best/conv54 was detected as cls_layer.
[info] The layer best/conv62 was detected as reg_layer.
[info] The layer best/conv65 was detected as cls_layer.
[info] The layer best/conv77 was detected as reg_layer.
[info] The layer best/conv80 was detected as cls_layer.
[info] Using the default score threshold of 0.001 (range is [0-1], where 1 performs maximum suppression) and IoU threshold of 0.7 (range is [0-1], where 0 performs maximum suppression).
Changing the values is possible using the nms_postprocess model script command.
[info] The activation function of layer best/conv54 was replaced by a Sigmoid
[info] The activation function of layer best/conv65 was replaced by a Sigmoid
[info] The activation function of layer best/conv80 was replaced by a Sigmoid
[info] Starting Model Optimization
I0000 00:00:1755811228.233650   10446 gpu_device.cc:2022] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 38380 MB memory:  -> device: 0, name: NVIDIA A100-SXM4-40GB, pci bus id: 0000:00:04.0, compute capability: 8.0
I0000 00:00:1755811228.717185   10446 gpu_device.cc:2022] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 38380 MB memory:  -> device: 0, name: NVIDIA A100-SXM4-40GB, pci bus id: 0000:00:04.0, compute capability: 8.0
[warning] Reducing optimization level to 1 (the accuracy won't be optimized and compression won't be used) because there's less data than the recommended amount (1024)
[info] Model received quantization params from the hn
I0000 00:00:1755811229.768961   10446 gpu_device.cc:2022] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 38380 MB memory:  -> device: 0, name: NVIDIA A100-SXM4-40GB, pci bus id: 0000:00:04.0, compute capability: 8.0
I0000 00:00:1755811230.263196   10446 gpu_device.cc:2022] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 38380 MB memory:  -> device: 0, name: NVIDIA A100-SXM4-40GB, pci bus id: 0000:00:04.0, compute capability: 8.0
[info] MatmulDecompose skipped
[info] Starting Mixed Precision
[info] Model Optimization Algorithm Mixed Precision is done (completion time is 00:00:00.92)
[info] LayerNorm Decomposition skipped
[info] Starting Statistics Collector
[info] Using dataset with 64 entries for calibration
Calibration:   0%|                                                                                                                                                                                                                                                                                   | 0/64 [00:00<?, ?entries/s]I0000 00:00:1755811285.870083   10629 cuda_dnn.cc:529] Loaded cuDNN version 90900
Could not load symbol cuFuncGetName. Error: /lib/x86_64-linux-gnu/libcuda.so.1: undefined symbol: cuFuncGetName
Calibration: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 64/64 [00:45<00:00,  1.40entries/s]
[info] Model Optimization Algorithm Statistics Collector is done (completion time is 00:00:48.81)
[info] Starting Fix zp_comp Encoding
[info] Model Optimization Algorithm Fix zp_comp Encoding is done (completion time is 00:00:00.00)
[info] Starting Matmul Equalization
[info] Model Optimization Algorithm Matmul Equalization is done (completion time is 00:00:00.03)
[info] Starting MatmulDecomposeFix
[info] Model Optimization Algorithm MatmulDecomposeFix is done (completion time is 00:00:00.00)
[info] activation fitting started for best/reduce_sum_softmax1/act_op
I0000 00:00:1755811361.690642   10446 gpu_device.cc:2022] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 38380 MB memory:  -> device: 0, name: NVIDIA A100-SXM4-40GB, pci bus id: 0000:00:04.0, compute capability: 8.0
I0000 00:00:1755811365.219133   10446 gpu_device.cc:2022] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 38380 MB memory:  -> device: 0, name: NVIDIA A100-SXM4-40GB, pci bus id: 0000:00:04.0, compute capability: 8.0
[info] Finetune encoding skipped
[info] Starting Bias Correction
[info] The algorithm Bias Correction will use up to 2.31 GB of storage space
[info] Using dataset with 64 entries for Bias Correction
Bias Correction:   0%|                                                                                                                                                                                                                                                | 0/99 [00:00<?, ?blocks/s, Layers=['best/conv1_output_0']]I0000 00:00:1755811384.999969   10928 cuda_dnn.cc:529] Loaded cuDNN version 90900
Could not load symbol cuFuncGetName. Error: /lib/x86_64-linux-gnu/libcuda.so.1: undefined symbol: cuFuncGetName
Bias Correction: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 99/99 [05:16<00:00,  3.19s/blocks, Layers=['best/postprocess_output_layer']]
[info] Model Optimization Algorithm Bias Correction is done (completion time is 00:05:19.42)
[info] Adaround skipped
[info] Quantization-Aware Fine-Tuning skipped
I0000 00:00:1755811733.066746   10446 gpu_device.cc:2022] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 38380 MB memory:  -> device: 0, name: NVIDIA A100-SXM4-40GB, pci bus id: 0000:00:04.0, compute capability: 8.0
I0000 00:00:1755811736.668816   10446 gpu_device.cc:2022] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 38380 MB memory:  -> device: 0, name: NVIDIA A100-SXM4-40GB, pci bus id: 0000:00:04.0, compute capability: 8.0
[info] Starting Layer Noise Analysis
Full Quant Analysis:   0%|                                                                                                                                                                                                                                                                         | 0/2 [00:00<?, ?iterations/s]I0000 00:00:1755811882.182885   20369 cuda_dnn.cc:529] Loaded cuDNN version 90900
Could not load symbol cuFuncGetName. Error: /lib/x86_64-linux-gnu/libcuda.so.1: undefined symbol: cuFuncGetName
Full Quant Analysis: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [02:14<00:00, 67.43s/iterations]
[info] Model Optimization Algorithm Layer Noise Analysis is done (completion time is 00:02:21.83)
[info] The calibration set seems to not be normalized, because the values range is [(0.0, 1.0), (0.0, 1.0), (0.0, 1.0)].
Since the neural core works in 8-bit (between 0 to 255), a quantization will occur on the CPU of the runtime platform.
Add a normalization layer to the model to offload the normalization to the neural core.
Refer to the user guide Hailo Dataflow Compiler user guide / Model Optimization / Optimization Related Model Script Commands / model_modification_commands / normalization for details.
[info] Model Optimization is done
[info] Saved HAR to: /home/stanislas.duprey/tmp/best_optimized.har
xxx:~$ hailo compiler best_optimized.har --hw-arch hailo8
[info] No GPU chosen, Selected GPU 0
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1755812407.881545   21205 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1755812407.888714   21205 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
[info] Current Time: 21:40:13, 08/21/25
[info] CPU: Architecture: x86_64, Model: Intel(R) Xeon(R) CPU @ 2.20GHz, Number Of Cores: 12, Utilization: 0.0%
[info] Memory: Total: 83GB, Available: 81GB
[info] System info: OS: Linux, Kernel: 6.8.0-1034-gcp
[info] Hailo DFC Version: 3.32.0
[info] HailoRT Version: Not Installed
[info] PCIe: No Hailo PCIe device was found
[info] Running `hailo compiler best_optimized.har --hw-arch hailo8`
[info] Compiling network
[info] To achieve optimal performance, set the compiler_optimization_level to "max" by adding performance_param(compiler_optimization_level=max) to the model script. Note that this may increase compilation time.
[info] Loading network parameters
[info] Starting Hailo allocation and compilation flow
[info] Adding an output layer after conv51
[info] Adding an output layer after conv54
[info] Adding an output layer after conv62
[info] Adding an output layer after conv65
[info] Adding an output layer after conv77
[info] Adding an output layer after conv80
[info] Building optimization options for network layers...
[info] Successfully built optimization options - 10s 337ms
[info] Trying to compile the network in a single context
[info] Single context flow failed: Recoverable single context error
[info] Building optimization options for network layers...
[info] Successfully built optimization options - 17s 878ms
[info] Using Multi-context flow
[info] Resources optimization params: max_control_utilization=60%, max_compute_utilization=60%, max_compute_16bit_utilization=60%, max_memory_utilization (weights)=60%, max_input_aligner_utilization=60%, max_apu_utilization=60%
[info] Finding the best partition to contexts...
[..................................<==>..] Duration: 00:00:49
Found valid partition to 2 contexts
[info] Searching for a better partition...
[.....................................   ] Duration: 00:00:34
Found valid partition to 2 contexts, Performance improved by 1.3%
[info] Searching for a better partition... Elapsed: 00:00:00
[.............<==>.......................] Duration: 00:00:36
Found valid partition to 2 contexts, Performance improved by 3.3%
[info] Searching for a better partition...
[...........................<==>.........] Duration: 00:00:35
Found valid partition to 2 contexts, Performance improved by 4.2%
[info] Searching for a better partition...
[..................<==>..................] Duration: 00:00:36
Found valid partition to 2 contexts, Performance improved by 4.4%
[info] Searching for a better partition...
[..................<==>..................] Duration: 00:00:36
Found valid partition to 2 contexts, Performance improved by 12.2%
[info] Searching for a better partition...
[.......<==>.............................] Duration: 00:00:54
Found valid partition to 2 contexts, Performance improved by 15.1%
[info] Searching for a better partition...
[.......<==>.............................] Duration: 00:04:51
Found valid partition to 3 contexts, Performance improved by 3.3%
[info] Searching for a better partition...
[<==>....................................] Duration: 00:03:26
Found valid partition to 3 contexts, Performance improved by 3.7%
[info] Searching for a better partition...
[...............................<==>.....] Duration: 00:00:35
Found valid partition to 3 contexts, Performance improved by 24.6%
[info] Searching for a better partition...
[........<==>............................] Duration: 00:11:50
Found valid partition to 3 contexts, Performance improved by 5.3%
[info] Searching for a better partition...
[...<==>.................................] Elapsed: 00:00:38
[info] Partition to contexts finished successfully
[info] Partitioner finished after 152 iterations, Time it took: 26m 15s 958ms
[info] Applying selected partition to 3 contexts...
[info] Validating layers feasibility

Validating best_context_0 layer by layer (100%)

 +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +
 +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +
 +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +
 +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +

● Finished


Validating best_context_1 layer by layer (100%)

 +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +
 +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +
 +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +
 +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +

● Finished


Validating best_context_2 layer by layer (100%)

 +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +
 +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +
 +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +
 +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +

● Finished

[info] Layers feasibility validated successfully
[info] Running resources allocation (mapping) flow, time per context: 59m 59s
Context:0/2 Iteration 4: Trying parallel mapping...
          cluster_0  cluster_1  cluster_2  cluster_3  cluster_4  cluster_5  cluster_6  cluster_7  prepost
 worker0  V          V          V          V          V          V          V          X          V
 worker1  V          V          V          V          V          X          V          V          V
 worker2  *          *          *          *          *          *          *          *          V
 worker3  V          V          V          V          V          V          V          V          V
Context:1/2 Iteration 4: Trying parallel mapping...
          cluster_0  cluster_1  cluster_2  cluster_3  cluster_4  cluster_5  cluster_6  cluster_7  prepost
 worker0  V          V          V          V          *          *          V          V          V
 worker1  *          *          *          *          *          *          *          *          V
 worker2  *          *          *          *          *          *          *          *          V
 worker3  V          V          V          V          V          V          V          V          V
Context:2/2 Iteration 4: Trying parallel mapping...
          cluster_0  cluster_1  cluster_2  cluster_3  cluster_4  cluster_5  cluster_6  cluster_7  prepost
 worker0  V          V          V          V          V          V          V          V          V
 worker1  V          V          V          V          V          V          V          V          V
 worker2  V          V          V          V          V          V          V          V          V
 worker3  V          V          V          V          V          V          V          V          V

  01:06
Reverts on cluster mapping: 0
Reverts on inter-cluster connectivity: 0
Reverts on pre-mapping validation: 0
Reverts on split failed: 0

[info] best_context_0 (best_context_0):
Iterations: 4
Reverts on cluster mapping: 2
Reverts on inter-cluster connectivity: 1
Reverts on pre-mapping validation: 0
Reverts on split failed: 0
[info] best_context_1 (best_context_1):
Iterations: 4
Reverts on cluster mapping: 0
Reverts on inter-cluster connectivity: 0
Reverts on pre-mapping validation: 0
Reverts on split failed: 0
[info] best_context_2 (best_context_2):
Iterations: 4
Reverts on cluster mapping: 0
Reverts on inter-cluster connectivity: 0
Reverts on pre-mapping validation: 0
Reverts on split failed: 0
[info] best_context_0 utilization:
[info] +-----------+---------------------+---------------------+--------------------+
[info] | Cluster   | Control Utilization | Compute Utilization | Memory Utilization |
[info] +-----------+---------------------+---------------------+--------------------+
[info] | cluster_0 | 75%                 | 68.8%               | 58.6%              |
[info] | cluster_1 | 50%                 | 56.3%               | 58.6%              |
[info] | cluster_2 | 62.5%               | 46.9%               | 40.6%              |
[info] | cluster_3 | 81.3%               | 62.5%               | 75%                |
[info] | cluster_4 | 56.3%               | 71.9%               | 98.4%              |
[info] | cluster_5 | 31.3%               | 28.1%               | 22.7%              |
[info] | cluster_6 | 62.5%               | 57.8%               | 54.7%              |
[info] | cluster_7 | 56.3%               | 57.8%               | 41.4%              |
[info] +-----------+---------------------+---------------------+--------------------+
[info] | Total     | 59.4%               | 56.3%               | 56.3%              |
[info] +-----------+---------------------+---------------------+--------------------+
[info] best_context_1 utilization:
[info] +-----------+---------------------+---------------------+--------------------+
[info] | Cluster   | Control Utilization | Compute Utilization | Memory Utilization |
[info] +-----------+---------------------+---------------------+--------------------+
[info] | cluster_0 | 87.5%               | 46.9%               | 60.9%              |
[info] | cluster_1 | 81.3%               | 53.1%               | 48.4%              |
[info] | cluster_2 | 18.8%               | 23.4%               | 12.5%              |
[info] | cluster_3 | 100%                | 62.5%               | 50.8%              |
[info] | cluster_6 | 100%                | 96.9%               | 77.3%              |
[info] | cluster_7 | 93.8%               | 70.3%               | 67.2%              |
[info] +-----------+---------------------+---------------------+--------------------+
[info] | Total     | 60.2%               | 44.1%               | 39.6%              |
[info] +-----------+---------------------+---------------------+--------------------+
[info] best_context_2 utilization:
[info] +-----------+---------------------+---------------------+--------------------+
[info] | Cluster   | Control Utilization | Compute Utilization | Memory Utilization |
[info] +-----------+---------------------+---------------------+--------------------+
[info] | cluster_0 | 87.5%               | 67.2%               | 53.1%              |
[info] | cluster_1 | 56.3%               | 70.3%               | 65.6%              |
[info] | cluster_2 | 100%                | 81.3%               | 63.3%              |
[info] | cluster_3 | 6.3%                | 1.6%                | 3.9%               |
[info] | cluster_4 | 12.5%               | 7.8%                | 5.5%               |
[info] | cluster_5 | 93.8%               | 96.9%               | 62.5%              |
[info] | cluster_6 | 37.5%               | 34.4%               | 28.9%              |
[info] | cluster_7 | 87.5%               | 64.1%               | 57.8%              |
[info] +-----------+---------------------+---------------------+--------------------+
[info] | Total     | 60.2%               | 52.9%               | 42.6%              |
[info] +-----------+---------------------+---------------------+--------------------+
[info] Successful Mapping (allocation time: 29m 0s)
[info] Compiling kernels of best_context_0...
[info] Compiling kernels of best_context_1...
[info] Compiling kernels of best_context_2...
[info] Bandwidth of model inputs: 9.375 Mbps, outputs: 6.79321 Mbps (for a single frame)
[info] Bandwidth of DDR buffers: 14.0625 Mbps (for a single frame)
[info] Bandwidth of inter context tensors: 60.9375 Mbps (for a single frame)
[info] Building HEF...
[info] Successful Compilation (compilation time: 28s)
[info] Compilation complete
[info] Saved HEF to: /home/stanislas.duprey/tmp/best.hef
[info] Saved HAR to: /home/stanislas.duprey/tmp/best_compiled.har

Done with the following hailo version:

xxx:~$ hailo --version
[info] No GPU chosen, Selected GPU 0
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1755814594.947982  175305 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1755814594.954882  175305 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
[info] Current Time: 22:16:40, 08/21/25
[info] CPU: Architecture: x86_64, Model: Intel(R) Xeon(R) CPU @ 2.20GHz, Number Of Cores: 12, Utilization: 0.0%
[info] Memory: Total: 83GB, Available: 81GB
[info] System info: OS: Linux, Kernel: 6.8.0-1034-gcp
[info] Hailo DFC Version: 3.32.0
[info] HailoRT Version: Not Installed
[info] PCIe: No Hailo PCIe device was found
[info] Running `hailo --version`
Hailo Dataflow Compiler v3.32.0

[majedabu]

can you share the onnx and the hef you got?

did you encounter any problem when running pre compiled models from model zoo?