batch_ratio calculation on 'mps' devices doing pipeline mode

[stelf]

🔎 Search before asking | 提交之前请先搜索

• I have searched the MinerU Readme and found no similar bug report.
• I have searched the MinerU Issues and found no similar bug report.
• I have searched the MinerU Discussions and found no similar bug report.

🤖 Consult the online AI assistant for assistance | 在线 AI 助手咨询

• I have consulted the online AI assistant but was unable to obtain a solution to the issue.

Description of the bug | 错误描述

batch tuning via batch_ratio is off on 'mps' since vram is not reported and thus not calculated at all in pipeline_analyze.py. the ratio is calculated only for 'npu' and 'cuda',.

adding the following, which approximates the VRAM to safely be 0.7 of the available memory results in reasonable performance gains for me.

 if str(device).startswith('mps'):
        # MPS-specific VRAM detection
        if vram is None:
            try:
                import torch
                if torch.backends.mps.is_available():
                    # Get recommended max memory from system
                    # MPS doesn't expose direct VRAM query, use system memory as proxy
                    import psutil
                    system_memory_gb = psutil.virtual_memory().total / (1024**3)
                    # On Apple Silicon, GPU shares system memory
                    # Use conservative estimate: 70% of system memory for GPU tasks
                    vram = system_memory_gb * 0.7
                    logger.info(f'MPS device detected, estimated shared memory: {vram:.1f} GB')
            except Exception as e:
                logger.warning(f'Could not determine MPS memory: {e}')
                vram = 16  # Conservative default for Apple Silicon

        # Determine batch ratio based on available memory
        if vram >= 32:
            batch_ratio = 16
        elif vram >= 24:
            batch_ratio = 12
        elif vram >= 16:
            batch_ratio = 8
        else:
            batch_ratio = 4

        logger.info(f'MPS device detected, estimated VRAM: {vram:.1f} GB, using batch_ratio: {batch_ratio}')

apart from that these rather aggressive values in the batch_analyze.py also produce improved performance

YOLO_LAYOUT_BASE_BATCH_SIZE = 8
MFD_BASE_BATCH_SIZE = 8
MFR_BASE_BATCH_SIZE = 16
OCR_DET_BASE_BATCH_SIZE = 16
TABLE_ORI_CLS_BATCH_SIZE = 32
TABLE_Wired_Wireless_CLS_BATCH_SIZE = 32

(note that those impacted by batch_ratio are preserved in line with HEAD)

currently also experimenting with the binning for the OCR-det batch step where a more agressive binning of the images (in larger patches) result in sometimes tenfold increase in performance. i'll open separate issue for it when I also get some time to test on my CUDA/24GB device.

How to reproduce the bug | 如何复现

---

Operating System Mode | 操作系统类型

MacOS

Operating System Version| 操作系统版本

OS 15.7 on M3 Max 64GB

Python version | Python 版本

3.11

Software version | 软件版本 (mineru --version)

=2.5

Backend name | 解析后端

pipeline

Device mode | 设备模式

mps

[dosubot]

Hi @stelf! I'm Dosu and I’m helping the MinerU team.

You're correct—batch_ratio calculation in pipeline mode currently only handles 'npu' and 'cuda' devices. There is no logic for 'mps' devices in pipeline_analyze.py, so your proposed VRAM estimation and batch_ratio logic for Apple Silicon is not implemented yet. The batching and VRAM detection for 'mps' would require a code change similar to your snippet, using psutil to estimate available memory and mapping that to batch_ratio tiers. Recent updates and documentation do not address this gap, and the aggressive batch size values you mentioned are also not set by default in the codebase. Your approach aligns well with how Apple Silicon shares memory and should help improve performance on MPS devices.
See code reference

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[myhloli]

On macOS, due to the use of unified memory, larger batch sizes cause contention between model runtime memory and application runtime memory. This issue becomes even more severe when numerous background applications are running, preventing larger batches from effectively improving processing speed. On my Mac mini with an M4 chip and 16GB of RAM, the best performance was achieved with a batch_ratio of 1—this was under conditions where all background applications were closed. Larger batch_ratio values, such as 2 or 4, consistently resulted in noticeable memory shortages and reduced parsing speed. Therefore, I believe it is reasonable to keep batch_ratio at 1 on Mac systems.

[stelf]

Potentially this is the situation with Mx's with less memory. The OS - like firefox, vscode, etc take significant amount of memory. Window manager takes ~1g, etc. But with 64 gigs a good portion (~40) is normally free, and this can be utilized. I understand batch_ratio values give you headaches, but for me batch_ratio of 12 works pretty much okay, even though i blew memory with batch_ratio 16.

after tweaking my current setup is

YOLO_LAYOUT_BASE_BATCH_SIZE = 8
MFD_BASE_BATCH_SIZE = 6
MFR_BASE_BATCH_SIZE = 8
OCR_DET_BASE_BATCH_SIZE = 16
TABLE_ORI_CLS_BATCH_SIZE = 32
TABLE_Wired_Wireless_CLS_BATCH_SIZE = 32

with this config

        # Determine batch ratio based on available memory
        if vram >= 32:
            batch_ratio = 16
        elif vram >= 24:
            batch_ratio = 12
        elif vram >= 16:
            batch_ratio = 8
        else:
            batch_ratio = 4

and also with tweaked ocr-det procedure where the binning is remade to actually use the memory. and some missing gc and deletes are added.

I'm processing 150 -page PDFs with heavy table use for a total of ~400secs on a M3Max with 64GB,... (so this is some %% slower than your M4 theoretically).

which means that the RAM should be utilized properly, and these machines can take it. im amazed at what they actually take, the 24GIG 3090TI does not even come close for some of the model steps.

the best approach would be adaptive batch_ratio which adapts with memory usage depending on the nature of the docs.

[stelf]

update: tbh the culprit to faster OCR processing is to rework the whole thing to aim for consistent size of the batch in respect to total memory.

this does the trick for me, but my bet is it can be approached in a much smarter manner as it also produces some massive

                MAX_GROUP_MEMORY = 1024 * 1024 * 64 
                MIN_BATCH_SIZE = 8  
                MAX_BATCH_SIZE = 128 
                
                resolution_groups = defaultdict(list)
                group_counter = 0
                current_group_memory = 0
                current_group_key = f"bin_{group_counter}"
                
                for idx, crop_info in sorted_crops:
                    h, w = crop_info[0].shape[:2]
                    img_memory = estimate_memory(h, w)
                    
                    should_create = (
                        (current_group_memory + img_memory > MAX_GROUP_MEMORY and 
                         len(resolution_groups[current_group_key]) >= MIN_BATCH_SIZE) or
                        len(resolution_groups[current_group_key]) >= MAX_BATCH_SIZE
                    )
                    
                    if should_create:
                        group_counter += 1
                        current_group_key = f"bin_{group_counter}"
                        current_group_memory = 0
                    
                    resolution_groups[current_group_key].append(crop_info)
                    current_group_memory += img_memory