Scaling Issue in new Pytorch Build method #258
Description
Hi, I am reporting my observations on two methods involved. They are with the following two commits 1) ac72dca, which uses Pytorch 2.5.0 and 2) 2dd009f , which uses Pytorch 2.6.0.
Steps followed in first method:
cd Tool-Solutions/docker/pytorch-aarch64
./build.sh --build-type pytorch --onednn acl
The above build will create new image with pytorch installed. I will create a container from it and run detr hugging face model (hug_detr.py script given at end). OMP_NUM_THREADS=64 python hug_detr.py is the command used to run the script, the script is run for 10 times and average time is taken. For 32 threads OMP_NUM_THREADS=32 is used.
Steps followed in second method:
cd Tool-Solutions/docker/pytorch-aarch64
./build.sh
sudo ./dockerize.sh results/torch-2.6.0.dev20241104-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
The above build will create new image with pytorch installed. I will run detr hugging face model (hug_detr.py script given at end). OMP_NUM_THREADS=64 python hug_detr.py is the command used to run the script, the script is run for 10 times and average time is taken. For 32 threads OMP_NUM_THREADS=32 is used.
Detr Model Script (hug_detr.py):
from transformers import AutoImageProcessor, DetrForObjectDetection
import torch
from time import time
import numpy as np
from PIL import Image
import requests
import os
from torch.profiler import profile, record_function, ProfilerActivity
url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)
image_processor = AutoImageProcessor.from_pretrained("facebook/detr-resnet-50")
model = DetrForObjectDetection.from_pretrained("facebook/detr-resnet-50")
inputs = image_processor(images=image, return_tensors="pt")
with torch.no_grad():
for i in range(4):
print(f"Warm up of cycle {i}",end='\r', flush=True)
outputs=model(**inputs)
times=[]
itr=6
with torch.no_grad():
for i in range(itr):
print(f"Actual run of cycle {i}",end='\r', flush=True)
start_time = time()
outputs = model(**inputs)
times.append(time() - start_time)
print("average time (milliseconds) for DETR Inference: %.2f " %(np.average(times[0:itr])*1e3))