Segmentation Fault Converting h264 to VP9 to 4k

[jgcaruso]

Hi there

I'm trying to convert a 2 minute 30fps 4k h264 file to a 4k vp9 file using ffmpeg and I'm getting a segmentation fault around 42 seconds of video transcoded (3 minutes of processing time). The same video worked fine transcoding to 1440p video file.

ffmpeg command:

ffmpeg -i bbb-2m-4k-30fps.mp4 -y -f mp4 -s 3840x2160 -acodec aac -c:v libsvt_vp9 -movflags faststart -pix_fmt yuv420p -r 30 -g 30 -minrate 6000k -maxrate 17400k -b:v 12000k bbb-2m-svt-vp9-4k.mp4

I also tried running SvtVp9EncApp and I get the following error:
command:

sudo SvtVp9EncApp -w 3840 -h 2160 -i ~/vp/bbb-2m-4k-30fps.mp4 -fps 30 -n 100 -tune 0 -q 32 -b test.bin

error:

double free or corruption (out)
Aborted

The full dump:

-------------------------------------------
SVT [version]	: SVT-VP9 Encoder Lib v0.3.0
SVT [build]	: GCC 9.3.0	 64 bit
LIB Build date: Oct 21 2021 15:46:52
-------------------------------------------
Number of logical cores available: 4
Number of PPCS 37
-------------------------------------------
SVT [config]: Profile [0]	Level (auto)
SVT [config]: EncoderMode / Tune 						: 9 / 0
SVT [config]: EncoderBitDepth 							: 8
SVT [config]: SourceWidth / SourceHeight					: 3840 / 2160
SVT [config]: FrameRate / Gop Size						: 30 / 33
SVT [config]: HierarchicalLevels / BaseLayerSwitchMode / PredStructure		: 3 / 0 / 2
SVT [config]: BRC Mode / QP 							: CQP / 32
-------------------------------------------


Encoding       100
SUMMARY --------------------------------- Channel 1  --------------------------------
Total Frames		Frame Rate		Byte Count		Bitrate
         100		30.00 fps		 300000000		720000.00 kbps


Channel 1
Average Speed:		2.46 fps
Total Encoding Time:	40624 ms
Total Execution Time:	41152 ms
Average Latency:	19192 ms
Max Latency:		28294 ms
double free or corruption (out)
Aborted

I'm running this in an EC2 m6i.xlarge instance running Ubuntu
CPU: Intel(R) Xeon(R) Platinum 8375C CPU @ 2.90GHz
RAM: 16GB

The video file is a 2 minute clip of Big Buck Bunny, trimmed in Quicktime so it shouldn't be a strange format causing the issue.

Any help or direction in debugging this issue would be helpful.

[1480c1]

Can you try some more minimal reproduction cases?

ffmpeg -i bbb-2m-4k-30fps.mp4 -an -c:v libsvt_vp9 -g 30 -minrate 6000k -maxrate 17400k -b:v 12000k -f null -

# and

ffmpeg -i bbb-2m-4k-30fps.mp4 temp.yuv
SvtVp9EncApp -w 3840 -h 2160 -i temp.yuv -fps 30 -n 100 -tune 0 -q 32 -b test.bin

?

As a note, SvtVp9EncApp only supports yuv input

[jgcaruso]

As a note, SvtVp9EncApp only supports yuv input

Oh good to know!

ffmpeg -i bbb-2m-4k-30fps.mp4 -an -c:v libsvt_vp9 -g 30 -minrate 6000k -maxrate 17400k -b:v 12000k -f null -

This still seg faults.

ffmpeg -i bbb-2m-4k-30fps.mp4 temp.yuv
SvtVp9EncApp -w 3840 -h 2160 -i temp.yuv -fps 30 -n 100 -tune 0 -q 32 -b test.bin

This completes with the message

Encoding       100
SUMMARY --------------------------------- Channel 1  --------------------------------
Total Frames		Frame Rate		Byte Count		Bitrate
         100		30.00 fps		   4681304		11235.13 kbps


Channel 1
Average Speed:		8.17 fps
Total Encoding Time:	12234 ms
Total Execution Time:	12777 ms
Average Latency:	4389 ms
Max Latency:		5583 ms
Encoder finished

If I re-run it without -n 100 it fails with the message

free(): invalid size
Aborted (core dumped)

Full dump

-------------------------------------------
SVT [version]	: SVT-VP9 Encoder Lib v0.3.0
SVT [build]	: GCC 9.3.0	 64 bit
LIB Build date: Oct 21 2021 15:46:52
-------------------------------------------

[WARNING] For best speed performance, run with sudo privileges !

Number of logical cores available: 4
Number of PPCS 37
-------------------------------------------
SVT [config]: Profile [0]	Level (auto)
SVT [config]: EncoderMode / Tune 						: 9 / 0
SVT [config]: EncoderBitDepth 							: 8
SVT [config]: SourceWidth / SourceHeight					: 3840 / 2160
SVT [config]: FrameRate / Gop Size						: 30 / 33
SVT [config]: HierarchicalLevels / BaseLayerSwitchMode / PredStructure		: 3 / 0 / 2
SVT [config]: BRC Mode / QP 							: CQP / 32
-------------------------------------------


Encoding      2000
Average System Encoding Speed:        9.06
     3640
SUMMARY --------------------------------- Channel 1  --------------------------------
Total Frames		Frame Rate		Byte Count		Bitrate
        3640		30.00 fps		 183565785		12103.24 kbps


Channel 1
Average Speed:		9.24 fps
Total Encoding Time:	394102 ms
Total Execution Time:	394627 ms
Average Latency:	6824 ms
Max Latency:		8976 ms
free(): invalid size
Aborted (core dumped)

[jgcaruso]

I also ran it again after running sudo sysctl -w kernel.sched_rt_runtime_us=1000000 and I got this error

munmap_chunk(): invalid pointer
Aborted (core dumped)

Just wondering if there is anything else I can look into, or any way to dump extra logs to find out what is going on?
Is it a hardware issue where it should only be run on bare metal and this is just not meant to run on AWS?

[1480c1]

I am able to reproduce the issue even on bare metal, so I will try to see if I can debug further

as a note, I was able to do so with just

./Bin/Debug/SvtVp9EncApp -i /dev/urandom -w 3840 -h 2160 -fps 1 -n 1 -tune 0 -q 63

so no need for any specific clip nor anything other than just the 4k part.

However, the error message is not always the same (munmap_chunk(): invalid pointer vs free(): invalid size) so this is leading me to believe there's a buffer overrun or similar occurring somewhere