BUG: H.265 MSE Streaming Bug Report (In Frigate)

[TCS-UK]

Edited — H.265 MSE Streaming Bug Report

Date: 2026-04-10
Affected versions: All versions including v1.9.13 (embedded in Frigate 0.18.0-dev) and v1.9.14+dev (skrashevich fork, commit ad2c09a)
Repositories: AlexxIT/go2rtc · skrashevich/go2rtc
Symptom: H.265 (HEVC) cameras fail all MSE playback. Browser receives ~168 bytes and the WebSocket closes within 3 seconds. H.264 cameras on the same server are unaffected.

---

Environment (discovery context)

• Frigate version: 0.18.0-8f13932c (dev branch)
• go2rtc versions tested: embedded 1.9.13 (/usr/local/go2rtc/bin/go2rtc) and custom fork 1.9.14+dev.ad2c09a.dirty (/config/go2rtc)
• Camera: Dahua IPC-HDW2849H-S-IL, 4000×3000, H.265 Main profile, Level 6.0
• Access path: Chrome 146 (Windows 10) → Cloudflare → Traefik → Frigate nginx → go2rtc
• go2rtc stream config: exec:ffmpeg -rtsp_transport tcp -i rtsp://.../cam/realmonitor?channel=1&subtype=0 -c copy -f rtsp -rtsp_transport tcp {output}
• go2rtc stream API confirms: codec hevc, SDP a=rtpmap:96 H265/90000, all three parameter sets present (sprop-vps, sprop-sps, sprop-pps)
• Confirmed that the bug exists in BOTH go2rtc versions — the MSE/H.265 code in pkg/mp4/, pkg/iso/, and internal/mp4/ is byte-for-byte identical between v1.9.13 and ad2c09a

---

Related open issues

• #1302 — MSE / H265 Console Errors / Video doesn't load — Multiple reporters describe H.265 MSE failing while H.264 succeeds. Root cause was not identified; attributed to browser codec support. The codec string mismatch described in this report is the likely cause.
• #1598 — Black screen in WebRTC when H265 codec is enabled — Related symptom (H.265 stream not rendering). Different mode, but the box name inconsistency is also relevant to WebRTC mode codec negotiation.
• #1711 — Reolink Duo 2 incompatible due to aspect ratio and H265 encoding — MSE failures with H.265 camera; unresolved. Box name mismatch is likely a contributing factor.
• #1789 — Advice on diagnosing an issue with MSE mode — Debugging MSE failures; H.265 streams involved.

None of the above identified the root cause described below.

---

Bug A — hev1 box written in fMP4, but hvc1 advertised in MIME type

Severity

High — confirmed root cause of MSE failures. This is a protocol-level mismatch. Chrome 107–110 silently tolerated it; Chrome 120+ rejects the init segment, causing MSE playback to fail for all H.265 cameras.

Files

• pkg/iso/codecs.go — line 14

Description

When go2rtc builds the fMP4 init segment for an H.265 stream, it writes an hev1 sample entry box:

// pkg/iso/codecs.go:14
case core.CodecH265:
    m.StartAtom("hev1")   // ← writes hev1 box

However, the MIME type returned to the browser and used in addSourceBuffer() advertises hvc1:

// pkg/mp4/mime.go:10
MimeH265 = "hvc1.1.6.L153.B0"   // ← declares hvc1

hvc1 and hev1 are not interchangeable. Per ISO 14496-15 §8.4.1:

Box name	Meaning
hvc1	H.265 parameter sets stored exclusively out-of-band in the hvcC box
hev1	H.265 parameter sets may be in-band (in the stream) or in hvcC

When a browser creates a SourceBuffer with video/mp4; codecs="hvc1.1.6.L153.B0" and then receives an init segment with an hev1 sample entry box, it has received a different codec type than declared. In Chrome 120+ this inconsistency causes appendBuffer to fire a QuotaExceededError or InvalidStateError, which the video-rtc.js error handler interprets as stream failure, closing the WebSocket and falling back to jsmpeg.

Note: go2rtc already correctly writes parameter sets to the hvcC box (not in-band), so changing the sample entry box name from hev1 to hvc1 is the correct and safe fix — the delivery mode matches the hvc1 semantics.

Fix (applied)

Change pkg/iso/codecs.go line 14 to write hvc1 instead of hev1:

 case core.CodecH265:
-    m.StartAtom("hev1")
+    m.StartAtom("hvc1")

With this change, the declared MIME codec string (hvc1.1.6.L153.B0) matches the fMP4 sample entry box (hvc1). This also improves Safari compatibility — Safari requires hvc1 and does not accept hev1.

Verification

After deploying the patched binary, probing the MSE WebSocket from inside the container confirms:

Text: {'type': 'mse', 'value': 'video/mp4; codecs="hvc1.1.6.L153.B0"'}
Binary: 759 bytes (init segment)
  Box name at offset 421: hvc1

The hev1 init segment was 168 bytes and caused immediate close; the hvc1 init segment is 759 bytes (full ftyp + moov) and the connection stays open for video frames.

---

Bug B — H.265 codec level hardcoded at 5.1 (investigated, not applied as fix)

Status

Investigated. Technically inaccurate, but cannot be corrected in the server response without client changes. No fix applied.

Description

For H.264, go2rtc derives the profile and level from the stream's SPS:

// pkg/mp4/mime.go
case core.CodecH264:
    s += "avc1." + h264.GetProfileLevelID(codec.FmtpLine)

For H.265, the codec string is hardcoded regardless of the actual stream:

MimeH265 = "hvc1.1.6.L153.B0"   // Level 5.1, hardcoded

case core.CodecH265:
    s += MimeH265

L153 encodes H.265 Level 5.1. The Dahua IPC-HDW2849H-S-IL at 4000×3000 is actually Level 6.0 (general_level_idc = 180, since 4000×3000 = 12,000,000 luma samples > Level 5.1 max of 8,912,896).

Why a "derive from SPS" fix does not work here

The natural fix would mirror H.264: decode the H.265 SPS and use the actual level in the response. A GetProfileLevelID() function for H.265 was implemented (added to pkg/h265/helper.go) and tested.

Problem: The codec string in the server response must match what the browser offered, not the actual stream level.

video-rtc.js hardcodes its CODECS list and offers only hvc1.1.6.L153.B0 (Level 5.1) to go2rtc. The browser checks MediaSource.isTypeSupported('video/mp4; codecs="hvc1.1.6.L153.B0"') before sending the request — that's the level the browser has confirmed it supports.

When go2rtc responds with video/mp4; codecs="hvc1.1.6.L180.B0" (Level 6.0, derived from SPS), the browser tries:

mediaSource.addSourceBuffer('video/mp4; codecs="hvc1.1.6.L180.B0"')

Level 6.0 support depends on hardware and driver. On systems where L180 is not in the supported codec list, addSourceBuffer throws and MSE fails — the same symptom as before but for a different reason. This was confirmed during testing: deploying the "derive from SPS" fix did not resolve the 168-byte close pattern.

What would actually fix it

The level mismatch is a client-server contract issue. The options are:

1. Update video-rtc.js to include higher levels in its CODECS list (e.g. hvc1.1.6.L180.B0, hvc1.1.6.L186.B0). This is the correct long-term fix but requires a frontend change in go2rtc/Frigate.

2. In go2rtc: Use the codec string from the browser's offer as the response, rather than deriving from the stream. This requires threading the offer codec string through the ParseCodecs → AddConsumer → ContentType path.

3. Accept the current behaviour: Level 5.1 is declared but Level 6.0 content is delivered. The actual SPS inside the hvcC box has the correct level; most HEVC decoders use the SPS for decoding decisions rather than the MIME type level. Level 5.1 declaration for Level 6.0 content is technically wrong but benign in practice once the browser successfully creates the SourceBuffer.

The GetProfileLevelID() function added to pkg/h265/helper.go is retained in the patch for completeness and potential future use (e.g., in ParseCodecs matching or logging), but it is not called from MimeCodecs().

---

Summary

	Bug A	Bug B
File	pkg/iso/codecs.go:14	pkg/mp4/mime.go:10
Issue	fMP4 box is hev1, MIME declares hvc1	H.265 codec level hardcoded at 5.1 for all streams
Impact	Chrome 120+ rejects init segment; MSE always fails for H.265	Level mismatch between MIME and actual stream
Fix	Change hev1 → hvc1 in codecs.go:14	Cannot fix response without client changes
Applied	Yes	No
Present in v1.9.13	Yes	Yes
Present in v1.9.14+dev	Yes	Yes
Related issues	#1302, #1598, #1711, #1789	#1302, #1711

---

Workarounds (no code change required)

1. Switch live stream to an H.264 sub-stream

If the camera has a configured H.264 sub-stream in go2rtc (e.g. via NVENC re-encode), configure the Frigate live.streams to default to that stream:

# config.yaml
cameras:
  dahua_test:
    live:
      streams:
        Detect Stream: dahua_test_sub   # H.264 — MSE works
        Record Stream: dahua_test       # H.265 — MSE fails until bugs are fixed

In the Frigate UI, select "Detect Stream" for live view on affected cameras.

2. Add an H.264 restream in go2rtc for the affected camera

# go2rtc streams config
streams:
  dahua_test_h264:
    - exec:ffmpeg -rtsp_transport tcp -hwaccel cuda -i rtsp://user:pass@camera_ip/stream
        -an -c:v h264_nvenc -preset p2 -tune ll -rc cbr -b:v 4M
        -f rtsp -rtsp_transport tcp {output}

Then point the Frigate live view at dahua_test_h264.

3. Test with embedded go2rtc

Note: this does not fix the bug. The pkg/mp4/ and pkg/iso/ MSE/H.265 code is identical in v1.9.13 and v1.9.14+dev. Switching binaries will produce the same MSE failure for H.265 streams.

[TCS-UK]

Edited Patch after further testing:

diff --git a/pkg/h265/helper.go b/pkg/h265/helper.go
index c2ad94d..e90378c 100644
--- a/pkg/h265/helper.go
+++ b/pkg/h265/helper.go
@@ -3,6 +3,8 @@ package h265
 import (
 	"encoding/base64"
 	"encoding/binary"
+	"fmt"
+	"strings"
 
 	"github.com/AlexxIT/go2rtc/pkg/core"
 )
@@ -58,6 +60,33 @@ func Types(data []byte) []byte {
 	return types
 }
 
+// GetProfileLevelID returns the HEVC codec string component derived from the
+// stream's SPS parameter set, e.g. "1.6.L180.B0" for Main profile Level 6.0.
+// Falls back to "1.6.L153.B0" (Main 5.1) if the SPS cannot be decoded.
+// This mirrors h264.GetProfileLevelID for consistent MIME type generation.
+func GetProfileLevelID(fmtp string) string {
+	_, sps, _ := GetParameterSet(fmtp)
+	if len(sps) == 0 {
+		return "1.6.L153.B0"
+	}
+
+	s := DecodeSPS(sps)
+	if s == nil {
+		return "1.6.L153.B0"
+	}
+
+	// Encode general_profile_compatibility_flags as hex with trailing zeros
+	// removed, matching the shortened form used in codec strings
+	// (e.g. 0x60000000 → "60000000" → "6").
+	flags := fmt.Sprintf("%08X", s.general_profile_compatibility_flags)
+	flags = strings.TrimRight(flags, "0")
+	if flags == "" {
+		flags = "0"
+	}
+
+	return fmt.Sprintf("%d.%s.L%d.B0", s.general_profile_idc, flags, s.general_level_idc)
+}
+
 func GetParameterSet(fmtp string) (vps, sps, pps []byte) {
 	if fmtp == "" {
 		return
diff --git a/pkg/iso/codecs.go b/pkg/iso/codecs.go
index f4a83c7..471f6c3 100644
--- a/pkg/iso/codecs.go
+++ b/pkg/iso/codecs.go
@@ -11,7 +11,7 @@ func (m *Movie) WriteVideo(codec string, width, height uint16, conf []byte) {
 	case core.CodecH264:
 		m.StartAtom("avc1")
 	case core.CodecH265:
-		m.StartAtom("hev1")
+		m.StartAtom("hvc1")
 	default:
 		panic("unsupported iso video: " + codec)
 	}
diff --git a/pkg/mp4/mime.go b/pkg/mp4/mime.go
index f1a74e4..8cd9341 100644
--- a/pkg/mp4/mime.go
+++ b/pkg/mp4/mime.go
@@ -25,8 +25,6 @@ func MimeCodecs(codecs []*core.Codec) string {
 		case core.CodecH264:
 			s += "avc1." + h264.GetProfileLevelID(codec.FmtpLine)
 		case core.CodecH265:
-			// H.265 profile=main level=5.1
-			// hvc1 - supported in Safari, hev1 - doesn't, both supported in Chrome
 			s += MimeH265
 		case core.CodecAAC:
 			s += MimeAAC