Skip to content

new TextDecoder('utf-8', { fatal: true }).decode(..., { stream: true }) behaves differently than node #1861

@hi-ogawa

Description

@hi-ogawa

Describe the bug

A following code tries to decode non utf-8 data.

// repro.js

function main() {
  const decoder = new TextDecoder('utf-8', { fatal: true });
  // reverse to force non-utf8 binary data
  const data = new TextEncoder().encode('🔥').reverse();
  try {
    const decoded = decoder.decode(data, { stream: true });
    console.log({ decoded });
  } catch (error) {
    console.log({ error });
  }
}

main();

This leads to catch case on Node when run locally:

$ node repro.js
{
  error: TypeError: The encoded data was not valid for encoding utf-8
      at TextDecoder.decode (node:internal/encoding:462:14)
      at main (file:///home/hiroshi/code/tmp/rsc-app/repro.js:6:29)
      at file:///home/hiroshi/code/tmp/rsc-app/repro.js:13:1
      at ModuleJob.run (node:internal/modules/esm/module_job:271:25)
      at async onImport.tracePromise.__proto__ (node:internal/modules/esm/loader:578:26)
      at async asyncRunEntryPointWithESMLoader (node:internal/modules/run_main:116:5) {
    code: 'ERR_ENCODING_INVALID_ENCODED_DATA'
  }
}

However, on Stackblitz, it returns a number 3

 node repro.js
{ decoded: 3 }

Link to the blitz that caused the error

https://stackblitz.com/edit/stackblitz-starters-nz6s4tic?file=repro.js

Steps to reproduce

Open stackblitz

Expected behavior

It should work like Node:

$ node repro.js
{
  error: TypeError: The encoded data was not valid for encoding utf-8
      at TextDecoder.decode (node:internal/encoding:462:14)
      at main (file:///home/hiroshi/code/tmp/rsc-app/repro.js:6:29)
      at file:///home/hiroshi/code/tmp/rsc-app/repro.js:13:1
      at ModuleJob.run (node:internal/modules/esm/module_job:271:25)
      at async onImport.tracePromise.__proto__ (node:internal/modules/esm/loader:578:26)
      at async asyncRunEntryPointWithESMLoader (node:internal/modules/run_main:116:5) {
    code: 'ERR_ENCODING_INVALID_ENCODED_DATA'
  }
}

Parity with Local

Screenshots

No response

Platform

  • OS: [e.g. ~macOS, Windows, Linux] Linux
  • Browser: [e.g. Chrome, Safari, Firefox] Chrome
  • Version: [e.g. 91.1]

Additional context

I came across the issue in hi-ogawa/vite-plugins#852. The binary payload is encoded as string to inject into html based on TextDecoder.decode where if it succeeds, injected as is and if it fails, it fallsbacks to base64 encoding. The original code is found here https://github.com/devongovett/rsc-html-stream/blob/9938c93975380d13189363565897fb66a59ffb05/server.js#L66-L76

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions