Fix stdio encoding issue: Enforce explicit UTF-8 for correct Unicode handling

[willibrandon]

Overview

This pull request addresses the encoding issue reported in #35, where JSON-RPC messages printed in the terminal showed corrupted characters (e.g., Chinese characters displayed as question marks).

The problem stemmed from the stdio transport layer relying on the system default encoding (Windows-1252 on Windows)
instead of explicitly using UTF-8.

Changes

• Encoding Fix: Updated both client and server transports to explicitly use UTF8Encoding (without BOM) for reading and writing:

  // Create streams with explicit UTF-8 encoding to ensure proper Unicode character handling
  // This is especially important for non-ASCII characters like Chinese text and emoji
  var utf8Encoding = new UTF8Encoding(false); // No BOM
  _stdInWriter = new StreamWriter(_process.StandardInput.BaseStream, utf8Encoding) { AutoFlush = true };
  _stdOutReader = new StreamReader(_process.StandardOutput.BaseStream, utf8Encoding);

• Tests Added: Tests have been implemented to verify that both BMP Unicode characters (Chinese: "上下文伺服器") and non-BMP Unicode characters (emoji: 🔍🚀👍) are correctly preserved during transport.

Impact

This fix resolves the Unicode character corruption by ensuring that the transport layer uses consistent UTF-8 encoding, improving the reliability of message display in all locales. The changes maintain the existing API surface while enhancing support for international characters.

Next Steps

Please review the changes and let me know if any further modifications or additional tests are needed.

[stephentoub]

Thanks, @willibrandon.