-
Notifications
You must be signed in to change notification settings - Fork 34
Add config for enabling the wasm version of shiki highlighter #322
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Adds a configurable way to choose between Shiki’s JavaScript regex engine and the WASM Oniguruma engine across the @pierre/diffs highlighter stack (shared highlighter and worker pool), and updates docs/examples accordingly.
Changes:
- Introduces
preferredHighlighter: 'shiki-js' | 'shiki-wasm'and threads it through shared highlighter + WorkerPoolManager initialization. - Switches worker highlighter initialization to async
createHighlighterCoreand adds Oniguruma engine support. - Updates Shiki imports/dependencies to rely on
shiki/*entry points and documents the new option in docs/demo.
Reviewed changes
Copilot reviewed 16 out of 17 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/diffs/src/worker/worker.ts | Async worker message handling + engine selection (js vs wasm) during highlighter creation. |
| packages/diffs/src/worker/types.ts | Adds preferredHighlighter to the worker initialization request/types. |
| packages/diffs/src/worker/WorkerPoolManager.ts | Stores preferredHighlighter on the pool and passes it into shared highlighter + worker initialization. |
| packages/diffs/src/utils/getHighlighterOptions.ts | Extends highlighter option derivation to include preferredHighlighter. |
| packages/diffs/src/types.ts | Defines HighlighterTypes and exposes preferredHighlighter on base code options. |
| packages/diffs/src/shiki-stream/types.ts | Updates Shiki type import path to shiki/core. |
| packages/diffs/src/shiki-stream/tokenizer.ts | Updates Shiki type import path to shiki/core. |
| packages/diffs/src/shiki-stream/stream.ts | Updates Shiki type import path to shiki/core. |
| packages/diffs/src/renderers/DiffHunksRenderer.ts | Adjusts option typing to exclude preferredHighlighter from required defaults. |
| packages/diffs/src/highlighter/shared_highlighter.ts | Adds engine selection to shared highlighter creation based on preferredHighlighter. |
| packages/diffs/package.json | Removes direct @shikijs/* deps now covered by shiki, bumps package version. |
| bun.lock | Lockfile updates reflecting dependency/version changes. |
| apps/docs/app/docs/WorkerPool/content.mdx | Documents preferredHighlighter for WorkerPool initialization. |
| apps/docs/app/docs/WorkerPool/constants.ts | Adds commented examples for preferredHighlighter. |
| apps/docs/app/docs/VanillaAPI/constants.ts | Documents preferredHighlighter in vanilla usage examples. |
| apps/docs/app/docs/ReactAPI/constants.ts | Documents preferredHighlighter in React usage examples. |
| apps/demo/src/main.ts | Demo opts into 'shiki-wasm'. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
there doesn't appear to be an easy way to tell if an instance is wasm or not :thonk:
This PR adds back the ability to use the WASM highlighter, given there are certain cases (especially in Webkit/Safari) where performance is drastically improved over the JS variant.
I'm keeping it still under the key
preferredHighlighterbecause once we've created one, you can't change the type (it would require a full disposal).I'm not quite sure what the bundle implications will be here, but i think it should ok...