Generic solution for serialization / deserialization of web components

[ttraenkler]

I am looking for a general purpose serialization and deserialization (rehydration) of custom elements including their declarative shadow DOM with server side rendering (SSR) being one important use case. I wonder what's in scope for linkedom, what goes beyond and if prior work exists that slices the problem right - in a generic, technology agnostic way, complete and maintained enough to be worth building on or if it's better to roll my own.

Since you are open to collaboration and referenced your project WESC in WebReflection/linkedom#211 (comment), I thought this would be the right place to start a discussion. I had a brief glimpse at WESC and would like to better understand how this contrasts with / complements other approaches. I think for a solution the key is how to slice the problem, it should be as much as possible technology and framework agnostic and just do what's needed, not more and not less so it is as agnostic, general purpose and reusable as can be, also giving it a higher chance of being of interest to and maintained by a larger community.

Since in the goals of WESC you describe declarative https://github.com/luwes/wesc#custom-element-server-side-rendering as your first goal, I wonder how it compares / contrasts to the existing approaches you reference. What does the renderToString function do in addition to what linkedom offers and how does WESC compare to the referenced existing solutions?

Linkedom - This project is powered by Linkedom.
Ocean - Web component server-side rendering.
Web Components Compiler (WCC) - Experimental native Web Components compiler.
custom-elements-ssr - Renders Lit custom elements on the server.

First observation that Ocean and custom-elements-ssr are both built on top of linkedom., so only the web components compiler seems to be standalone. I will dive a bit deeper to see what they offer and where they potentially fall short or are too specific.

• Ocean is deno only for now and last has been contributed to 2 years ago. Streaming and rehydration sound interesting, though I would have to dive deeper to judge this solution. The author works on https://github.com/withastro/astro, and it might be worthwhile to see how astro solves this issue in a generic way. Regarding Ocean, while I am sympathetic to Deno, I think it would be great to have a runtime agnostic solution as much as possible. I referenced a way to potentially make it compatible with node Node / NPM support matthewp/ocean#41 (comment)
• custom-elements-ssr is actually not rendering lit custom elements, but aims to be generic, using a linkedom polyfill:

custom-elements-ssr
This package contains an Astro SSR integration to render vanilla custom elements on the server, as well as a @lit-labs/ssr compatible ElementRenderer for usage with @lit-labs/ssr.

Enables server-side rendering and client-side hydration for your Lit custom elements.

Try it on Stackblitz

Differences with lit SSR
It could be the case that you were hoping the Lit SSR package would also support vanilla Custom Elements, and were surprised to find that it didnt. The reason for this is that to render custom elements on the server side, we need some browser APIs to be available in a Node.js environment. Lit however, makes surprisingly little use of browser APIs to be able to do efficient rendering. This means that the DOM shim that Lit SSR requires is really, really minimal, and doesn't include a bunch of things, like for example querySelectors. This package instead makes use of linkedom to shim browser functionality on the server, which does include the required browser APIs to render custom elements on the server.

Additionally the ElementRenderer for vanilla custom elements is a little bit different from Lit elements.

Since this is not lit specific, it might be worth looking into the pros and cons of this solution, which seems to go in a generic direction.

I haven't seen WCC before, so I cannot comment much on that.

[luwes]

thank, great feedback!

I created WeSC to research and develop a way to write components closer to the metal, a HTML first approach which is not tied to Javascript, ideally it would be back-end language agnostic so it would be more like a superset of HTML.

Think something like Mustache but more geared towards the front-end with efficient DOM updates etc.
Some projects I helped with in that direction:
https://www.media-chrome.org/docs/en/themes
https://github.com/luwes/template-extensions

That's 1 aim I have with this project but it could also turn out to be just a best practices guide how to write great web components / custom elements on the front-side of things.
A problem to solve in this area would be how to separate the HTML from the JS to make it server-language agnostic.
An exploration in that area: https://github.com/luwes/html-module

---

But the current goal and problem I want to solve is SSR of DSD in the JS space mainly because we would like it in https://github.com/muxinc/media-chrome

What makes WeSC/server different is, it supports more frameworks and ways to render the DSD. workers, string, stream.
You can see which changes I made to Linkedom so many of the other solutions are lacking some important web api's.
video, slot, etc...

Our use case in Media Chrome is kind of special because the <media-controller> parent propagates state to its controls / children and it's also and async operation. I tested all the solutions and they did not work to SSR Media Chrome.
Lit SSR seems to work on a "per component" basis, it evaluates and renders the DSD separately for each component which would not work for Media Chrome because of the inter element communication. WeSC evaluates the whole document or document fragments.

I think Linkedom would always require a thin wrapper because, depending on the size of your webpage of course, you might not want to parse and render the whole web page but only process the custom elements in the page.

hydrating can be a pain but I'm starting to think that using vanilla JS custom elements is the most efficient way to do it.
it's very hard problem for frameworks to leave out the declarative views of the JS bundle even when they're already SSR'd so most of the times you end up serving the view (HTML) twice. once in the page, once in the JS bundle.
with vanilla custom elements it's pretty easy, I'll add an example of this approach soon!

Hope this answers some of your questions. Cheers

[ttraenkler]

I created WeSC to research and develop a way to write components closer to the metal, a HTML first approach which is not tied to Javascript, ideally it would be back-end language agnostic so it would be more like a superset of HTML.

Think something like Mustache but more geared towards the front-end with efficient DOM updates etc. Some projects I helped with in that direction: https://www.media-chrome.org/docs/en/themes https://github.com/luwes/template-extensions

That's 1 aim I have with this project but it could also turn out to be just a best practices guide how to write great web components / custom elements on the front-side of things. A problem to solve in this area would be how to separate the HTML from the JS to make it server-language agnostic. An exploration in that area: https://github.com/luwes/html-module

Interesting, recently I have also written some light "HTML first" web components helper functions which generate templates from plain HTML and CSS files and custom element classes from templates and lifecycle methods (optionally wrapping the template in a shadow root). For updates I also chose the plain DOM APIs, resolving the id and class name collisions with shadow DOM instead of using custom incompatible component file formats or frameworks. That's where linkedom comes in on the server or in a build step when generating static sites.

But the current goal and problem I want to solve is SSR of DSD in the JS space mainly because we would like it in https://github.com/muxinc/media-chrome

What makes WeSC/server different is, it supports more frameworks and ways to render the DSD. workers, string, stream. You can see which changes I made to Linkedom so many of the other solutions are lacking some important web api's. video, slot, etc...

Yeah I think this is a common goal we have, that's motivating me to collaborate on contributing to linkedom.

Our use case in Media Chrome is kind of special because the <media-controller> parent propagates state to its controls / children and it's also and async operation. I tested all the solutions and they did not work to SSR Media Chrome. Lit SSR seems to work on a "per component" basis, it evaluates and renders the DSD separately for each component which would not work for Media Chrome because of the inter element communication. WeSC evaluates the whole document or document fragments.

In my web components helper library, I made the shadow root optional for that reason, but I would be interested to see how to communicate best across shadow DOM boundaries, since without this you cannot really exchange components with others without the risk of collisions. I have a few directions I would like to try out but at the moment getting DSD and SSR to work with linkedom would come first.

I think Linkedom would always require a thin wrapper because, depending on the size of your webpage of course, you might not want to parse and render the whole web page but only process the custom elements in the page.

hydrating can be a pain but I'm starting to think that using vanilla JS custom elements is the most efficient way to do it. it's very hard problem for frameworks to leave out the declarative views of the JS bundle even when they're already SSR'd so most of the times you end up serving the view (HTML) twice. once in the page, once in the JS bundle. with vanilla custom elements it's pretty easy, I'll add an example of this approach soon!

IIRC, I read the way to do this with plain vanilla JS custom elements would be to detect if the shadow root already exists in the constructor of the custom element.

I will have a look at the projects you referenced in more detail when I have some spare time, and would also share the helper functions I have written, looking for feedback.