Skip to content

Conversation

Flat
Copy link

@Flat Flat commented Feb 11, 2025

Sets the default sixel ratio to 1:1 since most terminals either do not support ratios or default to another ratio. Disables background erase, this fixes the black backgrounds on transparent sixels. zellij-org/zellij#2048 (comment)

Before:
image

After:
image

Sets the default sixel ratio to 1:1 since most terminals either do not
support ratios or default to another ratio. Disables background erase,
this fixes the black backgrounds on transparent sixels.
zellij-org/zellij#2048 (comment)
@AutumnMeowMeow
Copy link

@Flat

@imsnif invited me to take a quick look. Looks good, I'd say go for it.

A couple comments:

According to VT340 doc, the "9" could by 7, 8, or 9. However, DEC STD 070 section 7.1 says definitively that "9" is 1:1 (actually 100:100). Just documenting here why the 9 is better if one desires 1:1 aspect ratio.

The background select parameter (0 or 2 for background color, 1 for transparency) would ideally be a bool that goes with the image data itself. It's basically a part of the image registers/palette, so would have the same life cycle. Since zellij keeps the original palette without transforming it, it should also keep this and replicate it on output. This way images that desire the background erase option can still have that. (Most images don't, so I'd say keep this for now and deal with it later if needed.)

Now having mentioned how transparency is part of the palette, so too is aspect ratio. If you add the transparency to the image metadata for decode/encode, then you can also add at that time the aspect ratio, and choose either to stretch the image vertically at input time (convert from x:1 to 1:1) or calculate its correct pixel aspect ratio and pass that along at output time (assuming the user-facing terminal will do the stretch correctly).

Implement parsing Device Control String and Raster Attributes for sixels
to pass onto terminals, with defaults to spec.
@Flat
Copy link
Author

Flat commented Feb 13, 2025

Thanks for the information on those it does help a lot!

I've done a bit more to try to properly handle the DCS and RA codes for sixels. This seems to still be fine for handling proper ratio and transparency.

Some sixel tests before and after these changes
image

For reference here is how they look in foot without zellij:
image

I believe the differences/corrections need to be made to how zellij renders sixels as the sixel strings now match what is being deserialized and serialized.

@AutumnMeowMeow
Copy link

That looks lovely! :)

@imsnif
Copy link
Member

imsnif commented Feb 24, 2025

Thanks for taking a look @AutumnMeowMeow and my apologies for taking so long to get to this!

The part I'm missing is this (and maybe I totally misunderstood, so please correct me on anything here): if both the transparency and the aspect ratio are part of the original image, wouldn't this PR essentially be hard-coding them to opaque and 1:1 respectively? If so, what actually happens now that we omit them? Why does the terminal emulator make a different decision than we would with said hard-coding?

@Flat
Copy link
Author

Flat commented Feb 24, 2025

Thanks for taking a look @AutumnMeowMeow and my apologies for taking so long to get to this!

The part I'm missing is this (and maybe I totally misunderstood, so please correct me on anything here): if both the transparency and the aspect ratio are part of the original image, wouldn't this PR essentially be hard-coding them to opaque and 1:1 respectively? If so, what actually happens now that we omit them? Why does the terminal emulator make a different decision than we would with said hard-coding?

The original first commit of this PR did hard code these values, however after reading more about the resources that AutumnMeowMeow linked, I have implemented parsing for these values from the sixel images, so they are no longer hard coded but read from the sixel. If they are omitted from the image these values will not be set in the sixel and the terminal emulator (or consumer of this library) should use the default values as specified in the sixel spec.

@Flat Flat changed the title fix(serializer): sixel ratio 1:1 no bg erase fix(serializer): Implement sixel DCS and RA parsing Feb 24, 2025
@AutumnMeowMeow
Copy link

Yup. The current code looks to be a complete deserializer/reserializer of the entire sixel palette/metadata. :)

For zellij (or any other multiplexer using this), there will be some additional work to calculate the appropriate cutoffs so that images with different vertical aspect ratios will overlay/truncate properly along the terminal grid (esp. with a floating terminal obscuring images behind it). That could potentially come back as a "convert this w:h image to 1:1" function in this library, but it doesn't have to: zellij could also just horizontally/vertically truncate (x/w:y/h) on output, knowing that the terminal will handle the stretch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants