Skip to content

perf(media): decode inline images as thumbnails off the main thread - #232

Merged
Tarquinen merged 4 commits into
devfrom
media-thumbnails
Nov 21, 2026
Merged

Tarquinen merged 4 commits into
devfrom
media-thumbnails

Conversation

@Tarquinen

@Tarquinen Tarquinen commented Nov 20, 2026 •

Copy link
Copy Markdown
Member

Inline photos, tool screenshots and Session Media tiles now decode as thumbnails off the main thread, instead of at full size and sometimes on the main thread. A thumbnail is at most 1600 px on the long edge, and keeps at least 800 px on the short edge for square crops. Large pictures look the same or a little cleaner (less moiré). The full-screen viewer opens on the thumbnail and swaps in the original without resetting zoom, and Save and Share always export the original. Ports iOS #299 (011ee6bd), without its disk-cache, probe and Markdown parts.

Checked with a private session-lab copy whose images are 5120×2880, 1000×6000, 6000×1000 and 2560×1440:

Before (dev) After
Transcript Transcript before Transcript after
Session Media Session Media before Session Media after
Small images, light Small before Small after

Opening the viewer from the transcript (before, after) and from Session Media (before, after). On dev the transcript viewer flashes a spinner frame; here it opens on the picture and sharpens in place:

viewer-swap-before.mp4
viewer-swap-after.mp4
media-open-before.mp4
media-open-after.mp4

Verified: 3 iOS MediaDecodeTests ported, plus tests for Save and Share exporting the original (and failing cleanly when it can't decode) and for Session Media opening on its thumbnail. All fail without the change. Full scripts/check.sh passed on the VM. On the emulator I double-tap zoomed during the swap and the zoom held. Saving the 5120×2880 picture wrote a 5120×2880 file.

Not changed here: EXIF orientation is still ignored everywhere, the same as on dev, so rotated JPEGs show as they did before. Decoding the photo on send and tapping a composer photo still run on the main thread, also the same as on dev.

Inline images in the transcript, the queued sheet and Session Media tiles
decoded at full size, and the CachedImage composable did it inside
remember on the main thread. They now decode through
CachedImage.thumbnail: BitmapFactory sampling to at most 1600 px on the
long edge (keeping 800 px on the short edge for square crops), cached
under a thumb: key, and loaded off the main thread. The full-screen viewer
opens on the thumbnail and swaps in the original; Save and Share wait for
the original. Ports iOS #299 (011ee6bd), without its disk-cache, probe
and Markdown parts.

Verified: ported 3 MediaDecodeTests into MediaReferenceTests (they fail
with full-size decoding), vm.sh check for media, transcript and session,
then a full vm.sh check; emulator before/after on session-lab
session-media looks the same.
…t in

Review fixes for the thumbnail change:
- Save and Share decode the original off the main thread when only the
  thumbnail is on screen, instead of ignoring the tap while it loads or
  exporting the thumbnail when its decode failed.
- The viewer keeps its zoom when the original replaces the thumbnail on the
  same page, and the loaded page drops its placeholder size.
- A tile with only a media hash and no server shows nothing, as before; one
  whose bytes are in memory decodes without a spinner.
- The downsampled path recycles its intermediate bitmap, a cached full-size
  original no longer stands in for a thumbnail, and small images skip the
  header read on repeat calls.

Verified: vm.sh check :feature:media (the new Save/Share test and the updated
swap test fail without this change); on the emulator with a private
session-lab copy serving 5120x2880, 1000x6000, 6000x1000 and 2560x1440
images: tiles and Session Media show them downsampled at the right aspect,
the viewer opens on the picture, a double-tap zoom during the swap holds, and
Save writes a 5120x2880 file.
Session Media built image pages with no image, so with thumbnails the
viewer opened on a spinner for a few frames while it decoded the original
(dev hid this with its full-size decode). The page now starts on the cached
thumbnail with the original bytes, like the transcript opener, and the
viewer swaps the original in.

Verified: new SessionMediaTests.aMediaCardOpensOnTheThumbnailItsTileShows
fails without the change and passes with it; on the emulator with a
5120x2880 lab image, opening the Session Media tile shows the picture on
its first frame (screen recording, no spinner frames); full scripts/check.sh
on the VM.
@Tarquinen
Tarquinen merged commit edbfef2 into dev Nov 21, 2026
2 checks passed
@Tarquinen
Tarquinen deleted the media-thumbnails branch November 21, 2026 03:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant