Repository navigation
perf(media): decode inline images as thumbnails off the main thread - #232
Merged
Merged
Conversation
Inline images in the transcript, the queued sheet and Session Media tiles decoded at full size, and the CachedImage composable did it inside remember on the main thread. They now decode through CachedImage.thumbnail: BitmapFactory sampling to at most 1600 px on the long edge (keeping 800 px on the short edge for square crops), cached under a thumb: key, and loaded off the main thread. The full-screen viewer opens on the thumbnail and swaps in the original; Save and Share wait for the original. Ports iOS #299 (011ee6bd), without its disk-cache, probe and Markdown parts. Verified: ported 3 MediaDecodeTests into MediaReferenceTests (they fail with full-size decoding), vm.sh check for media, transcript and session, then a full vm.sh check; emulator before/after on session-lab session-media looks the same.
…t in Review fixes for the thumbnail change: - Save and Share decode the original off the main thread when only the thumbnail is on screen, instead of ignoring the tap while it loads or exporting the thumbnail when its decode failed. - The viewer keeps its zoom when the original replaces the thumbnail on the same page, and the loaded page drops its placeholder size. - A tile with only a media hash and no server shows nothing, as before; one whose bytes are in memory decodes without a spinner. - The downsampled path recycles its intermediate bitmap, a cached full-size original no longer stands in for a thumbnail, and small images skip the header read on repeat calls. Verified: vm.sh check :feature:media (the new Save/Share test and the updated swap test fail without this change); on the emulator with a private session-lab copy serving 5120x2880, 1000x6000, 6000x1000 and 2560x1440 images: tiles and Session Media show them downsampled at the right aspect, the viewer opens on the picture, a double-tap zoom during the swap holds, and Save writes a 5120x2880 file.
Session Media built image pages with no image, so with thumbnails the viewer opened on a spinner for a few frames while it decoded the original (dev hid this with its full-size decode). The page now starts on the cached thumbnail with the original bytes, like the transcript opener, and the viewer swaps the original in. Verified: new SessionMediaTests.aMediaCardOpensOnTheThumbnailItsTileShows fails without the change and passes with it; on the emulator with a 5120x2880 lab image, opening the Session Media tile shows the picture on its first frame (screen recording, no spinner frames); full scripts/check.sh on the VM.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Inline photos, tool screenshots and Session Media tiles now decode as thumbnails off the main thread, instead of at full size and sometimes on the main thread. A thumbnail is at most 1600 px on the long edge, and keeps at least 800 px on the short edge for square crops. Large pictures look the same or a little cleaner (less moiré). The full-screen viewer opens on the thumbnail and swaps in the original without resetting zoom, and Save and Share always export the original. Ports iOS #299 (011ee6bd), without its disk-cache, probe and Markdown parts.
Checked with a private session-lab copy whose images are 5120×2880, 1000×6000, 6000×1000 and 2560×1440:
Opening the viewer from the transcript (before, after) and from Session Media (before, after). On dev the transcript viewer flashes a spinner frame; here it opens on the picture and sharpens in place:
viewer-swap-before.mp4
viewer-swap-after.mp4
media-open-before.mp4
media-open-after.mp4
Verified: 3 iOS
MediaDecodeTestsported, plus tests for Save and Share exporting the original (and failing cleanly when it can't decode) and for Session Media opening on its thumbnail. All fail without the change. Fullscripts/check.shpassed on the VM. On the emulator I double-tap zoomed during the swap and the zoom held. Saving the 5120×2880 picture wrote a 5120×2880 file.Not changed here: EXIF orientation is still ignored everywhere, the same as on dev, so rotated JPEGs show as they did before. Decoding the photo on send and tapping a composer photo still run on the main thread, also the same as on dev.