fix(graphics): cap RLE decoder dimensions to prevent OOM from adversarial input - #1962
Merged
Marc-André Moreau (mamoreau-devolutions) merged 1 commit intoSep 22, 2026
Conversation
…rial input decompress_helper allocated its output buffer as dst.resize(row_delta * height, 0) with no cap on width/height, both of which come directly from TS_BITMAP_DATA's wire-decoded, unvalidated fields (MS-RDPBCGR 2.2.9.1.1.3.1.2.2, both 16-bit unsigned integers). Worst case: 65535 * 65535 * 3 bytes (24 bpp) is roughly 12.6 GB from a few attacker-controlled bytes. Added a per-axis MAX_DECODE_DIM = 8192 cap, checked before the allocation, matching the existing cap already used by ironrdp-graphics's ClearCodec decoder and MS-RDPBCGR's own documented maximum desktop width for current Windows RDP server versions (section 3.3.5.3.3, note 46). New RleError::DimensionsTooLarge variant carries the offending width/height for diagnostics.
Greg Lamberson (glamberson)
deployed
to
llm-providers
September 12, 2026 22:33 — with 
GitHub Actions
Active
GitHub Actions
Active
github-actions
Bot
added
needs-review
Marc-André Moreau (mamoreau-devolutions)
approved these changes
Sep 22, 2026
Marc-André Moreau (mamoreau-devolutions)
merged commit Sep 22, 2026
9b151c4
into
Devolutions:master
41 checks passed
holovchenko
pushed a commit
to holovchenko/IronRDP
that referenced
this pull request
Sep 25, 2026
…rial input (Devolutions#1962) ## Summary - `ironrdp_graphics::rle::decompress_helper` allocated its output buffer as `dst.resize(row_delta * height, 0)` with no cap on `width`/`height`, both of which come directly from `TS_BITMAP_DATA`'s wire-decoded, unvalidated fields (MS-RDPBCGR 2.2.9.1.1.3.1.2.2, both 16-bit unsigned integers). Worst case: 65535 * 65535 * 3 bytes (24 bpp) is roughly 12.6 GB from a few attacker-controlled bytes. - Added a per-axis `MAX_DECODE_DIM = 8192` cap, checked before the allocation, matching the existing cap already used by `ironrdp-graphics`'s ClearCodec decoder and MS-RDPBCGR's own documented maximum desktop width for current Windows RDP server versions (section 3.3.5.3.3, note 46). - New `RleError::DimensionsTooLarge` variant carries the offending width/height for diagnostics. ## Validation `cargo xtask check fmt/lints/tests/typos/locks` all pass. Added three regression tests: width over the limit, height over the limit, and a boundary test confirming `MAX_DECODE_DIM` itself is still accepted (not an off-by-one). ## Notes Filed as part of the audit tracked in Devolutions#1315. The existing `fuzz/fuzz_targets/rle_decompression.rs` target's `BitmapInput` generator currently caps width/height at `u8` (max 255), well under both the old and new limits, so it would not have found this on its own; widening that generator to the wire's actual `u16` range is a natural follow-up but is out of scope here since it also feeds three other oracles (`rdp6_encode_bitmap_stream`, `rdp6_decode_bitmap_stream_to_rgb24`, and one more) that have not been audited for the same class of issue. (cherry picked from commit 9b151c4)
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ironrdp_graphics::rle::decompress_helperallocated its output buffer asdst.resize(row_delta * height, 0)with no cap onwidth/height, both of which come directly fromTS_BITMAP_DATA's wire-decoded, unvalidated fields (MS-RDPBCGR 2.2.9.1.1.3.1.2.2, both 16-bit unsigned integers). Worst case: 65535 * 65535 * 3 bytes (24 bpp) is roughly 12.6 GB from a few attacker-controlled bytes.MAX_DECODE_DIM = 8192cap, checked before the allocation, matching the existing cap already used byironrdp-graphics's ClearCodec decoder and MS-RDPBCGR's own documented maximum desktop width for current Windows RDP server versions (section 3.3.5.3.3, note 46).RleError::DimensionsTooLargevariant carries the offending width/height for diagnostics.Validation
cargo xtask check fmt/lints/tests/typos/locksall pass. Added three regression tests: width over the limit, height over the limit, and a boundary test confirmingMAX_DECODE_DIMitself is still accepted (not an off-by-one).Notes
Filed as part of the audit tracked in #1315. The existing
fuzz/fuzz_targets/rle_decompression.rstarget'sBitmapInputgenerator currently caps width/height atu8(max 255), well under both the old and new limits, so it would not have found this on its own; widening that generator to the wire's actualu16range is a natural follow-up but is out of scope here since it also feeds three other oracles (rdp6_encode_bitmap_stream,rdp6_decode_bitmap_stream_to_rgb24, and one more) that have not been audited for the same class of issue.