Repository navigation
fix(uri_template): use surrogateescape for unquote and quote to preserve non-UTF-8 octets (#3721) - #3724
Conversation
…rve non-UTF-8 octets (modelcontextprotocol#3721)
|
This PR has been closed automatically. This repo only keeps pull requests open when they come from a maintainer, or from a contributor a maintainer has assigned to the linked issue, and you aren't currently assigned to #3721. If a maintainer assigns you to #3721, this PR reopens on its own and there's nothing more you need to do here. Assignment is a maintainer call based on capacity; comments that only ask to be assigned don't factor in. What does help is engaging on the issue itself by confirming the repro, explaining why it matters for your use case, or describing the approach you'd take. You're welcome to keep pushing commits here (just avoid force-pushing, since GitHub can't reopen a rewritten branch), but that on its own won't get the PR reviewed or the issue assigned, and realistically most auto-closed PRs stay closed. There's no need to open a new PR either way. CONTRIBUTING.md has the full reasoning, but in short:
Maintainers: reopen, remove |
Fixes #3721
Problem
RFC 3986 §2.1 percent-encoding encodes arbitrary octets, not UTF-8 characters exclusively. When matching URIs containing non-UTF-8 percent-encoded bytes (such as
file://caf%E9representing Latin-1café),UriTemplate.match()replaced undecodable bytes with Unicode replacement characterU+FFFD('caf\ufffd'). This destroyed the original byte data before reaching resource handlers and broke the bidirectional invertibility guarantee betweenmatch()andexpand()(expand({'name': 'caf\ufffd'})becomes'file://caf%EF%BF%BD').Root Cause
All
urllib.parse.unquote()calls acrosssrc/mcp/shared/uri_template.pyomitted theerrorsparameter, falling back to standard library defaulterrors="replace". Additionally,_encode()used standardquote()withouterrors="surrogateescape", causing re-expansion of surrogate-escaped strings to raiseUnicodeEncodeError.Changes
errors="surrogateescape"at all capture/unquote sites inuri_template.py:_parse_query(lenient query parameters)_extract_greedy(scalar and explode segments)_scan_suffix(ifemp named parameters and bounded suffix variables)_scan_prefix(bounded prefix variables)errors="surrogateescape"in_encode()forquote()so that surrogate-escaped strings accurately re-encode back to the original percent triplets.{var},{+var},{#var},{.var},{/var},{;var},{?var},{/list*}).Verification
tests/shared/test_uri_template.pypassed in 0.32s.