Proposal: Make public some members of System.Text.Encoding

[aka-nse]

Hello.

I’d like to propose making the following System.Text.Encoding members public:

• DecodeFirstRune Encoding.Internal.cs#L74-L78
• EncodeRune Encoding.Internal.cs#L80-L84
• TryGetByteCount Encoding.Internal.cs#L100-L117

Motivation

These APIs would enable efficient, Unicode scalar value–based text processing over data in arbitrary encodings.
Today, when working with encoded bytes, common workflows often require converting to UTF-16 (char/string) first, even when the caller only needs scalar-wise inspection or partial validation.
This adds avoidable allocation / conversion overhead and makes some incremental scenarios awkward.

Compatibility / behavior note

Currently these base implementation throws NotImplementedException.
Usually this exception means forgotten implementation, and it implies not to be appropriate for formal release.
I believe they should throw NotSupportedException insteadly when they get public.

[huoyaoyuan]

It feels weird to operate in runes with arbitrary encoding. Shouldn't all of the legacy non-UTF encodings be covered by UCS-2 range? Then they can be efficiently processed in UTF-16 chars.

For UTF-8 and UTF-16 processing, Span<char> -> Span<Rune> and Span<byte> -> Span<Rune> routines are useful. For the equivalent of the aforementioned APIs which process in single runes, there are already Rune.DecodeFromUtf8 and Rune.DecodeFromUtf16.

[aka-nse]

It feels weird to operate in runes with arbitrary encoding. Shouldn't all of the legacy non-UTF encodings be covered by UCS-2 range? Then they can be efficiently processed in UTF-16 chars.

First, even popular legacy encoding, there are characters which cannot map into BMP.
For example, Shift-JIS: commonly used Japanese encoding contains about 300 characters which get surrogate pair.
Windows CP932 is not same with Shift-JIS_2004, so that the encoding which called "Shift-JIS" in System.Text.Encoding.CodePages does not contain Supplementary Planes character. I apologize you.

Second, char base sequential APIs (such as DecodeFirstChar, EncodeChar, or TryGetCount(char, out int) ) exist neither in public or non-public.
At least these APIs in my proposal already exist and it is only required to change accessibility modifier (and exception type).
So, its risk seems less than appends new sequential APIs with char base.

[huoyaoyuan]

Second, char base sequential APIs (such as DecodeFirstChar, EncodeChar, or TryGetCount(char, out int)) exist neither in public or non-public.
At least these APIs in my proposal already exist and it is only required to change accessibility modifier (and exception type).

This doens't really matter. There isn't an action to "change visibility". Anything new in public must go through the API Review process. The internal prototypes and helpers are often changed in the review.

If you think they are important enough, you can open an API proposal, and propose the API as-if the internal methods doesn't exist.