detect_text_encoding does not trim trailing NULs, so NUL-padded text is reported as binary #3506

Summary

Before it classifies an encoding, libmagic strips the trailing NUL bytes from the buffer. PolyFile's
detect_text_encoding does not, so a buffer that is text followed by NUL padding is classified as
binary and falls through to application/octet-stream.

The two inputs differ only by four trailing NULs, and file classifies both as text.

Reproducer

$ printf '\xde\xca\xff\xed\x00\x00\x00\x00' > card.bin
$ ./file/src/file -m tests/magic.mgc -i --keep-going card.bin
card.bin: text/plain; charset=binary
$ ./file/src/file -m tests/magic.mgc --keep-going card.bin
card.bin: ISO-8859 text, with no line terminators

from polyfile.magic import MAGIC_DEFS, MagicMatcher, MatchContext, detect_text_encoding

matcher = MagicMatcher.parse(*MAGIC_DEFS)
print(detect_text_encoding(open("card.bin", "rb").read()))
print({mt for m in matcher.match(MatchContext.load("card.bin", only_match_mime=True))
       for mt in m.mimetypes})

None
{'application/octet-stream'}

Expected iso-8859-1 and {'text/plain'}. Dropping the four trailing NULs from the input already
produces both, so the classifier is correct and only the missing trim step separates the two
results.

Cause

file_ascmagic copies the buffer, trims its trailing NULs, and only then calls file_encoding
(file/src/ascmagic.c:82-89):

bb = *b;
bb.flen = trim_nuls(CAST(const unsigned char *, b->fbuf), b->flen);
/*
 * Avoid trimming at an odd byte if the original buffer was evenly
 * sized; this avoids losing the last character on UTF-16 LE text
 */
if ((bb.flen & 1) && !(b->flen & 1))
        bb.flen++;

trim_nuls (file/src/ascmagic.c:62-67) leaves at least one byte:

while (nbytes > 1 && buf[nbytes - 1] == '\0')
        nbytes--;

detect_text_encoding (polyfile/magic.py:3031) has no equivalent step. It classifies the raw
buffer, _only_contains(data, _TEXT_BYTES) fails on the NULs, and it returns None.
MagicMatcher.match then takes the OctetStreamTest branch (polyfile/magic.py:3453-3456).

A faithful port needs both halves: the trim itself and the odd-byte adjustment, which adds one byte
back when the trimmed length is odd and the original length was even, so that the last character of
UTF-16LE text survives.

Why file prints text/plain; charset=binary

The pairing looks self-contradictory but is intended, and it is worth recording because it makes the
expected value confusing to read. file_buffer classifies the buffer twice:

• funcs.c:351 initializes code_mime = "binary", and funcs.c:369-370 calls file_encoding on
  the untrimmed buffer. The NULs make that call fail, so code_mime keeps its default and the
  ; charset= suffix at funcs.c:517-521 reports binary.
• funcs.c:498 calls file_ascmagic, which classifies the trimmed buffer and supplies the
  text/plain type.

So the charset comes from the untrimmed buffer and the type from the trimmed one. Only the type is
at issue here; PolyFile emits no charset at all, which is #3488.

Impact

pocs-master/mocks/card.bin in the corkami corpus, currently skipped by KNOWN_BAD_FILES in
tests/test_corkami.py. Its skip comment claimed PolyFile was the more accurate of the two, which
no longer holds: after #3468 pinned the character class table to libmagic's text_chars, five of
the seven sibling mocks in that list began agreeing with file, and this one is left behind by the
missing trim rather than by a deliberate difference.

More generally this affects any NUL-padded text: fixed-width records, padded string tables, and
files rounded up to a block boundary.

The fix is small and self-contained in detect_text_encoding. It is independent of #3488, which
changes when the plain-text result is emitted rather than whether these bytes count as text.
Related to #3507, the EBCDIC gap in the same function.