Skip to content

gh-71986: Fix the base64 and bz2 codec incremental decoders - #160981

Open
b84330808 wants to merge 1 commit into
python:mainfrom
b84330808:gh-71986-base64-bz2-incremental-decoders
Open

b84330808 wants to merge 1 commit into
python:mainfrom
b84330808:gh-71986-base64-bz2-incremental-decoders

Conversation

@b84330808

@b84330808 b84330808 commented Oct 30, 2026 •

Copy link
Copy Markdown

The base64_codec incremental decoder decoded every chunk on its own, so splitting the input inside a group of four base64 characters failed:

>>> b"".join(codecs.iterdecode([b"AA", b"AA\r\n"], "base64"))
binascii.Error: Incorrect padding

It now subclasses codecs.BufferedIncrementalDecoder and decodes only complete groups of four alphabet characters, keeping the rest until the next call (or final=True). Bytes outside the alphabet (newlines, =) are not counted, since binascii.a2b_base64() ignores them in non-strict mode, so the joined output matches decoding the whole input at once.

The bz2_codec incremental decoder returned '' (a str) when called after the end of the compressed data; it now returns b''.

Martin Panter attached a patch to the original bpo issue in 2016. This is a new implementation against current main: that patch counted = as a data character, which no longer matches a2b_base64() (b"AA=AA=" decodes, but its first four characters b"AA=A" don't).

I used Claude Code to write this change.

The base64_codec incremental decoder decoded each chunk on its own, so
input split in the middle of a group of four characters failed with
binascii.Error. Keep the incomplete group until the next call.

The bz2_codec incremental decoder returned a str when called after the
end of the compressed data. Return b'' instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@python-cla-bot

python-cla-bot Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

All commit authors signed the Contributor License Agreement.

CLA signed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant