Skip to content

Document consume_schema_strategy default per release line (incl. 3.4, proton-enterprise#12508) - #702

Merged
kuiliu88 merged 3 commits into
kuiliu88-consume-schema-strategy-defaultfrom
kuiliu88-consume-schema-strategy-default-3.4
Oct 29, 2026
Merged

kuiliu88 merged 3 commits into
kuiliu88-consume-schema-strategy-defaultfrom
kuiliu88-consume-schema-strategy-default-3.4

Conversation

@kuiliu88

@kuiliu88 kuiliu88 commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

What

Companion to #701 (open); stacked on #701's branch so the diff is only this delta. Retarget to main after #701 merges.

The docs repo is single-branch (main), but the product ships from several lines (2.8.x, 3.2.x, 3.3.x, 3.4.x, 3.5 on develop). The consume_schema_strategy default differs per line, so the reference page now states it per release line in one table instead of scoping prose to one line. Triggered by timeplus-io/proton-enterprise#12508 ("Port/kafka fixes to 3.4", merged to 3.4 2026-10-29), which gives 3.4 the same implicit-default resolver as 3.3.

Changes

  • docs/shared/kafka-external-stream.md
    • New "Availability and default by release" table ({#consume_schema_strategy_versions}) replacing the 3.3.x-only "Default value" paragraph: per-release consume_schema_strategy availability, default when not set, and schema_message_name availability.
    • schema_message_name availability now points at that table.
    • count() note: 3.4.1 returns the raw Kafka message count.
    • Downgrade note made version-neutral; new upgrade note for 3.4.1 -> 3.4.2+ (TopicNameStrategy streams change single -> all, escape hatch consume_schema_strategy='single').
  • docs/shared/kafka-external-stream.md (commit 9c61597): documents two 3.4.1 single-mode gaps, both live-verified on 3.4.1 and 3.4.2-rc.1 and fixed on 3.3.4 / 3.4.2-rc.2 / 3.5.1-rc.0: a Protobuf sibling message in the same schema ID is not skipped, and (Avro, evolved subject) frames written with an earlier subject version are dropped. Added to the schema_message_name bullet, below the availability table, and to the 3.4.1 -> 3.4.2+ upgrade bullet.
  • docs/kafka-schema-registry.md, docs/shared/kafka-external-stream-read.md: per-strategy default sentence links to the table instead of repeating version lists.

Per-line facts

Source read at each ref (ExternalStreamSettings.h, MergeTreeSettings.h), updated 2026-10-29 ~15:10 UTC. "Live" = run on the official image on the local lab (Avro and Protobuf, same topic bytes, 11-12 cases, 35 variants). Results: ~/hotfix-qa/v3.4.1-..., v3.4.2-rc.1-..., v3.5.1-rc.0-consume-schema-strategy-validation/03-RESULTS.md and /home/test/hotfix-qa-342rc2/03-RESULTS.md (rc.2; the root disk was full).

ref consume_schema_strategy default when unset schema_message_name ttl_move_grace_seconds verified
v2.8.49, v3.2.15, v3.3.3 absent - absent absent source only
v3.3.4 present "" -> derived (#12502) present 3600 live (3.3.4 round)
v3.4.1, v3.4.2-rc.1 present fixed single absent 300 live on both (identical, 35/35 variants)
v3.4.2-rc.2 (d3c868b5 = 1a1a6d7f + version bump = 3.4 HEAD) present "" -> derived (#12508) present 300 live (35/35 variants identical to 3.5.1-rc.0 and to 3.3.4; 19 differ from 3.4.1)
v3.5.1-rc.0 (2af5614e = #12510 merge + version bump) present "" -> derived (#12510) present 3600 (source, develop) live (35/35 variants identical to 3.3.4)

Live-verified on 3.4.1 / 3.4.2-rc.1: with TopicNameStrategy or unset, frames with a foreign schema id are silently skipped; count() returns the raw message count (4 vs 2); schema_message_name -> code 115; consume_schema_strategy='' -> 471. Also in the docs text since 9c61597: with single, a Protobuf sibling message in the same schema id is not skipped, and (Avro, evolved subject) frames written with an earlier subject version are dropped (only the latest schema id is accepted). The earlier-version case was only run for Avro; the text says so.

Live-verified on 3.4.2-rc.2 and 3.5.1-rc.0 (identical on all 35 variants): default derived from subject_name_strategy exactly as on 3.3.4 (unset/TopicNameStrategy -> all 4 rows; Record/TopicRecord/custom -> Order only, count() 2); sibling and earlier-version frames handled; schema_message_name and '' accepted.

Why this is still a draft

Blocking (do not mark ready until resolved):

  1. Stacked on Document consume_schema_strategy implicit default (3.3.4) #701, which is open and blocked (review required). Either fold this into Document consume_schema_strategy implicit default (3.3.4) #701, or merge Document consume_schema_strategy implicit default (3.3.4) #701 first and retarget this PR to main.
  2. Docs text lacks the two live-verified 3.4.1 facts Done in 9c61597 (see Changes). Remaining caveat: the earlier-schema-version behavior was live-tested on Avro only; the Protobuf equivalent is not tested.
  3. No GA release for the versions named. Only candidates exist: v3.4.2-rc.2 and v3.5.1-rc.0. The table says "3.4.2 and later": live-verified true for rc.2 and false for rc.1, so any 3.4.2 GA must be cut from rc.2 or later. A "3.5.1 and later" row is not in the text yet; add it with the real number once 3.5.1 is released. 3.5.0 and earlier are not covered (before #12510, develop had a fixed all default).

Evidence gaps (weaken the claims, not blocking by themselves):
4. The 3.4.1 -> 3.4.2+ upgrade note (TopicNameStrategy single -> all) is derived from source, not from an in-place upgrade on one volume (also not run: ALTER + restart on the fixed build, downgrade). A 3.4.1 -> 3.5.x swap is a major-line jump and would not prove it.
5. 2.8.x / 3.2.x rows rest on the setting being absent in v2.8.49 / v3.2.15 source (local images exist; the Unknown-setting (115) check has not been run).
6. docusaurus build passes locally; spellcheck not run (binary not installed in the clone).

Closed today: 3.4 HEAD live behavior (rc.2), develop/3.5 live behavior (3.5.1-rc.0), ttl_move_grace_seconds values checked at the tags, blocker 2 (docs text, 9c61597).

Refs: proton-enterprise#12508, #12509 (closed), #12510 (merged); docs#701, docs#699.

@kuiliu88 kuiliu88 changed the title Sync consume_schema_strategy docs with 3.4 (proton-enterprise#12508) Document consume_schema_strategy default per release line (incl. 3.4, proton-enterprise#12508) Oct 29, 2026
@kuiliu88
kuiliu88 merged commit 9c61597 into kuiliu88-consume-schema-strategy-default Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant