Skip to content

CBG-5900: Add support to compact channel history with Channel Names - #8857

Merged
bbrks merged 6 commits into
mainfrom
CBG-5900
Oct 28, 2026
Merged

bbrks merged 6 commits into
mainfrom
CBG-5900

Conversation

@RIT3shSapata

@RIT3shSapata RIT3shSapata commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

CBG-5900

Add a channels option to the document channel history compact endpoint, POST /{keyspace}/_channel_history/{docid}/compact. You can now name the channels to compact. Before, you could only give a sequence number.

  • Add an optional channels array to CompactDocChannelHistoryRequest. A request must set exactly one of seq or channels.
  • Return 400 if a request sets both, or sets neither. An empty channels array counts as not set.
  • Add a channels parameter to CompactDocChannelHistory. With seq set to zero, the function removes the history of the named channels.
  • In channels mode, SG removes every history entry for the named channels, even if the document is still in one of them. The document's current channel membership does not change.
  • Update the OpenAPI spec for the endpoint: the request body, the oneOf rule, and the 400 description.
  • Parameterize the existing test to run for both seq and channel mode

Pre-review checklist

  • Logging sensitive data? Make sure it's tagged (e.g. base.UD(docID), base.MD(dbName))
  • Updated relevant information in the API specifications (such as endpoint descriptions, schemas, ...) in docs/api

Dependencies (if applicable)

  • Link upstream PRs
  • Update Go module dependencies when merged

Integration Tests

@RIT3shSapata
RIT3shSapata requested review from a team and a balanced review from Copilot October 6, 2026 12:40
@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Redocly previews

@RIT3shSapata RIT3shSapata changed the title Add support to compact channel history with Channel Names CBG-5900: Add support to compact channel history with Channel Names Oct 6, 2026
@factory-droid

factory-droid Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Droid finished @RIT3shSapata's task —— View job


Review summary

This PR's overall approach (mutual exclusivity between seq and channels, plus tests and OpenAPI updates) looks solid. The main correctness edge case is that compaction can currently delete Channels-map entries with Seq==0, even though those entries are treated as current memberships by getCurrentChannels.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Channel matching introduces quadratic processing for large documents and channel lists.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
What changed in this PR

Adds channel-name-based document channel-history compaction alongside sequence-based compaction.

Changes:

  • Adds mutually exclusive seq and channels request modes.
  • Implements channel-name filtering while preserving active memberships.
  • Updates API documentation and parameterizes compaction tests.
File Description
db/​crud.go Implements channel-based compaction.
rest/​doc_api.go Validates and handles the new request mode.
rest/​api_test.go Tests both compaction modes.
rest/​doc_api_test.go Tests request validation.
docs/​api/​paths/​admin/​keyspace-_channel_history-compact.yaml Documents the new API option.
db/​import_test.go Updates the compaction call signature.
db/​hybrid_logical_vector_test.go Updates the compaction call signature.
db/​attachment_test.go Updates the compaction call signature.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread db/crud.go Resolved
Comment thread db/crud.go
return del
})

for chanName, chanEntry := range doc.SyncData.Channels {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Avoid compacting Channels entries with Seq==0

SyncData.getCurrentChannels treats both nil removals and non-nil removals with Seq==0 as current channel memberships. CompactDocChannelHistory currently deletes non-nil entries whenever chanEntry.Seq <= seq (seq mode) or when the channel name is listed (channels mode), which can drop a channel that is still considered current and change the effective channel set.

Consider skipping deletions for chanEntry.Seq==0 (consistent with GetDocChannelHistory, which only reports removals where Seq != 0), for example:

Suggested change
for chanName, chanEntry := range doc.SyncData.Channels {
var del bool
if seq != 0 {
del = chanEntry.Seq != 0 && chanEntry.Seq <= seq
} else if len(channels) > 0 {
del = chanEntry.Seq != 0 && slices.Contains(channels, chanName)
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this comment accurate? That without the check on chanEntry.Seq != 0, this has the potential to remove active channels as well (which we don't want).

Please review and add a test for coverage of this case if we don't already have one.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the thing is chanEntry.Seq can never be set to 0? I don't see in any scenario a channel removal will have an entry of 0. Because if a channel is assigned then it will have a non zero value, then on removal I don't see SGW assigning a sequence of 0. The condition that I added in the function that droid pointed out is more of a defensive check.

This will not remove active channels because active channels usually have chanEntry is of type ChannelRemoval, and its nil for active channels.

I modified the test "multiple channels with mixed history" to test this scenario out better. Please let me know if this is okay

@RIT3shSapata
RIT3shSapata requested a review from bbrks October 6, 2026 14:17
@RIT3shSapata RIT3shSapata assigned bbrks and adamcfraser and unassigned bbrks Oct 6, 2026
Comment thread db/crud.go Resolved
Comment thread db/crud.go Resolved
Comment thread rest/doc_api.go

type CompactDocChannelHistoryRequest struct {
Seq uint64 `json:"seq"`
Seq uint64 `json:"seq"`

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realize it was the previous handling, but why didn't we allow setting seq=0 for users that wanted to delete all channel history? They needed to use seq=1 for that?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since I've set the type of seq to uint64, there is no definitive way to distinguish between no seq value being passed and setting the seq value to 0. Another thing is while implementing this initially, there was no discussion of completely deleting the channel history. All the requirements pointed to removing history before a certain sequence.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh that's my mistake - I'd forgotten that this API compacts everything older than a given sequence, not everything newer. There isn't really a use case for setting 0, then.

Comment thread rest/doc_api.go

type CompactDocChannelHistoryRequest struct {
Seq uint64 `json:"seq"`
Seq uint64 `json:"seq"`

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh that's my mistake - I'd forgotten that this API compacts everything older than a given sequence, not everything newer. There isn't really a use case for setting 0, then.

Comment thread db/crud.go Resolved
Comment thread db/crud.go
return del
})

for chanName, chanEntry := range doc.SyncData.Channels {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this comment accurate? That without the check on chanEntry.Seq != 0, this has the potential to remove active channels as well (which we don't want).

Please review and add a test for coverage of this case if we don't already have one.

@bbrks bbrks left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

API Spec description is slightly wrong and I think we can just remove the line.

Otherwise looks OK to me

@bbrks bbrks assigned RIT3shSapata and unassigned bbrks Oct 23, 2026
@RIT3shSapata
RIT3shSapata requested review from a team and bbrks October 26, 2026 14:06
@RIT3shSapata RIT3shSapata assigned bbrks and unassigned RIT3shSapata Oct 26, 2026
RIT3shSapata and others added 6 commits October 26, 2026 19:39
- compact channel history with channel names
- parameterize the existing tests to test for channel name based compaction
- update the REST API Endpoint
- update the API spec
- update inline comments and documentation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- change channels to compact from set to a map
- modify test case
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bbrks
bbrks merged commit 0d2e652 into main Oct 28, 2026
54 checks passed
@bbrks
bbrks deleted the CBG-5900 branch October 28, 2026 11:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants