Repository navigation
review-prs: link a finding to the rule it breaks, and propose a missing rule - #197
Merged
Merged
Conversation
…ng rule The bug pass and the project-guidance pass write findings with no rule link, so on bravebot 18 of 19 High comments carried none even where a documented rule covered them. With review.linkRules the validator gets an index of the rules the PR was checked against and names the rule in cited_rule_link. It is a separate field because rule_link is a dedup key in post-review.py: two findings cited to the same rule in one file would collapse into one comment. collect-results.py keeps a cited link only when it names a rule in the docs and takes the link text from them. With review.proposeBestPractices the validator may describe one rule the project lacks, and an automatic run files it as a "Best practice:" issue through file-proposals.py: one per PR, three per run, skipped when a rule heading or an earlier issue already says it, with mentions and foreign links removed. Both keys are set for bravebot only.
bbondy
approved these changes
Oct 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
User impact: a bravebot review comment from the bug pass or the project-guidance pass now links the best-practice rule it breaks, and a review can open an issue proposing a rule the project lacks.
The problem
Bravebot's bug pass and project-guidance pass are told to write findings with no rule link, and those are the High comments. Several break a documented rule: a test that passes with the fix reverted, a spec the change no longer matches. Of the reviewer's last 49 comments, 1 of 18 High comments links a rule, against all 31 Medium and Low ones. A review that sees a rule the project is missing has nowhere to say so.
Reproduce
Run from any directory with
ghlogged in. Observed today:High1 linked of 18,Medium11 of 11,Low20 of 20. Expected: a High comment that breaks a documented rule links it too.tests/test_review_rules_and_proposals.pycovers the new behaviour and cannot run on the parent commit, which has none of the modules it loads.The fix
review.linkRuleshas the validator, which reads the source, name the rule a finding breaks. It is given an index of the rules that PR was checked against and writes the URL into a new field,cited_rule_link, notrule_link: post-review merges two findings that share arule_linkin one file, so reusing it would turn two real findings into one comment. The collector keeps the link only if the anchor exists and takes the link text from the docs, not from the model.review.proposeBestPracticeslets the validator describe one rule the project lacks. An automatic run files it as an issue after posting. A sample, rendered by the real code and not filed:One issue per PR and three per run, none when a rule heading or an earlier issue already says it, mentions and links off the repo removed, and a failure is logged and never fails the review. Without
--autoit only prints. Both keys are set in bravebot's profile and not in brave-core's.Test plan
make lint— passedpython3 -m pytest tests/test_review_rules_and_proposals.py— 51 passedpython3 -m pytest testson the rebased branch — 1451 passed, 8 failed; the same 8 fail on master without this change, and 4 of them only fail with this host's localconfig.jsonmake check-reviewdog— no findings from opengrep on this branch; it and pip-audit report "could not run cleanly" on this host (nolandrun), so the dependency audit did not runFaults put back one at a time, and the test that caught each
rule_linktest_a_cited_link_that_names_a_rule_is_kept_with_the_docs_headingtest_a_cited_link_that_names_no_rule_is_dropped_and_the_finding_staystest_the_index_lists_the_rules_the_pr_was_checked_againsttest_mentions_are_defused_and_foreign_links_removedtest_at_most_one_per_pr_and_a_few_per_runlinkRulestest_bravebot_asks_for_both_and_brave_core_for_neither