Bayesian git bisection!
Use this to detect changes in likelihoods of events, for instance, to isolate a commit where a slightly flaky test became very flaky.
You don't need to know the likelihoods (although you can provide priors), just that something has changed at some point in some direction
Or:
uv tool install git_bayesect
git_bayesect uses Bayesian inference to identify the commit introducing a change, with
commit selection performed via greedy minimisation of expected entropy, and using a Beta-Bernoulli
conjugacy trick while calculating posterior probabilities to make handling unknown failure rates
tractable.
See https://hauntsaninja.github.io/git_bayesect.html for a write up.
Start a Bayesian bisection:
git bayesect start --old $COMMIT
Record an observation on the current commit:
Or on a specific commit:
git bayesect pass --commit $COMMIT
Check the overall status of the bisection:
Reset:
Set the prior for a given commit:
git bayesect prior --commit $COMMIT --weight 10
Set prior for all commits based on filenames:
git bayesect priors_from_filenames --filenames-callback "return 10 if any('suspicious' in f for f in filenames) else 1"
Set prior for all commits based on the text in the commit message + diff:
git bayesect priors_from_text --text-callback "return 10 if 'timeout' in text.lower() else 1"
Get a log of commands to let you reconstruct the state:
Undo the last observation:
Run the bisection automatically using a command to make observations:
Checkout the best commmit to test:
This repository contains a little demo, in case you'd like to play around:
# Create a fake repository with a history to bayesect over
python scripts/generate_fake_repo.py
cd fake_repo
# The fake repo contains a script called flaky.py
# This is a simple script that fails some fraction of the time
# At some point in the history of the repo, that fraction was changed
python flaky.py
git log --oneline
# Start the bayesection
OLD_COMMIT=$(git rev-list HEAD --reverse | head -n 2 | tail -n 1)
git bayesect start --new main --old $OLD_COMMIT
# Run a bayesection to find the commit that introduced the change
git bayesect run python flaky.py