Reproduce a reported bug against the last release
Check whether a bug is new, old, or already fixed.
QA · Software · When you ask · Updated
Works with
- Slack
SlackHow do you reproduce a reported bug against the last release?
To reproduce a reported bug against the last release, mention Maito with the report. It follows the steps in a browser on the current and previous release, records what happens, and finds the first commit where it broke. You learn whether it is new, old or fixed, with videos attached.
How it's done today
Engineers spend an hour finding where it started.
What Maito does
- Run the bug steps on current and previous versions
- Report where it started
Example
@Maito did the login bug exist in last week's release? Support says people with a + in their email can't sign in.
No. It started in 4.12. I tried sam+qa@northstar.example with the same password on both releases: • 4.11 (Oct 8): signs in • 4.12 (Oct 15): 'Invalid email address' Between the two tags there are 38 commits. The first bad one is 7c2d1e9, Tighten email validation (PR #860). The new pattern doesn't allow +. Catch: signup still accepts + addresses. Since 4.12, people can create accounts they can't log in to, so support will keep hearing about it. Attached: login-4.11.mp4 and login-4.12.mp4. Want me to open a fix PR?
@Maito yes, and tell Sam in support what to say meanwhile.
PR #924 is open: login and signup use the same email pattern, with a test for +. For Sam: a bug in 4.12 blocks + addresses, fix in review, workaround is signing in with the plain address if it's an alias.
Mia (11:40 AM):
did the login bug exist in last week's release? Support says people with a + in their email can't sign in.
Maito (12:02 PM):
No. It started in 4.12.
I tried sam+qa@northstar.example with the same password on both releases:
• 4.11 (Oct 8): signs in
• 4.12 (Oct 15): 'Invalid email address'
Between the two tags there are 38 commits. The first bad one is 7c2d1e9, Tighten email validation (PR #860). The new pattern doesn't allow +.
Catch: signup still accepts + addresses. Since 4.12, people can create accounts they can't log in to, so support will keep hearing about it.
Attached: login-4.11.mp4 and login-4.12.mp4. Want me to open a fix PR?
Mia (12:05 PM):
yes, and tell Sam in support what to say meanwhile.
Maito (12:21 PM):
PR #924 is open: login and signup use the same email pattern, with a test for +. For Sam: a bug in 4.12 blocks + addresses, fix in review, workaround is signing in with the plain address if it's an alias.
How to set it up
- Set up
Nothing to set up. Ask in Slack or by email whenever you need it.
What it won't do
A bug that depends on a specific customer's data or setup cannot be reproduced without it. It will ask for that instead of guessing.
Questions
Any tagged versions or builds your team can run. Tell it which two to compare.
It says so, lists exactly what it tried, and asks for missing details such as the account or browser.
Yes. It narrows down the commits between two releases and names the first one that breaks the behavior.
