Test a branch in a real browser before merge
Every change gets clicked through in a real browser, with a video.
QA · Software · When something happens · Updated
Works with
- Slack
SlackHow do you test a branch in a real browser before merge?
To test a branch in a real browser before merge, set a Loop in Maito that starts when a pull request opens. It opens the app, clicks through the flows the change touches at desktop and phone width, records a video, and posts passes and failures in your QA channel. It retests when you push fixes.
How it's done today
Testing happens on staging after merge, if at all.
What Maito does
- Start the app at the branch
- Run the user flows the change touches
- Record a video and report problems
Example
Loop run: Pull request opened · #881 Add coupon codes to checkout
Browser test of PR 881 is done: signup, add to cart, checkout with a coupon, remove the coupon. Desktop (1440) and phone (390). • Pass: SAVE10 takes $12.40 off a $124.00 cart. Total $111.60. • Pass: an invalid code shows an error. • Fail: at 390 px the coupon field overlaps the Pay button. Catch: removing the coupon after entering shipping leaves the total at $111.60 on screen. The confirmation page then says $124.00. The order is right, but customers will see two different totals. Attached: pr-881-checkout.mp4. Jess, both are yours.
@Maito I pushed fixes. Test again on 390 and desktop.
Retested on the latest commit. The Pay button is clear and the total updates to $124.00 when the coupon is removed. All 4 checks pass. New video attached.
Maito (4:20 PM):
Browser test of PR 881 is done: signup, add to cart, checkout with a coupon, remove the coupon. Desktop (1440) and phone (390).
• Pass: SAVE10 takes $12.40 off a $124.00 cart. Total $111.60.
• Pass: an invalid code shows an error.
• Fail: at 390 px the coupon field overlaps the Pay button.
Catch: removing the coupon after entering shipping leaves the total at $111.60 on screen. The confirmation page then says $124.00. The order is right, but customers will see two different totals.
Attached: pr-881-checkout.mp4. Jess, both are yours.
Jess (5:05 PM):
I pushed fixes. Test again on 390 and desktop.
Maito (5:14 PM):
Retested on the latest commit. The Pay button is clear and the total updates to $124.00 when the coupon is removed. All 4 checks pass. New video attached.
How to set it up
- Ask your agent
“When a pull request opens in the web app repo, run it in a browser, click through the flows it touches, record a video and post what you find in #qa.”
What it won't do
It tests the flows it can work out from the change and what you tell it. It does not replace a full test suite or exploratory testing by a person.
Questions
It uses a browser at desktop and phone widths. It does not test on physical phones.
With test accounts stored in the Vault (Settings, then Vault), so no passwords go in chat.
Yes. Mention Maito in the thread and it reruns on the latest commit with a new video.
