Check data for problems
Spot missing data, duplicates and sudden jumps before anyone trusts a bad number.
Data · On a schedule · Updated
Slack
BigQuery
SupabaseHow do you run data quality checks automatically?
To run data quality checks automatically, set up a Loop in Maito. On your schedule it queries BigQuery or Supabase, compares each table with its normal volume, and looks for gaps, duplicate ids and sudden jumps. It posts only the problems in Slack, with row counts and the likely cause.
How it's done today
Bad data is found in the board deck.
What Maito does
- Check key tables each day
- Flag missing rows, duplicates and odd jumps
Example
Loop run: Daily 7:00 AM · Data load check
2 problems in yesterday's loads (Oct 18) • orders: no rows from 02:00 to 04:00 UTC. That window normally has about 410 orders an hour, so roughly 820 are missing. • payments: 1,904 rows in the EU region, versus about 950 on a normal day. 951 payment_ids appear twice. The catch: the doubled payments make yesterday's revenue read $38,200 instead of about $19,100. The Monday digest will show the wrong number unless this is fixed before then. 14 other tables looked normal. I only read the data; nothing was changed.
@Maito List the duplicated payment ids.
Attached: duplicate-payments-oct-18.csv, 951 ids with both load timestamps. All were loaded at 03:10 and again at 03:40, so it looks like a retry of the same job.
Maito (7:00 AM):
2 problems in yesterday's loads (Oct 18)
• orders: no rows from 02:00 to 04:00 UTC. That window normally has about 410 orders an hour, so roughly 820 are missing.
• payments: 1,904 rows in the EU region, versus about 950 on a normal day. 951 payment_ids appear twice.
The catch: the doubled payments make yesterday's revenue read $38,200 instead of about $19,100. The Monday digest will show the wrong number unless this is fixed before then.
14 other tables looked normal. I only read the data; nothing was changed.
Nina (7:12 AM):
List the duplicated payment ids.
Maito (7:13 AM):
Attached: duplicate-payments-oct-18.csv, 951 ids with both load timestamps. All were loaded at 03:10 and again at 03:40, so it looks like a retry of the same job.
How to set it up
- Ask your agent
“Every morning at 7, check yesterday's loads in BigQuery for missing rows, duplicates and odd jumps. Post problems in #data-alerts. If everything is clean, say nothing.”
- Connect database
- BigQuery
- Supabase
Not connected yet? Your agent sends you a link to sign in.
What it won't do
Maito finds patterns that look wrong against recent history. It cannot tell whether an unusual number is a real event or a bug without someone confirming.
Questions
BigQuery and Supabase. Maito runs the checks as queries against the tables you point it at.
It compares each load with recent days and weeks, such as typical hourly volume. You can tell it the thresholds you care about, and it will remember them.
No. It reports what it found and lists the affected ids when asked. Cleaning up stays with your data team.
