AI and MCP

How to check AI categorisation accuracy on your own statement

A spot-check method: sample 30 transactions, mark right or wrong in a three-column sheet, work out the error rate and read the pattern of mistakes.

Editorial illustration for How to check AI categorisation accuracy on your own statement

To check how accurate AI categories are on your own statement, pick 30 transactions at random, mark each category Y if right or N if wrong, and divide the wrong ones by 30. A three-column sheet and two formulas are enough. The number is a rough guide, but the pattern of mistakes tells you more than the percentage.

How do you draw a fair sample?

Pick lines at random so you are not just checking the easy ones. In a sheet, put your transactions in rows 2 to 500, add =RAND() in a helper column next to each, then sort the whole table by that column and take the top 30 rows. RAND changes every time the sheet recalculates, so copy the 30 rows and paste them as values into a new tab before you start marking.

Do the check on a month that is not the one you spent most time correcting, since lines you already fixed will flatter the score.

What does the three-column sheet look like?

Set up the new tab with a header row and these columns:

  • Column A: the transaction, as date, narration and amount.
  • Column B: the category the AI chose.
  • Column C: your verdict, Y if right, N if wrong, with the correct category in a note if you wish.
  • Column D, optional: the amount as a number, for the weighted check below.

Do your own judgement before looking at the AI category if you can, to avoid agreeing with it just because it is there.

Which formulas give you the score?

  1. Error rate: =COUNTIF($C$2:$C$31,"N")/COUNTA($C$2:$C$31)
  2. Accuracy: =COUNTIF($C$2:$C$31,"Y")/COUNTA($C$2:$C$31)
  3. Wrong by category: =COUNTIFS($B$2:$B$31,"Shopping",$C$2:$C$31,"N")
  4. Share of rupees that were wrong: =SUMIFS($D$2:$D$31,$C$2:$C$31,"N")/SUM($D$2:$D$31)

Suppose 4 of 30 are N. The error rate is 4 ÷ 30 = 0.133, about 13 percent, and the accuracy is 26 ÷ 30, about 87 percent. If the four wrong ones are small, the rupee-weighted figure may be only 3 percent; if one is a ₹40,000 transfer, it could be far higher.

How do you read the mistakes?

Pattern in the N rowsWhat it usually meansWhat to do
Mostly UPI payments to peopleThe category depends on your contextAdd a rule or a tag for repeat payees
Mostly one merchantA narration that confuses the systemAdd an alias or a rule
Random mixNormal noise on small itemsAccept it, review the large ones
Mostly large amountsThe risk sits where it costs mostReview every large item by hand

A worked pass shows how the pattern view helps. Say the four N rows are two UPI payments to people, one shop with a shortened narration, and one annual premium filed as Shopping. That is three kinds of cause, and each has a different fix: a rule for the two people, an alias for the shop, and a one-off correction for the premium. Counting by cause matters more than the headline 13 percent, because you can remove causes, not percentages.

Write the date and the result at the top of the tab. When you repeat the test after adding rules, you can compare like with like, and you will see whether the changes helped.

What do you do with the result?

Decide in advance what you would accept; the right level depends on your use. For a rough personal view, a few wrong small items may be fine. For business books, any error on a tax-coded category deserves a look. Then act on the pattern: write rules for the repeat offenders and run the sample again a month later to see whether it improved.

In SpendixAI, corrections you make become history and merchant aliases can be added, so the same mistake is less likely to repeat. If AI is switched on, you can also lower the amount it handles or turn it off for features where you want hands-on control.

SPENDIXAI JOURNAL

Useful ideas start with a clearer picture of your own spending.

Explore your reports