Rule changes without surprises

Rules rarely act alone. A new decline rule takes payments away from the step-up rule that used to handle them, a lower threshold sends more cases to manual review, and a retrained model shifts the scores that every later rule depends on. The effect of one change depends on all the others, so a change that looked safe on paper can behave quite differently in production.

The cost also arrives late. A good customer who is declined may simply leave without complaining. Fraud that slips through appears weeks or months later as chargebacks or scam claims. By the time either is visible, several more changes have usually gone live, and the cause is hard to trace.

The sums involved are large on both sides. UK Finance reports that in 2025 its members prevented £1.68 billion of unauthorised fraud, 70p in every £1 attempted, while criminals still stole £1.28 billion through authorised and unauthorised fraud. At that scale, a small loss of precision moves real money either way.

Replay depends on a record of each decision with its inputs exactly as they stood at that moment. That includes answers from outside services, such as a device check or a watch-list lookup, because those answers change over time and cannot be fetched again later. A stored score is not enough. It shows what the system concluded but not why, and it cannot be run under a new rule.

The same record serves model risk and audit teams. The US banking agencies' revised guidance on model risk management, SR 26-2 of April 2026, which replaced SR 11-7, treats outcomes analysis, comparing model outputs with real-world outcomes through monitoring or back-testing, as part of sound validation. It also excludes deterministic rule-based processes from its definition of a model, so rule changes can fall outside formal validation. Fraud teams are wise to apply the same discipline to their rules themselves.

Run the candidate change over a period of past payments, using the archived inputs, and compare its decisions with the ones actually made. The useful output is the set of decisions that flip: approvals that would now be declined or challenged, and declines that would now be approved.

Break the flips down by customer segment, payment value, channel and merchant. Then sort each flipped payment by what is known about it: confirmed fraud, confirmed good, or not yet known. Confirmed fraud is what the change gains and confirmed good is what it costs. The unknown group needs care, because it shrinks as outcomes arrive.

A change meant to alter nothing, such as a rewrite of existing logic, should produce no flips at all. Any difference it shows is a defect found before customers meet it. This step matters most on instant payment systems such as Aani in the UAE and Pix in Brazil, where money leaves in seconds and a mistaken approval is hard to undo.

A change can look excellent in aggregate while the good payments it stops belong to your most valuable customers, or to a merchant you cannot afford to disrupt. Sample the flipped payments and read them as an analyst would, starting with high-value customers, long-standing customers and key merchants.

For each stop, ask whether the team could explain it to the customer in plain words. Cases that surprise the reviewers usually point to a condition in the rule that is broader than intended, and they are cheaper to find here than in the complaints queue.

A replay looks at the past, and behaviour changes. The next step is to run the candidate in shadow beside live traffic: it sees every payment and records what it would have done, while the current rules still decide. Compare its decisions with the live ones each day and check that the pattern matches what the replay predicted. A gap between the two is a finding in its own right.

Then release in steps, starting with a small share of traffic, one segment or one channel, with a rollback that has been tested. Record who approved the change, the evidence they saw and the reason for it. That record is what an auditor will ask for, and what your own team will need when a result looks odd months later.

Some changes are data rather than logic. Adding an account to a list of known mule accounts, or removing one after a complaint, should take effect within moments and be reversible at once, with a log of who made it. A new rule, a new threshold or a retrained model goes through replay, shadow and approval. Agree in advance which changes take which path, so an urgent list edit is never delayed and a logic change never skips review.

Track these before and after each change, by segment, and read the fraud figures again once outcomes have matured.

Approval rate

Share of payments approved, by segment and channel

Fraud caught and missed

By value and by count

Step-up rate

How often customers are challenged, and how many complete it

Review volume

Cases sent to analysts, and the time they take

Customer contacts

Calls and complaints about declined or held payments

Mantika Argus keeps every decision exactly as it was made, so a new rule can run on your past payments and then in shadow before it goes live.