Order tells the customer's story

Take two customers who each logged in four times and made one large transfer last month. In a monthly table they look the same. For the first, the transfer went to a savings account used for years. For the second, the salary was moved to another bank the day it arrived, shortly after a first login from a new device. The totals match while the stories differ.

Teams summarise because classical models need a fixed set of columns per customer. Every summary is a decision about what to forget: a 30-day count cannot tell a steady pattern from a sudden change in the last three days. Analysts compensate with more windows and ratios, and the feature list grows to hundreds of columns that are hard to maintain.

The transformer, introduced in 2017 for machine translation, is the architecture behind today's language models (Vaswani et al.). A language model is trained to predict the next word, and at scale that task makes it learn grammar and facts along the way, because both help with the prediction.

The same idea works on events. Treat each customer's history as a sentence whose words are events: a card payment, a login, a transfer, a call to support. A model trained to predict the next event, or to fill in hidden ones, has to learn what usually follows what and what a break in routine looks like, without being told which features to compute.

Several firms have published this approach. Stripe has described a payments foundation model trained on tens of billions of transactions and reported that it raised its detection rate for card-testing attacks on large businesses by 64% (TechCrunch, May 2025). Nubank, in Brazil, reported gains on large recommendation problems from transformer models trained on transaction histories it already had, with no new data sources (Braithwaite et al., 2025). Revolut's PRAGMA models are pretrained on banking event sequences and used for credit scoring, fraud detection and lifetime value (Ostroukhov et al., 2026).

The usual pattern is one model per question, for who may leave, who will take a loan or what credit limit to set, each with its own pipeline, monitoring and owner. Over a few years a team maintains dozens of pipelines that describe the same customers in slightly different ways.

A model trained on behaviour learns one representation of each customer from their events, and each question becomes a smaller task on top of it. The PRAGMA paper reports that a simple linear model on that representation already performs well, with further gains from light fine-tuning. A new question becomes a smaller project than a new pipeline, and every team works from the same view of the customer.

Three conditions decide whether this approach will pay off. Volume means enough events that patterns repeat, which is why payments and digital banking are the clearest cases. Continuity means the same customer is recognisable across systems and over time, so their history reads as one story instead of fragments held by the card platform, the app and the core system. Outcomes means a record of what happened after each decision, such as whether the customer stayed or repaid. If one condition is missing, start with that one.

Each event needs at least its time, to the second where possible, its type, its value and its channel. Keep the raw events as well as any aggregates, and enough history to cover the slow cycles of your business, such as seasons and loan terms.

A lender with a modest customer base and monthly snapshots, or a bank whose records cannot yet be joined across systems, will usually get more from classical models and careful features. For small tables, TabPFN, a model pretrained for small tabular datasets, outperformed earlier methods on datasets of up to 10,000 samples in its authors' tests (Hollmann et al., Nature, 2025).

In those cases the most valuable step is getting the event record in order: one customer identifier, timestamps kept, outcomes linked to decisions. That work pays off with any model.

A model trained on personal data should be treated as a personal-data asset. Write down the purposes it may serve and review any new use as you would for the data. Control who can query it, set retention rules for the model as well as the data, and test that it does not memorise or reveal individual records.

Rules differ by jurisdiction. In the UAE they differ between onshore and the financial free zones, and in Brazil the LGPD applies. Involve counsel early, keep training and serving where your obligations require, and record which data each model version was trained on.

Compare a sequence model with your current one on later periods it has not seen, at the lead time the team needs, and check that predicted rates match what happens: if it says 7 in 100 customers in a group will leave, about 7 should.

Then measure the business result: customers retained or products taken up in the group your team acted on, against a holdout group it did not contact, and count the separate pipelines the new approach replaces.

Pick one question with a clear owner and recorded outcomes, such as who may leave in the next quarter, and run the new approach beside the current one before widening.

Mantika Tessera, built on transformer models, reads each customer's events in order and predicts who may leave and who is ready for more; your team chooses what to do.