Home / Platform / Cleansing

A typo is not
a small problem.
It is a split entity.

A dataset contains a country column. Most records say CHINA. Six say CHINO. Nobody typed CHINO on purpose — it arrived through a keying error, a bad import mapping, or a supplier's export that was never validated. Six records out of fifty-three. It looks trivial. It is not.

Every aggregate is wrong Filters silently exclude The join fails outright

Interactive · run the real flow

Look, consent, audit.
Never a single destructive click.

Cleansing is deliberately a two-step consent flow with a permanent record. Nothing is modified until the impact has been shown and accepted.

Mapper · value panel · concept “COUNTRY”

groups 4 CHINA 47 state dirty

              Items under COUNTRY — 4 distinct values

Select the offending value to request a cleanup.

A cleanup that would touch six records and one that would touch six thousand look identical until the impact report is produced. The preview is a pure read — it exists so the person making the decision can see the blast radius first.

A single mistyped value does not corrupt one record. It splits an entity, and every aggregate over it is quietly wrong.

📉

Every aggregate is wrong, and none of them look wrong

Sales into China report 47 records instead of 53. The figure is plausible, internally consistent, and understated by 11%. Nothing raises an error, because nothing is malformed — CHINO is a perfectly valid string.

🔎

Filters silently exclude

A user filtering on CHINA gets 47 records and has no indication that six more exist. The six are not flagged; they are simply not there.

💔

And in a merged model, the join fails

This is the consequence most often discovered last. A record carrying CHINO cannot link to one carrying CHINA. The six records do not merely report separately — they fall out of the join entirely. Dirty values do not just distort a merge, they defeat it.

Why AtomicDB can correct this in one operation

Not a bulk edit.
One item merged into another.

In a relational database the value CHINO exists once per row that mentions it. Six rows hold six independent copies of the string, and correcting them means rewriting each one.

In AtomicDB a value is stored once per concept. The six records do not each hold the text CHINO; they hold associations to the single item that is CHINO. Correcting the data is therefore one item being merged into another — and every association follows.

This is why it is safe to offer to a non-technical user. There is no WHERE clause to get wrong, no risk of matching more rows than intended, and no possibility of correcting the value in one table but not another — because there is only one place the value lives.

Step	Action
Find the bad value	It appears in the concept's value list beside the correct one
Assess the damage	Request the cleanup; read the impact report
Choose the sources	Tick the datasets to correct — a value may be wrong in several
Correct it	Enter the correct spelling and confirm
Verify	The value list now shows one entry; the record count has moved
Explain it later	The history entry records the change and its operator

No SQL is written. No migration is scheduled. No downstream job is triggered. The person who knows that CHINO should be CHINA is the person who fixes it, at the moment they notice.

Cleansing you are allowed to undo

Correcting a value is the most
dangerous thing anyone does
to a dataset

Here it is a three-step, fully recorded transaction.

1

See the blast radius first

Before anything changes you are shown every system the value appears in, which column it sits in, how many records are affected, and example records to spot-check. You then tick which systems to correct.

nothing is implicit · a preview never writes

2

Correct it everywhere at once

Every affected record is rewritten, the links pointing at the old value are repointed, and the two spellings become one — across every system you ticked, in one action.

if any record fails, the values are deliberately NOT merged

3

The original is still there

A correction writes a NEW record and retires the old one rather than overwriting it. The tamper-evident ledger keeps the original, with who changed it and when.

auditable · attributable · reversible

The data is never left half-corrected. The item merge runs only if every row rewrite succeeded. On partial failure the merge is skipped, the response reports how many rows failed and identifies them, and the hierarchy is left consistent with the rows that did not change. Either the value is merged and every record agrees, or nothing is merged and the response says exactly what went wrong.

The correction is finished when it is applied

There is no
downstream refresh

In a conventional stack, correcting the source is the beginning of the work. The warehouse copy must be reloaded, dependent transformations re-run, extracts regenerated, dashboard caches invalidated, and any downstream mart rebuilt. Until each of those completes, different parts of the organisation report different numbers — and the correction itself becomes a small project.

In AtomicDB retrieval reads live state. There is no materialised copy to refresh, so the corrected value is what every reader sees at the next read: the source model, every merged model drawing on it, Ask's answers, and ATLAS's graph.

Conventional

Reload the warehouse copy
Re-run dependent transformations
Regenerate extracts
Invalidate dashboard caches
Rebuild every downstream mart
Until then, different teams report different numbers

AtomicDB

Nothing. It is already done.
The source model reads correct
Every merged model reads correct
Ask answers correct
ATLAS's graph is correct
Six records that fell out of every join now join

Cleansing does not only improve the accuracy of reports over one dataset — it repairs relationships across every model built on it. Six customer records that could not reach their orders now can.

Beyond typographical errors

In every case the problem is
the same shape

One real-world entity represented by several distinct items. Each is the same operation: two items that should be one, made one, with the records following automatically.

Kind	Example	Correction
Typographical	CHINO → CHINA	merge into the correct item
Synonym / abbreviation	USA · U.S.A. · United States	merge all into one
Legacy naming	ACME Corp → Acme Industries	merge forward
Inconsistent capitalisation	Chicago vs CHICAGO	resolved by the store's case handling
Stray whitespace	"CHINA " with a trailing space	merge into the trimmed item

And it does not have to be a hunt. Today, correcting CHINO requires somebody to notice CHINO. Posed as a single optimisation problem — see the Quantum page — the user is instead shown every value in the dataset that appears to be the same thing said differently, as one reviewable list, scored together so the proposal set is internally consistent by construction. Nothing is written on the solver's say-so: every proposal still passes through this preview-and-consent step.

Limits and honest notes

What this flow
does not do

A value drawn from more than four datasets reports the first four. In practice a merged concept draws from one or two.

The interface loops over the datasets the user ticked in the consent dialog, applying the change to each. A value that is wrong in one dataset is not necessarily wrong in another — and the user decides which.

The ledger retains the original rows and the concept retains the merge history. This is a feature for regulated work and a consideration where a value must be erased rather than corrected: cleansing is a correction mechanism, not a redaction mechanism.

Where two datasets disagree about what a column means rather than how a value is spelled, the answer is the Mapper's grid, not the cleanup flow.

The cleansing behaviour described here is the engine's documented two-step consent flow; CHINO → CHINA is the canonical example carried in the engine's own interface specification.

Back to the Mapper Talk to us

A typo is nota small problem.It is a split entity.

Six records say CHINO.One merge corrects them all.

Look, consent, audit.Never a single destructive click.