Post-editing effort across 27 million translated segments, 2024–2026
Our first study — Findings on Context Sensitive Efficiencies, published 2024 and revised Q1 2025 — measured 4.3 million segments from June 2022 to December 2023: 20 language pairs, translation edit rate (TER), Microsoft neural machine translation (NMT) as the baseline. Its headline: humans changed 18.23% of NMT output and 11.17% of context-sensitive output — 39% less editing. This paper repeats the measurement at production scale: 27.0 million anonymized segment measurements, recorded August 2024 to September 2026. Three results. First, the gap widened: reviewers now change 8.1% of context-sensitive suggestions against 15.5% of NMT output — 47% less editing than NMT, and 84% of suggestions ship with zero edits. Second, the improvement compounds: an account's edit distance falls from 41.7% in its first month to about 16% by month ten, because every correction feeds the account's knowledge base, never a model. Third, half of the measured segments — 51.6% — ship untouched. The study reads abstracted workflow measurements only: every figure is a group aggregate, no client is identifiable, and no source or target text was ever accessible to it.
Same metric, same question. The first study’s window closed in December 2023; this one opens in August 2024 — the two do not overlap. Editing fell on both sides. It fell faster where context is in the loop.
| Measure (TER) | 2024 study | 2026 production | Change |
|---|---|---|---|
| Editing on context-sensitive suggestions | 11.17% | 8.14% | −27% |
| Editing on NMT | 18.23% | 15.49% | −15% |
| Advantage over NMT | 39% | 47% | +8 pts |
Table 1. Editing effort, 2024 controlled study (4.3M segments, Microsoft neural MT baseline) vs 2026 production (7.7M origin-labeled segments, ≥5 accounts per bucket). Two years of better NMT models cut NMT editing by 15%. Context cut its own side by 27% and widened the advantage from 39% to 47%. The two editions average differently; under the matched construction the gap widens further, 22% to 47% — the note in section 14 has the detail.
The first study closed with open hypotheses. This edition answers three of them with production data: whether editing effort moves with working speed (its hypothesis 3 — section 8), what continuous learning earns an account over time (section 4), and how the spread across languages behaves at scale (section 6).
The corpus: 27.0 million anonymized segment measurements, August 2024 through September 2026. Each record is an abstracted workflow metric — an edit-distance value with coarse context such as origin and language pair — held apart from customer content, which the study never touches. Edit distance: how much of a suggested translation the human changed before confirming — 0% is accepted as written, 100% is fully rewritten. TER counts those changes the same way the 2024 study did, so the two measurements compare.
Since January 2026 every segment records where its starting text came from: a translation memory match (TM — a sentence your team already approved), a context-sensitive suggestion (built from your memories, glossaries, and preferences), neural machine translation (NMT — with none of that context), or human translation (HT).
Reviewers change 8.1% of context-sensitive output. They change 15.5% of NMT output — almost twice as much. And 84% of context-sensitive suggestions are confirmed without a single edit; NMT manages 67%.
Every confirmation and correction teaches the account's knowledge base — never a model. If that is true, accounts must edit less the longer they work. They do.
Accounts start at a 41.7% average edit distance in month zero. By month ten they sit near 16% — about 60% less correcting — and hold there through month seventeen. Each account counts once per month, so no single large customer moves the line. Months 0–2 run high by design: a new account's knowledge base is empty.
A trained model would keep this learning inside the vendor's weights. Here it lives in the customer's knowledge base, and this curve is what it earns.
The same curve, read from the other side: the share of segments shipped without a single edit climbs from about 40% in month zero to 64% by month seventeen.
The first study found translators changing as little as 1% and as much as 77% of a text. Production sharpens that picture into two poles: half of all segments ship untouched, and one in seven is fully rewritten. The middle — the patch-it-up editing that MTPE was named for — is the thinnest part of the distribution.
Fourteen busiest language pairs, ten or more accounts each. English into Spanish (US) gets 9% editing. English into Italian gets 26.3% — almost three times as much. The spread maps where human judgment spends its time, and where a glossary or style preference pays back first.
Six human pipeline stages span at least five accounts each. The first translation pass carries nearly all the change; the passes after it mostly confirm — review changes 0.5% of what it reads, in-country review 0.2%. The one stage that still edits heavily is the legacy full post-editing workflow, at 9.2%: the old MTPE pattern, visible in the data exactly where teams still run it.
Seconds per source word, over segments with a recorded working time: memory matches review fastest at 1.12. Context-sensitive suggestions review at human-translation speed — 1.35 against 1.38. NMT is slowest at 1.45. Modest gaps, shown from zero.
The corpus sits on forty-one months of production history. We chart it as an index rather than absolute volume — data abstraction applies to our own numbers too. Words processed per month grew more than fivefold from mid-2023 to 2026. Words routed through NMT are metered separately and stay a small share throughout.
The blended edit distance across the whole corpus holds at 15–20% every quarter — flat for two years. New accounts enter every month at month-zero editing rates and hold the blend up. The improvement is real; it shows per cohort (section 4), not in the blend. We chart it anyway.
The first study closed with eight hypotheses. Here is where each stands now. Section 13 adds this edition's own, numbered on from these.
| # | Hypothesis, as posed | Q1 2025 status | This edition |
|---|---|---|---|
| 1 | Language variance comes from tokenization of character-based languages | Confirmed — the revision standardized it | Settled |
| 2 | Translator methodology swings TER by ±50% | Confirmed — edits ranged from 1% to 77% of a text | Extended — production is bimodal (section 5) |
| 3 | TER correlates with speed | Not yet tested | Measured — lighter-edited origins review faster; the per-word gap is modest (section 8) |
| 4 | Domain variance of ±20% | Not yet tested | Open — the abstracted metrics layer carries no domain field |
| 5 | Learned terms improve TER by at least 10% | Not yet tested | Open |
| 6 | TER shifts ±10% across NMT engines | Not yet tested | Open |
| 7 | Translator well-being improves | More data required | Open — telemetry cannot answer it; a survey can |
| 8 | Translation quality improves | Not yet tested | Open — zero-edit share measures acceptance, not quality |
The numbers in this paper describe the pipeline as it ran. Three product layers now sit on top of it, and they turn these findings into a loop clients operate themselves.
Smells, the runtime semantic verifier, reads the whole document and flags potential meaning deviations — omissions, mistranslations, inconsistencies — as they happen. With half of all segments already shipping untouched (section 5), review can change shape: approve the document, and analyze what Smells raised.
Sous-Chef applies audience, tone, and formality at scale — defined once, applied to an account, a project, or a language. The pre-translation arrives already tuned to each client and context instead of as one deterministic output, so fewer of the remaining edits are about tone at all.
The Confidence Score totals what Smells found and tracks it over time. It gives a team the number to watch: where it holds, approve; where it moves, look.
Together they close the loop this paper measured: corrections teach the knowledge base, Sous-Chef carries preferences into the next suggestion, Smells inspects the output, and the Confidence Score keeps the running tally — constant improvement of the pre-translation, run by the client. Their effect on edit distance is deliberately not claimed here: it is not yet measured, and it belongs to the next edition alongside the open hypotheses.
The first study ended by thinking out loud. It said what was worth testing and what was worth noticing, and those questions shaped this edition. We end this one the same way.
Nothing in this section is a finding. Each item is something the data made us notice, the hypothesis it suggests, and how the next edition could test it. We number them on from the first study's eight, so one table can track every open question.
One more thing is worth noticing, and it is not a hypothesis. The blended average holds flat at 15 to 20% (section 9) while accounts edit less the longer they work (section 4). A single blended number hides the learning. The next edition should lead with cohort numbers.
That makes eleven open questions, five from the first study and six from this one. Some need data we do not collect yet, such as a content-type signal, a quality sample, and a survey. We name them now so the next edition can start collecting it.
Note on Table 1 and Figure 1. The 2024 figures come from a controlled study against one NMT baseline; the 2026 figures are observational production data — accounts choose their own mix of work. The averaging also differs: the study’s headline (18.23% and 11.17%) weighs its 20 language pairs equally, while its segment-weighted result was TER .162 against .128 — about a 22% advantage. The 2026 figures are segment-weighted, so the matched comparison is weighted against weighted: the advantage moved from 22% to 47%. We compare the editions because they use the same metric on the same platform, and we label every figure so you can weigh it.