The red-pen syndrome

This article was originally published in the No. 12 (2014) issue of Revista Tradumàtica

1. Introduction

Every translation-related quality standard issued so far has followed the same error-catching-and-assessment approach, always reaffirming the TEP model, but apparently overlooking the increasing complexity of tasks and, consequently, of operating costs and the risk of introducing new errors in every step of the process.

By the time of publication, ISO/DIS 17100 should have replaced EN 15038. However, the new standard will plausibly leave unsolved many of the typical issues related to quality management in translation.

This article is a review of current translation quality standards and a proposal for student training from a historical perspective based on checklists as a suitable means for quality assurance, to prevent errors rather than try to catch them.

The opinions and suggestions in it are not the results of any empirical research and only come from the author’s extensive working experience as a practitioner and a university teacher. The many publications listed in the reference section at the end of the article are hopefully intended as fuel for further study and investigation.

2. The reasons for quality management

Quality management started with a problem of measures in ancient Egypt, when building the pyramids. The sizing and spacing of stone blocks had to be consistent on each row to ensure easy handling, covering, and structure solidity.

Many centuries later, in times of war, manufacturing defects were the main reason for standards.

During World War I, in the UK, a high percentage of shells failed to explode because the two main ammunitions manufacturers had a different definition of an inch.

During World War II, accidental detonations in weapons factories led the government to require suppliers to write up the procedure for making their products, have the procedure approved by the Ministry, and ensure their workers followed it. At that time, bullets manufactured in one state of the United States had to work consistently in rifles made in another. The armed forces initially inspected every unit of product; then, to simplify and speed up this process without compromising safety, they began to use sampling techniques for inspection, following a military specification.

After the end of World War II, President Harry S. Truman tapped General Douglas MacArthur to oversee the re-building of Japan and exercise authority through the Japanese government machinery. The Japanese Union of Scientists and Engineers invited William Edwards Deming and Joseph Juran to work on statistical process control and on managing for quality respectively. By the 1970s, Japan’s ability to produce high-quality goods at competitive cost had broadsided the developed countries of North America and Western Europe. More than a decade later, William Edward Deming’s teachings had spread over as Total Quality Management (TQM).

Since then, the quality movement has matured beyond the foundations of Deming, Juran and the early Japanese practitioners of quality, moving from manufacturing into service, healthcare, education and government sectors.

2.1 Quality management today

Today, quality standards generally target processes, considering a product as the result of a process.

Since quality problems vary with the type of product or service, the making process is a critical determinant of quality, and must comply with requirements: work methods, tools, controls, or operator qualifications.

When different organizations agree on common requirements and comply with them, these requirements become standards, enabling the efficient functioning of a market, allowing partners to communicate and have common expectations on each other’s performances and products. By establishing a commonly agreed and shared reference framework and by offering solutions to known issues, standards ensure transparency, thus helping to reduce costs, expedite contract negotiation, and improve buyer/provider relationships.

For such a framework to be efficient, good practices must be in place, and metrics are necessary to track performance of the underlying business processes. Most process standards are then merely guidelines to good practice based on a set of requirements for compliant organizations to meet and obtain the buyers’ acceptance of the process outputs.

Quality management generally hinges on four basic rules:

  1. Write down what you do.
  2. Do what you have written.
  3. Substantiate what you have done.
  4. Reflect on how to improve it.

And pros and cons of process-oriented quality standards are generally well known:

Pros Cons
  • Repeatable quality
  • High degree of control
  • Reduced impact
  • Mechanism for continuous improvement
  • Greater confidence
  • Repeatability does not mean quality
  • Heavy initial investments
  • Major maintenance costs
  • Bureaucratization
  • Reduced flexibility

3. Translation quality standards

Today, quality is a relative concept broadly corresponding to product suitability. As relative to needs, quality varies with tasks, each one having its own requirements.

Even translation quality standards mostly rely on the idea that following certain procedures will increase the likelihood of good quality.

Like many other manufacturing companies at the dawn of quality management systems, translation providers look at compliance and certification only as a means to foster a positive perception in buyers, increase reputation, and gain a competitive advantage. Unfortunately, they are looking for a short-term payoff that could cost them much more in the aftermath.

3.1 Three issues on quality models and policies

A serious approach to quality models and policies requires addressing what translation buyers care about, the impact of technology on translation and translation quality, and the practicality of the traditional translation model.

3.1.1 Translation buyers goals

Buyers and users care about benefits while translation providers care about quality, which they see as a feature and, in fact, it is not a benefit.

Buyers want to know what they spend for and for what it is worth and do not want to put a project at stake on any tiny fraction of it. Therefore, to avoid guessing, buyers require factual data to assess their translation effort, budget it, and evaluate the product they receive.

3.1.2 The impact of technology

Online machine translation engines and freely available open-source machine translation engines have been disrupting the translation industry. They are also bringing a different perception in translation buyers. More and more people are now willing to acknowledge the importance of translation as a highly qualified service, especially as to reliability. They are also augmenting the already considerable burden of the quality-cost-delivery triangle on the ‘cheap’ and ‘fast’ sides. Also, the undeniably less effort required for translating, thanks to the availability of new tools and easily and freely accessible information, together with the ability to leverage from previously translated material have helped the slump of prices. This combination is at the origin of the increasingly growing amount of content, the consequent surge in the demand for translations and the asymmetric decrease in compensations.

3.1.3 The translation model

Agile is the new black in the translation business. Agile processes are lean, uncomplicated and straightforward, accurate but practical, and simple enough to be unambiguous. A typical agile methodology lies on close collaboration between self-organizing, cross-functional teams, adaptive planning, and flexibility to change, while the level of complexity and sophistication of processes in the translation business is increasing.

Anyway, no agility is possible with a serial, stiff, compartmentalized process like the traditional Translate-Edit-Proofread (TEP) model.

An agile quality-oriented model requires doing things right the first time every time. A quality assurance framework is then necessary for masterly doing every job, being accurate in every task, and solve any issue before the process is complete.

The TEP model is sluggish, inefficient and prone to introduce new errors at any step rather than removing them.

3.2 A long series of misfocused standards

In 1996, Italy issued the first translation quality standard for quality management in translation, UNI 10574. A year later, the Dutch followed with the ATA Taalmerk. In 1998, Germany published its own standard DIN 2345 and so did Austria two years later with ÖNORM D 1200. In 2001, it was the turn of the USA with the SAE J2450 translation quality metric, and in 2005 China came along with GB/T 19682 specifying target text quality requirements for translation services. In 2006, the EU issued EN 15038 to harmonize the European standards and the USA issued ASTM F2575-6. Lastly, in 2008, Canada issued its own standard CAN/CGSB-131.10 based on EN 15038.

Every translation-related quality standard issued so far has followed the same error-catching-and-assessment approach, always reaffirming the TEP model, but apparently overlooking the increasing complexity of tasks and, consequently, of operating costs and the risk of introducing new errors in every step of the process.

This approach came to overstatement in 2012 with ISO/TS 11669. Despite its analytical claim, ISO/TS 11669 provides translation requirements and selects the issues for quality assessment in the traditional error-catching approach. It provides strict guidance for best practices for all the phases of a translation project as the basis for qualitative assessment, but no clue for a quantitative measurement of the quality of a translation. Essentially, ISO/TS 11669 is a container for a long list of translation parameters for specifications, levying vague, blurry, and subjective criteria for quality assessment from the archetypal academic scenario.

The general architecture of ISO/TS 11669 is overly and unnecessarily complicated, lying on the assumption that buyers and providers share the same knowledge.

4. The quality paradox

Quality is a prerequisite for existence on market, for any business, it is expected. This is especially true in the translation business, where buyers cannot possibly assess the quality of all the products or services they receive.

A major goal for any quality standard should be allowing benchmarking, i.e. measuring performance according to agreed metrics and indicators for a simple rating scheme. For a unified global rating system, though, not only are independent evaluation organizations necessary; clear, plain and practical criteria should come first. Moreover, as technology advances, buyers’ expectations increase and business contexts get more complex, while demands get barer and stronger.

No translation-related quality standard so far contains any such clear, plain and practical criteria.

4.1 Quality assurance

Quality assurance (QA) aims at preventing defects in products and errors in services. Inspection is only a fraction of QA, although run systematically. In this respect, efficiency is crucial to make inspection practicable. This is the main reason for sampling techniques allowing producers to inspect and test product samples for quality to achieve the desired level of confidence in the quality of the entire production run.

All translation-related quality standards simply replicate the typical trial-and-error approach of teaching, with downstream rather than upstream adjustments, catching rather than preventing errors. This exception handling approach challenges QA’s principles.

In quality management, quality is a function of the contributing elements, some of which can be bearable and effectively controlled while others are not. Specification of requirements help define those elements and assure quality. In this respect, inspection relies on the analysis of objective data to track the cause for variances and remove them to prevent errors to occur again.

Metrics are the foundations of inspections to measure performances against requirements according to a common set of criteria.

In translation, all metrics reflect the typical error-catching approach and thus are usually subtractive for a final score, sometimes returning a pondered percentage.

In a requirement-oriented quality assurance model, performance metrics should be additive, i.e. summing the elements meeting a specification.

4.2 Error catching versus error prevention

At the Localization World Dublin conference in June 2014, translation quality scholar Sharon O’Brien reported that, to her knowledge, no translation educational institution waived the traditional error-catching approach in assessing translation quality. In other words, counting errors is still the one known or assumed way to instruct students in a centuries-old educational model forged to feed a system that has been flaking off for at least two decades.

This red-pen approach translates into a red-pen syndrome once students become translation providers and enter the translation business.

Not surprisingly, the whole translation business is still at the error-catching production model through serial, non-collaborative, additional steps that could easily, often and costly introduce new errors at every stage.

After TQM and ISO 9000, the manufacturing industry moved forward to introduce new models and techniques for process improvement, like the Capability Maturity Model Integration (CMMI) and Six Sigma. Six Sigma still embraces the concept of quality as relative to defectiveness, ranking organizations on a 7-grade scale.

Sigma level Defectiveness
1 69%
2 31%
3 6.7%
4 0.62%
5 0.023%
6 0.00034%
7 0.0000019%

A seemingly impressive level 4 corresponds to 620 errors in a 100,000 word projects, two errors every three pages. A level 4 also corresponds to two incorrect landings per day at major airports, to a 7-hour-per-month power failure, to 15,000 urgent packages lost per week, to five minutes every day of only non-drinkable water available, to 5,000 incorrect surgical operations per week, and to 200,000 wrong prescriptions per year.

4.3 Maslow’s hammer

On the other hand, quoting Abraham Maslow, “I suppose it is tempting, if the only tool you have is a hammer, to treat everything as if it were a nail.”

The trial-and-error teaching model also used for training translators is typically based on inspections. It is costly, but the very small, consistent samples make the otherwise impracticable 100 percent inspection sustainable.

The problem with this model is that it only focuses on the final output rather than on the process, and this leads the players to add steps to the process to catch errors to improve quality. Unfortunately, besides adding costs by multiplying efforts, this exposes to the risk of introducing new errors at every stage. An error-based chain is neither efficient nor lean or budget, and in fact, in translation, quality costs are mostly for quality control — reviews, rejections, and repairs.

5. Escaping the red-pen syndrome

The long-studied immeasurability of translation quality is mostly due to the intrinsic subjectivity of the task and the consequent assessment.

Right from the start, translation trainees are taught that translation errors are just one indicator of poor quality, that the number of errors alone cannot measure translation quality, and that a poor style, inappropriate tone or wrong register can make a translation bad, even if it contains no material mistakes.

In fact, even if a translation contains no blatant, material errors that could lead to a substantial cost of poor quality for any risks related to possible damages, it could always be of poor quality for someone, due to the highly subjective assessment criteria.

Similarly, most translators still think of quality in terms of tasks associated with inspection and reviewing, rather than with compliance with requirements, root-cause analysis and corrective actions.

The all-time common concept of quality in translation is then its major constraint, and the reason for aversion to change, and ISO 17100 will change nothing.

Claiming that unique features characterizing translation prevent it from assimilation to other, even essential, economic activities will widen the gap between translation buyers and providers, with the first struggling to find an escape path from information asymmetry, approach translation just like any other business, and buy translation like any other product or service. Like it or not, just like many other globalized services and products, translation is now a commodity.

Most buyers rely on standards as a base for common criteria, methods, processes and practices between the players in a transaction, and a transparent relationship between them.

Since quality is a prerequisite, most buyers who cannot overcome information asymmetry but can see translation as a commodity reasonably ask for budget prices. They would possibly welcome the option to pick a translation provider as they could pick a hotel room and be delighted to do without intermediaries using a platform like Booking.com or Expedia.

While the latter is becoming a reality, the former is still far away. Without a unified global system for rating, there is no chance for a common classification system ranking translation providers on a scale of different service levels.

On the other hand, the idea of different levels of translation quality or service itemization and unbundling conflicts with all current translation quality standards, including ISO 17100.

No more harmonization is necessary, then, but a radically new approach.

5.1 Checklists for error prevention

Checklists were born in the military field to reduce human errors when facing many complex and challenging tasks and save lives. Checklists were a simple, brief, and straightforward solution to a serious problem to prevent further training — and funding — for experienced and skilled staff. Checklists itemized tasks staff know how to do.

Today, even the once crafted translating work is getting increasingly too complex to carry out reliably from memory alone, and checklists are the foundations for quality of processes, for doing things right the first time every time.

A fundamental reason for documenting processes within an organization is to reduce errors by preventing them from occurring. Error prevention comes from training and reminding people on expected practices. When an organization commits to best practices, the first goal must be having guidelines be read and used. These documents must then be as simple, brief, and straightforward as possible.

Checklists are a way to concisely document practices and find errors in an organization’s work. Simple checklists can ensure that critical steps are not overlooked.

5.2 Key performance indicators to communicate quality

In the common language of business, quality is often associated with index. A multi-item measure of key dimensions of operational, product, and service quality can shape a quality index consisting of a group of key performance indicators (KPI) assessing the provider’s capacity to meet the buyer’s expectations at an acceptable cost.

A fresh translation quality standard could list a set of KPIs to make up a quality index to display the value brought that buyers can understand.

6. Final remarks

The traditional answer from translation academics and pundits to cope with complexity is hyper-specialization, obviously through training, possibly the one they suggest.

Apparently, that answer is wrong, mostly because those academics and pundits look at the world as it was when they began, while it is changing everyday faster.

As individuals, we will fail at our tasks no matter how smart or experienced we are; discipline in our processes is one way to overcome such failures.

Checklists could be a suitable means for quality assurance, to prevent errors rather than try to catch them.

Current translation quality standards stubbornly reaffirm obsolete, inefficient, unsuccessful, overcomplicated traditional theories and models, and insist in suggesting the traditional error-catching approach for quality, prone to subjectivity and fallacy.

Only new agile models envisaging translation providers to prevent errors can help respond to the increasing complexity of demand and work. For these models not to remain mockups, new educational programs are necessary to teach future and professional translation providers to abandon abstruse, convoluted and hard-to-apply technical specifications and start working with an open mind and a joint effort on automatic tools reducing human involvement, and thus subjectivity.

7. References

ASTM International (2014). F 2575 – 14 Standard Guide for Quality Assurance in Translation
Baker, M. (1998). “Quality of translation”, Encyclopedia of Translation Studies. Routledge
Carnegie Mellon University Software Engineering Institute (2010). CMMI for Services. Version 1.3.
Drucker, P. (1993). Innovation and Entrepreneurship. Collins
Drugan, J. (2013). Quality in Professional Translation. Bloomsbury Academic.
Edwards Deming, W. (1964). Statistical Adjustment of Data. Dover
European Committee for Standardization (2006). EN 15038:2006 Translation services – Service requirements
Gawande, A. (2009). The Checklist Manifesto. Metropolitan Books
House, J. (1997). Translation quality assessment: A model revisited. Gunter Narr
Hubbard, D. W. (2014). How to Measure Anything. Wiley
International Organization for Standardization (2005). ISO 9000:2005 Quality management systems – Fundamentals and vocabulary
International Organization for Standardization (2012). ISO/TS 11669:2012 Translation projects — General guidance
International Organization for Standardization (2014). ISO/FDIS 17100 Translation Services — Requirements for translation services (draft for publishing)
Juran, J. M. (2004). Architect of Quality: The Autobiography of Dr. Joseph M. Juran. McGraw-Hill
Mossop, B. (2007). Editing and Revising for Translators (Translation Practices Explained). St. Jerome Publishing
Muzii, L. (2005). “Building a Localization Kit”, Client Side News, Supplement to Volume 6 Issue 12
Muzii, L. (2006). “Quality Assessment and Economic Sustainability of Translation”, Rivista internazionale di tecnica della traduzione/International Journal of Translation 9. EUT – Edizioni Università di Trieste
Muzii, L. (2007). “A Roadmap to Quality Translations (Part 1)”, Client Side News, Volume 7 Issue 2
Muzii, L. (2007). “A Roadmap to Quality Translations (Part 2)”, Client Side News, Volume 7 Issue 3
Muzii, L. (2008). “Outlining a Translation Requirements Specification. Client Side News. Volume 6 Issue 12. Volume 8 Issue 5. May
Muzii, L. (2012). A Contrarian’s View on Translation Standards. sQuid/Lulu
Muzii, L. (2014). “Communicating value in business terms”, MultiLingual, July/August, pp. 46-49
Pyzdek, T. Keller, P. A. (2010). The Six Sigma Handbook. McGraw-Hill
Williams, M. (2004). Translation Quality Assessment: An Argumentation-Centred Approach. University of Ottawa Press

Author: Luigi Muzii

Luigi Muzii