If you want to know how we measure the quality of the translations you deliver, you are at the right place. Keep reading!
Performance Scorecard
Your dashboard features a personalized scorecard showing your performance as a Gengo translator. The scorecard focuses on quality, and also provides a summary of your work history at Gengo.
Quality
Language Specialists review jobs using a system called GoCheck, which automatically calculates a job score based on error count, error severity and word count. Scores range from 0 (lowest) to 10 (highest), and you are expected to achieve minimum scores of 7 at Standard, Pro and Edit level jobs.
How is your overall Translator Score calculated?
Your translator score is calculated based on a weighted average of your last 10 GoCheck scores. Then, in order to add a consistency factor, we deduct from that weighted average the standard deviation of all your GoCheck scores for the last 3 years.
As a result, your translator score is not merely a reflection of your delivered quality, but a prediction of quality for future jobs based on the consistency of previous scores. In other words, your score represents that most of the jobs delivered by you will be above this score.
To illustrate this, let’s use an example with two translators who both have 10 GoCheck scores for jobs with an equal number of units. The GoCheck scores are ranked from newest to oldest.
Translator A’s scores: 7, 10, 8, 9, 8, 9, 10, 9, 9, 10.
Translator B’s scores: 10, 9, 10, 10, 9, 9, 6, 8, 5, 10.
The basic score for each of the two translators is the same: 8.8. This is calculated as a weighted average of their last 10 GoCheck scores, where bigger and more recent jobs contribute more to the final average.
However, after deducting the standard deviation, their final Translator Score will be different. Standard deviation represents how much fluctuation there is from the average. Translator A’s scores are closer together, ranging from 7 to 10, so their standard deviation is lower at 1.0, and their final Translator Score will be higher (8.8 - 1.0 = 7.8). Translator B's scores, on the other hand, range from 5 to 10, which results in a higher standard deviation of 1.7 and a lower final Translator Score (8.8 - 1.7 = 7.1).
So, it is really important to maintain consistency in your GoCheck scores: the lower their variation is, the higher your final Translator Score will be.
To finish our example, let’s see how the final Translator Score will change for translators A and B if they receive a new score of 10 or a new score of 7:
|
Translator |
Initial Translator Score |
Translator Score if the new score is 10 |
Translator Score if the new score is 7 |
|
A |
7.8 |
7.9 |
7.4 |
|
B |
7.1 |
7.2 |
6.8 |
As we can see, the new GoCheck score of 7 affected Translator A more, because it is less typical for them, so it impacted more heavily on their consistency factor (or in other words, standard deviation). Translator B has already had some low GoCheck scores, so this new 7 didn’t make as much difference as it did for Translator A.
This shows why it's worth keeping in mind that the Translator Score is designed to measure both quality and consistency, meaning that every GoCheck score makes a difference.
Our formula automatically discards the lowest score in the last 3 years of your GoCheck history when calculating your overall translator score. This will ensure that a single bad day will not negatively affect an otherwise consistently good performance, while also giving newer translators a fair chance to prove their skills when they are just starting out with Gengo.
In order to get a better score, your goal as a translator is to maintain a consistent quality in all your translation jobs, with all scores above the minimum of 7. This way, you will see your score gradually increase, although this may take a little bit of time.
If you are just starting to get your first reviews, you may not see a translator score in your scorecard for a while. To allow us to calculate your score from a significant sample of your translations, it will only display after you have received at least 5 reviews.
What's the meaning of your Translator Score?
The quality metric on your scorecard tells you how well you are doing and whether you should be worried or celebrating your good work. It's color coded, and the small number next to the arrow tells you how much your score increased or decreased since your previous GoCheck. Here's what happens at each level:
- Below 5: your qualifications will get automatically revoked
- 5 to 6: your qualifications are at risk because your quality doesn’t meet our minimum quality standards and your translations will be reviewed more often
- 6 to 7: your quality has been inconsistent and your qualifications may be at risk; it’s important that you focus on consistently meeting our minimum threshold in your work
- 7-8: you're doing well, but there's room for improvement
- 8 and above: you're doing great and you'll get access to more projects! :)
You can see more details about each level here with some useful advice on how to improve your overall translator score.
For translators who are qualified in multiple language pairs, your dashboard will display separately the score of every language pair you are qualified in, as in this example:
Job Reviews
Jobs to be reviewed are selected at random periodically by the system. However, there are other reasons that may trigger further reviews, including but not limited to recent low scores, an investigation that requires us to take a closer look at your quality, a rejected job, and customers indicating dissatisfaction with the quality received.
*NOTE: Language Specialists cannot choose what to review.
How does it work?
- When Language Specialists perform a review, they will select errors as indicated in the screenshot below with specific comments.
- These comments are for you to learn from and implement starting with your next translation. Please note that there's no need for you to take action or revise the job unless specifically requested.
- The system will then calculate a score based on word count versus error severity.
- *NOTE: Language Specialists do not control this calculation.
By popular demand, Language Specialists also have the ability to add suggestions that will not deduct any points and are purely for your information and to help you improve your translations.
Low Scores
Please keep in mind that we will not remove your qualifications if you only have 1 bad score. Qualifications are removed if you consistently fail to meet Gengo's quality standards.
As with test decisions, please accept job assessments and scores as final. You can, however, send in feedback on your job review.
GoCheck History
From your scorecard, you will now be able to easily access and view your full GoCheck review history for each language pair. Just click on "View History" next to each language pair:
Here you can see a summary of the feedback from the LS, so you can review comments and scores at a glance, and you can easily switch between language pairs from the tabs at the top. Clicking each Job ID will bring you into the GoCheck page, and clicking the row itself will bring you to the job in the workbench.
0 Comments