1
I believe that the scoring system in Gengo is severly flaud, either that or they‘re trying to make my score lower on purpose. I calculated my overall score according to the information given in the score article about how they calculate our score and my score turned out to be much higher than it is right now. I sent an email to support and they replied that basically it‘s just the way it is. What‘s really going on here? Am I wasting my time in Gengo?

8 comments

  • 1
    Avatar
    Esraa

    I don’t mind getting bad reviews and being notified as well. I just don’t understand why is it when I get good ones and my overall score should be higher, it doesn’t get any higher! It only goes down!

  • 2
    Avatar
    Esraa

    It’s extremely frustrating and at times makes me think, “What’s the point in trying to get a good review then?” I don’t think discouraging us from doing good jobs achieves anything! They should rethink their strategy of dealing with the scoring system.

  • 1
    Avatar
    Lara Fernandez

    Hi @Esraa,

    Thanks for bringing this up :) I understand your frustration, but based on your post, I think there's been a misunderstanding regarding the way we calculate our score, which explains why your calculations don't match the numbers that are displayed on your profile:

    • The score is calculated through a two step process:
    • 1. A weighted average of your 10 most recent GoCheck scores (weighted here means that not all jobs have the same weight on the calculation: larger and more recent jobs weight heavier than smaller or older jobs. Because of this, the calculation is different than a simple average that you can calculate on your own)
    • 2. From the weighted average, we deduct your standard deviation, which is calculated based on your entire GoCheck history (minus your lowest score.) 

    I hope that the explanation above clarifies why your calculations are not resulting in the number you see on your scorecard. If you'd like, you can always write to our Support team for a breakdown of your scores :)

    To address your last point: “What’s the point in trying to get a good review then?” , I think it may be good to take a step back and remember that the purpose of your work with Gengo is not to try to get a good review, but to deliver high quality work to our customers, who are on the receiving end of your submissions.

    @Heike S. -- thank you for your contribution as well! I would like to clarify that no job is scanned for mistakes before being picked up for review -- reviews are triggered at random by the system, and at that stage, nobody knows whether there are errors in them or not. However, as you mention, besides the regular review process, low customer feedback and low scores do trigger further reviews. For example, if a customer leaves a low score on one of your jobs, an extra review will be triggered. And if you get a low score in a GoCheck, a further extra review of a separate job will be triggered as well, as a further step for quality assurance. In this way, low customers cores, as well as low-scoring GoChecks do trigger further reviews. Errors in jobs which the customer has not rated low, or in jobs which have not been reviewed, however, go undetected.

    Hope this helps!

    Lara

  • 0
    Avatar
    Esraa

    Thank you, Lara, for calrifying. This definitely helps calrifying how the scoring system is calculated more. And I understand that the purpose is to deliver high quality jobs, which I‘m always trying to do. However, with the current scoring system, it‘s highly demotivating. I hope that the team would consider changing the way scores are calculated. I understand that the current system might seem fair, but there are also other ways that can be fair for both the translator and the customer and be motivating at the same time. I hope that you would take it into consideration.

  • 0
    Avatar
    Lara Fernandez

    @Esraa - I'm glad that I could help clarify the calculation a bit :) I do understand, of course, how some aspects of the scoring system can feel frustrating and demotivating, too. This is a formula we reached and refined after many iterations, which at the moment is working well for us, but we are always looking to kaizen (continuously improve) all our processes, so there's nothing set in stone forever. I will definitely pass your feedback along to the team and, while I can't promise immediate changes, we will take it into account in the event that a reformulation of the scores calculation becomes part of the pipeline.

  • 1
    Avatar
    Esraa

    Thank you so much for your understanding and your responses. I appreciate that you would pass this on. And I hope that you have happy new year, and stay safe.

  • 0
    Avatar
    Lara Fernandez

    @Heike S. -- Thanks so much for clarifying! Now for Facebook, they have a QA process in place through which yes, some jobs may be returned by revisions -- but the re-reviews are triggered by the system as per usual (unless you were providing very low quality that granted a more thorough investigation, which doesn't sound like your case.) Now, you may have felt that the reviews were more numerous than usual, but since you mention that nobody in your language pair was picking up these jobs, I'm going to assume (at the risk of being wrong) that you delivered a fair amount of these jobs. Because, as you know, regular reviews are triggered by activity (the more active you are, the more reviews are triggered) this still makes sense :)

    So, as you mention -- it sounds like you're more dissatisfied with the QA process and reviews of FB jobs than with our Quality / Score / Gocheck system itself. I've seen some of these jobs in various language pairs and I agree that they can be difficult to handle, as they are user generated content grabbed from the internet, which doesn't always guarantee the best quality source text. 

    In light of your experience, I can 100% understand your resolution and where you're coming from and, as you know, nobody can force you to work on those jobs :) 

  • 1
    Avatar
    Lara Fernandez

    @Heike S. -- Thanks for your feedback on the scoring system, as well! I think we have to make a difference between the scoring system (formula through which your quality score is produced) and GoCheck reviews :) Just to reassure you, corrections should never be a mere matter of style, and if you're receiving corrections based on style and not on an actual error, I would recommend that you bring it up to our attention by filling out a request for re-review. There's actually a category called "Suggestion" that the LSs can use to highlight text which is not "wrong" per se, if they like to offer an alternative more natural suggestion, and this category doesn't affect the score. 

    Last but not least, I sometimes read translators mention dissatisfaction with the reviews they receive, but at the same time they refuse to request a re-review. Among other indicators, re-review request numbers are part of the data that we monitor to evaluate our LSs performance, so I'd recommend submitting a re-review request when applicable (i.e. if you have resources to support your claims that your translation is correct, or that the errors marked are not errors.) While this may not always result in a change in score, it leaves a track record for us to look at when reviewing LS performance, and it is important :)

Please sign in to leave a comment.