12

Today I woke up and my quality score was way lower than yesterday. This is because the new quality score has been implemented. Has anyone noticed the same? Are there still so many translators with scores of 8.0 or above? I would like to know which are those exclusive jobs that can be accessed JUST by them now. Can anyone share their thoughts?

13 comments

  • 1
    Avatar
    tashioh

    Hi Gamesofhands, today I also found my score drop by 1.4 without a review by Go-check/Language Specialist and I've been trying to find the reason before I bother support team and ask what's going on. It seems like it's due to the "consistency score" but I'm not a 100% sure. My profile page shows my old (and better) score and I still have access to Pro jobs so I'm not panicking but would love to know why. I used to be able to send a feedback when I couldn't agree with a review but now I can't even do that now.

  • 12
    Avatar
    anavictoria.vegaoz

    Guys, what can I say? I'm also among all those translators that were affected by this change. My QS dropped by 1.5 and I just lack of 0.1 to reach a score of 8.0.

    I understand the fact that Gengo does so to deliver the highest quality to clients - but I don't agree this is the right way to do so. I got a bad score 2 and a half years ago, and my skills have improved greatly in that time frame. The fact that this old, bad score is now having a negative impact on my current QS is just unfair because I'm not definitely delivering that quality nowadays. Even though I'm committed to offer an excellent work in each and every translation job I pick up, this isn't the best motivation to do so...

  • -9
    Avatar
    Lara Fernandez

    @gamesofhands - Generally, these are higher paying (larger) jobs and also jobs that are considered sensitive in quality terms.

    @tashiohnsman + @anavictoria.vegaoz - It's important to note that the new score is not an average of your scores, but a different calculation of the fluctuations in your quality. Accordingly, the minimum requirement is not the same as before. Whereas a QS of 7 was the minimum requirement before, we're looking at a minimum of 6 now, and anywhere between a consistency of 7 and 10 is actually a great score. I understand that seeing it in a numerical expression, it may seem low, but from our side, we look at our translators with a consistency of 7 and more as pretty amazing, actually. It means that your consistency is pretty solid! (Please see this Support article if you haven't yet)

    Because we're all used to seeing an "average" of actual scores, and not an "evaluation of our fluctuations", I understand the change may feel pretty awkward at first. But I hope that the explanation above helps you understand that the new score is in no way equivalent or comparable to an average, or to the previous score. 

    Also, the new job distribution system applied to translators with a minimum of 8 in the old score, and a minimum of 7 in the new one -- if you have a 7.9 right now, it still applies to you!

  • 24
    Avatar
    Sinfony

    I think the fact that all scores are now given equal rating is a misstep by Gengo. A translation delivered X years ago has little bearing on the quality that one would expect years later. Translating is not a mechanical process of pushing a button on a machine - a translator is constantly learning, developing and refining their skill set. 

    I find it puzzling that when for more than two years now, in which I have only fallen below the expected minimum of 7.0 once and have achieved a pure average score in that time of exactly 9.0, that I end up with a score of less than 7.0. Presumably due to several scores of less than 7 which I unfortunately racked up more than two years ago, at the very start of my time using Gengo. Does length of (good) service count for so little?

    I would also like a more transparent explanation of the consistency score. In the examples cited in the article "How Gengo measures quality" how do the example scores of "Translator A" and "Translator B" arise exactly? There seems to be no understandable mathematical formula for arriving at scores of 7.44 and 3.75. If there is a mathematical system being used to calculate the scorecard, it should be transparent in its workings.

     

  • 6
    Avatar
    AlexF

    I have to agree with Sinfony.

    Taking into account all our scores (especially those when we were taking our first steps in translation for some of us) is certainly a big disadvantage for those who have actually made a lot of progress in their skills.

    Suppose Person A is somebody having worked for Gengo for 3 years or so and who has seriously refined their skills to deliver consistently top quality translations today but who was struggling in 2015 or 2016.

    Then consider Person B: Person B is a recently qualified translator on Gengo and she/he delivers the same quality translations as Person A.

    Yet, in the new system, Person A's Gengo score is below Person B's.

    So what exactly does the new scoring system reflect? 

  • 11
    Avatar
    KevanSF

    Hi All,

     

    I'm trying to keep an open mind about this switch in systems to rate translators. I believe Gengo is honest and upfront, and they are doing what they think is best to meet their corporate goal of providing consistent good quality to their customers (a goal I fully buy into) and I'm confident Lara is sincere and listens to our concerns and truly wants to help us all succeed to the best of our potential and see Gengo continue to greater success.

    But. (You knew there was a "but" coming, right?) At this point I have to agree with the previous commenters on this thread. I fail to comprehend how a years-old score or two can carry so much weight on a translator's current rating. As previously stated, we learn and grow as translators, researchers, and writers. Many people initiate their translating careers with Gengo (as did I), and learn how to better research and write with practice (hardly surprising), so it's only natural for there to be inconsistency in their quality looking back in time.

    I thought the prior system of the most recent 10 (or 20, prior to that even) Go Check scores (weighted) was elegant in that it gave a clear incentive to improve, and that the translator could see that improvement gradually impact their scores with each new Go Check received.

    One thing is painfully clear, however: Gengo has failed to adequately communicate or explain (not through lack of trying, however) exactly how this new system works and why it's better, because none of us who have so far posted seems to be on board yet. I have honestly read every word of everything Gengo has sent me, and read all the info again from the link from my dashboard, and then read the official explanations here in the various forum threads, and I still don't really understand this new system or why it is better, or how it helps Gengo, or how it helps us translators.

    The second thing that is painfully clear to me is that the new system is powerfully demotivating to some, many, or most of the translators. Perhaps that last statement is too much a conjecture on my part, so I'll say it appears to me when I read the comments that it is demotivating to the translators who have so far commented, and additionally, I can speak for myself and say that it's annoying and a bit of a downer for me personally.

    But it is what it is, and now that I've voiced my opinion, I'll just try to ignore the whole thing and go about my business as best I can, silently hoping that Gengo either better explains the system, modifies it, or scraps it.

    Happy translating,

    Kevan

     

    OT PS, why does the embedded spell-check on this message system mark "Gengo" as a misspelled word? Can't the Gengo IT team simply add the name of their own company to the blasted dictionary the same way I add words to my personal Word software dictionary database? Inquiring minds want to know!

     

  • 4
    Avatar
    Cheryl

    This also means, if I understand correctly, that 'bad' scores never go away, which means that over the long haul, our score can never improve significantly. I remember getting a relatively low score and being so relieved when it was no longer included in my 'past', and got back up to a 9.9 that held for months until recently. maybe I'm just confused. I know that in the past few years of my dealing with them, they have always been VERY responsive to us and nothing less than a pleasure to deal with, so I'm confident that we'll all be satisfied (more or less, lol) soon!

  • 8
    Avatar
    gamesofhands

    Ok, I see almost everyone agrees in two facts: a) This system is disadvantageous for translator who have been working for Gengo and been active for many years and their skills are much better now; b) It is very difficult to get rid of older reviews that do not represent the current abilities of those translators. So who would join me to request a revision of this new scoring system and replace it by a combination of this and the previous one: taking consistency into account like now, but just in the 10/20 most recent jobs? I think this would make things more fair

  • 1
    Avatar
    Cheryl

    I think that games of hands' solution is brilliant- it takes the best of both systems: weighting consistency and only going back 10-20 jobs...

  • 2
    Avatar
    tashioh

    I completely agree. I would say in the last 10, rather than 20 if Gengo does GoChecks once a month. My skills have improved quite a lot compared to 11 months ago.

  • 6
    Avatar
    tashioh

    I've been back and forth between threads in English and Japanese. There's so much negativity towards the new system it's really sad because Gengo's been so helpful the whole time. After reading everyone's posts, I'd like to request Gengo the following:

    1. Improve the mysterious formula of the consistency rating
    2. Start the calculation from the day they implemented the new system
    3. Consistency rating takes account in the 10 most recent reviews
    4. At the same time, improve the "consistency of reviews by Gengo"

    Anything to add? Thoughts, everyone?

  • 1
    Avatar
    livor21

    Hello, tashioh

     

    I'm just joining this forum to add another unfair aspect of this new system. My qualifications in English-Spanish were removed a year ago or so and now I have just passed my Standard test again, but in fact I am not given a second opportunity because my score now takes into account all my errors in the past and is 3.5 from the beginning. 

     

    How can I start from scratch if I have such a low score, even if it is from 2016 and 2017 reviews?

  • 1
    Avatar
    memiposasa

    Hi fellow translators,

    This post is so interesting and I think we are making a great effort to let Gengo know what we think about their sooooo weird and unfair new quality system.  However, there is a VERY interesting post started by an user called Antoine which deals with it and also the MACHINE TRANSLATION NEW SYSTEM, giving a very interesting standpoint and showing why this MT system is unfair too. Could you please check it out and leave a comment?

     

    The post is A bit of feedback on translation scores and MT

Please sign in to leave a comment.