Hello everyone!
Today I logged in to see that my quality score fell from 8.1 to 6.6 (the lowest I ever encountered)
Now I came to understand that, as of today, all goChecks we went through will be taken into account, not just the last 10, I just checked and it represents 4 years of work and 70 GoChecks for me. I know that the consitency is taken into account, but bear with me: I went through my archives and compiled every single score earned these last 4 years.
My average is 8.74. That's two points higher than my current score. However, as gengo explains it, the consistency is also taken into account. Fair enough:
47 of the total scores are situated between 9 and 10 (that's 67% of them), 26 of them being straight up 10s.
On 70 GoCheck scores, 59 are rated above 7, that's 84.2%.
Now, I do realise that there is room for improvement, I certainly do.
My question is: if all of that sends me down at 6.6/10, how can I hope to pass above 8 again? I mean in a foreseable future? My digging for these scores showed me that, other than several exceptions, I was evaluated roughly about once a month. Even if I pulled 10s upon 10s from now on (which I fully intend to do), I feel like it would be years before getting a decent score at this rate.
Even if I look at my past 10 scores, 2 of them are below 9, one of them being a 2,94 which, for the record, I disagree with, I could fill up a whole post about that but that's not the point here. The other being a 7,43.
I genuinely feel trapped here.
Am I the only one in this kind of situation I wonder?
41件のコメント
The same thing happened to me, and it was even more dramatic in my case: I went from 9.5 to 6.7 just like that.
I understand the importance of consistency but this, sincerely, is brutal. We have all had bad days and bad reviews and we've all learnt from the experience. I think this change should have come with a warning before being implemented, giving us translators time to adjust and prepare, and perhaps calculating the consistency score from now and forwards, not for the *entire* work history.
This consistency score thing definitely has to be rethought. Is it now better to be a consistently poor translator than to be a good translator and have occasionally made a mistake on a short translation that resulted in a couple of unrepresentatively low scores? That seems to be what Gengo is saying with this new system. It's as though we're being discouraged from improving. If we get 7.2 in one review are we supposed to try to make 2.8 points worth of errors in every translation after that in order to be consistent? Because if we get a 10.0 next time it seems as though they'll be docking us points for the 2.8 difference rather than rewarding us for doing better.
As I wrote in a parallel discussion on the forum:
Taking into account all our scores (especially those when we were taking our first steps in translation for some of us) is certainly a big disadvantage for those who have actually made a lot of progress in their skills.
Suppose Person A is somebody having worked for Gengo for 3 years or so and who has seriously refined their skills to deliver consistently top quality translations today but who was struggling in 2015 or 2016.
Then consider Person B: Person B is a recently qualified translator on Gengo and she/he delivers the same quality translations as Person A.
Yet, in the new system, Person A's Gengo score is below Person B's.
So what exactly does the new scoring system reflect?
In my opinion, the consistency score is absolutely discouraging people to take on short jobs and collections. Even the best translator will make a mistake at some point every 100 or so sentences. Sometimes it is even due to an incorrect source text etc.
If every sentence is split up as a single job in a collection and that one sentence gets picked for review, he/she will get one very very low score. In the past, these low scores were weighted with the unit amount of the job and not that big of a deal - I am not saying that I didn't care about getting bad reviews or that I was careless but it didn't have such a massive impact on the score and it went away with a couple of good new reviews. Overall, a bad score was all the more motivation to keep on working with Gengo to replace that bad score with ten new great ones.
Now, one such bad score has a massive impact on the consistency score. And it will stay there forever. How should that be motivating to continue working more and harder?
I am confident to say that I can score very consistently over 8/10 for all translations with several sentences. But I cannot guarantee that there will be not one single mistake in any of the translations that I submit. Nobody can. And if that mistake happens in a short translation, Gengo assumes that I am extremely inconsistent and will never forgive or forget. That just doesn't make any sense.
You at Gengo seem to be a bit distracted with the new system as well. Within a couple of days in a row, I got three e-mails. The first one stated that due to my excellent score, I now get access to more jobs due to the new distribution system. A day or so later, I get an e-mail stating that due to my low scores, my qualifications are at risk. That was confusing to me and the time I learnt about the new glorious system. The next days, I was basically just waiting to have my qualifcations revoked thanks to my new 6.5 score. But then I got another email, inviting me to take the pro test in my language pair because I continuously deliver such high quality in my work. So what is it now?
I am in the same boat as many of you. My score dropped from 8.8 to 6.5. In the 33 reviews I've accumulated in my time with Gengo, I've had three scores that were below 6.5. Strange.
My two bad scores were translations with only 21 units. It seems to me the new system doesn't really factor in the amount of units a job has. So making one mistake in every 10th short translation seems to be worse than one mistake in one long translation. That just doesn't seem to make sense to me as human error happens at random and doesn't care if there is only one sentence in a translation or many. We all make mistakes at some point.
So to me, the new system would be highly discouraging to accept low volume jobs since the risk of having my score put at risk is simply too high. Unfortunately, most jobs at non-pro level are very short jobs.
The problem is the qualifications are taken into account forever. That makes that people who have been translating for years in Gengo now see their average score decreased because their scores were much lower in 2014 or2015. This is unfair, because it is obvious that they have now better skills and only consistency in their last months should be taken into account. Is not that your case, Alicia L. alonso?
@Alicia L. Alonso
My experience seems to echo yours. My 9.2 average plummetted to a 6.6 when I checked my score this morning. I hadn't visited Gengo for about a week up until that point, so it really came out of left field...
I was so shaken, I ended up averaging my past 35 scores (everything for 2018), and whaddya know--a 9.2/10. Three of those 34 were marked below a seven: one at 5.2, another at 2.9, and another at 6.7.
I've made my feelings on the matter known to Support and they have echoed every other Support member who has chimed in here and in other threads on the topic. I understand the need for a consistency score in theory, but I feel the implementation has been handled with a distinct lack of finesse. Anecdotally, if I am being brought in for a year-end review and I have NAILED my projects out of the park with glowing praise from my clients, three less-than-remarkable results should not drag my overall score down to a 6/10 from what should be a 9/10. A reasonable supervisor will say, "Yeah, you've had a few bad days, but 9 times out of 10, you're doing great." That 6 on my dashboard? Not helping me feel valued, Gengo.
I've been with Gengo since 2015, and I'll be the first to admit that I wasn't bringing my A game back then, but anyone with access to my scores can see that I knock it out of the park on a regular, and most importantly consistent, basis. With over 800,000 characters under my belt and way more happy clients than Gengo to worry about, I'd like to think I'm at a point in my career where I know what I'm doing.
Why punish us for mistakes we didn't even know we were making? Once again, I see the value of a consistency score and support its implementation, provided that it is done correctly. Personally, I would suggest that the new scores only account for reviews made after the date the news post was made. That would at least level the playing field and give people an idea of the standards expected. As it stands, I see absolutely no way to bring my score above a 6 based on almost 4 years of backdated reviews, meaning a new recruit with fewer reviews will assuredly be placed in a higher bracket by default, where I'm left walking the tight-rope being weighted down by obsolete results.
I spent a lot of my afternoon in tears because, for a dark moment, Gengo had me convinced I was less than what I thought I was worth when I know damn well that I'm a fine translator.
I can see a lot of people walking if Gengo doesn't get their act together over this, myself included. For now, though, I'll just look at the "Wordsmith" and "Top Scorer" headings on my profile and the Gengo T-shirt hanging up in my closet and just shake my head.
I too have been with Gengo since over 6 years, and my score dropped from 9.4 to 8.3 after the implementation of this very questionable new scoring system. But what is really strange here: I had just one review (a 10/10) after that implementation, and my score dropped further to 8.2. Now please, what is the logic here? I can certainly see the need for "consistency", but this new system seems very arbitrary and frustrating, for all of the reasons mentioned on this thread.
With my due respect and no offence intended, I still have to say the new scoring system is a big joke.
No more, no less. In fact, it's not wrong from the idea beyond this, actually the idea of tracking consistency is brilliant and ingenious, but the way of implementing it (algorithm, code, etc) is literally a disaster. It's just too hard for any of us, the human, to just admit that our brand new "product", the one that has just been introduced, is a failure...and Gengo is not an exception.
My account could be banned after posting this comment, but I think if no one has guts to say this, I will the the first to express the real thought and feeling as a translator who has been working for over 3 years for Gengo.
Thinh Le from Vietnam.
I've read many posts on the subject in this and another forum.
Everyone seems to find the new scoring system unfair, especially to older translators. All replies by Gengo's staff are only efforts to explain the "unexplainable" and they don't convince anyone. There are many threads opened, but see if you can find Nelson's posts who makes a very detailed and accurate analysis of the situation.
I personally find the new scoring system totally unfair and frustrating. The figures below indicate why I feel betrayed to see my score drop from 10 to 9.4 after more than 6 years as a Gengo translator:
Number of GoChecks 119
Number of jobs translated 26,035
Number of words translated 557,618 (plus another 50-60 thousand in special projects)
Customers served 380
I understand that 9.4 is still a high consistency score but it's based on a very small number of low ratings dated months or years ago. As others have said, even one low rating will never allow me to regain a score of 10.
Well done Gengo, someone(s) in your company will be very satisfied for adopting his/her/their idea of changing the scoring system.
You will certainly find that not even one translator is satisfied.
I don't expect to get a reasonable reply, just the usual canned responses.
Thanks for listening (?)
@Yasemin Erdem
@Lara
This is what Lara wrote in the discussion thread "Consistency Score"
"The previous version of the scorecard was based on a weighted average of the last 10 GoCheck scores. This gives a general idea of overall quality but fails to take into account fluctuations or inconsistencies. The new version adds a buffer around this average of 10 GoChecks based on the standard deviation of past Gocheck scores. This new score (average of the 10 last GoChecks minus standard deviation) represents the lower threshold of quality that a translator delivers consistently. Your new score represents that most of the jobs delivered will be above this score."
With this formula, an exceptionally good score will increase your standard deviation and therefore decrease your consistency score !
Wonderful, isn't it? I guess Gengo needs to go back to the drawing board on this one.
Alex
"This new score (average of the 10 last GoChecks minus standard deviation) represents the lower threshold of quality that a translator delivers consistently."
Well there's a big part of your problem right there. Standard deviation makes no discrimination between positive deviation from the average and negative deviation from the average. For starters you should only be counting deviation below the average if you want the final score to actually represent the lower threshold of quality.
The deviation also needs to be weighted by word count. And you should think about weighting the deviations by how recent they are. Less recent deviations should count for less than recent ones.
I also think we should have two separate scores. An average score and a consistency corrected score. That would be far less discouraging at the very least.
Just wanted to add my two cents to this pool of discouragement and disappointment created by the implementation of the new system.
My score dropped from 9.8 to 8.2. I've been working for Gengo since 2014, I have almost 600k words under my belt, worked on hundreds of projects, and have always taken my score and my performance as a professional linguist extremely seriously. I agree that consistency as a translator is very important, but having been punished by mistakes made a very long time ago, AGAIN, is absolutely terrible, discouraging, and frankly, is making me lose interest.
After a couple of bad reviews, I spent the past year working hard to heal my score and trying to bring it back to the levels of excellence I always strive to deliver for this company. This is a slap in the face, Gengo.
What is Gengo doing to keep hard-working translators happy and feeling valued? How does Gengo reward the loyalty of people that have been translating for them on a regular basis, often daily, for years? I don't think a t-shirt is enough. Members with a very good track record and proven loyalty, consistency and quality should not be *scared* to do a job they love. This system is not making me feel alert. It's making me feel weary, worried, scared to make a mistake and be punished for it for years.
Is something being done regarding all this?
I wish Gengo offered some sort of career progression for loyal translators.
Here's how to understand the new scores:
So with 6.6, "you're doing well". I think all of us got a much lower score with this consistency calculation. I had 10 in periods previously, and close to that most of the time, but I've had some bad scores too. However, it doesn't seem like you need a score above 8 to keep your pro qualification.
Yes, I feel extremely discouraged and demotivated. Japanese-speaking and Chinese-speaking translators are quite upset too. I just wanted to leave a note here so that I get a notice each time someone posts on this thread.
Hello,
Yes that has happened to me a number of times. The errors are counted twice or three times if they are all highlighted and by requesting a new review on the last one, I had that mistake corrected.
Previously, I hadn't bothered but things are getting a bit tight, so now I'll be extra careful with mistakes made by the reviewer themselves!
My score also went down a point from 9.9 to 8.8.
Alex
I've sent this same question to the support team, Antoine (still waiting for an answer).
"How could we, translators that have been with Gengo for a long time, 3+ years, achieve a 8+ (my QS decreased to 7.3) score? In a matter of fact, what 10 jobs reviewed with 10s would impact in my QS (notice that I should have 100+ jobs reviewed, I don't really now)?
For me, this new system seems highly disadvantageous to older translators (that have waaaay more jobs reviewed). For us, I don't really see how we can achieve a 8+ score. From what I see, after reading about the new system, that it is impossible. There's simply no way. I could get those 10 10s and get like a 0.1~0.2 upgrade in my quality score, while a, I don't know, 2.4 rating, would get it way down (since lower scores apparently are given way more importance).
Hi fellow translators,
This post is so interesting and I think we are making a great effort to let Gengo know what we think about their sooooo weird and unfair new quality system. However, there is a VERY interesting post started by an user called Antoine which deals with it and also the MACHINE TRANSLATION NEW SYSTEM, giving a very interesting standpoint and showing why this MT system is unfair too. Could you please check it out and leave a comment?
The post is A bit of feedback on translation scores and MT
I agree with most everything said here. I also think that, while support's comments that translators will have to readjust to scores' new meanings (7 in many ways being the new 8) are reasonable, there is something to be said about the new score calculations not being very intuitive. For example, my score just got brought down from a 9.3 to a 7.6. Okay, I've had some low scores before, I get how that happened. But then I get a job reviewed just today, with a score of 8.45. Now my overall score is 7.5. I can only imagine that anyone new coming to Gengo who sees their score drop after getting a better review than their current score would be very put off, and might hesitate to continue doing work here. Also, the example calculations on the support article are confusing, since even in the example of the translator who IS supposed to be consistent, their total score is lower than any one of their individual scores. I think scaling the scores to be more understandable, since I'm sure the actual calculations are a bit of a trade secret, would go a long way to keeping the relationship between support and translators positive.
Hi,
Another one on the same boat: 2 years on Gengo, always overall scores around 9 and now a bright red, disappointing and frustrating 6.1 reminding me that I am 'just above ok' (which is basically what my brain interprets when I see a 6.1)
Now, the fact that before mine there are 30+ comments genuinely questioning the new system speaks for itself.
The fact that so many aspects have been brought by translators into the discussion which seem not to have been taken into consideration when designing the new system speaks for itself (e.g. that 'older' gengo translator bear a weight on their shoulders which newer ones don't and this only makes the system unfair)
The fact that endless, complicated and far from easily understandable explanations have to be provided in order to EXPLAIN a scoring system, speaks for itself.
The fact that after those explanations no-one is however convinced of the good brought by the system speaks for itself.
Really, please, reconsider the system. We are after all those who know our trajectory and history with scores and are chipping in to highlight that the system is far from being fit for purpose.
I am really relieved to see that I am not the only one with this worry since the scoring system was modified.
I guess it's a bit more reassuring when seen with this perspective, thank you for that.
Still frustrating, but a bit reassuring.
I think a "visual feedback" a tad less anxiety inducing than your score appearing in bright red might have been a nice touch then, however.
P.S: By the way, while digging on my previous scores, I stumbled upon a gocheck that was utterly dismal but with only "minor errors" outlined by the reviewer. The thing I noticed though, is that when hovering on said mistakes, the "correction" or "suggestion" from the reviewer was sometimes displayed 2 or 3 times for each mistake. Is it possible that the scoring algorithm thus counted 2 or 3 distincts corrections for each mistake and thus made me end up with a score reflecting the double or triple amount of mistakes I actually was flagged with? Thank you if anyone as any input on this.
Yes, that would be my case. That and the fact that I have had some high scores and then a few low ones like I assume is the case with some. We all make slips and mistakes and learn from it.
I guess the best way to take this now is to try and be as consistent as possible moving forward.
To all,
You may would like to read the following topic named "Consistency Score".
https://support.gengo.com/hc/ja/community/posts/360017912794--Consistency-Score-
This topic is fairly long. So I would lke to summarize it for you.
-Accoridng to Gengo's member, this "consistency score" is not a new thing, but addition. They showed this to translators in order to make it easier for translators to where we stand in regard to consistency of our quality. (However, I doubt whether this actually apply to us in current situation. Change in scoring system cannot be taken as "addition" in my point of view.)
-At least, several translators are either received of warning, or revoked of their qualification, due to consistency score, in these several months. Gengo has been using consistency score for evaluating translators for quite a while.
As I wrote, that topic is fairly long, and some of information contained in the topic might be unnecessary, but hopefully it can be some help to you.
Hi, everyone. My score was also reduced from 8.3 to 6.2 after the implementation of this new system. Today I got a job reviewed and score was 10/10, shockingly my score was reduced to 6.1!!!! I checked my last 10 reviewed jobs, from the 5th reviewed to the latest one, 5 of them scored 10, and 1 was 9.18 (the 6th reviewed), the others are crazily and unbelievably low after this new system. My question is, the latest score 10 didn't make any sense on my total score, which I imaged it should be higher than 6.2 but the reality is lower again? It's just ridiculous!!!!
Unfortunately, no.
When I started working for Gengo, it was after I lost my dear mother on June 2, 2014, after a long and debilitating illness: she had a stroke. It was a terrible time both for her and me, especially as her condition progressively got worse. Those who have gone through it will know what it means, those who haven't will, sooner or later, and I hope it is much, much later.
I don't think one can ever get over it—one just learns to live with it.
So, my first job happened absolutely unexpectedly. At the time, I was not ready to do a translation, and just looked at the available jobs. Imagine my surprise when I came back to the computer after having a meal and saw that I had started translating a job. Frankly, I hadn't pushed the button. How it happened I have no idea. I panicked: the time left was less than an hour! It was a disaster, and the score was accordingly very, very low.
I couldn't work well at that time: my heart beat like crazy, my hands trembled, and it was terrible.
But a couple of years passed, and I suddenly felt better translating. I think my scores represent two different persons and, in a sense, they do.
The big question is: how can I live down my early unfortunate experiences with the new scoring system?
I wonder why the comment I made yesterday at 18:19 is still pending.
Lara,
As Yasmin, I have also experienced that my score kept dropping following new top scores. How is that possible? Regardless what your starting point is, every subsequent score which is consistently in the 9-10 range should contribute to raise your score, albeit slowly, but never reduce it. Is that not how it works?
Thanks Alex, I agree. Although I had quite a few jobs with a 10-rating (and not just recently, but consistently over the past years), so I'm not even sure about what already seems like a warped reasoning here. What is the logic? Shall we try to include minor errors, so that the standard deviation is lower and thus the score remains the same? In other words, aim for 9 rather than 10, so that you keep your score? That is just ridiculous.
I think that no matter what, a HIGHER SCORE should never reduce your existing score. So yes, back to the drawing board.
Yes I agree with you Lara,
But whatever the usual scenario, in the case of Yasmine and unless she is mistaken or missed another review, that is exactly what happened... and you can really understand that she is frustrated to see her score go down after a perfect 10.