0

Hi, I know that this has been discussed, but I don't understand. From Oct 6, 2019 to now, I've had ~43 scores. 36 were 10s. Of the other 7, they were

9.2 (x2), 9.3, 9.5 (x2), 9.6, and 97.

My last 10 scores were 9.7, 9.5, 9.2, 9.6, 10, 10, 10, 10, 10, 10.

I'm honestly asking - Why is my average 8.9? I mean, I get that scores are weighted and there is a consistency factor, but I don't understand how my average is lower than any of my scores in the last 2 years (my last low score was 5,1 on Sept 4, 2019).

In my whole time here (Sept 2017), I've  had over 18,000 approved translations and of those, only 7 really bad scores- 4.9, 5.3, 6.5, 6.3, 4, 7.7, 5.1 (that's the last bad score I've had. it was on 9/4/2019- there was one critical misspelling that took my score down almost 5 points to 5.1).

When will those bad grades drop off my record? If a translator improves and makes progress, at some point the standard deviation should improve. This takes into account that people improve. Thanks for clarifying!

 

14件のコメント

  • 0
    Avatar
    gunnarbu

    Hi Cheryl.

    Yes, this has been discussed at length in other threads. I think there is also the issue of a rolling window of your 10 last scores. Did you taket that into consideration? If I have understood this right (Please correct me, Lara, ift his is not correct …), this can have the effect that if you get for example get a score of 9.2 on your last job, and your consistency score is say 8.5, which is lower, you would logically expect your consistency score to increase a notch. However that is dependent of the value of the score that dropped out at the bottom of your rolling window of your last 10 scores. If that was higher than 9.2, your consistency score might even go down a notch, which seems strange, but that’ the way it works. Again – please arrest me anyone if this is not correctly understood – you could say I write this as a test of my own understanding 😊

  • 0
    Avatar
    Heike S.

    Well well, complicated is not always better ...

  • 0
    Avatar
    Cheryl

    Thank you, Gunnarbu. I do understand what you've put here. My last ten scores are listed above. The ten before that is 10, 10, 10, 10, 10, 10, 10, 10, 9.5, 10. 

    My question is more when do they recalculate the standard deviation so that our scores as 'beginners' don't affect us when we've improved. In my case, my last score below 7 was Sept 4, 2019. And apart from that, I've only had 6 scores below 7 in my whole time here (since 2017). 

  • 0
    Avatar
    Chris

    As far as I know, only the worst score ever will be ignored for the consistency score, everything else will weigh you down forever, which isn't exactly motivating.

  • 0
    Avatar
    gunnarbu

    Yes - "... weigh you down forever ...", I think I used a similar term in one of the other threads on the subject. It seems unfair that we should be held down for poor performances comitted many years ago. Everybody can improve over the years, by practice and experience, better proofreading, or in some cases by studying more, so perhaps Gengo should consider putting into the algorithm that very low scores that are more than x years old are wheigted down compared to more recent low scores. Just a suggestion - probably many ways to do this. A simpler way would simply be to delete the lowest score from every year, not just the single lowest score ever. Everybody can have a bad day ... Gunnar

  • 0
    Avatar
    Heike S.

    I hope, Gunnar, Gengo will consider your suggestion. There have been so many questions and no matter how often the system is explained, hardly anyone understands it or the scientific rationale behind it, if there is one, which I doubt.

  • 0
    Avatar
    gunnarbu

    Heike, I am sure there is a scientfic, educational and quality assurance rationale behind it, and for me it is not hard to agree that it makes sense to have something like this in place to ensure quality vs the customers and an incentive for us translators to deliver consistently high quality translations. I think what it boils down to is - how hard and how long should you be 'punished' for past sins? 

  • 1
    Avatar
    Heike S.

    Of course, I wasn't saying we don't need quality control at all, Gunnar. But this system doesn't seem to be scientific.

    Heike S.により編集されました
  • 1
    Avatar
    Heike S.

    I'd like to know, Gengo, is the GoCheck system based on science? Where does it come from, who designed it? Is it an official quality control system or just something some software engineer came up with? I'm genuinely interested. I, personally, am still satisfied with my score, but I also noticed it hardly ever goes up after I get a 10. I'm not surprised people complain and claim it isn't motivating.

    Heike S.により編集されました
  • 0
    Avatar
    Katrina Paterson

    Hello Cheryl, thanks for bringing this topic up. As Gunnar has mentioned, this topic is indeed covered on some of our other forum threads, as well as in this Support article here, but for sure we’re always happy to clarify. 

    As you probably know already, translator scores are calculated based on a weighted average of the ten most recent GoChecks, minus the standard deviation which is calculated based on a weighted average of all previous GoChecks other than the lowest-scoring one. This means that the standard deviation part of your score is based on your entire GoCheck history and every score other than the lowest one will always remain on that record.

    As you guys have mentioned, there have been lots of discussions around the quality score. To address Heike’s question regarding where our proprietary formula comes from: this was calculated and refined over numerous iterations by our Quality Team, in order to reach its current state, where it provides us with what we’ve come to consider a reliable way to forecast a translator’s performance. In this light, we’d like to take this opportunity to remind you that your score is not a reflection of your quality in the way that an average would be, but rather an indicator of a quality threshold that your delivered work consistently meets. In other words, if your score is 8.9, it doesn’t mean your quality is a 8.9 flat, it means that you are a reliable translator whom we can expect to consistently deliver work at or above that score.

    We hope this helps!

  • 1
    Avatar
    Cheryl

    The way you worded the last sentence made all this finally make sense. Thank you!

    My request (as well as many other people's) is two fold: 
    1. Every translator needs to have 2-3 years of Gocheck history or 200 (?) scores (those are just numbers I pulled out of the air). This would put a 'statute of limitations' on low scores. 
    2. Once a year (or as it happens, which ever is easier), all scores over those numbers get dropped. 

    Here's the scenario as I see it played out- I started in 2017. We're in 2021. what I'm proposing is everything before 2018 gets dropped or archived or its coefficient/weight lowers. In fact, that may be a better idea- the coefficient/standard deviation be on a scale where it gets lower as the users get more experience- first 2 years it's X, then 3rd year it drops a bit and 4th year a bit more so those low scores are less and less important. Once someone hits 5 years, the coeff/deviation will become permanent. 

    I'm not an IT person, it may not be feasible, but I think if we know that eventually those scores will work themselves out of the system we'd feel better about everything, lol. 

  • 1
    Avatar
    Katrina Paterson

    Hi Cheryl, thank you very much for your suggestions about the translator score system. We'll be sure to pass them on to the Quality Team for if and when they're planning any changes. We're happy to hear that our explanation helped!

  • 0
    Avatar
    gunnarbu

    Hello all, I remember when I started with Gengo it took quite some time before I discovered that each translator actually has a public Gengo profile, so in case there are others that are not aware of this - your 'normal' average score is displayed there. 

  • 0
    Avatar
    Heike S.

    Thank you Katrina!

サインインしてコメントを残してください。