This is my first time posting on the forums as far as I can recall, it could have been on better omens I guess, but I guess I'm more of a "no news good news" type.
So, I must say that I woke up whith a severe (and figurative) hangover when I logged in to work today and discovered my new translator score. Now, I'll get into the details later and, while I'm really not happy about it, this here post is not about ranting as much as it is about laying some hopefully constructive remarks that I hope can be backed up by my peers, or even rebuted by my colleagues in which case I would at least have learned something today.
First of all, the elephant in the room: while I have never (ever) seen my translator score dislpayed in red during the 4+ years I've worked with Gengo, I now start up and see a dreadful 6.6 on my scoreboard. That's quite a shock to be honest, I checked the mail we received explaining how all the previous scores are now taken into account, which I can understand, even if it raises a lot of questions I'll get to later (still, quite a shock). I don't know if I'm an isolated case, ending up with the lowest rating I ever had after all my work is taken into account instead of just the last 10. Now, while my score was above 8 during the past weeks, I still found that the job supply was quite weak despite the fact that it was announced that 8s and over would get jobs otherwise innaccessible for lower score holders. Now that I'm where I'm at, I must say I feel quite uneasy at the perspective of even less jobs (or even withdrawal of my qualifications).
Now, one can easily respond to all this: "get better". It's true, and I will, because it won't be said that I'm above getting better at my job. However, and this is my main gripe right here and now: the scoring system is flawed –not its very existence, mind you, just the way it is/was up to now; and I'd like to have the opinion of both my fellow translators and Gengo on what I'm about to develop.
- Fisrt off: as translators might get inconsistent with the quality of their work (guilty as charged, obviously) this is also true for the reviewers and their reviews. I've been more than once appalled by some reviews I got. Rarely because they saw flaws where I (still) didn't – althought it happened in a few rare cases, but mainly by the way it impacted the ratings in the end. I once had the displeasure of being at a variance on wordings with the reviewer (i.e: nothing so objectively penalisable as a typo or a grammatical mistakes), variance I would still debate to this day btw (having asked several colleagues for input on the matter afterwards), all of this resulting on a score of 3 or worse; not gonna lie, tough pills to swallow every time this occurs. Twice I tried to signal Gengo about this, knowing –and accepting and understanding, that there was no appeal on the matter, but I did it hoping that this could find an echo in the future or that I could get further explanations on why I could merely be wrong on the matter. Twice I got cookie cutter answers, "reviews are unalterable, please check the forum/FAQs if you want more details" (paraphrasing here). I reluctantly decided that the least worst course of action then was getting my head down, waiting for other jobs, getting other jobs done good and reviewed accordingly, hoping said reviews occur quickly enough to finally expunge the "dud score(s)" from my last 10 reviewed jobs and go back to a more, shall I say, "representative" scoreboard. As I said earlier, I always managed to maintain a score of at least 7 with all that, which leads me to my next point.
- Next point being: if an average of a translator's "career" is obviously more representative of the quality of their work on paper than just their last 10 jobs, and thus is ultimately a good idea for Gengo, its customers and even its tranlsators imo, it presupposes that the rating and review system is –and more importantely: always was– absolutely bulletproof. And I'll say it: it's not. Not enough to take such a gamble anyways. As it stands, you place the burden of every single quirk or mood from a reviewer (and I say this with all the more confidence, having had the occasion of working on both sides of the fence, and for Gengo to boot) and the burden of every potential fluke on the shoulders of a translator, and permanently so. I remember seeing in Gengo communication something along the lines of "at Gengo, we understand that a translator can have a bad day/phase and offer them a chance to win their momentum back and prove that a string of bad results was only momentary". Well, two remarks on this: firstly, by this logic, you have to take into account that the reviewers cans suffer from the same ordeal (and in my opinion, you don't), secondly, with the new system, this is utterly thrown down the sink.
- I wholeheartedly subscribe to the fact that a translator with a string of 7s is more reliable and sounder from a business standpoint than one with several 10s and a few 3s thrown in the mix, as exemplified by your "how it works" scoring page. But to hammer it on: you then have to make really sure that the 3s really are worth 3s. And by that I'm not necessarely saying "it should be 10 or 3", but rather: "3 obviously implies mistakes, but do those mistakes really account for such a plummet in ratings?". If not, you run the risk of having otherwise pretty decent translators working with a sword of Damocles above their heads every time they submit a translation, this is both unfair and detrimental to the business and the overall quality of the work. It's a lose/lose/lose situation for Gengo, the customers, and the translators. I get that Gengo is a business, I get that running it implies handling both carrot and stick with your contractors, but there's simply too much stick here, or rather, to be more precise: handling of said stick is carried on too haphazardly.
- Finally, I find it inconsistent –if not incoherent, that Gengo should within two days announce a revamping of its reviewing system on one hand, which could imply a change of scoring scale (be it minute or of significance, I can't say as I didn't experience the new system first hand yet), and on the other hand announce (and enforce) on the morrow the taking into account of the whole results of the older system, effective immediately. Consider this solely for the sake of argument–as I obviously lack any data on the matter: if the new system proves to be more forgiving than the previous one, then the translators present up until this point are at a huge disadvantage compared to anyone who integrates the Gengo ecosystem afterwards, and are so on criteria that are not relevant with their inherent qualities (or lack thereof) as translators. The opposite could be quite true as well, if the new system is harsher then newcomers could be at a disadvantage independently of their talent. This creates unfairness on one hand, and could fail on a systemic level at doing what it's purposed to do on the other: evaluate the qualities of every single translator, compare them to those of their peers (which works insofar as their respective evaluations are on the same scale, if you catch my drift) and then decide who to dish what jobs to. To reiterate: if that fails, then everybody accross-the-board suffers from it.
On a unrelated note: I suggest that you get rid of any MT stuff like those I've seen appear lately. Don't get me wrong: I really commend Gengo for not going the usual route of "here's a translation that has been MT'd beforehand, you thus have less work to do, we thus pay you less". Nothing's further from the truth, and that's why I am grateful. But please allow me to explain how it's "so far from the truth":
EDIT: Sadly, you can scratch that "grateful" part. Because after examination with other translators, it appears that Gengo indeed uses the excuse of a job that has been machine translated beforehand to scratch 25% off of a translator's fee on Pro level works. Standard jobs fees, on the other hand, remain untouched. Now, before this addendum I was about to explain why this was preposterous, so:
You have two courses of action with a machine translation that you're supposed to "smooth around the edges" (wich is often an understatement, might I had): you either try to effectively "smoothen" the existing text, which often implies mental gymnastics of epic proportions which in turn a) ends up taking more time to complete than a translation from the ground up and b) considerably increases the risks of incoherences due to careless mistakes. Or, you just delete all the text and translate everything from the ground up which, as explained in "a)" above, takes less time anyway.
Both options effectively render Machine Translation moot as a tool. Don't get me wrong, MT has its uses, especially when no translator is around, but considering the business Gengo and its contractors are in, it's moot.
Now this is going to sound like an outlandish exaggeration, but I'll stand by it for argument's sake because that remains true in absolute terms: either Gengo is developping a "learning" MT tool that will eventually be capable of effectively (and efficiently) replacing translators (I'm not judging: if it ends up working properly it's fair game), in which case: forget I said anything and please do carry on. Or Gengo is not, and it is just a tool whose sole purpose is and will be to aid translators in their work, in which case I'll refer you to the two previous paragraphs and let you draw your own conclusions.
I think I've said everything I needed now. Thank you for bearing through this and as I said you're welcome to chime in or answer anything that transpired in this post, I would just humbly ask that yout do so by setting out arguments regardless of the level of agreement or disagreement that you would experience.
Everybody have a nice one.