15

Hi,

This is my first time posting on the forums as far as I can recall, it could have been on better omens I guess, but I guess I'm more of a "no news good news" type.

So, I must say that I woke up whith a severe (and figurative) hangover when I logged in to work today and discovered my new translator score. Now, I'll get into the details later and, while I'm really not happy about it, this here post is not about ranting as much as it is about laying some hopefully constructive remarks that I hope can be backed up by my peers, or even rebuted by my colleagues in which case I would at least have learned something today.

First of all, the elephant in the room: while I have never (ever) seen my translator score dislpayed in red during the 4+ years I've worked with Gengo, I now start up and see a dreadful 6.6 on my scoreboard. That's quite a shock to be honest, I checked the mail we received explaining how all the previous scores are now taken into account, which I can understand, even if it raises a lot of questions I'll get to later (still, quite a shock). I don't know if I'm an isolated case, ending up with the lowest rating I ever had after all my work is taken into account instead of just the last 10. Now, while my score was above 8 during the past weeks, I still found that the job supply was quite weak despite the fact that it was announced that 8s and over would get jobs otherwise innaccessible for lower score holders. Now that I'm where I'm at, I must say I feel quite uneasy at the perspective of even less jobs (or even withdrawal of my qualifications).

 

Now, one can easily respond to all this: "get better". It's true, and I will, because it won't be said that I'm above getting better at my job. However, and this is my main gripe right here and now: the scoring system is flawed –not its very existence, mind you, just the way it is/was up to now; and I'd like to have the opinion of both my fellow translators and Gengo on what I'm about to develop.

  • Fisrt off: as translators might get inconsistent with the quality of their work (guilty as charged, obviously) this is also true for the reviewers and their reviews. I've been more than once appalled by some reviews I got. Rarely because they saw flaws where I (still) didn't – althought it happened in a few rare cases, but mainly by the way it impacted the ratings in the end. I once had the displeasure of being at a variance on wordings with the reviewer (i.e: nothing so objectively penalisable as a typo or a grammatical mistakes), variance I would still debate to this day btw (having asked several colleagues for input on the matter afterwards), all of this resulting on a score of 3 or worse; not gonna lie, tough pills to swallow every time this occurs. Twice I tried to signal Gengo about this, knowing –and accepting and understanding, that there was no appeal on the matter, but I did it hoping that this could find an echo in the future or that I could get further explanations on why I could merely be wrong on the matter. Twice I got cookie cutter answers, "reviews are unalterable, please check the forum/FAQs if you want more details" (paraphrasing here). I reluctantly decided that the least worst course of action then was getting my head down, waiting for other jobs, getting other jobs done good and reviewed accordingly, hoping said reviews occur quickly enough to finally expunge the "dud score(s)" from my last 10 reviewed jobs and go back to a more, shall I say, "representative" scoreboard. As I said earlier, I always managed to maintain a score of at least 7 with all that, which leads me to my next point.
  • Next point being: if an average of a translator's "career" is obviously more representative of the quality of their work on paper than just their last 10 jobs, and thus is ultimately a good idea for Gengo, its customers and even its tranlsators imo, it presupposes that the rating and review system is –and more importantely: always was– absolutely bulletproof. And I'll say it: it's not. Not enough to take such a gamble anyways. As it stands, you place the burden of every single quirk or mood from a reviewer (and I say this with all the more confidence, having had the occasion of working on both sides of the fence, and for Gengo to boot) and the burden of every potential fluke on the shoulders of a translator, and permanently so. I remember seeing in Gengo communication something along the lines of "at Gengo, we understand that a translator can have a bad day/phase and offer them a chance to win their momentum back and prove that a string of bad results was only momentary". Well, two remarks on this: firstly, by this logic, you have to take into account that the reviewers cans suffer from the same ordeal (and in my opinion, you don't), secondly, with the new system, this is utterly thrown down the sink.
  • I wholeheartedly subscribe to the fact that a translator with a string of 7s is more reliable and sounder from a business standpoint than one with several 10s and a few 3s thrown in the mix, as exemplified by your "how it works" scoring page. But to hammer it on: you then have to make really sure that the 3s really are worth 3s. And by that I'm not necessarely saying "it should be 10 or 3", but rather: "3 obviously implies mistakes, but do those mistakes really account for such a plummet in ratings?". If not, you run the risk of having otherwise pretty decent translators working with a sword of Damocles above their heads every time they submit a translation, this is both unfair and detrimental to the business and the overall quality of the work. It's a lose/lose/lose situation for Gengo, the customers, and the translators. I get that Gengo is a business, I get that running it implies handling both carrot and stick with your contractors, but there's simply too much stick here, or rather, to be more precise: handling of said stick is carried on too haphazardly.
  • Finally, I find it inconsistent –if not incoherent, that Gengo should within two days announce a revamping of its reviewing system on one hand, which could imply a change of scoring scale (be it minute or of significance, I can't say as I didn't experience the new system first hand yet), and on the other hand announce (and enforce) on the morrow the taking into account of the whole results of the older system, effective immediately. Consider this solely for the sake of argument–as I obviously lack any data on the matter: if the new system proves to be more forgiving than the previous one, then the translators present up until this point are at a huge disadvantage compared to anyone who integrates the Gengo ecosystem afterwards, and are so on criteria that are not relevant with their inherent qualities (or lack thereof) as translators. The opposite could be quite true as well, if the new system is harsher then newcomers could be at a disadvantage independently of their talent. This creates unfairness on one hand, and could fail on a systemic level at doing what it's purposed to do on the other: evaluate the qualities of every single translator, compare them to those of their peers (which works insofar as their respective evaluations are on the same scale, if you catch my drift) and then decide who to dish what jobs to. To reiterate: if that fails, then everybody accross-the-board suffers from it.

 

On a unrelated note: I suggest that you get rid of any MT stuff like those I've seen appear lately. Don't get me wrong: I really commend Gengo for not going the usual route of "here's a translation that has been MT'd beforehand, you thus have less work to do, we thus pay you less". Nothing's further from the truth, and that's why I am grateful. But please allow me to explain how it's "so far from the truth":

EDIT: Sadly, you can scratch that "grateful" part. Because after examination with other translators, it appears that Gengo indeed uses the excuse of a job that has been machine translated beforehand to scratch 25% off of a translator's fee on Pro level works. Standard jobs fees, on the other hand, remain untouched. Now, before this addendum I was about to explain why this was preposterous, so:

You have two courses of action with a machine translation that you're supposed to "smooth around the edges" (wich is often an understatement, might I had): you either try to effectively "smoothen" the existing text, which often implies mental gymnastics of epic proportions which in turn a) ends up taking more time to complete than a translation from the ground up and b) considerably increases the risks of incoherences due to careless mistakes. Or, you just delete all the text and translate everything from the ground up which, as explained in "a)" above, takes less time anyway.

Both options effectively render Machine Translation moot as a tool. Don't get me wrong, MT has its uses, especially when no translator is around, but considering the business Gengo and its contractors are in, it's moot.

Now this is going to sound like an outlandish exaggeration, but I'll stand by it for argument's sake because that remains true in absolute terms: either Gengo is developping a "learning" MT tool that will eventually be capable of effectively (and efficiently) replacing translators (I'm not judging: if it ends up working properly it's fair game), in which case: forget I said anything and please do carry on. Or Gengo is not, and it is just a tool whose sole purpose is and will be to aid translators in their work, in which case I'll refer you to the two previous paragraphs and let you draw your own conclusions.

 

I think I've said everything I needed now. Thank you for bearing through this and as I said you're welcome to chime in or answer anything that transpired in this post, I would just humbly ask that yout do so by setting out arguments regardless of the level of agreement or disagreement that you would experience.

 

Everybody have a nice one.

41 comments

  • 0
    Avatar
    Lara Fernandez

    Hi AlexF,

    All those jobs have the "TM" icon next to the LP, so I'd assume they are TM-enabled.

    Please see this Support article for more details on TM and how it affects your rewards.

    Thanks,

    Lara

  • 1
    Avatar
    Xavier

    Yeaaah, un autre français :p

    That project is indeed TM-enabled. They also provide machine translation. I let you imagine how useful that MT can be on the broken English often used by Amazon... 

    Since Gengo is now super strict with reviews and that those reviews impact which jobs I can access, I am not sure I will ever touch that project again. It is too easy for me to make mistakes, between the broken English and the weird product descriptions without pictures or so.

    I wonder how long it is going to take for Gengo to see jobs staying on the Dashboard for days because of the source language is of poor quality and the reward is not worth the risk... The Sony project is already affected for example, I see some jobs not being taken by anybody until Gengo adds 30% or so to the reward... 

    Edited by Xavier
  • 0
    Avatar
    Lara Fernandez

    @Xavier — I think you’re mixing two different features here.
    TM is translation memory, which is what’s enabled for the jobs AlexF showed :)
    TM is not new and it’s been at least 2 years since we’ve implemented it.
    MT is machine translation.

  • 0
    Avatar
    Xavier

    Hm good point Lara. I know TM have been used for a while, and I don't mind them much (apart from the fact that we can't edit them and that some of them have mistakes, but that's a different debate).

    This project uses both TM and machine translations provided by the client (or Gengo? I'm not sure), and I was referring to the latter ones being useless. They might not be affecting the rewards though, I will edit my previous comment, thanks!

     (Screenshot removed due to including Client Material - Lara)

  • 0
    Avatar
    Lara Fernandez

    @Xavier - It doesn't look in your screenshot that MT has been prefilled into the target area (which is empty). It does look like the customer provided MT for reference in a comment, but please rest assured that when this is the case, said MT does not affect your rewards in any way. MT can only affect your rewards when it has been prefilled into the target field of the workbench.

    Also, please note that I will be removing your screenshot after posting this comment, to protect our customer's privacy as per Section 1 of our Translator Agreement, which requires all customer material to be treated as confidential. This would include not only customer ID numbers and such, but also their source texts. 

    Thanks!
    Lara

  • 8
    Avatar
    Antoine

    I second Alex here, checked one those Amazon jobs that pop up constantly as of late.

    Terrible source text, but ok, you can't deem some stuff worthy or not worthy of being translated, that'd be pretty unhealty.

    What's not worth it however, is seeing a job that should pay $26 according to the wordcount being handed at 14$. Yeah, no. I don't see how this can fly, sorry.

    You can't expect us to deliver a quality job for that. You can only say "well don't bother too much and go with the MT, however broken, as long as it is understandable". Because this is the signal I read when I see this kind of practices, and it should only be logical in a commercial relationship: "you pay less, you get less".

    Thing is, it's very frustrating because no translator I know would happily let a half-a**ed translation slide, especially with their name stuck to it. But with rates like these, one has to wonder.

    And this chimes with a recent comment I saw from you, Lara, essentialy stating that Gengo enforces a strict peer-reviewing system to make sure that translators always give their best and never pulled punches. Well, no, that's flawed logic imo. I can't understand how you can expect someone working on a standard rate job to give it the same diligence they would give on a Pro level job. Why bother having two separate level of competence/princing in that case? Why should a client go for the Pro option when the company's policy is "every translation shall be equally treated by our wordsmith". Man, if I'm a customer and I read this, why should I bother paying extra?

    And of course this is even worse with those dreaded MT jobs right now.

    It should be simple: you go standard, you get a solid translation handled by people we have selected through a tough process, all this with a strong level of minimum quality strictly enforced by peer reviewing.

    You go Pro, you get a translation done by people selected through an even tougher process, you can be sure that the translation will be carried out with the utmost attention to render it as fateful as can be to your original text, perfect if you don't wanna take any chances with your target audience. On the other hand these translators and these translations will be observed all the more closely.

    You go MT: why bother? Go get Google translate.

    This system would be alright by my standards, the current system seems to keen on levelling everything downwards to my taste.

  • 4
    Avatar
    AlexF

    Hello again,

    For the Amazon jobs, I had of course noted that they were TM jobs, the thing that annoyed me was seeing the 0.035€/word indicated at the top of each job, instead of the real "reward" per word. Anybody new to TM isn't aware of how the reward is heavily discounted in some instances, and may think he is happily translating for the indicated reward as he hasn't bothered to do the maths... The price shown is what you should be paying, so please display the real reward per word! 

    On another front, I couldn't agree more with Antoine in his latest post. This has been bugging me for a while now.

    Why should we be evaluated on the same level for a Standard low-paying job than a Pro high-reward job? I understand that we are free to take or disregard Standard jobs (if we are Pro-level of course) but why should my overall score be impacted in the same way, if I do?

    Alex

     

     

    Edited by AlexF
  • 9
    Avatar
    Tony

    I will not do more work (editing a PEMT) for less pay (discounted reward). Have a rethink on this, Gengo, I'm giving you a 6.5/10 for this year's work.

  • 5
    Avatar
    Antoine

    Same here, I'm actively avoiding MT jobs. They're not worth the hassle, not moneywise, not from a risk/reward standpoint either with the rating system currently going on. And I must say that I'm none too happy about not accepting jobs on a regular basis.

    Edited by Antoine
  • 8
    Avatar
    AlexF

    After a few tries, I have also decided to stay away from MT. Not worth it, especially when it is a mumble-jumble of TM and MT keywords (Amazon), though I must admit that sometimes I have had a good laugh over some of them... 

    As the saying goes: Garbage In, Garbage Out. We should therefore be evaluated on the quality of our translation versus the quality of the source text. When a source text is filled with mistakes, missing words, punctuation or even clear meaning, you can't expect Garbage In Quality Out 

    For MT or TM, I reiterate that it would be nice to be able to opt out so that they don't show up on your dashboard and trigger off your RSS feed unnecessarily, making it difficult to identify "normal" jobs.

     

     

    Edited by AlexF
  • 5
    Avatar
    kvstegemann

    @AlexF: "For MT or TM, I reiterate that it would be nice to be able to opt out so that they don't show up on your dashboard and trigger off your RSS feed unnecessarily, making it difficult to identify "normal" jobs."

    Excellent idea! I am also bombarded with MT and TM jobs with unreasonable requests, useless pretranslation and much reduced rewards. It's nearly impossible to find the few regular jobs in the long list of junk jobs. I'd love to opt out of these too.

Please sign in to leave a comment.