I was debating whether to bring this up quietly as a support ticket, but after reading these steady streams of posts about the reviews process, I decided it might be better to bring it to a public forum and offer my two cents on the matter.
I accept GoChecks as a fact of life in order to continue working with Gengo. And they can serve as a wakeup call in some cases, so I try to receive them constructively. And in fact, in most cases I welcomed the reviews, because hopefully I can build back and rehabilitate my score (albeit at an excruciatingly slow rate). I concede that it's easy to feel victimized over receiving a low score for a "harmless" or unintentional mistake (a mistake that shouldn't have been made anyway). But lately, it feels as if I have to defend myself by asking for re-reviews more frequently. In addition, and this is merely a perception of mine, it feels as if the quality of the reviews has gotten worse.
I've read the many posts complaining about the reviews, and I would say that it's difficult to properly judge individual situations unless we all could see the entire source text, the translated text, and the full review of the jobs in question. But we can't do that, so I won't talk about reviews which I perceived to be "subjective." I will only talk in the simplest terms regarding what I believe to be two objectively "unfair" reviews.
In one case, the review was flat-out, unquestionably incorrect (a March 9.5 score that was changed to 10). That review indicated a comma splice that did not, in fact, exist. There's not much more I can say about it than that. But if you take my word for it, this sounds like a "critical error" on the reviewer's part.
A review in May indicated a "critical error" (7.2 changed to 10) -- I won't get into all the details, but the source text from the customer contained a sentence that seemed vague enough to allow for several possibilities. Since I only had 2 hours to complete the job, I submitted the translation, but wrote to the customer relaying my concerns, asking them to verify and correct my concerns. They wrote back after a few days with additional information (thereby justifying, in my opinion, the decision to submit and fix it later), and I submitted the correct translation via the comments section. However, the reviewer wrote in their comment that "revisions made after submission in the comments field are outside of the scope of reviews; reviews assess errors present at the time of submission." -- which sounds like a rather soul-crushing bureaucratic answer. Anyway, I argued that the review should take into account the totality of the job, including my communication with the customer, because that was the process which helped me to arrive at the correct translation. And furthermore, the correct translation I provided was basically the same phrasing as what the reviewer suggested in their comment.
These are simply common sense arguments. If the reviewer doesn't know what a comma splice is, they should be doing their homework before penalizing someone for an imagined offense. And if the reviewer thinks that comments don't count, then do they believe that they aren't being used in order to arrive at a correct translation? In fact, this reviewer understood that I was using the comments section for this purpose -- they said so in their review -- and yet they decided to penalize me anyway!
So those are the facts of the matter as I see it. Now let me tell you how this experience makes me feel from an emotional standpoint.
I don't know if I'm being reviewed more frequently than others (almost once a week on average, and I've received close to 300 in total), but it can feel at times like they're trying to catch me on an "off" day when the reviews keep coming one after another. That in itself is exhausting, but I'm okay with it, because the more good scores I can get, the better for my GoCheck score. However, it begins to feel like an assault when I get the sense that I'm being reviewed incorrectly or unfairly.
And even when I achieve a successful re-review, it's rare that I get any sufficient answers from the reviewer. Nor do I get an answer when I ask what I should do in the future if a similar situation occurs. The incorrect review in March (the non-existent comma splice) came with no explanation whatsoever. Just the revised score.
The re-review from May (7.2 changed to 10) came only with a comment that said, "Updated to reflect change in reviews policy." Which means, what exactly? That sounds like the reviewer saying that they revised the score only because of a change in policy. So I was at the mercy of a policy decision? Did that policy change suddenly occur in the day or two between my re-review submission and the subsequently revised score? If I am required to be up to date on grammar rules and whatever style guide a customer assigns (and updates constantly), then shouldn't I be assured that the reviewers are following the rules as well? Why is it up to me to catch an incorrect review? If I didn't contest these reviews, my score may have dropped low enough to get me revoked. If I'm going to be judged this severely, I would hope that the reviewers are held to the same standards themselves.
Maybe subconsciously, what I really wanted was an apology or acknowledgement of the mistakes. Perhaps that is a bit much to ask for, but you know what? If I had received some form of acknowledgement, it would at least make me feel better about the whole thing -- that I would know that there were fellow human beings behind the process too, that they make mistakes too, and that they're trying to get better as well. Being tossed a revised "10" score from the other side of that bureaucratic wall doesn't feel so encouraging.
Reading the forums, I get the sense that the Gengo staff who respond to the posts are trying their best, encouraging the frustrated translators to try various methods in order to work out their various issues. The reviews, meanwhile, feel as if they are coming from some kind of bean-counting mentality.
In the old days, if the boss didn't like you, he or she could call you in and say, "Hey, this isn't working out, etc." Here, it practically feels like I'm being subjected to death by a thousand paper cuts until my score gets low enough to eject me. Look, I get the reality. I'm nothing more than a gig worker in a larger ecosphere that can easily survive without my contributions, so I'll have to accept it or find something else to do.
I apologize for this long-winded post. Normally, I'd prefer to just keep my head down instead of complaining. But this entire situation has been frustrating, fills me with constant dread, provides little or no satisfaction (even when a re-review is successful), and is ultimately a soul-sucking experience, because it feels as if the time between reviews and re-reviews is nothing more than a short period of relief before the whole thing simply happens all over again. Groundhog Day.
11件のコメント
Like everybody, I do make mistakes. My profile score is high enough, but I also have two scores of less than 5.0. One was a fake-news article, weaving some real Hollywood personae with a scam miracle cosmetic cream. We were to translate also the fake "comments" section, which had grammatical mistakes sprinkled here and there for realism. Another, more recent disaster, was for the AI machine translation project: the source text turned out to be itself a translation of an English-language magazine article (which I later tracked down). Needless to say, my back-translation into English had nothing in common with the original English article :)
I have seen other jobs where the source text was either for a scam operation and/or irredeemably ungrammatical. Sure, I don't have to accept them and I can let others take it if they want to. Still, I would like to suggest to Gengo management that maybe we need to allow translators, or at least those with a longer track record here, to be able to flag and refuse these back to the customer.
In other words, if we are to take seriously the quality control of our output, we should also reject low quality input.
Hi Heike! Not really, to be fair. You should be aware that in general, when the source contains a grammatical mistake or inconsistency, we are supposed to correct it in the translation.
My point is, if I am struggling with all the semantical and grammatical nonsense in the source text, I cannot do a very good job. And I do want to do a good job. That particular time:
1) "The balm not only preserves youth but regenerates it!"
The reviewer gave a wrong term error for this, suggesting only -> merely.
2) A fake "doctor" is being asked about the cream:
"We turned to him for his opinion about the XYZ anti-aging balm."
The reviewer: "Not the same meaning as source" (the original said "turned to him for clarifications")
3) Comments section:
"Ritchie" said in the original: "This cream is a bomb!"
My translation: "This cream is like a supernova!"
Reviewer: "Doesn't work - the phrase requires an English equivalent - of the phrase, not the words in the phrase."
(I should have said "This cream is a blast!", but Ludwig search finds "like a supernova" in NYT etc. Why give me a bad mark for this?)
4) "... contains one of the most powerful antioxidants that neutralize free radicals."
The reviewer corrected this neutralize -> neutralizes. On english.stackexchange.com there is a long and inconclusive discussion about whether such a phrase requires harmony with singular "one" or with plural "antioxidants". Maybe the reviewer is not a chemist and does not know that any and all antioxidants neutralize free radicals?
Then of course there were a couple more serious errors made by me, which were caused by nonsensical phrases in the original.
So as you can see, the reviewer is a professional and trying to do their job but the end result is puzzling.
Finally, the customer complained that this was not worthy of a pro-level and gave me a zero-star rating. But check out this exchange:
Me: "it seems that the correct name of this creme is Budicene, not Budicence. Should this be corrected?"
Customer: "don't bother the name of the product, let it be 'XXX'. doesn't matter."
Bottom line is that I am not anymore taking jobs with ponzi scams, fake-news, or nonsensical/ungrammatical source text. Final straw was that AI machine translation project, and as a result I did not work at all for the past 12 months and only now came back. I guess that's my free choice, so I should be happy.
Sorry to the OP for hijacking the thread. I agree with his point, and just want to add that the reviews need to take into account the quality of the original text.
Hello KS, Heike S., thank you for your input.
As for "hijacking the threat," not the case at all! In fact, I was hoping to hear from others about their experiences. It's helpful to know that I'm not alone in my "misery!"
It's hard understanding how exactly unfair someone else's experience with a review is without being in their shoes, but I feel pretty certain that we share common ground in how it makes us feel.
Like you say, KS, about not wanting to take on certain jobs. I feel the same exact way. I try to stay away from jobs that seem like "trouble" -- lots of revisions, unreasonable requests, etc. -- those seem like just a hair away from triggering a low feedback score from the customer. Sometimes I take the job anyway, if I feel confident enough, as in, "I dare you to give me a 1!" But either way, the review specter has wormed its way into my brain and has been affecting my behavior now for some time.
You brought up another good point about source text quality. I also have noticed for a long time that there are many jobs that contain badly written source text. I would think that in many cases, the very things the reviewers like to ding us for -- "repeated words," phrases they don't like, etc. -- the way the source text is written inevitably affects how the translation is written. For example, if the customer writes awkwardly, and repeats the same word over and over again, is it the translator's job to pull out their Thesaurus and find different ways to say the same thing? Why should I care? That's for a managing editor or some other person on a higher pay grade to decide. It already takes a good amount of time to read a badly written source text, take it apart and find the correct meaning, and then translate it, without having to worry about whether it's ready for publication. We're just translating so that the customer knows what's being said. They can make their own decisions about massaging the wording. I know that argument doesn't work for all jobs, but if the translation is otherwise correct, it feels like an unnecessary extra paper cut.
In one case I have a customer that puts out their own style guide (that changes periodically without notice), have several terms that they want automatically translated into pre-determined terms (I don't know what to call that, it's that green highlighted text that you click on and automatically get transferred to the translated text box), which can make it challenging to work around, and then the source text is written pretty badly to the point of incoherence sometimes. Luckily, the customer's style guide says that they would rather have a creative translation than a straight-up translation (they used to even say so before, but for some reason don't express it as directly now).
Well, so what happens usually when I get a review regarding these jobs is that the reviewer disagrees with "how" I wrote the translation (as opposed to the more objective "what"). For example, they say "it should be written like this" or "this sounds awkward" or "this phrasing is incorrect," etc. All perfectly valid arguments in a vacuum, I would concede, but usually, the reason they say those things is because they're not reading the comments made by the customer, or they didn't read the style guide (I constantly have to bring up the style guide to the reviewers), or they didn't notice those automatic translated term things.
So I have to constantly bring up all of those things to explain why I made the decisions I did, and it seemed fine when I would usually be able to convince the reviewer to consider those factors. But after a while, it gets exhausting, making the same argument over and over again. And eventually, you'll hit on a reviewer who still won't agree. And then I start to wonder if it's all worth it, because now I feel like I'm writing translations for two competing agendas -- one, the customer, who has laid out all these rules, and two, the reviewer, who somehow doesn't see all those rules, and has to be walked through them each and every time (how many reviewers are out there? Are they seeing this for the first time each instance?)
Have you noticed by now how many times I've been using parentheses and em-dashes to bolster my arguments? I've gotten into the habit of trying to cram in every appeal and explanation into as short a space as possible because I always feel like I have to protect myself.
You know, I've read so many posts now explaining how the GoCheck score works, how it's calculated, this and that. And I could care less how it's done. Sure, I would certainly appreciate a more generous way of calculation so that my score could better reflect my current quality - like I keep saying, I believe that I have learned to be more diligent and more dedicated, but it's not so motivating looking at a score that barely ever moves no matter how many 10s I accumulate, but ready to drop like a rock at the first 9 score.
Actually, this isn't good for my health, I can feel my blood boiling already, and I've probably already punched out over 3,000 characters here. I'm sure we all have better things to do than b***h and moan, but it sure feels good to vent!
Great post and discussion. Regarding what the original poster and others have mentioned about feeling targeted after receiving multiple reviews in a row, one thing I have noticed is that when a bad score triggers a succession of reviews, there is a tendency for the reviewers to start to "smell blood" and become harsher in their grading -- becoming more likely to mark what should be "suggestions" as "errors" and so on. In essence, they come in already expecting that there will be problems with the job and this seems to noticeably color their judgment. I have noticed on multiple occasions that the same thing seems to happen when a rejection or bad feedback from a customer triggers a review.
All this is perfectly understandable given normal human psychology, but it is also utterly irrational and something that Gengo should train the Language Specialists to consciously guard against.
Ludwig is fantastic, and if any reviewers are reading this, I would encourage them to kindly look there when marking some phrase as a calque. For example, it was Ludwig which turned up the following phrases:
Don't tie up yourself in knots over this. The reviews are not there to punish people who have slipped from a 9.8 score to a 9.5. The reviews are there to kick out people who paid someone to cheat on the exam, got in, and post google translated jobs for a few quick bucks.
EDIT: Or maybe not. I would very much appreciate some comment from Gengo.
KS, thank you for elaborating! In my opinion a reviewer deciding a metaphor doesn't work doesn't necessarily mean that it doesn't work. And I totally agree with you on the subject of source text quality. I remember one AI project where I had to reject quite a few individual jobs as non-translatable. There was nothing else that I could do. I got multiple revision requests for them, asking why I considered them untranslable, but in fact, the language to be translated was total gibberish. In some of those cases I found out that the "source texts" were machine translations of texts written in yet another language. In later AI projects, the source texts were much better, but I'm not sure if they were from the same customer.
I don't know how gengo313 feels about this, but I think you are not hijacking the thread but responding to it and sharing your own, similar experience due to the way the system works. It must be a difficult process for Gengo to try and remedy the faults of this system which in many respects works very well but makes communication of its participants very difficult.
Hmm, I just noticed that I had written, "hijacking the threat" instead of "hijacking the thread." Interesting Freudian slip, if I say so myself ...
KS and gengo313: I hope the LS are aware of that ;-) In my experience: the less simple typos I do the more they seem to find things that the don't like now. Without the reviews I would have been blissly unaware of the fact that I have NOT gotten any better ;-)
Hi KS, what were your supposed mistakes in the fake comments section? Did you include mistakes in your translation for realism and got punished?
By the way, I didn't know Ludwig search :-)