4

Hello,

I wanted to share this to see if I'm the only one that has experienced these very frustrating issues or if this could be a wider problem with regards of how jobs are reviewed. 

First up there's the issue of subjective matters, and I can take direct examples of my own experiences so we're not even talking about imaginative scenarios. I've gotten marked multiple times as errors cases where I've translated words correctly but that the LS's consider that the English word is commonly used and should be left as is. Now, while I agree that a segment of the population, mainly the well-off and the ones that already speak English would have no issue with words in English being thrown in, I'm thinking here of people who speak no English at all. One example is the word: "vintage" and in this case it was used to describe clothing. I translated it as "con acabado antiguo" which in English would be something like "with an antique look". So, my translation isn't incorrect in the sense that I am not choosing the wrong words to convey what the original text says, but the LS's consider that it is an error because they consider that the word in English is widely used and should not be translated. All of this are subjective matters and what interests me is the question if my translation should be marked as an error? I would totally be okay with a suggestion (which is not marked as an error, mainly as a note for future jobs), but why is it an error? And who decides when the English word is sufficiently used to leave it as is? Yeah, I'd understand it and my English-fluent friends would as well, but I'm sure my immediate family who doesn't speak any English at all would struggle with those. Again, this is only an example but the list of words where this very same issue could happen is an extensive one. But the fundamental question is: in these cases; should they be marked as errors or simply as suggestions? I think it's also worth noting that our job is to translate and the goal is to translate one text and make it fully intelligible in the target language. When you consider this, I'd argue that my take on it, to translate them instead of leaving the source text as it is, is the better option of the two.

 

Another issue that I've seen happen is when it comes to large volume and multiple jobs from the same customer. I know we're not supposed to talk about who the customers are - according to the Translation Agreement -, as far as I understood, and I want to respect that, so let's just say that in this case I'm talking about a large e-commerce platform that sells all kinds of products. Pretty much anything you can imagine, you'd find it in such platform. Now, the issue here is mainly about brand names. And I want to point out that in the customer's instructions, there is no glossary of brand names nor any links to product's pages in order to see if the text is referring to the actual description of the product or if it's part of the name of the brand, and to see where in the text does the brand name begin and where it ends. I can also use a real-life example of one that occurred to me. So apparently there is a clothing brand whose name is: "Screaming Eagle", and in my case the text was describing a jacket from this brand. Now, I don't know if screaming eagle is the name of the brand or if the actual jacket has a screaming eagle and the product description is describing that. I should note that all of these jobs are TM jobs, so very poorly paid and again, no info or reference as to what are brand names or even links to the products so that translators can check them. It is also worth mentioning the immense volume of jobs coming in sometimes and the ranges of products that are covered, in essence: anything you could find at a large e-commerce store. So, I'm talking about products and categories that vary from auto parts - and those being both electrical and mechanical parts -, clothing, musical instruments, camera gear, baby products, food products, toys, furniture, home appliances, electronics of all kinds (computers, phones, tablets, etc.), audio equipment, beauty products, personal hygiene, cleaning products etc. And that is only to name a few or the ones that come to mind right now. Am I supposed to be familiar with all the brands that exist in all of the above mentioned (and beyond) categories of products? And with no index or listings of brand names or any links to the products themselves? And on top of that, with the incredible volume of these type of jobs and the very low compensation? And yet, they were marked as critical errors, and thus, severely impacting my final score of the review. 

At this point I'm simply tired and don't even want to submit re-review processes because they never seem to understand what my actual point is, and in many cases, they only double down or simply ignore some instances and they don't even comment on why they don't think it was wrong, so in some cases I don't even get any answers from the LS's. And sometimes people simply don't want to admit they were wrong and instead double down on their stance.

I thank you for taking the time to read all of this, and I wonder if there's others that have faced similar frustrating situations like the ones mentioned above?

87件のコメント

  • 0
    Avatar
    Nelson Bras

    Another day, another review and another example of a bad practice that should be urgently addressed by Gengo.

    This is the reason why a single mistake may have a disproportionate weight towards your final score.

    Wordcount: 1912

    Final score 9.87/10

    How may mistakes do you need to have this final score attributed to your work?

    Several minor mistakes?

    Two major mistakes?

    Nope. A single innocent, almost undetectable, microscopic mistake. The kind of mistake that does not change the meaning/understanding of the sentence in question.

    Here’s the “little monster”:

    “Passo 1.Escolha” instead of “Passo 1. Escolha

    Can you spot the difference?

    Reviewer: “Even though the original text also lacks a space between ‘1.’ and ‘Step’, this is a mistake that should not be reproduced.”

    Yep. The reviewer is absolutely right!!!

    So… what is the problem here? Why 9.87 instead of 9.97?

     

    • Option #1 - Because the reviewer marked the mistake as a “misspeling” instead of a “punctuation” mistake;
    • Option #2 - Because the reviewer selected 2 words + 1 number + 1 punctuation mark, when he/she should have limited his/her selection to the full stop (period), since the error only refers to the absence of a space after that.
    • Option #3 – All of the above.

     

    Yep. “All of the above” is the right answer!

     

    Now, if the reviewer applies this same method for every mistake he/she comes across with, of course, the result will always be darker than it should. In some cases, it can even become catastrophic.

     

    Dear Lara Fernandez,

    I know I have already asked you a lot, but… would you be so kind as to relay this message to the Portuguese reviewers?

    You are the best. Thank you!

    Nelson Brasにより編集されました
  • 0
    Avatar
    Nelson Bras

    BTW - I have also sent you my final remarks on the previous issues.

    Interesting fact:

    I had time to contact the experts from Ciberdúvidas (https://ciberduvidas.iscte-iul.pt/) regarding two questions. Guess what - they do not agree with your 3 reviewers.

    We really need to talk about some issues related to literal interpretation by the reviewers of  some of the "rules" that you can find in Gengo's Style Guide.

    Have a nice weekend!

  • 1
    Avatar
    Lara Fernandez

    Hi Nelson - I'll try to get back to your email later today, as I'm slightly swamped at the moment :) 

    However, I wanted to address your comment above, as the information you're sharing is incorrect and may cause misunderstandings among other translators reading this thread.

    So… what is the problem here? Why 9.87 instead of 9.97?

    • Option #1 - Because the reviewer marked the mistake as a “misspeling” instead of a “punctuation” mistake;
    • Option #2 - Because the reviewer selected 2 words + 1 number + 1 punctuation mark, when he/she should have limited his/her selection to the full stop (period), since the error only refers to the absence of a space after that.
    • Option #3 – All of the above.

    Actually:

    1. A minor error of any kind has the same weight. While it sounds like in this case the correct label would have more likely been "Punctuation" due to the nature of the error, the label is not affecting the calculation in the slightest. It would have been the same if the label had been "Misspelling".
    2. The number of words that the LS selects to mark the error also hold no bearing on the weight of said error. If only a character had been selected, or if the entire text had been selected, the weight of the error would have been exactly the same. The calculation takes into account the number of errors in the text, their categories and weights, but not how long they span within the text.

    As you already know, our LS/Reviewers do not decide the score a GoCheck receives. They only mark the errors, their categories, and their severities, and our proprietary formular automatically calculates the final score. In this particular case, even if the label had been different, or the LS had marked less characters, the outcome would have been exactly the same.

  • 0
    Avatar
    Nelson Bras

    Hi Lara!

     

    Thank you for the detailed explanation. It certainly expunges a lot of mulling and stomach aches :-)

    1 - I would like to clarify that I am not implying that this was an intentional action from the reviewer to produce lower scores, but rather an innocent tagging mistake.

    2 - Upon reading your explanation, I can only arrive to a simple conclusion: if the weight of a single, almost irrelevant mistake in a 1671 words' job is that huge, the "proprietary formula" is the one to blame and should definitely be reviewed.

    Best regards!

  • 0
    Avatar
    Lara Fernandez

    Hi Nelson - no worries! 

    I know that you didn't mean to imply malice on the LS's side - however, it's still important for everybody to know that it's not a tagging mistake either :) It's also important to put everybody's mind at ease, at least regarding how much the amount of text highlighted impacts, or not, the weight of their errors.

    Regarding the formula, it's of course okay for you to disagree with it :)

    Have a great start of the week!

    Lara Fernandezにより編集されました
  • 0
    Avatar
    Fernando

    Yes Lara, I will do that, thank you. I have been very busy as there were a couple of very big projects in my language pairs but I'll try to send that information soon. Though it seems like you have been terribly busy too.

    I also would like to address an "informality", for lack of a better or stronger word, with another review I got where the LS just states: "Incorrect" in his reasoning or explanation on why they marked something as an error. If it's an error, then it's an error, I have no issues with that but at least provide evidence to your claim and do the job properly as it is expected from the rest of us, as it should be.

    To illustrate a bit further, and for the sake of transparency, the original was sort of a "nonesense" piece of text, so I wonder what the "correct" answer would be. In any case, I don't think it is acceptable for an LS to simply mark something as incorrect without further explanation. That's simply like saying: It's wrong  because I say so, and I don't find that reasonable. But at this point we all know that requesting re-reviews seldom leads to a satisfactory outcome, to say the least. :)

    Thanks again Lara for your assistance and I'll send you that info as soon as I can.

    Fernandoにより編集されました
  • 3
    Avatar
    galyna.p

    A couple of days ago I submitted a re-review request (I even managed to do it via the new "improved" form Gengo has now) providing my comments regarding the numerous cases the LS marked as "wrong term" in a job. I strongly believe a re-review must be done by another LS, not the same person reviewing the job the first time. There used to be two language specialists in my language pair (RU-EN), and every time I requested a re-review, the job was reviewed by the second person. But Gengo seems to have "optimized" the process because today, instead of receiving the second reviewer's opinion, I just received my request back with the comments of the LS who reviewed the job in the first place and whose corrections I don't agree with. Moreover, some of his new comments were like this: "Reviewers aren't required to explain every correction." or "Don't argue corrections, try to learn from them."

    Learn? The same reviewer decreased my score some time ago because he believed that the word "tunic" didn't exist in the modern world (while the text was a description of a garment for a fashion retailer, and it was a tunic!). This person also thought that a belt can't be "skinny" and jeans can't be "high-rise", only buildings can... So what can I learn from this specialist? Back then, the second LS was a ray of hope for me. Now, all hope is gone. Most of all, I am frustrated by the fact that the second LS that used to work in this pair was actually a professional and provided valuable comments but, for some reason, that person doesn't work here any more.

    For me, that was the last drop... I will never pick up another job in this language pair. 

    I realize nothing will change. Just wanted to share my experience in case someone is wondering if he/she is the only one treated unfairly.

  • 0
    Avatar
    Chris

    "The same reviewer decreased my score some time ago because he believed that the word "tunic" didn't exist in the modern world (while the text was a description of a garment for a fashion retailer, and it was a tunic!). This person also thought that a belt can't be "skinny" and jeans can't be "high-rise", only buildings can..."

    Was that score then corrected by the second LS or was that already at the re-revision stage?

    Chrisにより編集されました
  • 0
    Avatar
    galyna.p

    Hi Chris,

    It was actually a couple of years ago, before Gengo changed their score system. Those corrections were in the initial review, which I requested to be re-reviewed. It wasn't the only such case at that time, and every re-review I requested was done by the second LS, usually receiving a higher score and thus saving my overall score.

  • 1
    Avatar
    galyna.p

    Just wanted to add that the re-review system doesn't seem to exist as such any more. All Gengo allows you to do is communicate your objections to the reviewer, but it can't help your score in any way.

    The system with two reviewers checking the translation offered at least some kind of justice. I don't see how justice can be done now. If you get a bad score by an unprofessional reviewer, you can't do anything about it. 

  • 0
    Avatar
    Chris

    I think the system is still in place. It's just the language leading up to the following page and the part at the bottom that might have gotten even more discouraging (can't really compare though):

    https://support.gengo.com/hc/en-us/articles/360043592993-What-is-a-re-review-request-and-how-do-I-submit-one-

     

    Glad to hear the score for your examples got corrected. That sounds bad enough for an initial review, but for a re-review it would be pretty much inexcusable.

    I think we should have a checkbox to decide if we want a re-review done by the same or a different language specialist. If I remember correctly, this used to be decided based on the kind of correction you requested and might be decided by support currently (if I interpret the messages correctly). Even consulting an external proofreader should be an option in case there is just one for a language pair. That's obviously connected with some expenses and effort and won't happen, but being at the mercy of one bad apple is a scary thought.

    Chrisにより編集されました
  • 1
    Avatar
    galyna.p

    Well, the revision system stops working when there is only one reviewer. This is the case in my language pair. The scary thing is that this person has been working here for years...

  • 1
    Avatar
    KevanSF

    In reply to galyna.p's comments above:

     

    "The same reviewer decreased my score some time ago because he believed that the word "tunic" didn't exist in the modern world (while the text was a description of a garment for a fashion retailer, and it was a tunic!). This person also thought that a belt can't be "skinny" and jeans can't be "high-rise", only buildings can... So what can I learn from this specialist? Back then, the second LS was a ray of hope for me. Now, all hope is gone. Most of all, I am frustrated by the fact that the second LS that used to work in this pair was actually a professional and provided valuable comments but, for some reason, that person doesn't work here any more."

     

    It seems to me that for these matters of English usage and style, ANY English-native LS could provide a second review. If there are no second LS's available in the RU > EN pair, why not have a A FR > EN LS, for example, review the English product? He or she could certainly adjudicate the "error" status for things like "skinny belts" and "high-rise jeans" and anything else dealing with the English usage, grammar, and punctuation, independent of the Russian source text. There must be many EN LS's who could be called upon to read the English translation in question with an eye to finding any "errors" in the English, should they actually exist.

     

    It wouldn't be a complete solution, but it would provide a sort of safety valve for translators such as galnya.p who seem to have no other recourse.

     

    Best wishes,

     

    Kevan

  • 0
    Avatar
    galyna.p

    To KevanSF:

     

  • 0
    Avatar
    galyna.p

    Sure, it would solve the issue, but it requires additional effort from the support, so I don't think it will ever happen.

  • 0
    Avatar
    Chris

    I don't even think it would be that much more effort for support (which is excellent, in my experience) if they already have to decide how to handle re-review requests. It's more a question of implementation and resource allocation as they can only work with the options and guidelines they have been given. It should also be possible to leave this decision to the translators themselves if the instances are rare enough and only take a closer look if a pattern emerges.

  • 3
    Avatar
    Lara Fernandez

    A couple of points to clarify regarding the issues brought up by @galyna.p :)

    • The system by which a job may be reviewed by a 2nd LS continues to be in place, but this is not something that translators can choose in their request for re-review, and is for our Quality Team to assess. In the past, the team needed to assess this regardless of the translators' choice and, therefore, we found that in a high number of cases, it didn't make sense to give the choice. For example, people would address questions to the 1st reviewer (why did you XYZ?) while choosing to have the job looked at by a 2nd reviewer, or ask for a complete re-review of the job but choose to have the same person look at it. Choices contradicted the actual comments and questions that people were sending in, and therefore we simplified the process. 
    • Some language pairs only have one LS, but in cases of absolute need, we are able to check with a third party. If the problem is not the interpretation of the source, but the expression in the target, sometimes an LS with the same target, or in a similar pair, may help as well (like Kevan mentions.)
    • Please do not worry about how much more or less effort it is for Support to handle your specific case (that's really not for you to worry, and it wouldn't make sense for you to hold back because of that!) If you believe you have a case, you are always welcome to reach out and, if you are experiencing issues post a second review, it's important that you voice them so that we can look into the situation. As a last resort, if nothing else works, you can always reach out to me directly at lara.fernandez@lionbridge.com.

     

    As a reminder, I also thought to share this Support article where we explain how we monitor LS performance, please be sure to take a look!

  • -1
    Avatar
    Ballmar

     I agree with Galyna on that re-reviews should be done by a 2nd LS.

    What I’m going to say will not sound politically correct but I’m speaking sincerely about my own experiences relating to unfair penalisations, the so called “quality team” and the explanations Lara keeps repeating:

    1)      What is the point of asking for a second review if it is going to be reviewed by the same LS and with the same prejudices against some translators?

    2)      The quality team (inexistent in my opinion) as every time I asked for a re-review due to an unfair penalisation, all the theoretical members of the “quality team” agreed to the unfair penalisation. That is simply unbelievable and impossible and I don’t think such team, in the hypothetical case of existing, could be so unanimous in face of an unfair decision..

    3)      Why no-one in Gengo/Lionbridge is able to understand the helplessness that the unfairly treated translators feel?

    4)      How is it possible that no-one in Gengo/Lionbridge really cares for the translators treated in a scornful manner?

    5)      The LS’s answer that Galyna mentions “you should not question the reviews but learn from them” when the penalisations were based on wrong and unjustified reasons, describes a profile both personal and professional of some LSs. There are exceptions in Gengo/Lionbridge, of course, LSs who are professional and fair, and whose penalisations are based on real errors.

    6)      What can translators do if there is no-one in Gengo/Lionbridge defending the truth? What to do when the correct words a translator uses are not among the ones prefered by an LS?

    7) What to do when the LS is unable to offer a better option to what he/she has penalised?

    8) What to do when a LS proves to be unsure about the errors he/she has penalised?

    Gengo/Lionbridge is wrong if it thinks that to allow some LSs this way of acting against some translators will not have consequences for the company. A company that does not defend its workers's truth and leaves them in hands of unprofessional people, will inevitably end in failure. 

  • 0
    Avatar
    Chris

    Thanks for pointing out that article, Lara!

    "One of the ways in which we monitor reviews is by tracking translator requests for re-reviews to make sure that the LSs carry out GoChecks fairly and accurately. If we find that an LS receives many re-review requests from different translators and the results of the second reviews often turn out to be very different from the original one, we further investigate to see if any retraining is necessary."

    It seems like this would incentivize the LSs to match their initial review as closely as possible (if they do the second review themselves) in order to avoid an investigation. Obviously a conscientious proofreader wouldn't fear an investigation as much and wouldn't mind admitting they've made an error in the first place but this could lead to an even worse behavior with some LSs who perhaps think their position is at risk.

  • 0
    Avatar
    Lara Fernandez

    Thanks for bringing this up, @Chris :)
    What the excerpt you quoted fails to mention is that the reviews are not always conducted by the same LS (as I already mentioned, the system where another LS may review your job is still in place!) — a second person has no incentive to stick to the first review.

    Lara Fernandezにより編集されました
  • 0
    Avatar
    Chris

    That's true of course, but it depends on how often a second person is actually consulted in cases where none is readily available.

    Now I realize we don't have all the facts (obviously there are also translators who can't accept they made a mistake) and these things can take time and are better settled behind the scenes to avoid mob justice. And like mentioned, I'm glad to report that the system has worked great for me so far. But the impression I'm left with is that the system hasn't worked for people like galyna.p, going by the information that's publicly available at this point in time.

    Chrisにより編集されました
  • 2
    Avatar
    gunnarbu

    Hello. Just to bring some balance into the picture - for me the system seems to work very well. I have requested a number of re-reviews over the years, and a number of them have actually resulted in a higher score. I am not sure if it has been done by a second LS or not, but I have so far always been treated fairly. Reading some of the comments here and in some of the other ongoing threads, like the one on 'Delay of job approvals' one could wrongly get the impression that Gengo is a bad place to work. If these are the only complaints from 20.000 + translators, then I would say that the rate of complaints is quite microscopic :-) (Probably there are more complaints submitted by other channels, but it would surprise me if this was a huge issue). Of course there can be instances of poor LS work, and this should be reported and dealt with, and as Lara commented somewhere it also seems that Gengo proactively is using other means to discover this, e.g. by monitoring abnormally high frequencencies of re-review requests towards certain LS etc.

  • 1
    Avatar
    galyna.p

    I had a chance to see how the system works with one LS and two of them.

    The first time I submitted a re-review request to Gengo was back in 2016 (I did it because of the same LS who reviewed my job last week). Back then, I was astonished by the fact that a LS didn’t even bother to check the glossary provided by the client, not to mention his complete incompetence at the subject (I wrote about being penalized for “skinny belts” and “high-rise jeans”), so I had to request a new revision. There were two LSs in my language pair back then. In the majority of cases, I got a much higher score from the second LS re-reviewing the same job (it was the perfect example of a case when “the results of the second reviews often turned out to be very different from the original one”). I assume I wasn’t the only one having problems with the first LS at that time.

    Back then I actually noticed that some “retraining” might have taken place because the LS at least started providing more comments to his corrections. It was not enough, of course, since the problem lay much deeper.

    The major problem in this case is that, if one LS is unprofessional, a translator has to constantly request re-reviews in order to save his/her score. Why do we have to waste our time doing it?

    What happened next is that at some point Gengo parted with the second LS who worked in this pair and stayed with the worst of the two. Dead end. 

  • 1
    Avatar
    galyna.p

    To gunnarbu:

    " I have requested a number of re-reviews over the years, and a number of them have actually resulted in a higher score."

    And how is that fair? It only means that the first review was unfair.

  • 1
    Avatar
    gunnarbu

    To galyna:

    It was fair because I had made a good argument with reference to examples and sources that confirmed my view, which the LS then acknowledged. This is not the same as to say that the first review was unfair; this is more like fruitful feedback-loop and learning. We are all constantly learning, both translators and LS, nobody is perfect. All LS should have this kind of mindset, but I understand very well from some of the gruelling examples above that this is evidently not  always the case, and I trust that Gengo is on the ball.

  • 0
    Avatar
    Lara Fernandez

    @galyna.p -- I actually got curious and wanted to confirm the number of LS in the language pair you mention, RU-EN, so I took a look at our records :) There's actually 2 LSs in this pair, and both are currently active.

    @Chris -- A second person will conduct the review based on the content of the request submitted by the translator, I think we're kind of going down the assumption that "A second review will always be conducted by default by the same person who conducted the first review, unless XYZ". There's no such default -- the team will evaluate the request and forward to the most appropriate channel (same LS, or different LS). Of course, on occasion, the result yielded may not be perfect -- and this is why, for exceptional circumstances, I advised @galyna.p (and I always advise other translators) to contact me if need be. This is not to say I have the power to remove errors or change scores -- what I can do is advocate for the translator and push for a closer look at their work, but it's necessary to understand that sometimes it turns out that the translator is wrong (it happens, we can all be wrong sometimes without even realizing so). In other words, reaching out to me is not a magical spell to make things go the translator's way, but rather a way to get your case looked at again, in a closer manner. 

    I would also like to point out that we not only monitor changes in re-reviews but, as @gunnarbu also mentions, we monitor the re-review request rate for each LS and pay close attention to the data. High re-review request ratios would definitely indicate a problem and dissatisfaction among translators, and that would also be a trigger for investigation.

    You also bring up an important point, Chris, with this comment:

    obviously there are also translators who can't accept they made a mistake

    This is also very real, and it does happen. It is, of course, beyond the scope of this conversation, but it is important to remember that there's all types of people in a pool of 20k+ translators. The system is designed to be as fair as possible, but of course, with such a large amount of people (and variety of cases) it's very hard to achieve 100% perfection. We also understand this (bringing us back to my points above regarding exceptional assistance if/when necessary, etc). For the most part, though, like @gunnarbu has mentioned, I feel like, for a pool of this size, the system does work mostly as intended -- and it goes without saying that we're always looking for ways to refine and improve it :)

    Lara Fernandezにより編集されました
  • 2
    Avatar
    galyna.p

    Ah, Gengo were finally able to reach the second LS and my job was re-reviewed. Now I see what’s going on.

    In the past, second reviews were “blind” reviews of the same job by another person. Now, the second LS reviews the corrections made by the first LS. It is absolutely obvious that it is done to eliminate cases when “the results of the second reviews often turn out to be very different from the original one”.

    So today I received my job with all the comments from the first LS and a tiny comment form the second LS at the bottom: “From your alternative reviewer: as you will see from my comments, I agree with most of my colleague's points. However, all of these issues were quite minor, so I removed some of them and "downgraded" some to suggestions.” This person mentions his/her “comments” while there are none of them in fact, only the same comments that the previous LS provided. As a result, the score for the job increased by 0.7.

    Now I feel sorry for some people working as a LS. They are pushed to make their review stay as close as possible to the previous one.

    galyna.pにより編集されました
サインインしてコメントを残してください。