See also this post:
https://support.gengo.com/hc/ja/community/posts/360038160474-Re-review-sent-to-the-same-reviewer-
So recently I made a re-review request where I gave detailed reasons for why I believe that the original reviewer was mistaken. I also requested explicitly that I would like the job to be reviewed again by a different Langauge Specialist.
Despite this, I received a re-review from the *same* reviewer. The tone of the messages was very defensive (and frankly, passive-aggressive bordering on unprofessional: "Oh, by the way, I should have marked you for this error as well...").
I have two things to say about this.
First, the point I made in my original post stands. I think it is just common sense that the second review should be done by someone other than the individual who did the original review, in order to keep the re-review process fair. Obviously, the original reviewer will feel inclined to be defensive about their choices, and this will likely bias them in favor of the original score. I think "re-review" should mean just that -- having a different Language Specialist conduct a separate review, to independently corroborate the original score.
This point is especially concerning as according to a support article by Gengo:
"If . . . an LS receives many re-review requests from different translators and the results of the second reviews often turn out to be very different from the original one, we further investigate to see if any retraining is necessary."
If this is the case, how on earth can it possibly make sense to allow the same LS to conduct the second review? What incentive would the LS have to make changes to his/her original score, if any changes are going to negatively impact his/her own evaluation? I hope that someone can give us a logical explanation of this point.
Second, I have noticed that a lot of big changes were made to the re-review system right after (and presumably as a response to) the concerns I raised in the post I linked to above. The main changes I have noticed are:
(1) It is no longer possible for us translators to see who is conducting the reviews (previously, the comments the LS left on the review displayed their translator number, but this has since been removed).
(2) There is no longer an explicit option available on the re-review request form that allows translators to request that their job be evaluated by a different reviewer.
So basically, it appears that what Gengo has done is to respond to my concerns about the transparency of the re-review process by lowering their transparency and making sure translators have less say in the process, rather than more.
Now I find these changes by Gengo very concerning, bordering on the downright disingenuous. I have noticed that people like Katrina and Lara are making an effort these days to create a sense of community within Gengo translators, but how can we translators continue to trust Gengo if this is how they respond to our very real concerns?
19 comments
Hi @KO,
Thanks for bringing this up :)
I believe I've explained this before in other threads, but allow me to clarify once again: the option to explicitly request that a second reviewer takes a look at the job was removed for two reasons:
1) Some language pairs have only one LS and it's not possible for us to have a second reviewer routinely look at re-review requests. For specific strong cases and/or if we have reasons to suspect something is amiss, we may bring in a third-party reviewer on a case-by-case basis, but this shouldn't be taken as standard procedure.
2) For languages that have more than one LS, unfortunately translators were routinely making mistakes in their re-review requests, marking they wanted a second reviewer to look at the job while addressing their questions regarding the first review to the first reviewer (questions which the second person would not be able to answer, since they are not the first reviewer) and viceversa, marking they wanted the same reviewer to look at the job while in the comments mentioning they wanted the job to be reviewed from scratch. In a nutshell: the options were being widely misused and causing a lot of unnecessary back and forth.
Based on the above, we removed that option and left it at the discretion of the Quality Team to carefully read through the requests and redirect them to the first or second reviewer based on the actual comments and claims left by the translator. In your specific case, I haven't taken a look at your re-review yet, but I will double check with the team on the reason why the job was sent to the same reviewer even if you had explicitly in the comments request that someone else look at it. I will get back to you once I have more clarity on this :)
I wasn't aware of the LS # being removed from reviews, so I will check on this for you as well :)
Now, as for this point:
"If . . . an LS receives many re-review requests from different translators and the results of the second reviews often turn out to be very different from the original one, we further investigate to see if any retraining is necessary."
If this is the case, how on earth can it possibly make sense to allow the same LS to conduct the second review? What incentive would the LS have to make changes to his/her original score, if any changes are going to negatively impact his/her own evaluation? I hope that someone can give us a logical explanation of this point.
I believe this was also explained in some other thread in the past. First and most importantly, this is not the only data that we look at to monitor LS performance, this is one among many other indicators, so you shouldn't take it as the absolute only reason why we would investigate or review performance. As I mentioned above as well, for example, if an LS receives an inordinate amount of re-review requests, showing a spike of incidences, we may bring in a third-party to review their work even if they have never changed scores. If we were to find an LS was deliberately producing inaccurate reviews in order not to change the scores, we would take immediate action towards termination, so it is in their best interest to change the scores when applicable.
Thanks!
Lara
Following up from my previous post -- @KO, I've spoken with our Quality Team and it seems that there was a misunderstanding. The member of the staff who redirected your request does not speak Japanese, and they relied on Google Translate, for which they apologize. I have taken a look at your re-review request myself, confirmed that you had asked explicitly for someone else to look at the job, and have requested that they trigger a new re-review accordingly for you. You should expect to receive your new re-review within the next few days :)
Thank you for your patience,
Lara
Hello Lara,
hopefully you'll be able to see my comment on this old thread (I didn't want to start a new one about an old topic).
I recently had a negative re-review experience where the same LS also took on the re-review. At least I am almost certain it is the same LS because they refer to choices they made or could have made in the first review.
Without, for the time being, getting into the details as to why this was a negative experience, it just seems that lately my re-review requests often tend to be assigned to the original specialist, with all the potential problems eloquently described by the original creator of this post.
I have read your comment above, but I still have some doubts. My language pair is ENG>ITA, so the first point (that there's possibly only one LS for a language pair) shouldn't apply here?
These are my questions:
- Can you confirm that, everything else being equal, the standard preference should go towards assigning a re-review to a *different* LS and that only for special reasons the quality team might assign the re-review to the same LS? It seems obvious to me that this should be the ideal solution whenever possible since, generally speaking, any type of "appeal process" is based on the principle that you should be judged by a different jury.
- If so, could you suggest what some possible reasons could be for which the Quality Team decides to reassign the review to the same specialist? I am starting to wonder whether I'm doing something wrong with my requests. Generally speaking, in my re-review requests, I tend to strictly discuss the error objectively and if I refer to the LS at all, I do so in the third person. What else could I do to maximize the chances that the QT assigns the re-review to a different LS?
- A slightly tangential question, in reference to your comment above regarding possible misunderstandings due to language gaps within the Quality Team: given that a re-review is an internal communication within Gengo, susceptible itself of review and examination, I always make the effort of writing my re-review requests in English, because official communication on Gengo happens in English and especially because the QT needs to first approve and "validate the details of my request". But it seems that LS mostly do their reviews and re-reviews in Italian. Since LS themselves need to undergo review and quality control, should they be encouraged to communicate in English, for the sake of greater transparency?
Thank you for your time and have a great weekend!
I would also love to hear some answers concerning the points that marcodnd has raised.
The last point, especially, is something that I am wondering about as well.
I think that the general rule should be that if the re-review request form is written in English, then the Language Specialist should answer in English as well, for the sake of greater transparency and also as a matter of common courtesy. In fact, that is already the case with communications between translators and customers or Gengo's support team, so I do not see why that is not the rule with communications between Language Specialists and translators.
It should not be too much of a problem, since according to Gengo's hiring guidelines, all the Language Specialists are fluent in English in addition to their respective languages:
http://careers.gengo.com/apply/Nrg0nN/Language-Specialist-Certain-Language-Pairs
Hi @marcodnd,
Thanks for your comment :) To address your questions:
I think I may have mentioned this in a different thread within the last few months, but I've lost track of where, so for the sake of clarity: the standard preference at the moment is to send the re-review to the same reviewer (unless the specifics of the request warrant for the Quality Team to deem it necessary to send it to a second reviewer, when available in that LP.) Please do note that this is to be decided at the discretion of our Quality Team and that the decision for this process has been made based on large amounts of data regarding the re-review requests received over the last few years.
As per the above, the standard process is, for most cases, for the request to go to the original LS. Our LSs are human, just like translators, and therefore we believe that, while they make mistakes, they can also correct them when they have the chance to review them once again and are provided with sources that may challenge their previous decision. If and when appropriate, many of our LSs actually rectify their reviews and make changes based on the feedback received from the translator. The Quality Team assigning a re-review request to a different LS depends, as I explained above, on each specific case, so I cannot provide any tips for this to happen.
The misunderstanding mentioned above was about the member of the staff misunderstanding the content of the re-review request when forwarding it, not about a language misunderstanding between the LS and the translator. Basically, the Quality Team will first and foremost serve as a filter (because, believe it or not, many re-review requests come worded in very rude language that we can't possibly forward to our LSs) and then also, based on the nature of the request, decide how to best route it through the process.
LSs are free to provide their feedback in the target language, since it is expected to be both their and the translator's native language, and therefore communication should be clearer, especially when it comes to specific nuances. While we don't really expect LSs to be rude or behave unprofessionally in their reviews, based solely on the fact that they'd hypothetically feel safe because they're using their target language and "we can't understand it", if any issue ever came up with their feedback, you are more than welcome to let us know by emailing our Support Team. If such a case came up, please rest assured that we would carefully look into their language, involving a third-party if necessary.
As for the requirement for LSs to be fluent in English (per our Careers page), this is for the purpose of communication with the Gengo Team.
Hope this helps :)
Lara
Lara, what you are telling us is that, at a certain point in time and without explicit announcement to us translators, Gengo has removed the right for translators to ask for a re-review.
I am sure you will disagree with this description of things, but as indicated by the confusion this has caused both in my case and the case of marcodnd, for most of us a "re-review" naturally means a separate review done by a different Language Specialist. Indeed, that was presumably the common understanding before this change of policy.
Also, I do not remember receiving a public announcement from Gengo regarding these changes (if I am mistaken, please correct me). Why was a change of policy of this magnitude not made more widely known to us translators? This seems to be exactly the sort of important information that should be announced to us through our dashboards, rather than news about cooking or archaic languages.
@KO -- We did make an announcement on the dashboard, and I believe also on the newsletter, when we updated our Support article on Re-review requests, which literally reads (and which doesn't contradict anything that I have explained so far, despite your own personal preferences):
After your submission has been processed, your job will undergo a second review by either the same LS or a different LS (for pairs with more than one LS.) Our Quality team will decide the best course of action based on the nature of your request.
Even in the past, we never promised that a re-review request meant that a second person would be looking at the job, and effectively, this was never even possible in all of our language pairs, to begin with, because some language pairs only have one LS (and this is no secret). And, while at some point the request form did include the option to choose the same reviewer or a second reviewer, it has been repeatedly explained that this form was severely misused (causing us to process re-reviews thrice more often than we should because translators chose the wrong option for their specific case). It is based on this extensive internal data that we deferred the decision to the Quality Team, with this not being equal, under any circumstances, to the right to request a re-review being taken away.
Lara, I do not mean to argue with you since I understand that you are only the messenger in this situation, but where does it state that in the article you cited that "the standard preference at the moment is to send the re-review to the same reviewer (unless the specifics of the request warrant for the Quality Team to deem it necessary to send it to a second reviewer, when available in that LP.)"?
@KO -- That is information that I am providing as a response to you and other translators' inquiries. It, however, does not contradict or negate the fact that:
After your submission has been processed, your job will undergo a second review by either the same LS or a different LS (for pairs with more than one LS.) Our Quality team will decide the best course of action based on the nature of your request.
Both statements mean the same exact thing: the Quality Team will decide the best course of action based on the nature of your request (whether it is to send the request to the original LS, or to another one). The statement could be twisted a number of different ways, but at no point we're hiding the fact that the Quality Team will make a decision regarding a request, and at no point have people's right to request a re-review has been taken away (in fact, they're being processed as we speak).
I clarify not for the sake of arguing with you, but rather because I'm mindful that, while participation may be quite low, many people read these forums (with 5,000+ unique views just last week) and therefore it's important that the information shared is clear.
Lara, my point is not that what you are saying is logically inconsistent with the Support Article. But as you have just explained, there was a significant change in Gengo's policy on this matter; before, "the request form did include the option to choose the same reviewer or a second reviewer", whereas now "the standard preference at the moment is to send the re-review to the same reviewer". What I am saying is that this important change of policy should have been explicitly flagged and widely communicated to the community at large, not mentioned somewhere on the forums where even you cannot recall.
The Support article was updated with the statement I am mentioning below at the same timing that changes were implemented to the request form, and announced on the dashboard and the newsletter :) We have also been repeatedly open on here to everyone that has asked regarding the fact that: the changes were made based on extensive internal data, and based on repeated misuse.
As always, @KO, while we do appreciate the feedback, we can't promise that changes will be made to the system (or that a previous status quo will be maintained) because of it. We take feedback in and analyze it across teams based on alignment with business vision and strategy, how it fits within our roadmap, and also how relevant it is within the community (in terms of impact, both qualitative and quantitative.) I think I've mentioned this in passing in some other thread recently too, but with a community of 30,000+ qualified translators, the amount of complaints regarding re-review requests on the forums are mostly edge cases (which we always make an effort to look into). These cases amount to less than 1% of our total qualified translators and, when put in numerical context, illustrate the fact that the system is largely working as intended for the purpose of scalability.
With all due respect, Lara, I don't really see how any of that is relevant to my previous point, but I guess there is no point pressing further.
I have to say that I have tried very, very hard to be as reasonable and patient as I possibly can throughout this whole ordeal, but frankly at this point I am left feeling empty and frustrated. I would like to emphasize that the reason that I (as well as the many others who have offered their feedback in other threads) have done so because we trust that the people at Gengo are on the whole well-meaning, reasonable people who are trying their best under a difficult situation to be responsive to our concerns. Otherwise, why spend so much time and energy trying to offer our constructive feedback? But this kind of exchange is very disheartening.
@KO — As repeatedly stated, the constructive feedback is being taken and passed along for consideration if/when appropriate by the relevant teams on an ongoing basis. I understand the disappointment of not being able to have it reflected in an immediate change, but what I said above remains relevant: changes and improvements to the system are always justified by feedback in combination with data. At the moment, the data is as I mentioned above. The feedback has been relayed to the team regardless to keep in mind, and to keep a track record of these discussions in case we see changes in the future that justify a different approach.
Hello Lara,
I'm sorry if I missed the previous post where you already answered some of my questions. Sometimes I feel we abuse of your patience on this forum :)
So thank you for your answer. Not exactly the answer I wanted to hear, but helpful nonetheless!
Knowing that my re-review will most likely go to the same specialist might help me word my requests in a manner that better reflects this type of communication. To be clear, I am *always* absolutely respectful (otherwise my request wouldn't even go through) but, for example, instead of saying:
"The LS has missed the client's glossary entry x and y...."
I would say:
"please consider that - while your suggestion is clearly correct in general - the client's glossary specifies a different translation for this term".
Because, while it is impossible to prove, I am certain that a number of errors in my last review that could have been retracted or simply demoted to suggestions have not been retracted simply because the LS was put in a position of self-defensiveness and perhaps stung pride. There are subtle rhetorical jabs, or conveniently missing of some of the most salient points I made in defense of my decisions, or deliberate misconstruing of my arguments that convince me that the LS was partly acting in pride, not 100% objectively. All these subtle signals would be impossible to detect for a non-Italian speaker, hence my question about (re)-reviews not being made in English.
But regardless of my case, I remain firm in my position that, in an ideal world, a re-review should be sent to a different reviewer. However, I do understand that a company makes decisions based on multiple factors that we have no access to. In that regard, I have a very simple question. Perhaps it is an indiscreet question, so I understand if you won't answer. First of all, I am assuming that LS are gig workers, not employees, so they are paid by job, similar to translators. I also assume that if a re-review is sent to a different LS, they will obviously have to be compensated independently from the first reviewer. So, my question:
- If I am a language specialist and I am assigned to re-review one of my own reviews, do I get extra compensation, or is that considered part of the same job? (For comparison, I know that, as a translator, when a client asks for revisions, I do not get any extra compensation - even when the revisions are due to a client's mistake.)
I hope this doesn't sound like a contentious question, I am simply trying to understand the company's point of view.
Another quick question: In case I have concerns over the performance of a LS, what would be the best channel of communication to express my concern? Is the Quality Team reacheable directly by the translator? If I contact support with such concerns, would I be forwarded to someone in the QT who supervises the work of LS?
Thank you again!
Hi @marcodnd,
I fully understand your position, and it's definitely a valid opinion to have. I also appreciate your understanding that any decision that affects a system designed to scale and leverage technology to manage a large community like ours needs to take into account multiple factors before changes can be made (or for changes to be made at all). There's a lot of value in understanding this, whether one agrees or not with the current system, because it helps put feedback in perspective :)
With that said, on to your questions:
Just like a translator doesn't receive extra compensation when they need to revise their work, LSs don't either.
The Quality Team is not reachable directly, but if you email our Support Team, they will be able to forward your concerns to the team, and also relay back any relevant information, when applicable. Please note that, while all feedback is shared with the team, and it may trigger internal investigations, trainings, etc, it's not always the case that this internal information will be shared with the translator. What I can assure you of is that all feedback sent in through Support reaches the whole of our Operations Team (which includes the Quality Team) :)
Thanks,
Lara
OK Lara, just to sum up, I feel that at the very least the language used in the Re-review request support article doesn't exactly reflect the reality you have described here and it might lead again to misunderstanding in the future. In particular:
1. "your job will undergo a second review by either the same LS or a different LS"
The formula "either/or" seems to imply that both solutions (same LS vs. different LS) are on equal terms, or at least it doesn't reflect the reality that one of the two solutions is mostly preferred to the other, by default. Perhaps this could be made more explicit, for greater clarity? For ex., "your job will undergo a second review by the same LS (or, in some select cases, by a different LS)".
2. "Our Quality team will decide the best course of action based on the nature of your request."
I did ask in my first comment about how exactly I could formulate my request in such a way that the QT might be inclined to assign the re-review to a different LS but there doesn't seem to be a clear answer to that. This part, again, is at least potentially misleading because it suggests that I, the translator, can *do something* to point the QT towards one of the two possible courses of action. Maybe the "based on the nature of your request" part could simply be removed?
I'm not saying that these statements are "false", but they do seem to be misleading based on what you told us here. As you said above, they can be twisted in different ways. So why not make them so clear that they *cannot* be twisted in any way?
Finally, about this bit of information you provided above:
"Just like a translator doesn't receive extra compensation when they need to revise their work, LSs don't either."
Yes, and I feel dumb for not realizing sooner that there are extra costs associated with assigning the re-review to a different specialist. This detail might be obvious to a lot of people but evidently not to everyone. And this is the one detail that might have saved a lot of discussion here, because I'm sure most people will understand that it is futile to argue against a company's budgeting and cost concerns. Perhaps even this bit of information could somehow be added in the article, for maximum transparency?
Please note, I understand that I shouldn't expect any of my suggestions to be taken up in any way. I am just saying that, in my case, had the communication been the way I described above, it might have saved us a lot of this discussion (not that there was anything wrong with it!).
Thank you again,
Marco
Hi Marco :)
Your feedback is much appreciated, but it seems to be based on a fundamental misunderstanding of what “based on the nature of your request” means. There’s no answer to how you can formulate your request to ensure it gets sent to your LS of choice because this is indeed based on the *nature* of the request, where “the nature of the request” does not mean “your preference indicated in the request” but “the types of inquiries, and claims (and everything in between) in your request”, which are looked at on a case by case basis, in order to make the decision, as explained, to send the request to the original LS or a different one.
As for LS payment details, they’re not relevant to an article on what re-review requests are or how to submit one, simply because these details are not the sole (or even a major) factor that has shaped the current process. As explained in this thread, too, this has had more to do with trends observed in our re-review request data for the last few years. It’s an anecdotal answer that I gave you because you explicitly asked about the compensation scheme, so the response was objectively in that context, and not to be misconstrued to mean that’s the reason why we send a re-review request to the same LS when we deem it appropriate.
Thank you for your understanding,
Lara
Well, yes, the fact that I misunderstood what "the nature of your request" means was exactly my point: that the language used there is susceptible of misunderstanding, especially given how in the past it *was* possible to express a preference about same vs. different LS. I am simply suggesting to update the language, not the policies behind the language.
I understand your point about the payment details and why you woulnd't want to add this to the support page. However I can assure you that, for me, knowing now how this payment scheme works is absolutely relevant and useful, and might significantly help me better understand the interpersonal dynamics going on in a re-review process.
Anyway, I feel we exhausted our points of conversation here and that I might be abusing of your patience, so I will stop here. This was honestly useful, thank you :)
Marco
Glad to hear you found it useful :)
Have a great rest of your week, Marco!