In this post, I would like to point out several recurring issues I have noticed with regard to a review I recently received for Job 74874742.
At the outset, I would like to emphasize that I have great respect for Gengo's Language Specialists and have certainly learned a lot about the art of translation from the feedback I have received over the years. I am not writing this in order to target the competence or integrity of a specific reviewer, but rather to raise some questions regarding the standards and expectations that Gengo as a company sets for its translators, and how these expectations are communicated to the Language Specialists when they are onboarded and trained.
I hope that the following feedback will be helpful in further improving Gengo's review process. I would also appreciate it very much if other translators could share their own experiences on the matter.
(1)
First and foremost, I would like to point out that oftentimes, the only justification that the reviewer cites for marking a given translation as an error is vague and subjective.
I understand that while this is an issue that has repeatedly been brought up in these forums in the past, Gengo has said and done very little to address it, and so it may seem a bit like I am flogging a dead horse.
Nevertheless, I would like to unpack a little bit more precisely why this practice is damaging in being both unfair and unhelpful, with the hopes that my feedback will work towards encouraging Gengo to finally implement the necessary improvements in its review system.
For example, in the review in question, the reviewer issued a "Medium" semantic error for translating the word "nation" as 国家 instead of 国. The (sole) reason he/she/they has given for this is that 国家 is "大げさ", or literally, "grandiose" or "a bit much" (http://www.romajidesu.com/dictionary/meaning-of-%E5%A4%A7%E3%81%92%E3%81%95.html).
In another place, the reviewer marked the translation of "ancient" as 古色蒼然たる, rather than the reviewer's favored 旧式の, as a "Low" semantic error. The only reason cited is that "そこまでは言っていません" or "that is going too far".
Now I realize that translation is often more an art than a science and that sometimes in such matters it is necessary to defer to the judgment of an experienced translator. However, I have at least two issues with vaguely worded justifications of this kind:
-
For one thing, such kinds of comments are unfair, in that they are difficult to validate and contest on objective grounds. I hope that Gengo does not disagree that as much as possible, reviewers should give objective grounds for why a given translation is an error, by pointing out concretely why it either distorts the original meaning or is ungrammatical, in line with the criteria outlined in: https://support.gengo.com/hc/en-us/articles/115002394227-Job-Reviews-GoCheck-Error-Types.
-
For another, such comments are vague and therefore unhelpful. Gengo claims that one of the objectives of the review process is to "provide constructive feedback to the translators that helps them grow and improve" (as it is stated in the above article). But since each person will have different ideas of what counts as "grandiose", or "a bit much", for instance, it is difficult for us translators to figure out how exactly to apply this sort of feedback in our future work at Gengo.
I should point out that Gengo states that it expects Language Specialists to "be objective and fair in their reviews" (https://support.gengo.com/hc/en-us/articles/231441587-What-is-a-Language-Specialist-and-how-does-Gengo-manage-the-LS-team). It would appear to me for the reasons stated above that feedback of this sort is neither.
(2)
Second, I have noticed multiple instances where the reviewer in question does not seem to be behaving in accordance with the criteria Gengo provides to us translators.
For instance, in the example above, the reviewer has marked my translation of "nation" as 国家 as a "Medium" error. However, the Support Article defines a Medium error as "A noticeable error, including misrepresentation of the source or a clearly ungrammatical rendering of the target". It is not clear to me exactly why (even if we grant for the sake of argument that the choice of 国家 is somehow overly "grandiose" in the context) the mere fact that my original rendering is "grandiose" puts it on a par in terms of seriousness with other errors that have a clear impact on the accuracy and/or usability of a translation, such as "misrepresentation of the source" or "clearly ungrammatical rendering[s] of the target".
In fact, I would submit that, if this is indeed an error, then it is a good example of a "Low" semantic error, which Gengo defines as one that "does not significantly change the meaning of the translation".
As another example, the reviewer has marked a minor error concerning the same word that occurs several times as "Medium", writing that 頻出しますのでmediumとさせていただきます。 (= I have marked it as medium because it occurs "frequently").
Now, for one thing, the Support Article cited earlier does not say that multiple instances of the same error will count towards increasing the severity of that error, so I do not see what grounds the reviewer is relying on in making this judgment.
For another, this appears to be clearly contrary to the expectations that Gengo has communicated to us translators in the past: https://support.gengo.com/hc/en-us/community/posts/360014633413-Multiple-counting-for-the-same-error.
(3)
Third, and perhaps most worryingly from my perspective, I would like to point out the reviewer's application of the criteria is inconsistent, even within the same review.
For instance, in the above examples, the reviewer has marked my "grandiose" translation of "nation" as a "Medium" error, while my translation of "ancient", which is supposedly "going too far", was marked as a "Low" error. However, it is unclear to me just what the difference is supposed to be between a "grandiose" translation and a translation that is "going too far", and more importantly, why this difference would suffice to objectively justify the drastically different judgments of severity between the two cases.
Moreover, it is unclear to me how the reviewer's fluctuating judgments here relate to the criteria Gengo has outlined for us translators in https://support.gengo.com/hc/en-us/articles/115002394227-Job-Reviews-GoCheck-Error-Types.
It should be clear by now that the second and third points both circle back to the first. If the reviewer had been required to offer more clear and objective reasons, providing accessible supporting evidence and explicitly referencing the criteria set out by Gengo, for why a given error falls under a given type or category of severity, then this kind of seemingly arbitrary application of the criteria would be less likely to occur.
Indeed, since the Gengo Quality Control Team requires us translators to provide "detailed" evidence and "accessible references" in submitting the re-review request, I would respectfully ask that Gengo do us the bare human courtesy of applying the same standard to the reviewers themselves
48件のコメント
Is it possible that some reviewers are not entirely clear what the reviewing process should do? I feel that there's potential for a bit of conflict between the two goals that Gengo sets for reviews:
1. assess the quality of translations
2. provide constructive feedback to the translators that helps them grow and improve
Assessing the quality of translations is something that should leave no space for subjectivity or personal preference. I feel there must be ways to define a mistake in a relatively objective and rigorous manner, and reviewers should comply to that, strictly. They need to be acutely aware that their evaluation affects a translator's score, which in turn might determine their ability to keep working on gengo in the future.
When it comes to the second goal, providing feedback, this is were subjectivity comes into play. If your goal is to provide feedback, you absolutely can rely on your personal preferences and opinion: there's always a way to improve a translation even when it has no objective mistakes. But! This should not have an impact on my scorecard.
And the thing is, reviewers do have a Suggestion option available, which does not deduct any points. But somehow I practically never see them on my reviewed jobs. If one major goal of reviews is to provide constructive feedback, why do I never see suggestions? I would love it if every single one of my jobs received a detailed feedback with multiple suggestions! But somehow it's either no suggestions at all (which implies the translation is 'perfect' and un-improvable?) or objective mistakes, with nothing in between.
Thank you for your input, marcodnd. I definitely agree with the points you make.
Lately, I have come to think that any given translation can be roughly classified into the following three categories:
(A) Translations that are clearly an error, either because they misrepresent the meaning of the source text or are otherwise ungrammatical
(B) Translations that do not meet the reviewer's idea of the perfect translation, but are not objectively errors either
(C) The reviewer's idea of the perfect translation (i.e., how they personally would have translated the passage themselves if they were the translator)
Quite often, as in the case of my original post, what I see is the reviewer penalizing a translation because it falls under (B) rather than (C) (= that is, for not conforming to what is the "ideal" translation, in the reviewer's own personal opinion), when actually what they should be focusing on is clearly identifying which renderings fall under (A) as opposed to (B), based on objective evidence and criteria.
Of course, it may also be helpful for us translators to have some feedback with regard to (C); but as you point out, that is presumably what the Suggestion feature is for.
Hi KO, I'm a translator in the JA→EN language pair, which is the reverse of yours.
First of all, for what it's worth--and I know that my opinion is nowhere as important as that of an LS--I really don't see any issues with your translations of 国家 and 古色蒼然たる, much less errors that are serious enough to warrant a 'medium' error classification (at least in the absence of any further context).
Like what you've mentioned, I've always believed that an "error" should mean precisely what it means, i.e., a rendering of the source text that is not conceivably acceptable. Accordingly, a translation that renders the source text in a manner that is not to the satisfaction of the LS or which does not conform to their personal preferences, but which is otherwise not technically incorrect, should not be marked as an error. Even the "low" error classification ought to be bounded by this requirement, since the "low" is supposed to refer to the impact on or extent of distortion to the source text, and not how strongly the LS happens to feel that the said translation is an error. A textbook example of a "low" error would be inconsistent subject/verb agreement like "Oranges is"--a construction that is not conceivably acceptable under existing grammatical rules and has a small impact on the overall translation.
Since translation is as much a creative enterprise as it is an analytical one, it would be nice if Gengo can do more to encourage LS's to be more even-handed and give more leeway to translators to develop their own style and become better at the craft, something I'm sure Gengo's customers will appreciate as well. In my view, the only way this can happen is if the review process is governed by an entirely rules-based system that is insulated against the whims of subjective preferences and opinions by focusing exclusively on identifying errors that cannot be disputed, which are then classified as "low," "medium," or "high" according to their impact on the translation.
I happen to think that the "Suggestions" feature is a fantastic introduction, but as others have already mentioned, it would be nice if this feature can be utilized more frequently as a means of sharing the experience of the LS's with other translators. (In my case, the only time the Suggestions feature was ever used was when I had successfully appealed an "error" and justified why it should be downgraded to a "suggestion" instead.)
One possible way of establishing such a rules-based system is for a translation to count as an error only after it has been independently marked as such by at least two LS's without access to each other's reviews. From the standpoint of consistency and fairness, if only one out of two LS's regards a certain passage as an error, it is difficult for the translator to accept that they should be marked down and threatened with the potential loss of qualifications for it. (After all, you would think that any error egregious enough to merit a "medium" or "critical" classification would be immediately apparent to all vastly experienced LS's.) Obviously this might create more work for LS's and slow down the review process, but I believe that such a reform is certainly worth it if it makes the review process and the work of the LS's more credible, gives assurance to the translators that they are being assessed in a fair and transparent manner, and saves the community managers the exasperation of having to deal with this perennial issue on the forums every two days.
Thank you, aardhiroki.
I kindly appreciate your input with regards to my translations; but, at this point, I am not even all that concerned with the legitimacy of the score I have received in this specific case. It is more that I feel frustrated and discouraged by (1) the overall lack of reasonable explanation, (2) opaque standards and expectations, and (3) inconsistent enforcement of the rules which currently pervade the review process as a whole.
The points you make are very interesting and strike me as exactly right. I especially like how you point out the creative aspect of translation as an enterprise. I think translating well is not merely a matter of applying a fixed and definite criterion that you can learn in advance to identify the single correct answer (although, of course, there is an element of that too; insofar as there are certain more or less well-established rules of usage and grammar that any objectively acceptable translation will need to follow). Rather, beyond a certain point, coming up with a good translation is a creative and open-ended exercise in which there are multiple and sometimes conflicting standards for what counts as a "good" translation that a translator will need to balance, and in which sometimes there is no obviously correct answer with regard to how this is done.
I also think the point you raise about the importance of ensuring fairness by having reviews conducted by multiple independent Language Specialists is spot-on. That is exactly why I have repeatedly stressed that the re-review process, at the very least, should in principle be conducted by a separate and independent reviewer, rather than being something that has to be negotiated by the translator on a case-by-case basis (although apparently, Gengo disagrees):
https://support.gengo.com/hc/en-us/community/posts/360038160474-Re-review-sent-to-the-same-reviewer-
https://support.gengo.com/hc/en-us/community/posts/1500000202282-Re-review-still-being-sent-to-the-same-reviewer
Indeed, there is surely a reason why it is virtually a universally accepted practice in the field of academic publishing across the world (another industry that, much like translation, relies heavily on the experience and good judgment of experts to ensure quality) to have reports of new findings peer-reviewed by at least two (and often more!) experts before it is recommended for publication or rejection, in order to approximate a fair and objective outcome.
In the end, I think that what we all want to see is just slightly more humility on the part of the reviewers themselves; a recognition that translation can be a messy business, and that sometimes one's judgment regarding what counts as an error or not is not a clear-cut matter, but rather can depend heavily on one's own personal preference and opinion, over which reasonable persons can disagree.
aardhiroki, thank you for your comment, it expresses exactly how I feel, I just would have found it difficult to find the right words. I just had a another disheartening review. Not really a bad one, but my score went down two decimal points. I'm so proud of myself that I seem to have managed to eliminate the typos that sometimes crept into my translations when I started out translating for Gengo a year ago. But some LS seem to be determined to find something anyway. If I repeat a word in a sentence, even in the form of a compound noun and after careful consideration, the LS might consider it an "unnecessary" repetition. If I decide not to include a half-sentence in my translation, because it really is an unnecessary repetition of something that has already been said in the source sentence (perhaps for emphasis, but in a way, that doesn't work in German), the LS might consider it an omission. In the beginning, I also found the reviews instructive. But this has not been the case for a long time now, most of the time, I'm just confused and becoming more and more careful in a negative way, i.e. rejecting translations that I might have tackled before and being less creative in finding the right expression for the target text.
I have some input about this issue. When I make a re-review request, even when my opinions are corroborated by native English speakers, the reviewers still tend to stick to the original reviews. In fact, it's rare to get a different score after a re-review now. And for the very same mistake, the reviewers may designate it as a minor error this time and a medium error next time.
My score is consistently above 9.0 in 2018 and 2019 and I think the quality of my translation has been improving over time, but my score has in fact declined to lower than 8.0 now. I agreed with the review results on most occasions before 2020, but since 2020, I can only agree with less than half of the reviews. I become reluctant to file a re-review request now because the results are highly unlikely to change. And the reviews indeed can go wrong and sometimes the opinions from the reviewers are even rejected by the customers. For example, the following is an instruction from a customer.
For people's names, please don’t translate only if they are super famous and have a well-known Chinese Name. You may search on https://baike.baidu.com/ (a website similar to Wikipedia) to find their name in Chinese and use the format of e.g. 大卫·贝克汉姆(David Beckham).
In a translation, I didn't translate a name that is not familiar to most Chinese people, and the reviewer marked it as an error. I requested a re-review, and the reviewer explained that the name should be translated because it appears on https://baike.baidu.com/. Then I followed the reviewer's instruction in subsequent translations. Later in a revision request, however, the customer made it clear that the names should be translated only if they are super famous, not if they have an entry on https://baike.baidu.com/.
I really hope this issue will be considered seriously by the leadership. One of my suggestions is that the reviewer team should include at least one source language specialist because many disagreements arise around the understanding of the original text. And I don't think the target language specialists are competent enough to deal with this issue by themselves. Another suggestion is that a re-review should be done by a different reviewer. The insights from another reviewer may help settle a disagreement.
I have been working for Gengo for over 5years in EN/JP pair and I thought I should share my experience.
Hi tamura_w,
I think the ID is simply shown if the reviewer leaves an additional comment for the whole review. This seems to use the default comment system. While some might avoid leaving a comment because of that, it's probably coincidence in most cases.
I agree that showing the ID by default (via the system) is a good idea, though.
Hello all. I think very much of these issues could be avoided if you use the comment field more actively to ask the customer for clarification underway, or, if there is not enough time for that, by posting comments like e.g. "For the expression "xxxxx" I have assumed "yyyyyy", however this can also be viewed in a different way (describe ...), and in that case I would use "zzzzz". This way the customer has to make a choice, and the LS cannot penalise you, because they are obliged to take the comments into consideration. (Unless the LS thinks that both "yyyyyy" and "zzzzz" ar wrong ...) In my experience most customers appreciate receiving clarification questions. Some customer change the word by themselves, and others send the job back to you for making the change if they so prefer. Gunnar (EN-NO)
Hi Chris,
It might be a coincidence as you suggest, but sometimes it's confusing. If it doesn't bother anyone, I think this should be changed.
I have already asked whether someone from the Gengo Quality Control Team could perhaps clarify Gengo's official stance on these issues several days ago, but while we all wait for the response, let me share another experience that I think might be pertinent to some of the points that cnaky and tamura_w have raised.
A while ago (due to circumstances chronicled here: https://support.gengo.com/hc/en-us/community/posts/1500000202282-Re-review-still-being-sent-to-the-same-reviewer) I had the experience of receiving two separate re-reviews for the same job, one by the reviewer who had been in charge of the original review and then another by a different translator (Job # 72421647).
In my re-reviewed request, I pointed out six different passages where I believed that the error in question should either be retracted or modified in severity. Here were the results of the respective reviews:
- Re-review 1 (by the same reviewer): Retracted one minor error and gave a long-winded and hostile rationalization of why the remaining five were still errors.
- Re-review 2 (by a different reviewer): Retracted the remaining five errors (in effect, retracting all six errors), on the grounds that for all the cases at issue, "either translation is acceptable" (= 訳語としてどちらでも良いと判断しました)
Now, I have two thoughts on this experience:
(1) Regarding the consistency of Gengo's reviews
If I had not gone out of my way to appeal Gengo's judgment to have my review sent to a different reviewer rather than the same reviewer, I would have been penalized for all five errors that Gengo ultimately acknowledged were incorrectly marked *and upheld by the first reviewer* (which in my case translated to more than a 1.0 difference in the job score and a +- 0.2 difference in Consistency Score).
Now I ask; how on earth is a system in which this kind of stark inconsistency between reviewers' judgments routinely occurs "objective" or "fair"?
Gengo makes a great deal of the importance of ensuring "Consistency" among the translators. But it seems to me that it is not spending nearly as much effort as it should in ensuring consistency among its reviewers.
(2) Regarding the oversight by Gengo's Quality Control Team
It has come to my attention through the above saga that the staff at the Gengo Quality Control Team uses Google Translate to check the re-review requests that are sent to them by translators (I could note the inherent irony of this practice in a company like Gengo, but I will not go there).
This suggests to me that those in the Quality Control Team are not necessarily conversant in the target language; in my case, Japanese. If so, I think this is highly problematic, for obvious reasons:
(i) For one thing, Lara has informed me that Gengo leaves "it at the discretion of the Quality Team to carefully read through the requests and redirect them to the first or second reviewer based on the actual comments and claims left by the translator." But how on earth can the Quality Team "carefully read through" the requests when they do not even understand the language?
(ii) For another thing, if the above is indeed the case, then it makes me highly doubtful of the Quality Team's ability to exercise effective oversight of the reviewers, since most of the reviewer's feedback and the back and forth between translators occurs in the target language. If the Team needs to resort to machine translation to understand what is being said, how can they make a fair and objective assessment with regard to the correctness of the claims of the respective parties?
Indeed, I have come to suspect that this lack of transparency and oversight may be part of why the issue of problematic reviewers has been so difficult to resolve. That is part of why I think it might be good for us translators and also Gengo as a whole to air these issues with each other publicly, such as on these forums.
gunnarbu, communicating with the customer can be very important, no doubt about that. I try to do it moderately, only when I feel it is necessary and promising. But in my experience, if I feel sure about what I'm doing, asking or explaining to the customer might come across as insecure - and it probably is.
I also think that once I start thinking only about was an LS might consider nice as a translation and taking translation decisions only to make sure an LS will not rate me down, I should probably consider finding other clients to replace Gengo.
In my opinion, the way Gengo works has many advantages and is sort of genius. But the way reviews are done in this system is one of its drawbacks. I'm not sure there is a way for Gengo to make reviews more fair and beneficial, but I definitely hope so.
Hi KO,
First and foremost, please allow me to apologize for the delay in addressing this thread. As I'm sure you're aware, and as I understand our Support Team has also communicated to you, we've had quite a long string of national holidays in Japan due to Golden Week. I personally was out of office since April 29th, and part of our Quality Team will continue to be out of office until early next week. Please do note that while our platform continues to run 24/7 throughout the year, teams in the back office do have weekends and national holidays off, as well as their own vacations, so sometimes we cannot get back to you immediately. We do, however, always make an effort to catch up and be back at full speed as soon as possible when we're back at work :)
This thread has grown quite a bit, and I'll be going through specific feedback later in the day to pass along to our Quality Team so they can review once they're back, however, I wanted to take a moment to address the 3 main points in the original post, if that's okay. Separate from this, I would like to ask that you please send me via email (lara.fernandez@lionbridge.com) the job number for the job whose examples you've used here, as well as the job you mention in a separate thread about political beliefs. I would like to take a closer look at both to push for further re-review if necessary.
1) First and foremost, I would like to point out that oftentimes, the only justification that the reviewer cites for marking a given translation as an error is vague and subjective.
Thanks for bringing this up. What I have often seen happen in cases like this is rather poor communication from the LSs. This is something that, as Experience Lead, I am very concerned about, and which I have repeatedly brought up internally with our Quality Team so they can encourage the LSs to leave better feedback and proper explanations for their decisions. I am sure that, if they did that, some decisions that may seem vague would show in a different light (at least, we would understand the logic behind them) and it would be easier to contest or objectively discuss them.
From the examples that you bring up, one thought that came to my mind was that 旧式 and 古色蒼然たる, while similar, are not fully interchangeable. While 旧式 states an objective fact of something being "old, or ancient", 古色蒼然たる adds color and paints a picture of something actually looking extremely ancient or run down (ひどく古びたさま。いかにも古めかしいさま。また、古めかしく趣のあるさま。). As translators, I think we all can agree that while some words may have similar meanings, they can't always be used interchangeably, and that context will weigh heavily on which one is more appropriate. Please do note that I am not defending the comment of the reviewer -- I am simply trying to somehow relate to what they might have meant by "そこまでは言っていません" or "that is going too far". Whether the reviewer is right or not likely depends on the source text, so I won't go into that here (that's something I want to look at when I receive the job number), but what I would surely like to emphasize is that I do agree with you that that piece of feedback isn't really feedback. Even in the event that the LS was right and that their thinking process was the one I just described above, I think they should have been able to articulate in a better way than "that is going too far."
As I said, I personally find this highly concerning. Our Quality Team has let me know in the past that they don't require LSs to go into lengthy explanations for the errors marked, but I do believe that it's important that when they do provide explanations, these are actually proper explanations, so I will be sure to speak to them on this once again.
2) Second, I have noticed multiple instances where the reviewer in question does not seem to be behaving in accordance with the criteria Gengo provides to us translators.
This is another one where we'll need to look at the source text in order to fully understand why the errors were marked with a certain severity (or why this severity may be an error in and of itself.)
One thing I would like to point out here is that, while the Support Article you mention does not say anything about "repeated errors" in a text (in fact, the Support Article is not intended to be an extensive explanation of the LS reference and training materials, and can't be taken as such), it is actually the case that our LSs are instructed to mark repeated errors only once and raise their severity, rather than marking them several times as independent errors within the text. In this regard, the LS was following protocol.
3)Third, and perhaps most worryingly from my perspective, I would like to point out the reviewer's application of the criteria is inconsistent, even within the same review.
This one, in a way, is related to point 2, where we will need to look at both the source text and the translation, as well as the errors marked and their severities, to assess whether they've been actually marked according to Gengo standards or whether there are indeed any inconsistencies. Again, I'm looking forward to receiving the job numbers to take a closer look, and please do know that if inconsistencies are found, I will make sure that this is addressed by the Quality Team.
Last but not least, I wanted to quickly address your comment regarding the Quality Team using GT to interpret the review requests received. The Quality Team is a 4 person team who manage our LSs and executes all processes around quality. Hence, their role is not to assess the quality itself but to keep the system running smoothly. Since it is not realistic to expect 4 people to speak all source and target languages that we offer, they may use GT when needed in order to discern the types of questions that are being sent, not their specific details -- because, as you state, that wouldn't make sense, and that's for our LSs to look at. In an ideal world, perhaps we wouldn't need the QT to step in at this stage, and perhaps the re-review requests could be sent directly to the LSs. However, due to the abusive nature of many of the re-review requests that we receive, this is certainly not possible at the moment, and we need a human filter in place, so to speak. The Gengo model is built to scale, and differs from a traditional translation agency in many aspects. Hope this clarifies a bit how this works :)
As I said, I will go through the feedback in this topic to pass on to the team -- so please do keep it coming :) As always, I cannot promise any immediate changes (or any changes, at all) since these depend not only on feedback, but also on how feasible certain suggestions are, how they fit within the business vision or our roadmap (if they do) and, last but nonetheless extremely important, actual resources that we can dedicate to the development of features to address said feedback. It is still important for us to hear it, so we can log it and refer back to it when we have the bandwidth to explore improvements.
Thanks,
Lara
Thank you Lara. Your comments are definitely helpful and give me a lot to think about.
In the meanwhile, all issues raised in the following post relate to Job # 74874742.
https://support.gengo.com/hc/en-us/community/posts/1500000863621-Yet-Another-Post-on-the-Subjectivity-of-Gengo-s-Reviews
All issues raised in the following post relate to Job # 74614777.
https://support.gengo.com/hc/en-us/community/posts/1500000893341-Reviews-and-Political-Beliefs
Thanks, KO! :)
I will take a look at both jobs in-depth as I prepare to speak with our Quality Team about them.
As I mentioned here, and in the other thread, a big part of the team is still on holidays, so please do not expect any movement on this until next week. I will definitely make a note to bring it up with them as soon as they're back in office.
While I have your attention, Lara, I would also like to request a response regarding the following points:
(1)
As I already communicated to the Quality Team, the current re-review request form is unreasonably limiting.
(i) First, the current form only allows the translator to list 6 complaints. This limitation may have been understandable in the past, when the translators had the option to explicitly choose the option for a full re-review by a different reviewer (and so only had to cite enough evidence to justify the job being sent for re-review), but under the current system it is not reasonable or fair, since sometimes the job gets sent back to the same reviewer, who will only look at the complaints explicitly raised in the review form.
(ii) Second, the current form's 1000 character limit is also unreasonably confining, especially as Gengo will sometimes reject re-review requests on grounds that they do not provide detailed explanations and links. Since links, especially, eat up the character count quite a bit, it is sometimes not possible for the translator to do both within the specified word count. Perhaps this problem could be addressed by a separate box where the translators can list their links and references.
(2)
I only bring this up because I know that you are fluent in Japanese, but I would like to point out that I am noticing a lot of issues with the tone that some of these Japanese reviewers are employing in their comments. This includes the subjective and one-sided way in which this "feedback" is often communicated, as pointed out in my original post, but it also includes the use of overly casual and cozy expressions ( 「～ですね。」、「～では。」、「～してみては。」) that may be appropriate in the context of a teacher addressing his students in a classroom, but arguably not so much for associates addressing each other in a formal environment. At times, it frankly comes off as somewhat belittling and patronizing (or, as we say in Japanese, 上から目線).
I am not saying that translators should be treated with kid gloves, but I would like to remind Gengo that we are not students or subordinates, either (if anything, we are users of a platform that Gengo provides, as are our customers), and I would really appreciate it if we could be treated with a little bit more respect.
Hi @KO,
Thanks for this! To be honest, at this stage, rather than a response in terms of whether your suggestions can or will be implemented or not, what I can tell you is that I will be sure to pass this feedback along as well next week :)
(1) (i) Noted :) and (ii) Noted as well. While this one makes sense, and I understand what you mean about the links, in the meanwhile, you may also want to consider using free link shortening services such as bitly and the like to save a significant amount of characters.
(2) I think this one ties back with our discussion above regarding LS feedback and instances of poor communication, and I agree with you that the tone used should be friendly, helpful and professional (with friendly not necessarily meaning things can get overly casual, if that makes sense.)
Have a great weekend,
Lara
It seems all I can do at this point is to wait for my case to be heard by the Quality Team. However, as I mentioned at the outset, my original intention in writing this post was not to harp on my personal grievances but rather to start a public discussion with the community on how to make things better. So here are some brief thoughts in response to Lara's reply to my original post, as well as to some of the interesting and important points that others have raised in this thread.
(1)
I am becoming increasingly frustrated at how these issues concerning problematic reviews and reviewers often do not get addressed or resolved by Gengo unless the translator goes out of their way to have something done about them (as was the case with my experience with the reviewer who stubbornly refused to retract their original score, detailed above).
In effect, this is Gengo pushing the burden of identifying and rooting out problematic behavior by reviewers onto the translators themselves. We translators should not have to be sitting here expending time and energy writing paragraphs fighting tooth and nail to have our issues fairly heard -- essentially doing the work of the Quality Control Team for them.
Either (i) Gengo should more closely oversee the activities of individual reviewers, or (ii) they should set up the system in such a way that ensures that a fair and objective outcome is reached in each instance without the need for constant oversight; for example (as multiple other people including aardhiroki, cnaky, and tamura_w have already suggested in this thread, quite reasonably so), by having two or more reviewers independently evaluate every job that comes under review.
It really is that simple.
(2)
I happen to have some experience giving oral and written feedback on college essays and papers on a daily basis in my other career, and one thing that I have learned is that the ability to produce good work oneself, on the one hand, and the ability to identify and explain issues with the work of others in a succinct and accurate way and to give constructive and fair feedback, on the other, are two entirely different skills.
Of course, the former ability is crucial, but as a reviewer, the latter ability should be just as important as well. That is presumably why these individuals merit the respect and responsibilities that come with the title "Language Specialist", rather than "Senior/Veteran Translator" or some such.
Yet the recurring issues with vaguely worded and subjective feedback as outlined above (which oftentimes essentially amount to: "This is wrong because it just feels wrong, or because I like my way better — end of story") makes me wonder just how much the latter ability is emphasized in hiring and training the Language Specialists at Gengo.
I would like to stress that many people who produce good work themselves do not automatically acquire the latter ability (if only!), which needs to be deliberately developed through experience and practice. I would really like to see this skill emphasized and rewarded more by those in charge at Gengo in the future.
Hi KO,
I would like to present my own real-world example here and I would love to hear opinions by my fellow translators.
Recently, a collection submitted by me was reviewed. I wasn't convinced by some of the points raised, so I filed a re-review request. It seems the same LS who originally reviewed the translation also did the re-review.
Although I already decided to stop spending time and energy on this, your recent comment made me change my mind.
Source text:
------
*These prices are in US Dollars (USD). Pricing in other currencies and countries may vary and actual charges may be converted to your local currency depending on your country of residence.
My translation:
*Diese Preise sind in US-Dollar (USD). Die Preise in anderen Währungen und Ländern können abweichen und in die lokale Währung umgerechnet sein.
------
For context, the customer came back to me with the following request:
"Hi, unfortunately, the translation is too long, can you please reduce it to 4000 characters?"
That's what I did. I went through the whole text, shortening whereever it was possible. I don't remember exactly if I shortened the above translation before or after the customer's request, but I think it was after that. The LS must have been aware of the customer comment, but I also wrote an email to the support team, because I forgot to mention the fact in the re-review request. The support team passed on my comments to the LS.
-----
Here's the LS' comment from the re-review:
------
„Local“ und „depending on your country of residence“ sind zwar semantisch ähnliche Begriffe jedoch keine Synonyme, da „local“ ortsbestimmend, „depending on your country of residence“ aber konditional ist. Aus diesem Grund darf Letzteres auch nicht ausgelassen werden.
------
For those who don't read German, she says something like "Local" and "depending on your country of residence" might be semantically similar, however, they are not synonyms, as "local" defines a place while "depending on your country of residence" is conditional. So the latter cannot be ommitted.
My own opinion, which I cannot let the reviewer know directly, is that it doesn't matter. I'm quite sure that a literal translation on "depending on your country of residence" is unnessary, as "lokal" says it all in a pithy way, also bearing in mind that the customer specifically asked for a shorter translation. I omitted the half-sentence on purpose, it was a conscious decision because I consider my translation much more elegant that way, and as currency is also mentioned, there is no room for misunderstanding.
If you want to be pedantic, the LS may be right. But as I said, I was asked to shorten the translation, and as I always try to produce a pithy translation, there was not much room for maneuvre left for making it even shorter.
I would also like to mention, that while we are being punished for using a word twice ("unnecessary repetition"), it looks like we are being asked to translate each and every unnessary repetition. Except that when we do so, the next LS is going to comment, that the translation is "too literal".
Thank you for that, Heike S.
As I have also repeatedly brought up on these forums, in my opinion the practice of sending re-reviews to the original reviewer simply does not make sense and is indefensible from any point of view (except possibly to cut back on costs).
It is truly baffling to me how anyone with a basic appreciation of how actual human beings and incentives work, as well as of the unavoidable need for subjective interpretation and choice that is inherent in translation in general, could think that it would make any kind of sense to have a given translation be sent back to the same reviewer in the case of a dispute, rather than a different one.
The problem isn’t even necessarily the obvious conflict of interest; although it is that too. It could simply be that different translators can genuinely disagree in good faith about the validity of a given choice, as it appears to be what happened in your example, in which case the only thing left to do is then to appeal to the judgment of an independent third party (or multiple parties, under ideal circumstances).
@Heike
I'd say it depends on what kind of shop this is and if the author really meant "country of residence". Assuming they did and they added that condition on purpose, then technically, the original and the translation are expressing two different ideas right now. If you kept the "your" currency, it might be a bit clearer, but also not conclusive.
Granted, it might not matter most of the time and most customers will draw the right conclusion, but I wouldn't go as far as saying it doesn't matter. If I was to remove something like that, I'd probably point that fact out to the customer (and thus the LS), just to be safe.
Looking at that sentence, I'm also wondering if we can use a contraction like "in anderen Währungen und Ländern" in German, purely technically speaking. It's the same word, but once the use is local and once modal(?). But then there's also the character limit, of course, which often will require some compromises.
Hi Chris,
Thank you very much for your comment! I still think it was the best solution I chose, but I am still interested in what others think. Anyway, I probably should have made the comment. I was fully aware that an LS going "by the book" might consider it an omission. Thinking some more about it, a better solution might have been "Landeswährung". But that would not give the LS her conditional either, because that's a compound noun. She obviously wanted some similar grammatical structure, although most of us know that you cannot always translate grammatical structures 1:1. Even if it's possible, it isn't always the best solution.
P.S.
Actually, the more often I read the English sentence, the more confusing I find it, trying to say everything twice or more often and in different ways. This could have been said with even fewer words without the confusion and without any omission, I'm quite sure about it.
Hello all. It is very tempting to make a META observation here. Reading through this thread and many other similar threads here in the forums I think it becomes very evident that translations and their reviews are NOT an exact science – very far from it. This should not come as a surprise though, if we think about it, as language people, we all know that! If we all agree on that, a natural consequence will be that reviews will invariably, from time to time, result in different views and interpretations, often with no obvious RIGHT or WRONG answer, simply because this is NOT an exact science. That aside, I still think that many of these discussions/disputes over reviews could have been avoided altogether by actively making assumptions or seeking clarification from the customer in the comment field whenever the source contains issues. For example as Chris suggested to Heike above … “If I was to remove something like that, I'd probably point that fact out to the customer (and thus the LS), just to be safe.”
Perhaps the point is that we don't know for sure if all LS are really so much more competent than we are.
@gunnarbu
Thank you, gunnarbu. I think your point about translators being more proactive is sage advice and is definitely something that I will be doing more going forward in order to avoid situations like these.
But, as long as we are being meta here, one thing that I think might be relevant in this context is that translators are usually working on a deadline with no extra time or compensation allotted to proofreading, customer communications, or research. Of course, that doesn't excuse submitting objectively mistaken translations, but I do think that it means that it is often not reasonable for reviewers to penalize translators for failing to choose (what the reviewer thinks is) the most natural or faithful rendering in each case -- especially since the "naturalness" or "faithfulness" of a translation is a matter of degree and will include some subjective elements.
I think aardhiroki put it perfectly above; the standard for marking a translation as an error should be whether it is "a rendering of the source text that is not conceivably acceptable". Any other feedback should be marked as a "Suggestion".
@Heike S.
Perhaps that has something to do with it; but for me, personally, I suspect the issue is more a matter of the lack of feedback and oversight of the reviewers by Gengo's Quality Control Team. I hate to repeat myself, but I really do think that, in this respect, not sending re-reviews to a different reviewer from the original one is bad practice all around and harmful to the reviewers as well, since it denies them the opportunity to receive feedback regarding their feedback from a peer.
Hi @KO - just dropping you a quick line to inform you that I spoke with our Quality Team earlier in the week. The job you mention in the initial post of this thread, as well as the other in your other thread regarding political beliefs, have both been accepted for a third-party review which I hope you will receive soon (if you haven't yet.) As for your other feedback regarding LS communication, etc, it has certainly been passed along so that the team may sync with the LS again on best practices when it comes to communication with the translators, as well as for future improvements. Thanks so much for your patience!
Thank you, Lara. It is great to hear that things are finally moving forward on this.
While we are at it, I would appreciate a quick clarification regarding what you say here:
"One thing I would like to point out here is that, while the Support Article you mention does not say anything about "repeated errors" in a text (in fact, the Support Article is not intended to be an extensive explanation of the LS reference and training materials, and can't be taken as such), it is actually the case that our LSs are instructed to mark repeated errors only once and raise their severity, rather than marking them several times as independent errors within the text. In this regard, the LS was following protocol."
I understand that the Support Article cannot be taken to cover every little detail, but I would argue that it should provide a fairly comprehensive guideline for how errors are marked. After all, the Re-Review Request Form prompts translators to point out "Error(s) [that] should have not been marked according to Gengo's Error Types classification", giving detailed explanations and citing references. How are translators expected to do this if there are certain criteria that only the Language Specialists have access to and which are not made known to the translators in advance?
I would also like to know a little bit more about the reasoning behind the rule concerning repeated errors. Presumably, the rationale behind not marking repeated errors is to not punish the translator for the same mistake more than once. But if reviewers are allowed to mark the same error once and raise the severity, doesn't that amount to the same thing?
In fact, the whole idea at it stands is still murky to me. Does this mean that, for instance, the same error will be marked as "medium" if repeated more than (say) five times, as "critical" if repeated (say) more than ten times? And even if that were the case, again we have a problem with inconsistent application of the criteria. For instance, I have seen the same reviewer as the one in question here mark a repeated error found in a different job as "low", even though the term in question was repeated many more times than the term that was marked as a "medium" error in this job.
I agree with KO, if there are rules, they should be accessible by everyone.
@Heike
The point is that a person isn't necessarily in their country of residence when ordering something and there might also be a different delivery address. That probably doesn't happen frequently, but the original author decided to specify exactly what they mean with local currency. Perhaps they simply phrased things badly or even made an error, but we can't simply assume that.
Thank you Chris! Next time I'll ask the customer. Serves them right for asking me to shorten my translation without providing a character limit in the first place 😉
Yeah, that's certainly annoying, especially if it leads to even more problems down the road.