Pages in topic:   [1 2] >
Feedback on the 9th Translation Contest: ideas for improvement
Thread poster: Silvia Prendin
Silvia Prendin
Silvia Prendin  Identity Verified
Italy
Local time: 12:15
Member (2009)
English to Italian
+ ...
Oct 16, 2009

I would like to use this topic to gather your comments about the contest process, so that it can be improved. Yes, I think there's room for improvement.

So here's something to start with.

Suppose you have 60 entries for a given language pair (e.g. English to French), they will appear on your screen in random order. I'm not sure we all have the time to go through 60 entries... Anyway, the voters will pick some of them and vote. Now, this method won't guarantee that these
... See more
I would like to use this topic to gather your comments about the contest process, so that it can be improved. Yes, I think there's room for improvement.

So here's something to start with.

Suppose you have 60 entries for a given language pair (e.g. English to French), they will appear on your screen in random order. I'm not sure we all have the time to go through 60 entries... Anyway, the voters will pick some of them and vote. Now, this method won't guarantee that these 60 entries will have the same *chances* to pass the first voting round. Some of them might get to be read by 3 people only and consequently be turned down for lack of votes. The risk is: even if quality is good and the entry has what it takes to pass the round, it won't pass. On the other hand other entries getting more votes but questionable quality will pass.

In such case, the first thing I can think of would be making the number of votes visible for each entry, so as a voter, if I see that an entry got 50 votes and another one got 1 vote, I would read the latter and then vote.

What do you think? Other ideas/comments?


[Subject edited by staff or moderator 2009-10-16 11:23 GMT]
Collapse


 
Juan González Pérez
Juan González Pérez
Spain
Local time: 12:15
Member (2008)
English to Spanish
+ ...
Translations to Spanish Oct 16, 2009

Hi Silvia,

Thanks for starting this thread, I love this contest and every idea that could improve it must be welcome

Here I go with my two cents: I think the contest could have some different categories for Spanish (i.e., European Spanish, Latin American Spanish...). It has been already done for other languages, and I consider it could be appropriate doing the same in this case.

Regards!


 
Iwona Szymaniak
Iwona Szymaniak  Identity Verified
Poland
Local time: 12:15
Member
English to Polish
+ ...
SITE LOCALIZER
Number of reviewers Oct 16, 2009

Well, Silvia is right we do not have time to review all the translations.

Sometimes we do not have time to review more than one and decide not to vote at all (I have to admit I was so busy this time I didn't cast my vote).

Perhaps it would be good to keep track of both reviewers and voters, and then devise a sort of proportional formula to get the voting in the fair perspective and numbers.

I am not sure, however, if it is a good idea to make the number
... See more
Well, Silvia is right we do not have time to review all the translations.

Sometimes we do not have time to review more than one and decide not to vote at all (I have to admit I was so busy this time I didn't cast my vote).

Perhaps it would be good to keep track of both reviewers and voters, and then devise a sort of proportional formula to get the voting in the fair perspective and numbers.

I am not sure, however, if it is a good idea to make the number of voters visible to the voting public during the evaluation stage. Why? Simply becasue we are likely to "follow the flock" in our voting decisions.


Iwona

[Zmieniono 2009-10-16 10:47 GMT]
Collapse


 
Lianne Wilson
Lianne Wilson
United Kingdom
Local time: 11:15
Japanese to English
+ ...
Translations to English Oct 16, 2009

At least in the French->English pair (I don't know about others) we had a lot of comments about texts sounding 'unamerican' or something like that. However, unless I am wrong, the to English translation contests do not specify whether to use American or British English, making such comments irrelevant.

Since no AE/BE preference is stated, I believe entries should not be criticised for using AE or BE terms or style in their translations.
So, I would suggest either:

... See more
At least in the French->English pair (I don't know about others) we had a lot of comments about texts sounding 'unamerican' or something like that. However, unless I am wrong, the to English translation contests do not specify whether to use American or British English, making such comments irrelevant.

Since no AE/BE preference is stated, I believe entries should not be criticised for using AE or BE terms or style in their translations.
So, I would suggest either:

A) making it more obvious that either AE or BE is fine and that comments against one or the other are not appropriate.
B) splitting the English competitions (at least those with enough entries) into AE and BE categories.
C) some other solution I'm too stoopid to think of right now
Collapse


 
Silvia Prendin
Silvia Prendin  Identity Verified
Italy
Local time: 12:15
Member (2009)
English to Italian
+ ...
TOPIC STARTER
Thank you both Oct 16, 2009

for your comments and ideas. Yes, I think that two different categories for Spanish would be appropriate, and this would be advisable for other languages too.

About the number of voters, I think it has nothing to do with likes and dislikes, rather the chances to be read. An entry might have 30 votes and an average rating of 1.2!


 
Silvia Prendin
Silvia Prendin  Identity Verified
Italy
Local time: 12:15
Member (2009)
English to Italian
+ ...
TOPIC STARTER
Thank you Lianne! Oct 16, 2009

Yes, the same goes for UK/US.

 
Gyula Jerkovich
Gyula Jerkovich  Identity Verified
Hungary
Local time: 12:15
Member (2006)
English to Hungarian
Screening? Oct 16, 2009

To address the question of too many entries and few votes to qualify them: in this case, maybe some kind of screening could work, e.g. if tagging would be required in the first round and items with above a certain no. of spelling mistakes or with 'mistranslation' tags, etc. would automatically not be qualified for the next round (i.e. a 'negative' qualification in the first round). This could help resolve the 'underrating' problem.

 
Henry Dotterer
Henry Dotterer
Local time: 07:15
SITE FOUNDER
Clarification Oct 16, 2009

Thanks for the ideas. Coming up with the best possible structure for contests is a fun challenge, and the job is definitely still underway. There are suggestions each time around that result in improvements to the contest structure, and I am sure this will continue for some time.

To clear up any possible misunderstandings:

* You don't have to rate all entries in a given language pair. Few people do.

* No entry has ever been eliminated due to lack of votes.<
... See more
Thanks for the ideas. Coming up with the best possible structure for contests is a fun challenge, and the job is definitely still underway. There are suggestions each time around that result in improvements to the contest structure, and I am sure this will continue for some time.

To clear up any possible misunderstandings:

* You don't have to rate all entries in a given language pair. Few people do.

* No entry has ever been eliminated due to lack of votes.

* Showing the number of raters (in the qualification round) is something that was suggested and implemented in the current contest.

Thanks!
Collapse


 
Henry Dotterer
Henry Dotterer
Local time: 07:15
SITE FOUNDER
We have "ruling out" for that, Gyula Oct 16, 2009

Gyula Jerkovich wrote:

To address the question of too many entries and few votes to qualify them: in this case, maybe some kind of screening could work, e.g. if tagging would be required in the first round and items with above a certain no. of spelling mistakes or with 'mistranslation' tags, etc. would automatically not be qualified for the next round (i.e. a 'negative' qualification in the first round). This could help resolve the 'underrating' problem.

What we have along these lines is "ruling out". When it becomes highly improbable, mathematically, that a given entry will wind up among the top 7 entries in terms of average rating, it gets removed from contention. This works.


 
Geraldine Oudin
Geraldine Oudin  Identity Verified
United Kingdom
Japanese to French
+ ...
Random order Oct 18, 2009

I think that the entries should appear in a random order, and that the order should change each time the section is accessed. Since most users (including myself) give up after rating the first 10 or 20, it would give a chance for all texts to be reviewed.

*****
http://geraldineoudin.com


 
Silvia Prendin
Silvia Prendin  Identity Verified
Italy
Local time: 12:15
Member (2009)
English to Italian
+ ...
TOPIC STARTER
Thank you Gyula, Henry and Geraldine Oct 18, 2009

I think the rules - as they are - are not clear enough. The steps taken are unknown.

1. How do you deal with an entry that was rated 4 just by one reader in the first round?(provided it was rated only once)

2. How do you deal with an entry that was rated 1 just by one reader in the first round?
(provided it was rated only once)

3. What if an entry is rated three times only?

4. Is there a minimum of ratings to pass the first round? Or can
... See more
I think the rules - as they are - are not clear enough. The steps taken are unknown.

1. How do you deal with an entry that was rated 4 just by one reader in the first round?(provided it was rated only once)

2. How do you deal with an entry that was rated 1 just by one reader in the first round?
(provided it was rated only once)

3. What if an entry is rated three times only?

4. Is there a minimum of ratings to pass the first round? Or can an entry pass with just two ratings for example?

5. Would it be possible to make the number of raters public in the first round? The number of ratings is now visible, but it is a sum of all ratings (not just the first round of course). If we have an idea about how many entries were rated and how many times, we could focus on the entries that received few ratings. I don't think such a step would take anything away from the other entries.

In short, I am thinking of a way to balance the distribution of ratings. This should not affect the rating in any way, because the quality is judged by the reader.

Geraldine, I know that entries are shown at random, but this is no guarantee that they will have equal chances to be read, especially if you consider that there might have been 30 raters in all for the first round. Also, as far as I am concerned, I rated the first entries on the list and the last ones, skipping the entries in the middle. So as you see, we cannot take anything for granted.
Collapse


 
Silvia Prendin
Silvia Prendin  Identity Verified
Italy
Local time: 12:15
Member (2009)
English to Italian
+ ...
TOPIC STARTER
9th Contest Oct 18, 2009

Henry, could you please tell us how many readers rated entries in the first round for each language pair?
Also, what do you think about separating language variants?


 
James_xia
James_xia  Identity Verified
China
English to Chinese
+ ...
To review part of the entries should be fine Oct 19, 2009

Restricting the time to consume, also to ensure the prudity and quality of the reviewing results, part of the entries can be reviewed instead of showing a comment on most or even the whole of the entries in a rush.

For a contest involving dozens of languages that covers 140 pairs, to organize an highly effective process is not as easy as it looks. Nonetheless, one thing is not for sure: I'm wonder why the participants were invited to give a comment on others.

Personal
... See more
Restricting the time to consume, also to ensure the prudity and quality of the reviewing results, part of the entries can be reviewed instead of showing a comment on most or even the whole of the entries in a rush.

For a contest involving dozens of languages that covers 140 pairs, to organize an highly effective process is not as easy as it looks. Nonetheless, one thing is not for sure: I'm wonder why the participants were invited to give a comment on others.

Personally, when given the chance to make a comment by ticking 'like' or 'dislike', I simply did not know how to start but make the careful identification on a couple of obvious mistakes (if apprpriate), and dare not to provide a relatively comprehensive review on each entry.

Later I thought maybe this way is in the hope of not neglecting the true comments, not only for their participation, but also to ensure a fair contest to the maxium level.

Thanks!
Collapse


 
Silvia Prendin
Silvia Prendin  Identity Verified
Italy
Local time: 12:15
Member (2009)
English to Italian
+ ...
TOPIC STARTER
Comments Oct 19, 2009

Thank you, James.

I think comments on mistakes are very important, but that's just my point of view. Some comments I saw were unsubstantiated, and there was nothing we could do about them: something else to think of too!


 
Geraldine Oudin
Geraldine Oudin  Identity Verified
United Kingdom
Japanese to French
+ ...
Are you sure? Oct 19, 2009

Silvia P. wrote:

I know that entries are shown at random, but this is no guarantee that they will have equal chances to be read, especially if you consider that there might have been 30 raters in all for the first round. Also, as far as I am concerned, I rated the first entries on the list and the last ones, skipping the entries in the middle. So as you see, we cannot take anything for granted.


Are you sure about that? During the second round, there were still many entries in the pairs I was allowed to rate, and I rated them in several times. However, the order did not seem to change...


 
Pages in topic:   [1 2] >


To report site rules violations or get help, contact a site moderator:

Moderator(s) of this forum
Lucia Leszinsky[Call to this topic]

You can also contact site staff by submitting a support request »

Feedback on the 9th Translation Contest: ideas for improvement






Anycount & Translation Office 3000
Translation Office 3000

Translation Office 3000 is an advanced accounting tool for freelance translators and small agencies. TO3000 easily and seamlessly integrates with the business life of professional freelance translators.

More info »
CafeTran Espresso
You've never met a CAT tool this clever!

Translate faster & easier, using a sophisticated CAT tool built by a translator / developer. Accept jobs from clients who use Trados, MemoQ, Wordfast & major CAT tools. Download and start using CafeTran Espresso -- for free

Buy now! »