So what are these leaderboard rankings anyway

So what are these leaderboard rankings anyway Its week 6 of the italki World Cup Language Challenge.  (Yes, Germany has won the Cup, but a more pressing and curious question remains: who will win the italki World Cup Challenge?). Or, as many of you may be quick to point out, what does winning the italki challenge really mean?  How do I read the leaderboard? Should I boo or cheer? (Cheer is the answer to the last one learning a language is good for you). Offical 2014 italki World Cup Language Challenge Leaderboard Short Answer: The Country ranking attempts to answer the question Participants from which country have taken the most classes during the World Cup Challenge (on average)?. The Language ranking attempts to answer the question Students of which language showed the most dedication to taking classes (on average)?. This latter one got a bit confusing because many of the participants are actively studying multiple languages. Long Answer: Basic Methodology. Trying to put together a ranking of groups of people as diverse as italki users is always a strange proposition very few fit well in easily-defined groups. Ultimately we have had to take some shortcuts to be able to process the data and present it in some sort of a consistent fashion. The top-10 rankings you see have been limited only to teams of 4 or more. By teams here we mean ways of grouping participants. Country teams. The obvious way was by country. Even here, however, we had to make a decision about what counts as ones country, as we have both countries of origin, as well as countries in which our users live. For the top ten by country we decided to allocate students into teams based on their profile listing of the country of origin, add up all their session hours, and divide by number of people on the team. The reason for us using an average was to find a way to rank these teams on an overall metric. So, for example, the average participant from Mexico has spent 19.09 hours taking lessons with an italki teacher. As you can quickly imagine this has flaws. The Angola team wound up beng just one student, who has completed over 50 hours of lessons since the beginning of the challenge. Trying to defray the statistical anomalies like this, but still give a shout-out to the dedicated lone representatives of their countries, we have dropped teams with less than three students. Why 3? Frankly, we needed a number that would not eliminate too many teams, but could still be seen as a team-effort. Yes, it is rather arbitrary. Target Language teams. The math on this just got a little weird. Many of our participants are taking multiple languages. Some are even taking languages which are not listed in our site (one of our more prolific users who has racked up numerous hours in Tagalog is actually learning Ilokano from his teacher- a language we do not have formally listed on the site yet). After loads of hand-wringing and fights with our spreadsheet programs, we have decided to use this metric in a simple and crude way: Your target language team is determined by what language you have studied most of in the period of World Cup Challenge. Then, all the hours that you have taken regardless of language get tallied up and divided by the number of other members of your team. Yup, its very crude. Cantonese and Shanghainese dialects got dropped entirely for example. That said, the reason we chose this approach is: although its easy to tabulate the number of hours in a specific language, it is much harder to figure out how to divide that number to find the average. Do we divide them by total number of participants of the challenge? That would be unfair to the small dedicated groups learning Catalan or pretty much every language but English. Do we divide them by number of people who have at all studied this language? That also yields meaningless results, as it doesnt represent the amount of effort many of our students have put into studying a total of 3 or 4 languages. Ultimately we decided that a participants primary language will be his or her team, and created this relatively abstract measure. What the ranking says is that, on average, people studying Spanish (as a primary language) have spent approximately 7.51 hours taking language classes. Final Thoughts. We do not want our participants to miss out on the glory, so we are planning to do a final ranking by number of hours of all the users who have completed the challenge target. These will be individual rankings, with a breakdown of number of hours learned in at least their top-two or top-three languages. Doing this breakdown every week, however, would be very distracting for our team, and would take away from many other activities that are necessary to keep the site running: community management, customer service, handling the publication of articles, and promoting italki resources to inspired language-learners all over the world. Most importantly, we feel that the real winners of the challenge, whether completers or not, will be those who can look at the before video and the after video, and see how much they have accomplished in understanding another language and culture in avery short span of time. By the way, when your before and after videos are ready, please send them to support@italki.com. Good luck everyone in the last few days of the challenge!