Editing Semester A Progress Report 2012 (section)

====Language Verification====
In previous years there has not been a lot of focus on what language the code could be written in. In 2009 they tested 10 languages and came to the assumption that English fit the best. This year we have decided to verify this assumption by testing more languages and performing further analysis. This has been my main focus for contribution towards the project.  

To start this analysis, it began with reviewing previous years work; previously they tested languages by using a frequency analysis of the most used letters, and compared this with the code.  To test this we first went through the dictionary and calculated the number of words that begins with each letter of the alphabet. This was then converted into a frequency analysis and compared to the Somerton code.  The results from this were not very promising, as there was very little matching between the two lots of frequencies. The results are shown in figure 1 of the appendix.

Considering the previous years had mainly focused on Western-European languages, and records had shown the victim was most likely of Eastern-European decent, we decided to expand and consider more languages.  I was able to find a fantastic web site, which had the Tower of Babel bible passage translated into over a hundred different languages. Using this site, I was able to get 85 different languages that contained approximately 1000 characters in each translation.  This allow for frequency analysis to be performed by running a Java program which I had modified to calculate the number of times each letter appeared in the texts. This analysis was then repeated; by calculating the number of times each letter was the initial letter of a word.

The results from this were quite interesting. When we compared the initial letter frequency results with the Somerton code, by computing the difference in frequency and standard deviation. The results showed that English was the third closest, behind two Philippians’ languages Ilocano and Tagalog.  This helps to support the theory that the code is in English since it has such a high frequency comparison to the code. A table of these results is included within the appendix.

Further analysis need to be done in this area, the best thing to do would be to tae the top 20 languages and perform more frequency analysis using longer texts to refine the results.  As a result of the data we have collected, we will be constructing a Language Cross-Off List, similar to the cipher version that will explain why we have discounted each of the languages tested.