Difference between revisions of "Cipher cracking 2018 weekly progress"

From Derek
Jump to: navigation, search
(Created page with "== Weekly progress == === Semester 1 Week 1=== # Week 1 progress: #* Contacted Jikai and had preliminary meeting #* Contacted Professor Abbott and held first meeting #* Meet...")
 
(→‎Weekly progress)
 
(38 intermediate revisions by the same user not shown)
Line 4: Line 4:
  
 
# Week 1 progress:
 
# Week 1 progress:
−
#* Contacted Jikai and had preliminary meeting
+
#* Organise meeting with Prof. Abbott.
−
#* Contacted Professor Abbott and held first meeting
+
#* Contacted project partner and had a meeting.
−
#* Meeting occurred on Wednesday 4th March from 11:00am to 12:10pm
+
#* Attended the literature training session with project partner at Bar Smith library.
−
#* Booked Research Seminar with Librarian for Week 4
+
−
#* Conducted Background research by reading about previous final year projects
+
  
 
# Goals for Week 2:
 
# Goals for Week 2:
−
#* Research google n-gram
+
#* Meet Prof. Abbott.
−
#* Find out how much space the database requires
+
#* Understand the aims of the project
−
#* Find out about applying to use some of the budget on the purchase of a hard drive to store the database
+
−
#* Find out the cost of a copy of the google n-gram database (if any)
+
−
 
+
−
 
+
−
# Week 1 progress:
+
−
#* Meet Prof. Abbott and Nicholas.
+
−
#* Read and understand the background of the project.
+
−
#* Analyse all the documents from previous groups.
+
−
 
+
−
# Goals for Week 2:
+
−
#* Install some software given from Dr Matthew.
+
−
#* Prepare for Project Proposal Seminar in week 5.
+
  
 
=== Semester 1 Week 2===
 
=== Semester 1 Week 2===
  
 
# Week 2 progress:
 
# Week 2 progress:
−
#* Researched Google n-gram, size, cost.
+
#* Met with Prof. Abbott.
−
#* Discovered Microsoft Web N-Gram Services
+
#* Conducted Background research by reading about previous final year projects.
−
#* Researched Microsoft Web N-Gram Services
+
  
 
# Goals for Week 3:
 
# Goals for Week 3:
−
#* Read about Git Hub
+
#* Start preparing for the Proposal Seminar.
−
#* Create Git Hub account
+
#* Research about Somertons Mans letters.
−
#* Clone repository and install GUI on my Mac and Windows systems
+
−
#* Send Matthew Berryman user info to be granted permissions to repository
+
−
#* Begin work on Proposal Seminar
+
−
#* Create project Gantt chart
+
−
#* Create project Work Breakdown Structure
+
−
#* Organise meeting with Professor Abbott
+
−
 
+
−
 
+
−
# Week 2 progress:
+
−
#* Finish Github installment and creat an account in Github.
+
−
#* Download and use Github to check the folders Cipher_GUI and Decoding Toolkit from 2013 to find some useful information to continue our work.
+
−
#* Prepare for Proposal Seminar.
+
−
 
+
−
# Goals for Week 3
+
−
#* Trying to do the P-value for all language.
+
−
#* Doing Proposal Seminar.
+
−
#* Meet Prof. Abbott.
+
  
 
=== Semester 1 Week 3===
 
=== Semester 1 Week 3===
Line 58: Line 26:
  
 
# Week 3 progress:
 
# Week 3 progress:
−
#* Created Git Hub Account
+
#* Start Work Break Down Structure
−
#* Commenced work on Proposal Seminar
+
−
#* Created Work Breakdown Structure
+
 
#* Commenced work on project Gantt Chart
 
#* Commenced work on project Gantt Chart
−
#* Organised meeting with Professor Abbott for Monday 23rd March (Week 4) at 3pm
+
#* Organised meeting with Professor Abbott
−
#* Emailed Microsoft and applied for user token to be granted access to the Microsoft Web N-gram Services and received a reply
+
#* Worked on Proposal
 +
#* Met with project partner
 +
 
 
# Goals for Week 4:
 
# Goals for Week 4:
−
#* Attend meeting with Professor Abbott on Monday 23rd March at 3pm
+
#* Attend meeting with Professor Abbott
−
#* Send Matthew Berryman user info to be granted permissions to repository
+
 
#* Finish project Gantt Chart
 
#* Finish project Gantt Chart
−
#* Continue work on Proposal Seminar
 
−
#* Attend Literature Search Training Session on Tuesday 24th March at 10am
 
−
 
−
 
−
# Week 3 progress:
 
−
#* Begin calculate P-value of the code as some languages.
 
−
#* Worked on Proposal
 
−
# Goals for Week 4:
 
−
#* Have meeting with Prof. Abbott on Monday.
 
−
#* Finish Proposal draft and send to both Prof. Abbott and Dr. Mattew for feedback.
 
  
 
=== Semester 1 Week 4===
 
=== Semester 1 Week 4===
Line 83: Line 40:
  
 
# Week 4 progress:
 
# Week 4 progress:
−
#* Continued work on Proposal Seminar
+
#* Finished project Gantt Chart
−
#* Emailed Matthew Berryman GitHub user information
+
#* Met with Professor Abbott
−
#* Attended Literature Search Training Session on Tuesday 24th March at 10am
+
#* Worked on Proposal
−
#* Attended Professor Abbotts presentation on giving presentations
+
#* Met with project partner
 +
#* MATLAB coding started for Task 1
  
 
# Goals for Week 5:
 
# Goals for Week 5:
−
#* Work on Work Breakdown Structure, Gantt Chart, Task Allocation, Risk Assessment and Budgeting
+
#* Finish Power point for proposal
−
#* Submit draft of PowerPoint presentation and talk for Proposal Seminar to Professor Abbott
+
#* Make more progress in with Task 1
−
#* Rehearse for Proposal Seminar
+
−
#* Present Proposal Seminar on Tuesday 31st March
+
−
 
+
−
 
+
−
# Week 4 progress:
+
−
#* Finished draft of PPT presentation and met Prof Abbott on Monday.
+
−
#* Attended Literature Search Training Session on Tuesday.
+
−
 
+
−
# Goals for Week 5:
+
−
#* Present Proposal Seminar on Tuesday 31st March.
+
−
#* Continue doing P-value about the European language.
+
  
 
=== Semester 1 Week 5===
 
=== Semester 1 Week 5===
  
 
# Week 5 progress:
 
# Week 5 progress:
−
#* Completed Work Breakdown Structure, Gantt Chart, Task Allocation, Risk Assessment and Budgeting
+
#* Power Point nearly completed
−
#* Submitted draft of PowerPoint presentation and talk for Proposal Seminar to Professor Abbott
+
#* Names of horses, city names etc. have been finalized
−
#* Rehearsed for Proposal Seminar
+
#* Task 1 about 50% complete
−
#* Presented Proposal Seminar on Tuesday 31st March
+
  
−
# Goals for Week 6:
+
# Goals for Mid-Semester Break:
−
#* Commence background research for draft research proposal
+
#* Power Point has to be completed
−
#* Obtain previous groups statistics code from Professor Abbott
+
#* Prepare speech for proposal
 +
#* Meet with project partner in break to complete Task 1
  
 +
=== Semester 1 Mid-Semester Break===
  
−
# Week 5 progress:
 
−
#* Finish Seminar presentation
 
  
 +
# Mid-Semester Break progress:
 +
#* Met with project partner
 +
#* More coding done for Task 1
 +
#* Graphs have been produced for Task 1
 +
#* Task 1 75% complete
 +
#* Power Point got drafted by Prof. Abbott
 +
#* Proposal seminar completed
 +
#* DNA testing kit has been delivered
  
 
# Goals for Week 6:
 
# Goals for Week 6:
−
 
+
#* Organise meeting with Prof. Abbott
−
#* Continue doing task 1: P-value about languages
+
#* Complete Task 1
 +
#* Collect the DNA kit
 +
#* Send DNA kit ASAP
  
 
=== Semester 1 Week 6===
 
=== Semester 1 Week 6===
Line 128: Line 84:
  
 
# Week 6 progress:
 
# Week 6 progress:
−
#* Commenced background research for draft research proposal.
+
#* Sent DNA to 23andMe
−
 
+
#* Had meeting with Derek
−
# Goals for Mid-Semester Break:
+
#* An extension of Task 1 was given, to further look into the book
−
#* Complete draft research proposal by due date on Friday 17th April.
+
#* Still working on Task 1
−
#* Commence work on critical review of previous groups statistical analysis and begin to implement P-Value statistical testing.
+
−
#* Attend National Trust Somerton Man Tour on Tuesday 14th April.
+
−
 
+
−
 
+
−
# Week 6 progress:
+
−
#* Begin draft research proposal.
+
−
 
+
−
# Goals for Week Mid-Semester Break:
+
−
#* Finish draft research proposal before Friday 17th April.
+
−
 
+
−
=== Semester 1 Mid-Semester Break===
+
−
 
+
−
 
+
−
# Mid-Semester Break progress:
+
−
#* Completed draft research proposal by due date on Friday 17th April.
+
−
#* Commenced work on critical review of previous groups statistical analysis.
+
−
#* Attended National Trust Somerton Man Tour on Tuesday 14th April.
+
−
 
+
−
# Goals for Week 7:
+
−
#* Background research on how compile and program in Java for potential use in extension of 2013 group’s statistical frequency analysis.
+
−
 
+
−
 
+
−
# Mid-Semester Break progress:
+
−
#* Finished draft research proposal
+
−
#* Have a trip to Somerton beach
+
  
 
# Goals for Week 7:
 
# Goals for Week 7:
−
#* Based on 2013 group's work, continue doing P-Value test for Europen languages
+
#* Work on the extension task
 +
#* Start working on thesis
  
 
=== Semester 1 Week 7===
 
=== Semester 1 Week 7===
Line 165: Line 97:
  
 
# Week 7 progress:
 
# Week 7 progress:
−
#* Conducted background research on how compile and program in Java for potential use in extension of 2013 group’s statistical frequency analysis.
+
#* Original of Task 1 is completed
−
 
+
#* Extension of Task 1 near completion
 +
#* The code is complete to count the letter in each line of the book
 +
#* Thesis layout completed
 +
#* Thesis introduction started
 +
 
# Goals for Week 8:
 
# Goals for Week 8:
−
#* Organise meeting with Professor Abbott.
+
#* Complete extension of Task 1
−
#* Organise meeting with Jikai.
+
#* Complete Thesis introduction
−
#* Evaluate usability of 2013 group’s statistical analysis for extension.
+
#* Oragnize meeting with Derek
−
 
+
−
 
+
−
# Week 7 progress:
+
−
#* Used mathematic method to do the P-value test for Europen languages.
+
−
 
+
−
 
+
−
# Goals for Week 8:
+
−
#* Considering to use java or matlab to do the P-value test.
+
−
#* Meeting prof. Abbott.
+
  
 
=== Semester 1 Week 8===
 
=== Semester 1 Week 8===
  
 
# Week 8 progress:
 
# Week 8 progress:
−
#* Met with Professor Abbott.
+
#* Meeting with Derek
−
#* Met with Jikai to discuss progress.
+
#* Derek said that we have good progress
−
#* Evaluated usability of 2013 group’s statistical analysis for extension: Decided to use a combination of Java and Excel.  Java to be used for text processing, and outputting letter frequency results, and Excel to perform statistical analysis calculations and graph outputs.
+
#* Meeting with partner, to talk about thesis
 +
#* Task 1 is fully completed
 +
#* Results show that horse names are not the mysterious code
  
 
# Goals for Week 9:
 
# Goals for Week 9:
−
#* Organise meeting with Jikai and Professor Abbott.
+
#* Work on thesis
−
#* Contact Peter Varsos
+
−
#* Download programming resources
+
−
#* Begin compiling and formatting text resources for use in statistical frequency analysis.
+
−
 
+
−
 
+
−
# Week 8 progress:
+
−
#* Met with Nick to discuss P-value task progress.
+
−
#* Used excel to calculated P-value for English and Scots.
+
−
 
+
−
 
+
−
# Goals for Week 9:
+
−
#* Making a meeting with Professor Abbott.
+
−
#* Wait for Peter's reply about statistic value for each language in text resources.
+
−
#* Finish the P-value test between English and Somerton code.
+
  
 
=== Semester 1 Week 9===
 
=== Semester 1 Week 9===
Line 209: Line 124:
  
 
# Week 9 progress:
 
# Week 9 progress:
−
#* Organised and attended meeting with Professor Abbott for Friday 15th May
+
#* DNA results have come back, need to have meeting with Derek ASAP
−
#* Researched how to create branches and commit to GitHub
+
#* Thesis introduction completed
−
#* Committed 2013 group's work to GitHub
+
#* Sent drafts to each other
−
#* Downloaded and installed Eclipse
+
−
#* Emailed Peter Varsos about obtaining frequency analysis text files
+
 
   
 
   
 
# Goals for Week 10:
 
# Goals for Week 10:
−
#* Hopefully obtain text files from 2013 group for use in frequency analysis
+
#* Have meeting with Derek
−
#* Prepare test text files to test statistical analysis P-value method
+
#* Continue working on the thesis
−
 
+
−
 
+
−
# Week 9 progress:
+
−
#* Met with Professor Abbott and discuss our progress.
+
−
#* Complete the P-value test between English and Somerton code.
+
−
 
+
−
# Goals for Week 10:
+
−
#* Begin the P-value test for English, French and German.
+
−
#* Compare the 44 letters in somerton code with letters in English, French and German by Declaration of the human rights.
+
  
 
=== Semester 1 Week 10===
 
=== Semester 1 Week 10===
Line 232: Line 136:
  
 
# Week 10 progress:
 
# Week 10 progress:
−
#* Received reply from Peter Varsos, directed me to contact Lucy Griffith
+
#* Meeting with Derek
−
#* Emailed Lucy Griffith and received a reply, awaiting an additional reply to see whether or not she can find original text files for 2013 group’s statistical frequency analysis
+
#* Informed that we can't do any of the DNA task until the ethics is signed
−
#* Received reply from Lucy Griffith with zip file including all text files
+
#* Showed Derek results for task 1 and told us it was good
−
#* Prepared test text files for English, German and French to test our p-value statistical frequency analysis code
+
#* Drafted each others thesis and sent it back to each other
−
#* Committed working version and modifications of 2013’s group to Git Hub for use with text processing
+
−
#* Organised and attended meeting with Professor Abbot for Friday 22nd May
+
−
+
−
# Goals for Week 11:
+
−
#* Research how to create and edit wiki page
+
−
#* Create Research Project Proposal & Progress Report 2015 Wiki page
+
−
#* Download Anova package for Excel to test Jikai’s P-value method
+
−
#* Run test case for P-value in Matlab to check method against Excel and by hand
+
−
#* Commence writing progress report section of Research Project Proposal & Progress Report document
+
−
#* Organise meeting with Professor Abbott for Friday 29th May
+
−
 
+
−
 
+
−
# Week 10 progress:
+
−
#* Received test text files for English, German and French from Nick, and use excel to calculate the P-value for each language declaration against Code, other languages declaration against English statistics and English decalaration against itself.
+
−
#* Meet Nick and Professor Abbot for Friday 22nd May.
+
  
 
# Goals for Week 11:
 
# Goals for Week 11:
−
#* Use mathmatic methed to manually calculate P-value to check the result from excel is correct or not.
+
#* Only task to do now is the thesis
−
#* Use Matlab methed to check the result from excel is correct or not.
+
  
 
=== Semester 1 Week 11===
 
=== Semester 1 Week 11===
Line 260: Line 148:
  
 
# Week 11 progress:
 
# Week 11 progress:
−
#* Researched how to create and edit wiki page
+
#* Thesis introduction completed again
−
#* Created Research Project Proposal & Progress Report 2015 Wiki page
+
#* Thesis technical background started
−
#* Download Stat Plus for Mac to run Anova P-value test in Excel to test Jikai’s P-value method
+
#* Thesis project management started
−
#* Ran test case for P-value in Matlab to check method against Excel and by hand
+
−
#* Commenced writing progress report section of Research Project Proposal & Progress Report document
+
−
#* Attended meeting with Professor Abbott on Friday 29th May
+
 
   
 
   
 
# Goals for Week 12:
 
# Goals for Week 12:
−
#* Prepare text files for statistical analysis
+
#* Have the thesis completed by Tuesday, so we can send our drafts to each other
−
#* Complete Task 1: Statistical Frequency Analysis of Letters
+
−
#* Complete and submit Research Project Proposal & Progress Report onto MyUni by Friday 5th June
+
−
 
+
−
 
+
−
# Week 11 progress:
+
−
#* Used mathmatic methed to manually calculated P-value, the result was same as excel did.
+
−
#* Use Matlab to calculated the P-value, the result was same as excel did.
+
−
#* Attended meeting with Professor Abbott and Nick on Friday 29th May.
+
−
# Goals for Week 12:
+
−
#* Calculate the P-value for European languages form Declaration against Somerton code and compare results with 2013 group's.
+
−
#* Finish Semester A proposal and progress report.
+
  
 
=== Semester 1 Week 12===
 
=== Semester 1 Week 12===
Line 285: Line 159:
  
 
# Week 12 progress:
 
# Week 12 progress:
−
#* Emailed Lucy Griffith for past code or excel spreadsheets used to perform the 2013 group's statistical analysis and received excel spreadsheets containing raw data and method
+
#* Thesis completed and sent to partner
−
#* Emailed Matthew Berryman for assistance with statistical analysis
+
#* Draft each others thesis
−
#* Prepared text files for statistical analysis
+
#* Final copy of draft thesis completed and sent to my uni
−
#* Carried out statistical analysis using Normal Distribution methods
+
−
#* Carried out statistical analysis using Non-Parametric methods
+
−
#* Modified statistical analysis method to obtain more accurate results
+
−
#* Completed Task 1: Statistical Frequency Analysis of Letters
+
−
#* Completed and submitted Research Project Proposal & Progress Report onto MyUni on Friday 5th June
+
  
 
# Goals for Week 13:
 
# Goals for Week 13:
−
#* Upload content from Research Project Proposal & Progress Report onto 2015 group's Wiki page
+
#* Have meeting with Derek to talk about the next task that need to be completed
−
 
+
−
 
+
−
# Week 12 progress:
+
−
#* Calculated the P-value for European languages form Declaration against Somerton code and compare results with 2013 group's.
+
−
#* Calculated chi-squared value between top 20 European languages from declaration against code and top 21 popular European language from Wikipedia against code.
+
−
#* Finishd and submited Semester A proposal and progress report on 5th June.
+
−
# Goals for Week 13:
+
−
#* Finish and upload gourp progress report to Wiki page.
+
−
#* Prepare begin task 3.
+
  
 
=== Semester 1 Week 13===
 
=== Semester 1 Week 13===
Line 310: Line 170:
  
 
# Week 13 progress:
 
# Week 13 progress:
−
#* Uploaded content from Research Project Proposal & Progress Report onto 2015 group's Wiki page
+
#* Exam study, therefore no project work
−
 
+
−
# Goals for Exam Period:
+
−
#* Exam Revision
+
−
#* Commence Task 2
+
−
 
+
−
 
+
−
# Week 13 progress:
+
−
#* Uploaded research project proposal & progress report onto 2015 group's Wiki page.
+
  
 
# Goals for Exam Period:
 
# Goals for Exam Period:
−
#* Review for exam.
+
#* Do Exam
  
 
===Mid Year Break Week 1===
 
===Mid Year Break Week 1===
Line 327: Line 179:
  
 
# Week 1 progress:
 
# Week 1 progress:
−
#* Installed Microsoft Visual Studio.
+
#* Meet up with partner to talk about next task
−
#* Created and ran n-gram test code using quick start guide.
+
#* Ethics been approved
−
 
+
 
# Goals for Week 2:
 
# Goals for Week 2:
−
#* Re-evaluate the use of the Microsoft Database and perhaps consider the Google alternative.
+
#* Start doing DNA task
−
 
+
−
 
+
−
# Week 1 progress:
+
−
#* Finish Chi-squared tests update(If the letter appeared 0 time.
+
−
 
+
−
Use number 1 to instead of 0 for numbers)
+
−
 
+
−
# Goals for Week 2:
+
−
 
+
−
#* Go to Gold Coast.
+
  
 
===Mid Year Break Week 2===
 
===Mid Year Break Week 2===
  
 
# Week 2 progress:
 
# Week 2 progress:
−
#* Upon further research, and consulting Dr Matthew Berryman and the Microsoft Web N-Gram Team, I found that the Microsoft database would require too many calls for use with our application and so the Google alternative must be considered.
+
#* Degrading the DNA is nearly completed
−
#* Enquired about access to the collated database from the University of Pennsylvania Linguistic Data Consortium, was quoted a cost of 150 USD for the license, plus 50 USD postage of the physical DVDs upon which they are stored.
+
#* Matlab code is nearly done
 
    
 
    
 
# Goals for Week 3:
 
# Goals for Week 3:
−
#* Begin writing search engine for use with Google corpus.
+
#* Have the code completed
−
 
+
−
# Week 2 progress:
+
−
#* Travel in Gold Coast.
+
−
 
+
−
# Goals for Week 3:
+
−
#* Review One-time pad work from 2013's group.
+
−
#* Search information about One-time pad.
+
  
 
===Mid Year Break Week 3===
 
===Mid Year Break Week 3===
  
 
# Week 3 progress:
 
# Week 3 progress:
−
#* Sought assistance from Dr Matthew Berryman, who wrote code to download the n-gram files from Google and collate them into a single file.
+
#* Code is completed
−
#* Dr Matthew Berryman advised me to use Python to write the database search engine
+
#* Started degrading the DNA, by removing SNPs
−
#* Began to learn python language
+
 
−
#* Began writing search engine
+
−
#* Had Skype conference with Dr Matthew Berryman
+
−
#* Organised meeting with Professor Abbott for next week
+
−
 
+
 
# Goals for Semester 2 week 1:
 
# Goals for Semester 2 week 1:
−
#* Continue learning python and continue working on search engine
+
#* Meet with Derek and show him our progress
−
#* Meet with Professor Abbott
+
−
 
+
−
# Week 3 progress:
+
−
#* Reviewed One-time pad work from 2013's group.
+
−
#* Searched information about One-time pad.
+
−
#* Organised meeting with Professor Abbott for next week.
+
−
# Goals for Semester 2 Week 1:
+
−
#* Met with Professor Abbott and Nick.
+
  
 
===Semester 2 Week 1===
 
===Semester 2 Week 1===
  
 
# Week 1 progress:
 
# Week 1 progress:
−
#* Met with Professor Abbott on Tuesday 28 July.
+
#* Met with Derek, says our work is going well
−
#* Continued working on search engine code in conjunction with Dr Matthew Berryman.
+
#* Degraded the DNA by 10% intervals and now waiting for results
−
#* Ran code on database for first time.
+
−
#* Continued learning python functions used in code for use in thesis writeup.
+
  
 
# Goals for Week 2:
 
# Goals for Week 2:
−
#* Assist Jikai with Task 3
+
#* After results have come, start to analyse the result and figure what the next steps are
−
#* Continue writing up elements for use in thesis.
+
−
 
+
−
# Week 1 progress:
+
−
 
+
−
#* Met with Professor Abbott and Nick on Tuesday 28 July.
+
−
#* Discussed task 3 with Nick.
+
−
# Goals for Week 2:
+
−
 
+
−
#* Started writing code for task 3 One-time pad.
+
  
 
===Semester 2 Week 2===
 
===Semester 2 Week 2===
  
 
# Week 2 progress:
 
# Week 2 progress:
−
#* Recieved results from first test from Matthew Berryman.
+
#* Another set of degrading data was put into Gedmatch to compare results
−
#* Commenced working on obtaining statistical data from results.
+
#* Met with Derek to inform us on what the next step is, which is to find false positives and false negatives and graph results
−
#* Continued writing up elements for thesis.
+
#* Also got told to look into the heritage stuff on Gedmatch
  
 
# Goals for Week 3:
 
# Goals for Week 3:
−
#* Try to complete statistical results.
+
#* Plot false positives and false negatives
−
#* Email Dr Andrew Allison to book consulting time to discuss our task 1 statistical analysis results
+
#* Start looking into booking the mass spectrometer lab
−
 
+
−
# Week 2 progress:
+
−
#* Try to use Matlab to write code for task 3 One-time pad.
+
−
 
+
−
# Goals for Week 3:
+
−
#* Continue writing code for task 3.
+
  
 
===Semester 2 Week 3===
 
===Semester 2 Week 3===
  
 
# Week 3 progress:
 
# Week 3 progress:
−
#* Emailed Dr Andrew Allison and received no reply.
+
#* Mass Spect booked
−
#* Organised appointment with Professor Abbott for Friday 21st of August.
+
#* Obtained Hair for Mass spect
−
#* Emailed Dr Matthew Berryman about discrepancy between raw results and initial results using count column.
+
  
 
# Goals for Week 4:
 
# Goals for Week 4:
−
#* Meet with Professor Abbott on Friday.
+
#* Complete Mass Spect
−
#* Discuss n-gram search results with Dr Berryman and Professor Abbott to decide whether/what type of statistical analysis is to be conducted.
+
#* Write Thesis
−
 
+
−
# Week 3 progress:
+
−
#* Continued using Matlab to write code for task 3.
+
−
 
+
−
 
+
−
# Goals for Week 4:
+
−
#* Use Matlab to write code for task 3.
+
−
#* Finish case(4M and 2W) Chi-squared value calculations.
+
  
 
===Semester 2 Week 4===
 
===Semester 2 Week 4===
  
 
# Week 4 progress:
 
# Week 4 progress:
−
#* Met with Professor Abbott for Friday 21st of August.
+
#* Mass Spect Completed
−
#* Decided that we are unable to conduct statistical analysis from results of n-gram search, so results to be cleaned up and presented as part of final report.
+
#* Finished Task 3
  
 
# Goals for Week 5:
 
# Goals for Week 5:
−
#* Clean up search results for presentation
+
#* Analysis results from mass spect
−
#* Visit Dr Allison's office to organise a meeting
+
−
#* Conduct verification and testing of Jikai's one-time pad code.
+
−
 
+
−
# Week 4 progress:
+
−
#* Meet with Professor Abbott on Friday 21st of August.
+
−
#* Finish programming code for one time pad.
+
−
 
+
−
# Goals for Week 5:
+
−
#* Add last function to the code.
+
−
#* Review our task 1 statistical analysis.
+
  
 
===Semester 2 Week 5===
 
===Semester 2 Week 5===
  
 
# Week 5 progress:
 
# Week 5 progress:
−
#* Emailed Hong Gunn Chew about University of Wollongong invoicing University of Adelaide for server and processing costs, using project budget.
+
#* Made some progress with the results
−
#* Tried reaching Dr Andrew Allison via his office phone on multiple days but he was not in.
+
#* Met with Derek
−
#* Spoke to Electrical Engineering office and was given an alternate email to try to reach him at.
+
−
#* Emailed Dr Allison to organise meeting.
+
−
#* Verified Jikai's One Time Pad code and suggested changes and improvements.
+
  
 
# Goals for Week 6:
 
# Goals for Week 6:
−
#* Clean up search results for presentation.
+
#* Continue on analyzing results
−
#* Continue writing up n-gram search section of thesis.
+
#* Continue writing thesis  
−
#* Obtain mass spectrometer data from Professor Abbott to begin extension task (task 4).
+
−
 
+
−
# Week 5 progress:
+
−
#* Improve the matlab code and add last function in order to decipher the code by last character.
+
−
#* Review our chi-squared result.
+
−
#* Complete our group wiki page under IEEE home page.
+
−
# Goals for Week 6:
+
−
#* Continue improving code and add function.
+
−
#* Organise a meeting with Dr Allison to discusss our statistic result.
+
  
 
===Semester 2 Week 6===
 
===Semester 2 Week 6===
  
 
# Week 6 progress:
 
# Week 6 progress:
−
#* Met with Dr Andrew Allison and discussed possibility of attempting statistical analysis from task 1 again using Bayes' theorem.
+
#* Made some progress with the results
−
#* Continued writing code to clean up search results for presentation.
+
−
#* Continued writing up n-gram search section of thesis.
+
−
 
+
 
# Goals for Week 7:
 
# Goals for Week 7:
−
#* Arrange meeting with Professor Abbott
+
#* Continue on analyzing results
−
#* Send formal application for funding to Dr Hong-Gunn Chew including amended and forecasted budget.
+
#* Continue writing thesis
−
#* Email Matthew Berryman for advice on proposed method.
+
−
#* Finish clean up code and excel import of data.
+
−
 
+
−
# Week 6 progress:
+
−
#* Continued improving code and add last function.
+
−
#* Met Dr Allison and get some useful suggetions to improve our statistic tests.
+
−
# Goals for Week 7:
+
−
#* Finished one time pad code improvement.
+
−
#* Organise a meeting with Prof Abbott and Dr Matthew to discuss our statistic results.
+
  
 
===Semester 2 Week 7===
 
===Semester 2 Week 7===
  
 
# Week 7 progress:
 
# Week 7 progress:
−
#* Arranged to have meeting with Professor Abbott and Dr Berryman in week 8.
+
#* Found Strontium levels are lower in Adelaide compared to Bali
−
#* Created agenda of items of importance to discuss in meeting.
+
#* Continue analysing other elements to find any significants
−
#* Had email discussion with Dr Berryman and Dr Allison about statistical analysis approach.
+
−
#* Completed cleanup code.
+
−
#* Ran completed cleanup code on search results.
+
−
#* Cleanup code produced repeated results, so commenced writing double remover code.
+
−
#* Completed application for funding to be sent to Dr Hong-Gunn Chew, waiting on invoices for proof of purchases from Dr Berryman.
+
  
 
# Goals for Week 8:
 
# Goals for Week 8:
−
#* Complete and run doubles remover code and import outputs into excel document.
+
#* Continue writing thesis
−
#* Email application for funding to Dr Hong-Gunn Chew.
+
−
#* Meet with Professor Abbott and Dr Berryman to discuss project progress so far and direction of further work.
+
−
 
+
−
# Week 7 progress:
+
−
#* Finished one time pad code improvement.
+
−
#* Organise a meeting with Prof Abbott and Dr Matthew to discuss our statistic results.
+
−
#* The matlab code was able to decode the Somerton code by using different letter locations and last letter as key.
+
−
 
+
−
# Goals for Week 8:
+
−
#* Meet both Professor Abbott and Dr Berryman to discuss project progress so far and we will start task 4 or update our task 1 results in the midbreak.
+
−
#* Use matlab code to output decoding messages and find some meaningful words.
+
  
 
===Semester 2 Week 8===
 
===Semester 2 Week 8===
  
 
# Week 8 progress:
 
# Week 8 progress:
−
#* Created agenda for meeting with Professor Abbott, Dr Berryman and Jikai.
+
#* Graphs made with the different elements
−
#* Met with Professor Abbott, Dr Berryman and Jikai on Friday 18 September.
+
 
−
#* In meeting, decided upon using texts from project Gutenberg to redo chi-squared statistical analysis on the language of origin of the code to overcome the issue of letters appearing 0 times in using the UN Declaration of Human Rights as the base text.
+
−
#* Received invoices from Dr Berryman and emailed completed application for funding to Dr Hong-Gunn Chew.
+
−
#* Completed doubles remover code.
+
  
 
# Goals for Mid-Semester Break:
 
# Goals for Mid-Semester Break:
−
#* Process results using doubles remover code and import into excel.
+
#* Continue writing thesis
−
#* Obtain useful results from n-gram search for use in final report.
+
#* Start making the poster
−
#* Submit exhibition requirements.
+
−
#* Fix up all original chi-squared test results.
+
−
#* Perform Chi-squared reanalysis using project Gutenberg texts.
+
  
−
# Week 8 progress:
 
−
#* Met Professor Abbott and Dr Berryman on Friday 18/09/2015.
 
−
#* In the meeting, decided we should redo our task 1 by using texts from project Gutenberg instead of using the UN Declaration of Human Rights as the base text.
 
−
#* Used matlab code to output decoding messages and try to find some meaningful words.
 
  
−
 
−
# Goals for Mid-Semester Break:
 
−
#* Redo our chi-squared test on task 1.
 
−
#* Search text files on project Gutenberg webpage and format them.
 
−
#* Analyse results in task 3.
 
  
 
===Mid Semester Break===
 
===Mid Semester Break===
 
# Week Mid-Semester Break progress:
 
# Week Mid-Semester Break progress:
−
#* Submitted Exhibition requirements and created shortened project title and project description for display.
+
#* Poster nearly completed
−
#* Extended doubles remover code to work for results that appear more than twice.
+
−
#* Processed results and output final results in alphabetical order for comparison with raw results and cleaned up results.
+
−
#* Also processed results and output in order of highest number of year occurrence to lowest.
+
−
#* Tried importing results into Excel but ran out of rows, as data was too large, so instead exported full results for use in final dump of work, as well as top 20 results ranked in order of occurrence for each group of n-grams from 1-5 for inclusion in appendices of final report/thesis/wiki.
+
−
#* Discussed finding interesting/useful combinations of 5 grams for presenting in final report with Matthew Berryman
+
−
#* Received most popular interesting/useful 5 gram combinations from Dr. Berryman.
+
−
#* Fixed up task 1 initial results with count changed from 0 to 1, fixed up second refinement, for letters with 0 count, changing frequency to 0.001, and commenced re-analysis of some of the top 20 European languages with larger text files from project Gutenberg to allow no letters with count 0 to occur to verify our top 20 European results.
+
−
#* Wrote instructions on how to use initial and all letter counter java files from 2013 group’s work
+
  
 
# Goals for week 9:
 
# Goals for week 9:
−
#* Organise meeting and meet with Professor Abbott
+
#* Continue writing thesis
−
#* Visit Maths Learning Centre to see if there is any way to obtain a p-value from our new data sets.
+
#* Complete poster so Derek can draft
−
#* Finalise all results, figures, graphs etc. for use in presentation and thesis/final report
+
−
#* Commence creating slideshow for final presentation
+
−
#* Commence writing thesis skeleton for use in final presentation
+
−
 
+
−
====Jikai====
+
−
# Week Mid-Semester Break progress:
+
−
#* Redo our chi-squared test on task 1.
+
−
#* Searched text files on project Gutenberg webpage and format them. The new text files from project Gutenberg allows no letters with count 0 and we uesd them to verify our chi-squared test results.
+
−
#* Analyse results in task 3, pick some of them to used in our Final Seminar and Final Report.
+
−
 
+
−
# Goals for week 9:
+
−
#* Meet Professor Abbott and Dr Berryman to discuss our update results for task 1.
+
−
#* Start writing slides for our Final Seminar presentation.
+
−
#* Start writing our Final thesis.
+
  
 
===Semester 2 Week 9===
 
===Semester 2 Week 9===
 
# Week 9 progress:
 
# Week 9 progress:
−
#* Obtained promising results from re-analysis of a portion of the top 20 European languages using project Gutenberg text files and the chi-squared method.
+
#* Poster completed, sent to get drafted
−
#* Organised meeting and met with Professor Abbott and Dr. Berryman on Thursday 8 October at 11am.
+
#* Thesis needs to be completed, so it can be drafted
−
#* Conclusions from meeting were to increase the sample size of the English portion of our Project Gutenberg corpus to 20 concatenated novels, and to take 100 44 letter samples from this to obtain our average chi-squared value for English to calculate our English p-value significance level.
+
−
#* Visited Maths Learning Centre and was given some useful advice on how to compare our control p-values vs code p-values.
+
−
#* Attempted normal distribution p-value analysis method on chi-squared value of code versus average chi-squared value of English samples but did not work since lower chi-squared values are better, not closer chi-squared values.
+
−
#* Finalised all results, figures, graphs etc. for use in presentation and thesis/final report.
+
−
#* Commenced creating slideshow for final presentation.
+
−
#* Continued writing thesis for inclusion of task 4: statistical frequency of letters reanalysis in final presentation.
+
  
 
# Goals for week 10:
 
# Goals for week 10:
−
#* Complete slideshow and talk for final presentation.
+
#* Complete thesis
−
#* Present final presentation on Tuesday 13 October at 5pm.
+
#* Edit poster after it is drafted
−
#* Write python code in order to increase the sample size of the English portion of our Project Gutenberg corpus to 20 concatenated novels, and to take 100 44 letter samples from this.
+
#* Start the final seminar slides
−
#* Obtain average chi-squared value for English to calculate our English p-value significance level.
+
−
#* Obtain chi-squared and p-value for new English base text vs Somerton Man code.
+
−
#* Continue writing thesis.
+
−
 
+
−
# Week 9 progress:
+
−
#* Meet Professor Abbott and Dr Berryman on Thursday 08/10/2015 and discuss our update results on task 1.
+
−
#* Discuss with Nick about the Final Seminar workplan and time.
+
−
#* Start writing slides for our Final Seminar presentation.
+
−
 
+
−
 
+
−
# Goals for week 10:
+
−
#* The final seminar will hold on Tuesday 13/10/2015.
+
−
#* Continue updating and improving our Chi-squared test for task 1.
+
−
#* Start writing our Final thesis.
+
  
 
===Semester 2 Week 10===
 
===Semester 2 Week 10===
 
# Week 10 progress:
 
# Week 10 progress:
−
#* Completed slideshow and talk for final presentation.
+
#* Poster completed and submitted
−
#* Presented final presentation on Tuesday 13 October at 5pm.
+
#* Final seminar slides done and sent to Derek to get drafted
−
#* Completed python codes for implementing increased sample size reanalysis for Task 4
+
#* Sent Derek our thesis to get drafted
−
#* Obtained average chi-squared value for English to calculate our English p-value significance level.
+
−
#* Obtained chi-squared and p-value for new English base text vs Somerton Man code.
+
−
#* Continued writing thesis.
+
  
 
# Goals for week 11:
 
# Goals for week 11:
−
#* Complete and submit final report/honours thesis
+
#* Edit final seminar slides when drafted
−
#* Complete honours poster
+
#* Edit thesis
−
 
+
#* Prepare for expo
−
# Week 10 progress:
+
−
#* Finished the PPT slides for final seminar.
+
−
#* Presented The final seminar on Tuesday 13/10/2015.
+
−
#* Downloaded and formatted English novels from Project Gutenberg, and sent results to Nick.
+
−
#* Started writing our Final thesis.
+
−
.
+
−
 
+
−
# Goals for week 11:
+
−
#* Finish our Final thesis in Wiki page.
+
  
 
===Semester 2 Week 11===
 
===Semester 2 Week 11===
 
# Week 11 progress:
 
# Week 11 progress:
−
#* Completed and submitted Final Report/Honours Thesis
+
#* Get Thesis Drafted
−
#* Designed and submitted project exhibition poster
+
#* Final seminar slides submitted
−
#* Organised project exhibition resources
+
#* Expo completed
  
 
# Goals for week 12:
 
# Goals for week 12:
−
#* Present at project exhibition
+
#* Finish Thesis
−
 
+
#* Prepare for final seminar
−
# Week 11 progress:
+
#* Give all relevant files to Derek
−
*# Finished final thesis before Wednesday 21/10/2015.
+
−
*# Prepared for exhibition next week.
+
−
 
+
−
 
+
−
# Goals for week 12:
+
−
*# Exhibition will hold between Monday (26/10/2015) and Tuesday (27/10/2015).
+
−
*#
+
  
 
===Semester 2 Week 12===
 
===Semester 2 Week 12===
 
# Week 12 progress:
 
# Week 12 progress:
−
#* Presented at Ingenuity Exhibition
+
#* Thesis done and submitted
−
#* Collated work for final dump to be submitted Friday 6 November
+
#* Work on Final Seminar
−
#* Organised final meeting with Derek for Friday 6 November
+
  
−
# Week 12 progress:
+
# Goals for week 13:
−
#*  
+
#* Prepare for final seminar
−
#*
+
 
−
#*  
+
===Semester 2 Week 13===
−
#*
+
# Week 13 progress:
 +
#* Final seminar completed
 +
#* Closed project
  
 
==See also==
 
==See also==
−
* [[Cipher Cracking 2013]]
+
 
−
* [[Semester A Progress Report 2013 - Cipher cracking]]
+
−
* [[Semester B Final Report 2013]]
+
−
* [[Cipher Cross-off List]]
+
  
 
==Back==
 
==Back==

Latest revision as of 10:35, 3 November 2018

Weekly progress

Semester 1 Week 1

  1. Week 1 progress:
    • Organise meeting with Prof. Abbott.
    • Contacted project partner and had a meeting.
    • Attended the literature training session with project partner at Bar Smith library.
  1. Goals for Week 2:
    • Meet Prof. Abbott.
    • Understand the aims of the project

Semester 1 Week 2

  1. Week 2 progress:
    • Met with Prof. Abbott.
    • Conducted Background research by reading about previous final year projects.
  1. Goals for Week 3:
    • Start preparing for the Proposal Seminar.
    • Research about Somertons Mans letters.

Semester 1 Week 3

  1. Week 3 progress:
    • Start Work Break Down Structure
    • Commenced work on project Gantt Chart
    • Organised meeting with Professor Abbott
    • Worked on Proposal
    • Met with project partner
  1. Goals for Week 4:
    • Attend meeting with Professor Abbott
    • Finish project Gantt Chart

Semester 1 Week 4

  1. Week 4 progress:
    • Finished project Gantt Chart
    • Met with Professor Abbott
    • Worked on Proposal
    • Met with project partner
    • MATLAB coding started for Task 1
  1. Goals for Week 5:
    • Finish Power point for proposal
    • Make more progress in with Task 1

Semester 1 Week 5

  1. Week 5 progress:
    • Power Point nearly completed
    • Names of horses, city names etc. have been finalized
    • Task 1 about 50% complete
  1. Goals for Mid-Semester Break:
    • Power Point has to be completed
    • Prepare speech for proposal
    • Meet with project partner in break to complete Task 1

Semester 1 Mid-Semester Break

  1. Mid-Semester Break progress:
    • Met with project partner
    • More coding done for Task 1
    • Graphs have been produced for Task 1
    • Task 1 75% complete
    • Power Point got drafted by Prof. Abbott
    • Proposal seminar completed
    • DNA testing kit has been delivered
  1. Goals for Week 6:
    • Organise meeting with Prof. Abbott
    • Complete Task 1
    • Collect the DNA kit
    • Send DNA kit ASAP

Semester 1 Week 6

  1. Week 6 progress:
    • Sent DNA to 23andMe
    • Had meeting with Derek
    • An extension of Task 1 was given, to further look into the book
    • Still working on Task 1
  1. Goals for Week 7:
    • Work on the extension task
    • Start working on thesis

Semester 1 Week 7

  1. Week 7 progress:
    • Original of Task 1 is completed
    • Extension of Task 1 near completion
    • The code is complete to count the letter in each line of the book
    • Thesis layout completed
    • Thesis introduction started
  1. Goals for Week 8:
    • Complete extension of Task 1
    • Complete Thesis introduction
    • Oragnize meeting with Derek

Semester 1 Week 8

  1. Week 8 progress:
    • Meeting with Derek
    • Derek said that we have good progress
    • Meeting with partner, to talk about thesis
    • Task 1 is fully completed
    • Results show that horse names are not the mysterious code
  1. Goals for Week 9:
    • Work on thesis

Semester 1 Week 9

  1. Week 9 progress:
    • DNA results have come back, need to have meeting with Derek ASAP
    • Thesis introduction completed
    • Sent drafts to each other
  1. Goals for Week 10:
    • Have meeting with Derek
    • Continue working on the thesis

Semester 1 Week 10

  1. Week 10 progress:
    • Meeting with Derek
    • Informed that we can't do any of the DNA task until the ethics is signed
    • Showed Derek results for task 1 and told us it was good
    • Drafted each others thesis and sent it back to each other
  1. Goals for Week 11:
    • Only task to do now is the thesis

Semester 1 Week 11

  1. Week 11 progress:
    • Thesis introduction completed again
    • Thesis technical background started
    • Thesis project management started
  1. Goals for Week 12:
    • Have the thesis completed by Tuesday, so we can send our drafts to each other

Semester 1 Week 12

  1. Week 12 progress:
    • Thesis completed and sent to partner
    • Draft each others thesis
    • Final copy of draft thesis completed and sent to my uni
  1. Goals for Week 13:
    • Have meeting with Derek to talk about the next task that need to be completed

Semester 1 Week 13

  1. Week 13 progress:
    • Exam study, therefore no project work
  1. Goals for Exam Period:
    • Do Exam

Mid Year Break Week 1

  1. Week 1 progress:
    • Meet up with partner to talk about next task
    • Ethics been approved
  1. Goals for Week 2:
    • Start doing DNA task

Mid Year Break Week 2

  1. Week 2 progress:
    • Degrading the DNA is nearly completed
    • Matlab code is nearly done
  1. Goals for Week 3:
    • Have the code completed

Mid Year Break Week 3

  1. Week 3 progress:
    • Code is completed
    • Started degrading the DNA, by removing SNPs
  1. Goals for Semester 2 week 1:
    • Meet with Derek and show him our progress

Semester 2 Week 1

  1. Week 1 progress:
    • Met with Derek, says our work is going well
    • Degraded the DNA by 10% intervals and now waiting for results
  1. Goals for Week 2:
    • After results have come, start to analyse the result and figure what the next steps are

Semester 2 Week 2

  1. Week 2 progress:
    • Another set of degrading data was put into Gedmatch to compare results
    • Met with Derek to inform us on what the next step is, which is to find false positives and false negatives and graph results
    • Also got told to look into the heritage stuff on Gedmatch
  1. Goals for Week 3:
    • Plot false positives and false negatives
    • Start looking into booking the mass spectrometer lab

Semester 2 Week 3

  1. Week 3 progress:
    • Mass Spect booked
    • Obtained Hair for Mass spect
  1. Goals for Week 4:
    • Complete Mass Spect
    • Write Thesis

Semester 2 Week 4

  1. Week 4 progress:
    • Mass Spect Completed
    • Finished Task 3
  1. Goals for Week 5:
    • Analysis results from mass spect

Semester 2 Week 5

  1. Week 5 progress:
    • Made some progress with the results
    • Met with Derek
  1. Goals for Week 6:
    • Continue on analyzing results
    • Continue writing thesis

Semester 2 Week 6

  1. Week 6 progress:
    • Made some progress with the results
  2. Goals for Week 7:
    • Continue on analyzing results
    • Continue writing thesis

Semester 2 Week 7

  1. Week 7 progress:
    • Found Strontium levels are lower in Adelaide compared to Bali
    • Continue analysing other elements to find any significants
  1. Goals for Week 8:
    • Continue writing thesis

Semester 2 Week 8

  1. Week 8 progress:
    • Graphs made with the different elements


  1. Goals for Mid-Semester Break:
    • Continue writing thesis
    • Start making the poster


Mid Semester Break

  1. Week Mid-Semester Break progress:
    • Poster nearly completed
  1. Goals for week 9:
    • Continue writing thesis
    • Complete poster so Derek can draft

Semester 2 Week 9

  1. Week 9 progress:
    • Poster completed, sent to get drafted
    • Thesis needs to be completed, so it can be drafted
  1. Goals for week 10:
    • Complete thesis
    • Edit poster after it is drafted
    • Start the final seminar slides

Semester 2 Week 10

  1. Week 10 progress:
    • Poster completed and submitted
    • Final seminar slides done and sent to Derek to get drafted
    • Sent Derek our thesis to get drafted
  1. Goals for week 11:
    • Edit final seminar slides when drafted
    • Edit thesis
    • Prepare for expo

Semester 2 Week 11

  1. Week 11 progress:
    • Get Thesis Drafted
    • Final seminar slides submitted
    • Expo completed
  1. Goals for week 12:
    • Finish Thesis
    • Prepare for final seminar
    • Give all relevant files to Derek

Semester 2 Week 12

  1. Week 12 progress:
    • Thesis done and submitted
    • Work on Final Seminar
  1. Goals for week 13:
    • Prepare for final seminar

Semester 2 Week 13

  1. Week 13 progress:
    • Final seminar completed
    • Closed project

See also

Back

Retrieved from "https://www.eleceng.adelaide.edu.au/personal/dabbott/wiki/index.php?title=Cipher_cracking_2018_weekly_progress&oldid=21187"