Cipher Cracking 2019 - Revision history

A1684198: /* Semester 2 */

2019-11-10T18:01:20Z

Semester 2

← Older revision		Revision as of 03:31, 11 November 2019
Line 57:		Line 57:
	* CD or stick containing your whole project directories (Week 13)		* CD or stick containing your whole project directories (Week 13)
	* YouTube video (Week 13) - add the URL to this wiki		* YouTube video (Week 13) - add the URL to this wiki
	** https://~~www~~.~~youtube.com~~/~~insert_here~~		** https://youtu.be/FcJbR_diNMg

	== Weekly progress and questions ==		== Weekly progress and questions ==

A1684198: /* See also */

2019-11-09T05:07:31Z

A1722261: /* Semester 2 */

2019-11-06T21:07:47Z

Semester 2

← Older revision		Revision as of 06:37, 7 November 2019
Line 51:		Line 51:
	** [[File:141 Final Seminar2019.pdf]]		** [[File:141 Final Seminar2019.pdf]]
	* Final report (Week 11) - only one report needed in wiki format		* Final report (Week 11) - only one report needed in wiki format
	** [[Final ~~Cipher cracking report~~ 2019]]		** [[Final Report/Thesis 2019]]
	* Poster (Week 12) - one poster only needed		* Poster (Week 12) - one poster only needed
	** [[File:141 Poster2019.pdf]]		** [[File:141 Poster2019.pdf]]

A1684198: /* See also */

2019-11-06T07:54:18Z

A1065010: /* Expectations */

2019-03-31T02:08:14Z

Expectations

← Older revision		Revision as of 11:38, 31 March 2019
Line 73:		Line 73:

	It is perfectly acceptable to have a long list of ideas that didn't work, provide they are carefully tested in a structured way. Finding things that don't work is part of the scientific process. When we don't know what is supposed to work or not, being able to eliminate ideas that don't work is still very exciting and worthwhile.		It is perfectly acceptable to have a long list of ideas that didn't work, provide they are carefully tested in a structured way. Finding things that don't work is part of the scientific process. When we don't know what is supposed to work or not, being able to eliminate ideas that don't work is still very exciting and worthwhile.

			* It is important to regularly see your main supervisors. Don't let more than 2 week go by without them seeing your face briefly.

			* You should be making at least one formal progress meeting with supervisors per month. It does not strictly have to be exactly a month, but roughly each month you should be in a position to show some progress and have some problems and difficulties to discuss. On the other hand the meetings can be very frequent in periods when you have a lot of activity and progress to show.

			* The onus is on you to drive the meetings, make the appointments, and set them up.

	== Relationship to possible career path==		== Relationship to possible career path==

A1065010 at 02:03, 31 March 2019

2019-03-31T02:03:15Z

← Older revision		Revision as of 11:33, 31 March 2019
Line 31:		Line 31:
	* Using a known good DNA reference file, deliberately drop out those SNPs that are missing in the Somerton Man. As you know the groundtruth in this case, you can investigate which padding methods failed and which produced partial good results.		* Using a known good DNA reference file, deliberately drop out those SNPs that are missing in the Somerton Man. As you know the groundtruth in this case, you can investigate which padding methods failed and which produced partial good results.

	* Another padding trick is you might introduce ~~opposite homozygotes~~ (~~ie.~~ AA, GG, TT, CC) in various places to beef up the number of ~~pairs~~. These segments might distort the relationship estimates, but if you keep them to smaller segments, they might not make a difference. You can test this out on the good DNA file that has been deliberately stripped.		* Another padding trick is you might introduce homozygous pairs (AA, GG, TT, CC) in various places to beef up the number of SNPs. These segments might distort the relationship estimates, but if you keep them to smaller segments, they might not make a difference. You can test this out on the good DNA file that has been deliberately stripped.

	* Further to padding by just the minimum amount, you can then investigate if you can pad by more by finding SNPs that are common to males or have a high liklihood in males. You can test this on good DNA files.		* Further to padding by just the minimum amount, you can then investigate if you can pad by more by finding SNPs that are common to males or have a high liklihood in males. You can test this on good DNA files.
Line 51:		Line 51:
	** [[File:141 Final Seminar2019.pdf]]		** [[File:141 Final Seminar2019.pdf]]
	* Final report (Week 11) - only one report needed in wiki format		* Final report (Week 11) - only one report needed in wiki format
	** [[Final ~~Report/Thesis~~ 2019]]		** [[Final Cipher cracking report 2019]]
	* Poster (Week 12) - one poster only needed		* Poster (Week 12) - one poster only needed
	** [[File:141 Poster2019.pdf]]		** [[File:141 Poster2019.pdf]]
Line 87:		Line 87:
	* Guanchen Li's Master's thesis [[File:Thesis_a1652167_Guanchen_Li.pdf]]'		* Guanchen Li's Master's thesis [[File:Thesis_a1652167_Guanchen_Li.pdf]]'
	* [https://genesis.gedmatch.com/login1.php Gedmatch Genesis]		* [https://genesis.gedmatch.com/login1.php Gedmatch Genesis]
			* [https://hirisplex.erasmusmc.nl/ Hair and eye colour calculator]

	==Back==		==Back==

A1065010: /* Possible extension */

2019-03-31T01:30:31Z

Possible extension

← Older revision		Revision as of 11:00, 31 March 2019
Line 67:		Line 67:

	==Possible extension==		==Possible extension==
	* If there is time we can get hold of the original FASTQ file and perform statistical tests on it. And you can attempt statistical imputation to infer missing ~~bases pairs~~.		* If there is time we can get hold of the original FASTQ file and perform statistical tests on it. And you can attempt statistical imputation to infer missing SNPs. Idea behind imputation is like what we do in electronic engineering when we do error correction in digital communication systems. For example if a communication system transmits the sentence "the cat sat on the mat" and we receive "th* ct st n th m*t" with drop outs in information, we can still reconstruct the sentence. You can think of a DNA file as being like a long message that also suffers drop outs, and so you can use statistical techniques to infer values. We electrical engineering we call this "error correction" but in bioinformatics this is called "imputation."

	== Expectations ==		== Expectations ==

A1065010: /* Expectations */

2019-03-31T01:20:06Z

Expectations

← Older revision		Revision as of 10:50, 31 March 2019
Line 72:		Line 72:
	We don't really expect you to find the killer or identify the Somerton Man, though that would be cool if you do and you'll become very famous overnight. To get good marks we expect you to show a logical engineering approach to squeezing information out of the data, and using known good files to always groundtruth each idea.		We don't really expect you to find the killer or identify the Somerton Man, though that would be cool if you do and you'll become very famous overnight. To get good marks we expect you to show a logical engineering approach to squeezing information out of the data, and using known good files to always groundtruth each idea.

	It is perfectly acceptable to have a long list of ideas that didn't work, provide they are carefully ~~test~~ in a structured way. Finding things that don't work is part of the scientific process. When we ~~know~~ know what is supposed to work or not, being able to eliminate ideas that don't work is still very exciting and worthwhile.		It is perfectly acceptable to have a long list of ideas that didn't work, provide they are carefully tested in a structured way. Finding things that don't work is part of the scientific process. When we don't know what is supposed to work or not, being able to eliminate ideas that don't work is still very exciting and worthwhile.

	== Relationship to possible career path==		== Relationship to possible career path==

A1065010: /* Specific tasks */

2019-03-31T01:15:43Z

Specific tasks

← Older revision		Revision as of 10:45, 31 March 2019
Line 19:		Line 19:
	==Specific tasks==		==Specific tasks==

	We have supplied you with a file of about 700,000 rs numbers and the corresponding ~~SNP base pairs~~ for the Somerton Man. Unfortunately there's a very high drop out rate. The idea is to ~~ground truth~~ this data, characterize it, and squeeze out of it any information you can. The idea is to find out what the data can tell us and also what id definitely fails at telling us. We need to know both. Here are some ideas to get you started:		We have supplied you with a file of about 700,000 rs and i numbers and the corresponding SNPs for the Somerton Man. Unfortunately there's a very high drop out rate. The idea is to groundtruth this data, characterize it, and squeeze out of it any information you can. The idea is to find out what the data can tell us and also what it definitely fails at telling us. We need to know both. Here are some ideas to get you started:

	* Write a script to count the rs numbers in each chromosome. Count the base pairs in each chromosome. Tabulate these results. Do this quickly.		* Write a script to count the rs and i numbers in each chromosome. Count the base pairs in each chromosome. Tabulate these results. Do this quickly.

	* Create a synthetic "random human" file to get some idea what random ~~base pairs~~ do. Upload this in research mode on Gedmatch Genesis and test its characteristics. Does it actually link to any humans?		* Create a synthetic "random human" file to get some idea what random SNPs do. Upload this in research mode on Gedmatch Genesis and test its characteristics. Does it actually link to any humans?

	* Create a ~~synthetics~~ human that say has all A's or a short sequence of ~~base pairs~~ that is repeated over and over periodically. How do these files behave?		* Create a synthetic human that say has all A's or a short sequence of SNPs that is repeated over and over periodically. How do these files behave?

	* Upload the Somerton Man in research mode. Find out which chromosomes are under the minimum required based papers. Try many different ways of padding these base pairs with dummy data in order to meet the minimum. To are looking to find the padding method that influences the results the least.		* Upload the Somerton Man in research mode. Find out which chromosomes are under the minimum required based papers. Try many different ways of padding these base pairs with dummy data in order to meet the minimum. You are looking to find the padding method that influences the results the least.

	* Using a known good DNA reference file, deliberately drop out those SNPs ~~the~~ are missing in the Somerton Man. As you know the groundtruth in this case, you can investigate which padding methods failed and which produced partial results.		* Using a known good DNA reference file, deliberately drop out those SNPs that are missing in the Somerton Man. As you know the groundtruth in this case, you can investigate which padding methods failed and which produced partial good results.

	* Another padding trick is you might introduce opposite homozygotes in various places to beef up the number of pairs. These segments might distort the relationship estimates, but if you keep them to smaller segments, they might not make a difference. You can test this out on the good DNA file that has been deliberately stripped.		* Another padding trick is you might introduce opposite homozygotes (ie. AA, GG, TT, CC) in various places to beef up the number of pairs. These segments might distort the relationship estimates, but if you keep them to smaller segments, they might not make a difference. You can test this out on the good DNA file that has been deliberately stripped.

	* Further to padding by just the minimum amount, you can then investigate if you can pad by more by finding ~~basepairs~~ that are common to males or have a high liklihood in males. You can ~~have~~ good DNA files ~~you can test this on~~.		* Further to padding by just the minimum amount, you can then investigate if you can pad by more by finding SNPs that are common to males or have a high liklihood in males. You can test this on good DNA files.

	* You can go to ~~an rs number~~ database that ~~describes~~ the ~~effect of~~ each ~~pair~~ (eg. if it ~~affect~~ eyecolour, a disease, or whatever). You can then create you own file containing all the rs descriptions for the Somerton Man. Then you can write your own software to datamine this file for interesting physiological features, characteristics, and diseases.		* You can go to a SNP database that is indexed by rs and i numbers and find the affect each SNP has (eg. if it affects eyecolour, a disease, or whatever). You can then create you own file containing all the descriptions for the Somerton Man. Then you can write your own software to datamine this file for interesting physiological features, characteristics, and diseases.

	==Deliverables==		==Deliverables==

A1065010: /* General project description */

2019-03-31T01:09:29Z

General project description

← Older revision		Revision as of 10:39, 31 March 2019
Line 15:		Line 15:
	The idea of this project is to advance the techniques of human identification by focusing on a difficult case that that is too hard for current techniques. You can read the details about the dead body and the circumstances [http://en.wikipedia.org/wiki/Taman_Shud_Case here.]		The idea of this project is to advance the techniques of human identification by focusing on a difficult case that that is too hard for current techniques. You can read the details about the dead body and the circumstances [http://en.wikipedia.org/wiki/Taman_Shud_Case here.]

	We also want you to bring the skills of an electrical engineer to bear on the area of e-forensics and bioinformatics see if you can apply these to ~~other areas~~ of the case ~~(eg~~. analyzing degraded DNA sequences).		We also want you to bring the skills of an electrical engineer to bear on the area of e-forensics and bioinformatics see if you can apply these to an important area of the case, ie. analyzing degraded DNA sequences.

	==Specific tasks==		==Specific tasks==

← Older revision		Revision as of 14:37, 9 November 2019
Line 84:		Line 84:

	==See also==		==See also==
	* [[Final Report/Thesis 2018]]
	* [[Final Report/Thesis 2019]]		* [[Final Report/Thesis 2019]]
	*[https://github.com/ratherto/Who-killed-the-Somerton-Man-2019 Codes used in this project]		*[https://github.com/ratherto/Who-killed-the-Somerton-Man-2019 Codes used in this project]

← Older revision		Revision as of 17:24, 6 November 2019
Line 86:		Line 86:
	* [[Final Report/Thesis 2018]]		* [[Final Report/Thesis 2018]]
	* [[Final Report/Thesis 2019]]		* [[Final Report/Thesis 2019]]
			*[https://github.com/ratherto/Who-killed-the-Somerton-Man-2019 Codes used in this project]

	== References and useful resources==		== References and useful resources==