Template:User Page Link - Revision history

Mpetredi: /* Mitchell Petredis */ added Team Name to my template

2013-11-12T04:56:19Z

‎Mitchell Petredis: added Team Name to my template

Mpetredi: Replaced content with "=='''Mitchell Petredis'''=="

2013-10-25T04:21:49Z

Replaced content with "=='''Mitchell Petredis'''=="

Mpetredi: /* Mitchell Petredis */ not sure why my individual assignment posted, but it should now be gone

2013-10-25T04:15:42Z

‎Mitchell Petredis: not sure why my individual assignment posted, but it should now be gone

Mpetredi: added categories

2013-10-25T04:05:55Z

added categories

Mpetredi: /* OriginalRowCounts Comparison */ added note about possible error

2013-10-25T04:02:06Z

‎OriginalRowCounts Comparison: added note about possible error

Mpetredi: /* Mitchell Petredis */ added references link

2013-10-25T03:59:49Z

‎Mitchell Petredis: added references link

Mpetredi: /* Using SQL Queries to Validate the PostgreSQL Database Results from the TallyEngine */ added results from PostgreSQL

2013-10-25T03:56:56Z

‎Using SQL Queries to Validate the PostgreSQL Database Results from the TallyEngine: added results from PostgreSQL

Mpetredi: /* Using XMLPipeDB match to Validate the XML Results from the TallyEngine */ added results from XMLPipeDB Match

2013-10-25T03:51:39Z

‎Using XMLPipeDB match to Validate the XML Results from the TallyEngine: added results from XMLPipeDB Match

Mpetredi: /* OriginalRowCounts Comparison */ added original row counts

2013-10-25T03:47:52Z

‎OriginalRowCounts Comparison: added original row counts

← Older revision		Revision as of 04:56, 12 November 2013
Line 1:		Line 1:
	==[[user:mpetredi\|'''Mitchell Petredis''']]==		==[[user:mpetredi\|'''Mitchell Petredis''']]==
		+
		+	[[[[Team Name]]]]

← Older revision		Revision as of 04:21, 25 October 2013
Line 1:		Line 1:
	==[[user:mpetredi\|'''Mitchell Petredis''']]==		==[[user:mpetredi\|'''Mitchell Petredis''']]==
−
−	~~==Gene Database Information==~~
−
−	~~===Export Information===~~
−
−	~~Version of GenMAPP Builder: gmbuilder-2.0b70~~
−
−	~~Computer on which export was run: Keck Lab desktop~~
−
−	~~Postgres Database name: vc-mpetredi-20131022-gmb2b70~~
−
−	~~UniProt XML filename: Vc-uniprot-mpetredi-20131022.xml~~
−	* UniProt XML version (The version information can be found at [http://uniprot.org/news the UniProt News Page]): Vc-uniprot-mpetredi-20131022.xml located in downloads (for now)
−	* Time taken to import: 8.72 minutes
−
−	~~GO OBO-XML filename: go_daily-termdb-mpetredi-20131022.obo.xml~~
−	* GO OBO-XML version (The version information can be found in the file properties after the file downloaded from the [http://beta.geneontology.org/page/download-ontology GO Download page] has been unzipped):
−	* Time taken to import: ?
−	* Time taken to process: 14.77 minutes
−
−	~~GOA filename: 46.V_cholerae_ATCC_39315._mpetredi_20131022.goa~~
−	* ''Direct download from wiki due to network connectivity issue.''
−	* GOA version (News on [http://www.ebi.ac.uk/GOA/ this page] records past releases; current information can be found in the Last modified field on the [ftp://ftp.ebi.ac.uk/pub/databases/GO/goa/proteomes/ FTP site]):
−	* Time taken to import: 0.16 minutes
−
−	~~Name of .gdb file: Vc-Std_20131022-Mpetredi-gmb2b70.gdb~~
−	* Time taken to export .gdb:
−	* Upload your file and link to it here.
−
−	~~Note: Export started around 11 am; ended at 4:03:36 pm.~~
−
−	~~==TallyEngine==~~
−
−	~~Run the TallyEngine in GenMAPP Builder and record the number of records for UniProt and GO in the XML data and in the PostgreSQL databases (or you can upload and link to a screenshot of the results).~~
−
−	~~[[Image: GMBuilder_Capture_10-24-2013_9.40am.PNG]]~~
−
−	~~== Using XMLPipeDB match to Validate the XML Results from the TallyEngine==~~
−
−	~~[[How_Do_I_Count_Thee%3F_Let_Me_Count_The_Ways#XMLPipeDB_Match \| Follow the instructions found on this page to run XMLPipeDB match.]]~~
−
−	~~[[Image: Java_-jar_unique_IDs_10-24-2013_10.01am.PNG]]~~
−
−	~~Are your results the same as you got for the TallyEngine? Why or why not?~~
−
−	~~Yes, the results are the same because both XMLPipeDB Match and TallyEngine were looking for the same ID tags in VC and VCA.~~
−
−	~~== Using SQL Queries to Validate the PostgreSQL Database Results from the TallyEngine==~~
−
−	~~[[Image: PgAdmin_SQL_command_10-24-2013_10.05am.PNG]]~~
−
−	~~[[How_Do_I_Count_Thee%3F_Let_Me_Count_The_Ways#SQL \| Follow the instructions on this page to query the PostgreSQL Database.]]~~
−
−	~~==OriginalRowCounts Comparison==~~
−
−	~~Within the .gdb file, look at the OriginalRowCounts table to see if the database has the expected tables with the expected number of records. Compare the tables and records with a benchmark .gdb file.~~
−
−	Benchmark .gdb file: (for the [[Week 9]] Assignment, use the "Vc-Std_External_20101022.gdb" as your benchmark, downloadable from [http://sourceforge.net/projects/xmlpipedb/files/V.%20cholerae%20Gene%20Database/V.%20cholerae%2020101022/Vc-Std_External_20101022.zip/download here].
−
−	~~Copy the OriginalRowCounts table and paste it here:~~
−
−	~~[[Image: Micosoft_Access_Capture_10-24-2013_10.20am.PNG]]~~
−
−	~~Note: Data doesn't just magically appear, so the extra value obtained in Microsoft Access must be wrong. Perhaps there is a flaw in GenMAPP~~
−
−	~~==Visual Inspection==~~
−
−	~~Perform visual inspection of individual tables to see if there are any problems.~~
−
−	* Look at the Systems table. Is there a date in the Date field for all gene ID systems present in the database?
−	* Open the UniProt, RefSeq, and OrderedLocusNames tables. Scroll down through the table. Do all of the IDs look like they take the correct form for that type of ID?
−
−	~~Note:~~
−
−	~~==.gdb Use in GenMAPP==~~
−
−	~~Note:~~
−
−	~~===Putting a gene on the MAPP using the GeneFinder window===~~
−
−	* Try a sample ID from each of the gene ID systems. Open the Backpage and see if all of the cross-referenced IDs that are supposed to be there are there.
−
−	~~Note:~~
−
−	~~===Creating an Expression Dataset in the Expression Dataset Manager===~~
−
−	* How many of the IDs were imported out of the total IDs in the microarray dataset? How many exceptions were there? Look in the EX.txt file and look at the error codes for the records that were not imported into the Expression Dataset. Do these represent IDs that were present in the UniProt XML, but were somehow not imported? or were they not present in the UniProt XML?
−
−	~~Note:~~
−
−	~~===Coloring a MAPP with expression data===~~
−
−	~~Note:~~
−
−	~~===Running MAPPFinder===~~
−
−	~~Note:~~
−
−	~~== Compare Gene Database to Outside Resource==~~
−
−	The OrderedLocusNames IDs in the exported Gene Database are derived from the UniProt XML. It is a good idea to check your list of OrderedLocusNames IDs to see how complete it is using the original source of the data (the sequencing organization, the MOD, etc.) Because UniProt is a protein database, it does not reference any non-protein genome features such as genes that code for functional RNAs, centromeres, telomeres, etc.
−
−	~~Note:~~
−
−	~~<!--In this example, this query tells you which IDs are in Riley only:<br>~~
−	~~select Riley.ID from Riley left outer join Blattner on Riley.ID = Blattner.ID where Blattner.ID is null;-->~~
−
−	~~[[Category: Individual Homework]]~~
−	~~[[Category: Journal Entry]]~~

← Older revision		Revision as of 04:15, 25 October 2013
Line 1:		Line 1:
	==[[user:mpetredi\|'''Mitchell Petredis''']]==		==[[user:mpetredi\|'''Mitchell Petredis''']]==
−
−	~~===References===~~
−
−	~~[[Week 9]] and all of its embedded links~~
−
−	~~===Pre-requisites===~~
−
−	~~Download all of these programs/files before continuing.~~
−
−	*[[http://www.postgresql.org/ PostgreSQL]]
−	*[[http://sourceforge.net/projects/xmlpipedb/files/GenMAPP%20Builder/ Latest GenMAPP Builder]]
−	*[[http://www.genmapp.org/download_v2.1.php GenMAPP Classic]]
−	*[[https://sourceforge.net/projects/xmlpipedb/files/ XMLPipeDB match utility]]
−
−	~~===UniProt XML===~~
−
−	~~#Go to [[http://www.uniprot.org/taxonomy/complete-proteomes UniProt Complete Proteomes]]~~
−	~~#Choose a topic from the "Complete proteomes" list~~
−	~~#*I picked [[http://www.uniprot.org/taxonomy/?query=complete:yes%20ancestor:2 Bacteria]]~~
−	~~#Pick a specimen from the list of results and click on "Complete proteome set" under its link.~~
−	~~#*For this exercise, pick [[http://www.uniprot.org/uniprot/?query=organism:243277+keyword:1185 Vibrio cholerae serotype O1 (strain 39315 / El Tor Inaba N16961)]]~~
−	#Once on your specimen page, click on the yellow/orange box in the upper right hand corner called [[http://www.uniprot.org/uniprot/?query=organism%3a243277+keyword%3a1185&format=* Download]] to choose a filetype
−	~~#Prior to selecting a file type, make sure [[http://www.uniprot.org/uniprot/?query=organism%3a243277+keyword%3a1185&format=&compress=yes compressed]] results are enabled by clicking on the link.~~
−	~~#*select [[http://www.uniprot.org/uniprot/?query=organism%3a243277+keyword%3a1185&compress=yes&format=xml XML]]~~
−	~~#**Extract the .gz file by right-clicking on the file and scrolling to 7-zip/Extract here~~
−
−	~~===GOA===~~
−
−	~~#Go to [[http://www.ebi.ac.uk/GOA/downloads UniProt-GOA Downloads]]~~
−	#Current/previous UniProt-GOA files are located [[ftp://ftp.ebi.ac.uk/pub/databases/GO/goa/ here]]. #*NOTE: Google Chrome from the Keck lab can't find the link, and redirects to [[http://www.ebi.ac.uk/ European Bionformatics Institute]] instead.
−	~~#**For now, the V. cholerae GOA file can be downloaded [[https://xmlpipedb.cs.lmu.edu/biodb/fall2013/images/0/0f/46.V_cholerae_ATCC_39315.goa here]]~~
−	~~#Click on the link for [[ftp://ftp.ebi.ac.uk/pub/databases/GO/goa/proteomes/ proteomes]]~~
−	#Pick an organism and right click on its link to download the GO annotations and select "Save target/link as" to save the GOA file. Simply clicking on the link will open the file as a text file within the browser, so it is necessary to use the "Save target/link as" command.
−
−	~~===GO OBO-XML===~~
−
−	~~#Download OBO-XML file from [[http://archive.geneontology.org/latest-termdb/go_daily-termdb.obo-xml.gz here]]~~
−	~~#Use 7zip to extract the file's contents properly (don't use Windows native extraction program).~~
−
−	~~===PostgreSQL===~~
−
−	~~#Make a new database as described in [[Week 6]].~~
−	#In the SQL editor, click on "Open File" in the toolbar and select the gmbuilder.sql from wherever you extracted the GenMAPP Builder package. This imports all the data tables needed to run the SQL commands.
−	~~#Click on the execute button (looks like a green Play button or F5) to load the data tables.~~
−
−	In an effort to save time and reduce redundancy, I followed all the steps in [[https://xmlpipedb.cs.lmu.edu/biodb/fall2013/index.php/Running_GenMAPP_Builder Running GenMAPP Builder]] to the letter with varying file locations. The most important information is below in Gene Database Information.

	==Gene Database Information==		==Gene Database Information==

← Older revision		Revision as of 04:05, 25 October 2013
Line 152:		Line 152:
	<!--In this example, this query tells you which IDs are in Riley only:<br>		<!--In this example, this query tells you which IDs are in Riley only:<br>
	select Riley.ID from Riley left outer join Blattner on Riley.ID = Blattner.ID where Blattner.ID is null;-->		select Riley.ID from Riley left outer join Blattner on Riley.ID = Blattner.ID where Blattner.ID is null;-->
		+
		+	[[Category: Individual Homework]]
		+	[[Category: Journal Entry]]

@@ Line 109: / Line 109: @@
 [[Image: Micosoft_Access_Capture_10-24-2013_10.20am.PNG]]
-Note:
+Note: Data doesn't just magically appear, so the extra value obtained in Microsoft Access must be wrong. Perhaps there is a flaw in GenMAPP
 ==Visual Inspection==

← Older revision		Revision as of 03:59, 25 October 2013
Line 1:		Line 1:
	==[[user:mpetredi\|'''Mitchell Petredis''']]==		==[[user:mpetredi\|'''Mitchell Petredis''']]==
		+
		+	===References===
		+
		+	[[Week 9]] and all of its embedded links

	===Pre-requisites===		===Pre-requisites===

← Older revision		Revision as of 03:56, 25 October 2013
Line 90:		Line 90:

	== Using SQL Queries to Validate the PostgreSQL Database Results from the TallyEngine==		== Using SQL Queries to Validate the PostgreSQL Database Results from the TallyEngine==
		+
		+	[[Image: PgAdmin_SQL_command_10-24-2013_10.05am.PNG]]

	[[How_Do_I_Count_Thee%3F_Let_Me_Count_The_Ways#SQL \| Follow the instructions on this page to query the PostgreSQL Database.]]		[[How_Do_I_Count_Thee%3F_Let_Me_Count_The_Ways#SQL \| Follow the instructions on this page to query the PostgreSQL Database.]]

@@ Line 82: / Line 82: @@
 [[How_Do_I_Count_Thee%3F_Let_Me_Count_The_Ways#XMLPipeDB_Match | Follow the instructions found on this page to run XMLPipeDB match.]]
 Are your results the same as you got for the TallyEngine?  Why or why not?
 == Using SQL Queries to Validate the PostgreSQL Database Results from the TallyEngine==

← Older revision		Revision as of 03:47, 25 October 2013
Line 96:		Line 96:

	Copy the OriginalRowCounts table and paste it here:		Copy the OriginalRowCounts table and paste it here:
		+
		+	[[Image: Micosoft_Access_Capture_10-24-2013_10.20am.PNG]]

	Note:		Note: