Difference between revisions of "Nanguiano Week 9"
(→OriginalRowCounts Comparison: add more info) |
(→Visual Inspection: Added information) |
||
Line 110: | Line 110: | ||
* Look at the Systems table. Is there a date in the Date field for all gene ID systems present in the database? | * Look at the Systems table. Is there a date in the Date field for all gene ID systems present in the database? | ||
+ | ** No. The majority of systems to not have a date included in their date field. | ||
* Open the UniProt, RefSeq, and OrderedLocusNames tables. Scroll down through the table. Do all of the IDs look like they take the correct form for that type of ID? | * Open the UniProt, RefSeq, and OrderedLocusNames tables. Scroll down through the table. Do all of the IDs look like they take the correct form for that type of ID? | ||
− | + | ** The UniProt IDs all look correct. | |
− | + | ** The RefSeq IDs also all look correct. | |
+ | ** The IDs in OrderedLocusNames appear to take two forms: one which is of the form VC_####, and one of the form VC_A####. This seems to prove the hypothesis that the difference between Tally Engine and the match/SQL results was due to a difference in searched names. | ||
+ | *** To test this, the command <code>java -jar xmlpipedb-match-1.1.1.jar "VC_A?[0-9][0-9][0-9][0-9]" < uniprot-organism%3A243277.xml</code> returns 3,831 unique matches, the same as the Tally Engine. Running the command <code>select count(*) from genenametype where type = 'ordered locus' and value ~ 'VC_A?[0-9][0-9][0-9][0-9';</code> in SQL also returns 3,831. Therefore, I am decently confident that this was the reason the utilities were returning different results. I am still uncertain as to why SQL returned one less value than the match utility at first, before this new filter. | ||
===.gdb Use in GenMAPP=== | ===.gdb Use in GenMAPP=== |
Revision as of 03:06, 17 September 2016
Contents
- 1 Running GenMAPP Builder
- 2 Quality Assurance
- 3 Links
Running GenMAPP Builder
I followed the process from the Running GenMAPP Builder Tutorial Page.
Software
The following software was utilized:
- Any tool that can unpack .gz and .zip files
- We use 7-zip
- PostgreSQL on Windows (http://www.enterprisedb.com/products-services-training/pgdownload)
- GenMAPP Builder version 3.0.0 build 5 (https://github.com/lmu-bioinformatics/xmlpipedb/releases)
- Java JDK 1.8 64-bit
- Download page
- File to download is: jdk-8u65-windows-x64.exe
- GenMAPP 2 can be downloaded here. The file to download is "GenMAPPv2Setup.exe".
- XMLPipeDB match utility (https://sourceforge.net/projects/xmlpipedb/files/) for counting IDs in XML files
- Microsoft Access or any other tool that can read .mdb files
Quality Assurance
The template for this information was taken from the Gene Database Testing Report Sample Page.
Export Information
Version of GenMAPP Builder: 3.0.0 build 5
Computer on which export was run: Seaver 120-14, HP LV2311 Windows 7 Enterprise
Postgres Database name: V_Cholerae_20160916_gmbuilder03b5
UniProt XML filename (give filename and upload and link to compressed file): uniprot-organism%3A243277.xml
- UniProt XML version (The version information can be found at the UniProt News Page): UniProt Release 2016_08
- UniProt XML download link: http://www.uniprot.org/uniprot/?query=organism:243277
- Time taken to import: 2.92 minutes
- Note: None
GO OBO-XML filename (give filename and upload and link to compressed file): go_daily-termdb.obo-xml
- GO OBO-XML version (The version information can be found in the file properties after the file downloaded from the GO Download page has been unzipped): 10/27/2015, 2:24am
- GO OBO-XML download link: http://geneontology.org/page/download-ontology#Legacy_Downloads
- Time taken to import: 6.87 minutes
- Time taken to process: 4.26 minutes
- Note: The GO OBO-XML file and version is from Anu Varshneya's Week 9 Journal. Currently available GO OBO-XML files do not run properly in GenMAPP builder.
GOA filename (give filename and upload and link to compressed file): 46.V_cholerae_ATCC_39315.goa
- GOA version (News on this page records past releases; current information can be found in the Last modified field on the FTP site): 07/05/16
- GOA download link: http://ftp.ebi.ac.uk/pub/databases/GO/goa/proteomes/46.V_cholerae_ATCC_39315.goa
- Time taken to import: 0.06 minutes
- Note: None
Name of .gdb file (give filename and upload and link to compressed file): Vc-Std_20160916_NA.gdb
- Time taken to export: 5.15 hours
- Start time: 09/16/16, 12:26:08 PM
- End time: 09/16/16, 7:16:23 PM
- Note: There was a time during the export in which the computer was locked and not running the export, from 2:56pm - 4:19pm.
TallyEngine
- Run the TallyEngine in GenMAPP Builder and record the number of records for UniProt and GO in the XML data and in the Postgres databases.
- Choose the menu item Tallies > Run XML and Database Tallies for UniProt and GO...
- Take a screenshot of the results. Upload the image to the wiki and display it on this page.
Using XMLPipeDB match to Validate the XML Results from the TallyEngine
- After entering into the directory containing the match utility and the uniprot xml file, I ran the command
java -jar xmlpipedb-match-1.1.1.jar "VC_[0-9][0-9][0-9][0-9]" < uniprot-organism%3A243277.xml
- It returned a total of 2,738 unique matches.
Are your results the same as you got for the TallyEngine? Why or why not?
- My results were not the same as my results for Tally Engine. Tally Engine returned 3,789 unique matches, while this returned 2,738. This is likely due to potentially due to differently formatted names in the xml file, as we are only checking for names in the structure VC_####.
Using SQL Queries to Validate the PostgreSQL Database Results from the TallyEngine
- Next, I checked the values in SQL to see if they would validate the results from Tally Engine or the match utility.
- In PgAdminIII, I ran the command:
select count(*) from genenametype where type = 'ordered locus' and value ~ 'VC_[0-9][0-9][0-9][0-9]';
- Are your results the same as reported by the TallyEngine? Why or why not?
- The results were not the same as the ones reported by Tally Engine, nor were they the same as the ones reported by the match utility. It returned 2,737 matches, one less than the result returned by the match utility. While it is less than the Tally Engine perhaps due to a different naming convention, I don't fully understand why it would be off by one from the match utility.
OriginalRowCounts Comparison
- Within the .gdb file, look at the OriginalRowCounts table to see if the database has the expected tables with the expected number of records. Compare the tables and records with a benchmark .gdb file.
- Benchmark .gdb file (2010): Vc-Std_External_20101022
Copy the OriginalRowCounts table from the benchmark and new gdb and paste them here:
- Original GDB:
- New GDB:
- The new gdb had 10 additional tables.
- The have the same number of OrderedLocusNames, 7,664, but most of the other values are different.
Note:
Visual Inspection
Perform visual inspection of individual tables to see if there are any problems.
- Look at the Systems table. Is there a date in the Date field for all gene ID systems present in the database?
- No. The majority of systems to not have a date included in their date field.
- Open the UniProt, RefSeq, and OrderedLocusNames tables. Scroll down through the table. Do all of the IDs look like they take the correct form for that type of ID?
- The UniProt IDs all look correct.
- The RefSeq IDs also all look correct.
- The IDs in OrderedLocusNames appear to take two forms: one which is of the form VC_####, and one of the form VC_A####. This seems to prove the hypothesis that the difference between Tally Engine and the match/SQL results was due to a difference in searched names.
- To test this, the command
java -jar xmlpipedb-match-1.1.1.jar "VC_A?[0-9][0-9][0-9][0-9]" < uniprot-organism%3A243277.xml
returns 3,831 unique matches, the same as the Tally Engine. Running the commandselect count(*) from genenametype where type = 'ordered locus' and value ~ 'VC_A?[0-9][0-9][0-9][0-9';
in SQL also returns 3,831. Therefore, I am decently confident that this was the reason the utilities were returning different results. I am still uncertain as to why SQL returned one less value than the match utility at first, before this new filter.
- To test this, the command
.gdb Use in GenMAPP
While the above sections perform quality assurance on the exported Gene Database via verifying ID counts, the "proof in the pudding" is to actually use the Gene Database in GenMAPP. You can follow the instructions in Part 2 of the Vibrio cholerae Microarray Data Analysis to verify that the Gene Database works in GenMAPP. In this case, the emphasis is not on the findings of the data analysis itself, but that the Gene Database functions appropriate in GenMAPP.
For assistance with using the GenMAPP program, the GenMAPP Help is very extensive. To access it within GenMAPP, go to the menu item Help > GenMAPP Help and either browse or search for your topic of interest.
Note:
Putting a gene on the MAPP using the GeneFinder window
- In the main GenMAPP Drafting Board window, left-click on the icon for "Gene" in the upper left corner of the window. Click on the Drafting Board to place the Gene on the MAPP. Now, right-click on the gene to access the GeneFinder window. Type or paste a gene ID into the Gene ID field. Select the appropriate Gene ID system from the drop-down menu and click the Search button. For example, for Vibrio cholerae, you could search for the ID "VC0028", which is an OrderedLocusNames ID. Once the ID has been found, click the OK button to return to the Drafting Board window.
- For the Final Project, you will need to try a sample ID from each of the gene ID systems, not just OrderedLocusNames.
- Open the Backpage by left-clicking on the gene box on the Drafting Board to see if all of the cross-referenced IDs that are supposed to be there are there.
Note:
Creating an Expression Dataset in the Expression Dataset Manager
- How many of the IDs were imported out of the total IDs in the microarray dataset? How many exceptions were there? Look in the EX.txt file and look at the error codes for the records that were not imported into the Expression Dataset. Do these represent IDs that were present in the UniProt XML, but were somehow not imported? or were they not present in the UniProt XML?
Note:
Coloring a MAPP with expression data
Note:
Running MAPPFinder
Note:
Links
Nicole Anguiano
BIOL 367, Fall 2015
Assignment Links
- Week 1 Assignment
- Week 2 Assignment
- Week 3 Assignment
- Week 4 Assignment
- Week 5 Assignment
- Week 6 Assignment
- Week 7 Assignment
- Week 8 Assignment
- Week 9 Assignment
- Week 10 Assignment
- Week 11 Assignment
- Week 12 Assignment
- Week 14 Assignment
- Week 15 Assignment
Individual Journals
- Individual Journal Week 2
- Individual Journal Week 3
- Individual Journal Week 4
- Individual Journal Week 5
- Individual Journal Week 6
- Individual Journal Week 7
- Individual Journal Week 8
- Individual Journal Week 9
- Individual Journal Week 10
- Individual Journal Week 11
- Individual Assessment
- Deliverables