Difference between revisions of "Malverso Week 9"

Revision as of 06:10, 3 November 2015

Export Information

Version of GenMAPP Builder: 3 build 5 Computer on which export was run: NON003461 (Bio Laptop 2) Postgres Database name: vcholerae_20151027_gmb3build5

UniProt XML filename (give filename and upload and link to compressed file):

• UniProt XML version (The version information can be found at the UniProt News Page):
• UniProt XML download link:V. cholerae download link
• Time taken to import to GenMAPP Builder: 3.90 minutes

GO OBO-XML filename (give filename and upload and link to compressed file):

• GO OBO-XML version (The version information can be found in the file properties after the file downloaded from the GO Download page has been unzipped): Last modified 10/27/2015 3:24 am
• GO OBO-XML download link: [1]
• Time taken to import to GenMAPP Builder: 8.20 minutes
• Time taken to process: 6.45 minutes

GOA filename (give filename and upload and link to compressed file):

• GOA version (News on this page records past releases; current information can be found in the Last modified field on the FTP site): Last modified 10/13/2015 6:31 am
• GOA download link: [2]
• Time taken to import: .07 minutes
  • Note: file was last modified 13 Oct 20016 at 07:31

Name of .gdb file (give filename and upload and link to compressed file):

• Time taken to export: 8 hours, 48 minutes, 26 seconds
  • Start time: 3:55:30 PM
  • End time: 12:43:56 AM
  • This was a much longer export time than others.

Note: Export in progress says 1% but is broken, it will skip up eventually. I also left the classroom before the export completed and Dr.Dahlquist recorded my end time.

TallyEngine

• Run the TallyEngine in GenMAPP Builder and record the number of records for UniProt and GO in the XML data and in the Postgres databases.
  • Choose the menu item Tallies > Run XML and Database Tallies for UniProt and GO...
  • Take a screenshot of the results. Upload the image to the wiki and display it on this page.
  • For more information, see this page.

Using XMLPipeDB match to Validate the XML Results from the TallyEngine

Follow the instructions found on this page to run XMLPipeDB match.

Are your results the same as you got for the TallyEngine? Why or why not?

• Not at first. I got 2738 for the number of vibrio cholerae genes using the command java -jar xmlpipedb-match-1.1.1.jar "VC_[0-9][0-9][0-9][0-9]" < "uniprot-taxonomy%3A243277.xml"
• TallyEngine gave the number 3831 (ordered locus names) for both XML and Postgres.
• This is a big difference.

• So I made a new match command:

java -jar xmlpipedb-match-1.1.1.jar "VC_A?[0-9][0-9][0-9][0-9]" < "uniprot-taxonomy%3A243277.xml"

• This takes into account that many genes have an A in their names. Also, every gene has a double but we only need to count one of each.
• I got the same answer, 3831 when I used this command.

Using SQL Queries to Validate the PostgreSQL Database Results from the TallyEngine

For more information, see this page.

You can also look for counts at the SQL level, using some variation of a select count(*) query. This requires some knowledge of which table received what data. Here’s an initial tip: the gene/name tags in the XML file land in the genenametype table. A query on this table counting values from this table that were marked as ordered locus in the XML file matching the pattern VC_[0-9][0-9][0-9][0-9] would look like this:

select count(*) from genenametype where type = 'ordered locus' and value ~ 'VC_[0-9][0-9][0-9][0-9]';

In pgAdmin III, you can issue these queries by clicking on the pencil/SQL icon in the toolbar, typing the query into the SQL Editor tab, then clicking on the green triangular Play button to run.

Are your results the same as reported by the TallyEngine? Why or why not?

• I got 2737 at first, which is a different number.

• This is because those with A's in their names are not accounted for.

• Using the new SQL pattern:

select count(*) from genenametype where type = 'ordered locus' and value ~ 'VC_A?[0-9][0-9][0-9][0-9]';

• This returned 3831, which is the same value the other methods used.

OriginalRowCounts Comparison

Within the .gdb file, look at the OriginalRowCounts table to see if the database has the expected tables with the expected number of records. Compare the tables and records with a benchmark .gdb file.

• I got 7664 OrderedLocusNames using the OrigonalRowCounts on Microsoft Access for both .gdb files.
• On the OrigonalRowCounts table for my database, the values were often higher than the benchmark. For example, for the GeneOntology and Uniprot tables.
• Looking at the orderedLocusNames in my database, I found that:
  • some of the names begin with VCA, not all begin with VC. ALso, some have underscores and some don't. About twice as many in this table than the tallyEngine results.
  • This is because every gene has a two versions - one with an underscore and one without.

Benchmark .gdb file:

Copy the OriginalRowCounts table from the benchmark and new gdb and paste them here:

Note: The benchmark .gdb file was downloaded at sourceforge

Visual Inspection

Perform visual inspection of individual tables to see if there are any problems.

• Look at the Systems table. Is there a date in the Date field for all gene ID systems present in the database?
• Open the UniProt, RefSeq, and OrderedLocusNames tables. Scroll down through the table. Do all of the IDs look like they take the correct form for that type of ID?

Note:

.gdb Use in GenMAPP

Note:

Putting a gene on the MAPP using the GeneFinder window

• Try a sample ID from each of the gene ID systems. Open the Backpage and see if all of the cross-referenced IDs that are supposed to be there are there.

Note:

Creating an Expression Dataset in the Expression Dataset Manager

• How many of the IDs were imported out of the total IDs in the microarray dataset? How many exceptions were there? Look in the EX.txt file and look at the error codes for the records that were not imported into the Expression Dataset. Do these represent IDs that were present in the UniProt XML, but were somehow not imported? or were they not present in the UniProt XML?

Note:

Coloring a MAPP with expression data

Note:

Running MAPPFinder

Note:

Compare Gene Database to Outside Resource

The OrderedLocusNames IDs in the exported Gene Database are derived from the UniProt XML. It is a good idea to check your list of OrderedLocusNames IDs to see how complete it is using the original source of the data (the sequencing organization, the MOD, etc.) Because UniProt is a protein database, it does not reference any non-protein genome features such as genes that code for functional RNAs, centromeres, telomeres, etc.

Note:

Team Page

Heavy Metal HaterZ

Assignments

• Week 1
• Week 2
• Week 3
• Week 4
• Week 5
• Week 6
• Week 7
• Week 8
• Week 9
• Week 10
• Week 11
• Week 12
• Week 13
• Week 14
• Week 15

Individual Journal Entries

• Malverso User Page (Week 1)
• Week 2
• Week 3
• Week 4
• Week 5
• Week 6
• Week 7
• Week 8
• Week 9
• Week 10
• Week 11
• Week 12
• Week 13
• Week 14
• Week 15

Shared Journal Entries

• Week 1
• Week 2
• Week 3
• Week 4
• Week 5
• Week 6
• Week 7
• Week 8
• Week 9
• Week 10
• Week 11
• Week 12
• Week 13
• Week 14
• Week 15