Difference between revisions of "Rlegaspi Week 14"

From LMU BioDB 2015
Jump to: navigation, search
(Deleting error messages and replacing them with blank spaces and recorded how many error messages contained on the mastersheet.)
(→‎December 3, 2015 thru December 8, 2015: updated file link)
 
(8 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
{{Heavy Metal HaterZ}}
 
{{Heavy Metal HaterZ}}
  
−
= GenMAPP User Milestones =
+
= Goals for Week 14 =
−
 
+
== Data Preparation and Statistical Analysis ==
−
== Milestone 1: Startup from Original Microarray Paper and Data ==
+
−
 
+
−
# Download microarray data in its “rawest” form that you can find. (Consult with Dr. Dahlquist about this.)
+
−
# Verify that the gene IDs in the microarray data match the chosen species and '''''strain''''' that is being used to create the ''.gdb''. (Needs to be done in conjunction with the QA and Coder.)
+
−
# File management: on your team's home page
+
−
#* Link to the source of the microarray data
+
−
#* Upload the microarray data files to the wiki
+
−
 
+
−
== Milestone 2: Data Preparation ==
+
−
 
+
−
# Create a table or list that shows the correspondence between the samples in the experiment and the files you have downloaded.
+
−
# Determine how many biological or technical replicates, and which samples were labeled with Cy3 or Cy5.
+
 
# Create a Master Raw Data file that contains the IDs and columns of data required for further analysis.
 
# Create a Master Raw Data file that contains the IDs and columns of data required for further analysis.
 
# Consult with Dr. Dahlquist on how to process the data (normalization, statistics)
 
# Consult with Dr. Dahlquist on how to process the data (normalization, statistics)
−
 
−
== Milestone 3: On-going Analysis Cycle ==
 
−
 
−
This milestone represents the on-going work of the GenMAPP user, as the project gradually converges toward a final version of its gene database and compares its own results to the results in the original microarray paper. (Note: These milestones may continue on to Week 13.)
 
−
 
 
# Perform the statistical analysis in Excel.
 
# Perform the statistical analysis in Excel.
 
# Format the gene expression data for import into GenMAPP.
 
# Format the gene expression data for import into GenMAPP.
−
# Import data into GenMAPP, create ColorSets, and run MAPPFinder.
 
−
# Document and take notes on test runs with GenMAPP.
 
−
#* Use the ''EX.txt'' file to help the Coder/Quality Assurance team members to validate the ''.gdb''.
 
−
# Do a journal club outline of the paper so that you can use it in the Discussion section of your group report and your final presentation.
 
−
# Create a ''.mapp'' file showing one pathway that is changed in your data.
 
−
 
−
== Week 14 Notes ==
 
−
*Compiling Raw Data and Statistical Analysis
 
−
*Created a CompiledRawData Sheet.
 
−
*Created a MasterSheet and deleted data with GeneID containing the following:
 
−
*#Number of deletions: 705
 
−
*Found the error message <code>#NUM!</code> and replaced with a blank space ("nothing") and there were 2118 spaces.
 
−
 
−
== Week 12 Feedback ==
 
  
−
* I have reviewed the work that Emily and you did to compile your raw data this last week and want to acknowledge the hard work that you both put in to organize the files and perform the calculations. I'm putting the feedback here on your page and will put a note on Emily's talk page to refer her to this page.
+
= Summary of Progress and Procedure =
−
* In light of the effort that it took to get to this point and to cut down on the additional workload that your particular dataset entails, from this point forward, let's cut out two of the timepoints for the depletion and repletion experiments (ignoring the 10 minute and 40 minute timepoints and keeping the 0, 5, 20, and 60 minute timepoints).
+
== Compiling Raw Data and Statistical Analysis ==
−
*# All of your calculations at this point exist as individual files; you need to compile all of the log ratios you computed into a single file.  In a new workbook, name the first sheet "CompiledRawData".  Name Column A "ID" and copy and paste in the list of IDs from one of your files.
+
=== December 1, 2015 through December 3, 2015 ===
−
*# Create a "MasterIndex" column as follows.  Insert a new column to the right of the "ID" column and name it "MasterIndex". In this column you will create a numerical index of genes so that you can always sort them back into the same order that they started out in.
+
Referencing the Week 12 Feedback provided by Dr. Dahlquist, I was able to begin compiling the raw data on single Excel File:
−
*#* Type a "1" in cell B2 and a "2" in cell B3.
+
*Created an Excel File and named file ''Raw Data Shewanella RARL 20151201''
−
*#* Select both cells. Hover your mouse over the bottom-right corner of the selection until it makes a thin black + sign. Double-click on the + sign to fill the entire column with a series of numbers from 1 to 11520 (the number of spots on the microarray).
+
*Sheet 1 was entitled CompiledRawData Sheet:
−
*# Then you need to copy and paste (values) the "log2" column from your individual files. They should be in order Log2FC-CO-rep1, Log2FC-CO-rep2, Log2FC-C0-rep3, Log2FC-C0-rep4, Log2FC-C5-rep1, etc.
+
**Column 1 = Gene ID
−
*# The next set of manipulations should be performed in a new sheet called "MasterSheet".
+
**Column 2 = MasterIndex (numbered from 1 to 11520)
−
*# Sort the data A-->Z based on the "ID" column.  Delete all rows that have an ID of "Blank", "blank", "gDNA", start with "NC-", start with "ORF". Record how many rows got deleted.
+
**The rest of the columns was log data taken from the 0, 5, 20, and 60 time points respectively
−
*# Some of your cells are going to have error messages in them because of the previous calculations you did.  Find and replace all of these with nothing, record how many cells that is.
+
***7 timepoints total (C0, C5, C20, C60, F5, F20, F60) and 4 replicates total; therefore, 28 total columns of data
−
*# Create a new worksheet called "ScalingCentering" and copy and paste special all of your data into this new sheet. [http://www.openwetware.org/wiki/BIOL398-01/S10:Sample_Microarray_Analysis_Vibrio_cholerae#Normalize_the_log_ratios_for_the_set_of_slides_in_the_experiment You will perform the scaling and centering operations like you did for the ''Vibrio cholerae'' data.]
+
*Created a MasterSheet and copied information from CompiledRawData Sheet into this new sheet
−
*#* Once you have done this, e-mail Dr. Dahlquist and provide a link to your file.  Your microarray data has duplicated spots.  I have a script that will separate these out so that you can average them as technical replicates.
+
**Sorted the Gene ID's in alphabetical order (A-Z) and deleted the rows that contained an ID of '''Blank, blank, gDNA, NC-, or ORF''' resulting in the deletion of '''705 rows.'''
−
*# Create a new worksheet called "Statistics", copy and paste the values into this new sheet.
+
**Deleted the cells that contained the error message of <code>#NUM!</code> which resulted in the deletion of '''2,118 cells.'''
−
*# You will average the technical replicate spots for each sample to get one value for each sample.
+
**Deleted the cells that contained the error message of <code>#DIV/0!</code> which resulted in the deletion of '''23 cells.'''
−
*# You will average the biological replicates for each timepoint to get an average for each timepoint (C0, C5, C20, C60, F5, F20, F60).
+
*Created a ScalingCentering Sheet
−
*# You will take the ratio of the average log ratios so that you have values for the Average Log Ratio of C5/C0, C20/C0, C60/C0, F5/C60, F20/C60, and F60/C60).  Since this is in log space, to take the ratio, you will actually subtract instead of divide.
+
**Copied over data from the MasterSheet
−
*# You will perform a two-sample T test between C5 and C0, C20 and C0, C60 and C0, F5 and C60, etc. and perform the Bonferroni and Benjamini and Hochberg corrections on these p values. This computation is not the same as what we did for ''Vibrio''.  Instead you will use the <code>TTEST</code> function in Excel (see me when you are ready to do this).  The corrections will be the same as what you did before.
+
**Added two rows right below the title row to represent the calculations for the Average and the Standard Deviation of each column
−
*# After that, then you will need to then format the data for GenMAPP and you'll be ready to import it into GenMAPP and run MAPPFinder.  
+
**For the Scaled and Centered Columns of data, typed the equation <code>=(C4-C$2)/C$3</code> in the first cell under scaled and centered column for replicate 1 at timepoint C0, and used Excel functions in order to scale and center the rest of the data with the equation as a template.
−
* Let me know if you have any questions.
+
*Sent this file to Dr. Dahlquist to split the data to get rid of duplicates: [[File:Raw Data Shewanella RARL 20151201.xlsx]]
 +
=== December 3, 2015 thru December 8, 2015 ===
 +
Discrepancies and issues arose with data between my partner Emily Simso and I that were brought to our attention by Dr. Dahlquist; thus, a review of the compiled raw data needed to be done in order for our Excel Sheets to match and to continue on with statistical analysis:
 +
*Repeated procedure from [[File:Raw Data Shewanella RARL 20151201.xlsx]]; however, feedback from Dr. Dahlquist was kept in mind and created a new Excel file called ''UpdatedCompiledRawData Shewanella RARL 20151201 HMH''
 +
**Correct set of timepoints were used in my previous Excel file so no changes were needed to be done there
 +
**Ensured that I had 11520 Gene IDs; in which the last row which had a "Gene ID" as its label was changed to the correct Gene ID that is "SO4357."
 +
**Once all the necessary changes were made and I had touched base with my partner Emily on Sunday and Monday, I had uploaded the file for splitting by Dr. Dahlquist: [[File:UpdatedCompiledRawData Shewanella RARL 20151201 HMH.xlsx]]
 +
*Split data was received and posted as a file on our team's file page by Dr. Dahlquist: [[File:UpdatedCompiledRawData Shewanella RARL 20151201 HMH forsplitting.xlsx]]
 +
**Downloaded this file and copied the sheets of data into a new Excel file entitled ''StatisticalAnalysis Shewanella RARL 20151207 HMH''
 +
**Created a new sheet called Averages
 +
***Averaged together the replicate data from the two spots that are now split and used the equation <code>=AVERAGE(C2,AG2)</code> under the column for C0 replicate 1
 +
***Used excel to copy this equation to the entire column and get a derivative of the equation copied for the other columns of averages for each replicate
 +
**Created a new sheet called Statistics
 +
***Copied and pasted values from the Averages sheet into this new sheet
 +
***Computed the average of the biological replicates for each treatment, biological average was calculated with the following equation for C0: <code>=AVERAGE(C2:F2)</code> and a derivative of this equation was used for every timepoint.
 +
***Calculated the average log ratios of C5/C0, C20/C0, C60/C0, F5/C60, F20/C60, and F60/C60
 +
****Since its in log space, I just needed to subtract the average from the C5 to the average from the C0.
 +
***Performed a two-sample T test between C5 and C0, C20 and C0, C60 and C0, F5 and C60, and so on:
 +
  =TTEST(<range of cells containing the biological replicates for C0>, <range of cells containing the biological replicates for C5>, 2,3]
 +
: This will returned the p value and uploaded the file to the team's file page to be reviewed by Dr. Dahlquist, while performing a sanity check: [[File:StatisticalAnalysis Shewanella RARL 20151207 HMH.xlsx]]
  
−
''&mdash; [[User:Kdahlquist|Kdahlquist]] ([[User talk:Kdahlquist|talk]]) 13:34, 24 November 2015 (PST)''
+
= External Links =
 +
{{Template:Rlegaspi}}

Latest revision as of 07:39, 8 December 2015

HeavyMetal.jpg

Shewanella oneidensis

Our Gene Database Testing Report

Group Paper - File:Final Report 20151218 2 HMH.docx

Group Members

Important Links

Our Files

Our Deliverables

Gene Database Project Links
Overview Deliverables Reference Format Guilds Project Manager GenMAPP User Quality Assurance Coder
Teams Heavy Metal HaterZ The Class Whoopers GÉNialOMICS Oregon Trail Survivors
Individual Journal Entries
Mary Alverson Week 11 Week 12 Week 14 Week 15
Emily Simso Week 11 Week 12 Week 14 Week 15
Ron Legaspi Week 11 Week 12 Week 14 Week 15
Josh Kuroda Week 11 Week 12 Week 14 Week 15

Goals for Week 14

Data Preparation and Statistical Analysis

  1. Create a Master Raw Data file that contains the IDs and columns of data required for further analysis.
  2. Consult with Dr. Dahlquist on how to process the data (normalization, statistics)
  3. Perform the statistical analysis in Excel.
  4. Format the gene expression data for import into GenMAPP.

Summary of Progress and Procedure

Compiling Raw Data and Statistical Analysis

December 1, 2015 through December 3, 2015

Referencing the Week 12 Feedback provided by Dr. Dahlquist, I was able to begin compiling the raw data on single Excel File:

  • Created an Excel File and named file Raw Data Shewanella RARL 20151201
  • Sheet 1 was entitled CompiledRawData Sheet:
    • Column 1 = Gene ID
    • Column 2 = MasterIndex (numbered from 1 to 11520)
    • The rest of the columns was log data taken from the 0, 5, 20, and 60 time points respectively
      • 7 timepoints total (C0, C5, C20, C60, F5, F20, F60) and 4 replicates total; therefore, 28 total columns of data
  • Created a MasterSheet and copied information from CompiledRawData Sheet into this new sheet
    • Sorted the Gene ID's in alphabetical order (A-Z) and deleted the rows that contained an ID of Blank, blank, gDNA, NC-, or ORF resulting in the deletion of 705 rows.
    • Deleted the cells that contained the error message of #NUM! which resulted in the deletion of 2,118 cells.
    • Deleted the cells that contained the error message of #DIV/0! which resulted in the deletion of 23 cells.
  • Created a ScalingCentering Sheet
    • Copied over data from the MasterSheet
    • Added two rows right below the title row to represent the calculations for the Average and the Standard Deviation of each column
    • For the Scaled and Centered Columns of data, typed the equation =(C4-C$2)/C$3 in the first cell under scaled and centered column for replicate 1 at timepoint C0, and used Excel functions in order to scale and center the rest of the data with the equation as a template.
  • Sent this file to Dr. Dahlquist to split the data to get rid of duplicates: File:Raw Data Shewanella RARL 20151201.xlsx

December 3, 2015 thru December 8, 2015

Discrepancies and issues arose with data between my partner Emily Simso and I that were brought to our attention by Dr. Dahlquist; thus, a review of the compiled raw data needed to be done in order for our Excel Sheets to match and to continue on with statistical analysis:

  • Repeated procedure from File:Raw Data Shewanella RARL 20151201.xlsx; however, feedback from Dr. Dahlquist was kept in mind and created a new Excel file called UpdatedCompiledRawData Shewanella RARL 20151201 HMH
    • Correct set of timepoints were used in my previous Excel file so no changes were needed to be done there
    • Ensured that I had 11520 Gene IDs; in which the last row which had a "Gene ID" as its label was changed to the correct Gene ID that is "SO4357."
    • Once all the necessary changes were made and I had touched base with my partner Emily on Sunday and Monday, I had uploaded the file for splitting by Dr. Dahlquist: File:UpdatedCompiledRawData Shewanella RARL 20151201 HMH.xlsx
  • Split data was received and posted as a file on our team's file page by Dr. Dahlquist: File:UpdatedCompiledRawData Shewanella RARL 20151201 HMH forsplitting.xlsx
    • Downloaded this file and copied the sheets of data into a new Excel file entitled StatisticalAnalysis Shewanella RARL 20151207 HMH
    • Created a new sheet called Averages
      • Averaged together the replicate data from the two spots that are now split and used the equation =AVERAGE(C2,AG2) under the column for C0 replicate 1
      • Used excel to copy this equation to the entire column and get a derivative of the equation copied for the other columns of averages for each replicate
    • Created a new sheet called Statistics
      • Copied and pasted values from the Averages sheet into this new sheet
      • Computed the average of the biological replicates for each treatment, biological average was calculated with the following equation for C0: =AVERAGE(C2:F2) and a derivative of this equation was used for every timepoint.
      • Calculated the average log ratios of C5/C0, C20/C0, C60/C0, F5/C60, F20/C60, and F60/C60
        • Since its in log space, I just needed to subtract the average from the C5 to the average from the C0.
      • Performed a two-sample T test between C5 and C0, C20 and C0, C60 and C0, F5 and C60, and so on:
=TTEST(<range of cells containing the biological replicates for C0>, <range of cells containing the biological replicates for C5>, 2,3]
This will returned the p value and uploaded the file to the team's file page to be reviewed by Dr. Dahlquist, while performing a sanity check: File:StatisticalAnalysis Shewanella RARL 20151207 HMH.xlsx

External Links

Ron Legaspi
BIOL 367, Fall 2015

Assignment Links
Individual Weekly Journals
Shared Weekly Journals
Retrieved from "https://xmlpipedb.lmucs.io/biodb/fall2015/index.php?title=Rlegaspi_Week_14&oldid=7488"