Difference between revisions of "Rlegaspi Week 14"

From LMU BioDB 2015
Jump to: navigation, search
(Copying and pasting procedure from Dr. D.)
(Summary of progress was updated to the point in which I acquired the split data; however, need to talk about what I did with data.)
Line 1: Line 1:
 
= Goals for Week 14 =
 
= Goals for Week 14 =
−
== Milestone 2: Data Preparation ==
+
== Data Preparation and Statistical Analysis ==
 
# Create a Master Raw Data file that contains the IDs and columns of data required for further analysis.
 
# Create a Master Raw Data file that contains the IDs and columns of data required for further analysis.
 
# Consult with Dr. Dahlquist on how to process the data (normalization, statistics)
 
# Consult with Dr. Dahlquist on how to process the data (normalization, statistics)
 
# Perform the statistical analysis in Excel.
 
# Perform the statistical analysis in Excel.
 
# Format the gene expression data for import into GenMAPP.
 
# Format the gene expression data for import into GenMAPP.
−
# Import data into GenMAPP, create ColorSets, and run MAPPFinder.
 
−
# Document and take notes on test runs with GenMAPP.
 
−
#* Use the ''EX.txt'' file to help the Coder/Quality Assurance team members to validate the ''.gdb''.
 
−
# Do a journal club outline of the paper so that you can use it in the Discussion section of your group report and your final presentation.
 
−
# Create a ''.mapp'' file showing one pathway that is changed in your data.
 
  
−
= Summary of Progress =
+
= Summary of Progress and Procedure =
−
==Compiling Raw Data and Statistical Analysis==
+
== Compiling Raw Data and Statistical Analysis ==
−
Referencing the Week 12 Feedback provided by Dr. Dahlquist, I was able to begin compiling the raw data on single data sheet
+
=== December 1, 2015 through December 3, 2015 ===
 +
Referencing the Week 12 Feedback provided by Dr. Dahlquist, I was able to begin compiling the raw data on single Excel File:
 +
*Created an Excel File and named file ''Raw Data Shewanella RARL 20151201''
 +
*Sheet 1 was entitled CompiledRawData Sheet:
 +
**Column 1 = Gene ID
 +
**Column 2 = MasterIndex (numbered from 1 to 11520)
 +
**The rest of the columns was log data taken from the 0, 5, 20, and 60 time points respectively
 +
***7 timepoints total (C0, C5, C20, C60, F5, F20, F60) and 4 replicates total; therefore, 28 total columns of data
 +
*Created a MasterSheet and copied information from CompiledRawData Sheet into this new sheet
 +
**Sorted the Gene ID's in alphabetical order (A-Z) and deleted the rows that contained an ID of '''Blank, blank, gDNA, NC-, or ORF''' resulting in the deletion of '''705 rows.'''
 +
**Deleted the cells that contained the error message of <code>#NUM!</code> which resulted in the deletion of '''2,118 cells.'''
 +
**Deleted the cells that contained the error message of <code>#DIV/0!</code> which resulted in the deletion of '''23 cells.'''
 +
*Created a ScalingCentering Sheet
 +
**Copied over data from the MasterSheet
 +
**Added two rows right below the title row to represent the calculations for the Average and the Standard Deviation of each column
 +
**For the Scaled and Centered Columns of data, typed the equation <code>=(C4-C$2)/C$3</code> in the first cell under scaled and centered column for replicate 1 at timepoint C0, and used Excel functions in order to scale and center the rest of the data with the equation as a template.
 +
*Sent this file to Dr. Dahlquist to split the data to get rid of duplicates: [[File:Raw Data Shewanella RARL 20151201.xlsx]]
 +
=== December 3, 2015 thru December 8, 2015 ===
 +
Discrepancies and issues arose with data between my partner Emily Simso and I that were brought to our attention by Dr. Dahlquist; thus, a review of the compiled raw data needed to be done in order for our Excel Sheets to match and to continue on with statistical analysis:
 +
*Repeated procedure from [[File:Raw Data Shewanella RARL 20151201.xlsx]]; however, feedback from Dr. Dahlquist was kept in mind and created a new Excel file called ''UpdatedCompiledRawData Shewanella RARL 20151201 HMH''
 +
**Correct set of timepoints were used in my previous Excel file so no changes were needed to be done there
 +
**Ensured that I had 11520 Gene IDs; in which the last row which had a "Gene ID" as its label was changed to the correct Gene ID that is "SO4357."
 +
**Once all the necessary changes were made and I had touched base with my partner Emily on Sunday and Monday, I had uploaded the file for splitting by Dr. Dahlquist: [[File:UpdatedCompiledRawData Shewanella RARL 20151201 HMH.xlsx]]
 +
*Split data was received and posted as a file on our team's file page by Dr. Dahlquist: [[UpdatedCompiledRawData_Shewanella_RARL_20151201_HMH_forsplitting.xlsx]]
 +
**Downloaded this file and copied the sheets of data into a new Excel file entitled ''StatisticalAnalysis Shewanella RARL 20151207 HMH''
 +
**
  
−
*Created a CompiledRawData Sheet.
+
[[File:StatisticalAnalysis Shewanella RARL 20151207 HMH.xlsx]]
−
*Created a MasterSheet and deleted data with GeneID containing the following:
+
−
*#Number of deletions: 705
+
−
*Found the error message <code>#NUM!</code> and replaced with a blank space ("nothing") and there were 2118 replacements made.
+
−
*Found the error message <code>#DIV/0!</code> and replaced with blank space ("nothing") and there were 23 replacements made.
+
  
−
File was sent to Dr. Dahlquist: [[File:Raw Data Shewanella RARL 20151201.xlsx]]
+
You now need to do the following:
 +
# Average together the replicate data from the two spots that are now split. This means that you need to average the "Log2FC-C0-rep1-scaledandcentered" in cell C2 with the value in cell AG2, for example.
 +
# Copy and paste special > paste values into a new sheet called "statistics".
 +
# Compute the average of the biological replicates for each treatment and timepoint.  For example, average together all four biological replicates for Log2FC-C0.  Repeat for each timepoint.
 +
# Compute the ratio of the average log ratios so that you have values for the Average Log Ratio of C5/C0, C20/C0, C60/C0, F5/C60, F20/C60, and F60/C60).  Since this is in log space, to take the ratio, you will actually subtract instead of divide.
 +
# Perform a two-sample T test between C5 and C0, C20 and C0, C60 and C0, F5 and C60, etc.  Use the equation:
 +
=TTEST(<range of cells containing the biological replicates for C0>, <range of cells containing the biological replicates for C5>, 2,3]
 +
: This will return the p value.  Send me the link to the file at this point so I can check the results.  You can also perform the sanity check.  Let me know how it goes.
  
−
== Week 12 Feedback ==
+
914 instances of error "#DIV/0!" replaced with a blank cell.
  
−
* In light of the effort that it took to get to this point and to cut down on the additional workload that your particular dataset entails, from this point forward, let's cut out two of the timepoints for the depletion and repletion experiments (ignoring the 10 minute and 40 minute timepoints and keeping the 0, 5, 20, and 60 minute timepoints).
 
−
*# All of your calculations at this point exist as individual files; you need to compile all of the log ratios you computed into a single file.  In a new workbook, name the first sheet "CompiledRawData".  Name Column A "ID" and copy and paste in the list of IDs from one of your files.
 
−
*# Create a "MasterIndex" column as follows.  Insert a new column to the right of the "ID" column and name it "MasterIndex". In this column you will create a numerical index of genes so that you can always sort them back into the same order that they started out in.
 
−
*#* Type a "1" in cell B2 and a "2" in cell B3.
 
−
*#* Select both cells. Hover your mouse over the bottom-right corner of the selection until it makes a thin black + sign. Double-click on the + sign to fill the entire column with a series of numbers from 1 to 11520 (the number of spots on the microarray).
 
−
*# Then you need to copy and paste (values) the "log2" column from your individual files.  They should be in order Log2FC-CO-rep1, Log2FC-CO-rep2, Log2FC-C0-rep3, Log2FC-C0-rep4, Log2FC-C5-rep1, etc.
 
−
*# The next set of manipulations should be performed in a new sheet called "MasterSheet".
 
 
*# Sort the data A-->Z based on the "ID" column.  Delete all rows that have an ID of "Blank", "blank", "gDNA", start with "NC-", start with "ORF".  Record how many rows got deleted.
 
*# Sort the data A-->Z based on the "ID" column.  Delete all rows that have an ID of "Blank", "blank", "gDNA", start with "NC-", start with "ORF".  Record how many rows got deleted.
 
*# Some of your cells are going to have error messages in them because of the previous calculations you did.  Find and replace all of these with nothing, record how many cells that is.
 
*# Some of your cells are going to have error messages in them because of the previous calculations you did.  Find and replace all of these with nothing, record how many cells that is.
Line 43: Line 59:
 
*# After that, then you will need to then format the data for GenMAPP and you'll be ready to import it into GenMAPP and run MAPPFinder.  
 
*# After that, then you will need to then format the data for GenMAPP and you'll be ready to import it into GenMAPP and run MAPPFinder.  
 
* Let me know if you have any questions.
 
* Let me know if you have any questions.
−
 
−
== 2015-12-07 ==
 
−
 
−
* A new file with the split data has been uploaded to your team's files page: UpdatedCompiledRawData_Shewanella_RARL_20151201_HMH_forsplitting.xlsx
 
−
** Note that this file is based on "UpdatedCompiledRawData_Shewanella_RARL_20151201_HMH.xlsx".  I still found an error in the other version of the file that there was a gene called "Gene ID" on the CompiledRawData sheet.  This led to a missing gene on the MasterSheet and a discrepancy in the data for the scaling and centering between the two files in the fourth decimal place.
 
−
 
−
You now need to do the following:
 
−
# Average together the replicate data from the two spots that are now split.  This means that you need to average the "Log2FC-C0-rep1-scaledandcentered" in cell C2 with the value in cell AG2, for example.
 
−
# Copy and paste special > paste values into a new sheet called "statistics".
 
−
# Compute the average of the biological replicates for each treatment and timepoint.  For example, average together all four biological replicates for Log2FC-C0.  Repeat for each timepoint.
 
−
# Compute the ratio of the average log ratios so that you have values for the Average Log Ratio of C5/C0, C20/C0, C60/C0, F5/C60, F20/C60, and F60/C60).  Since this is in log space, to take the ratio, you will actually subtract instead of divide.
 
−
# Perform a two-sample T test between C5 and C0, C20 and C0, C60 and C0, F5 and C60, etc.  Use the equation:
 
−
=TTEST(<range of cells containing the biological replicates for C0>, <range of cells containing the biological replicates for C5>, 2,3]
 
−
: This will return the p value.  Send me the link to the file at this point so I can check the results.  You can also perform the sanity check.  Let me know how it goes.
 
−
 
−
914 instances of error "#DIV/0!" replaced with a blank cell.
 

Revision as of 07:09, 8 December 2015

Goals for Week 14

Data Preparation and Statistical Analysis

  1. Create a Master Raw Data file that contains the IDs and columns of data required for further analysis.
  2. Consult with Dr. Dahlquist on how to process the data (normalization, statistics)
  3. Perform the statistical analysis in Excel.
  4. Format the gene expression data for import into GenMAPP.

Summary of Progress and Procedure

Compiling Raw Data and Statistical Analysis

December 1, 2015 through December 3, 2015

Referencing the Week 12 Feedback provided by Dr. Dahlquist, I was able to begin compiling the raw data on single Excel File:

  • Created an Excel File and named file Raw Data Shewanella RARL 20151201
  • Sheet 1 was entitled CompiledRawData Sheet:
    • Column 1 = Gene ID
    • Column 2 = MasterIndex (numbered from 1 to 11520)
    • The rest of the columns was log data taken from the 0, 5, 20, and 60 time points respectively
      • 7 timepoints total (C0, C5, C20, C60, F5, F20, F60) and 4 replicates total; therefore, 28 total columns of data
  • Created a MasterSheet and copied information from CompiledRawData Sheet into this new sheet
    • Sorted the Gene ID's in alphabetical order (A-Z) and deleted the rows that contained an ID of Blank, blank, gDNA, NC-, or ORF resulting in the deletion of 705 rows.
    • Deleted the cells that contained the error message of #NUM! which resulted in the deletion of 2,118 cells.
    • Deleted the cells that contained the error message of #DIV/0! which resulted in the deletion of 23 cells.
  • Created a ScalingCentering Sheet
    • Copied over data from the MasterSheet
    • Added two rows right below the title row to represent the calculations for the Average and the Standard Deviation of each column
    • For the Scaled and Centered Columns of data, typed the equation =(C4-C$2)/C$3 in the first cell under scaled and centered column for replicate 1 at timepoint C0, and used Excel functions in order to scale and center the rest of the data with the equation as a template.
  • Sent this file to Dr. Dahlquist to split the data to get rid of duplicates: File:Raw Data Shewanella RARL 20151201.xlsx

December 3, 2015 thru December 8, 2015

Discrepancies and issues arose with data between my partner Emily Simso and I that were brought to our attention by Dr. Dahlquist; thus, a review of the compiled raw data needed to be done in order for our Excel Sheets to match and to continue on with statistical analysis:

  • Repeated procedure from File:Raw Data Shewanella RARL 20151201.xlsx; however, feedback from Dr. Dahlquist was kept in mind and created a new Excel file called UpdatedCompiledRawData Shewanella RARL 20151201 HMH
    • Correct set of timepoints were used in my previous Excel file so no changes were needed to be done there
    • Ensured that I had 11520 Gene IDs; in which the last row which had a "Gene ID" as its label was changed to the correct Gene ID that is "SO4357."
    • Once all the necessary changes were made and I had touched base with my partner Emily on Sunday and Monday, I had uploaded the file for splitting by Dr. Dahlquist: File:UpdatedCompiledRawData Shewanella RARL 20151201 HMH.xlsx
  • Split data was received and posted as a file on our team's file page by Dr. Dahlquist: UpdatedCompiledRawData_Shewanella_RARL_20151201_HMH_forsplitting.xlsx
    • Downloaded this file and copied the sheets of data into a new Excel file entitled StatisticalAnalysis Shewanella RARL 20151207 HMH

File:StatisticalAnalysis Shewanella RARL 20151207 HMH.xlsx

You now need to do the following:

  1. Average together the replicate data from the two spots that are now split. This means that you need to average the "Log2FC-C0-rep1-scaledandcentered" in cell C2 with the value in cell AG2, for example.
  2. Copy and paste special > paste values into a new sheet called "statistics".
  3. Compute the average of the biological replicates for each treatment and timepoint. For example, average together all four biological replicates for Log2FC-C0. Repeat for each timepoint.
  4. Compute the ratio of the average log ratios so that you have values for the Average Log Ratio of C5/C0, C20/C0, C60/C0, F5/C60, F20/C60, and F60/C60). Since this is in log space, to take the ratio, you will actually subtract instead of divide.
  5. Perform a two-sample T test between C5 and C0, C20 and C0, C60 and C0, F5 and C60, etc. Use the equation:
=TTEST(<range of cells containing the biological replicates for C0>, <range of cells containing the biological replicates for C5>, 2,3]
This will return the p value. Send me the link to the file at this point so I can check the results. You can also perform the sanity check. Let me know how it goes.

914 instances of error "#DIV/0!" replaced with a blank cell.

    1. Sort the data A-->Z based on the "ID" column. Delete all rows that have an ID of "Blank", "blank", "gDNA", start with "NC-", start with "ORF". Record how many rows got deleted.
    2. Some of your cells are going to have error messages in them because of the previous calculations you did. Find and replace all of these with nothing, record how many cells that is.
    3. Create a new worksheet called "ScalingCentering" and copy and paste special all of your data into this new sheet. You will perform the scaling and centering operations like you did for the Vibrio cholerae data.
      • Once you have done this, e-mail Dr. Dahlquist and provide a link to your file. Your microarray data has duplicated spots. I have a script that will separate these out so that you can average them as technical replicates.
    4. Create a new worksheet called "Statistics", copy and paste the values into this new sheet.
    5. You will average the technical replicate spots for each sample to get one value for each sample.
    6. You will average the biological replicates for each timepoint to get an average for each timepoint (C0, C5, C20, C60, F5, F20, F60).
    7. You will take the ratio of the average log ratios so that you have values for the Average Log Ratio of C5/C0, C20/C0, C60/C0, F5/C60, F20/C60, and F60/C60). Since this is in log space, to take the ratio, you will actually subtract instead of divide.
    8. You will perform a two-sample T test between C5 and C0, C20 and C0, C60 and C0, F5 and C60, etc. and perform the Bonferroni and Benjamini and Hochberg corrections on these p values. This computation is not the same as what we did for Vibrio. Instead you will use the TTEST function in Excel (see me when you are ready to do this). The corrections will be the same as what you did before.
    9. After that, then you will need to then format the data for GenMAPP and you'll be ready to import it into GenMAPP and run MAPPFinder.
  • Let me know if you have any questions.
Retrieved from "https://xmlpipedb.lmucs.io/biodb/fall2015/index.php?title=Rlegaspi_Week_14&oldid=7476"