Difference between revisions of "Mavila9 Week 10"
(output data) |
|||
Line 5: | Line 5: | ||
==Methods== | ==Methods== | ||
− | === | + | === Creat the GRNmap Input Workbook === |
− | [https://github.com/kdahlquist/DahlquistLab/raw/master/data/Spring2019/15-genes_28-edges_sample_Sigmoid_estimation_2019.xlsx This sample workbook] was downloaded on which to | + | [https://github.com/kdahlquist/DahlquistLab/raw/master/data/Spring2019/15-genes_28-edges_sample_Sigmoid_estimation_2019.xlsx This sample workbook] was downloaded to base the network and microarray data worksheet on. |
+ | |||
+ | ==== production_rates sheet ==== | ||
+ | |||
+ | *** The production rates were provided in a Microsoft Access database, which was[https://github.com/kdahlquist/DahlquistLab/raw/master/data/Spring2019/Expression-and-Degradation-rate-database_2019.accdb downloaded from here.] | ||
+ | *** A query was performed to get the list of production rates for each gene as a group. | ||
+ | *** To perform the query, the following steps were taken: | ||
+ | ***# A list of the genes was imported to a new table in the database. The "External Data" tab was clicked and the Excel icon with the "up" arrow on it was selected. | ||
+ | ***# The "Browse" button was clicked and the Excel file containing the network that was used to upload to GRNsight was selected. | ||
+ | ***# The button next to "Import the source data into a new table in the current database" was selected and "OK" was clicked. | ||
+ | ***# In the next window, the "network" worksheet was selected then "Next" was clicked . | ||
+ | ***# In the next window, the "First Row Contains Column Headings" was checked and "Next" was cliked. | ||
+ | ***# In the next window, the left-most column was be highlighted and the"Field Name" was changed to "id" then "Next" was clicked. | ||
+ | ***# In the next window, the button for "Choose my own primary key." was selected and the "id" field was chosen from the drop down next to it then "Next" was clicked. | ||
+ | ***# In the next field, "Import to Table: network" was input then "Finish" was clicked. | ||
+ | ***# In the next window the import step was not saved, then "Close" was clicked. | ||
+ | ***# A table called "network" appeared in the list of tables at the left of the window. | ||
+ | ***# "Query Design" was selected from the "Create" tab. | ||
+ | ***# In the window that appears, the "network" table was chosen and "Add" was clicked. The "production_rates" table was selected and "Add" was clicked then "Close" was clicked. | ||
+ | ***# The two tables appear in the main part of the window. Then the word "id" was clicked in the network table and dragged to the "standard_name" field in the "production_rates" table, and released. A line appeared between those two words. | ||
+ | ***# After right-clicking the line between those words and "Join Properties" was selected from the menu that appears. Option "2: Include ALL records from 'network' and only those records from 'production_rates' where the joined fields are equal." was selected and "OK" was clicked. | ||
+ | ***# The "id" word was clicked on in the "network" table and dragged to the bottom of the screen to the first column next to the word "Field" and release. | ||
+ | ***# The "production_rate" field was clicked in the "production_rates" table and dragged it to the bottom of the screen to the second column next to the word "Field" and released. | ||
+ | ***# Anywhere in the gray area near the two tables was right-clicked then in the menu that appears, "Query Type > Make Table Query..." was selected. | ||
+ | ***# In the window that appears, the table was named "production_rates_1" then "Current Database" was selected and "OK" was clicked. | ||
+ | ***# On the "Query Tools: Menus" tab the exclamation point icon was selected and "Yes" was selected. | ||
+ | ***# The "production_rates_1" table appeared in the list at the left. The table name was double-clicked to open. | ||
+ | ***# The data in this table was copied and pasted back into the Excel workbook. | ||
+ | |||
+ | ==== degradation_rates sheet ==== | ||
+ | |||
+ | * Same steps as "production_rates sheet" | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
==== Expression Data Sheets for Individual Yeast Strains ==== | ==== Expression Data Sheets for Individual Yeast Strains ==== |
Revision as of 20:33, 6 November 2019
Contents
Links
Purpose
This investigation serves to help gain experience with database queries and modeling on Microsoft Access, GRNMAP, and MATLAB, to prepare for the the final project.
Methods
Creat the GRNmap Input Workbook
This sample workbook was downloaded to base the network and microarray data worksheet on.
production_rates sheet
- The production rates were provided in a Microsoft Access database, which wasdownloaded from here.
- A query was performed to get the list of production rates for each gene as a group.
- To perform the query, the following steps were taken:
- A list of the genes was imported to a new table in the database. The "External Data" tab was clicked and the Excel icon with the "up" arrow on it was selected.
- The "Browse" button was clicked and the Excel file containing the network that was used to upload to GRNsight was selected.
- The button next to "Import the source data into a new table in the current database" was selected and "OK" was clicked.
- In the next window, the "network" worksheet was selected then "Next" was clicked .
- In the next window, the "First Row Contains Column Headings" was checked and "Next" was cliked.
- In the next window, the left-most column was be highlighted and the"Field Name" was changed to "id" then "Next" was clicked.
- In the next window, the button for "Choose my own primary key." was selected and the "id" field was chosen from the drop down next to it then "Next" was clicked.
- In the next field, "Import to Table: network" was input then "Finish" was clicked.
- In the next window the import step was not saved, then "Close" was clicked.
- A table called "network" appeared in the list of tables at the left of the window.
- "Query Design" was selected from the "Create" tab.
- In the window that appears, the "network" table was chosen and "Add" was clicked. The "production_rates" table was selected and "Add" was clicked then "Close" was clicked.
- The two tables appear in the main part of the window. Then the word "id" was clicked in the network table and dragged to the "standard_name" field in the "production_rates" table, and released. A line appeared between those two words.
- After right-clicking the line between those words and "Join Properties" was selected from the menu that appears. Option "2: Include ALL records from 'network' and only those records from 'production_rates' where the joined fields are equal." was selected and "OK" was clicked.
- The "id" word was clicked on in the "network" table and dragged to the bottom of the screen to the first column next to the word "Field" and release.
- The "production_rate" field was clicked in the "production_rates" table and dragged it to the bottom of the screen to the second column next to the word "Field" and released.
- Anywhere in the gray area near the two tables was right-clicked then in the menu that appears, "Query Type > Make Table Query..." was selected.
- In the window that appears, the table was named "production_rates_1" then "Current Database" was selected and "OK" was clicked.
- On the "Query Tools: Menus" tab the exclamation point icon was selected and "Yes" was selected.
- The "production_rates_1" table appeared in the list at the left. The table name was double-clicked to open.
- The data in this table was copied and pasted back into the Excel workbook.
degradation_rates sheet
- Same steps as "production_rates sheet"
Expression Data Sheets for Individual Yeast Strains
- Expression data can be provided for either a single strain or multiple strains of yeast (for example, the wild type strain and a transcription factor deletion strain).
- Each strain will have its own sheet in the workbook.
- Each sheet should be given a unique name that follows the convention "STRAIN_log2_expression", where the word "STRAIN" is replaced by the strain designation, which will appear in the optimization_diagnostics sheet.
- Everyone in the class will have at least one expression worksheet called "wt_log2_expression".
- You should have included the transcription factors GLN3, HAP4, and CIN5 in your network. Thus, we will use the expression data from the dGLN3, dHAP4, dCIN5 deletion strains in our workbooks as well, naming the worksheets "dgln3_log2_expression", "dhap4_log2_expression", and "dcin5_expression".
- If, for some reason, you don't have all three of those genes in your network, only include expression data for the wild type and the genes out of those three that you have in your network.
- The sheet should have the following columns in this order:
- "id": list of all genes. The genes should be listed in the same order in all the sheets in the Excel workbook.
- The next series of columns should contain the expression data for each gene at a given timepoint given as log2 ratios (log2 fold changes). The column header should be the time at which the data were collected, without any units. For example, the 15 minute timepoint would have a column header "15" and the 30 minute timepoint would have the column header "30". GRNmap supports replicate data for each of the timepoints. Replicate data for the same timepoint should be in columns immediately next to each other and have the same column headers. For example, three replicates of the 15 minute timepoint would have "15", "15", "15" as the column headers.
- If data are provided for multiple strains, each strain should have data for the same timepoints, although the number of replicates can vary.
- Include the data for the 15, 30, and 60 minute timepoints, but not the 90 or 120 minute timepoints.
- The data you will be using is contained in the Expression-and-Degradation-rate-database_2019.accdb file that you used to obtain the production and degradation rates.
- It is tedious to copy and paste all of these data by hand, so we will execute a query in Microsoft Access to do it for you. Follow the steps listed for the "production_rates" sheet for each strains expression data. After you import the data into Excel, you will need to change the column headers to "15", "15", etc., as described above.
- Missing values in the expression data sheets are OK; you don't need to put any values there like you did for the production_rates or degradation_rates sheets.
network sheet
- The network you derived from the YEASTRACT database for the Week 9 assignment can be copied and pasted into this sheet directly. You may need to edit the contents of cell A1, but the rest should be good to go (especially since you previewed it in GRNsight). The description below just explains what is already in this worksheet.
- This sheet contains an adjacency matrix representation of the gene regulatory network.
- The columns correspond to the transcription factors and the rows correspond to the target genes controlled by those transcription factors.
- A “1” means there is an edge connecting them and a “0” means that there is no edge connecting them.
- The upper-left cell (A1) should contain the text “cols regulators/rows targets”. This text is there as a reminder of the direction of the regulatory relationships specified by the adjacency matrix.
- The rest of row 1 should contain the names of the transcription factors that are controlling the other genes in the network, one transcription factor name per column.
- The rest of column A should contain the names of the target genes that are being controlled by the transcription factors heading each of the columns in the matrix, one target gene name per row.
- The transcription factor names should correspond to the "id" in the other sheets in the workbook. They should be capitalized the same way and occur in the same order along the top and side of the matrix. The matrix needs to be symmetric, i.e., the same transcription factors should appear along the top and left side of the matrix. The genes should be listed in the same order in all the sheets in the Excel workbook.
- Each cell in the matrix should then contain a zero (0) if there is no regulatory relationship between those two transcription factors, or a one (1) if there is a regulatory relationship between them. Again, the columns correspond to the transcription factors and the rows correspond to the target genes controlled by those transcription factors.
network_weights sheet
- These are the initial guesses for the estimation of the weight parameters, w.
- Since these weights are initial guesses which will be optimized by GRNmap, the content of this sheet can be identical to the "network" sheet.
optimization_parameters sheet
- The optimization_parameters sheet should have two columns (from left to right) entitled, "optimization_parameter" and "value".
- You should copy this worksheet from the sample workbook provided. The only row that you need to modify is row 15, "Strain". Include just the strain designations for which you have a corresponding STRAIN_log2_expression sheet. If you don't have the dgln3, dhap4, or dcin5 expression sheets, then you will delete those from this row. If you do so, make sure that you don't leave any gaps between cells.
- What follows below is an explanation of what the optimization_parameters mean.
- alpha: Penalty term weighting (from the L-curve analysis)
- kk_max: Number of times to re-run the optimization loop. In some cases re-starting the optimization loop can improve performance of the estimation.
- MaxIter: Number of times MATLAB iterates through the optimization scheme. If this is set too low, MATLAB will stop before the parameters are optimized.
- TolFun: How different two least squares evaluations should be before the program determines that it is not making any improvement
- MaxFunEval: maximum number of times the program will evaluate the least squares cost
- TolX: How close successive least squares cost evaluations should be before the program determines that it is not making any improvement.
- production_function: = Sigmoid (case-insensitive) if sigmoidal model, =MM (case-insensitive) if Michaelis-Menten model
- L_curve: =0 if an L-curve analysis should NOT be run or =1 if an L-curve analysis SHOULD be run. The L-curve analysis will automatically run sequential rounds of estimation for an array of fixed alpha values (0.8, 0.5, 0.2, 0.1,0.08, 0.05,0.02,0.01, 0.008, 0.005, 0.002, 0.001, 0.0008, 0.0005, 0.0002, and 0.0001). GRNmap makes a copy of the user's selected input workbook and changes alpha to the first alpha in the list. The estimation runs and the resulting parameter values are used as the initial guesses for the next round of estimation with the next alpha value. This process repeats until all alpha values have been run. New input and output workbooks are generated for each alpha value, although currently, the graphs are only saved for the last run.
- estimate_params =1 if want to estimate parameters and =0 if the user wants to do just one forward run
- make_graphs =1 to output graphs; =0 to not output graphs
- fix_P =1 if the user does not want to estimate the production rate, P, parameter, just use the initial guess and never change; =0 to estimate
- fix_b =1 if the user does not want to estimate the b parameter, just use the initial guess and never change; =0 to estimate
- expression_timepoints: A row containing a list of the time points when the data was collected experimentally. Should correspond to the timepoint column headers in the STRAIN_log2_expression sheets.
- Strain: A row containing a list of all of the strains for which there is expression data in the workbook. Should correspond to the "STRAIN" portion of the names of the STRAIN_log2_expression sheets for each strain. Note that GRNmap will run the model for the wild type network (all genes present in the network) and for networks where the gene deleted from the designated STRAIN has been deleted from the network.
- simulation_timepoints: A row containing a list of the time points at which to evaluate the differential equations to generate the simulated data. This does not need to correspond to the actual measurement times, but should be in the same units (e.g. minutes).
threshold_b sheet
- These are the initial guesses for the estimation of the threshold_b parameters.
- There should be two columns.
- The left-most column should contain the header "id" and list the standard names for the genes in the model in the same order as in the other sheets.
- The second column should have the header "threshold_b" and should contain the initial guesses, we are going to use all 0.
Data and Files
Your data and files section should include:
- Your Excel workbook that is your GRNmap input workbook.
Stopping point for Tuesday's interim deadline.
Dynamical Systems Modeling of your Gene Regulatory Network
Now you are ready to run the model and analyze the results. The software we will use is called GRNmap, which stands for Gene Regulatory Network Modeling and Parameter Estimation. It is written in MATLAB and can be run from code or run as a stand-alone executable if you don't have MATLAB installed. However, it can only be run in Windows, not on Macs.
- To run GRNmap from code, you must have MATLAB R2014b installed on your computer.
- Download the GRNmap v1.10 code from the GRNmap Downloads page.
- Unzip the file. (Right-click, 7-zip > Extract here)
- Launch MATLAB R2014b.
- Open GRNmodel.m, which will be in the directory that you unzipped GRNmap-1.10 > matlab
- Click the Run button (green "play" arrow).
- You will be prompted to select your input workbook.
- You will see an optimization diagnostics graphic that shows the progress of the estimation.
- When the run is over, expression plots will display.
- Output .xlsx and .mat files will be saved in the same folder as your input folder, along with .jpg files containing the optimization diagnostic and individual expression plots. Save these files.
- Note that if you need to run GRNmap again, you should not use the same directory for the input file. Currently, GRNmap will overwrite previous output.
- You can upload your output .xlsx file into GRNsight to visualize the results!
Class/Team Journal Assignment has been cancelled for Week 10.
Data
media:GRNmap output mavila9.zip