LMU BioDB 2019 - User contributions [en]

FunGals Deliverables

2019-12-14T00:36:56Z

Eyoung20: /* Statement of Work */ added info

{{template:FunGals}}

==Checklist==
{| class="wikitable"
|-
! Tasks to be completed
! Complete/Incomplete
|-
| Organized Team deliverables wiki page (or other media (CD or flash drive) with table of contents)
| '''Complete'''
|-
| Group Report (''.doc'', ''.docx'' or ''.pdf'' file)
| Incomplete
|-
| Individual statements of work, assessments, reflections (wiki page, ''.doc'', ''.docx'', ''.pdf'', or e-mailed to Dr. Dahlquist)
| Incomplete
|-
| Group PowerPoint presentation (given on Tuesday, December 10, ''.ppt'', ''.pptx'' or ''.pdf'' file)
| '''Complete'''
|-
| Sample-data relationship table in Excel (''.xlsx'')
| '''Complete'''
|-
| Excel spreadsheet with ANOVA results/stem formatting (''.xlsx'')
| '''Complete'''
|-
| PowerPoint of ANOVA table, screenshots of stem results (''.pptx''), screenshot of black and white GRNsight input network and colored GRNsight output networks
| '''Complete'''
|-
| Gene List and GO List files from each significant profile (''.txt'' compressed together in a ''.zip'' file)
| '''Complete'''
|-
| YEASTRACT "rank by TF" results (''.xlsx'')
| '''Complete'''
|-
| GRNmap input workbook (with network adjacency matrix, ''.xlsx'')
| '''Complete'''
|-
| GRNmap output workbook (''.xlsx'') and output plots (''.jpg'') zipped together
| '''Complete'''
|-
| MS Access database, unified by the three teams with expression tables and metadata table(s) created (''.accdb'')
| '''Complete'''
|-
| ReadMe for the database that describes the design of the database, references the sources of the data, and has a [https://www.quackit.com/microsoft_access/microsoft_access_2016/howto/how_to_create_a_database_diagram_in_access_2016.cfm database schema diagram] (''.doc'', ''.docx'', ''.pdf'')
| '''Complete'''
|-
| Query design for populating a GRNmap input workbook from the database (screenshot of MS Access, or SQL code, ''.txt'')
| '''Complete'''
|-
| Electronic notebook corresponding to these the microarray results files ([[Week 12/13]] and [[Week 15]]) to support ''reproducible research'' so that all manipulations of the data and files are documented so that someone else could begin with your starting file, follow the protocol, and obtain your results.
| Incomplete
|-
|}

==Data and Files==
*Group Report
**[[Media:FunGals BioDB Final Paper.docx|Final Report]]
*Group PowerPoint presentation
**[[Media:FunGals_Final_Presentation.pptx | Final Presentation]]
*Metadata Workbook
**[[Media:Metadata_Sheet_(1).xlsx | Sample-Data Relationship Table (Metadata Sheet)]]
*ANOVA results/STEM formatting
**[[media: Thiuram_yeast_experiment.xlsx| ANOVA Results/STEM formatting (.xlsx)]]
*Figure Slides
**[[Media:FunGals_Figures_Slides.pptx| Figures Slides]]
*Gene List and GO List
**[[media: FunGals_genelist.zip|Genelist from STEM]]
**[[media: FunGals_GOlist.zip|GOlist from STEM]]
*YEASTRACT "rank by TF" results
**[[media:Genelist_combined_profiles_FunGals.xlsx| YEASTRACT Rank by TF - Green and Red Profiles (.xlsx)]]
*GRNmap Input Workbooks
**[[media:FunGals Green Profile RegulationMatrix Input Workbook.xlsx | Input Workbook for Green profile (.xlsx)]]
**[[media:FunGals Red Profile RegulationMatrix Documented 2019123 2318 1127808084.xlsx| Input Workbook Red network (.xlsx)]]
*GRNmap Output Workbooks
**[[media:GRNmap FunGals Green profile.zip | Output Workbook & GRNModel from MatLab for Green profile (.zip)]]
**[[Media:GRN_Model_Red_from_Matlab.zip | Output Workbook & GRNModel from MatLab for Red profile (.zip)]]
*MS Access Database
**[[media:FunGals_BioDB_CombinedDatabase_final.accdb.zip|Combined Database]]
*ReadMe
**[[media:FunGals_ReadMe.docx | ReadMe with Relationship Report]]
*Query Design Screenshot
**[[media:Queries_Run_for_the_Red_Profile_in_Acess_Database.pptx|Query Designs Red Profile]]
**[[media:Green Query Acess.pptx|Query Designs Green Profile]]

==Individual Assessment and Reflection==

===Michael Armas===

==== Statement of Work ====

* Describe exactly what you did on the project.
** As a coder is was my responsibility to do the database work. I created the database in MS Access, the metadata sheets, and the read me. This included working with the database to organize and universalize the data with the other groups. For the final report, I wrote the Introduction for the paper as well as added methods concerning the creation of the database and a few concluding points about the facilitation of results through the database. The presentation was mostly formatted by some of the other group members, but I was able to help a lot with the explanation of data during the presentation. The whole group worked hard to help each other with what was to be said during the presentation as well! As the project manager, I organized meeting times and made sure the entire group was on top of milestones and deliverables. I feel as if my contribution, alongside the help of my fellow group members, helped the outcome of the final project.
* Provide references or links to artifacts of your work, such as:
** Wiki pages
***[[Marmas Week 11|Individual Journal Week 11]]
***[[Marmas Week 12/13|Individual Journal Week 12/13]]
***[[Marmas Week 15|Individual Journal Week 15]]
** Other files or documents
***Significant Files I produced or helped produce:
****[[Media:Metadata_Sheet_(1).xlsx | Sample-Data Relationship Table (Metadata Sheet)]]
****[[media:FunGals_BioDB_CombinedDatabase_final.accdb.zip|Combined Database]]
****[[media:FunGals_ReadMe.docx | ReadMe with Relationship Report]]
** Code or scripts
*** Not very applicable, but the database is outlined in the files above.

==== Assessment of Project ====

* Give an objective assessment of the success of your project workflow and teamwork.
**I felt as if my success during this project was very satisfying. Hearing from the DA's that the database worked successfully was great to hear that I was able to help out with the assessment of data. For the other components, I feel as if I held my weight very much so when writing for the paper or presenting to the class. I enjoyed being the project manager for this group and felt as if my organization of meetings and milestones helped keep the group on track!
* What worked and what didn't work?
** Meeting at certain times was great and it really helped to communicate in person rather than over text or call. This worked well with our team and I feel as if this was the most successful way to work. What I felt didn't work was sometimes leaving some assignments to the last minute. While we were able to get the work in on time always, it was stressful to be working sometimes until late into the night.
* What would you do differently if you could do it all over again?
** I would get started with things much earlier. I am a person that tries to avoid procrastination, but sometimes that doesn't go as planned. If I could encourage myself and others to get started early on this project, I would be much less stressed during finals week.
* Evaluate your team’s portion of the Final Project and Group Report in the following areas:
*# Content: What is the quality of the work?
*#* The quality of the work was great, I think we are all happy with how the project turned out and with our discoveries. Going through and organizing some pages gave the final touches to the presentation of the project to make it look clean.
*# Organization: Comment on the organization of the project and of your group's wiki pages.
*#* The pages are organized and easy to follow. Headers are established and a table of contents allowed for things to be found easily. Most importantly, our deliverables are all organized into a page that has different sections to find files that will be graded.
*# Completeness: Did your team achieve all of the project objectives? Why or why not?
*#* Yes, we did achieve all of the project objectives. As I type this, we are just finishing the final touches on the final report, and will be ready to turn in the assignment before 4:00pm on Friday.

==== Reflection on the Process ====

* What did you learn?
** With your head (biological or computer science principles)
***I learned a lot about bioinformatics, a field of study which I really knew nothing about. This field allows for the ability to do something that humans could never do. The efficiency of bioinformatics blows me away with how fast data can be analyzed.
** With your heart (personal qualities and teamwork qualities that make things work or not work)?
***I continue to develop interpersonal skills with people when doing group projects. Working with a group always comes with its set of challenges, but working through problems is what being human is all about! I really did enjoy working with this group!
** With your hands (technical skills)?
***Being able to work with MS Access and virtual machines is a skill I never thought I would need. However, I understand how the familiarity with MS Access and all of the bioinformatics software is valuable to employers in that field. These skills will absolutely be going on my CV!
* What lesson will you take away from this project that you will still use a year from now?
**I hope that my interpersonal skills with group members continues to develop. Working with groups will be something I absolutely do in the future. Additionally, maybe in the next year I will be working for a company that will need me to work with databases or bioinformatics.

===Kaitlyn Nguyen===

==== Statement of Work ====

* Describe exactly what you did on the project.
** As a Data Analyst in this assignment, I was in charge of statistical analysis (ie. creating the ANOVA sheet), downloading GOlists and plugging it into the geneontology database, creating input workbooks and output workbooks for the Green Profile, running queries for MS Access as well as, formatting and editing powerpoints and group pages, and helping with additional tasks necessary for the project to flow cohesively.
* Provide references or links to artifacts of your work, such as:
** Wiki pages: Combined journal page with Emma Young
** https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_11
** https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Knguye66_Eyoung20_Week_12/13
** https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Knguye66_Eyoung20_Week_15
** Other files or documents
*** Stopping point of my work is up until STEM worksheet: [[media:Thiuram_yeast_experiment.xlsx|ANOVA & STEM workbook (.xlsx)]]
*** [[media:Genelist_combined_profiles_FunGals.xlsx| YEASTRACT Rank by TF - Green and Red Profiles (.xlsx)]]
*** [[media:FunGals Green Profile RegulationMatrix Input Workbook.xlsx | Input Workbook for Green profile (.xlsx)]]
*** [[media:GRNmap FunGals Green profile.zip | Output Workbook & GRNModel from MatLab for Green profile (.zip)]]
*** [[media:Green Query Acess.pptx|Query Designs Green Profile]]

==== Assessment of Project ====

* Give an objective assessment of the success of your project workflow and teamwork.
* What worked and what didn't work?
** What worked was everyone's ability to understand each other's situations, final exams, and work patiently to help each other. What didn't work was waiting until the last minute to finish all the work (final presentation, wiki page, final paper) rather than slowly work on it throughout the weeks.
* What would you do differently if you could do it all over again?
** I would definitely start the paper much earlier to leave more space to help reformatting and editing.
* Evaluate your team’s portion of the Final Project and Group Report in the following areas:
*# Content: What is the quality of the work? 9/10
*# Organization: Comment on the organization of the project and of your group's wiki pages.
*** Our organization is quite good actually, we follow off the requirements and deliverables in order.
*# Completeness: Did your team achieve all of the project objectives? Why or why not? Yes!

==== Reflection on the Process ====

* What did you learn?
** With your head (biological or computer science principles)
*** I learned that understanding an entirely new language, whether it be biological, computational, or both require a lot of existing knowledge and/or hours of practice to figure out the methods.
** With your heart (personal qualities and teamwork qualities that make things work or not work)?
*** Personal qualities that work are communication skills and leadership. A teamwork quality that is always effective is patience. Sometimes a team member in the group may not understand something and rather than rushing through the work just to merely finish it, other team members would be patient and understanding in helping them understand the work.
** With your hands (technical skills)?
*** I have funnily learned that I have a knack for typing fast, but hat I need to slow down to catch mistakes earlier on, rather than later due to typos that become much larger errors later on.
* What lesson will you take away from this project that you will still use a year from now?
** I definitely understand the use of running Queries now. I've also learned that there are so many other functions on Excel that I could use over and over again in the future.

===Emma Young===

==== Statement of Work ====

* As a Data Analysis I worked with [[User:knguye66|Kaitlyn Nguyen]] to analysis the data from the Kitagawa et al. 2002 journal article. Here is a break down of what I did out of the Data Analysis work we split.
**Ran the Sanity Check on the ANOVA DATA
**Created the STEM input sheet
**Helped to Run STEM
**Created the Combined Gene Lists
**Ran The GENElist through YEASTRACT
**Chose the Genes for the network and ran them Through YEATTRACT
**Visualized the unweighted networks in GRNsighted
**Created the Red profile GRNmap input worksheet
**Ran the Red profile GRNmap input worksheet through GRNmap using MATLAb
** Visualized the GRNmap weighted network results in GRNsight
** worked on the shared individual journal pages to record the process
**added milestones and reflections to group pages
**worked on presentation
**worked on final paper

* Provide references or links to artifacts of your work, such as:
** Wiki pages
***[[Knguye66 Eyoung20 Week 12/13]]
***[[Knguye66 Eyoung20 Week 15]]
***[[FunGals]]
***[[FunGals Deliverables]]
** Other files or documents
***[[media:Data_for_FunGals.pptx|FunGals Data on Powerpoint]]
***[[media:Thiuram_yeast_experiment.xlsx|ANOVA & STEM workbook (.xlsx)]]
***[[media:Genelist_combined_profiles_FunGals.xlsx| YEASTRACT Rank by TF - Green and Red Profiles (.xlsx)]]
***[[media:FunGals_Green_Profile_RegulationMatrix_Documented_2019123_2322_1272399515.xlsx| Green Regulation Matrix/network (.xlsx)]] ***[[media:FunGals Red Profile RegulationMatrix Documented 2019123 2318 1127808084.xlsx| Input Workbook Red network (.xlsx)]]
***[[media:Queries_Run_for_the_Red_Profile_in_Acess_Database.pptx|Query Designs Red Profile]]
*** [[media:FunGals Red Profile RegulationMatrix Documented 2019123 2318 1127808084.xlsx| Input Workbook Red network (.xlsx)]]
***[[Media:GRN_Model_Red_from_Matlab.zip | Output Workbook & GRNModel from MatLab for Red profile (.zip)]]
***[[Media:FunGals_Final_Presentation.pptx | Final Presentation]]
***[[Media:FunGals BioDB Final Paper.docx|Final Report]]
** Code or scripts
***Not code or scripts to add

==== Assessment of Project ====

* Give an objective assessment of the success of your project workflow and teamwork.
* What worked and what didn't work?
**We worked well together when we could all work together at the same time. There was not a clear communication on when everyone was free to work. This lead to some tense moments were the work flow was not very consistent. It also left some people ahead and some people rushing to get work done.
* What would you do differently if you could do it all over again?
** I would work out meeting times that worked on everyones schedule and possibly divide up the workload a little more evenly.
* Evaluate your team’s portion of the Final Project and Group Report in the following areas:
*# Content: What is the quality of the work?
** the quality of the work was reasonable for the time frame we had. I think each of us put as mush as we were able into the work. Is it the best we could do, no. But I think in order to achieve that best we would have at least need another week. So given the time frame and the stressful and busy time of the semester I am proud of the work we did.
*# Organization: Comment on the organization of the project and of your group's wiki pages.
** I found the formate of our team page a little hard to look through. Everything is present but I feel it could have been organized a bit better. I also found the project organization a little weird. I felt like we rewrote a lot of information between the presentation, the deliverables, the group page and the individual journals. I feel like the repletion although useful for remembering the steps and what we did a little unnecessary. Maybe in the future combining the delieverables with he paper would be a good idea since they both contain the same information.
*# Completeness: Did your team achieve all of the project objectives? Why or why not?
** Yes we achieved all the project objectives although on a time crunch. I think it has to do with the fact that we became flexible with each others needs especially when writing the paper. We helped each other out with each section. If someone was running behind or overwhelmed we helped each other. Illiana and Kaitlyn especially had my back when the methods section overwhelmed me a bit.

==== Reflection on the Process ====

* What did you learn?
** With your head (biological or computer science principles)
*** I learned a lot about data anaylsis and how to tell if a paper has good data or if its contents should be looked at more critically.
** With your heart (personal qualities and teamwork qualities that make things work or not work)?
***I learned that team work can need a lot of compassion. A lot of the times our team worked best when we helped each other out.
** With your hands (technical skills)?
*** I learned a lot of new tricks for excel, as well as how to use an access database something I have never used before.
* What lesson will you take away from this project that you will still use a year from now?\
**I think the lesson on how to critically look at a research article is one I will most definitely use for the rest of my scientific career.
[[User:Eyoung20|Eyoung20]] ([[User talk:Eyoung20|talk]]) 16:23, 13 December 2019 (PST)

===Iliana Crespin===
[[Media:Crespin_AssessmentandReflection.docx| Individual Assessment and Reflection]]

==Acknowledgments==
This section is in acknowledgement to partners Michael Armas, Iliana Crespin, Emma Young, and Kaitlyn Nguyen.

Thank you Dr. Dahlquist for helping us on this project and being able to answer our questions. It was great having you as our professor this semester.

'''Except for what is noted above, this individual journal entry was completed by me and not copied from another source.''' 
[[User:Icrespin|Icrespin]] ([[User talk:Icrespin|talk]]) 21:34, 12 December 2019 (PST)
[[User:Marmas|Marmas]] ([[User talk:Marmas|talk]]) 10:52, 13 December 2019 (PST)
[[User:Eyoung20|Eyoung20]] ([[User talk:Eyoung20|talk]]) 16:23, 13 December 2019 (PST)

==References==
Dahlquist, K. (2019, November 19). Week 12/13. In Wikipedia, Biological Databases. Retrieved 6:25, November 20, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_12/13

Dahlquist, K. (2019, October 17). Week 8. In Wikipedia, Biological Databases. Retrieved 6:30, October 21, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_8

Final Project. (2019). Deliverables. Retrieved December 10, 2019 from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Final_Project_Deliverables

FunGals. (2019). Home Page. Retrieved December 12, 2019 from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/FunGals

Kitagawa, E., Takahashi, J., Momose, Y., & Iwahashi, H. (2002). Effects of the pesticide thiuram: genome-wide screening of indicator genes by yeast DNA microarray. Environmental science & technology, 36(18), 3908-3915. DOI: 10.1021/es015705v

[[Category: Group Projects]]

FunGals Deliverables

2019-12-14T00:32:53Z

Eyoung20: /* Data and Files */ added report

{{template:FunGals}}

==Checklist==
{| class="wikitable"
|-
! Tasks to be completed
! Complete/Incomplete
|-
| Organized Team deliverables wiki page (or other media (CD or flash drive) with table of contents)
| '''Complete'''
|-
| Group Report (''.doc'', ''.docx'' or ''.pdf'' file)
| Incomplete
|-
| Individual statements of work, assessments, reflections (wiki page, ''.doc'', ''.docx'', ''.pdf'', or e-mailed to Dr. Dahlquist)
| Incomplete
|-
| Group PowerPoint presentation (given on Tuesday, December 10, ''.ppt'', ''.pptx'' or ''.pdf'' file)
| '''Complete'''
|-
| Sample-data relationship table in Excel (''.xlsx'')
| '''Complete'''
|-
| Excel spreadsheet with ANOVA results/stem formatting (''.xlsx'')
| '''Complete'''
|-
| PowerPoint of ANOVA table, screenshots of stem results (''.pptx''), screenshot of black and white GRNsight input network and colored GRNsight output networks
| '''Complete'''
|-
| Gene List and GO List files from each significant profile (''.txt'' compressed together in a ''.zip'' file)
| '''Complete'''
|-
| YEASTRACT "rank by TF" results (''.xlsx'')
| '''Complete'''
|-
| GRNmap input workbook (with network adjacency matrix, ''.xlsx'')
| '''Complete'''
|-
| GRNmap output workbook (''.xlsx'') and output plots (''.jpg'') zipped together
| '''Complete'''
|-
| MS Access database, unified by the three teams with expression tables and metadata table(s) created (''.accdb'')
| '''Complete'''
|-
| ReadMe for the database that describes the design of the database, references the sources of the data, and has a [https://www.quackit.com/microsoft_access/microsoft_access_2016/howto/how_to_create_a_database_diagram_in_access_2016.cfm database schema diagram] (''.doc'', ''.docx'', ''.pdf'')
| '''Complete'''
|-
| Query design for populating a GRNmap input workbook from the database (screenshot of MS Access, or SQL code, ''.txt'')
| '''Complete'''
|-
| Electronic notebook corresponding to these the microarray results files ([[Week 12/13]] and [[Week 15]]) to support ''reproducible research'' so that all manipulations of the data and files are documented so that someone else could begin with your starting file, follow the protocol, and obtain your results.
| Incomplete
|-
|}

==Data and Files==
*Group Report
**[[Media:FunGals BioDB Final Paper.docx|Final Report]]
*Group PowerPoint presentation
**[[Media:FunGals_Final_Presentation.pptx | Final Presentation]]
*Metadata Workbook
**[[Media:Metadata_Sheet_(1).xlsx | Sample-Data Relationship Table (Metadata Sheet)]]
*ANOVA results/STEM formatting
**[[media: Thiuram_yeast_experiment.xlsx| ANOVA Results/STEM formatting (.xlsx)]]
*Figure Slides
**[[Media:FunGals_Figures_Slides.pptx| Figures Slides]]
*Gene List and GO List
**[[media: FunGals_genelist.zip|Genelist from STEM]]
**[[media: FunGals_GOlist.zip|GOlist from STEM]]
*YEASTRACT "rank by TF" results
**[[media:Genelist_combined_profiles_FunGals.xlsx| YEASTRACT Rank by TF - Green and Red Profiles (.xlsx)]]
*GRNmap Input Workbooks
**[[media:FunGals Green Profile RegulationMatrix Input Workbook.xlsx | Input Workbook for Green profile (.xlsx)]]
**[[media:FunGals Red Profile RegulationMatrix Documented 2019123 2318 1127808084.xlsx| Input Workbook Red network (.xlsx)]]
*GRNmap Output Workbooks
**[[media:GRNmap FunGals Green profile.zip | Output Workbook & GRNModel from MatLab for Green profile (.zip)]]
**[[Media:GRN_Model_Red_from_Matlab.zip | Output Workbook & GRNModel from MatLab for Red profile (.zip)]]
*MS Access Database
**[[media:FunGals_BioDB_CombinedDatabase_final.accdb.zip|Combined Database]]
*ReadMe
**[[media:FunGals_ReadMe.docx | ReadMe with Relationship Report]]
*Query Design Screenshot
**[[media:Queries_Run_for_the_Red_Profile_in_Acess_Database.pptx|Query Designs Red Profile]]
**[[media:Green Query Acess.pptx|Query Designs Green Profile]]

==Individual Assessment and Reflection==

===Michael Armas===

==== Statement of Work ====

* Describe exactly what you did on the project.
** As a coder is was my responsibility to do the database work. I created the database in MS Access, the metadata sheets, and the read me. This included working with the database to organize and universalize the data with the other groups. For the final report, I wrote the Introduction for the paper as well as added methods concerning the creation of the database and a few concluding points about the facilitation of results through the database. The presentation was mostly formatted by some of the other group members, but I was able to help a lot with the explanation of data during the presentation. The whole group worked hard to help each other with what was to be said during the presentation as well! As the project manager, I organized meeting times and made sure the entire group was on top of milestones and deliverables. I feel as if my contribution, alongside the help of my fellow group members, helped the outcome of the final project.
* Provide references or links to artifacts of your work, such as:
** Wiki pages
***[[Marmas Week 11|Individual Journal Week 11]]
***[[Marmas Week 12/13|Individual Journal Week 12/13]]
***[[Marmas Week 15|Individual Journal Week 15]]
** Other files or documents
***Significant Files I produced or helped produce:
****[[Media:Metadata_Sheet_(1).xlsx | Sample-Data Relationship Table (Metadata Sheet)]]
****[[media:FunGals_BioDB_CombinedDatabase_final.accdb.zip|Combined Database]]
****[[media:FunGals_ReadMe.docx | ReadMe with Relationship Report]]
** Code or scripts
*** Not very applicable, but the database is outlined in the files above.

==== Assessment of Project ====

* Give an objective assessment of the success of your project workflow and teamwork.
**I felt as if my success during this project was very satisfying. Hearing from the DA's that the database worked successfully was great to hear that I was able to help out with the assessment of data. For the other components, I feel as if I held my weight very much so when writing for the paper or presenting to the class. I enjoyed being the project manager for this group and felt as if my organization of meetings and milestones helped keep the group on track!
* What worked and what didn't work?
** Meeting at certain times was great and it really helped to communicate in person rather than over text or call. This worked well with our team and I feel as if this was the most successful way to work. What I felt didn't work was sometimes leaving some assignments to the last minute. While we were able to get the work in on time always, it was stressful to be working sometimes until late into the night.
* What would you do differently if you could do it all over again?
** I would get started with things much earlier. I am a person that tries to avoid procrastination, but sometimes that doesn't go as planned. If I could encourage myself and others to get started early on this project, I would be much less stressed during finals week.
* Evaluate your team’s portion of the Final Project and Group Report in the following areas:
*# Content: What is the quality of the work?
*#* The quality of the work was great, I think we are all happy with how the project turned out and with our discoveries. Going through and organizing some pages gave the final touches to the presentation of the project to make it look clean.
*# Organization: Comment on the organization of the project and of your group's wiki pages.
*#* The pages are organized and easy to follow. Headers are established and a table of contents allowed for things to be found easily. Most importantly, our deliverables are all organized into a page that has different sections to find files that will be graded.
*# Completeness: Did your team achieve all of the project objectives? Why or why not?
*#* Yes, we did achieve all of the project objectives. As I type this, we are just finishing the final touches on the final report, and will be ready to turn in the assignment before 4:00pm on Friday.

==== Reflection on the Process ====

* What did you learn?
** With your head (biological or computer science principles)
***I learned a lot about bioinformatics, a field of study which I really knew nothing about. This field allows for the ability to do something that humans could never do. The efficiency of bioinformatics blows me away with how fast data can be analyzed.
** With your heart (personal qualities and teamwork qualities that make things work or not work)?
***I continue to develop interpersonal skills with people when doing group projects. Working with a group always comes with its set of challenges, but working through problems is what being human is all about! I really did enjoy working with this group!
** With your hands (technical skills)?
***Being able to work with MS Access and virtual machines is a skill I never thought I would need. However, I understand how the familiarity with MS Access and all of the bioinformatics software is valuable to employers in that field. These skills will absolutely be going on my CV!
* What lesson will you take away from this project that you will still use a year from now?
**I hope that my interpersonal skills with group members continues to develop. Working with groups will be something I absolutely do in the future. Additionally, maybe in the next year I will be working for a company that will need me to work with databases or bioinformatics.

===Kaitlyn Nguyen===

==== Statement of Work ====

* Describe exactly what you did on the project.
** As a Data Analyst in this assignment, I was in charge of statistical analysis (ie. creating the ANOVA sheet), downloading GOlists and plugging it into the geneontology database, creating input workbooks and output workbooks for the Green Profile, running queries for MS Access as well as, formatting and editing powerpoints and group pages, and helping with additional tasks necessary for the project to flow cohesively.
* Provide references or links to artifacts of your work, such as:
** Wiki pages: Combined journal page with Emma Young
** https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_11
** https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Knguye66_Eyoung20_Week_12/13
** https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Knguye66_Eyoung20_Week_15
** Other files or documents
*** Stopping point of my work is up until STEM worksheet: [[media:Thiuram_yeast_experiment.xlsx|ANOVA & STEM workbook (.xlsx)]]
*** [[media:Genelist_combined_profiles_FunGals.xlsx| YEASTRACT Rank by TF - Green and Red Profiles (.xlsx)]]
*** [[media:FunGals Green Profile RegulationMatrix Input Workbook.xlsx | Input Workbook for Green profile (.xlsx)]]
*** [[media:GRNmap FunGals Green profile.zip | Output Workbook & GRNModel from MatLab for Green profile (.zip)]]
*** [[media:Green Query Acess.pptx|Query Designs Green Profile]]

==== Assessment of Project ====

* Give an objective assessment of the success of your project workflow and teamwork.
* What worked and what didn't work?
** What worked was everyone's ability to understand each other's situations, final exams, and work patiently to help each other. What didn't work was waiting until the last minute to finish all the work (final presentation, wiki page, final paper) rather than slowly work on it throughout the weeks.
* What would you do differently if you could do it all over again?
** I would definitely start the paper much earlier to leave more space to help reformatting and editing.
* Evaluate your team’s portion of the Final Project and Group Report in the following areas:
*# Content: What is the quality of the work? 9/10
*# Organization: Comment on the organization of the project and of your group's wiki pages.
*** Our organization is quite good actually, we follow off the requirements and deliverables in order.
*# Completeness: Did your team achieve all of the project objectives? Why or why not? Yes!

==== Reflection on the Process ====

* What did you learn?
** With your head (biological or computer science principles)
*** I learned that understanding an entirely new language, whether it be biological, computational, or both require a lot of existing knowledge and/or hours of practice to figure out the methods.
** With your heart (personal qualities and teamwork qualities that make things work or not work)?
*** Personal qualities that work are communication skills and leadership. A teamwork quality that is always effective is patience. Sometimes a team member in the group may not understand something and rather than rushing through the work just to merely finish it, other team members would be patient and understanding in helping them understand the work.
** With your hands (technical skills)?
*** I have funnily learned that I have a knack for typing fast, but hat I need to slow down to catch mistakes earlier on, rather than later due to typos that become much larger errors later on.
* What lesson will you take away from this project that you will still use a year from now?
** I definitely understand the use of running Queries now. I've also learned that there are so many other functions on Excel that I could use over and over again in the future.

===Emma Young===

==== Statement of Work ====

* As a Data Analysis I worked with [[User:knguye66|Kaitlyn Nguyen]] to analysis the data from the Kitagawa et al. 2002 journal article. Here is a break down of what I did out of the Data Analysis work we split.
**Ran the Sanity Check on the ANOVA DATA
**Created the STEM input sheet
**Helped to Run STEM
**Created the Combined Gene Lists
**Ran The GENElist through YEASTRACT
**Chose the Genes for the network and ran them Through YEATTRACT
**Visualized the unweighted networks in GRNsighted
**Created the Red profile GRNmap input worksheet
**Ran the Red profile GRNmap input worksheet through GRNmap using MATLAb
** Visualized the GRNmap weighted network results in GRNsight
** worked on the shared individual journal pages to record the process
**added milestones and reflections to group pages
**worked on presentation
**worked on final paper

* Provide references or links to artifacts of your work, such as:
** Wiki pages
***[[Knguye66 Eyoung20 Week 12/13]]
***[[Knguye66 Eyoung20 Week 15]]
***[[FunGals]]
** Other files or documents
***[[media:Data_for_FunGals.pptx|FunGals Data on Powerpoint]]
***[[media:Thiuram_yeast_experiment.xlsx|ANOVA & STEM workbook (.xlsx)]]
***[[media:Genelist_combined_profiles_FunGals.xlsx| YEASTRACT Rank by TF - Green and Red Profiles (.xlsx)]]
***[[media:FunGals_Green_Profile_RegulationMatrix_Documented_2019123_2322_1272399515.xlsx| Green Regulation Matrix/network (.xlsx)]] ***[[media:FunGals Red Profile RegulationMatrix Documented 2019123 2318 1127808084.xlsx| Input Workbook Red network (.xlsx)]]
***[[media:Queries_Run_for_the_Red_Profile_in_Acess_Database.pptx|Query Designs Red Profile]]
*** [[media:FunGals Red Profile RegulationMatrix Documented 2019123 2318 1127808084.xlsx| Input Workbook Red network (.xlsx)]]
***[[Media:GRN_Model_Red_from_Matlab.zip | Output Workbook & GRNModel from MatLab for Red profile (.zip)]]
***[[Media:FunGals_Final_Presentation.pptx | Final Presentation]]
** Code or scripts

==== Assessment of Project ====

* Give an objective assessment of the success of your project workflow and teamwork.
* What worked and what didn't work?
**We worked well together when we could all work together at the same time. There was not a clear communication on when everyone was free to work. This lead to some tense moments were the work flow was not very consistent. It also left some people ahead and some people rushing to get work done.
* What would you do differently if you could do it all over again?
** I would work out meeting times that worked on everyones schedule and possibly divide up the workload a little more evenly.
* Evaluate your team’s portion of the Final Project and Group Report in the following areas:
*# Content: What is the quality of the work?
** the quality of the work was reasonable for the time frame we had. I think each of us put as mush as we were able into the work. Is it the best we could do, no. But I think in order to achieve that best we would have at least need another week. So given the time frame and the stressful and busy time of the semester I am proud of the work we did.
*# Organization: Comment on the organization of the project and of your group's wiki pages.
** I found the formate of our team page a little hard to look through. Everything is present but I feel it could have been organized a bit better. I also found the project organization a little weird. I felt like we rewrote a lot of information between the presentation, the deliverables, the group page and the individual journals. I feel like the repletion although useful for remembering the steps and what we did a little unnecessary. Maybe in the future combining the delieverables with he paper would be a good idea since they both contain the same information.
*# Completeness: Did your team achieve all of the project objectives? Why or why not?
** Yes we achieved all the project objectives although on a time crunch. I think it has to do with the fact that we became flexible with each others needs especially when writing the paper. We helped each other out with each section. If someone was running behind or overwhelmed we helped each other. Illiana and Kaitlyn especially had my back when the methods section overwhelmed me a bit.

==== Reflection on the Process ====

* What did you learn?
** With your head (biological or computer science principles)
*** I learned a lot about data anaylsis and how to tell if a paper has good data or if its contents should be looked at more critically.
** With your heart (personal qualities and teamwork qualities that make things work or not work)?
***I learned that team work can need a lot of compassion. A lot of the times our team worked best when we helped each other out.
** With your hands (technical skills)?
*** I learned a lot of new tricks for excel, as well as how to use an access database something I have never used before.
* What lesson will you take away from this project that you will still use a year from now?\
**I think the lesson on how to critically look at a research article is one I will most definitely use for the rest of my scientific career.
[[User:Eyoung20|Eyoung20]] ([[User talk:Eyoung20|talk]]) 16:23, 13 December 2019 (PST)

===Iliana Crespin===
[[Media:Crespin_AssessmentandReflection.docx| Individual Assessment and Reflection]]

==Acknowledgments==
This section is in acknowledgement to partners Michael Armas, Iliana Crespin, Emma Young, and Kaitlyn Nguyen.

Thank you Dr. Dahlquist for helping us on this project and being able to answer our questions. It was great having you as our professor this semester.

'''Except for what is noted above, this individual journal entry was completed by me and not copied from another source.''' 
[[User:Icrespin|Icrespin]] ([[User talk:Icrespin|talk]]) 21:34, 12 December 2019 (PST)
[[User:Marmas|Marmas]] ([[User talk:Marmas|talk]]) 10:52, 13 December 2019 (PST)
[[User:Eyoung20|Eyoung20]] ([[User talk:Eyoung20|talk]]) 16:23, 13 December 2019 (PST)

==References==
Dahlquist, K. (2019, November 19). Week 12/13. In Wikipedia, Biological Databases. Retrieved 6:25, November 20, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_12/13

Dahlquist, K. (2019, October 17). Week 8. In Wikipedia, Biological Databases. Retrieved 6:30, October 21, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_8

Final Project. (2019). Deliverables. Retrieved December 10, 2019 from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Final_Project_Deliverables

FunGals. (2019). Home Page. Retrieved December 12, 2019 from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/FunGals

Kitagawa, E., Takahashi, J., Momose, Y., & Iwahashi, H. (2002). Effects of the pesticide thiuram: genome-wide screening of indicator genes by yeast DNA microarray. Environmental science & technology, 36(18), 3908-3915. DOI: 10.1021/es015705v

[[Category: Group Projects]]

FunGals

2019-12-14T00:32:22Z

Eyoung20: /* Week 15 */ added data

==Effects of the Pesticide Thiuram: Genome-wide Screening of Indicator Genes by Yeast DNA Microarray==

'''Team Information'''
Project Manager: [[User:Marmas|Michael Armas]]
Quality Assurance: [[User:Icrespin| Iliana Crespin]]
Data Analysis: [[User:eyoung20|Emma Young]], [[User:knguye66|Kaitlyn Nguyen]]
Coder: [[User:Marmas|Michael Armas]]

== Methods and Results ==
=== Week 11 ===
====Setting up the Project====
*Selected [[User:Marmas|Michael Armas]] as team's Project Manager.
*Added the name of of selected project manager to the Project Manager guild page and Overview pages.
*Named the team FunGals and created [[FunGals]] home page on the wiki.
*The name of the team home page was the selected team name.

*Created a link to team's page on the course Main page.
*Created a template for FunGal with useful information and links that you will invoke on all pages that you will create for the project.
*Created a category using team name and included it the FunGal template so that it is used on all pages created for the project. Also included the category "Group Projects" in the template. 
*Each person needs to write a short executive summary of that person's progress on the project for the week, with links to the relevant individual journal pages (which will have more detailed information).
*Each team member should reflect on the team's progress '''(Note that you will be directed to add specific information to your team's pages in the individual portion of the assignment for this and future weeks)''':
**What worked? What didn't work? What will I do next to fix what didn't work?
*** Michael Armas: Working with this team is a great pleasure! Everyone on the team is pulling their weight and providing information that contributes to the group work. Next time, as the project manager, I want to be more organized and come up with interim deadlines to make sure we are not cramming right before the actual deadline. In fact, I would have something small due every day than procrastinate until the end. However, with this team, we are making it work and doing exceptional work!
*** Kaitlyn Nguyen: What worked this week is starting the presentation formatting early, thus allocating the rest of the time to "filling in the missing parts" of the powerpoint. What also worked this week was team communication via group-chats. What didn't work this week was starting the individual assignment later, leaving less time to practicing and fully understanding the material from the journal article for the presentation in class. To fix what didn't work, I will start my individual assignment much earlier, to leave the rest of the time to focus on collaborative teamwork and group assignments. [[User:Knguye66|Knguye66]] ([[User talk:Knguye66|talk]]) 23:37, 13 November 2019 (PST)
*** Emma Young: The team seems to work really well together. I think in upcoming weeks we will really thrive in working together. A foundation for communication has already been established and we have already started reviewing each others work and providing constructive criticism and help where it is needed. The main issue this week was a matter of timing. With the amount of work we had to do in the time frame given we were not able to meet up due to busy and conflicting prior engagements. This meant that we did not reach our full potential of working together effectively as a group this week. Next week, we hopefully will be able to set better deadlines and be able to have the ability to plan out time to work on this in a less rushed manner.
*** Iliana Crespin: Overall, this group has been great. Everyone has worked together and understands what must be done. In addition, each member is understanding of the special circumstances before the deadline of this assignment. The only thing that didn't work out was how scattered each of us were because of all the assignments that had to be completed from other classes. In addition, mandated plans (school- or work-related) popped up which made it difficult to contribute equally. For the future, I will make sure to manage my time better, because I have work and school. Therefore, by doing a fair share of certain things before classes/work, I will be able to contribute more.[[User:Icrespin|Icrespin]] ([[User talk:Icrespin|talk]]) 23:46, 13 November 2019 (PST)

====Annotated Bibliography====

#Braconi, D., Bernardini, G., & Santucci, A. (2016). Saccharomyces cerevisiae as a model in ecotoxicological studies: A post-genomics perspective. Journal of Proteomics, 137, 19-34. DOI: 10.1016/j.jprot.2015.09.001
#Hinkle, K. L., & Olsen, D. (2018). Exposure to the lampricide TFM elicits an environmental stress response in yeast. FEMS yeast research, 19(1), foy121. doi: 10.1093/femsyr/foy121
#Iwahashi, Y., Hosoda, H., Park, J. H., Lee, J. H., Suzuki, Y., Kitagawa, E., ... & Iwahashi, H. (2006). Mechanisms of patulin toxicity under conditions that inhibit yeast growth. Journal of agricultural and food chemistry, 54(5), 1936-1942. doi: 10.1021/jf052264g.
#Iwahashi, H., Ishidou, E., Kitagawa, E., & Momose, Y. (2007). Combined Cadmium and Thiuram Show Synergistic Toxicity and Induce Mitochondrial Petite Mutants. Environmental Science & Technology, 41(22), 7941–7946. doi: 10.1021/es071313y
#Iwahashi, H., Ishidou, E., Kitagawa, E., & Momose, Y. (2007). Combined Cadmium and Thiuram Show Synergistic Toxicity and Induce Mitochondrial Petite Mutants. Environmental Science & Technology, 41(22), 7941–7946. doi: 10.1021/es071313y
#Kitagawa, E., Momose, Y., & Iwahashi, H. (2003). Correlation of the Structures of Agricultural Fungicides to Gene Expression in Saccharomyces cerevisiaeupon Exposure to Toxic Doses. Environmental Science & Technology, 37(12), 2788–2793. doi: 10.1021/es026156b
#Lu Yu, Na Guo, Rizeng Meng, Bin Liu, Xudong Tang, Jing Jin, Yumei Cui, Xuming Deng. Allicin-induced global gene expression profile of Saccharomyces cerevisiae. Applied Microbiology and Biotechnology 2010, 88 (1) , 219-229. DOI: 10.1007/s00253-010-2709-x.
#Pierron, A., Mimoun, S., Murate, L. S., Loiseau, N., Lippi, Y., Bracarense, A. P. F., ... & Oswald, I. P. (2016). Intestinal toxicity of the masked mycotoxin deoxynivalenol-3-β-D-glucoside. Archives of toxicology, 90(8), 2037-2046. doi: 10.1007/s00204-015-1592-8

===Week 12/13===
====Milestone 3: Getting the data ready for analysis ====

# As a group Downloaded and examined the microarray dataset, compared it to the samples and experiment described in the journal club article.
#* [https://sgd-prod-upload.s3.amazonaws.com/S000204415/Kitagawa_2002_PMID_12269742.zip Kitagawa et al. (2002)]
#* downloaded and unzipped
#* file was downloaded but was PLC format
#* Mike figured out how to drag and drop it one to excel to open the file
# Along with the QA's, make a "sample-data relationship table" that lists all of the samples (microarray chips), noting the treatment, time point, replicate number, and other information pertaining to each sample.
#* This "Metadata" sheet was created in Excel and is present on the excel work under the week 12/13 data and files.
#* In collaboration with other groups, the Metadata sheet was filled out so that nomenclature is consistent.
# The class collectively came up with consistent column headers that summarized the information
#* As decided by all groups, the format for column headers was yeastStrain_treatmentOrMutation_concentrationAndUnits_dataType_timeAndUnits-replicateNumber.
# Organized the data in a worksheet in an Excel workbook so that:
#* MasterIndex was in the first column, ID was in the second column
#* Data columns were to the right, in increasing chronological order, using the column header pattern that was created.
#* Replicates were grouped together
# Data analysts (DA) [[User:Knguye66|Kaitlyn]] set-up the ANOVA worksheet via referencing [[Week 8]]
#* The steps for the ANOVA: Part I, Benjamini, Bonferroni, and p-value correction, as well as, a quick sanity check were followed.
# DAs [[User:Eyoung20|Eyoung20]] begin setting up for STEM analysis to complete the first stage of milestones and deliverables
# All the calculations were checked by the QA [[User:Icrespin|Iliana]].

*Each team member should reflect on the team's progress '''(Note that you will be directed to add specific information to your team's pages in the individual portion of the assignment for this and future weeks)''':
**What worked? What didn't work? What will I do next to fix what didn't work?
*** Michael Armas: What worked this week was the communication between my guild and me, and my group and me. Everyone was able to help each other out and make sure the project is coming along smoothly. What didn't work this week for me was understanding exactly what I had to do to set up spreadsheets for data analysis. However, thanks to Mihir, I was able to understand how to create a metadata sheet and reach the milestone for this week. Before I leave class from now on, I will reassure myself that I understand the information by asking Dr. Dahlquist the best way to reach the week's milestone.
*** Kaitlyn Nguyen: This week, the pattern and organization to which each group member worked flowed smoothly as every member of the group knew exactly what each person was accomplishing. Having extra time in class and staying after class helped us jump-start on our tasks together and find out what assignments work best for each team member. What didn't work this week was that each milestone and part of this project works chronologically, whereby other members have to wait for the first person to finish his/her task before beginning theirs. As the days and weeks move on, it will be good to continue this method, but switch off on who starts the tasks first for the assignment. [[User:Knguye66|Knguye66]] ([[User talk:Knguye66|talk]]) 18:44, 20 November 2019 (PST)
*** Emma Young: For this week I worked closely with [[User:Knguye66|Kaitlyn]] to start the data analysis for the rest of the project. We worked with the whole team to analyze the data from the journal article and format for the rest of the data analysis. [[User:Marmas|Mike]] worked with the other programers to formulate the metadata sheet and the format for the ANOVA sheet. More details on this can be found in his personal electronic notebook [[Marmas Week 12/13]] Then [[User:Knguye66|Kaitlyn]] worked on the ANOVA analysis of the data. [[User:Icrespin|Iliana]] double checked the the results of the analysis the details can be found in [[Icrespin Journal Week 12/13]]. After the checks were completed and any errors were found, I ran a sanity check on the ANOVA data and then formatted the STEM worksheet for future analysis. The details about what [[User:Knguye66|Kaitlyn] and [[user:eyoung20|I]] did can be found in our shared journal [[Knguye66 Eyoung20 Week 12/13]]. As can be seen in this summary this week your group was really good at communicating and working together to work on this project. We worked really well together this week. One thing that did not work as well was trying to work on the excel spreadsheet online when others were working on it, there was a large lag on the sheet. I think for the next steps I might do the work on a saved downloaded version of the excel sheet and then copy my work to the shared excel. [[User:Eyoung20|Eyoung20]] ([[User talk:Eyoung20|talk]]) 16:36, 25 November 2019 (PST)
*** Iliana Crespin: For this week, it was a better organization on completing the assignment. There is more of a pattern of what should be done throughout each day to prepare for the deadline. What didn't work out for me is still trying to see how a QA can incorporate any work, since my teammates are great. For the next few weeks, I will try to continue double checking the work and contribute more. This contribution can include working with the data analysts and seeing how I can help out a bit more. In addition, texting teammates a bit more to remind any little things that must be due before the deadline. [[User:Icrespin|Icrespin]] ([[User talk:Icrespin|talk]]) 14:48, 21 November 2019 (PST)

===Week 15===
'''Milestone 4'''
#Begin wrapping up with running analysis on the created database using GRNmap
#*Data Analysts: Create GRNmap using individual database from MS Access.
#*Coder: Communicate with the QA to ensure the database is correct and if any changes need to be made
#*QA: Review the database and communicate to the Coder if any changes need to be made.
#Each team member should reflect on the team's progress
#*What worked? What didn't work? What will I do next to fix what didn't work?
#**Michael Armas
#***I believe that the best thing we as a group did for the progress of this project was meeting up in person. Our in-person meetings in the lab we’re very productive; communicating over text or calling was always less fluid. What didn’t really work for me was being last minute on certain milestones or assignments. The desire to procrastinate is natural, but avoiding it should be a top priority to decrease stress towards the end of the project. However, I was always confident of actually getting the work done. In the future, I want to put getting things done early at the top of the priority list, making deadlines much less stressed and rushed. 
[[User:Marmas|Marmas]] ([[User talk:Marmas|talk]]) 13:59, 13 December 2019 (PST)
#**Kaitlyn Nguyen
#*** What worked was the ability for us to communicate fairly with each other to finish the assignment on time. While we always finished assignments and milestones on time, we would always be stressed the night before to make it up towards the end. In the future, I would start my portion and ask the group members to start adding outlines and bullet points onto the paper to begin brainstorming what to write rather than waiting until the later moments.
[[User:Knguye66|Knguye66]] ([[User talk:Knguye66|talk]]) 14:33, 13 December 2019 (PST)
#**Emma Young:The team worked really well to get everything done to the best of our ability. We were able to meet all of our milestones and worked well on providing support to one and other. However we did have some disagreements on how to time things out. I think we should have really worked out a schedule ahead of time. We all seemed to misconstrue when we could all work on each of our parts and work on things together. I think in the future having a better understanding of each others work loads and what can reasonable be completed when would be very beneficial.
[[User:Eyoung20|Eyoung20]] ([[User talk:Eyoung20|talk]]) 15:40, 13 December 2019 (PST)

#**Iliana Crespin: After everything, the team was able to complete all of the necessary items on time. I had an amazing team who went above and beyond on accommodations. There was a lot of communication that allowed us to make sure we were doing our tasks. One thing that didn't work was how I handled each deadline. Due to work circumstances, it was very difficult to do a lot of things, which made me a bit helpless. However, my team members were very flexible. For the future, one thing to work on is time management.[[User:Icrespin|Icrespin]] ([[User talk:Icrespin|talk]]) 19:29, 12 December 2019 (PST)

== Data and Files ==
=== Week 11 ===
* Presentation file: [[Media: Marmas_FunGals_Presentation.pptx|Group Presentation]]
=== Week 12/13 ===
*[[media: Thiuram_yeast_experiment.xlsx|Excel Workbook]]
*[[media: FunGals_genelist.zip|Genelist from STEM]]
*[[media: FunGals_GOlist.zip|GOlist from STEM]]
*[[media:Data_for_FunGals.pptx|FunGals Data on Powerpoint]]
===Week 15===
*[[Media:Marmas_finalProject_Expression-and-Degradation-rate-database_120319.zip|FunGals Expression Database]]
*[[Media:GRN_Model_Red_from_Matlab.zip|GRN Model Red Profile]]
*[[media:Data_for_FunGals.pptx|FunGals Data on Powerpoint]]
*[[media:Queries_Run_for_the_Red_Profile_in_Acess_Database.pptx|Query Designs Red Profile]]
*[[Media:FunGals_Final_Presentation.pptx | Final Presentation]]
*[[Media:FunGals BioDB Final Paper.docx|Final Report]]

== Milestones ==
{| class="wikitable"
|-
! Tasks to be completed
! Due Date
|-
| Write down Methods and Results for Week 12/13
*Coder/designer: find out the column names and edit them, sample to data relationship table
*Data Analysis: ANOVA analysis and set-up, STEM set-up (will run STEM on Thursday)
*QA: make sure all data in original file actually made it into the database, let coder and analysts know if there are issues
| Thursday, 12:00am, 11/19/19
|-
| Team Journal Assignment, continue writing down Methods and Results for Week 12/13
*Data Analysis: finish editing missing information for ANOVA and (STRAIN) worksheet, run STEM
*Coder and QA: add standard names to genes, make sure all data in original file actually made it into the database, let coder and analysts know if there are issues
| Tuesday, 12:00am, 11/26/19
|-
| Creating the database and running queries, working with all guilds to assure project is in line to be completed
*Data Analysts: Run YEASTRACT and modify GO Terms.
*Coder: Create individual database for the team.
*QA:Design Expression Tables used to create the final individual database.
| Thursday, 12:00am, 11/28/19
|-
|Begin wrapping up with running analysis on the created database using GRNmap
*Data Analysts: Create GRNmap using individual database from MS Access.
*Coder: Communicate with the QA to ensure the database is correct and if any changes need to be made
*QA: Review the database and communicate to the Coder if any changes need to be made.
| Tuesday, 12:00am, 12/03/19
|-
| Wrap up data analysis and individual milestones and begin to gather deliverables for turn-in
*Data Analysts: Provide feedback on the database and its ease-of-use. Work with QA to gather deliverables.
*Coder: Work with the entire guild to properly combine all databases for the entire class to use.
*QA: Work with the coder to finalize the [https://www.quackit.com/microsoft_access/microsoft_access_2016/howto/how_to_create_a_database_diagram_in_access_2016.cfm database schema diagram]
| Thursday, 12:00am, 12/05/19
|-
| Final Presentation
| Tuesday, 12:00am, 12/10/19
|-
| Report submitted
| Friday, 4:00 pm, 12/13/19
|-
|}

==Acknowledgements==
This section is in acknowledgement to partner [[User:Marmas|Michael Armas]], [[User:Icrespin|Iliana Crespin]], [[User:eyoung20|Emma Young]], and [[User:knguye66|Kaitlyn Nguyen]]. I would also like to acknowledge [[User:Kdahlquist|Dr. Dahlquist]] for introducing and teaching the topic and direction of this assignment.

"Except for what is noted above, this individual journal entry was completed by me and not copied from another source."
 
[[User:Marmas|Marmas]] ([[User talk:Marmas|talk]]) 23:40, 13 November 2019 (PST)
[[User:Icrespin|Icrespin]] ([[User talk:Icrespin|talk]]) 23:46, 13 November 2019 (PST)

"Except for what is noted above, this individual journal entry was completed by me and not copied from another source."
[[User:Eyoung20|Eyoung20]] ([[User talk:Eyoung20|talk]]) 23:57, 13 November 2019 (PST) [[User:Knguye66|Knguye66]] ([[User talk:Knguye66|talk]]) 18:45, 20 November 2019 (PST)

{{FunGals}}

==References==
===Week 11===
*Kitagawa, E., Takahashi, J., Momose, Y., & Iwahashi, H. (2002). Effects of the pesticide thiuram: genome-wide screening of indicator genes by yeast DNA microarray. Environmental science & technology, 36(18), 3908-3915. DOI: 10.1021/es015705v
*Dahlquist, K. (2019, November 7). Week 11. In Wikipedia, Biological Databases. https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_1https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_11
===Week 12/13===
*Dahlquist, K. (2019, November 19). Data Analysis. In Wikipedia, Biological Databases. Retrieved 6:25, November 20, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Data_Analysis
*Dahlquist, K. (2019, November 20). Final Project Deliverables. In Wikipedia, Biological Databases. Retrieved 6:25, November 20, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_12/13https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Final_Project_Deliverables
*Dahlquist, K. (2019, November 19). Week 12/13. In Wikipedia, Biological Databases. Retrieved 6:25, November 20, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_12/13
*Dahlquist, K. (2019, October 17). Week 8. In Wikipedia, Biological Databases. Retrieved 6:30, October 21, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_8
===Week 15===
Dahlquist, K. (2019, November 19). Week 12/13. In Wikipedia, Biological Databases. Retrieved 6:25, November 20, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_12/13

Dahlquist, K. (2019, October 17). Week 8. In Wikipedia, Biological Databases. Retrieved 6:30, October 21, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_8

Final Project. (2019). Deliverables. Retrieved December 10, 2019 from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Final_Project_Deliverables

FunGals. (2019). Home Page. Retrieved December 12, 2019 from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/FunGals

Kitagawa, E., Takahashi, J., Momose, Y., & Iwahashi, H. (2002). Effects of the pesticide thiuram: genome-wide screening of indicator genes by yeast DNA microarray. Environmental science & technology, 36(18), 3908-3915. DOI: 10.1021/es015705v

2019-12-13T20:17:26Z

Eyoung20: /* Week 15 */ added data

==Effects of the Pesticide Thiuram: Genome-wide Screening of Indicator Genes by Yeast DNA Microarray==

'''Team Information'''
Project Manager: [[User:Marmas|Michael Armas]]
Quality Assurance: [[User:Icrespin| Iliana Crespin]]
Data Analysis: [[User:eyoung20|Emma Young]], [[User:knguye66|Kaitlyn Nguyen]]
Coder: [[User:Marmas|Michael Armas]]

== Methods and Results ==
=== Week 11 ===
====Setting up the Project====
*Selected [[User:Marmas|Michael Armas]] as team's Project Manager.
*Added the name of of selected project manager to the Project Manager guild page and Overview pages.
*Named the team FunGals and created [[FunGals]] home page on the wiki.
*The name of the team home page was the selected team name.

*Created a link to team's page on the course Main page.
*Created a template for FunGal with useful information and links that you will invoke on all pages that you will create for the project.
*Created a category using team name and included it the FunGal template so that it is used on all pages created for the project. Also included the category "Group Projects" in the template. 
*Each person needs to write a short executive summary of that person's progress on the project for the week, with links to the relevant individual journal pages (which will have more detailed information).
*Each team member should reflect on the team's progress '''(Note that you will be directed to add specific information to your team's pages in the individual portion of the assignment for this and future weeks)''':
**What worked? What didn't work? What will I do next to fix what didn't work?
*** Michael Armas: Working with this team is a great pleasure! Everyone on the team is pulling their weight and providing information that contributes to the group work. Next time, as the project manager, I want to be more organized and come up with interim deadlines to make sure we are not cramming right before the actual deadline. In fact, I would have something small due every day than procrastinate until the end. However, with this team, we are making it work and doing exceptional work!
*** Kaitlyn Nguyen: What worked this week is starting the presentation formatting early, thus allocating the rest of the time to "filling in the missing parts" of the powerpoint. What also worked this week was team communication via group-chats. What didn't work this week was starting the individual assignment later, leaving less time to practicing and fully understanding the material from the journal article for the presentation in class. To fix what didn't work, I will start my individual assignment much earlier, to leave the rest of the time to focus on collaborative teamwork and group assignments. [[User:Knguye66|Knguye66]] ([[User talk:Knguye66|talk]]) 23:37, 13 November 2019 (PST)
*** Emma Young: The team seems to work really well together. I think in upcoming weeks we will really thrive in working together. A foundation for communication has already been established and we have already started reviewing each others work and providing constructive criticism and help where it is needed. The main issue this week was a matter of timing. With the amount of work we had to do in the time frame given we were not able to meet up due to busy and conflicting prior engagements. This meant that we did not reach our full potential of working together effectively as a group this week. Next week, we hopefully will be able to set better deadlines and be able to have the ability to plan out time to work on this in a less rushed manner.
*** Iliana Crespin: Overall, this group has been great. Everyone has worked together and understands what must be done. In addition, each member is understanding of the special circumstances before the deadline of this assignment. The only thing that didn't work out was how scattered each of us were because of all the assignments that had to be completed from other classes. In addition, mandated plans (school- or work-related) popped up which made it difficult to contribute equally. For the future, I will make sure to manage my time better, because I have work and school. Therefore, by doing a fair share of certain things before classes/work, I will be able to contribute more.[[User:Icrespin|Icrespin]] ([[User talk:Icrespin|talk]]) 23:46, 13 November 2019 (PST)

====Annotated Bibliography====

#Braconi, D., Bernardini, G., & Santucci, A. (2016). Saccharomyces cerevisiae as a model in ecotoxicological studies: A post-genomics perspective. Journal of Proteomics, 137, 19-34. DOI: 10.1016/j.jprot.2015.09.001
#Hinkle, K. L., & Olsen, D. (2018). Exposure to the lampricide TFM elicits an environmental stress response in yeast. FEMS yeast research, 19(1), foy121. doi: 10.1093/femsyr/foy121
#Iwahashi, Y., Hosoda, H., Park, J. H., Lee, J. H., Suzuki, Y., Kitagawa, E., ... & Iwahashi, H. (2006). Mechanisms of patulin toxicity under conditions that inhibit yeast growth. Journal of agricultural and food chemistry, 54(5), 1936-1942. doi: 10.1021/jf052264g.
#Iwahashi, H., Ishidou, E., Kitagawa, E., & Momose, Y. (2007). Combined Cadmium and Thiuram Show Synergistic Toxicity and Induce Mitochondrial Petite Mutants. Environmental Science & Technology, 41(22), 7941–7946. doi: 10.1021/es071313y
#Iwahashi, H., Ishidou, E., Kitagawa, E., & Momose, Y. (2007). Combined Cadmium and Thiuram Show Synergistic Toxicity and Induce Mitochondrial Petite Mutants. Environmental Science & Technology, 41(22), 7941–7946. doi: 10.1021/es071313y
#Kitagawa, E., Momose, Y., & Iwahashi, H. (2003). Correlation of the Structures of Agricultural Fungicides to Gene Expression in Saccharomyces cerevisiaeupon Exposure to Toxic Doses. Environmental Science & Technology, 37(12), 2788–2793. doi: 10.1021/es026156b
#Lu Yu, Na Guo, Rizeng Meng, Bin Liu, Xudong Tang, Jing Jin, Yumei Cui, Xuming Deng. Allicin-induced global gene expression profile of Saccharomyces cerevisiae. Applied Microbiology and Biotechnology 2010, 88 (1) , 219-229. DOI: 10.1007/s00253-010-2709-x.
#Pierron, A., Mimoun, S., Murate, L. S., Loiseau, N., Lippi, Y., Bracarense, A. P. F., ... & Oswald, I. P. (2016). Intestinal toxicity of the masked mycotoxin deoxynivalenol-3-β-D-glucoside. Archives of toxicology, 90(8), 2037-2046. doi: 10.1007/s00204-015-1592-8

===Week 12/13===
====Milestone 3: Getting the data ready for analysis ====

# As a group Downloaded and examined the microarray dataset, compared it to the samples and experiment described in the journal club article.
#* [https://sgd-prod-upload.s3.amazonaws.com/S000204415/Kitagawa_2002_PMID_12269742.zip Kitagawa et al. (2002)]
#* downloaded and unzipped
#* file was downloaded but was PLC format
#* Mike figured out how to drag and drop it one to excel to open the file
# Along with the QA's, make a "sample-data relationship table" that lists all of the samples (microarray chips), noting the treatment, time point, replicate number, and other information pertaining to each sample.
#* This "Metadata" sheet was created in Excel and is present on the excel work under the week 12/13 data and files.
#* In collaboration with other groups, the Metadata sheet was filled out so that nomenclature is consistent.
# The class collectively came up with consistent column headers that summarized the information
#* As decided by all groups, the format for column headers was yeastStrain_treatmentOrMutation_concentrationAndUnits_dataType_timeAndUnits-replicateNumber.
# Organized the data in a worksheet in an Excel workbook so that:
#* MasterIndex was in the first column, ID was in the second column
#* Data columns were to the right, in increasing chronological order, using the column header pattern that was created.
#* Replicates were grouped together
# Data analysts (DA) [[User:Knguye66|Kaitlyn]] set-up the ANOVA worksheet via referencing [[Week 8]]
#* The steps for the ANOVA: Part I, Benjamini, Bonferroni, and p-value correction, as well as, a quick sanity check were followed.
# DAs [[User:Eyoung20|Eyoung20]] begin setting up for STEM analysis to complete the first stage of milestones and deliverables
# All the calculations were checked by the QA [[User:Icrespin|Iliana]].

*Each team member should reflect on the team's progress '''(Note that you will be directed to add specific information to your team's pages in the individual portion of the assignment for this and future weeks)''':
**What worked? What didn't work? What will I do next to fix what didn't work?
*** Michael Armas: What worked this week was the communication between my guild and me, and my group and me. Everyone was able to help each other out and make sure the project is coming along smoothly. What didn't work this week for me was understanding exactly what I had to do to set up spreadsheets for data analysis. However, thanks to Mihir, I was able to understand how to create a metadata sheet and reach the milestone for this week. Before I leave class from now on, I will reassure myself that I understand the information by asking Dr. Dahlquist the best way to reach the week's milestone.
*** Kaitlyn Nguyen: This week, the pattern and organization to which each group member worked flowed smoothly as every member of the group knew exactly what each person was accomplishing. Having extra time in class and staying after class helped us jump-start on our tasks together and find out what assignments work best for each team member. What didn't work this week was that each milestone and part of this project works chronologically, whereby other members have to wait for the first person to finish his/her task before beginning theirs. As the days and weeks move on, it will be good to continue this method, but switch off on who starts the tasks first for the assignment. [[User:Knguye66|Knguye66]] ([[User talk:Knguye66|talk]]) 18:44, 20 November 2019 (PST)
*** Emma Young: For this week I worked closely with [[User:Knguye66|Kaitlyn]] to start the data analysis for the rest of the project. We worked with the whole team to analyze the data from the journal article and format for the rest of the data analysis. [[User:Marmas|Mike]] worked with the other programers to formulate the metadata sheet and the format for the ANOVA sheet. More details on this can be found in his personal electronic notebook [[Marmas Week 12/13]] Then [[User:Knguye66|Kaitlyn]] worked on the ANOVA analysis of the data. [[User:Icrespin|Iliana]] double checked the the results of the analysis the details can be found in [[Icrespin Journal Week 12/13]]. After the checks were completed and any errors were found, I ran a sanity check on the ANOVA data and then formatted the STEM worksheet for future analysis. The details about what [[User:Knguye66|Kaitlyn] and [[user:eyoung20|I]] did can be found in our shared journal [[Knguye66 Eyoung20 Week 12/13]]. As can be seen in this summary this week your group was really good at communicating and working together to work on this project. We worked really well together this week. One thing that did not work as well was trying to work on the excel spreadsheet online when others were working on it, there was a large lag on the sheet. I think for the next steps I might do the work on a saved downloaded version of the excel sheet and then copy my work to the shared excel. [[User:Eyoung20|Eyoung20]] ([[User talk:Eyoung20|talk]]) 16:36, 25 November 2019 (PST)
*** Iliana Crespin: For this week, it was a better organization on completing the assignment. There is more of a pattern of what should be done throughout each day to prepare for the deadline. What didn't work out for me is still trying to see how a QA can incorporate any work, since my teammates are great. For the next few weeks, I will try to continue double checking the work and contribute more. This contribution can include working with the data analysts and seeing how I can help out a bit more. In addition, texting teammates a bit more to remind any little things that must be due before the deadline. [[User:Icrespin|Icrespin]] ([[User talk:Icrespin|talk]]) 14:48, 21 November 2019 (PST)

===Week 15===
'''Milestone 4'''
#Begin wrapping up with running analysis on the created database using GRNmap
#*Data Analysts: Create GRNmap using individual database from MS Access.
#*Coder: Communicate with the QA to ensure the database is correct and if any changes need to be made
#*QA: Review the database and communicate to the Coder if any changes need to be made.
#Each team member should reflect on the team's progress
#*What worked? What didn't work? What will I do next to fix what didn't work?
#**Michael Armas
#**Kaitlyn Nguyen
#**Emma Young
#**Iliana Crespin: After everything, the team was able to complete all of the necessary items on time. I had an amazing team who went above and beyond on accommodations. There was a lot of communication that allowed us to make sure we were doing our tasks. One thing that didn't work was how I handled each deadline. Due to work circumstances, it was very difficult to do a lot of things, which made me a bit helpless. However, my team members were very flexible. For the future, one thing to work on is time management.[[User:Icrespin|Icrespin]] ([[User talk:Icrespin|talk]]) 19:29, 12 December 2019 (PST)

== Data and Files ==
=== Week 11 ===
* Presentation file: [[Media: Marmas_FunGals_Presentation.pptx|Group Presentation]]
=== Week 12/13 ===
*[[media: Thiuram_yeast_experiment.xlsx|Excel Workbook]]
*[[media: FunGals_genelist.zip|Genelist from STEM]]
*[[media: FunGals_GOlist.zip|GOlist from STEM]]
*[[media:Data_for_FunGals.pptx|FunGals Data on Powerpoint]]
===Week 15===
*[[Media:Marmas_finalProject_Expression-and-Degradation-rate-database_120319.zip|FunGals Expression Database]]
*[[Media:GRN_Model_Red_from_Matlab.zip|GRN Model Red Profile]]
*[[media:Data_for_FunGals.pptx|FunGals Data on Powerpoint]]
*[[media:Queries_Run_for_the_Red_Profile_in_Acess_Database.pptx|Query Designs Red Profile]]

== Milestones ==
{| class="wikitable"
|-
! Tasks to be completed
! Due Date
|-
| Write down Methods and Results for Week 12/13
*Coder/designer: find out the column names and edit them, sample to data relationship table
*Data Analysis: ANOVA analysis and set-up, STEM set-up (will run STEM on Thursday)
*QA: make sure all data in original file actually made it into the database, let coder and analysts know if there are issues
| Thursday, 12:00am, 11/19/19
|-
| Team Journal Assignment, continue writing down Methods and Results for Week 12/13
*Data Analysis: finish editing missing information for ANOVA and (STRAIN) worksheet, run STEM
*Coder and QA: add standard names to genes, make sure all data in original file actually made it into the database, let coder and analysts know if there are issues
| Tuesday, 12:00am, 11/26/19
|-
| Creating the database and running queries, working with all guilds to assure project is in line to be completed
*Data Analysts: Run YEASTRACT and modify GO Terms.
*Coder: Create individual database for the team.
*QA:Design Expression Tables used to create the final individual database.
| Thursday, 12:00am, 11/28/19
|-
|Begin wrapping up with running analysis on the created database using GRNmap
*Data Analysts: Create GRNmap using individual database from MS Access.
*Coder: Communicate with the QA to ensure the database is correct and if any changes need to be made
*QA: Review the database and communicate to the Coder if any changes need to be made.
| Tuesday, 12:00am, 12/03/19
|-
| Wrap up data analysis and individual milestones and begin to gather deliverables for turn-in
*Data Analysts: Provide feedback on the database and its ease-of-use. Work with QA to gather deliverables.
*Coder: Work with the entire guild to properly combine all databases for the entire class to use.
*QA: Work with the coder to finalize the [https://www.quackit.com/microsoft_access/microsoft_access_2016/howto/how_to_create_a_database_diagram_in_access_2016.cfm database schema diagram]
| Thursday, 12:00am, 12/05/19
|-
| Final Presentation
| Tuesday, 12:00am, 12/10/19
|-
| Report submitted
| Friday, 4:00 pm, 12/13/19
|-
|}

==Acknowledgements==
This section is in acknowledgement to partner [[User:Marmas|Michael Armas]], [[User:Icrespin|Iliana Crespin]], [[User:eyoung20|Emma Young]], and [[User:knguye66|Kaitlyn Nguyen]]. I would also like to acknowledge [[User:Kdahlquist|Dr. Dahlquist]] for introducing and teaching the topic and direction of this assignment.

"Except for what is noted above, this individual journal entry was completed by me and not copied from another source."
 
[[User:Marmas|Marmas]] ([[User talk:Marmas|talk]]) 23:40, 13 November 2019 (PST)
[[User:Icrespin|Icrespin]] ([[User talk:Icrespin|talk]]) 23:46, 13 November 2019 (PST)

"Except for what is noted above, this individual journal entry was completed by me and not copied from another source."
[[User:Eyoung20|Eyoung20]] ([[User talk:Eyoung20|talk]]) 23:57, 13 November 2019 (PST) [[User:Knguye66|Knguye66]] ([[User talk:Knguye66|talk]]) 18:45, 20 November 2019 (PST)

{{FunGals}}

==References==
===Week 11===
*Kitagawa, E., Takahashi, J., Momose, Y., & Iwahashi, H. (2002). Effects of the pesticide thiuram: genome-wide screening of indicator genes by yeast DNA microarray. Environmental science & technology, 36(18), 3908-3915. DOI: 10.1021/es015705v
*Dahlquist, K. (2019, November 7). Week 11. In Wikipedia, Biological Databases. https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_1https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_11
===Week 12/13===
*Dahlquist, K. (2019, November 19). Data Analysis. In Wikipedia, Biological Databases. Retrieved 6:25, November 20, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Data_Analysis
*Dahlquist, K. (2019, November 20). Final Project Deliverables. In Wikipedia, Biological Databases. Retrieved 6:25, November 20, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_12/13https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Final_Project_Deliverables
*Dahlquist, K. (2019, November 19). Week 12/13. In Wikipedia, Biological Databases. Retrieved 6:25, November 20, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_12/13
*Dahlquist, K. (2019, October 17). Week 8. In Wikipedia, Biological Databases. Retrieved 6:30, October 21, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_8
===Week 15===
Dahlquist, K. (2019, November 19). Week 12/13. In Wikipedia, Biological Databases. Retrieved 6:25, November 20, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_12/13

Dahlquist, K. (2019, October 17). Week 8. In Wikipedia, Biological Databases. Retrieved 6:30, October 21, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_8

Final Project. (2019). Deliverables. Retrieved December 10, 2019 from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Final_Project_Deliverables

FunGals. (2019). Home Page. Retrieved December 12, 2019 from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/FunGals

Kitagawa, E., Takahashi, J., Momose, Y., & Iwahashi, H. (2002). Effects of the pesticide thiuram: genome-wide screening of indicator genes by yeast DNA microarray. Environmental science & technology, 36(18), 3908-3915. DOI: 10.1021/es015705v

FunGals Deliverables

2019-12-13T20:16:44Z

Eyoung20: /* Data and Files */ added data file

File:Queries Run for the Red Profile in Acess Database.pptx

2019-12-13T20:15:10Z

Eyoung20:

Knguye66 Eyoung20 Week 15

2019-12-13T19:44:44Z

Eyoung20: /* Methods and Results: Progress */

'''Combined Individual Journals for Kaitlyn Nguyen and Emma Young (Data Analysts).'''

== Purpose ==
The purpose of this assignment is to record our progress towards the [[FunGals]] group [[Final_Project_Deliverables | deliverables]] as the [[Data_Analysis | Data Analysts]] for this week and the future weeks to come. The purpose of week 12 specifically was to download and adapt the data to the formatting we need for analysis. Then to begin the analysis with ANOVA and preparations for STEM.

== Methods and Results: Progress ==
===== - Progress 11/26/19 - =====

==== Analyzing and Interpreting STEM Results ====
#Why did you select this profile? In other words, why was it interesting to you?
#* We collectively combined the 4 red profiles together because they had similar trends and to increase the number of genes for analysis (called "Red".) Similarly this was done to the 3 green profiles as well (upward trends), with the addition of the blue profile #29 added. We will call this group "Green".
#How many genes belong to this profile?
#* Red (composed of Profile #9,26,34,11): 289
#* Green (Profile #40,42,18,29): 214
#How many genes were expected to belong to this profile?
#* Red (composed of Profile #9,26,34,11): 51.5
#* Green (Profile #40,42,18,29): 48.5
#What is the p value for the enrichment of genes in this profile?
#* Due to combining the profiles, we do not have p-values for the enrichment of genes in the 2 different groups.

- '''Viewing and Saving STEM Results''' -
# A new window will open called "All STEM Profiles (1)". Each box corresponds to a model expression profile. Colored profiles have a statistically significant number of genes assigned; they are arranged in order from most to least significant p value. Profiles with the same color belong to the same cluster of profiles. The number in each box is simply an ID number for the profile.
#* Click on the button that says "Interface Options...". At the bottom of the Interface Options window that appears below where it says "X-axis scale should be:", click on the radio button that says "Based on real time". Then close the Interface Options window.
#*Take a screenshot of this window (on a PC, simultaneously press the <code>Alt</code> and <code>PrintScreen</code> buttons to save the view in the active window to the clipboard) and paste it into a PowerPoint presentation to save your figures.
# Click on each of the SIGNIFICANT profiles (the colored ones) to open a window showing a more detailed plot containing all of the genes in that profile.
#* Take a screenshot of each of the individual profile windows and save the images in your PowerPoint presentation.
#* At the bottom of each profile window, there are two yellow buttons "Profile Gene Table" and "Profile GO Table". For each of the profiles, click on the "Profile Gene Table" button to see the list of genes belonging to the profile. In the window that appears, click on the "Save Table" button and save the file to your desktop. Make your filename descriptive of the contents, e.g. "wt_profile#_genelist.txt", where you replace the number symbol with the actual profile number.
#** Upload these files to the wiki and link to them on your individual journal page. (Note that it will be easier to [[Week_4#Compressing_and_Decompressing_Files_with_7-Zip | zip all the files together]] and upload them as one file).
#* For each of the significant profiles, click on the "Profile GO Table" to see the list of Gene Ontology terms belonging to the profile. In the window that appears, click on the "Save Table" button and save the file to your desktop. Make your filename descriptive of the contents, e.g. "wt_profile#_GOlist.txt", where you use "wt", "dGLN3", etc. to indicate the dataset and where you replace the number symbol with the actual profile number. At this point you have saved all of the primary data from the STEM software and it's time to interpret the results!
#** Upload these files to the wiki and link to them on your individual journal page. (Note that it will be easier to [[Week_4#Compressing_and_Decompressing_Files_with_7-Zip | zip all the files together]] and upload them as one file).

==== Clustering the data with STEM, as did on [[Week 9]]. ====
# Note that we will make some adjustments to the GO term analysis because stem was not providing GO term names. We are going to use the GO enrichment tool at GeneOntology.org instead.
#* Go to [http://geneontology.org/ http://geneontology.org/].
#* For the cluster you want to analyze, open the gene list and copy the list of genes.
#* Paste the list of genes into the "Go Enrichment Analysis" box on the right hand side of the GeneOntology.org page.
#* Select "Saccharomyces cerevisiae" from the species drop-down menu.
# Click the "Launch" buton.
#* Near the bottom of the results page, click on the button to Export "Table".
#* This will prompt you to save a .txt file that can be opened in Excel to view your results.
# Use YEASTRACT to generate a candidate gene regulatory network as you did on [[Week 9]].

==== Using YEASTRACT to Infer which Transcription Factors Regulate a Cluster of Genes ====

In the previous analysis using STEM, we found a number of gene expression profiles (aka clusters) which grouped genes based on similarity of gene expression changes over time. The implication is that these genes share the same expression pattern because they are regulated by the same (or the same set) of transcription factors. We will explore this using the YEASTRACT database.

# Open the gene list in Excel for the one of the significant profiles from your stem analysis. Choose a cluster with a clear cold shock/recovery up/down or down/up pattern. You should also choose one of the largest clusters.
#* Copy the list of gene IDs onto your clipboard.
# Launch a web browser and go to the [http://www.yeastract.com/ YEASTRACT database].
#* On the left panel of the window, click on the link to [http://www.yeastract.com/formrankbytf.php ''Rank by TF''].
#* Paste your list of genes from your cluster into the box labeled ''ORFs/Genes''.
#* Check the box for ''Check for all TFs''.
#* Accept the defaults for the Regulations Filter (Documented, DNA binding plus expression evidence)
#* Do '''''not''''' apply a filter for "Filter Documented Regulations by environmental condition".
#* Rank genes by TF using: The % of genes in the list and in YEASTRACT regulated by each TF.
#* Click the ''Search'' button.
# Answer the following questions:
#* In the results window that appears, the p values colored green are considered "significant", the ones colored yellow are considered "borderline significant" and the ones colored pink are considered "not significant". '''''How many transcription factors are green or "significant"?'''''
#**Green: 31
#**Red: 30
#* '''''Copy the table of results from the web page and paste it into a new Excel workbook to preserve the results.'''''
#** '''''Upload the Excel file to the wiki and link to it in your electronic lab notebook.'''''
#** '''''Are CIN5, GLN3, and/or HAP4 on the list? If so, what is their "% in user set", "% in YEASTRACT", and "p value".'''''
#**Green
#*** CIN5
#**** 0.2991 % in user set
#****0.0294
#**** p-value: 0.5492
#***GLN3
#**** 0.3645% in user set
#****0.0324% in YEASTRACT
#**** p-value: 0.18046
#***HAP4
#**** 0.2383% in user set
#****0.0467% in YEASTRACT
#**** P-value:0.00031337
#**Red
#*** CIN5
#**** 0.2111%% in user set
#**** 0.028% in YEASTRACT
#**** p-value: 0.9998
#***GLN3
#**** 0.4152% in user set
#****0.0498% in YEASTRACT
#**** p-value:0.00207752
#***HAP4
#****0.2353% in user set
#****0.0623% in YEASTRACT
#**** P-value:0.000006235
# For the mathematical model that we will build, we need to define a ''gene regulatory network'' of transcription factors that regulate other transcription factors. We can use YEASTRACT to assist us with creating the network. We want to generate a network with approximately 15-20 transcription factors in it.
#* You need to select from this list of "significant" transcription factors, which ones you will use to run the model. You will use these transcription factors and add GLN3, HAP4, and CIN5 if they are not in your list. Explain in your electronic notebook how you decided on which transcription factors to include. Record the list and your justification in your electronic lab notebook. Each group member will select a different network (they can have some overlapping transcription factors, but some should also be different).
#* Go back to the YEASTRACT database and follow the link to ''[http://www.yeastract.com/formregmatrix.php Generate Regulation Matrix]''.
#* Copy and paste the list of transcription factors you identified (plus HAP4, GLN3, and CIN5) into both the "Transcription factors" field and the "Target ORF/Genes" field.
#* We are going to use the "Regulations Filter" options of "Documented", "'''Only''' DNA binding evidence"
#** Click the "Generate" button.
#** In the results window that appears, click on the link to the "Regulation matrix (Semicolon Separated Values (CSV) file)" that appears and save it to your Desktop. Rename this file with a meaningful name so that you can distinguish it from the other files you will generate.

==== Visualizing Your Gene Regulatory Networks with GRNsight ====

We will analyze the regulatory matrix files you generated above in Microsoft Excel and visualize them using GRNsight to determine which one will be appropriate to pursue further in the modeling.
# First we need to properly format the output files from YEASTRACT.
#* Open the file in Excel. It will not open properly in Excel because a semicolon was used as the column delimiter instead of a comma. To fix this, Select the entire Column A. Then go to the "Data" tab and select "Text to columns". In the Wizard that appears, select "Delimited" and click "Next". In the next window, select "Semicolon", and click "Next". In the next window, leave the data format at "General", and click "Finish". This should now look like a table with the names of the transcription factors across the top and down the first column and all of the zeros and ones distributed throughout the rows and columns. This is called an "adjacency matrix." If there is a "1" in the cell, that means there is a connection between the trancription factor in that row with that column.
#* Save this file in Microsoft Excel workbook format (.xlsx).

#* For this adjacency matrix to be usable in GRNmap (the modeling software) and GRNsight (the visualization software), we need to transpose the matrix. Insert a new worksheet into your Excel file and name it "network". Go back to the previous sheet and select the entire matrix and copy it. Go to you new worksheet and click on the A1 cell in the upper left. Select "Paste special" from the "Home" tab. In the window that appears, check the box for "Transpose". This will paste your data with the columns transposed to rows and vice versa. This is necessary because we want the transcription factors that are the "regulatORS" across the top and the "regulatEES" along the side.
#* The labels for the genes in the columns and rows need to match. Thus, delete the "p" from each of the gene names in the columns. Adjust the case of the labels to make them all upper case.
#* In cell A1, copy and paste the text "rows genes affected/cols genes controlling".
#* Finally, for ease of working with the adjacency matrix in Excel, we want to alphabatize the gene labels both across the top and side.
#** Select the area of the entire adjacency matrix.
#** Click the Data tab and click the custom sort button.
#** Sort Column A alphabetically, being sure to exclude the header row.
#** Now sort row 1 from left to right, excluding cell A1. In the Custom Sort window, click on the options button and select sort left to right, excluding column 1.
#* Name the worksheet containing your organized adjacency matrix "network" and Save.
# Now we will visualize what these gene regulatory networks look like with the GRNsight software.
#* Go to the [http://dondi.github.io/GRNsight/ GRNsight] home page.
#* Select the menu item File > Open and select the regulation matrix .xlsx file that has the "network" worksheet in it that you formatted above. If the file has been formatted properly, GRNsight should automatically create a graph of your network. You can click the "Grid Layout" button to arrange the nodes in a grid, or you can click and drag the nodes (genes) around until you get a layout that you like and take a screenshot of the results. Paste it into your PowerPoint presentation.
#** If you have nodes (genes) floating around in the display that are not connected to any other nodes, we need to delete them from the network for the modeling to work properly. Go back to the Excel workbook and network sheet and delete both the row and column with the floating gene's name. Then re-upload the edited file to GRNsight to visualize it. Use this final version in your PowerPoint and subsequent modeling.
#** The deleted genes (floating) on GRNsight were:
#*** Green: MSN and BAS1

===== - Progress 12/03/19 - =====

==== Creating the GRNmap Input Workbook ====

Now that you have identified the gene regulatory network that you want to model, the next step is to generate the input Excel workbook that you will run in the GRNmap modeling software.

[https://github.com/kdahlquist/DahlquistLab/raw/master/data/Spring2019/15-genes_28-edges_sample_Sigmoid_estimation_2019.xlsx Click here to download a sample workbook] on which to base the one specific to your network and microarray data.

Note that when following the instructions below, you need to follow them precisely, to the letter, or GRNmap will return an error.

==== production_rates sheet ====

* This sheet contained the initial guesses for the production rate parameters, ''P'', for all genes in the network.
* It was Assumed that the system was in steady state with the relative expression of all genes equal to 1, (P/2) - lambda = 0, where lambda was the degradation rate, was a reasonable initial guess.
* The sheet contained two columns (from left to right) entitled, "id", "production_rate".
** The id was an identifier that the user will use to identify a particular gene. In this case, "StandardName" was what was being used, for example, GLN3.
** The "production_rate" column contained the initial guesses for the ''P'' parameter as described above, rounded to four decimal places.
*** The production rates were provided in a Microsoft Access database, which yo could be [https://github.com/kdahlquist/DahlquistLab/raw/master/data/Spring2019/Expression-and-Degradation-rate-database_2019.accdb download from here.]
*** A query was preformed to get the list of production rates for each gene as a group.
*** When the query was performed, these steps were followed.
***# Imported the list of genes to a new table in the database. Clicked on the "External Data" tab and selected the Excel icon with the "up" arrow on it.
***# Clicked the "Browse" button and selected the Excel file containing the network that was used to upload to GRNsight.
***# Made sure the button next to "Import the source data into a new table in the current database" and clicked "OK".
***# In the next window, selected the "network" worksheet, if it wasn't already automatically selected. Clicked "Next".
***# In the next window, made sure the "First Row Contains Column Headings" was checked. Clicked "Next".
***# In the next window, the left-most column was be highlighted. Changed the "Field Name" to "id". Clicked "Next".
***# In the next window, selected the button for "Choose my own primary key." and chose the "id" field from the drop down next to it. Clicked "Next".
***# In the next field, made sure it said "Import to Table: network". Clicked Finish.
***# In the next window there was no need to save the import steps, so "Close" was clicked.
***# A table called "network" appeared in the list of tables at the left of the window.
***# The "Create" tab was opened. The icon for "Query Design" was then selected.
***# In the window that appearsed, the "network" table was clicked and the "Add" option was sleeted. Clicked on the "production_rates" table and clicked the "Add". Then closed the window by clicking "Close".
***# The two tables appeared in the main part of the window. Then Access was told which fields in the two tables correspond to each other.To do that the word "id" in the network table was clicked and dragged to the "standard_name" field in the "production_rates" table, and released. Then a line appeared between those two words.
***# Right-clicked on the line between those words and selected "Join Properties" from the menu that appeared. Selected Option "2: Include ALL records from 'network' and only those records from 'production_rates' where the joined fields are equal." Clicked "OK".
***# Clicked on the "id" word in the "network" table and dragged it to the bottom of the screen to the first column next to the word "Field" and released it.
***# Clicked on the "production_rate" field in the "production_rates" table and dragged it to the bottom of the screen to the second column next to the word "Field" and released it.
***# Right-clicked anywhere in the gray area near the two tables. In the menu that appeared, selected "Query Type > Make Table Query...".
***# In the window that appeared, the table was named "production_rates_1" because their could not have been two tables with the same name in the database. Made sure that "Current Database"was selected and Clicked "OK".
***# The "Query Tools: Menus" tab was opened. Clicked on the exclamation point icon. A window appeared that gave informations on how many rows were pasting into the new table. Clicked "Yes".
***# The "production_rates_1" table then appeared in the list at the left. The table name was Double-clicked to open it.
***# The data in this table was copied and pasted it back into the Excel workbook. When pasted "Paste Special > Paste values" was used so that the Access formatting did not get carried along.
* Note that the genes should be listed in the same order in all the sheets in the Excel workbook.

==== degradation_rates sheet ====

* This sheet contained degradation rates of all genes in the network, which were provided by the user.
* Currently, the Dahlquist Lab is using data based on published mRNA half-life data from [http://rnajournal.cshlp.org/content/20/10/1645.full Neymotin et al. (2006)].
** We converted the half-life data values to the degradation rates by taking the natural log of the half-life and dividing by 2. * The sheet contained two columns (from left to right) entitled "id", and "degradation_rate".
** The id is an identifier that the user will use to identify a particular gene.
** The "degradation_rate" column should contained the absolute value of the degradation rate for the corresponding gene as described above, rounded to four decimal places.
*** To obtain these values, the same file was used, [https://github.com/kdahlquist/DahlquistLab/raw/master/data/Spring2019/Expression-and-Degradation-rate-database_2019.accdb Microsoft Access database] that was used to obtain the production rates in the first worksheet. Again, The instructions were followed to execute a query, substituting the appropriate "degradation_rates" table in the query.
* Again note, the genes were listed in the same order in all the sheets in the Excel workbook.

==== Expression Data Sheets for Individual Yeast Strains ====

* Expression data can be provided for either a single strain or multiple strains of yeast (for example, the wild type strain and a transcription factor deletion strain).
** Each strain has its own sheet in the workbook.
** Each sheet was given a unique name that follows the convention "STRAIN_log2_expression", where the word "STRAIN" was replaced by the strain designation, which will appear in the optimization_diagnostics sheet.
*** Everyone in the class had at least one expression worksheet called "wt_log2_expression".
*** The network included the transcription factors GLN3, HAP4, and CIN5. Thus, the expression data from the dGLN3, dHAP4, dCIN5 deletion strains were used in the workbooks as well, The worksheets were named "dgln3_log2_expression", "dhap4_log2_expression", and "dcin5_expression".
* The sheet had the following columns in this order:
*# "id": list of all genes. The genes were listed in the same order in all the sheets in the Excel workbook.*# The next series of columns contained the expression data for each gene at a given timepoint given as log2 ratios (log2 fold changes). The column header was the time at which the data were collected, without any units. For example, the 15 minute timepoint had the column header "15" and the 30 minute timepoint had the column header "30". GRNmap supported replicate data for each of the timepoints. Replicated data for the same timepoint should was placed in columns immediately next to each other and had the same column headers. For example, the three replicates of the 15 minute timepoint had "15", "15", "15" as the column headers.
*# Data was provided for multiple strains, each strain had data for the same timepoints, although the number of replicates varied.
* The worksheets included the data for the 15, 30, and 60 minute timepoints, but not the 90 or 120 minute timepoints.
* The data used is contained in the [https://github.com/kdahlquist/DahlquistLab/raw/master/data/Spring2019/Expression-and-Degradation-rate-database_2019.accdb Expression-and-Degradation-rate-database_2019.accdb] file that was used to obtain the production and degradation rates.
* It is tedious to copy and paste all of these data by hand, so a query was executed in Microsoft Access to do it. The steps listed for the "production_rates" sheet for each strains expression data were used to complete the query. After the data was imported into Excel, the column headers were changed to "15", "15", etc., as described above.

==== network sheet ====

* The network that wasderived from the YEASTRACT database for the [[BIOL388/S19:Week 5 | Week 5]] assignment was copied and pasted into the network sheet directly.** This sheet contained an adjacency matrix representation of the gene regulatory network.
** The columns corresponded to the transcription factors and the rows corresponded to the target genes controlled by those transcription factors.
** A “1” meant there is an edge connecting them and a “0” meant that there is no edge connecting them.
** The upper-left cell (A1) contained the text “cols regulators/rows targets”. This text was there as a reminder of the direction of the regulatory relationships specified by the adjacency matrix.
** The rest of row 1 contained the names of the transcription factors that were controlling the other genes in the network, one transcription factor name per column.
** The rest of column A contained the names of the target genes that were being controlled by the transcription factors heading each of the columns in the matrix, one target gene name per row.
** The transcription factor names corresponded to the "id" in the other sheets in the workbook. They were capitalized the same way and occurred in the same order along the top and side of the matrix. The matrix was symmetrical, i.e., the same transcription factors appeared along the top and left side of the matrix. The genes were listed in the same order in all the sheets in the Excel workbook.
** Each cell in the matrix contained a zero (0) if there was no regulatory relationship between those two transcription factors, or a one (1) if there was a regulatory relationship between them. Again, the columns corresponded to the transcription factors and the rows corresponded to the target genes controlled by those transcription factors.

==== network_weights sheet ====* These were the initial guesses for the estimation of the weight parameters, ''w''.
* Since these weights were initial guesses which will be optimized by GRNmap, the content of this sheet could be identical to the "network" sheet.

==== optimization_parameters sheet ====

* The optimization_parameters sheet had two columns (from left to right) entitled, "optimization_parameter" and "value".
* This worksheet was copied from the [https://github.com/kdahlquist/DahlquistLab/raw/master/data/Spring2019/15-genes_28-edges_sample_Sigmoid_estimation_2019.xlsx sample workbook] provided. The only row that was modified is row 15, "Strain". Included just the strain designations for which there were corresponding STRAIN_log2_expression sheet.
* What is below is an explanation of what the optimization_parameters mean.
** alpha: Penalty term weighting (from the L-curve analysis)
** kk_max: Number of times to re-run the optimization loop. In some cases re-starting the optimization loop can improve performance of the estimation.
** MaxIter: Number of times MATLAB iterates through the optimization scheme. If this is set too low, MATLAB will stop before the parameters are optimized.
** TolFun: How different two least squares evaluations should be before the program determines that it is not making any improvement
** MaxFunEval: maximum number of times the program will evaluate the least squares cost
** TolX: How close successive least squares cost evaluations should be before the program determines that it is not making any improvement.** production_function: = Sigmoid (case-insensitive) if sigmoidal model, =MM (case-insensitive) if Michaelis-Menten model
** L_curve: =0 if an L-curve analysis should NOT be run or =1 if an L-curve analysis SHOULD be run. The L-curve analysis will automatically run sequential rounds of estimation for an array of fixed alpha values (0.8, 0.5, 0.2, 0.1,0.08, 0.05,0.02,0.01, 0.008, 0.005, 0.002, 0.001, 0.0008, 0.0005, 0.0002, and 0.0001). GRNmap makes a copy of the user's selected input workbook and changes alpha to the first alpha in the list. The estimation runs and the resulting parameter values are used as the initial guesses for the next round of estimation with the next alpha value. This process repeats until all alpha values have been run. New input and output workbooks are generated for each alpha value, although currently, the graphs are only saved for the last run.
** estimate_params =1 if want to estimate parameters and =0 if the user wants to do just one forward run
** make_graphs =1 to output graphs; =0 to not output graphs
** fix_P =1 if the user does not want to estimate the production rate, ''P'', parameter, just use the initial guess and never change; =0 to estimate
** fix_b =1 if the user does not want to estimate the ''b'' parameter, just use the initial guess and never change; =0 to estimate
** expression_timepoints: A row containing a list of the time points when the data was collected experimentally. Should correspond to the timepoint column headers in the STRAIN_log2_expression sheets.
** Strain: A row containing a list of all of the strains for which there is expression data in the workbook. Should correspond to the "STRAIN" portion of the names of the STRAIN_log2_expression sheets for each strain. Note that GRNmap will run the model for the wild type network (all genes present in the network) and for networks where the gene deleted from the designated STRAIN has been deleted from the network.
** simulation_timepoints: A row containing a list of the time points at which to evaluate the differential equations to generate the simulated data. This does not need to correspond to the actual measurement times, but should be in the same units (e.g. minutes).

==== threshold_b sheet ====

* These were the initial guesses for the estimation of the ''threshold_b'' parameters.
* There should be two columns.
** The left-most column contained the header "id" and listed the standard names taken from the earlier sheets in the workbook so that they were in the same order as in the other sheets.
** The second column had the header "threshold_b" and contained the initial guesses, used all 0 for this column.=== Dynamical Systems Modeling of your Gene Regulatory Network ===
* To run GRNmap from code, had to have MATLAB R2014b installed on computer.
*# Downloaded the GRNmap v1.10 code from the [http://kdahlquist.github.io/GRNmap/downloads/ GRNmap Downloads page].
*#* [https://github.com/kdahlquist/GRNmap/archive/v1.10.zip This is a direct link to start downloading (81 MB).]
*# Unzipped the file. (Right-click, 7-zip > Extract here)
*# Launched MATLAB R2014b.
*# Open GRNmodel.m, which was in the directory that was unzipped GRNmap-1.10 > matlab
*# Clicked the Run button (green "play" arrow).
*# then there was a prompt to select your input workbook.

*# Saw an optimization diagnostics graphic that shows the progress of the estimation.
*# There were errors that occured in attempting to run this on Tuesday 11/5. [[user:Kdahlquist|Dr. Dahlquist]] is going to trouble shoot to figure out the issue with the input.
*##[[user:Kdahlquist|Dr. Dahlquist]] was able to find and correct the error in the workbook file.*# When the run was over, expression plots were displayed.
*# Output .xlsx and .mat files were be saved in the same folder as the input folder, along with .jpg files containing the optimization diagnostic and individual expression plots. These files were saved.

==== degradation_rates sheet ====

* This sheet contains degradation rates for all genes in the network, which are provided by the user.
* Currently, the Dahlquist Lab is using data based on published mRNA half-life data from [http://rnajournal.cshlp.org/content/20/10/1645.full Neymotin et al. (2006)].
** We converted the half-life data values to the degradation rates by taking the natural log of the half-life and dividing by 2.
* The sheet should contain two columns (from left to right) entitled "id", and "degradation_rate".
** The id is an identifier that the user will use to identify a particular gene.
** The "degradation_rate" column should then contain the absolute value of the degradation rate for the corresponding gene as described above, rounded to four decimal places.
*** To obtain these values, you will use the same file, [https://github.com/kdahlquist/DahlquistLab/raw/master/data/Spring2019/Expression-and-Degradation-rate-database_2019.accdb Microsoft Access database] that you used to obtain the production rates in the first worksheet. Again, you can copy and paste the values one-by-one or you can follow the instructions to execute a query, substituting the appropriate "degradation_rates" table in the query. Note that you don't need to re-import your "network" table, you just need to create and execute the query.
* Again note, the genes should be listed in the same order in all the sheets in the Excel workbook.
* If there are missing values, substitute the value <code>0.0990</code> for the missing degradation rates.

==== Expression Data Sheets for Individual Yeast Strains ====

* Expression data can be provided for either a single strain or multiple strains of yeast (for example, the wild type strain and a transcription factor deletion strain).
** Each strain will have its own sheet in the workbook.
** Each sheet should be given a unique name that follows the convention "STRAIN_log2_expression", where the word "STRAIN" is replaced by the strain designation, which will appear in the optimization_diagnostics sheet.
*** Everyone in the class will have at least one expression worksheet called "wt_log2_expression".
*** You should have included the transcription factors GLN3, HAP4, and CIN5 in your network. Thus, we will use the expression data from the dGLN3, dHAP4, dCIN5 deletion strains in our workbooks as well, naming the worksheets "dgln3_log2_expression", "dhap4_log2_expression", and "dcin5_expression".
**** If, for some reason, you don't have all three of those genes in your network, only include expression data for the wild type and the genes out of those three that you have in your network.
* The sheet should have the following columns in this order:
*# "id": list of all genes. The genes should be listed in the same order in all the sheets in the Excel workbook.
*# The next series of columns should contain the expression data for each gene at a given timepoint given as log2 ratios (log2 fold changes). The column header should be the time at which the data were collected, without any units. For example, the 15 minute timepoint would have a column header "15" and the 30 minute timepoint would have the column header "30". GRNmap supports replicate data for each of the timepoints. Replicate data for the same timepoint should be in columns immediately next to each other and have the same column headers. For example, three replicates of the 15 minute timepoint would have "15", "15", "15" as the column headers.
*# If data are provided for multiple strains, each strain should have data for the same timepoints, although the number of replicates can vary.
* Include the data for the 15, 30, and 60 minute timepoints, but not the 90 or 120 minute timepoints.
* The data you will be using is contained in the [https://github.com/kdahlquist/DahlquistLab/raw/master/data/Spring2019/Expression-and-Degradation-rate-database_2019.accdb Expression-and-Degradation-rate-database_2019.accdb] file that you used to obtain the production and degradation rates.
* It is tedious to copy and paste all of these data by hand, so we will execute a query in Microsoft Access to do it for you. Follow the steps listed for the "production_rates" sheet for each strains expression data. After you import the data into Excel, you will need to change the column headers to "15", "15", etc., as described above.
* Missing values in the expression data sheets are OK; you don't need to put any values there like you did for the production_rates or degradation_rates sheets.

==== network sheet ====

* The network you derived from the YEASTRACT database for the [[Week 9]] assignment can be copied and pasted into this sheet directly. You may need to edit the contents of cell A1, but the rest should be good to go (especially since you previewed it in GRNsight). The description below just explains what is already in this worksheet.
** This sheet contains an adjacency matrix representation of the gene regulatory network.
** The columns correspond to the transcription factors and the rows correspond to the target genes controlled by those transcription factors.
** A “1” means there is an edge connecting them and a “0” means that there is no edge connecting them.
** The upper-left cell (A1) should contain the text “cols regulators/rows targets”. This text is there as a reminder of the direction of the regulatory relationships specified by the adjacency matrix.
** The rest of row 1 should contain the names of the transcription factors that are controlling the other genes in the network, one transcription factor name per column.
** The rest of column A should contain the names of the target genes that are being controlled by the transcription factors heading each of the columns in the matrix, one target gene name per row.
** The transcription factor names should correspond to the "id" in the other sheets in the workbook. They should be capitalized the same way and occur in the same order along the top and side of the matrix. The matrix needs to be symmetric, i.e., the same transcription factors should appear along the top and left side of the matrix. The genes should be listed in the same order in all the sheets in the Excel workbook.
** Each cell in the matrix should then contain a zero (0) if there is no regulatory relationship between those two transcription factors, or a one (1) if there is a regulatory relationship between them. Again, the columns correspond to the transcription factors and the rows correspond to the target genes controlled by those transcription factors.

==== network_weights sheet ====

* These are the initial guesses for the estimation of the weight parameters, ''w''.
* Since these weights are initial guesses which will be optimized by GRNmap, the content of this sheet can be identical to the "network" sheet.

==== optimization_parameters sheet ====

* The optimization_parameters sheet should have two columns (from left to right) entitled, "optimization_parameter" and "value".
* You should copy this worksheet from the [https://github.com/kdahlquist/DahlquistLab/raw/master/data/Spring2019/15-genes_28-edges_sample_Sigmoid_estimation_2019.xlsx sample workbook] provided. The only row that you need to modify is row 15, "Strain". Include just the strain designations for which you have a corresponding STRAIN_log2_expression sheet. If you don't have the dgln3, dhap4, or dcin5 expression sheets, then you will delete those from this row. If you do so, make sure that you don't leave any gaps between cells.
* What follows below is an explanation of what the optimization_parameters mean.
** alpha: Penalty term weighting (from the L-curve analysis)
** kk_max: Number of times to re-run the optimization loop. In some cases re-starting the optimization loop can improve performance of the estimation.
** MaxIter: Number of times MATLAB iterates through the optimization scheme. If this is set too low, MATLAB will stop before the parameters are optimized.
** TolFun: How different two least squares evaluations should be before the program determines that it is not making any improvement
** MaxFunEval: maximum number of times the program will evaluate the least squares cost
** TolX: How close successive least squares cost evaluations should be before the program determines that it is not making any improvement.
** production_function: = Sigmoid (case-insensitive) if sigmoidal model, =MM (case-insensitive) if Michaelis-Menten model
** L_curve: =0 if an L-curve analysis should NOT be run or =1 if an L-curve analysis SHOULD be run. The L-curve analysis will automatically run sequential rounds of estimation for an array of fixed alpha values (0.8, 0.5, 0.2, 0.1,0.08, 0.05,0.02,0.01, 0.008, 0.005, 0.002, 0.001, 0.0008, 0.0005, 0.0002, and 0.0001). GRNmap makes a copy of the user's selected input workbook and changes alpha to the first alpha in the list. The estimation runs and the resulting parameter values are used as the initial guesses for the next round of estimation with the next alpha value. This process repeats until all alpha values have been run. New input and output workbooks are generated for each alpha value, although currently, the graphs are only saved for the last run.
** estimate_params =1 if want to estimate parameters and =0 if the user wants to do just one forward run
** make_graphs =1 to output graphs; =0 to not output graphs
** fix_P =1 if the user does not want to estimate the production rate, ''P'', parameter, just use the initial guess and never change; =0 to estimate
** fix_b =1 if the user does not want to estimate the ''b'' parameter, just use the initial guess and never change; =0 to estimate
** expression_timepoints: A row containing a list of the time points when the data was collected experimentally. Should correspond to the timepoint column headers in the STRAIN_log2_expression sheets.
** Strain: A row containing a list of all of the strains for which there is expression data in the workbook. Should correspond to the "STRAIN" portion of the names of the STRAIN_log2_expression sheets for each strain. Note that GRNmap will run the model for the wild type network (all genes present in the network) and for networks where the gene deleted from the designated STRAIN has been deleted from the network.
** simulation_timepoints: A row containing a list of the time points at which to evaluate the differential equations to generate the simulated data. This does not need to correspond to the actual measurement times, but should be in the same units (e.g. minutes).

==== threshold_b sheet ====

* These are the initial guesses for the estimation of the ''threshold_b'' parameters.
* There should be two columns.
** The left-most column should contain the header "id" and list the standard names for the genes in the model in the same order as in the other sheets.
** The second column should have the header "threshold_b" and should contain the initial guesses, we are going to use all 0.

====Visualizing in GRNmap results in GRNsight====

== Conclusion ==
The first stage of our group's project was completed via referencing [[Week 8]] and using Microsoft Excel to complete the tasks. The excel file will be located in the [[FunGals]] page for viewing and download. We were able to successfully complete the ANOVA analysis and correct found mistakes. We were able to complete a sanity check with results showing. For the sanity check the unadjusted p- values showed 37.2% of the genes had a p<0.05, 18.16% had a p<0.01, 5.04% have p<0.001, and 0.87% have p<0.0001. The sanity check for the Bonferroni-corrected p-value 0.11% of the genes have p<0.05. For the sanity check on the Benjamini and Hochberg-corrected p-value 16.36% go the genes had a p <0.05. Finally we were able to prepare the Data to run STEM in the next step of working on this project.

Extra Notes (will delete later): PDR3 did not show up in Access while running Query's

==Data and files==
[[media:Thiuram_yeast_experiment.xlsx|ANOVA & STEM workbook (.xlsx)]]

[[media:Genelist_combined_profiles_FunGals.xlsx| YEASTRACT Rank by TF - Green and Red Profiles (.xlsx)]]

[[media:FunGals_Green_Profile_RegulationMatrix_Documented_2019123_2322_1272399515.xlsx| Green Regulation Matrix/network (.xlsx)]]

[[media:FunGals Red Profile RegulationMatrix Documented 2019123 2318 1127808084.xlsx| Input Workbook Red network (.xlsx)]]

[[media:FunGals Green Profile RegulationMatrix Input Workbook.xlsx | Input Workbook for Green profile (.xlsx)]]

[[media:GRNmap FunGals Green profile.zip | Output Workbook & GRNModel from MatLab for Green profile (.zip)]]

[[Media:GRN_Model_Red_from_Matlab.zip | Output Workbook & GRNModel from MatLab for Red profile (.zip)]]

[[media:Data_for_FunGals.pptx|FunGals Data on Powerpoint]]

== Acknowledgements ==
This section is in acknowledgement to partner Kaitlyn Nguyen (User:knguye66), Michael Armas (User:Marmas), as well as, Iliana Crespin (User:Icrespin), and Emma Young (User:eyoung20). We would also like to acknowledge Dr. Dahlquist (User:KDahlquist) for introducing and teaching the topic and direction of this assignment.
Also to acknowledge that this is a shared electronic notebook between [[user:knguye66|Kaitlyn Nguyen]] and [[user:eyoung20|Emma Young]].

"Except for what is noted above, this individual journal entry was completed by me and not copied from another source."
[[User:Knguye66|Knguye66]] ([[User talk:Knguye66|talk]]) 18:49, 20 November 2019 (PST)

"Except for what is noted above, this individual journal entry was completed by me and not copied from another source."
[[User:Eyoung20|Eyoung20]] ([[User talk:Eyoung20|talk]]) 16:40, 25 November 2019 (PST)

== References ==
*Dahlquist, K. (2019, November 19). Data Analysis. In Wikipedia, Biological Databases. Retrieved 6:25, November 20, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Data_Analysis
*Dahlquist, K. (2019, November 20). Final Project Deliverables. In Wikipedia, Biological Databases. Retrieved 2:59, December 5, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Final_Project_Deliverables
*Dahlquist, K. (2019, October 29). Week 9. In Wikipedia, Biological Databases. Retrieved 2:57, December 2, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_9
*Dahlquist, K. (2019, November 7). Week 10. In Wikipedia, Biological Databases. Retrieved 2:59, December 5, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_10

{{Template:knguye66}}
{{Template:Eyoung20}}

[[Category: Group Projects]]
[[Category: FunGals]]

Knguye66 Eyoung20 Week 15

2019-12-13T19:37:32Z

Eyoung20: /* production_rates sheet */

'''Combined Individual Journals for Kaitlyn Nguyen and Emma Young (Data Analysts).'''

== Purpose ==
The purpose of this assignment is to record our progress towards the [[FunGals]] group [[Final_Project_Deliverables | deliverables]] as the [[Data_Analysis | Data Analysts]] for this week and the future weeks to come. The purpose of week 12 specifically was to download and adapt the data to the formatting we need for analysis. Then to begin the analysis with ANOVA and preparations for STEM.

== Methods and Results: Progress ==
===== - Progress 11/26/19 - =====

==== Analyzing and Interpreting STEM Results ====
#Why did you select this profile? In other words, why was it interesting to you?
#* We collectively combined the 4 red profiles together because they had similar trends and to increase the number of genes for analysis (called "Red".) Similarly this was done to the 3 green profiles as well (upward trends), with the addition of the blue profile #29 added. We will call this group "Green".
#How many genes belong to this profile?
#* Red (composed of Profile #9,26,34,11): 289
#* Green (Profile #40,42,18,29): 214
#How many genes were expected to belong to this profile?
#* Red (composed of Profile #9,26,34,11): 51.5
#* Green (Profile #40,42,18,29): 48.5
#What is the p value for the enrichment of genes in this profile?
#* Due to combining the profiles, we do not have p-values for the enrichment of genes in the 2 different groups.

- '''Viewing and Saving STEM Results''' -
# A new window will open called "All STEM Profiles (1)". Each box corresponds to a model expression profile. Colored profiles have a statistically significant number of genes assigned; they are arranged in order from most to least significant p value. Profiles with the same color belong to the same cluster of profiles. The number in each box is simply an ID number for the profile.
#* Click on the button that says "Interface Options...". At the bottom of the Interface Options window that appears below where it says "X-axis scale should be:", click on the radio button that says "Based on real time". Then close the Interface Options window.
#*Take a screenshot of this window (on a PC, simultaneously press the <code>Alt</code> and <code>PrintScreen</code> buttons to save the view in the active window to the clipboard) and paste it into a PowerPoint presentation to save your figures.
# Click on each of the SIGNIFICANT profiles (the colored ones) to open a window showing a more detailed plot containing all of the genes in that profile.
#* Take a screenshot of each of the individual profile windows and save the images in your PowerPoint presentation.
#* At the bottom of each profile window, there are two yellow buttons "Profile Gene Table" and "Profile GO Table". For each of the profiles, click on the "Profile Gene Table" button to see the list of genes belonging to the profile. In the window that appears, click on the "Save Table" button and save the file to your desktop. Make your filename descriptive of the contents, e.g. "wt_profile#_genelist.txt", where you replace the number symbol with the actual profile number.
#** Upload these files to the wiki and link to them on your individual journal page. (Note that it will be easier to [[Week_4#Compressing_and_Decompressing_Files_with_7-Zip | zip all the files together]] and upload them as one file).
#* For each of the significant profiles, click on the "Profile GO Table" to see the list of Gene Ontology terms belonging to the profile. In the window that appears, click on the "Save Table" button and save the file to your desktop. Make your filename descriptive of the contents, e.g. "wt_profile#_GOlist.txt", where you use "wt", "dGLN3", etc. to indicate the dataset and where you replace the number symbol with the actual profile number. At this point you have saved all of the primary data from the STEM software and it's time to interpret the results!
#** Upload these files to the wiki and link to them on your individual journal page. (Note that it will be easier to [[Week_4#Compressing_and_Decompressing_Files_with_7-Zip | zip all the files together]] and upload them as one file).

==== Clustering the data with STEM, as did on [[Week 9]]. ====
# Note that we will make some adjustments to the GO term analysis because stem was not providing GO term names. We are going to use the GO enrichment tool at GeneOntology.org instead.
#* Go to [http://geneontology.org/ http://geneontology.org/].
#* For the cluster you want to analyze, open the gene list and copy the list of genes.
#* Paste the list of genes into the "Go Enrichment Analysis" box on the right hand side of the GeneOntology.org page.
#* Select "Saccharomyces cerevisiae" from the species drop-down menu.
# Click the "Launch" buton.
#* Near the bottom of the results page, click on the button to Export "Table".
#* This will prompt you to save a .txt file that can be opened in Excel to view your results.
# Use YEASTRACT to generate a candidate gene regulatory network as you did on [[Week 9]].

==== Using YEASTRACT to Infer which Transcription Factors Regulate a Cluster of Genes ====

In the previous analysis using STEM, we found a number of gene expression profiles (aka clusters) which grouped genes based on similarity of gene expression changes over time. The implication is that these genes share the same expression pattern because they are regulated by the same (or the same set) of transcription factors. We will explore this using the YEASTRACT database.

# Open the gene list in Excel for the one of the significant profiles from your stem analysis. Choose a cluster with a clear cold shock/recovery up/down or down/up pattern. You should also choose one of the largest clusters.
#* Copy the list of gene IDs onto your clipboard.
# Launch a web browser and go to the [http://www.yeastract.com/ YEASTRACT database].
#* On the left panel of the window, click on the link to [http://www.yeastract.com/formrankbytf.php ''Rank by TF''].
#* Paste your list of genes from your cluster into the box labeled ''ORFs/Genes''.
#* Check the box for ''Check for all TFs''.
#* Accept the defaults for the Regulations Filter (Documented, DNA binding plus expression evidence)
#* Do '''''not''''' apply a filter for "Filter Documented Regulations by environmental condition".
#* Rank genes by TF using: The % of genes in the list and in YEASTRACT regulated by each TF.
#* Click the ''Search'' button.
# Answer the following questions:
#* In the results window that appears, the p values colored green are considered "significant", the ones colored yellow are considered "borderline significant" and the ones colored pink are considered "not significant". '''''How many transcription factors are green or "significant"?'''''
#**Green: 31
#**Red: 30
#* '''''Copy the table of results from the web page and paste it into a new Excel workbook to preserve the results.'''''
#** '''''Upload the Excel file to the wiki and link to it in your electronic lab notebook.'''''
#** '''''Are CIN5, GLN3, and/or HAP4 on the list? If so, what is their "% in user set", "% in YEASTRACT", and "p value".'''''
#**Green
#*** CIN5
#**** 0.2991 % in user set
#****0.0294
#**** p-value: 0.5492
#***GLN3
#**** 0.3645% in user set
#****0.0324% in YEASTRACT
#**** p-value: 0.18046
#***HAP4
#**** 0.2383% in user set
#****0.0467% in YEASTRACT
#**** P-value:0.00031337
#**Red
#*** CIN5
#**** 0.2111%% in user set
#**** 0.028% in YEASTRACT
#**** p-value: 0.9998
#***GLN3
#**** 0.4152% in user set
#****0.0498% in YEASTRACT
#**** p-value:0.00207752
#***HAP4
#****0.2353% in user set
#****0.0623% in YEASTRACT
#**** P-value:0.000006235
# For the mathematical model that we will build, we need to define a ''gene regulatory network'' of transcription factors that regulate other transcription factors. We can use YEASTRACT to assist us with creating the network. We want to generate a network with approximately 15-20 transcription factors in it.
#* You need to select from this list of "significant" transcription factors, which ones you will use to run the model. You will use these transcription factors and add GLN3, HAP4, and CIN5 if they are not in your list. Explain in your electronic notebook how you decided on which transcription factors to include. Record the list and your justification in your electronic lab notebook. Each group member will select a different network (they can have some overlapping transcription factors, but some should also be different).
#* Go back to the YEASTRACT database and follow the link to ''[http://www.yeastract.com/formregmatrix.php Generate Regulation Matrix]''.
#* Copy and paste the list of transcription factors you identified (plus HAP4, GLN3, and CIN5) into both the "Transcription factors" field and the "Target ORF/Genes" field.
#* We are going to use the "Regulations Filter" options of "Documented", "'''Only''' DNA binding evidence"
#** Click the "Generate" button.
#** In the results window that appears, click on the link to the "Regulation matrix (Semicolon Separated Values (CSV) file)" that appears and save it to your Desktop. Rename this file with a meaningful name so that you can distinguish it from the other files you will generate.

==== Visualizing Your Gene Regulatory Networks with GRNsight ====

We will analyze the regulatory matrix files you generated above in Microsoft Excel and visualize them using GRNsight to determine which one will be appropriate to pursue further in the modeling.
# First we need to properly format the output files from YEASTRACT.
#* Open the file in Excel. It will not open properly in Excel because a semicolon was used as the column delimiter instead of a comma. To fix this, Select the entire Column A. Then go to the "Data" tab and select "Text to columns". In the Wizard that appears, select "Delimited" and click "Next". In the next window, select "Semicolon", and click "Next". In the next window, leave the data format at "General", and click "Finish". This should now look like a table with the names of the transcription factors across the top and down the first column and all of the zeros and ones distributed throughout the rows and columns. This is called an "adjacency matrix." If there is a "1" in the cell, that means there is a connection between the trancription factor in that row with that column.
#* Save this file in Microsoft Excel workbook format (.xlsx).

#* For this adjacency matrix to be usable in GRNmap (the modeling software) and GRNsight (the visualization software), we need to transpose the matrix. Insert a new worksheet into your Excel file and name it "network". Go back to the previous sheet and select the entire matrix and copy it. Go to you new worksheet and click on the A1 cell in the upper left. Select "Paste special" from the "Home" tab. In the window that appears, check the box for "Transpose". This will paste your data with the columns transposed to rows and vice versa. This is necessary because we want the transcription factors that are the "regulatORS" across the top and the "regulatEES" along the side.
#* The labels for the genes in the columns and rows need to match. Thus, delete the "p" from each of the gene names in the columns. Adjust the case of the labels to make them all upper case.
#* In cell A1, copy and paste the text "rows genes affected/cols genes controlling".
#* Finally, for ease of working with the adjacency matrix in Excel, we want to alphabatize the gene labels both across the top and side.
#** Select the area of the entire adjacency matrix.
#** Click the Data tab and click the custom sort button.
#** Sort Column A alphabetically, being sure to exclude the header row.
#** Now sort row 1 from left to right, excluding cell A1. In the Custom Sort window, click on the options button and select sort left to right, excluding column 1.
#* Name the worksheet containing your organized adjacency matrix "network" and Save.
# Now we will visualize what these gene regulatory networks look like with the GRNsight software.
#* Go to the [http://dondi.github.io/GRNsight/ GRNsight] home page.
#* Select the menu item File > Open and select the regulation matrix .xlsx file that has the "network" worksheet in it that you formatted above. If the file has been formatted properly, GRNsight should automatically create a graph of your network. You can click the "Grid Layout" button to arrange the nodes in a grid, or you can click and drag the nodes (genes) around until you get a layout that you like and take a screenshot of the results. Paste it into your PowerPoint presentation.
#** If you have nodes (genes) floating around in the display that are not connected to any other nodes, we need to delete them from the network for the modeling to work properly. Go back to the Excel workbook and network sheet and delete both the row and column with the floating gene's name. Then re-upload the edited file to GRNsight to visualize it. Use this final version in your PowerPoint and subsequent modeling.
#** The deleted genes (floating) on GRNsight were:
#*** Green: MSN and BAS1

===== - Progress 12/03/19 - =====

==== Creating the GRNmap Input Workbook ====

Now that you have identified the gene regulatory network that you want to model, the next step is to generate the input Excel workbook that you will run in the GRNmap modeling software.

[https://github.com/kdahlquist/DahlquistLab/raw/master/data/Spring2019/15-genes_28-edges_sample_Sigmoid_estimation_2019.xlsx Click here to download a sample workbook] on which to base the one specific to your network and microarray data.

Note that when following the instructions below, you need to follow them precisely, to the letter, or GRNmap will return an error.

==== production_rates sheet ====

* This sheet contained the initial guesses for the production rate parameters, ''P'', for all genes in the network.
* It was Assumed that the system was in steady state with the relative expression of all genes equal to 1, (P/2) - lambda = 0, where lambda was the degradation rate, was a reasonable initial guess.
* The sheet contained two columns (from left to right) entitled, "id", "production_rate".
** The id was an identifier that the user will use to identify a particular gene. In this case, "StandardName" was what was being used, for example, GLN3.
** The "production_rate" column contained the initial guesses for the ''P'' parameter as described above, rounded to four decimal places.
*** The production rates were provided in a Microsoft Access database, which yo could be [https://github.com/kdahlquist/DahlquistLab/raw/master/data/Spring2019/Expression-and-Degradation-rate-database_2019.accdb download from here.]
*** A query was preformed to get the list of production rates for each gene as a group.
*** When the query was performed, these steps were followed.
***# Imported the list of genes to a new table in the database. Clicked on the "External Data" tab and selected the Excel icon with the "up" arrow on it.
***# Clicked the "Browse" button and selected the Excel file containing the network that was used to upload to GRNsight.
***# Made sure the button next to "Import the source data into a new table in the current database" and clicked "OK".
***# In the next window, selected the "network" worksheet, if it wasn't already automatically selected. Clicked "Next".
***# In the next window, made sure the "First Row Contains Column Headings" was checked. Clicked "Next".
***# In the next window, the left-most column was be highlighted. Changed the "Field Name" to "id". Clicked "Next".
***# In the next window, selected the button for "Choose my own primary key." and chose the "id" field from the drop down next to it. Clicked "Next".
***# In the next field, made sure it said "Import to Table: network". Clicked Finish.
***# In the next window there was no need to save the import steps, so "Close" was clicked.
***# A table called "network" appeared in the list of tables at the left of the window.
***# The "Create" tab was opened. The icon for "Query Design" was then selected.
***# In the window that appearsed, the "network" table was clicked and the "Add" option was sleeted. Clicked on the "production_rates" table and clicked the "Add". Then closed the window by clicking "Close".
***# The two tables appeared in the main part of the window. Then Access was told which fields in the two tables correspond to each other.To do that the word "id" in the network table was clicked and dragged to the "standard_name" field in the "production_rates" table, and released. Then a line appeared between those two words.
***# Right-clicked on the line between those words and selected "Join Properties" from the menu that appeared. Selected Option "2: Include ALL records from 'network' and only those records from 'production_rates' where the joined fields are equal." Clicked "OK".
***# Clicked on the "id" word in the "network" table and dragged it to the bottom of the screen to the first column next to the word "Field" and released it.
***# Clicked on the "production_rate" field in the "production_rates" table and dragged it to the bottom of the screen to the second column next to the word "Field" and released it.
***# Right-clicked anywhere in the gray area near the two tables. In the menu that appeared, selected "Query Type > Make Table Query...".
***# In the window that appeared, the table was named "production_rates_1" because their could not have been two tables with the same name in the database. Made sure that "Current Database"was selected and Clicked "OK".
***# The "Query Tools: Menus" tab was opened. Clicked on the exclamation point icon. A window appeared that gave informations on how many rows were pasting into the new table. Clicked "Yes".
***# The "production_rates_1" table then appeared in the list at the left. The table name was Double-clicked to open it.
***# The data in this table was copied and pasted it back into the Excel workbook. When pasted "Paste Special > Paste values" was used so that the Access formatting did not get carried along.
* Note that the genes should be listed in the same order in all the sheets in the Excel workbook.

==== degradation_rates sheet ====

* This sheet contained degradation rates of all genes in the network, which were provided by the user.
* Currently, the Dahlquist Lab is using data based on published mRNA half-life data from [http://rnajournal.cshlp.org/content/20/10/1645.full Neymotin et al. (2006)].
** We converted the half-life data values to the degradation rates by taking the natural log of the half-life and dividing by 2. * The sheet contained two columns (from left to right) entitled "id", and "degradation_rate".
** The id is an identifier that the user will use to identify a particular gene.
** The "degradation_rate" column should contained the absolute value of the degradation rate for the corresponding gene as described above, rounded to four decimal places.
*** To obtain these values, the same file was used, [https://github.com/kdahlquist/DahlquistLab/raw/master/data/Spring2019/Expression-and-Degradation-rate-database_2019.accdb Microsoft Access database] that was used to obtain the production rates in the first worksheet. Again, The instructions were followed to execute a query, substituting the appropriate "degradation_rates" table in the query.
* Again note, the genes were listed in the same order in all the sheets in the Excel workbook.

==== Expression Data Sheets for Individual Yeast Strains ====

* Expression data can be provided for either a single strain or multiple strains of yeast (for example, the wild type strain and a transcription factor deletion strain).
** Each strain has its own sheet in the workbook.
** Each sheet was given a unique name that follows the convention "STRAIN_log2_expression", where the word "STRAIN" was replaced by the strain designation, which will appear in the optimization_diagnostics sheet.
*** Everyone in the class had at least one expression worksheet called "wt_log2_expression".
*** The network included the transcription factors GLN3, HAP4, and CIN5. Thus, the expression data from the dGLN3, dHAP4, dCIN5 deletion strains were used in the workbooks as well, The worksheets were named "dgln3_log2_expression", "dhap4_log2_expression", and "dcin5_expression".
* The sheet had the following columns in this order:
*# "id": list of all genes. The genes were listed in the same order in all the sheets in the Excel workbook.*# The next series of columns contained the expression data for each gene at a given timepoint given as log2 ratios (log2 fold changes). The column header was the time at which the data were collected, without any units. For example, the 15 minute timepoint had the column header "15" and the 30 minute timepoint had the column header "30". GRNmap supported replicate data for each of the timepoints. Replicated data for the same timepoint should was placed in columns immediately next to each other and had the same column headers. For example, the three replicates of the 15 minute timepoint had "15", "15", "15" as the column headers.
*# Data was provided for multiple strains, each strain had data for the same timepoints, although the number of replicates varied.
* The worksheets included the data for the 15, 30, and 60 minute timepoints, but not the 90 or 120 minute timepoints.
* The data used is contained in the [https://github.com/kdahlquist/DahlquistLab/raw/master/data/Spring2019/Expression-and-Degradation-rate-database_2019.accdb Expression-and-Degradation-rate-database_2019.accdb] file that was used to obtain the production and degradation rates.
* It is tedious to copy and paste all of these data by hand, so a query was executed in Microsoft Access to do it. The steps listed for the "production_rates" sheet for each strains expression data were used to complete the query. After the data was imported into Excel, the column headers were changed to "15", "15", etc., as described above.

==== network sheet ====

* The network that wasderived from the YEASTRACT database for the [[BIOL388/S19:Week 5 | Week 5]] assignment was copied and pasted into the network sheet directly.** This sheet contained an adjacency matrix representation of the gene regulatory network.
** The columns corresponded to the transcription factors and the rows corresponded to the target genes controlled by those transcription factors.
** A “1” meant there is an edge connecting them and a “0” meant that there is no edge connecting them.
** The upper-left cell (A1) contained the text “cols regulators/rows targets”. This text was there as a reminder of the direction of the regulatory relationships specified by the adjacency matrix.
** The rest of row 1 contained the names of the transcription factors that were controlling the other genes in the network, one transcription factor name per column.
** The rest of column A contained the names of the target genes that were being controlled by the transcription factors heading each of the columns in the matrix, one target gene name per row.
** The transcription factor names corresponded to the "id" in the other sheets in the workbook. They were capitalized the same way and occurred in the same order along the top and side of the matrix. The matrix was symmetrical, i.e., the same transcription factors appeared along the top and left side of the matrix. The genes were listed in the same order in all the sheets in the Excel workbook.
** Each cell in the matrix contained a zero (0) if there was no regulatory relationship between those two transcription factors, or a one (1) if there was a regulatory relationship between them. Again, the columns corresponded to the transcription factors and the rows corresponded to the target genes controlled by those transcription factors.

==== network_weights sheet ====* These were the initial guesses for the estimation of the weight parameters, ''w''.
* Since these weights were initial guesses which will be optimized by GRNmap, the content of this sheet could be identical to the "network" sheet.

==== optimization_parameters sheet ====

* The optimization_parameters sheet had two columns (from left to right) entitled, "optimization_parameter" and "value".
* This worksheet was copied from the [https://github.com/kdahlquist/DahlquistLab/raw/master/data/Spring2019/15-genes_28-edges_sample_Sigmoid_estimation_2019.xlsx sample workbook] provided. The only row that was modified is row 15, "Strain". Included just the strain designations for which there were corresponding STRAIN_log2_expression sheet.
* What is below is an explanation of what the optimization_parameters mean.
** alpha: Penalty term weighting (from the L-curve analysis)
** kk_max: Number of times to re-run the optimization loop. In some cases re-starting the optimization loop can improve performance of the estimation.
** MaxIter: Number of times MATLAB iterates through the optimization scheme. If this is set too low, MATLAB will stop before the parameters are optimized.
** TolFun: How different two least squares evaluations should be before the program determines that it is not making any improvement
** MaxFunEval: maximum number of times the program will evaluate the least squares cost
** TolX: How close successive least squares cost evaluations should be before the program determines that it is not making any improvement.** production_function: = Sigmoid (case-insensitive) if sigmoidal model, =MM (case-insensitive) if Michaelis-Menten model
** L_curve: =0 if an L-curve analysis should NOT be run or =1 if an L-curve analysis SHOULD be run. The L-curve analysis will automatically run sequential rounds of estimation for an array of fixed alpha values (0.8, 0.5, 0.2, 0.1,0.08, 0.05,0.02,0.01, 0.008, 0.005, 0.002, 0.001, 0.0008, 0.0005, 0.0002, and 0.0001). GRNmap makes a copy of the user's selected input workbook and changes alpha to the first alpha in the list. The estimation runs and the resulting parameter values are used as the initial guesses for the next round of estimation with the next alpha value. This process repeats until all alpha values have been run. New input and output workbooks are generated for each alpha value, although currently, the graphs are only saved for the last run.
** estimate_params =1 if want to estimate parameters and =0 if the user wants to do just one forward run
** make_graphs =1 to output graphs; =0 to not output graphs
** fix_P =1 if the user does not want to estimate the production rate, ''P'', parameter, just use the initial guess and never change; =0 to estimate
** fix_b =1 if the user does not want to estimate the ''b'' parameter, just use the initial guess and never change; =0 to estimate
** expression_timepoints: A row containing a list of the time points when the data was collected experimentally. Should correspond to the timepoint column headers in the STRAIN_log2_expression sheets.
** Strain: A row containing a list of all of the strains for which there is expression data in the workbook. Should correspond to the "STRAIN" portion of the names of the STRAIN_log2_expression sheets for each strain. Note that GRNmap will run the model for the wild type network (all genes present in the network) and for networks where the gene deleted from the designated STRAIN has been deleted from the network.
** simulation_timepoints: A row containing a list of the time points at which to evaluate the differential equations to generate the simulated data. This does not need to correspond to the actual measurement times, but should be in the same units (e.g. minutes).

==== threshold_b sheet ====

* These were the initial guesses for the estimation of the ''threshold_b'' parameters.
* There should be two columns.
** The left-most column contained the header "id" and listed the standard names taken from the earlier sheets in the workbook so that they were in the same order as in the other sheets.
** The second column had the header "threshold_b" and contained the initial guesses, used all 0 for this column.=== Dynamical Systems Modeling of your Gene Regulatory Network ===
* To run GRNmap from code, had to have MATLAB R2014b installed on computer.
*# Downloaded the GRNmap v1.10 code from the [http://kdahlquist.github.io/GRNmap/downloads/ GRNmap Downloads page].
*#* [https://github.com/kdahlquist/GRNmap/archive/v1.10.zip This is a direct link to start downloading (81 MB).]
*# Unzipped the file. (Right-click, 7-zip > Extract here)
*# Launched MATLAB R2014b.
*# Open GRNmodel.m, which was in the directory that was unzipped GRNmap-1.10 > matlab
*# Clicked the Run button (green "play" arrow).
*# then there was a prompt to select your input workbook.

*# Saw an optimization diagnostics graphic that shows the progress of the estimation.
*# There were errors that occured in attempting to run this on Tuesday 11/5. [[user:Kdahlquist|Dr. Dahlquist]] is going to trouble shoot to figure out the issue with the input.
*##[[user:Kdahlquist|Dr. Dahlquist]] was able to find and correct the error in the workbook file.*# When the run was over, expression plots were displayed.
*# Output .xlsx and .mat files were be saved in the same folder as the input folder, along with .jpg files containing the optimization diagnostic and individual expression plots. These files were saved.

==== degradation_rates sheet ====

* This sheet contains degradation rates for all genes in the network, which are provided by the user.
* Currently, the Dahlquist Lab is using data based on published mRNA half-life data from [http://rnajournal.cshlp.org/content/20/10/1645.full Neymotin et al. (2006)].
** We converted the half-life data values to the degradation rates by taking the natural log of the half-life and dividing by 2.
* The sheet should contain two columns (from left to right) entitled "id", and "degradation_rate".
** The id is an identifier that the user will use to identify a particular gene.
** The "degradation_rate" column should then contain the absolute value of the degradation rate for the corresponding gene as described above, rounded to four decimal places.
*** To obtain these values, you will use the same file, [https://github.com/kdahlquist/DahlquistLab/raw/master/data/Spring2019/Expression-and-Degradation-rate-database_2019.accdb Microsoft Access database] that you used to obtain the production rates in the first worksheet. Again, you can copy and paste the values one-by-one or you can follow the instructions to execute a query, substituting the appropriate "degradation_rates" table in the query. Note that you don't need to re-import your "network" table, you just need to create and execute the query.
* Again note, the genes should be listed in the same order in all the sheets in the Excel workbook.
* If there are missing values, substitute the value <code>0.0990</code> for the missing degradation rates.

==== Expression Data Sheets for Individual Yeast Strains ====

* Expression data can be provided for either a single strain or multiple strains of yeast (for example, the wild type strain and a transcription factor deletion strain).
** Each strain will have its own sheet in the workbook.
** Each sheet should be given a unique name that follows the convention "STRAIN_log2_expression", where the word "STRAIN" is replaced by the strain designation, which will appear in the optimization_diagnostics sheet.
*** Everyone in the class will have at least one expression worksheet called "wt_log2_expression".
*** You should have included the transcription factors GLN3, HAP4, and CIN5 in your network. Thus, we will use the expression data from the dGLN3, dHAP4, dCIN5 deletion strains in our workbooks as well, naming the worksheets "dgln3_log2_expression", "dhap4_log2_expression", and "dcin5_expression".
**** If, for some reason, you don't have all three of those genes in your network, only include expression data for the wild type and the genes out of those three that you have in your network.
* The sheet should have the following columns in this order:
*# "id": list of all genes. The genes should be listed in the same order in all the sheets in the Excel workbook.
*# The next series of columns should contain the expression data for each gene at a given timepoint given as log2 ratios (log2 fold changes). The column header should be the time at which the data were collected, without any units. For example, the 15 minute timepoint would have a column header "15" and the 30 minute timepoint would have the column header "30". GRNmap supports replicate data for each of the timepoints. Replicate data for the same timepoint should be in columns immediately next to each other and have the same column headers. For example, three replicates of the 15 minute timepoint would have "15", "15", "15" as the column headers.
*# If data are provided for multiple strains, each strain should have data for the same timepoints, although the number of replicates can vary.
* Include the data for the 15, 30, and 60 minute timepoints, but not the 90 or 120 minute timepoints.
* The data you will be using is contained in the [https://github.com/kdahlquist/DahlquistLab/raw/master/data/Spring2019/Expression-and-Degradation-rate-database_2019.accdb Expression-and-Degradation-rate-database_2019.accdb] file that you used to obtain the production and degradation rates.
* It is tedious to copy and paste all of these data by hand, so we will execute a query in Microsoft Access to do it for you. Follow the steps listed for the "production_rates" sheet for each strains expression data. After you import the data into Excel, you will need to change the column headers to "15", "15", etc., as described above.
* Missing values in the expression data sheets are OK; you don't need to put any values there like you did for the production_rates or degradation_rates sheets.

==== network sheet ====

* The network you derived from the YEASTRACT database for the [[Week 9]] assignment can be copied and pasted into this sheet directly. You may need to edit the contents of cell A1, but the rest should be good to go (especially since you previewed it in GRNsight). The description below just explains what is already in this worksheet.
** This sheet contains an adjacency matrix representation of the gene regulatory network.
** The columns correspond to the transcription factors and the rows correspond to the target genes controlled by those transcription factors.
** A “1” means there is an edge connecting them and a “0” means that there is no edge connecting them.
** The upper-left cell (A1) should contain the text “cols regulators/rows targets”. This text is there as a reminder of the direction of the regulatory relationships specified by the adjacency matrix.
** The rest of row 1 should contain the names of the transcription factors that are controlling the other genes in the network, one transcription factor name per column.
** The rest of column A should contain the names of the target genes that are being controlled by the transcription factors heading each of the columns in the matrix, one target gene name per row.
** The transcription factor names should correspond to the "id" in the other sheets in the workbook. They should be capitalized the same way and occur in the same order along the top and side of the matrix. The matrix needs to be symmetric, i.e., the same transcription factors should appear along the top and left side of the matrix. The genes should be listed in the same order in all the sheets in the Excel workbook.
** Each cell in the matrix should then contain a zero (0) if there is no regulatory relationship between those two transcription factors, or a one (1) if there is a regulatory relationship between them. Again, the columns correspond to the transcription factors and the rows correspond to the target genes controlled by those transcription factors.

==== network_weights sheet ====

* These are the initial guesses for the estimation of the weight parameters, ''w''.
* Since these weights are initial guesses which will be optimized by GRNmap, the content of this sheet can be identical to the "network" sheet.

==== optimization_parameters sheet ====

* The optimization_parameters sheet should have two columns (from left to right) entitled, "optimization_parameter" and "value".
* You should copy this worksheet from the [https://github.com/kdahlquist/DahlquistLab/raw/master/data/Spring2019/15-genes_28-edges_sample_Sigmoid_estimation_2019.xlsx sample workbook] provided. The only row that you need to modify is row 15, "Strain". Include just the strain designations for which you have a corresponding STRAIN_log2_expression sheet. If you don't have the dgln3, dhap4, or dcin5 expression sheets, then you will delete those from this row. If you do so, make sure that you don't leave any gaps between cells.
* What follows below is an explanation of what the optimization_parameters mean.
** alpha: Penalty term weighting (from the L-curve analysis)
** kk_max: Number of times to re-run the optimization loop. In some cases re-starting the optimization loop can improve performance of the estimation.
** MaxIter: Number of times MATLAB iterates through the optimization scheme. If this is set too low, MATLAB will stop before the parameters are optimized.
** TolFun: How different two least squares evaluations should be before the program determines that it is not making any improvement
** MaxFunEval: maximum number of times the program will evaluate the least squares cost
** TolX: How close successive least squares cost evaluations should be before the program determines that it is not making any improvement.
** production_function: = Sigmoid (case-insensitive) if sigmoidal model, =MM (case-insensitive) if Michaelis-Menten model
** L_curve: =0 if an L-curve analysis should NOT be run or =1 if an L-curve analysis SHOULD be run. The L-curve analysis will automatically run sequential rounds of estimation for an array of fixed alpha values (0.8, 0.5, 0.2, 0.1,0.08, 0.05,0.02,0.01, 0.008, 0.005, 0.002, 0.001, 0.0008, 0.0005, 0.0002, and 0.0001). GRNmap makes a copy of the user's selected input workbook and changes alpha to the first alpha in the list. The estimation runs and the resulting parameter values are used as the initial guesses for the next round of estimation with the next alpha value. This process repeats until all alpha values have been run. New input and output workbooks are generated for each alpha value, although currently, the graphs are only saved for the last run.
** estimate_params =1 if want to estimate parameters and =0 if the user wants to do just one forward run
** make_graphs =1 to output graphs; =0 to not output graphs
** fix_P =1 if the user does not want to estimate the production rate, ''P'', parameter, just use the initial guess and never change; =0 to estimate
** fix_b =1 if the user does not want to estimate the ''b'' parameter, just use the initial guess and never change; =0 to estimate
** expression_timepoints: A row containing a list of the time points when the data was collected experimentally. Should correspond to the timepoint column headers in the STRAIN_log2_expression sheets.
** Strain: A row containing a list of all of the strains for which there is expression data in the workbook. Should correspond to the "STRAIN" portion of the names of the STRAIN_log2_expression sheets for each strain. Note that GRNmap will run the model for the wild type network (all genes present in the network) and for networks where the gene deleted from the designated STRAIN has been deleted from the network.
** simulation_timepoints: A row containing a list of the time points at which to evaluate the differential equations to generate the simulated data. This does not need to correspond to the actual measurement times, but should be in the same units (e.g. minutes).

==== threshold_b sheet ====

* These are the initial guesses for the estimation of the ''threshold_b'' parameters.
* There should be two columns.
** The left-most column should contain the header "id" and list the standard names for the genes in the model in the same order as in the other sheets.
** The second column should have the header "threshold_b" and should contain the initial guesses, we are going to use all 0.

== Conclusion ==
The first stage of our group's project was completed via referencing [[Week 8]] and using Microsoft Excel to complete the tasks. The excel file will be located in the [[FunGals]] page for viewing and download. We were able to successfully complete the ANOVA analysis and correct found mistakes. We were able to complete a sanity check with results showing. For the sanity check the unadjusted p- values showed 37.2% of the genes had a p<0.05, 18.16% had a p<0.01, 5.04% have p<0.001, and 0.87% have p<0.0001. The sanity check for the Bonferroni-corrected p-value 0.11% of the genes have p<0.05. For the sanity check on the Benjamini and Hochberg-corrected p-value 16.36% go the genes had a p <0.05. Finally we were able to prepare the Data to run STEM in the next step of working on this project.

Extra Notes (will delete later): PDR3 did not show up in Access while running Query's

==Data and files==
[[media:Thiuram_yeast_experiment.xlsx|ANOVA & STEM workbook (.xlsx)]]

[[media:Genelist_combined_profiles_FunGals.xlsx| YEASTRACT Rank by TF - Green and Red Profiles (.xlsx)]]

[[media:FunGals_Green_Profile_RegulationMatrix_Documented_2019123_2322_1272399515.xlsx| Green Regulation Matrix/network (.xlsx)]]

[[media:FunGals Red Profile RegulationMatrix Documented 2019123 2318 1127808084.xlsx| Input Workbook Red network (.xlsx)]]

[[media:FunGals Green Profile RegulationMatrix Input Workbook.xlsx | Input Workbook for Green profile (.xlsx)]]

[[media:GRNmap FunGals Green profile.zip | Output Workbook & GRNModel from MatLab for Green profile (.zip)]]

[[Media:GRN_Model_Red_from_Matlab.zip | Output Workbook & GRNModel from MatLab for Red profile (.zip)]]

[[media:Data_for_FunGals.pptx|FunGals Data on Powerpoint]]

== Acknowledgements ==
This section is in acknowledgement to partner Kaitlyn Nguyen (User:knguye66), Michael Armas (User:Marmas), as well as, Iliana Crespin (User:Icrespin), and Emma Young (User:eyoung20). We would also like to acknowledge Dr. Dahlquist (User:KDahlquist) for introducing and teaching the topic and direction of this assignment.
Also to acknowledge that this is a shared electronic notebook between [[user:knguye66|Kaitlyn Nguyen]] and [[user:eyoung20|Emma Young]].

"Except for what is noted above, this individual journal entry was completed by me and not copied from another source."
[[User:Knguye66|Knguye66]] ([[User talk:Knguye66|talk]]) 18:49, 20 November 2019 (PST)

"Except for what is noted above, this individual journal entry was completed by me and not copied from another source."
[[User:Eyoung20|Eyoung20]] ([[User talk:Eyoung20|talk]]) 16:40, 25 November 2019 (PST)

== References ==
*Dahlquist, K. (2019, November 19). Data Analysis. In Wikipedia, Biological Databases. Retrieved 6:25, November 20, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Data_Analysis
*Dahlquist, K. (2019, November 20). Final Project Deliverables. In Wikipedia, Biological Databases. Retrieved 2:59, December 5, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Final_Project_Deliverables
*Dahlquist, K. (2019, October 29). Week 9. In Wikipedia, Biological Databases. Retrieved 2:57, December 2, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_9
*Dahlquist, K. (2019, November 7). Week 10. In Wikipedia, Biological Databases. Retrieved 2:59, December 5, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_10

{{Template:knguye66}}
{{Template:Eyoung20}}

[[Category: Group Projects]]
[[Category: FunGals]]

File:Data for FunGals.pptx

2019-12-09T00:21:49Z

Eyoung20: Eyoung20 uploaded a new version of File:Data for FunGals.pptx

FunGals

2019-12-08T23:57:58Z

Eyoung20: /* Data and Files */ added data

==Effects of the Pesticide Thiuram: Genome-wide Screening of Indicator Genes by Yeast DNA Microarray==

'''Team Information'''
Project Manager: [[User:Marmas|Michael Armas]]
Quality Assurance: [[User:Icrespin| Iliana Crespin]]
Data Analysis: [[User:eyoung20|Emma Young]], [[User:knguye66|Kaitlyn Nguyen]]
Coder: [[User:Marmas|Michael Armas]]

== Methods and Results ==
=== Week 11 ===
====Setting up the Project====
*Selected [[User:Marmas|Michael Armas]] as team's Project Manager.
*Added the name of of selected project manager to the Project Manager guild page and Overview pages.
*Named the team FunGals and created [[FunGals]] home page on the wiki.
*The name of the team home page was the selected team name.

*Created a link to team's page on the course Main page.
*Created a template for FunGal with useful information and links that you will invoke on all pages that you will create for the project.
*Created a category using team name and included it the FunGal template so that it is used on all pages created for the project. Also included the category "Group Projects" in the template. 
*Each person needs to write a short executive summary of that person's progress on the project for the week, with links to the relevant individual journal pages (which will have more detailed information).
*Each team member should reflect on the team's progress '''(Note that you will be directed to add specific information to your team's pages in the individual portion of the assignment for this and future weeks)''':
**What worked? What didn't work? What will I do next to fix what didn't work?
*** Michael Armas: Working with this team is a great pleasure! Everyone on the team is pulling their weight and providing information that contributes to the group work. Next time, as the project manager, I want to be more organized and come up with interim deadlines to make sure we are not cramming right before the actual deadline. In fact, I would have something small due every day than procrastinate until the end. However, with this team, we are making it work and doing exceptional work!
*** Kaitlyn Nguyen: What worked this week is starting the presentation formatting early, thus allocating the rest of the time to "filling in the missing parts" of the powerpoint. What also worked this week was team communication via group-chats. What didn't work this week was starting the individual assignment later, leaving less time to practicing and fully understanding the material from the journal article for the presentation in class. To fix what didn't work, I will start my individual assignment much earlier, to leave the rest of the time to focus on collaborative teamwork and group assignments. [[User:Knguye66|Knguye66]] ([[User talk:Knguye66|talk]]) 23:37, 13 November 2019 (PST)
*** Emma Young: The team seems to work really well together. I think in upcoming weeks we will really thrive in working together. A foundation for communication has already been established and we have already started reviewing each others work and providing constructive criticism and help where it is needed. The main issue this week was a matter of timing. With the amount of work we had to do in the time frame given we were not able to meet up due to busy and conflicting prior engagements. This meant that we did not reach our full potential of working together effectively as a group this week. Next week, we hopefully will be able to set better deadlines and be able to have the ability to plan out time to work on this in a less rushed manner.
*** Iliana Crespin: Overall, this group has been great. Everyone has worked together and understands what must be done. In addition, each member is understanding of the special circumstances before the deadline of this assignment. The only thing that didn't work out was how scattered each of us were because of all the assignments that had to be completed from other classes. In addition, mandated plans (school- or work-related) popped up which made it difficult to contribute equally. For the future, I will make sure to manage my time better, because I have work and school. Therefore, by doing a fair share of certain things before classes/work, I will be able to contribute more.[[User:Icrespin|Icrespin]] ([[User talk:Icrespin|talk]]) 23:46, 13 November 2019 (PST)

====Annotated Bibliography====

#Braconi, D., Bernardini, G., & Santucci, A. (2016). Saccharomyces cerevisiae as a model in ecotoxicological studies: A post-genomics perspective. Journal of Proteomics, 137, 19-34. DOI: 10.1016/j.jprot.2015.09.001
#Hinkle, K. L., & Olsen, D. (2018). Exposure to the lampricide TFM elicits an environmental stress response in yeast. FEMS yeast research, 19(1), foy121. doi: 10.1093/femsyr/foy121
#Iwahashi, Y., Hosoda, H., Park, J. H., Lee, J. H., Suzuki, Y., Kitagawa, E., ... & Iwahashi, H. (2006). Mechanisms of patulin toxicity under conditions that inhibit yeast growth. Journal of agricultural and food chemistry, 54(5), 1936-1942. doi: 10.1021/jf052264g.
#Iwahashi, H., Ishidou, E., Kitagawa, E., & Momose, Y. (2007). Combined Cadmium and Thiuram Show Synergistic Toxicity and Induce Mitochondrial Petite Mutants. Environmental Science & Technology, 41(22), 7941–7946. doi: 10.1021/es071313y
#Iwahashi, H., Ishidou, E., Kitagawa, E., & Momose, Y. (2007). Combined Cadmium and Thiuram Show Synergistic Toxicity and Induce Mitochondrial Petite Mutants. Environmental Science & Technology, 41(22), 7941–7946. doi: 10.1021/es071313y
#Kitagawa, E., Momose, Y., & Iwahashi, H. (2003). Correlation of the Structures of Agricultural Fungicides to Gene Expression in Saccharomyces cerevisiaeupon Exposure to Toxic Doses. Environmental Science & Technology, 37(12), 2788–2793. doi: 10.1021/es026156b
#Lu Yu, Na Guo, Rizeng Meng, Bin Liu, Xudong Tang, Jing Jin, Yumei Cui, Xuming Deng. Allicin-induced global gene expression profile of Saccharomyces cerevisiae. Applied Microbiology and Biotechnology 2010, 88 (1) , 219-229. DOI: 10.1007/s00253-010-2709-x.
#Pierron, A., Mimoun, S., Murate, L. S., Loiseau, N., Lippi, Y., Bracarense, A. P. F., ... & Oswald, I. P. (2016). Intestinal toxicity of the masked mycotoxin deoxynivalenol-3-β-D-glucoside. Archives of toxicology, 90(8), 2037-2046. doi: 10.1007/s00204-015-1592-8

===Week 12/13===
====Milestone 3: Getting the data ready for analysis ====

# As a group Downloaded and examined the microarray dataset, compared it to the samples and experiment described in the journal club article.
#* [https://sgd-prod-upload.s3.amazonaws.com/S000204415/Kitagawa_2002_PMID_12269742.zip Kitagawa et al. (2002)]
#* downloaded and unzipped
#* file was downloaded but was PLC format
#* Mike figured out how to drag and drop it one to excel to open the file
# Along with the QA's, make a "sample-data relationship table" that lists all of the samples (microarray chips), noting the treatment, time point, replicate number, and other information pertaining to each sample.
#* This "Metadata" sheet was created in Excel and is present on the excel work under the week 12/13 data and files.
#* In collaboration with other groups, the Metadata sheet was filled out so that nomenclature is consistent.
# The class collectively came up with consistent column headers that summarized the information
#* As decided by all groups, the format for column headers was yeastStrain_treatmentOrMutation_concentrationAndUnits_dataType_timeAndUnits-replicateNumber.
# Organized the data in a worksheet in an Excel workbook so that:
#* MasterIndex was in the first column, ID was in the second column
#* Data columns were to the right, in increasing chronological order, using the column header pattern that was created.
#* Replicates were grouped together
# Data analysts (DA) [[User:Knguye66|Kaitlyn]] set-up the ANOVA worksheet via referencing [[Week 8]]
#* The steps for the ANOVA: Part I, Benjamini, Bonferroni, and p-value correction, as well as, a quick sanity check were followed.
# DAs [[User:Eyoung20|Eyoung20]] begin setting up for STEM analysis to complete the first stage of milestones and deliverables
# All the calculations were checked by the QA [[User:Icrespin|Iliana]].

*Each team member should reflect on the team's progress '''(Note that you will be directed to add specific information to your team's pages in the individual portion of the assignment for this and future weeks)''':
**What worked? What didn't work? What will I do next to fix what didn't work?
*** Michael Armas: What worked this week was the communication between my guild and me, and my group and me. Everyone was able to help each other out and make sure the project is coming along smoothly. What didn't work this week for me was understanding exactly what I had to do to set up spreadsheets for data analysis. However, thanks to Mihir, I was able to understand how to create a metadata sheet and reach the milestone for this week. Before I leave class from now on, I will reassure myself that I understand the information by asking Dr. Dahlquist the best way to reach the week's milestone.
*** Kaitlyn Nguyen: This week, the pattern and organization to which each group member worked flowed smoothly as every member of the group knew exactly what each person was accomplishing. Having extra time in class and staying after class helped us jump-start on our tasks together and find out what assignments work best for each team member. What didn't work this week was that each milestone and part of this project works chronologically, whereby other members have to wait for the first person to finish his/her task before beginning theirs. As the days and weeks move on, it will be good to continue this method, but switch off on who starts the tasks first for the assignment. [[User:Knguye66|Knguye66]] ([[User talk:Knguye66|talk]]) 18:44, 20 November 2019 (PST)
*** Emma Young: For this week I worked closely with [[User:Knguye66|Kaitlyn]] to start the data analysis for the rest of the project. We worked with the whole team to analyze the data from the journal article and format for the rest of the data analysis. [[User:Marmas|Mike]] worked with the other programers to formulate the metadata sheet and the format for the ANOVA sheet. More details on this can be found in his personal electronic notebook [[Marmas Week 12/13]] Then [[User:Knguye66|Kaitlyn]] worked on the ANOVA analysis of the data. [[User:Icrespin|Iliana]] double checked the the results of the analysis the details can be found in [[Icrespin Journal Week 12/13]]. After the checks were completed and any errors were found, I ran a sanity check on the ANOVA data and then formatted the STEM worksheet for future analysis. The details about what [[User:Knguye66|Kaitlyn] and [[user:eyoung20|I]] did can be found in our shared journal [[Knguye66 Eyoung20 Week 12/13]]. As can be seen in this summary this week your group was really good at communicating and working together to work on this project. We worked really well together this week. One thing that did not work as well was trying to work on the excel spreadsheet online when others were working on it, there was a large lag on the sheet. I think for the next steps I might do the work on a saved downloaded version of the excel sheet and then copy my work to the shared excel. [[User:Eyoung20|Eyoung20]] ([[User talk:Eyoung20|talk]]) 16:36, 25 November 2019 (PST)
*** Iliana Crespin: For this week, it was a better organization on completing the assignment. There is more of a pattern of what should be done throughout each day to prepare for the deadline. What didn't work out for me is still trying to see how a QA can incorporate any work, since my teammates are great. For the next few weeks, I will try to continue double checking the work and contribute more. This contribution can include working with the data analysts and seeing how I can help out a bit more. In addition, texting teammates a bit more to remind any little things that must be due before the deadline. [[User:Icrespin|Icrespin]] ([[User talk:Icrespin|talk]]) 14:48, 21 November 2019 (PST)

===Week 15===
'''Milestone 4'''
#Begin wrapping up with running analysis on the created database using GRNmap
#*Data Analysts: Create GRNmap using individual database from MS Access.
#*Coder: Communicate with the QA to ensure the database is correct and if any changes need to be made
#*QA: Review the database and communicate to the Coder if any changes need to be made.
#Each team member should reflect on the team's progress
#*What worked? What didn't work? What will I do next to fix what didn't work?
#**Michael Armas
#**Kaitlyn Nguyen
#**Emma Young
#**Iliana Crespin

== Data and Files ==
=== Week 11 ===
* Presentation file: [[Media: Marmas_FunGals_Presentation.pptx|Group Presentation]]
=== Week 12/13 ===
*[[media: Thiuram_yeast_experiment.xlsx|Excel Workbook]]
*[[meda: FunGals_genelist.zip|Genelist from STEM]]
*[[media: FunGals_GOlist.zip|GOlist from STEM]]
*[[media:Data_for_FunGals.pptx|FunGals Data on Powerpoint]]
===Week 15===
*[[Media:Marmas_finalProject_Expression-and-Degradation-rate-database_120319.zip|FunGals Expression Database]]
*[[Media:GRN_Model_Red_from_Matlab.zip|GRN Model Red Profile]]
*[[media:Data_for_FunGals.pptx|FunGals Data on Powerpoint]]

== Milestones ==
{| class="wikitable"
|-
! Tasks to be completed
! Due Date
|-
| Write down Methods and Results for Week 12/13
*Coder/designer: find out the column names and edit them, sample to data relationship table
*Data Analysis: ANOVA analysis and set-up, STEM set-up (will run STEM on Thursday)
*QA: make sure all data in original file actually made it into the database, let coder and analysts know if there are issues
| Thursday, 12:00am, 11/19/19
|-
| Team Journal Assignment, continue writing down Methods and Results for Week 12/13
*Data Analysis: finish editing missing information for ANOVA and (STRAIN) worksheet, run STEM
*Coder and QA: add standard names to genes, make sure all data in original file actually made it into the database, let coder and analysts know if there are issues
| Tuesday, 12:00am, 11/26/19
|-
| Creating the database and running queries, working with all guilds to assure project is in line to be completed
*Data Analysts: Run YEASTRACT and modify GO Terms.
*Coder: Create individual database for the team.
*QA:Design Expression Tables used to create the final individual database.
| Thursday, 12:00am, 11/28/19
|-
|Begin wrapping up with running analysis on the created database using GRNmap
*Data Analysts: Create GRNmap using individual database from MS Access.
*Coder: Communicate with the QA to ensure the database is correct and if any changes need to be made
*QA: Review the database and communicate to the Coder if any changes need to be made.
| Tuesday, 12:00am, 12/03/19
|-
| Wrap up data analysis and individual milestones and begin to gather deliverables for turn-in
*Data Analysts: Provide feedback on the database and its ease-of-use. Work with QA to gather deliverables.
*Coder: Work with the entire guild to properly combine all databases for the entire class to use.
*QA: Work with the coder to finalize the [https://www.quackit.com/microsoft_access/microsoft_access_2016/howto/how_to_create_a_database_diagram_in_access_2016.cfm database schema diagram]
| Thursday, 12:00am, 12/05/19
|-
| Final Presentation
| Tuesday, 12:00am, 12/10/19
|-
| Report submitted
| Friday, 4:00 pm, 12/13/19
|-
|}

==Acknowledgements==
This section is in acknowledgement to partner [[User:Marmas|Michael Armas]], [[User:Icrespin|Iliana Crespin]], [[User:eyoung20|Emma Young]], and [[User:knguye66|Kaitlyn Nguyen]]. I would also like to acknowledge [[User:Kdahlquist|Dr. Dahlquist]] for introducing and teaching the topic and direction of this assignment.

"Except for what is noted above, this individual journal entry was completed by me and not copied from another source."
 
[[User:Marmas|Marmas]] ([[User talk:Marmas|talk]]) 23:40, 13 November 2019 (PST)
[[User:Icrespin|Icrespin]] ([[User talk:Icrespin|talk]]) 23:46, 13 November 2019 (PST)

"Except for what is noted above, this individual journal entry was completed by me and not copied from another source."
[[User:Eyoung20|Eyoung20]] ([[User talk:Eyoung20|talk]]) 23:57, 13 November 2019 (PST) [[User:Knguye66|Knguye66]] ([[User talk:Knguye66|talk]]) 18:45, 20 November 2019 (PST)

{{FunGals}}

==References==
===Week 11===
*Kitagawa, E., Takahashi, J., Momose, Y., & Iwahashi, H. (2002). Effects of the pesticide thiuram: genome-wide screening of indicator genes by yeast DNA microarray. Environmental science & technology, 36(18), 3908-3915. DOI: 10.1021/es015705v
*Dahlquist, K. (2019, November 7). Week 11. In Wikipedia, Biological Databases. https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_1https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_11
===Week 12/13===
*Dahlquist, K. (2019, November 19). Data Analysis. In Wikipedia, Biological Databases. Retrieved 6:25, November 20, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Data_Analysis
*Dahlquist, K. (2019, November 20). Final Project Deliverables. In Wikipedia, Biological Databases. Retrieved 6:25, November 20, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_12/13https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Final_Project_Deliverables
*Dahlquist, K. (2019, November 19). Week 12/13. In Wikipedia, Biological Databases. Retrieved 6:25, November 20, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_12/13
*Dahlquist, K. (2019, October 17). Week 8. In Wikipedia, Biological Databases. Retrieved 6:30, October 21, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_8
===Week 15===

Knguye66 Eyoung20 Week 15

2019-12-08T23:57:26Z

Eyoung20: /* Data and files */

'''Combined Individual Journals for Kaitlyn Nguyen and Emma Young (Data Analysts).'''

== Purpose ==
The purpose of this assignment is to record our progress towards the [[FunGals]] group [[Final_Project_Deliverables | deliverables]] as the [[Data_Analysis | Data Analysts]] for this week and the future weeks to come. The purpose of week 12 specifically was to download and adapt the data to the formatting we need for analysis. Then to begin the analysis with ANOVA and preparations for STEM.

== Methods and Results: Progress ==
===== - Progress 11/26/19 - =====

==== Analyzing and Interpreting STEM Results ====
#Why did you select this profile? In other words, why was it interesting to you?
#* We collectively combined the 4 red profiles together because they had similar trends and to increase the number of genes for analysis (called "Red".) Similarly this was done to the 3 green profiles as well (upward trends), with the addition of the blue profile #29 added. We will call this group "Green".
#How many genes belong to this profile?
#* Red (composed of Profile #9,26,34,11): 289
#* Green (Profile #40,42,18,29): 214
#How many genes were expected to belong to this profile?
#* Red (composed of Profile #9,26,34,11): 51.5
#* Green (Profile #40,42,18,29): 48.5
#What is the p value for the enrichment of genes in this profile?
#* Due to combining the profiles, we do not have p-values for the enrichment of genes in the 2 different groups.

- '''Viewing and Saving STEM Results''' -
# A new window will open called "All STEM Profiles (1)". Each box corresponds to a model expression profile. Colored profiles have a statistically significant number of genes assigned; they are arranged in order from most to least significant p value. Profiles with the same color belong to the same cluster of profiles. The number in each box is simply an ID number for the profile.
#* Click on the button that says "Interface Options...". At the bottom of the Interface Options window that appears below where it says "X-axis scale should be:", click on the radio button that says "Based on real time". Then close the Interface Options window.
#*Take a screenshot of this window (on a PC, simultaneously press the <code>Alt</code> and <code>PrintScreen</code> buttons to save the view in the active window to the clipboard) and paste it into a PowerPoint presentation to save your figures.
# Click on each of the SIGNIFICANT profiles (the colored ones) to open a window showing a more detailed plot containing all of the genes in that profile.
#* Take a screenshot of each of the individual profile windows and save the images in your PowerPoint presentation.
#* At the bottom of each profile window, there are two yellow buttons "Profile Gene Table" and "Profile GO Table". For each of the profiles, click on the "Profile Gene Table" button to see the list of genes belonging to the profile. In the window that appears, click on the "Save Table" button and save the file to your desktop. Make your filename descriptive of the contents, e.g. "wt_profile#_genelist.txt", where you replace the number symbol with the actual profile number.
#** Upload these files to the wiki and link to them on your individual journal page. (Note that it will be easier to [[Week_4#Compressing_and_Decompressing_Files_with_7-Zip | zip all the files together]] and upload them as one file).
#* For each of the significant profiles, click on the "Profile GO Table" to see the list of Gene Ontology terms belonging to the profile. In the window that appears, click on the "Save Table" button and save the file to your desktop. Make your filename descriptive of the contents, e.g. "wt_profile#_GOlist.txt", where you use "wt", "dGLN3", etc. to indicate the dataset and where you replace the number symbol with the actual profile number. At this point you have saved all of the primary data from the STEM software and it's time to interpret the results!
#** Upload these files to the wiki and link to them on your individual journal page. (Note that it will be easier to [[Week_4#Compressing_and_Decompressing_Files_with_7-Zip | zip all the files together]] and upload them as one file).

==== Clustering the data with STEM, as did on [[Week 9]]. ====
# Note that we will make some adjustments to the GO term analysis because stem was not providing GO term names. We are going to use the GO enrichment tool at GeneOntology.org instead.
#* Go to [http://geneontology.org/ http://geneontology.org/].
#* For the cluster you want to analyze, open the gene list and copy the list of genes.
#* Paste the list of genes into the "Go Enrichment Analysis" box on the right hand side of the GeneOntology.org page.
#* Select "Saccharomyces cerevisiae" from the species drop-down menu.
# Click the "Launch" buton.
#* Near the bottom of the results page, click on the button to Export "Table".
#* This will prompt you to save a .txt file that can be opened in Excel to view your results.
# Use YEASTRACT to generate a candidate gene regulatory network as you did on [[Week 9]].

==== Using YEASTRACT to Infer which Transcription Factors Regulate a Cluster of Genes ====

In the previous analysis using STEM, we found a number of gene expression profiles (aka clusters) which grouped genes based on similarity of gene expression changes over time. The implication is that these genes share the same expression pattern because they are regulated by the same (or the same set) of transcription factors. We will explore this using the YEASTRACT database.

# Open the gene list in Excel for the one of the significant profiles from your stem analysis. Choose a cluster with a clear cold shock/recovery up/down or down/up pattern. You should also choose one of the largest clusters.
#* Copy the list of gene IDs onto your clipboard.
# Launch a web browser and go to the [http://www.yeastract.com/ YEASTRACT database].
#* On the left panel of the window, click on the link to [http://www.yeastract.com/formrankbytf.php ''Rank by TF''].
#* Paste your list of genes from your cluster into the box labeled ''ORFs/Genes''.
#* Check the box for ''Check for all TFs''.
#* Accept the defaults for the Regulations Filter (Documented, DNA binding plus expression evidence)
#* Do '''''not''''' apply a filter for "Filter Documented Regulations by environmental condition".
#* Rank genes by TF using: The % of genes in the list and in YEASTRACT regulated by each TF.
#* Click the ''Search'' button.
# Answer the following questions:
#* In the results window that appears, the p values colored green are considered "significant", the ones colored yellow are considered "borderline significant" and the ones colored pink are considered "not significant". '''''How many transcription factors are green or "significant"?'''''
#**Green: 31
#**Red: 30
#* '''''Copy the table of results from the web page and paste it into a new Excel workbook to preserve the results.'''''
#** '''''Upload the Excel file to the wiki and link to it in your electronic lab notebook.'''''
#** '''''Are CIN5, GLN3, and/or HAP4 on the list? If so, what is their "% in user set", "% in YEASTRACT", and "p value".'''''
#**Green
#*** CIN5
#**** 0.2991 % in user set
#****0.0294
#**** p-value: 0.5492
#***GLN3
#**** 0.3645% in user set
#****0.0324% in YEASTRACT
#**** p-value: 0.18046
#***HAP4
#**** 0.2383% in user set
#****0.0467% in YEASTRACT
#**** P-value:0.00031337
#**Red
#*** CIN5
#**** 0.2111%% in user set
#**** 0.028% in YEASTRACT
#**** p-value: 0.9998
#***GLN3
#**** 0.4152% in user set
#****0.0498% in YEASTRACT
#**** p-value:0.00207752
#***HAP4
#****0.2353% in user set
#****0.0623% in YEASTRACT
#**** P-value:0.000006235
# For the mathematical model that we will build, we need to define a ''gene regulatory network'' of transcription factors that regulate other transcription factors. We can use YEASTRACT to assist us with creating the network. We want to generate a network with approximately 15-20 transcription factors in it.
#* You need to select from this list of "significant" transcription factors, which ones you will use to run the model. You will use these transcription factors and add GLN3, HAP4, and CIN5 if they are not in your list. Explain in your electronic notebook how you decided on which transcription factors to include. Record the list and your justification in your electronic lab notebook. Each group member will select a different network (they can have some overlapping transcription factors, but some should also be different).
#* Go back to the YEASTRACT database and follow the link to ''[http://www.yeastract.com/formregmatrix.php Generate Regulation Matrix]''.
#* Copy and paste the list of transcription factors you identified (plus HAP4, GLN3, and CIN5) into both the "Transcription factors" field and the "Target ORF/Genes" field.
#* We are going to use the "Regulations Filter" options of "Documented", "'''Only''' DNA binding evidence"
#** Click the "Generate" button.
#** In the results window that appears, click on the link to the "Regulation matrix (Semicolon Separated Values (CSV) file)" that appears and save it to your Desktop. Rename this file with a meaningful name so that you can distinguish it from the other files you will generate.

==== Visualizing Your Gene Regulatory Networks with GRNsight ====

We will analyze the regulatory matrix files you generated above in Microsoft Excel and visualize them using GRNsight to determine which one will be appropriate to pursue further in the modeling.
# First we need to properly format the output files from YEASTRACT.
#* Open the file in Excel. It will not open properly in Excel because a semicolon was used as the column delimiter instead of a comma. To fix this, Select the entire Column A. Then go to the "Data" tab and select "Text to columns". In the Wizard that appears, select "Delimited" and click "Next". In the next window, select "Semicolon", and click "Next". In the next window, leave the data format at "General", and click "Finish". This should now look like a table with the names of the transcription factors across the top and down the first column and all of the zeros and ones distributed throughout the rows and columns. This is called an "adjacency matrix." If there is a "1" in the cell, that means there is a connection between the trancription factor in that row with that column.
#* Save this file in Microsoft Excel workbook format (.xlsx).

#* For this adjacency matrix to be usable in GRNmap (the modeling software) and GRNsight (the visualization software), we need to transpose the matrix. Insert a new worksheet into your Excel file and name it "network". Go back to the previous sheet and select the entire matrix and copy it. Go to you new worksheet and click on the A1 cell in the upper left. Select "Paste special" from the "Home" tab. In the window that appears, check the box for "Transpose". This will paste your data with the columns transposed to rows and vice versa. This is necessary because we want the transcription factors that are the "regulatORS" across the top and the "regulatEES" along the side.
#* The labels for the genes in the columns and rows need to match. Thus, delete the "p" from each of the gene names in the columns. Adjust the case of the labels to make them all upper case.
#* In cell A1, copy and paste the text "rows genes affected/cols genes controlling".
#* Finally, for ease of working with the adjacency matrix in Excel, we want to alphabatize the gene labels both across the top and side.
#** Select the area of the entire adjacency matrix.
#** Click the Data tab and click the custom sort button.
#** Sort Column A alphabetically, being sure to exclude the header row.
#** Now sort row 1 from left to right, excluding cell A1. In the Custom Sort window, click on the options button and select sort left to right, excluding column 1.
#* Name the worksheet containing your organized adjacency matrix "network" and Save.
# Now we will visualize what these gene regulatory networks look like with the GRNsight software.
#* Go to the [http://dondi.github.io/GRNsight/ GRNsight] home page.
#* Select the menu item File > Open and select the regulation matrix .xlsx file that has the "network" worksheet in it that you formatted above. If the file has been formatted properly, GRNsight should automatically create a graph of your network. You can click the "Grid Layout" button to arrange the nodes in a grid, or you can click and drag the nodes (genes) around until you get a layout that you like and take a screenshot of the results. Paste it into your PowerPoint presentation.
#** If you have nodes (genes) floating around in the display that are not connected to any other nodes, we need to delete them from the network for the modeling to work properly. Go back to the Excel workbook and network sheet and delete both the row and column with the floating gene's name. Then re-upload the edited file to GRNsight to visualize it. Use this final version in your PowerPoint and subsequent modeling.
#** The deleted genes (floating) on GRNsight were:
#*** Green: MSN and BAS1

===== - Progress 12/03/19 - =====

==== production_rates sheet ====

* This sheet contains initial guesses for the production rate parameters, ''P'', for all genes in the network.
* Assuming that the system is in steady state with the relative expression of all genes equal to 1, (P/2) - lambda = 0, where lambda is the degradation rate, is a reasonable initial guess.
* The sheet should contain two columns (from left to right) entitled, "id", "production_rate".
** The id is an identifier that the user will use to identify a particular gene. In our case, we are using the "StandardName", for example, GLN3.
** The "production_rate" column should then contain the initial guesses for the ''P'' parameter as described above, rounded to four decimal places.
*** The production rates are provided in a Microsoft Access database, which you can [https://github.com/kdahlquist/DahlquistLab/raw/master/data/Spring2019/Expression-and-Degradation-rate-database_2019.accdb download from here.]
*** You will perform a query to get the list of production rates for each gene as a group.
*** To perform the query, you will need to follow these steps.
***# Import a your list of genes to a new table in the database. Click on the "External Data" tab and select the Excel icon with the "up" arrow on it.
***# Click the "Browse" button and select your Excel file containing your network that you used to upload to GRNsight.
***# Make sure the button next to "Import the source data into a new table in the current database" and click "OK".
***# In the next window, select the "network" worksheet, if it hasn't already been automatically selected for you. Click "Next".
***# In the next window, make sure the "First Row Contains Column Headings" is checked. Click "Next".
***# In the next window, the left-most column will be highlighted. Change the "Field Name" to "id" if it doesn't say that already. Click "Next".
***# In the next window, select the button for "Choose my own primary key." and choose the "id" field from the drop down next to it. Click "Next".
***# In the next field, make sure it says "Import to Table: network". Click Finish.
***# In the next window you do not need to save the import steps, so just click "Close".
***# A table called "network" should appear in the list of tables at the left of the window.
***# Go to the "Create" tab. Click on the icon for "Query Design".
***# In the window that appears, click on the "network" table and click "Add". Click on the "production_rates" table and click "Add". Click "Close".
***# The two tables should appear in the main part of the window. We need to tell Access which fields in the two tables correspond to each other. Click on the word "id" in the network table and drag your mouse to the "standard_name" field in the "production_rates" table, and release. You will see a line appear between those two words.
***# Right-click on the line between those words and select "Join Properties" from the menu that appears. Select Option "2: Include ALL records from 'network' and only those records from 'production_rates' where the joined fields are equal." Click "OK".
***# Click on the "id" word in the "network" table and drag it to the bottom of the screen to the first column next to the word "Field" and release.
***# Click on the "production_rate" field in the "production_rates" table and drag it to the bottom of the screen to the second column next to the word "Field" and release.
***# Right-click anywhere in the gray area near the two tables. In the menu that appears, select "Query Type > Make Table Query...".
***# In the window that appears, name your table "production_rates_1" because you can't have two tables with the same name in the database. Make sure that "Current Database" is selected and Click "OK".
***# Go to the "Query Tools: Menus" tab. Click on the exclamation point icon. A window will appear that tells you how many rows you are pasting into a new table. Click "Yes".
***# Your new "production_rates_1" table will appear in the list at the left. Double-click on that table name to open it.
***# You can copy the data in this table and paste it back into your Excel workbook. Make sure that when you paste that you use "Paste Special > Paste values" so that the Access formatting doesn't get carried along. You can also choose to export this table to Excel going to the "External Data" tab and selecting the Excel icon with the arrow pointing to the right. Select the workbook you want to export the table to, making sure that "Preserve Access formatting" is '''not''' checked. Click "OK", click "Close".
* If there are missing values, substitute the value <code>0.1980</code> for the missing production rates.
* Note that the genes should be listed in the same order in all the sheets in the Excel workbook.

==== degradation_rates sheet ====

* This sheet contains degradation rates for all genes in the network, which are provided by the user.
* Currently, the Dahlquist Lab is using data based on published mRNA half-life data from [http://rnajournal.cshlp.org/content/20/10/1645.full Neymotin et al. (2006)].
** We converted the half-life data values to the degradation rates by taking the natural log of the half-life and dividing by 2.
* The sheet should contain two columns (from left to right) entitled "id", and "degradation_rate".
** The id is an identifier that the user will use to identify a particular gene.
** The "degradation_rate" column should then contain the absolute value of the degradation rate for the corresponding gene as described above, rounded to four decimal places.
*** To obtain these values, you will use the same file, [https://github.com/kdahlquist/DahlquistLab/raw/master/data/Spring2019/Expression-and-Degradation-rate-database_2019.accdb Microsoft Access database] that you used to obtain the production rates in the first worksheet. Again, you can copy and paste the values one-by-one or you can follow the instructions to execute a query, substituting the appropriate "degradation_rates" table in the query. Note that you don't need to re-import your "network" table, you just need to create and execute the query.
* Again note, the genes should be listed in the same order in all the sheets in the Excel workbook.
* If there are missing values, substitute the value <code>0.0990</code> for the missing degradation rates.

==== Expression Data Sheets for Individual Yeast Strains ====

* Expression data can be provided for either a single strain or multiple strains of yeast (for example, the wild type strain and a transcription factor deletion strain).
** Each strain will have its own sheet in the workbook.
** Each sheet should be given a unique name that follows the convention "STRAIN_log2_expression", where the word "STRAIN" is replaced by the strain designation, which will appear in the optimization_diagnostics sheet.
*** Everyone in the class will have at least one expression worksheet called "wt_log2_expression".
*** You should have included the transcription factors GLN3, HAP4, and CIN5 in your network. Thus, we will use the expression data from the dGLN3, dHAP4, dCIN5 deletion strains in our workbooks as well, naming the worksheets "dgln3_log2_expression", "dhap4_log2_expression", and "dcin5_expression".
**** If, for some reason, you don't have all three of those genes in your network, only include expression data for the wild type and the genes out of those three that you have in your network.
* The sheet should have the following columns in this order:
*# "id": list of all genes. The genes should be listed in the same order in all the sheets in the Excel workbook.
*# The next series of columns should contain the expression data for each gene at a given timepoint given as log2 ratios (log2 fold changes). The column header should be the time at which the data were collected, without any units. For example, the 15 minute timepoint would have a column header "15" and the 30 minute timepoint would have the column header "30". GRNmap supports replicate data for each of the timepoints. Replicate data for the same timepoint should be in columns immediately next to each other and have the same column headers. For example, three replicates of the 15 minute timepoint would have "15", "15", "15" as the column headers.
*# If data are provided for multiple strains, each strain should have data for the same timepoints, although the number of replicates can vary.
* Include the data for the 15, 30, and 60 minute timepoints, but not the 90 or 120 minute timepoints.
* The data you will be using is contained in the [https://github.com/kdahlquist/DahlquistLab/raw/master/data/Spring2019/Expression-and-Degradation-rate-database_2019.accdb Expression-and-Degradation-rate-database_2019.accdb] file that you used to obtain the production and degradation rates.
* It is tedious to copy and paste all of these data by hand, so we will execute a query in Microsoft Access to do it for you. Follow the steps listed for the "production_rates" sheet for each strains expression data. After you import the data into Excel, you will need to change the column headers to "15", "15", etc., as described above.
* Missing values in the expression data sheets are OK; you don't need to put any values there like you did for the production_rates or degradation_rates sheets.

==== network sheet ====

* The network you derived from the YEASTRACT database for the [[Week 9]] assignment can be copied and pasted into this sheet directly. You may need to edit the contents of cell A1, but the rest should be good to go (especially since you previewed it in GRNsight). The description below just explains what is already in this worksheet.
** This sheet contains an adjacency matrix representation of the gene regulatory network.
** The columns correspond to the transcription factors and the rows correspond to the target genes controlled by those transcription factors.
** A “1” means there is an edge connecting them and a “0” means that there is no edge connecting them.
** The upper-left cell (A1) should contain the text “cols regulators/rows targets”. This text is there as a reminder of the direction of the regulatory relationships specified by the adjacency matrix.
** The rest of row 1 should contain the names of the transcription factors that are controlling the other genes in the network, one transcription factor name per column.
** The rest of column A should contain the names of the target genes that are being controlled by the transcription factors heading each of the columns in the matrix, one target gene name per row.
** The transcription factor names should correspond to the "id" in the other sheets in the workbook. They should be capitalized the same way and occur in the same order along the top and side of the matrix. The matrix needs to be symmetric, i.e., the same transcription factors should appear along the top and left side of the matrix. The genes should be listed in the same order in all the sheets in the Excel workbook.
** Each cell in the matrix should then contain a zero (0) if there is no regulatory relationship between those two transcription factors, or a one (1) if there is a regulatory relationship between them. Again, the columns correspond to the transcription factors and the rows correspond to the target genes controlled by those transcription factors.

==== network_weights sheet ====

* These are the initial guesses for the estimation of the weight parameters, ''w''.
* Since these weights are initial guesses which will be optimized by GRNmap, the content of this sheet can be identical to the "network" sheet.

==== optimization_parameters sheet ====

* The optimization_parameters sheet should have two columns (from left to right) entitled, "optimization_parameter" and "value".
* You should copy this worksheet from the [https://github.com/kdahlquist/DahlquistLab/raw/master/data/Spring2019/15-genes_28-edges_sample_Sigmoid_estimation_2019.xlsx sample workbook] provided. The only row that you need to modify is row 15, "Strain". Include just the strain designations for which you have a corresponding STRAIN_log2_expression sheet. If you don't have the dgln3, dhap4, or dcin5 expression sheets, then you will delete those from this row. If you do so, make sure that you don't leave any gaps between cells.
* What follows below is an explanation of what the optimization_parameters mean.
** alpha: Penalty term weighting (from the L-curve analysis)
** kk_max: Number of times to re-run the optimization loop. In some cases re-starting the optimization loop can improve performance of the estimation.
** MaxIter: Number of times MATLAB iterates through the optimization scheme. If this is set too low, MATLAB will stop before the parameters are optimized.
** TolFun: How different two least squares evaluations should be before the program determines that it is not making any improvement
** MaxFunEval: maximum number of times the program will evaluate the least squares cost
** TolX: How close successive least squares cost evaluations should be before the program determines that it is not making any improvement.
** production_function: = Sigmoid (case-insensitive) if sigmoidal model, =MM (case-insensitive) if Michaelis-Menten model
** L_curve: =0 if an L-curve analysis should NOT be run or =1 if an L-curve analysis SHOULD be run. The L-curve analysis will automatically run sequential rounds of estimation for an array of fixed alpha values (0.8, 0.5, 0.2, 0.1,0.08, 0.05,0.02,0.01, 0.008, 0.005, 0.002, 0.001, 0.0008, 0.0005, 0.0002, and 0.0001). GRNmap makes a copy of the user's selected input workbook and changes alpha to the first alpha in the list. The estimation runs and the resulting parameter values are used as the initial guesses for the next round of estimation with the next alpha value. This process repeats until all alpha values have been run. New input and output workbooks are generated for each alpha value, although currently, the graphs are only saved for the last run.
** estimate_params =1 if want to estimate parameters and =0 if the user wants to do just one forward run
** make_graphs =1 to output graphs; =0 to not output graphs
** fix_P =1 if the user does not want to estimate the production rate, ''P'', parameter, just use the initial guess and never change; =0 to estimate
** fix_b =1 if the user does not want to estimate the ''b'' parameter, just use the initial guess and never change; =0 to estimate
** expression_timepoints: A row containing a list of the time points when the data was collected experimentally. Should correspond to the timepoint column headers in the STRAIN_log2_expression sheets.
** Strain: A row containing a list of all of the strains for which there is expression data in the workbook. Should correspond to the "STRAIN" portion of the names of the STRAIN_log2_expression sheets for each strain. Note that GRNmap will run the model for the wild type network (all genes present in the network) and for networks where the gene deleted from the designated STRAIN has been deleted from the network.
** simulation_timepoints: A row containing a list of the time points at which to evaluate the differential equations to generate the simulated data. This does not need to correspond to the actual measurement times, but should be in the same units (e.g. minutes).

==== threshold_b sheet ====

* These are the initial guesses for the estimation of the ''threshold_b'' parameters.
* There should be two columns.
** The left-most column should contain the header "id" and list the standard names for the genes in the model in the same order as in the other sheets.
** The second column should have the header "threshold_b" and should contain the initial guesses, we are going to use all 0.

== Conclusion ==
The first stage of our group's project was completed via referencing [[Week 8]] and using Microsoft Excel to complete the tasks. The excel file will be located in the [[FunGals]] page for viewing and download. We were able to successfully complete the ANOVA analysis and correct found mistakes. We were able to complete a sanity check with results showing. For the sanity check the unadjusted p- values showed 37.2% of the genes had a p<0.05, 18.16% had a p<0.01, 5.04% have p<0.001, and 0.87% have p<0.0001. The sanity check for the Bonferroni-corrected p-value 0.11% of the genes have p<0.05. For the sanity check on the Benjamini and Hochberg-corrected p-value 16.36% go the genes had a p <0.05. Finally we were able to prepare the Data to run STEM in the next step of working on this project.

Extra Notes (will delete later): PDR3 did not show up in Access while running Query's

==Data and files==
[[media:Thiuram_yeast_experiment.xlsx|ANOVA & STEM workbook (.xlsx)]]

[[media:Genelist_combined_profiles_FunGals.xlsx| YEASTRACT Rank by TF - Green and Red Profiles (.xlsx)]]

[[media:FunGals_Green_Profile_RegulationMatrix_Documented_2019123_2322_1272399515.xlsx| Green Regulation Matrix/network (.xlsx)]]

[[media:FunGals Red Profile RegulationMatrix Documented 2019123 2318 1127808084.xlsx| Input Workbook Red network (.xlsx)]]

[[media:FunGals Green Profile RegulationMatrix Input Workbook.xlsx | Input Workbook for Green profile (.xlsx)]]

[[media:GRNmap FunGals Green profile.zip | Output Workbook & GRNModel from MatLab for Green profile (.zip)]]

[[Media:GRN_Model_Red_from_Matlab.zip | Output Workbook & GRNModel from MatLab for Red profile (.zip)]]

[[media:Data_for_FunGals.pptx|FunGals Data on Powerpoint]]

== Acknowledgements ==
This section is in acknowledgement to partner Kaitlyn Nguyen (User:knguye66), Michael Armas (User:Marmas), as well as, Iliana Crespin (User:Icrespin), and Emma Young (User:eyoung20). We would also like to acknowledge Dr. Dahlquist (User:KDahlquist) for introducing and teaching the topic and direction of this assignment.
Also to acknowledge that this is a shared electronic notebook between [[user:knguye66|Kaitlyn Nguyen]] and [[user:eyoung20|Emma Young]].

"Except for what is noted above, this individual journal entry was completed by me and not copied from another source."
[[User:Knguye66|Knguye66]] ([[User talk:Knguye66|talk]]) 18:49, 20 November 2019 (PST)

"Except for what is noted above, this individual journal entry was completed by me and not copied from another source."
[[User:Eyoung20|Eyoung20]] ([[User talk:Eyoung20|talk]]) 16:40, 25 November 2019 (PST)

== References ==
*Dahlquist, K. (2019, November 19). Data Analysis. In Wikipedia, Biological Databases. Retrieved 6:25, November 20, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Data_Analysis
*Dahlquist, K. (2019, November 20). Final Project Deliverables. In Wikipedia, Biological Databases. Retrieved 2:59, December 5, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Final_Project_Deliverables
*Dahlquist, K. (2019, October 29). Week 9. In Wikipedia, Biological Databases. Retrieved 2:57, December 2, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_9
*Dahlquist, K. (2019, November 7). Week 10. In Wikipedia, Biological Databases. Retrieved 2:59, December 5, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_10

{{Template:knguye66}}
{{Template:Eyoung20}}

[[Category: Group Projects]]
[[Category: FunGals]]

Knguye66 Eyoung20 Week 15

2019-12-08T23:56:59Z

Eyoung20: /* Data and files */ added data

'''Combined Individual Journals for Kaitlyn Nguyen and Emma Young (Data Analysts).'''

== Purpose ==
The purpose of this assignment is to record our progress towards the [[FunGals]] group [[Final_Project_Deliverables | deliverables]] as the [[Data_Analysis | Data Analysts]] for this week and the future weeks to come. The purpose of week 12 specifically was to download and adapt the data to the formatting we need for analysis. Then to begin the analysis with ANOVA and preparations for STEM.

== Methods and Results: Progress ==
===== - Progress 11/26/19 - =====

==== Analyzing and Interpreting STEM Results ====
#Why did you select this profile? In other words, why was it interesting to you?
#* We collectively combined the 4 red profiles together because they had similar trends and to increase the number of genes for analysis (called "Red".) Similarly this was done to the 3 green profiles as well (upward trends), with the addition of the blue profile #29 added. We will call this group "Green".
#How many genes belong to this profile?
#* Red (composed of Profile #9,26,34,11): 289
#* Green (Profile #40,42,18,29): 214
#How many genes were expected to belong to this profile?
#* Red (composed of Profile #9,26,34,11): 51.5
#* Green (Profile #40,42,18,29): 48.5
#What is the p value for the enrichment of genes in this profile?
#* Due to combining the profiles, we do not have p-values for the enrichment of genes in the 2 different groups.

- '''Viewing and Saving STEM Results''' -
# A new window will open called "All STEM Profiles (1)". Each box corresponds to a model expression profile. Colored profiles have a statistically significant number of genes assigned; they are arranged in order from most to least significant p value. Profiles with the same color belong to the same cluster of profiles. The number in each box is simply an ID number for the profile.
#* Click on the button that says "Interface Options...". At the bottom of the Interface Options window that appears below where it says "X-axis scale should be:", click on the radio button that says "Based on real time". Then close the Interface Options window.
#*Take a screenshot of this window (on a PC, simultaneously press the <code>Alt</code> and <code>PrintScreen</code> buttons to save the view in the active window to the clipboard) and paste it into a PowerPoint presentation to save your figures.
# Click on each of the SIGNIFICANT profiles (the colored ones) to open a window showing a more detailed plot containing all of the genes in that profile.
#* Take a screenshot of each of the individual profile windows and save the images in your PowerPoint presentation.
#* At the bottom of each profile window, there are two yellow buttons "Profile Gene Table" and "Profile GO Table". For each of the profiles, click on the "Profile Gene Table" button to see the list of genes belonging to the profile. In the window that appears, click on the "Save Table" button and save the file to your desktop. Make your filename descriptive of the contents, e.g. "wt_profile#_genelist.txt", where you replace the number symbol with the actual profile number.
#** Upload these files to the wiki and link to them on your individual journal page. (Note that it will be easier to [[Week_4#Compressing_and_Decompressing_Files_with_7-Zip | zip all the files together]] and upload them as one file).
#* For each of the significant profiles, click on the "Profile GO Table" to see the list of Gene Ontology terms belonging to the profile. In the window that appears, click on the "Save Table" button and save the file to your desktop. Make your filename descriptive of the contents, e.g. "wt_profile#_GOlist.txt", where you use "wt", "dGLN3", etc. to indicate the dataset and where you replace the number symbol with the actual profile number. At this point you have saved all of the primary data from the STEM software and it's time to interpret the results!
#** Upload these files to the wiki and link to them on your individual journal page. (Note that it will be easier to [[Week_4#Compressing_and_Decompressing_Files_with_7-Zip | zip all the files together]] and upload them as one file).

==== Clustering the data with STEM, as did on [[Week 9]]. ====
# Note that we will make some adjustments to the GO term analysis because stem was not providing GO term names. We are going to use the GO enrichment tool at GeneOntology.org instead.
#* Go to [http://geneontology.org/ http://geneontology.org/].
#* For the cluster you want to analyze, open the gene list and copy the list of genes.
#* Paste the list of genes into the "Go Enrichment Analysis" box on the right hand side of the GeneOntology.org page.
#* Select "Saccharomyces cerevisiae" from the species drop-down menu.
# Click the "Launch" buton.
#* Near the bottom of the results page, click on the button to Export "Table".
#* This will prompt you to save a .txt file that can be opened in Excel to view your results.
# Use YEASTRACT to generate a candidate gene regulatory network as you did on [[Week 9]].

==== Using YEASTRACT to Infer which Transcription Factors Regulate a Cluster of Genes ====

In the previous analysis using STEM, we found a number of gene expression profiles (aka clusters) which grouped genes based on similarity of gene expression changes over time. The implication is that these genes share the same expression pattern because they are regulated by the same (or the same set) of transcription factors. We will explore this using the YEASTRACT database.

# Open the gene list in Excel for the one of the significant profiles from your stem analysis. Choose a cluster with a clear cold shock/recovery up/down or down/up pattern. You should also choose one of the largest clusters.
#* Copy the list of gene IDs onto your clipboard.
# Launch a web browser and go to the [http://www.yeastract.com/ YEASTRACT database].
#* On the left panel of the window, click on the link to [http://www.yeastract.com/formrankbytf.php ''Rank by TF''].
#* Paste your list of genes from your cluster into the box labeled ''ORFs/Genes''.
#* Check the box for ''Check for all TFs''.
#* Accept the defaults for the Regulations Filter (Documented, DNA binding plus expression evidence)
#* Do '''''not''''' apply a filter for "Filter Documented Regulations by environmental condition".
#* Rank genes by TF using: The % of genes in the list and in YEASTRACT regulated by each TF.
#* Click the ''Search'' button.
# Answer the following questions:
#* In the results window that appears, the p values colored green are considered "significant", the ones colored yellow are considered "borderline significant" and the ones colored pink are considered "not significant". '''''How many transcription factors are green or "significant"?'''''
#**Green: 31
#**Red: 30
#* '''''Copy the table of results from the web page and paste it into a new Excel workbook to preserve the results.'''''
#** '''''Upload the Excel file to the wiki and link to it in your electronic lab notebook.'''''
#** '''''Are CIN5, GLN3, and/or HAP4 on the list? If so, what is their "% in user set", "% in YEASTRACT", and "p value".'''''
#**Green
#*** CIN5
#**** 0.2991 % in user set
#****0.0294
#**** p-value: 0.5492
#***GLN3
#**** 0.3645% in user set
#****0.0324% in YEASTRACT
#**** p-value: 0.18046
#***HAP4
#**** 0.2383% in user set
#****0.0467% in YEASTRACT
#**** P-value:0.00031337
#**Red
#*** CIN5
#**** 0.2111%% in user set
#**** 0.028% in YEASTRACT
#**** p-value: 0.9998
#***GLN3
#**** 0.4152% in user set
#****0.0498% in YEASTRACT
#**** p-value:0.00207752
#***HAP4
#****0.2353% in user set
#****0.0623% in YEASTRACT
#**** P-value:0.000006235
# For the mathematical model that we will build, we need to define a ''gene regulatory network'' of transcription factors that regulate other transcription factors. We can use YEASTRACT to assist us with creating the network. We want to generate a network with approximately 15-20 transcription factors in it.
#* You need to select from this list of "significant" transcription factors, which ones you will use to run the model. You will use these transcription factors and add GLN3, HAP4, and CIN5 if they are not in your list. Explain in your electronic notebook how you decided on which transcription factors to include. Record the list and your justification in your electronic lab notebook. Each group member will select a different network (they can have some overlapping transcription factors, but some should also be different).
#* Go back to the YEASTRACT database and follow the link to ''[http://www.yeastract.com/formregmatrix.php Generate Regulation Matrix]''.
#* Copy and paste the list of transcription factors you identified (plus HAP4, GLN3, and CIN5) into both the "Transcription factors" field and the "Target ORF/Genes" field.
#* We are going to use the "Regulations Filter" options of "Documented", "'''Only''' DNA binding evidence"
#** Click the "Generate" button.
#** In the results window that appears, click on the link to the "Regulation matrix (Semicolon Separated Values (CSV) file)" that appears and save it to your Desktop. Rename this file with a meaningful name so that you can distinguish it from the other files you will generate.

==== Visualizing Your Gene Regulatory Networks with GRNsight ====

We will analyze the regulatory matrix files you generated above in Microsoft Excel and visualize them using GRNsight to determine which one will be appropriate to pursue further in the modeling.
# First we need to properly format the output files from YEASTRACT.
#* Open the file in Excel. It will not open properly in Excel because a semicolon was used as the column delimiter instead of a comma. To fix this, Select the entire Column A. Then go to the "Data" tab and select "Text to columns". In the Wizard that appears, select "Delimited" and click "Next". In the next window, select "Semicolon", and click "Next". In the next window, leave the data format at "General", and click "Finish". This should now look like a table with the names of the transcription factors across the top and down the first column and all of the zeros and ones distributed throughout the rows and columns. This is called an "adjacency matrix." If there is a "1" in the cell, that means there is a connection between the trancription factor in that row with that column.
#* Save this file in Microsoft Excel workbook format (.xlsx).

#* For this adjacency matrix to be usable in GRNmap (the modeling software) and GRNsight (the visualization software), we need to transpose the matrix. Insert a new worksheet into your Excel file and name it "network". Go back to the previous sheet and select the entire matrix and copy it. Go to you new worksheet and click on the A1 cell in the upper left. Select "Paste special" from the "Home" tab. In the window that appears, check the box for "Transpose". This will paste your data with the columns transposed to rows and vice versa. This is necessary because we want the transcription factors that are the "regulatORS" across the top and the "regulatEES" along the side.
#* The labels for the genes in the columns and rows need to match. Thus, delete the "p" from each of the gene names in the columns. Adjust the case of the labels to make them all upper case.
#* In cell A1, copy and paste the text "rows genes affected/cols genes controlling".
#* Finally, for ease of working with the adjacency matrix in Excel, we want to alphabatize the gene labels both across the top and side.
#** Select the area of the entire adjacency matrix.
#** Click the Data tab and click the custom sort button.
#** Sort Column A alphabetically, being sure to exclude the header row.
#** Now sort row 1 from left to right, excluding cell A1. In the Custom Sort window, click on the options button and select sort left to right, excluding column 1.
#* Name the worksheet containing your organized adjacency matrix "network" and Save.
# Now we will visualize what these gene regulatory networks look like with the GRNsight software.
#* Go to the [http://dondi.github.io/GRNsight/ GRNsight] home page.
#* Select the menu item File > Open and select the regulation matrix .xlsx file that has the "network" worksheet in it that you formatted above. If the file has been formatted properly, GRNsight should automatically create a graph of your network. You can click the "Grid Layout" button to arrange the nodes in a grid, or you can click and drag the nodes (genes) around until you get a layout that you like and take a screenshot of the results. Paste it into your PowerPoint presentation.
#** If you have nodes (genes) floating around in the display that are not connected to any other nodes, we need to delete them from the network for the modeling to work properly. Go back to the Excel workbook and network sheet and delete both the row and column with the floating gene's name. Then re-upload the edited file to GRNsight to visualize it. Use this final version in your PowerPoint and subsequent modeling.
#** The deleted genes (floating) on GRNsight were:
#*** Green: MSN and BAS1

===== - Progress 12/03/19 - =====

==== production_rates sheet ====

* This sheet contains initial guesses for the production rate parameters, ''P'', for all genes in the network.
* Assuming that the system is in steady state with the relative expression of all genes equal to 1, (P/2) - lambda = 0, where lambda is the degradation rate, is a reasonable initial guess.
* The sheet should contain two columns (from left to right) entitled, "id", "production_rate".
** The id is an identifier that the user will use to identify a particular gene. In our case, we are using the "StandardName", for example, GLN3.
** The "production_rate" column should then contain the initial guesses for the ''P'' parameter as described above, rounded to four decimal places.
*** The production rates are provided in a Microsoft Access database, which you can [https://github.com/kdahlquist/DahlquistLab/raw/master/data/Spring2019/Expression-and-Degradation-rate-database_2019.accdb download from here.]
*** You will perform a query to get the list of production rates for each gene as a group.
*** To perform the query, you will need to follow these steps.
***# Import a your list of genes to a new table in the database. Click on the "External Data" tab and select the Excel icon with the "up" arrow on it.
***# Click the "Browse" button and select your Excel file containing your network that you used to upload to GRNsight.
***# Make sure the button next to "Import the source data into a new table in the current database" and click "OK".
***# In the next window, select the "network" worksheet, if it hasn't already been automatically selected for you. Click "Next".
***# In the next window, make sure the "First Row Contains Column Headings" is checked. Click "Next".
***# In the next window, the left-most column will be highlighted. Change the "Field Name" to "id" if it doesn't say that already. Click "Next".
***# In the next window, select the button for "Choose my own primary key." and choose the "id" field from the drop down next to it. Click "Next".
***# In the next field, make sure it says "Import to Table: network". Click Finish.
***# In the next window you do not need to save the import steps, so just click "Close".
***# A table called "network" should appear in the list of tables at the left of the window.
***# Go to the "Create" tab. Click on the icon for "Query Design".
***# In the window that appears, click on the "network" table and click "Add". Click on the "production_rates" table and click "Add". Click "Close".
***# The two tables should appear in the main part of the window. We need to tell Access which fields in the two tables correspond to each other. Click on the word "id" in the network table and drag your mouse to the "standard_name" field in the "production_rates" table, and release. You will see a line appear between those two words.
***# Right-click on the line between those words and select "Join Properties" from the menu that appears. Select Option "2: Include ALL records from 'network' and only those records from 'production_rates' where the joined fields are equal." Click "OK".
***# Click on the "id" word in the "network" table and drag it to the bottom of the screen to the first column next to the word "Field" and release.
***# Click on the "production_rate" field in the "production_rates" table and drag it to the bottom of the screen to the second column next to the word "Field" and release.
***# Right-click anywhere in the gray area near the two tables. In the menu that appears, select "Query Type > Make Table Query...".
***# In the window that appears, name your table "production_rates_1" because you can't have two tables with the same name in the database. Make sure that "Current Database" is selected and Click "OK".
***# Go to the "Query Tools: Menus" tab. Click on the exclamation point icon. A window will appear that tells you how many rows you are pasting into a new table. Click "Yes".
***# Your new "production_rates_1" table will appear in the list at the left. Double-click on that table name to open it.
***# You can copy the data in this table and paste it back into your Excel workbook. Make sure that when you paste that you use "Paste Special > Paste values" so that the Access formatting doesn't get carried along. You can also choose to export this table to Excel going to the "External Data" tab and selecting the Excel icon with the arrow pointing to the right. Select the workbook you want to export the table to, making sure that "Preserve Access formatting" is '''not''' checked. Click "OK", click "Close".
* If there are missing values, substitute the value <code>0.1980</code> for the missing production rates.
* Note that the genes should be listed in the same order in all the sheets in the Excel workbook.

==== degradation_rates sheet ====

* This sheet contains degradation rates for all genes in the network, which are provided by the user.
* Currently, the Dahlquist Lab is using data based on published mRNA half-life data from [http://rnajournal.cshlp.org/content/20/10/1645.full Neymotin et al. (2006)].
** We converted the half-life data values to the degradation rates by taking the natural log of the half-life and dividing by 2.
* The sheet should contain two columns (from left to right) entitled "id", and "degradation_rate".
** The id is an identifier that the user will use to identify a particular gene.
** The "degradation_rate" column should then contain the absolute value of the degradation rate for the corresponding gene as described above, rounded to four decimal places.
*** To obtain these values, you will use the same file, [https://github.com/kdahlquist/DahlquistLab/raw/master/data/Spring2019/Expression-and-Degradation-rate-database_2019.accdb Microsoft Access database] that you used to obtain the production rates in the first worksheet. Again, you can copy and paste the values one-by-one or you can follow the instructions to execute a query, substituting the appropriate "degradation_rates" table in the query. Note that you don't need to re-import your "network" table, you just need to create and execute the query.
* Again note, the genes should be listed in the same order in all the sheets in the Excel workbook.
* If there are missing values, substitute the value <code>0.0990</code> for the missing degradation rates.

==== Expression Data Sheets for Individual Yeast Strains ====

* Expression data can be provided for either a single strain or multiple strains of yeast (for example, the wild type strain and a transcription factor deletion strain).
** Each strain will have its own sheet in the workbook.
** Each sheet should be given a unique name that follows the convention "STRAIN_log2_expression", where the word "STRAIN" is replaced by the strain designation, which will appear in the optimization_diagnostics sheet.
*** Everyone in the class will have at least one expression worksheet called "wt_log2_expression".
*** You should have included the transcription factors GLN3, HAP4, and CIN5 in your network. Thus, we will use the expression data from the dGLN3, dHAP4, dCIN5 deletion strains in our workbooks as well, naming the worksheets "dgln3_log2_expression", "dhap4_log2_expression", and "dcin5_expression".
**** If, for some reason, you don't have all three of those genes in your network, only include expression data for the wild type and the genes out of those three that you have in your network.
* The sheet should have the following columns in this order:
*# "id": list of all genes. The genes should be listed in the same order in all the sheets in the Excel workbook.
*# The next series of columns should contain the expression data for each gene at a given timepoint given as log2 ratios (log2 fold changes). The column header should be the time at which the data were collected, without any units. For example, the 15 minute timepoint would have a column header "15" and the 30 minute timepoint would have the column header "30". GRNmap supports replicate data for each of the timepoints. Replicate data for the same timepoint should be in columns immediately next to each other and have the same column headers. For example, three replicates of the 15 minute timepoint would have "15", "15", "15" as the column headers.
*# If data are provided for multiple strains, each strain should have data for the same timepoints, although the number of replicates can vary.
* Include the data for the 15, 30, and 60 minute timepoints, but not the 90 or 120 minute timepoints.
* The data you will be using is contained in the [https://github.com/kdahlquist/DahlquistLab/raw/master/data/Spring2019/Expression-and-Degradation-rate-database_2019.accdb Expression-and-Degradation-rate-database_2019.accdb] file that you used to obtain the production and degradation rates.
* It is tedious to copy and paste all of these data by hand, so we will execute a query in Microsoft Access to do it for you. Follow the steps listed for the "production_rates" sheet for each strains expression data. After you import the data into Excel, you will need to change the column headers to "15", "15", etc., as described above.
* Missing values in the expression data sheets are OK; you don't need to put any values there like you did for the production_rates or degradation_rates sheets.

==== network sheet ====

* The network you derived from the YEASTRACT database for the [[Week 9]] assignment can be copied and pasted into this sheet directly. You may need to edit the contents of cell A1, but the rest should be good to go (especially since you previewed it in GRNsight). The description below just explains what is already in this worksheet.
** This sheet contains an adjacency matrix representation of the gene regulatory network.
** The columns correspond to the transcription factors and the rows correspond to the target genes controlled by those transcription factors.
** A “1” means there is an edge connecting them and a “0” means that there is no edge connecting them.
** The upper-left cell (A1) should contain the text “cols regulators/rows targets”. This text is there as a reminder of the direction of the regulatory relationships specified by the adjacency matrix.
** The rest of row 1 should contain the names of the transcription factors that are controlling the other genes in the network, one transcription factor name per column.
** The rest of column A should contain the names of the target genes that are being controlled by the transcription factors heading each of the columns in the matrix, one target gene name per row.
** The transcription factor names should correspond to the "id" in the other sheets in the workbook. They should be capitalized the same way and occur in the same order along the top and side of the matrix. The matrix needs to be symmetric, i.e., the same transcription factors should appear along the top and left side of the matrix. The genes should be listed in the same order in all the sheets in the Excel workbook.
** Each cell in the matrix should then contain a zero (0) if there is no regulatory relationship between those two transcription factors, or a one (1) if there is a regulatory relationship between them. Again, the columns correspond to the transcription factors and the rows correspond to the target genes controlled by those transcription factors.

==== network_weights sheet ====

* These are the initial guesses for the estimation of the weight parameters, ''w''.
* Since these weights are initial guesses which will be optimized by GRNmap, the content of this sheet can be identical to the "network" sheet.

==== optimization_parameters sheet ====

* The optimization_parameters sheet should have two columns (from left to right) entitled, "optimization_parameter" and "value".
* You should copy this worksheet from the [https://github.com/kdahlquist/DahlquistLab/raw/master/data/Spring2019/15-genes_28-edges_sample_Sigmoid_estimation_2019.xlsx sample workbook] provided. The only row that you need to modify is row 15, "Strain". Include just the strain designations for which you have a corresponding STRAIN_log2_expression sheet. If you don't have the dgln3, dhap4, or dcin5 expression sheets, then you will delete those from this row. If you do so, make sure that you don't leave any gaps between cells.
* What follows below is an explanation of what the optimization_parameters mean.
** alpha: Penalty term weighting (from the L-curve analysis)
** kk_max: Number of times to re-run the optimization loop. In some cases re-starting the optimization loop can improve performance of the estimation.
** MaxIter: Number of times MATLAB iterates through the optimization scheme. If this is set too low, MATLAB will stop before the parameters are optimized.
** TolFun: How different two least squares evaluations should be before the program determines that it is not making any improvement
** MaxFunEval: maximum number of times the program will evaluate the least squares cost
** TolX: How close successive least squares cost evaluations should be before the program determines that it is not making any improvement.
** production_function: = Sigmoid (case-insensitive) if sigmoidal model, =MM (case-insensitive) if Michaelis-Menten model
** L_curve: =0 if an L-curve analysis should NOT be run or =1 if an L-curve analysis SHOULD be run. The L-curve analysis will automatically run sequential rounds of estimation for an array of fixed alpha values (0.8, 0.5, 0.2, 0.1,0.08, 0.05,0.02,0.01, 0.008, 0.005, 0.002, 0.001, 0.0008, 0.0005, 0.0002, and 0.0001). GRNmap makes a copy of the user's selected input workbook and changes alpha to the first alpha in the list. The estimation runs and the resulting parameter values are used as the initial guesses for the next round of estimation with the next alpha value. This process repeats until all alpha values have been run. New input and output workbooks are generated for each alpha value, although currently, the graphs are only saved for the last run.
** estimate_params =1 if want to estimate parameters and =0 if the user wants to do just one forward run
** make_graphs =1 to output graphs; =0 to not output graphs
** fix_P =1 if the user does not want to estimate the production rate, ''P'', parameter, just use the initial guess and never change; =0 to estimate
** fix_b =1 if the user does not want to estimate the ''b'' parameter, just use the initial guess and never change; =0 to estimate
** expression_timepoints: A row containing a list of the time points when the data was collected experimentally. Should correspond to the timepoint column headers in the STRAIN_log2_expression sheets.
** Strain: A row containing a list of all of the strains for which there is expression data in the workbook. Should correspond to the "STRAIN" portion of the names of the STRAIN_log2_expression sheets for each strain. Note that GRNmap will run the model for the wild type network (all genes present in the network) and for networks where the gene deleted from the designated STRAIN has been deleted from the network.
** simulation_timepoints: A row containing a list of the time points at which to evaluate the differential equations to generate the simulated data. This does not need to correspond to the actual measurement times, but should be in the same units (e.g. minutes).

==== threshold_b sheet ====

* These are the initial guesses for the estimation of the ''threshold_b'' parameters.
* There should be two columns.
** The left-most column should contain the header "id" and list the standard names for the genes in the model in the same order as in the other sheets.
** The second column should have the header "threshold_b" and should contain the initial guesses, we are going to use all 0.

== Conclusion ==
The first stage of our group's project was completed via referencing [[Week 8]] and using Microsoft Excel to complete the tasks. The excel file will be located in the [[FunGals]] page for viewing and download. We were able to successfully complete the ANOVA analysis and correct found mistakes. We were able to complete a sanity check with results showing. For the sanity check the unadjusted p- values showed 37.2% of the genes had a p<0.05, 18.16% had a p<0.01, 5.04% have p<0.001, and 0.87% have p<0.0001. The sanity check for the Bonferroni-corrected p-value 0.11% of the genes have p<0.05. For the sanity check on the Benjamini and Hochberg-corrected p-value 16.36% go the genes had a p <0.05. Finally we were able to prepare the Data to run STEM in the next step of working on this project.

Extra Notes (will delete later): PDR3 did not show up in Access while running Query's

==Data and files==
[[media:Thiuram_yeast_experiment.xlsx|ANOVA & STEM workbook (.xlsx)]]

[[media:Genelist_combined_profiles_FunGals.xlsx| YEASTRACT Rank by TF - Green and Red Profiles (.xlsx)]]

[[media:FunGals_Green_Profile_RegulationMatrix_Documented_2019123_2322_1272399515.xlsx| Green Regulation Matrix/network (.xlsx)]]

[[media:FunGals Red Profile RegulationMatrix Documented 2019123 2318 1127808084.xlsx| Input Workbook Red network (.xlsx)]]

[[media:FunGals Green Profile RegulationMatrix Input Workbook.xlsx | Input Workbook for Green profile (.xlsx)]]

[[media:GRNmap FunGals Green profile.zip | Output Workbook & GRNModel from MatLab for Green profile (.zip)]]

[[Media:GRN_Model_Red_from_Matlab.zip | Output Workbook & GRNModel from MatLab for Red profile (.zip)]]

[[File:Green_GRNsight_output.png|500px|thumb|center| Output GRNsight for Green profile]]

[[media:Data_for_FunGals.pptx|FunGals Data on Powerpoint]]

== Acknowledgements ==
This section is in acknowledgement to partner Kaitlyn Nguyen (User:knguye66), Michael Armas (User:Marmas), as well as, Iliana Crespin (User:Icrespin), and Emma Young (User:eyoung20). We would also like to acknowledge Dr. Dahlquist (User:KDahlquist) for introducing and teaching the topic and direction of this assignment.
Also to acknowledge that this is a shared electronic notebook between [[user:knguye66|Kaitlyn Nguyen]] and [[user:eyoung20|Emma Young]].

"Except for what is noted above, this individual journal entry was completed by me and not copied from another source."
[[User:Knguye66|Knguye66]] ([[User talk:Knguye66|talk]]) 18:49, 20 November 2019 (PST)

"Except for what is noted above, this individual journal entry was completed by me and not copied from another source."
[[User:Eyoung20|Eyoung20]] ([[User talk:Eyoung20|talk]]) 16:40, 25 November 2019 (PST)

== References ==
*Dahlquist, K. (2019, November 19). Data Analysis. In Wikipedia, Biological Databases. Retrieved 6:25, November 20, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Data_Analysis
*Dahlquist, K. (2019, November 20). Final Project Deliverables. In Wikipedia, Biological Databases. Retrieved 2:59, December 5, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Final_Project_Deliverables
*Dahlquist, K. (2019, October 29). Week 9. In Wikipedia, Biological Databases. Retrieved 2:57, December 2, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_9
*Dahlquist, K. (2019, November 7). Week 10. In Wikipedia, Biological Databases. Retrieved 2:59, December 5, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_10

{{Template:knguye66}}
{{Template:Eyoung20}}

[[Category: Group Projects]]
[[Category: FunGals]]

Knguye66 Eyoung20 Week 12/13

2019-12-08T23:56:34Z

Eyoung20: /* Data and files */ formatting

Knguye66 Eyoung20 Week 12/13

2019-12-08T23:56:18Z

Eyoung20: /* Data and files */ added file

File:Data for FunGals.pptx

2019-12-08T23:55:06Z

Eyoung20:

FunGals

2019-12-06T00:25:52Z

Eyoung20: /* Week 15 */ added data

==Effects of the Pesticide Thiuram: Genome-wide Screening of Indicator Genes by Yeast DNA Microarray==

'''Team Information'''
Project Manager: [[User:Marmas|Michael Armas]]
Quality Assurance: [[User:Icrespin| Iliana Crespin]]
Data Analysis: [[User:eyoung20|Emma Young]], [[User:knguye66|Kaitlyn Nguyen]]
Coder: [[User:Marmas|Michael Armas]]

== Methods and Results ==
=== Week 11 ===
====Setting up the Project====
*Selected [[User:Marmas|Michael Armas]] as team's Project Manager.
*Added the name of of selected project manager to the Project Manager guild page and Overview pages.
*Named the team FunGals and created [[FunGals]] home page on the wiki.
*The name of the team home page was the selected team name.

*Created a link to team's page on the course Main page.
*Created a template for FunGal with useful information and links that you will invoke on all pages that you will create for the project.
*Created a category using team name and included it the FunGal template so that it is used on all pages created for the project. Also included the category "Group Projects" in the template. 
*Each person needs to write a short executive summary of that person's progress on the project for the week, with links to the relevant individual journal pages (which will have more detailed information).
*Each team member should reflect on the team's progress '''(Note that you will be directed to add specific information to your team's pages in the individual portion of the assignment for this and future weeks)''':
**What worked? What didn't work? What will I do next to fix what didn't work?
*** Michael Armas: Working with this team is a great pleasure! Everyone on the team is pulling their weight and providing information that contributes to the group work. Next time, as the project manager, I want to be more organized and come up with interim deadlines to make sure we are not cramming right before the actual deadline. In fact, I would have something small due every day than procrastinate until the end. However, with this team, we are making it work and doing exceptional work!
*** Kaitlyn Nguyen: What worked this week is starting the presentation formatting early, thus allocating the rest of the time to "filling in the missing parts" of the powerpoint. What also worked this week was team communication via group-chats. What didn't work this week was starting the individual assignment later, leaving less time to practicing and fully understanding the material from the journal article for the presentation in class. To fix what didn't work, I will start my individual assignment much earlier, to leave the rest of the time to focus on collaborative teamwork and group assignments. [[User:Knguye66|Knguye66]] ([[User talk:Knguye66|talk]]) 23:37, 13 November 2019 (PST)
*** Emma Young: The team seems to work really well together. I think in upcoming weeks we will really thrive in working together. A foundation for communication has already been established and we have already started reviewing each others work and providing constructive criticism and help where it is needed. The main issue this week was a matter of timing. With the amount of work we had to do in the time frame given we were not able to meet up due to busy and conflicting prior engagements. This meant that we did not reach our full potential of working together effectively as a group this week. Next week, we hopefully will be able to set better deadlines and be able to have the ability to plan out time to work on this in a less rushed manner.
*** Iliana Crespin: Overall, this group has been great. Everyone has worked together and understands what must be done. In addition, each member is understanding of the special circumstances before the deadline of this assignment. The only thing that didn't work out was how scattered each of us were because of all the assignments that had to be completed from other classes. In addition, mandated plans (school- or work-related) popped up which made it difficult to contribute equally. For the future, I will make sure to manage my time better, because I have work and school. Therefore, by doing a fair share of certain things before classes/work, I will be able to contribute more.[[User:Icrespin|Icrespin]] ([[User talk:Icrespin|talk]]) 23:46, 13 November 2019 (PST)

====Annotated Bibliography====

#Braconi, D., Bernardini, G., & Santucci, A. (2016). Saccharomyces cerevisiae as a model in ecotoxicological studies: A post-genomics perspective. Journal of Proteomics, 137, 19-34. DOI: 10.1016/j.jprot.2015.09.001
#Hinkle, K. L., & Olsen, D. (2018). Exposure to the lampricide TFM elicits an environmental stress response in yeast. FEMS yeast research, 19(1), foy121. doi: 10.1093/femsyr/foy121
#Iwahashi, Y., Hosoda, H., Park, J. H., Lee, J. H., Suzuki, Y., Kitagawa, E., ... & Iwahashi, H. (2006). Mechanisms of patulin toxicity under conditions that inhibit yeast growth. Journal of agricultural and food chemistry, 54(5), 1936-1942. doi: 10.1021/jf052264g.
#Iwahashi, H., Ishidou, E., Kitagawa, E., & Momose, Y. (2007). Combined Cadmium and Thiuram Show Synergistic Toxicity and Induce Mitochondrial Petite Mutants. Environmental Science & Technology, 41(22), 7941–7946. doi: 10.1021/es071313y
#Iwahashi, H., Ishidou, E., Kitagawa, E., & Momose, Y. (2007). Combined Cadmium and Thiuram Show Synergistic Toxicity and Induce Mitochondrial Petite Mutants. Environmental Science & Technology, 41(22), 7941–7946. doi: 10.1021/es071313y
#Kitagawa, E., Momose, Y., & Iwahashi, H. (2003). Correlation of the Structures of Agricultural Fungicides to Gene Expression in Saccharomyces cerevisiaeupon Exposure to Toxic Doses. Environmental Science & Technology, 37(12), 2788–2793. doi: 10.1021/es026156b
#Lu Yu, Na Guo, Rizeng Meng, Bin Liu, Xudong Tang, Jing Jin, Yumei Cui, Xuming Deng. Allicin-induced global gene expression profile of Saccharomyces cerevisiae. Applied Microbiology and Biotechnology 2010, 88 (1) , 219-229. DOI: 10.1007/s00253-010-2709-x.
#Pierron, A., Mimoun, S., Murate, L. S., Loiseau, N., Lippi, Y., Bracarense, A. P. F., ... & Oswald, I. P. (2016). Intestinal toxicity of the masked mycotoxin deoxynivalenol-3-β-D-glucoside. Archives of toxicology, 90(8), 2037-2046. doi: 10.1007/s00204-015-1592-8

===Week 12/13===
====Milestone 3: Getting the data ready for analysis ====

# As a group Downloaded and examined the microarray dataset, compared it to the samples and experiment described in the journal club article.
#* [https://sgd-prod-upload.s3.amazonaws.com/S000204415/Kitagawa_2002_PMID_12269742.zip Kitagawa et al. (2002)]
#* downloaded and unzipped
#* file was downloaded but was PLC format
#* Mike figured out how to drag and drop it one to excel to open the file
# Along with the QA's, make a "sample-data relationship table" that lists all of the samples (microarray chips), noting the treatment, time point, replicate number, and other information pertaining to each sample.
#* This "Metadata" sheet was created in Excel and is present on the excel work under the week 12/13 data and files.
#* In collaboration with other groups, the Metadata sheet was filled out so that nomenclature is consistent.
# The class collectively came up with consistent column headers that summarized the information
#* As decided by all groups, the format for column headers was yeastStrain_treatmentOrMutation_concentrationAndUnits_dataType_timeAndUnits-replicateNumber.
# Organized the data in a worksheet in an Excel workbook so that:
#* MasterIndex was in the first column, ID was in the second column
#* Data columns were to the right, in increasing chronological order, using the column header pattern that was created.
#* Replicates were grouped together
# Data analysts (DA) [[User:Knguye66|Kaitlyn]] set-up the ANOVA worksheet via referencing [[Week 8]]
#* The steps for the ANOVA: Part I, Benjamini, Bonferroni, and p-value correction, as well as, a quick sanity check were followed.
# DAs [[User:Eyoung20|Eyoung20]] begin setting up for STEM analysis to complete the first stage of milestones and deliverables
# All the calculations were checked by the QA [[User:Icrespin|Iliana]].

*Each team member should reflect on the team's progress '''(Note that you will be directed to add specific information to your team's pages in the individual portion of the assignment for this and future weeks)''':
**What worked? What didn't work? What will I do next to fix what didn't work?
*** Michael Armas: What worked this week was the communication between my guild and me, and my group and me. Everyone was able to help each other out and make sure the project is coming along smoothly. What didn't work this week for me was understanding exactly what I had to do to set up spreadsheets for data analysis. However, thanks to Mihir, I was able to understand how to create a metadata sheet and reach the milestone for this week. Before I leave class from now on, I will reassure myself that I understand the information by asking Dr. Dahlquist the best way to reach the week's milestone.
*** Kaitlyn Nguyen: This week, the pattern and organization to which each group member worked flowed smoothly as every member of the group knew exactly what each person was accomplishing. Having extra time in class and staying after class helped us jump-start on our tasks together and find out what assignments work best for each team member. What didn't work this week was that each milestone and part of this project works chronologically, whereby other members have to wait for the first person to finish his/her task before beginning theirs. As the days and weeks move on, it will be good to continue this method, but switch off on who starts the tasks first for the assignment. [[User:Knguye66|Knguye66]] ([[User talk:Knguye66|talk]]) 18:44, 20 November 2019 (PST)
*** Emma Young: For this week I worked closely with [[User:Knguye66|Kaitlyn]] to start the data analysis for the rest of the project. We worked with the whole team to analyze the data from the journal article and format for the rest of the data analysis. [[User:Marmas|Mike]] worked with the other programers to formulate the metadata sheet and the format for the ANOVA sheet. More details on this can be found in his personal electronic notebook [[Marmas Week 12/13]] Then [[User:Knguye66|Kaitlyn]] worked on the ANOVA analysis of the data. [[User:Icrespin|Iliana]] double checked the the results of the analysis the details can be found in [[Icrespin Journal Week 12/13]]. After the checks were completed and any errors were found, I ran a sanity check on the ANOVA data and then formatted the STEM worksheet for future analysis. The details about what [[User:Knguye66|Kaitlyn] and [[user:eyoung20|I]] did can be found in our shared journal [[Knguye66 Eyoung20 Week 12/13]]. As can be seen in this summary this week your group was really good at communicating and working together to work on this project. We worked really well together this week. One thing that did not work as well was trying to work on the excel spreadsheet online when others were working on it, there was a large lag on the sheet. I think for the next steps I might do the work on a saved downloaded version of the excel sheet and then copy my work to the shared excel. [[User:Eyoung20|Eyoung20]] ([[User talk:Eyoung20|talk]]) 16:36, 25 November 2019 (PST)
*** Iliana Crespin: For this week, it was a better organization on completing the assignment. There is more of a pattern of what should be done throughout each day to prepare for the deadline. What didn't work out for me is still trying to see how a QA can incorporate any work, since my teammates are great. For the next few weeks, I will try to continue double checking the work and contribute more. This contribution can include working with the data analysts and seeing how I can help out a bit more. In addition, texting teammates a bit more to remind any little things that must be due before the deadline. [[User:Icrespin|Icrespin]] ([[User talk:Icrespin|talk]]) 14:48, 21 November 2019 (PST)

===Week 15===
'''Milestone 4'''
#Begin wrapping up with running analysis on the created database using GRNmap
#*Data Analysts: Create GRNmap using individual database from MS Access.
#*Coder: Communicate with the QA to ensure the database is correct and if any changes need to be made
#*QA: Review the database and communicate to the Coder if any changes need to be made.
#Each team member should reflect on the team's progress
#*What worked? What didn't work? What will I do next to fix what didn't work?
#**Michael Armas
#**Kaitlyn Nguyen
#**Emma Young
#**Iliana Crespin

== Data and Files ==
=== Week 11 ===
* Presentation file: [[Media: Marmas_FunGals_Presentation.pptx|Group Presentation]]
=== Week 12/13 ===
*[[media: Thiuram_yeast_experiment.xlsx|Excel Workbook]]
*[[meda: FunGals_genelist.zip|Genelist from STEM]]
*[[media: FunGals_GOlist.zip|GOlist from STEM]]
===Week 15===
*[[Media:Marmas_finalProject_Expression-and-Degradation-rate-database_120319.zip|FunGals Expression Database]]
*[[Media:GRN_Model_Red_from_Matlab.zip|GRN Model Red Profile]]

== Milestones ==
{| class="wikitable"
|-
! Tasks to be completed
! Due Date
|-
| Write down Methods and Results for Week 12/13
*Coder/designer: find out the column names and edit them, sample to data relationship table
*Data Analysis: ANOVA analysis and set-up, STEM set-up (will run STEM on Thursday)
*QA: make sure all data in original file actually made it into the database, let coder and analysts know if there are issues
| Thursday, 12:00am, 11/19/19
|-
| Team Journal Assignment, continue writing down Methods and Results for Week 12/13
*Data Analysis: finish editing missing information for ANOVA and (STRAIN) worksheet, run STEM
*Coder and QA: add standard names to genes, make sure all data in original file actually made it into the database, let coder and analysts know if there are issues
| Tuesday, 12:00am, 11/26/19
|-
| Creating the database and running queries, working with all guilds to assure project is in line to be completed
*Data Analysts: Run YEASTRACT and modify GO Terms.
*Coder: Create individual database for the team.
*QA:Design Expression Tables used to create the final individual database.
| Thursday, 12:00am, 11/28/19
|-
|Begin wrapping up with running analysis on the created database using GRNmap
*Data Analysts: Create GRNmap using individual database from MS Access.
*Coder: Communicate with the QA to ensure the database is correct and if any changes need to be made
*QA: Review the database and communicate to the Coder if any changes need to be made.
| Tuesday, 12:00am, 12/03/19
|-
| Wrap up data analysis and individual milestones and begin to gather deliverables for turn-in
*Data Analysts: Provide feedback on the database and its ease-of-use. Work with QA to gather deliverables.
*Coder: Work with the entire guild to properly combine all databases for the entire class to use.
*QA: Work with the coder to finalize the [https://www.quackit.com/microsoft_access/microsoft_access_2016/howto/how_to_create_a_database_diagram_in_access_2016.cfm database schema diagram]
| Thursday, 12:00am, 12/05/19
|-
| Final Presentation
| Tuesday, 12:00am, 12/10/19
|-
| Report submitted
| Friday, 4:00 pm, 12/13/19
|-
|}

==Acknowledgements==
This section is in acknowledgement to partner [[User:Marmas|Michael Armas]], [[User:Icrespin|Iliana Crespin]], [[User:eyoung20|Emma Young]], and [[User:knguye66|Kaitlyn Nguyen]]. I would also like to acknowledge [[User:Kdahlquist|Dr. Dahlquist]] for introducing and teaching the topic and direction of this assignment.

"Except for what is noted above, this individual journal entry was completed by me and not copied from another source."
 
[[User:Marmas|Marmas]] ([[User talk:Marmas|talk]]) 23:40, 13 November 2019 (PST)
[[User:Icrespin|Icrespin]] ([[User talk:Icrespin|talk]]) 23:46, 13 November 2019 (PST)

"Except for what is noted above, this individual journal entry was completed by me and not copied from another source."
[[User:Eyoung20|Eyoung20]] ([[User talk:Eyoung20|talk]]) 23:57, 13 November 2019 (PST) [[User:Knguye66|Knguye66]] ([[User talk:Knguye66|talk]]) 18:45, 20 November 2019 (PST)

{{FunGals}}

==References==
===Week 11===
*Kitagawa, E., Takahashi, J., Momose, Y., & Iwahashi, H. (2002). Effects of the pesticide thiuram: genome-wide screening of indicator genes by yeast DNA microarray. Environmental science & technology, 36(18), 3908-3915. DOI: 10.1021/es015705v
*Dahlquist, K. (2019, November 7). Week 11. In Wikipedia, Biological Databases. https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_1https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_11
===Week 12/13===
*Dahlquist, K. (2019, November 19). Data Analysis. In Wikipedia, Biological Databases. Retrieved 6:25, November 20, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Data_Analysis
*Dahlquist, K. (2019, November 20). Final Project Deliverables. In Wikipedia, Biological Databases. Retrieved 6:25, November 20, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_12/13https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Final_Project_Deliverables
*Dahlquist, K. (2019, November 19). Week 12/13. In Wikipedia, Biological Databases. Retrieved 6:25, November 20, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_12/13
*Dahlquist, K. (2019, October 17). Week 8. In Wikipedia, Biological Databases. Retrieved 6:30, October 21, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_8
===Week 15===

File:GRN Model Red from Matlab.zip

2019-12-06T00:24:43Z

Eyoung20: Eyoung20 uploaded a new version of File:GRN Model Red from Matlab.zip

Knguye66 Eyoung20 Week 15

2019-12-06T00:23:25Z

Eyoung20: /* Data and files */ added file

'''Combined Individual Journals for Kaitlyn Nguyen and Emma Young (Data Analysts).'''

== Purpose ==
The purpose of this assignment is to record our progress towards the [[FunGals]] group [[Final_Project_Deliverables | deliverables]] as the [[Data_Analysis | Data Analysts]] for this week and the future weeks to come. The purpose of week 12 specifically was to download and adapt the data to the formatting we need for analysis. Then to begin the analysis with ANOVA and preparations for STEM.

== Methods and Results: Progress ==
===== - Progress 11/26/19 - =====

==== Analyzing and Interpreting STEM Results ====
#Why did you select this profile? In other words, why was it interesting to you?
#* We collectively combined the 4 red profiles together because they had similar trends and to increase the number of genes for analysis (called "Red".) Similarly this was done to the 3 green profiles as well (upward trends), with the addition of the blue profile #29 added. We will call this group "Green".
#How many genes belong to this profile?
#* Red (composed of Profile #9,26,34,11): 289
#* Green (Profile #40,42,18,29): 214
#How many genes were expected to belong to this profile?
#* Red (composed of Profile #9,26,34,11): 51.5
#* Green (Profile #40,42,18,29): 48.5
#What is the p value for the enrichment of genes in this profile?
#* Due to combining the profiles, we do not have p-values for the enrichment of genes in the 2 different groups.

- '''Viewing and Saving STEM Results''' -
# A new window will open called "All STEM Profiles (1)". Each box corresponds to a model expression profile. Colored profiles have a statistically significant number of genes assigned; they are arranged in order from most to least significant p value. Profiles with the same color belong to the same cluster of profiles. The number in each box is simply an ID number for the profile.
#* Click on the button that says "Interface Options...". At the bottom of the Interface Options window that appears below where it says "X-axis scale should be:", click on the radio button that says "Based on real time". Then close the Interface Options window.
#*Take a screenshot of this window (on a PC, simultaneously press the <code>Alt</code> and <code>PrintScreen</code> buttons to save the view in the active window to the clipboard) and paste it into a PowerPoint presentation to save your figures.
# Click on each of the SIGNIFICANT profiles (the colored ones) to open a window showing a more detailed plot containing all of the genes in that profile.
#* Take a screenshot of each of the individual profile windows and save the images in your PowerPoint presentation.
#* At the bottom of each profile window, there are two yellow buttons "Profile Gene Table" and "Profile GO Table". For each of the profiles, click on the "Profile Gene Table" button to see the list of genes belonging to the profile. In the window that appears, click on the "Save Table" button and save the file to your desktop. Make your filename descriptive of the contents, e.g. "wt_profile#_genelist.txt", where you replace the number symbol with the actual profile number.
#** Upload these files to the wiki and link to them on your individual journal page. (Note that it will be easier to [[Week_4#Compressing_and_Decompressing_Files_with_7-Zip | zip all the files together]] and upload them as one file).
#* For each of the significant profiles, click on the "Profile GO Table" to see the list of Gene Ontology terms belonging to the profile. In the window that appears, click on the "Save Table" button and save the file to your desktop. Make your filename descriptive of the contents, e.g. "wt_profile#_GOlist.txt", where you use "wt", "dGLN3", etc. to indicate the dataset and where you replace the number symbol with the actual profile number. At this point you have saved all of the primary data from the STEM software and it's time to interpret the results!
#** Upload these files to the wiki and link to them on your individual journal page. (Note that it will be easier to [[Week_4#Compressing_and_Decompressing_Files_with_7-Zip | zip all the files together]] and upload them as one file).

==== Clustering the data with STEM, as did on [[Week 9]]. ====
# Note that we will make some adjustments to the GO term analysis because stem was not providing GO term names. We are going to use the GO enrichment tool at GeneOntology.org instead.
#* Go to [http://geneontology.org/ http://geneontology.org/].
#* For the cluster you want to analyze, open the gene list and copy the list of genes.
#* Paste the list of genes into the "Go Enrichment Analysis" box on the right hand side of the GeneOntology.org page.
#* Select "Saccharomyces cerevisiae" from the species drop-down menu.
# Click the "Launch" buton.
#* Near the bottom of the results page, click on the button to Export "Table".
#* This will prompt you to save a .txt file that can be opened in Excel to view your results.
# Use YEASTRACT to generate a candidate gene regulatory network as you did on [[Week 9]].

==== Using YEASTRACT to Infer which Transcription Factors Regulate a Cluster of Genes ====

In the previous analysis using STEM, we found a number of gene expression profiles (aka clusters) which grouped genes based on similarity of gene expression changes over time. The implication is that these genes share the same expression pattern because they are regulated by the same (or the same set) of transcription factors. We will explore this using the YEASTRACT database.

# Open the gene list in Excel for the one of the significant profiles from your stem analysis. Choose a cluster with a clear cold shock/recovery up/down or down/up pattern. You should also choose one of the largest clusters.
#* Copy the list of gene IDs onto your clipboard.
# Launch a web browser and go to the [http://www.yeastract.com/ YEASTRACT database].
#* On the left panel of the window, click on the link to [http://www.yeastract.com/formrankbytf.php ''Rank by TF''].
#* Paste your list of genes from your cluster into the box labeled ''ORFs/Genes''.
#* Check the box for ''Check for all TFs''.
#* Accept the defaults for the Regulations Filter (Documented, DNA binding plus expression evidence)
#* Do '''''not''''' apply a filter for "Filter Documented Regulations by environmental condition".
#* Rank genes by TF using: The % of genes in the list and in YEASTRACT regulated by each TF.
#* Click the ''Search'' button.
# Answer the following questions:
#* In the results window that appears, the p values colored green are considered "significant", the ones colored yellow are considered "borderline significant" and the ones colored pink are considered "not significant". '''''How many transcription factors are green or "significant"?'''''
#**Green: 31
#**Red: 30
#* '''''Copy the table of results from the web page and paste it into a new Excel workbook to preserve the results.'''''
#** '''''Upload the Excel file to the wiki and link to it in your electronic lab notebook.'''''
#** '''''Are CIN5, GLN3, and/or HAP4 on the list? If so, what is their "% in user set", "% in YEASTRACT", and "p value".'''''
#**Green
#*** CIN5
#**** 0.2991 % in user set
#****0.0294
#**** p-value: 0.5492
#***GLN3
#**** 0.3645% in user set
#****0.0324% in YEASTRACT
#**** p-value: 0.18046
#***HAP4
#**** 0.2383% in user set
#****0.0467% in YEASTRACT
#**** P-value:0.00031337
#**Red
#*** CIN5
#**** 0.2111%% in user set
#**** 0.028% in YEASTRACT
#**** p-value: 0.9998
#***GLN3
#**** 0.4152% in user set
#****0.0498% in YEASTRACT
#**** p-value:0.00207752
#***HAP4
#****0.2353% in user set
#****0.0623% in YEASTRACT
#**** P-value:0.000006235
# For the mathematical model that we will build, we need to define a ''gene regulatory network'' of transcription factors that regulate other transcription factors. We can use YEASTRACT to assist us with creating the network. We want to generate a network with approximately 15-20 transcription factors in it.
#* You need to select from this list of "significant" transcription factors, which ones you will use to run the model. You will use these transcription factors and add GLN3, HAP4, and CIN5 if they are not in your list. Explain in your electronic notebook how you decided on which transcription factors to include. Record the list and your justification in your electronic lab notebook. Each group member will select a different network (they can have some overlapping transcription factors, but some should also be different).
#* Go back to the YEASTRACT database and follow the link to ''[http://www.yeastract.com/formregmatrix.php Generate Regulation Matrix]''.
#* Copy and paste the list of transcription factors you identified (plus HAP4, GLN3, and CIN5) into both the "Transcription factors" field and the "Target ORF/Genes" field.
#* We are going to use the "Regulations Filter" options of "Documented", "'''Only''' DNA binding evidence"
#** Click the "Generate" button.
#** In the results window that appears, click on the link to the "Regulation matrix (Semicolon Separated Values (CSV) file)" that appears and save it to your Desktop. Rename this file with a meaningful name so that you can distinguish it from the other files you will generate.

==== Visualizing Your Gene Regulatory Networks with GRNsight ====

We will analyze the regulatory matrix files you generated above in Microsoft Excel and visualize them using GRNsight to determine which one will be appropriate to pursue further in the modeling.
# First we need to properly format the output files from YEASTRACT.
#* Open the file in Excel. It will not open properly in Excel because a semicolon was used as the column delimiter instead of a comma. To fix this, Select the entire Column A. Then go to the "Data" tab and select "Text to columns". In the Wizard that appears, select "Delimited" and click "Next". In the next window, select "Semicolon", and click "Next". In the next window, leave the data format at "General", and click "Finish". This should now look like a table with the names of the transcription factors across the top and down the first column and all of the zeros and ones distributed throughout the rows and columns. This is called an "adjacency matrix." If there is a "1" in the cell, that means there is a connection between the trancription factor in that row with that column.
#* Save this file in Microsoft Excel workbook format (.xlsx).

#* For this adjacency matrix to be usable in GRNmap (the modeling software) and GRNsight (the visualization software), we need to transpose the matrix. Insert a new worksheet into your Excel file and name it "network". Go back to the previous sheet and select the entire matrix and copy it. Go to you new worksheet and click on the A1 cell in the upper left. Select "Paste special" from the "Home" tab. In the window that appears, check the box for "Transpose". This will paste your data with the columns transposed to rows and vice versa. This is necessary because we want the transcription factors that are the "regulatORS" across the top and the "regulatEES" along the side.
#* The labels for the genes in the columns and rows need to match. Thus, delete the "p" from each of the gene names in the columns. Adjust the case of the labels to make them all upper case.
#* In cell A1, copy and paste the text "rows genes affected/cols genes controlling".
#* Finally, for ease of working with the adjacency matrix in Excel, we want to alphabatize the gene labels both across the top and side.
#** Select the area of the entire adjacency matrix.
#** Click the Data tab and click the custom sort button.
#** Sort Column A alphabetically, being sure to exclude the header row.
#** Now sort row 1 from left to right, excluding cell A1. In the Custom Sort window, click on the options button and select sort left to right, excluding column 1.
#* Name the worksheet containing your organized adjacency matrix "network" and Save.
# Now we will visualize what these gene regulatory networks look like with the GRNsight software.
#* Go to the [http://dondi.github.io/GRNsight/ GRNsight] home page.
#* Select the menu item File > Open and select the regulation matrix .xlsx file that has the "network" worksheet in it that you formatted above. If the file has been formatted properly, GRNsight should automatically create a graph of your network. You can click the "Grid Layout" button to arrange the nodes in a grid, or you can click and drag the nodes (genes) around until you get a layout that you like and take a screenshot of the results. Paste it into your PowerPoint presentation.
#** If you have nodes (genes) floating around in the display that are not connected to any other nodes, we need to delete them from the network for the modeling to work properly. Go back to the Excel workbook and network sheet and delete both the row and column with the floating gene's name. Then re-upload the edited file to GRNsight to visualize it. Use this final version in your PowerPoint and subsequent modeling.
#** The deleted genes (floating) on GRNsight were:
#*** Green: MSN and BAS1

===== - Progress 12/03/19 - =====

==== production_rates sheet ====

* This sheet contains initial guesses for the production rate parameters, ''P'', for all genes in the network.
* Assuming that the system is in steady state with the relative expression of all genes equal to 1, (P/2) - lambda = 0, where lambda is the degradation rate, is a reasonable initial guess.
* The sheet should contain two columns (from left to right) entitled, "id", "production_rate".
** The id is an identifier that the user will use to identify a particular gene. In our case, we are using the "StandardName", for example, GLN3.
** The "production_rate" column should then contain the initial guesses for the ''P'' parameter as described above, rounded to four decimal places.
*** The production rates are provided in a Microsoft Access database, which you can [https://github.com/kdahlquist/DahlquistLab/raw/master/data/Spring2019/Expression-and-Degradation-rate-database_2019.accdb download from here.]
*** You will perform a query to get the list of production rates for each gene as a group.
*** To perform the query, you will need to follow these steps.
***# Import a your list of genes to a new table in the database. Click on the "External Data" tab and select the Excel icon with the "up" arrow on it.
***# Click the "Browse" button and select your Excel file containing your network that you used to upload to GRNsight.
***# Make sure the button next to "Import the source data into a new table in the current database" and click "OK".
***# In the next window, select the "network" worksheet, if it hasn't already been automatically selected for you. Click "Next".
***# In the next window, make sure the "First Row Contains Column Headings" is checked. Click "Next".
***# In the next window, the left-most column will be highlighted. Change the "Field Name" to "id" if it doesn't say that already. Click "Next".
***# In the next window, select the button for "Choose my own primary key." and choose the "id" field from the drop down next to it. Click "Next".
***# In the next field, make sure it says "Import to Table: network". Click Finish.
***# In the next window you do not need to save the import steps, so just click "Close".
***# A table called "network" should appear in the list of tables at the left of the window.
***# Go to the "Create" tab. Click on the icon for "Query Design".
***# In the window that appears, click on the "network" table and click "Add". Click on the "production_rates" table and click "Add". Click "Close".
***# The two tables should appear in the main part of the window. We need to tell Access which fields in the two tables correspond to each other. Click on the word "id" in the network table and drag your mouse to the "standard_name" field in the "production_rates" table, and release. You will see a line appear between those two words.
***# Right-click on the line between those words and select "Join Properties" from the menu that appears. Select Option "2: Include ALL records from 'network' and only those records from 'production_rates' where the joined fields are equal." Click "OK".
***# Click on the "id" word in the "network" table and drag it to the bottom of the screen to the first column next to the word "Field" and release.
***# Click on the "production_rate" field in the "production_rates" table and drag it to the bottom of the screen to the second column next to the word "Field" and release.
***# Right-click anywhere in the gray area near the two tables. In the menu that appears, select "Query Type > Make Table Query...".
***# In the window that appears, name your table "production_rates_1" because you can't have two tables with the same name in the database. Make sure that "Current Database" is selected and Click "OK".
***# Go to the "Query Tools: Menus" tab. Click on the exclamation point icon. A window will appear that tells you how many rows you are pasting into a new table. Click "Yes".
***# Your new "production_rates_1" table will appear in the list at the left. Double-click on that table name to open it.
***# You can copy the data in this table and paste it back into your Excel workbook. Make sure that when you paste that you use "Paste Special > Paste values" so that the Access formatting doesn't get carried along. You can also choose to export this table to Excel going to the "External Data" tab and selecting the Excel icon with the arrow pointing to the right. Select the workbook you want to export the table to, making sure that "Preserve Access formatting" is '''not''' checked. Click "OK", click "Close".
* If there are missing values, substitute the value <code>0.1980</code> for the missing production rates.
* Note that the genes should be listed in the same order in all the sheets in the Excel workbook.

==== degradation_rates sheet ====

* This sheet contains degradation rates for all genes in the network, which are provided by the user.
* Currently, the Dahlquist Lab is using data based on published mRNA half-life data from [http://rnajournal.cshlp.org/content/20/10/1645.full Neymotin et al. (2006)].
** We converted the half-life data values to the degradation rates by taking the natural log of the half-life and dividing by 2.
* The sheet should contain two columns (from left to right) entitled "id", and "degradation_rate".
** The id is an identifier that the user will use to identify a particular gene.
** The "degradation_rate" column should then contain the absolute value of the degradation rate for the corresponding gene as described above, rounded to four decimal places.
*** To obtain these values, you will use the same file, [https://github.com/kdahlquist/DahlquistLab/raw/master/data/Spring2019/Expression-and-Degradation-rate-database_2019.accdb Microsoft Access database] that you used to obtain the production rates in the first worksheet. Again, you can copy and paste the values one-by-one or you can follow the instructions to execute a query, substituting the appropriate "degradation_rates" table in the query. Note that you don't need to re-import your "network" table, you just need to create and execute the query.
* Again note, the genes should be listed in the same order in all the sheets in the Excel workbook.
* If there are missing values, substitute the value <code>0.0990</code> for the missing degradation rates.

==== Expression Data Sheets for Individual Yeast Strains ====

* Expression data can be provided for either a single strain or multiple strains of yeast (for example, the wild type strain and a transcription factor deletion strain).
** Each strain will have its own sheet in the workbook.
** Each sheet should be given a unique name that follows the convention "STRAIN_log2_expression", where the word "STRAIN" is replaced by the strain designation, which will appear in the optimization_diagnostics sheet.
*** Everyone in the class will have at least one expression worksheet called "wt_log2_expression".
*** You should have included the transcription factors GLN3, HAP4, and CIN5 in your network. Thus, we will use the expression data from the dGLN3, dHAP4, dCIN5 deletion strains in our workbooks as well, naming the worksheets "dgln3_log2_expression", "dhap4_log2_expression", and "dcin5_expression".
**** If, for some reason, you don't have all three of those genes in your network, only include expression data for the wild type and the genes out of those three that you have in your network.
* The sheet should have the following columns in this order:
*# "id": list of all genes. The genes should be listed in the same order in all the sheets in the Excel workbook.
*# The next series of columns should contain the expression data for each gene at a given timepoint given as log2 ratios (log2 fold changes). The column header should be the time at which the data were collected, without any units. For example, the 15 minute timepoint would have a column header "15" and the 30 minute timepoint would have the column header "30". GRNmap supports replicate data for each of the timepoints. Replicate data for the same timepoint should be in columns immediately next to each other and have the same column headers. For example, three replicates of the 15 minute timepoint would have "15", "15", "15" as the column headers.
*# If data are provided for multiple strains, each strain should have data for the same timepoints, although the number of replicates can vary.
* Include the data for the 15, 30, and 60 minute timepoints, but not the 90 or 120 minute timepoints.
* The data you will be using is contained in the [https://github.com/kdahlquist/DahlquistLab/raw/master/data/Spring2019/Expression-and-Degradation-rate-database_2019.accdb Expression-and-Degradation-rate-database_2019.accdb] file that you used to obtain the production and degradation rates.
* It is tedious to copy and paste all of these data by hand, so we will execute a query in Microsoft Access to do it for you. Follow the steps listed for the "production_rates" sheet for each strains expression data. After you import the data into Excel, you will need to change the column headers to "15", "15", etc., as described above.
* Missing values in the expression data sheets are OK; you don't need to put any values there like you did for the production_rates or degradation_rates sheets.

==== network sheet ====

* The network you derived from the YEASTRACT database for the [[Week 9]] assignment can be copied and pasted into this sheet directly. You may need to edit the contents of cell A1, but the rest should be good to go (especially since you previewed it in GRNsight). The description below just explains what is already in this worksheet.
** This sheet contains an adjacency matrix representation of the gene regulatory network.
** The columns correspond to the transcription factors and the rows correspond to the target genes controlled by those transcription factors.
** A “1” means there is an edge connecting them and a “0” means that there is no edge connecting them.
** The upper-left cell (A1) should contain the text “cols regulators/rows targets”. This text is there as a reminder of the direction of the regulatory relationships specified by the adjacency matrix.
** The rest of row 1 should contain the names of the transcription factors that are controlling the other genes in the network, one transcription factor name per column.
** The rest of column A should contain the names of the target genes that are being controlled by the transcription factors heading each of the columns in the matrix, one target gene name per row.
** The transcription factor names should correspond to the "id" in the other sheets in the workbook. They should be capitalized the same way and occur in the same order along the top and side of the matrix. The matrix needs to be symmetric, i.e., the same transcription factors should appear along the top and left side of the matrix. The genes should be listed in the same order in all the sheets in the Excel workbook.
** Each cell in the matrix should then contain a zero (0) if there is no regulatory relationship between those two transcription factors, or a one (1) if there is a regulatory relationship between them. Again, the columns correspond to the transcription factors and the rows correspond to the target genes controlled by those transcription factors.

==== network_weights sheet ====

* These are the initial guesses for the estimation of the weight parameters, ''w''.
* Since these weights are initial guesses which will be optimized by GRNmap, the content of this sheet can be identical to the "network" sheet.

==== optimization_parameters sheet ====

* The optimization_parameters sheet should have two columns (from left to right) entitled, "optimization_parameter" and "value".
* You should copy this worksheet from the [https://github.com/kdahlquist/DahlquistLab/raw/master/data/Spring2019/15-genes_28-edges_sample_Sigmoid_estimation_2019.xlsx sample workbook] provided. The only row that you need to modify is row 15, "Strain". Include just the strain designations for which you have a corresponding STRAIN_log2_expression sheet. If you don't have the dgln3, dhap4, or dcin5 expression sheets, then you will delete those from this row. If you do so, make sure that you don't leave any gaps between cells.
* What follows below is an explanation of what the optimization_parameters mean.
** alpha: Penalty term weighting (from the L-curve analysis)
** kk_max: Number of times to re-run the optimization loop. In some cases re-starting the optimization loop can improve performance of the estimation.
** MaxIter: Number of times MATLAB iterates through the optimization scheme. If this is set too low, MATLAB will stop before the parameters are optimized.
** TolFun: How different two least squares evaluations should be before the program determines that it is not making any improvement
** MaxFunEval: maximum number of times the program will evaluate the least squares cost
** TolX: How close successive least squares cost evaluations should be before the program determines that it is not making any improvement.
** production_function: = Sigmoid (case-insensitive) if sigmoidal model, =MM (case-insensitive) if Michaelis-Menten model
** L_curve: =0 if an L-curve analysis should NOT be run or =1 if an L-curve analysis SHOULD be run. The L-curve analysis will automatically run sequential rounds of estimation for an array of fixed alpha values (0.8, 0.5, 0.2, 0.1,0.08, 0.05,0.02,0.01, 0.008, 0.005, 0.002, 0.001, 0.0008, 0.0005, 0.0002, and 0.0001). GRNmap makes a copy of the user's selected input workbook and changes alpha to the first alpha in the list. The estimation runs and the resulting parameter values are used as the initial guesses for the next round of estimation with the next alpha value. This process repeats until all alpha values have been run. New input and output workbooks are generated for each alpha value, although currently, the graphs are only saved for the last run.
** estimate_params =1 if want to estimate parameters and =0 if the user wants to do just one forward run
** make_graphs =1 to output graphs; =0 to not output graphs
** fix_P =1 if the user does not want to estimate the production rate, ''P'', parameter, just use the initial guess and never change; =0 to estimate
** fix_b =1 if the user does not want to estimate the ''b'' parameter, just use the initial guess and never change; =0 to estimate
** expression_timepoints: A row containing a list of the time points when the data was collected experimentally. Should correspond to the timepoint column headers in the STRAIN_log2_expression sheets.
** Strain: A row containing a list of all of the strains for which there is expression data in the workbook. Should correspond to the "STRAIN" portion of the names of the STRAIN_log2_expression sheets for each strain. Note that GRNmap will run the model for the wild type network (all genes present in the network) and for networks where the gene deleted from the designated STRAIN has been deleted from the network.
** simulation_timepoints: A row containing a list of the time points at which to evaluate the differential equations to generate the simulated data. This does not need to correspond to the actual measurement times, but should be in the same units (e.g. minutes).

==== threshold_b sheet ====

* These are the initial guesses for the estimation of the ''threshold_b'' parameters.
* There should be two columns.
** The left-most column should contain the header "id" and list the standard names for the genes in the model in the same order as in the other sheets.
** The second column should have the header "threshold_b" and should contain the initial guesses, we are going to use all 0.

== Conclusion ==
The first stage of our group's project was completed via referencing [[Week 8]] and using Microsoft Excel to complete the tasks. The excel file will be located in the [[FunGals]] page for viewing and download. We were able to successfully complete the ANOVA analysis and correct found mistakes. We were able to complete a sanity check with results showing. For the sanity check the unadjusted p- values showed 37.2% of the genes had a p<0.05, 18.16% had a p<0.01, 5.04% have p<0.001, and 0.87% have p<0.0001. The sanity check for the Bonferroni-corrected p-value 0.11% of the genes have p<0.05. For the sanity check on the Benjamini and Hochberg-corrected p-value 16.36% go the genes had a p <0.05. Finally we were able to prepare the Data to run STEM in the next step of working on this project.

Extra Notes (will delete later): PDR3 did not show up in Access while running Query's

==Data and files==
[[media:Thiuram_yeast_experiment.xlsx|ANOVA & STEM workbook (.xlsx)]]

[[media:Genelist_combined_profiles_FunGals.xlsx| Combined Genelist and GOlist (.xlsx)]]

[[media:FunGals_Green_Profile_RegulationMatrix_Documented_2019123_2322_1272399515.xlsx| Green Regulation Matrix/network (.xlsx)]]

[[media:FunGals Red Profile RegulationMatrix Documented 2019123 2318 1127808084.xlsx| Red network (.xlsx)]]

[[media:FunGals Green Profile RegulationMatrix Input Workbook.xlsx | Input Workbook for Green profile (.xlsx)]]

[[media:File:GRNmap FunGals Green profile.zip | Output Workbook for Green profile (.zip)]]

[[Media:GRN_Model_Red_from_Matlab.zip]]

== Acknowledgements ==
This section is in acknowledgement to partner Kaitlyn Nguyen (User:knguye66), Michael Armas (User:Marmas), as well as, Iliana Crespin (User:Icrespin), and Emma Young (User:eyoung20). We would also like to acknowledge Dr. Dahlquist (User:KDahlquist) for introducing and teaching the topic and direction of this assignment.
Also to acknowledge that this is a shared electronic notebook between [[user:knguye66|Kaitlyn Nguyen]] and [[user:eyoung20|Emma Young]].

"Except for what is noted above, this individual journal entry was completed by me and not copied from another source."
[[User:Knguye66|Knguye66]] ([[User talk:Knguye66|talk]]) 18:49, 20 November 2019 (PST)

"Except for what is noted above, this individual journal entry was completed by me and not copied from another source."
[[User:Eyoung20|Eyoung20]] ([[User talk:Eyoung20|talk]]) 16:40, 25 November 2019 (PST)

== References ==
*Dahlquist, K. (2019, November 19). Data Analysis. In Wikipedia, Biological Databases. Retrieved 6:25, November 20, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Data_Analysis
*Dahlquist, K. (2019, November 20). Final Project Deliverables. In Wikipedia, Biological Databases. Retrieved 2:59, December 5, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Final_Project_Deliverables
*Dahlquist, K. (2019, October 29). Week 9. In Wikipedia, Biological Databases. Retrieved 2:57, December 2, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_9
*Dahlquist, K. (2019, November 7). Week 10. In Wikipedia, Biological Databases. Retrieved 2:59, December 5, 2019, from https://xmlpipedb.cs.lmu.edu/biodb/fall2019/index.php/Week_10

{{Template:knguye66}}
{{Template:Eyoung20}}

[[Category: Group Projects]]
[[Category: FunGals]]

File:GRN Model Red from Matlab.zip

2019-12-06T00:20:31Z

Eyoung20:

File:FunGals Red Profile RegulationMatrix Documented 2019123 2318 1127808084.xlsx

2019-12-04T01:15:53Z

Eyoung20: Eyoung20 uploaded a new version of File:FunGals Red Profile RegulationMatrix Documented 2019123 2318 1127808084.xlsx

File:FunGals Green Profile RegulationMatrix Documented 2019123 2322 1272399515.xlsx

2019-12-04T01:12:38Z

Eyoung20: Eyoung20 uploaded a new version of File:FunGals Green Profile RegulationMatrix Documented 2019123 2322 1272399515.xlsx

Knguye66 Eyoung20 Week 15

2019-12-03T23:44:40Z