Revision as of 23:37, 30 November 2017

Gene hAPI Helpful Links

Our Deliverables: Gene hAPI Deliverables

Main Page: Main Page

Project Page: GRNsight Gene Page Project

GRNsight

Project Deliverables Checklist

[ ]Organized Team deliverables wiki page (or other media (CD or flash drive) with table of contents)
[ ]Group Report (.doc, .docx or .pdf file)
[ ]Individual statements of work, assessments, reflections (wiki page, .doc, .docx, .pdf, or e-mailed to both Dr. Dahlquist and Dr. Dionisio)
[ ]Group PowerPoint presentation (given on Tuesday, December 12, .ppt, .pptx or .pdf file)
[ ]Code (GitHub pull request)
[ ]Each team should coordinate in performing a final integration and integration testing iteration (see Coder milestone for details) which the Interaction and Integration team then submits to the original
[ ]GRNsight GitHub repository as a single, unified pull request from the class project’s fork
[ ]Supply a README that summarizes the functionality of your team's new feature (.txt or .md, one README per team)
[ ]Excel spreadsheet with ANOVA results/stem formatting (.xlsx)
[ ]PowerPoint of screenshots of stem results (.pptx)
[ ]Gene List and GO List files from each significant profile (.txt compressed together in a .zip file)
[ ]YEASTRACT "rank by TF" results (.xlsx)
[ ]GRNmap input workbook (with network adjacency matrix, .xlsx)
[ ]GRNmap output workbook (.xlsx)
[ ]Electronic notebook corresponding to these the microarray results files (Week 8, Week 10, and Weeks 11-15) support reproducible research so that all manipulations of the data and files are documented so that someone else could begin with your starting file, follow the protocol, and obtain your results.

Project Manager

[ ]Milestone 1: Project “Scaffolding”

This milestone pertains to setting up an initial schedule and any resources that your team will use for the duration of the project. It will be useful to get an overview of every team member’s own milestones so that you have an accurate big picture view.

1. In consultation with your team, work backward from the final deadline to set intermediate deadlines for each deliverable. In particular you need to set deadlines for what you will accomplish by the journal deadline for Week 12, Week 14, and Week 15.
2. Organize management tools for your team:
   • Workflow narratives
   • Action items
   • Testing results/reports
     • Bugs/feature requests
     • Question/answer sequences

[ ]Milestone 2: Periodic Updates

Not as much a milestone as an on-going task, once the project is up and running the Project Manager is responsible for keeping track of everyone’s progress.

1. Get periodic updates on progress; in particular, the project’s “place” in the overall flow should be known at all times (transparency). Team members will be giving a status reports in class for the rest of the semester. However, the instructors will expect you to know and be able to report on the status of each member of your team at any time.
2. Familiarize yourselves with the specific milestones of each team member so that you know how to monitor the team’s overall progress.
3. Monitor the status of the report-in-progress and other related documentation.
4. Coordinate team decisions and action items addressing any unforeseen delays or roadblocks.

Data Analyst

[X]Milestone 1: Annotated Bibliography

• For the Week 11 assignment, the Data Analysts will work with the QAs to develop an annotated bibliography of papers that perform the global transcriptional analysis of DNA microarray data in yeast.

[X]Milestone 2: Journal Club Presentation

• For the Week 12 assignment, the Data Analysts will work with the QAs to prepare a PowerPoint presentation to be delivered in class on Tuesday, November 21.

[ ]Milestone 3: Complete Data Analysis of Dahlquist Lab Data for Visualization with GRNsight

• For Week 14 and Week 15, the Data Analysts will review and complete the microarray data analysis begun with the Week 8 and Week 10 assignments with the eventual goal of visualizing a gene regulatory network derived from their dataset in GRNsight. Specifically:
  1. Review the ANOVA results from Week 8 for accuracy, making corrections if necessary.
  2. If corrections were made to the ANOVA results, re-running stem (Week 10).
  3. Using the YEASTRACT Database to determine which transcription factors are candidates for regulating the genes in a cluster from stem (part of Week 10 that we postponed.
  4. Using the YEASTRACT Database to develop a candidate gene regulatory network (part of Week 10 that we postponed).
  5. Using the GRNmap software to model the gene regulatory network (confer with Dr. Dahlquist when you are ready for this).
  6. Visualizing the results with GRNsight.
• As the end-user of the GRNsight software, the Data Analysts will provide feedback to the QAs and Coders about the usability of the new features.

Quality Assurance

[X]Milestone 1: Annotated Bibliography

• For the Week 11 assignment, the Data Analysts will work with the QAs to develop an annotated bibliography of papers that perform the global transcriptional analysis of DNA microarray data in yeast.

[X]Milestone 2: Journal Club Presentation

• For the Week 12 assignment, the Data Analysts will work with the QAs to prepare a PowerPoint presentation to be delivered in class on Tuesday, November 21.

[ ]Milestone 3: Requirements Analysis

As the Coders begin their development work and the Data Analysts start working with their assigned microarray data sets, the QA team members should familiarize themselves with and help specify the expected correct functionalities of their respective teams.

• The entire QA guild should become an expert on the information that can be retrieved for a gene, so that they know how IDs should look, how certain data types will be displayed, etc. This will help them detect flaws and areas of improvement as development work proceeds.
• Gene Database APIs team: QA should get to know how the four web service APIs are to be used in order to retrieve gene data given a gene symbol. They should be able to perform these steps themselves using curl or a web browser, so that they can provide independent verification that the Coders’ work is functioning correctly.

[ ]Milestone 4: On-going Testing of Respective Team Deliverables

The rest of the semester is expected to be an on-going process of verifying and validating the correctness of a QA member’s assigned team. Specific concerns include but are not limited to the following:

• Gene Database APIs and JASPAR teams: Manual testing will involve some combination of curl and web browser developer tool use in order to get to know the data returned by the various web services. The Coder members should find ways to show the Quality Assurance members the work-in-progress data returned by getGeneInformation, which QA can then compare to the raw web service API calls for accuracy. After the first integration milestone, QA members can examine the behavior of the prototype gene page at both the end-user and Developer Tools levels (i.e., examining network traffic to make sure the correct requests are going out with the expected responses coming in; inspecting the gene page elements to make sure that they received the correct data).

Coders

[X]Milestone 0: Journal Club Presentation

• For the Week 11 assignment, the Coders (including Designers) of each team will prepare a PowerPoint presentation of their respective assigned software design/development/engineering/best practices reading to be delivered in class on Tuesday, November 14.

[ ]Milestone 1: Working Environment Setup

Coder work will require the following software. The Seaver 120 lab computers are already set up for this; this list is provided for Coders who need to work on a different computer or outside of the lab.

• Node.js 8.4.0 or newer
• Code-savvy editor such as Atom or Microsoft Visual Studio Code
• Web browser with developer tools (Seaver 120 uses Google Chrome)
• git version control software
• (depends on team) curl command

Make sure that this software is installed and operational before beginning. If any Coder needs help with any of these packages, please consult your fellow guild members or ask Dr. Dionisio.

[ ]Milestone 2: Version Control Setup

Coding work will be done on a fork of the open source GRNsight project, which is hosted on GitHub. The software that interacts with GitHub to perform version control is git. If any Coder needs help with git or version control concepts in general, please consult your fellow guild members or ask Dr. Dionisio.

The Interaction and Integration team is responsible for this fork, and will do their own work on the master branch of this fork. Thus, many of the early steps in this procedure involve them, and they must accomplish these steps first before the others can proceed:

1. All members of the Coder guild should acquire a GitHub account, if they don’t already have one (it’s free).
2. One of the Coders of the Interaction and Integration team creates a fork of the GRNsight project.
3. The Interaction and Integration team adds the GitHub accounts of all Coder guild members as collaborators on the fork.
4. Once every team is a collaborator on this fork, they can then create their respective branches on the fork:
   • The Page Design team creates a branch called page-design.
   • The Gene Database APIs team creates a branch called gene-database-apis.
   • After the gene-database-apis branch is created, the JASPAR API team creates a branch from gene-database-apis called jaspar-api.

The teams will then do their work on their respective branches. The protocol for integrating your work is described in the Integration and Integration Testing milestone below.

The structure described in this milestone is a typical GitHub fork-and-branch approach which some Coders are already familiar with. If any Coder needs help with this methodology, please consult your fellow guild members or ask Dr. Dionisio.

[ ]Milestone 3: “Developer Rig” Setup and Initial As-Is Build

Before development can begin in earnest, the following initial setup steps must be performed per Coder after version control setup is complete:

1. git clone the GitHub fork created in the version control setup milestone on where you plan to work on the project.
2. cd into the folder that contains the cloned repository
3. git checkout the branch that is assigned to you:
   • Interaction and Integration: master (no need for an explicit checkout because the clone defaults to this branch)
   • Page Design: page-design
   • Gene Database APIs: gene-database-apis
   • JASPAR: jasper-api (note that as instructed above, this should be a branch of the gene-database-apis branch)
4. Follow the instructions in the GRNsight wiki to perform the following:
   1. Installation of necessary third-party libraries
   2. Initial startup of the GRNsight application on your computer

If anyone runs into problems with this procedure, please consult with your fellow guild members or notify Dr. Dionisio.

[ ]Milestone 4: Development, Implementation, and Localized Testing

At this point, each team proceeds with their own development work. In general, each team will do most of their work (and place most of their files) in the following locations within the GRNsight repository. Some files may need to go elsewhere; check in with Dr. Dionisio if there are any file organization/location issues:

• Page Design: web-client/public/html (this folder does not yet exist; you will need to create it) and web-client/test (in case you are able to write unit tests for your work)
  • While running GRNsight, you will be able to access files in this folder through the URL http://localhost:5001/html/filename
• Gene Database APIs: web-client/public/js and web-client/test
• JASPAR: web-client/public/js and web-client/test
• Interaction and Integration: Because you are integrating the other teams’ work, you will likely interact with the entire web-client folder

For the Gene Database APIs and JASPAR teams, your work will not be visible to end-users until the first overall integration is done, so use unit tests as your initial approach for verifying that your code is working well. You will probably need to mock up your API calls so that you don’t bombard the web services with requests while unit testing. However, these sites are generally open access so you can probably write your initial set of tests using real requests and responses.

The Quality Assurance members of each team should take charge of manual testing. Specific suggested manual testing tasks are described in the Quality Assurance guild page.

The Interaction and Integration team should initially get to know the files being created and worked on by the other teams prior to the first integration milestone.

Teams should perform their own internal testing (both manual and automated) on their respective files throughout development. Commit and push your work to your designated branch at appropriate intervals. Coders who are not familiar with this style of working should consult with fellow guild members or talk to Dr. Dionisio.

[ ]Milestone 5: Integration and Integration Testing

Integration and integration testing should take place periodically to make sure that the overall gene page project is coming together well. Most of the Coders’ work for the remainder of the semester will be a cycle of the development and integration milestones. The Interaction and Integration team is in charge of coordinating when these integrations take place. The first integration will likely be the most difficult one to perform, but it is hoped that later iterations will proceed more smoothly.

When the other teams have reached the following sub-milestones, the first integration cycle can take place. Future iterations are at the discretion of the Interaction and Integration team in coordination with how the other teams are progressing:

• Page Design: first working version of the gene page
• Gene Database APIs: first working version of the getGeneInformation function
• JASPAR: This team’s work does not have to participate in the first integration cycle; instead, they should integrate their work with the Gene Database APIs team whenever they have a working prototype. JASPAR functionality simply “shows up” at some future integration cycle.
• Interaction and Integration: You are not blocked from working prior to the first integration; you can work on exploring the existing GRNsight code to implement the right-click functionality that will open the gene page in a new tab or window. You can start by making this code open a blank page. Upon the first integration iteration, you can then connect that page to the gene page provided by the Page Design team.

An individual integration iteration proceeds in this way:

1. The Page Design and Gene Database APIs teams issue pull requests to merge their branches with the master branch.
2. The Coders guild reviews each other’s code and points out any needed revisions.
3. Once all revision requests are fulfilled, the Interaction and Integration team merges the pull requests into master.
4. The Coders guild resolves any merge conflicts then tests the combined code.
5. The combined code is allowed to be unfinished, but it should not be broken. The combined GRNsight build should still run and show steady progress toward the fully-envisioned gene page functionality.
6. Code revisions at this stage are committed and pushed to master.
7. When the Interaction and Integration team is satisfied with the state of the combined code, the other teams can then merge master back into their respective branches, and localized development and testing can proceed to the next phase.
8. When the teams feel that they have reached another integration point, they notify the Interaction and Integration team of this and another integration iteration can take place.
9. Rinse and repeat until the project is done!