Welcome
Welcome! You must be baffled by the project, not having any clear idea as you just joined. Dread not! We'll walk you through the onboarding.
Hopefully after spending some time reading sections below, you'll have a more clear idea.
By all means, please feel free to to ask on Slack general channel!
Understanding the structure of 2 workflows and 5 teams
The project is divided into 2 workflows: online and offline
The members are divided into 5 teams: frontend-mobile, frontend-dashboard, backend-mobile, backend-dashboard, and infra
The table below gives a roughly good idea to understand the relations between 2 workflows and 5 teams
| Workflow | User | Interface (frontend) | API (backend) | database |
|---|---|---|---|---|
| Mobile (Online) | e.g. Parents uploading data/photo | app (iOS, Android) | APIs | Database named "BioHPC" |
| (by frontend-mobile team) | (by backend-mobile team) | |||
| Dashboard (Offline) | e.g. Scientists view data | web (React) | APIs | Database named "BioHPC" |
| (by frontend-dashboard team) | (by backend-dashboard team) |
You might notice infra team is missing in the table. Infra team builds up the environments (servers/GitHub/certs/domains/etc) to support other teams.
In one word, infra team underlies all other teams.
Workflows and envs
Now having a rough idea about each team's role, let's understand how their code is interacting with each other (workflows) and 3 envs.
To understand the workflows and envs, from the link below, read the the last diagram in the bottom
Team Specific Onboarding
There is a separate robust set of docs on the confluence for onboarding to your specific team, and you should go through the appropriate team docs after reading this document. The Infrastructure documentation is especially robust and this documentation will be critical for members joining the infrastructure team. Members of other teams may also find the infrastructure docs helpful for building a more in-depth understanding of how their work fits in with the work of other teams. Infrastructure team members should also become familiar with the documentation for all other teams, as you will be more effective in your role after building a high level understanding of each team's work.
Confidential info: Login secrets, SSL certs, key files, etc
There are login secrets (email/username/password) to log in the websites that we use (GitHub, Gmail, Freenom, etc)
There are also SSL certs, key files, etc that we need for encryption/authorisation purposes.
- If you are working on infra team, you will need access to all of them (see the box link inside the doc link below)
- If you are working on backend teams (mobile & dashboard) or dashboard frontend team, you will need access to
SSL certsandAWS EC2 key files(see the box link inside the doc link below) - If you are working on mobile frontend team, sorry we haven't got much to say about this: Please refer to Jennifer's doc or ask her if she's available.
For convenience, we are using Cornell subscription version of box.com. This is of course not the most ideal method, nor the most secure way. It is convenient though. Feel free to come up with a better solution if you feel so.
❌ Confidential files (deprecated)
Codebases (GitHub)
You can find a list of all codebases on GitHub for each team at this link.
How do I write/debug code?
For backend (mobile & dashboard) developers
After cloning your codebase from GitHub, now it's time to write/debug code.
Probably you won't yet have every required library installed, but worry not! We use docker container to standardise everything (versions/libraries).
In one word, you'll find it more convenient and elegant to write/debug code in docker container env (instead of on your laptop's local env).
Read the doc below to know how.
For frontend (dashboard) developer
Clone the codebase from GitHub as well.
However, since already using npm, you will not need docker - it is not that useful for you.
You are ready to develop right away after cloning from GitHub.
For frontend (mobile) developer
Please refer to the mobile frontend onboarding page.
Where are these 3 envs? (dev, test, prod)
The previous section "Workflows" shows 3 envs (dev, test, prod) in the diagram. Where are they?
After reading the previous docs "How do I write/debug code?", you will be able to write/debug code locally - that is your dev env! (dev env is on everyone's laptop).
Test env and prod env are NOT on your laptop, but rather shared by all members. Where are they and how can I access them? Here's the answer.
Deploy your code
For backend (mobile & dashboard)
Once you have implemented new features locally and pushed your code to the GitHub repo, you can deploy your changes on the test env to see if they integrate well with other components of the project. After making sure that everything works well, it's time to deploy the new feature on the prod env and have it accessible to all our users!
Read the doc below for instructions on how to deploy.
❌ Deploy Services on AWS (Deprecated)
For frontend (dashboard)
For this semester, we do not set up a test server, but only local/dev environment (laptop) and production server.
To run the code in the development server, which uses the environment variable in the .env.development file, run npm start;
To run the code in the production server, which uses the environment variable in the .env.production file, run npx serve -s build;
To know how to deploy React web app onto AWS server, read this doc below
Monitor and send report emails
Preview: This part is rather cool! We are utilising serverless model (i.e. AWS Lambda) to build this part. Hope you find this interesting!
How do we monitor everything? How can we know if our frontend website (sorry, we haven't done so for frontend mobile), backend mobile APIs, and backend dashboard APIs are working properly? Can we make sure they are configured correctly and connected to the correct APIs/databases?
The answer is: We've built a monitoring and reporting system for this!
Read the link (and its sub-pages) below to know more details about rules of monitoring and sending reports/alerts.
Another more subtle question is: If sending reports/alerts to project members, should we send to multiple recipient email addresses? This is obviously labourious to set up.
How to keep it lazy and simple? Let's use uni's e-list!
Do we have to wait for emails to know the monitoring result? Absolutely not, we have a website for active checking.
Sometimes, to make it worse, the uni's email will falsely filter out our daily emails. Worry not! You can still see everything via the website below!
We do have built a system status page that you can actively go and check.
Access databases
During your development, you might want to connect to our databases to view/modify the data. For data security reasons, databases are host differently for different environments. Our development and test databases are hosted on MongoDB Atlas and our production database is hosted on BioHPC.
Connection strings to the databases and set-up instructions are given in the doc below.
BioHPC and MongoDB Atlas Set-up
Send emails (The centralised API to send emails)
If you need to send emails (e.g. sending password-reset emails when users request so) in your code, please refer to Send Emails in Your Code (AWS Lambda) and use the approach discussed there.
Access AWS
Normally, if you are not in the infra team, you won't need to access the AWS account that we've been using. If your want to access it due to your specific needs, please reach out to Liz.
Resource configuration details
As you have seen in the sections above, this project utilizes many cloud resources (such as AWS, BioHPC, etc.). Currently they are working well, but it's still important to know how to configure them in case the current configuration no longer fits our needs. This doc Configuration Notes contains some important configuration details about our resources. You don't have to read it during your onboarding, just keep in mind that it contains useful information that you can refer to when you want to configure cloud resources.