Design Details

1. Describe how you intend to develop the API module and provide the ability to run it in Web service mode (@Zifan Wei)

RESTful API can be developed on our local machines. Github would be used as a source control tool in our project, The reason for choosing Github as our source control tool because it is a common tool used in industry and it also provides all basic functions, we need to manage our project.

Different children branches are required for each team member to make sure the project is easy-manageable. After specific features are done by team members, they can push to their own branch and wait for other members’ code check. Finally, merging to the main branch by pulling request. It is an agile methodology for a team project.

And our data source is from http://outbreaks.globalincidentmap.com/ , we would use Puppeteer to scrape data we want. And store that JSON format data in MongoDB database. The request to database would be made when clients are calling API. And web server then sends back responses to clients. We would develop RESTful API to let clients get information they need. There are four HTTP methods we need to design for web service: GET, POST, PUT and DELETE.

Heroku is the cloud platform we will use to host API for application deployment, because it is easier to deploy small-size API for one project, Heroku also meets low computational demands and much more customer-oriented. So that’s why we use Heroku.

Open

Above shows how our application work in development, the deployment strategy have not been doen by use therefore we will add deployment architecture later.

2. Discuss your current thinking about how parameters can be passed to your module and how results are collected. Show an example of a possible interaction. (e.g.- sample HTTP calls with URL and parameters) (@Tingzhuang Zhou )

Initially we sign to put everything query parameter in the request body, in order to make the URL string more readable for users, but soon we realize that GET request doesn’t contain request body, therefore we have two options: to put query in query string, or to use POST request for the usage of request body, but to keep request format in standard, we have finally choose the query string as our selection.

Accessing Swagger Documentation

In local machine: http://localhost:4000/api-docs

In deployed public URL: https://news-website-seng3011.herokuapp.com/api-docs

API usage

Return all reports that match the parameters given

Parameters:

• country - the country to find reports for

• city - the city to find reports for

• start_date - the starting date and time to search for

• end_date - the ending date and time to search for

• key_terms - comma-separated list of key terms to search for e.g. disease names, symptoms. Can be left empty if the user wants to get all reports

If a report cannot be extracted, a message saying how many reports could not be extracted is provided, as well as a list of each article url and publish date of the article.

Example: getting all reports that mention “Outbreak” or are about “Hantavirus“ from Canberra, Australia between 2011-04-19T11:48:00 and 2022-03-16T09:38:00

Request

Response

User can query a single report by providing

User (frontend) related Actions

Each user will be login to our web APP by username and password. After login, user have ability to save/like the report as they wish. The saved report will be written in the database and can be Unsave as they wish. The potential usesage of this will be

Open

Return all user saved report by username

Request:

Response:

User can Save report to corresponding usernameand reportId return success saved report.

Request:

Response:

User can Unsave/Remove report by usernameand reportId, return success deleted report

Request:

Response:

Sample disease report

We design a light weight database with 4 fields, the country and city are storing for fast enumerate purpose in query suggestions.

• possible interactions other than get request

3. Present and justify implementation language, development and deployment environment (e.g. Linux, Windows) and specific libraries that you plan to use. (@Mashira Farid ) (@Lin Thit Myat Hsu Library)

Implementation Languages:

Our team decided to use full-stack JavaScript for our project. We mainly chose this as:

• We would only need to use one programming language throughout our app, allowing us to work efficiently and consistently

• Many online resources are available for it, so the few members of our team who were unfamiliar with JavaScript would be able to learn it quickly

• JavaScript is the best language for web apps like ours

Backend - NodeJS, ExpressJS

• NodeJS: an open-source, cross-platform runtime environment for JavaScript built on Google’s V8 engine

  • We chose to use NodeJS as it is very simple to use and easy to learn.

  • Since Node is event-driven, it can also enable asynchronous programming, which provides great performance compared to Python

  • Since NodeJS is open-source and cross-platform, everyone in the team could develop the app regardless of their development environment

  • By using NodeJS, we can also use NPM (node package manager), which allows us to install numerous libraries to use in our code to help in development. ExpressJS is an example of one of these libraries which is installed using NPM

• ExpressJS: a backend web application framework that's used to design and build web apps quickly and easily

  • We decided to use Express as it allows us to create RESTful routes easily. This is particularly useful as we will be developing our own API as part of the project, which we will need to create custom routes for

  • Since Express is an very popular NPM library, it works well with Node

  • Express also supports JSON, which we plan on using for our API to return results

Frontend - ReactJS

• ReactJS: an open-source frontend JavaScript library

  • We chose ReactJS for the frontend because it supports the use of components, which allows us to reuse existing code instead of having to rewrite similar features, which in turn allows for faster development

  • React is also better in terms of performance compared to other frameworks like Angular, as it only updates components in the DOM whose state has changed, instead of updating every component again, making our app lightweight and flexible

  • React also has plenty of documentation and online resources, so the members of our team who were unfamiliar with it would be able to pick it up quickly

Development and Deployment Environment:

Development - Cross-Platform

• Since everyone in our team is using different operating systems on our personal machines, we ensured that all the technologies we chose as part of our tech stack worked across different platforms

• This allows everyone in our team to work on our app in the development environment they were most comfortable with and had access to, which in turn leads to more efficient development of our app, compared to setting a specific development environment that not everyone may have access to

Deployment - Heroku (later switch to AWS if time permits)

• Heroku: a cloud platform as a service for app deployment

  • We chose to deploy our API using Heroku as it only requires a few steps in order for deployment. This will make it quicker and easier to deploy our API compared to other platforms such as AWS

  • Additionally, since our API is small, it doesn’t need all the functions that AWS provides, making it much simpler for us to deploy it using Heroku

  • However, if time permits, we plan on deploying our API using AWS

Database - MongoDB

• MongoDB: a cross-platform, NoSQL database

  • Initial database options are: MongoDB, PostgreSQL, SQLite

  • We chose MongoDB as our database as our app isn’t very big, so we didn't need to use SQL

  • MongoDB is more scable as it will initial a new data fields whenever we create the documents in database, hence suitable for agile iteration when adding new feature later.

  • Additionally, MongoDB integrates well with NodeJS, and stores data in JSON format, which matches the format our API will output in

Testing - Postman

• Postman: API testing application

  • We chose Postman over other tools such as Insomnia to test our API as a few of our members had experience using it

  • Postman also provides the ability to collaborate as a team to write, store and run test cases

Specific Libraries:

Library - PuppeteerJS, Morgan(logger), Winston(Logger), Nodemon, CORS (TODO)

• PuppeteerJS: an open-source NodeJS library

  • We use PuppeteerJS for scraping the website as the library allows the backend to run a headless (a code-only) version of Chrome to load the websites on.

  • A headless version of Chrome doesn’t use as much resources and as such can be easily run in any server, such as the hosting servers that we are using.

  • PuppeteerJS also have high-level API control of the Headless Chrome browser via the DevTools Protocol. This allows for the code to read the structure code of the website directly and to access the required parts of the website.

• Morgan: HTTP request logger middleware for Node.js

• Winston: Another logger for both develop and production environment.

• CORS: NPM package for solving Cross Origin Resource Sharing problems

• dotEnv: abstracting environment variable and url configurations in .env file, making swaping of environment between development/production feasible.

UI design - Material UI

• Material UI (MUI): an open-source frontend framework for React components based on Google’s Material Design

  • We decided to use MUI as it would make the development of our app quicker and easier. Since MUI defines its own components, we can use them to quickly design the UI of our app, instead of spending time writing our own CSS classes

  • Since we had decided to use React for our frontend, and MUI uses React components, it was only logical we used MUI over other non-React frameworks such as Bootstrap

  • Additionally, some of our team members already had some experience using it. The extensive documentation also means it will be quick and easy for the team members who haven’t used it before to pick up, as well as debug

  • By using MUI, we can keep our frontend UI theme consistent, and even customise it to suit our needs