Content Comparison

1. Describe how you intend to develop the API module and provide the ability to run it in Web service mode (Zifan Wei )

RESTful API can be developed on our local machines. Github would be used as a source control tool in our project, The reason for choosing Github as our source control tool because it is a common tool used in industry and it also provides all basic functions, we need to manage our project.

Different children branches are required for each team member to make sure the project is easy-manageable. After specific features are done by team members, they can push to their own branch and wait for other members’ code check. Finally, merging to the main branch by pulling request. It is an agile methodology for a team project.

And our data source is from http://outbreaks.globalincidentmap.com/ , we would use Puppeteer to scrape data we want. And store that JSON format data in MongoDB database. The request to database would be made when clients are calling API. And web server then sends back responses to clients. We would develop RESTful API to let clients get information they need. There are four HTTP methods we need to design for web service: GET, POST, PUT and DELETE.

Heroku is the cloud platform we will use to host API for application deployment, because it is easier to deploy small-size API for one project, Heroku also meets low computational demands and much more customer-oriented. So that’s why we use Heroku.

2. Discuss your current thinking about how parameters can be passed to your module and how results are collected. Show an example of a possible interaction. (e.g.- sample HTTP calls with URL and parameters) (Tingzhuang Zhou )

...

• Postman: API testing application

  • We chose Postman over other tools such as Insomnia to test our API as a few of our members had experience using it

  • Postman also provides the ability to collaborate as a team to write, store and run test cases

Specific Libraries:

Web scraping - Puppeteer

• Puppeteer: an NPM library that provides a high-level API to control Chrome or Chromium over the DevTools Protocol

  • We chose Puppeteer as it can be used to scrape the data from our data source, which we need to do as part of our API

...

• PuppeteerJS: an open-source NodeJS library

  • We use PuppeteerJS for scraping the website as the library allows the backend to run a headless (a code-only) version of Chrome to load the websites on.

  • A headless version of Chrome doesn’t use as much resources and as such can be easily run in any server, such as the hosting servers that we are using.

  • PuppeteerJS also have high-level API control of the Headless Chrome browser via the DevTools Protocol. This allows for the code to read the structure code of the website directly and to access the required parts of the website.

• Morgan: HTTP request logger middleware for node.js

• Winston: A Another logger for just about everythingboth develop and production environment.

• CORS: NPM package for solving Cross Origin Resouce Sharing problems

...