Describe how you intend to develop the API module and provide the ability to run it in Web service mode (Zifan Wei )

Discuss your current thinking about how parameters can be passed to your module and how results are collected. Show an example of a possible interaction. (e.g.- sample HTTP calls with URL and parameters) (Tingzhuang Zhou )

Present and justify implementation language, development and deployment environment (e.g. Linux, Windows) and specific libraries that you plan to use. (Mashira Farid ) (Lin Thit Myat Hsu Library)

• Frontend: ReactJS, MUI, *Rendering(CSR - CreateReactApp, SSR- NextJS),

• Backend: NodeJs, ExpressJS

• Database: MongoDB

• Libraries: PuppeteerJS, Morgan(logger), Winston(Logger)

• Deployment: AWS *ES2,*ESC, *Lambda, *Heroku, *CI/CD(?)

Implementation Languages:

Our team decided to use full-stack JavaScript for our project. We mainly chose this as:

• We would only need to use one programming language throughout our app, allowing us to work efficiently and consistently

• Many online resources are available for it, so the few members of our team who were unfamiliar with JavaScript would be able to learn it quickly

• JavaScript is the best language for web apps like ours

Backend - NodeJS, ExpressJS

• NodeJS: an open-source, cross-platform runtime environment for JavaScript built on Google’s V8 engine

  • We chose to use NodeJS as it is very simple to use, easy to learn, and provides great performance

  • Since NodeJS is open-source and cross-platform, everyone in the team could develop the app regardless of the operating system they used (Windows, Mac)

  • By using NodeJS, we can also use NPM (node package manager), which allows us to install numerous libraries to use in our code to help in development. ExpressJS is an example of one of these libraries which is installed using NPM

• ExpressJS: a backend web application framework thats used to design and build web apps quickly and easily

  • We decided to use ExpressJS as it allows us to create RESTful routes easily. This is particularly useful as we will be developing our own API as part of the project, which we will need to create custom routes for

  • ExpressJS also supports JSON, which we plan on using for our API to return results

Frontend - ReactJS

• ReactJS: an open-source frontend JavaScript library

  • We chose ReactJS for the frontend because it supports the use of components, which allows us to reuse existing code instead of having to rewrite similar features, which in turn allows for faster development

  • React also improves performance by only updating components in the DOM whose state has changed, instead of updating every component again, making our app lightweight and flexible

  • React also has plenty of documentation and online resources, so the members of our team who were unfamiliar with it would be able to pick it up quickly

Library - PuppeteerJS, Morgan(logger), Winston(Logger)

• PuppeteerJS: an open-source NodeJS library

  • We use PuppeteerJS for scraping the website as the library allows the backend to run a headless (a code-only) version of Chrome to load the websites on.

  • A headless version of Chrome doesn’t use as much resources and as such can be easily run in any server, such as the hosting servers that we are using.

  • PuppeteerJS also have high-level API control of the Headless Chrome browser via the DevTools Protocol. This allows for the code to read the structure code of the website directly and to access the required parts of the website.

Development and Deployment Environment:

Development: Windows/Linux/Mac? Cross-platform, so doesn't matter

Deployment: Heroku or AWS?

Testing: Postman?

Development

Deployment - Heroku (later switch to AWS if time permits)

• Heroku:

Database - MongoDB

• MongoDB: a cross-platform, NoSQL database

  • We chose MongoDB as our database as our app isn’t very big, so we didn't need to use SQL

  • Additionally, MongoDB integrates well with NodeJS, and stores data in JSON format, which matches the format our API will output in

Testing

Specific Libraries:

Material UI for frontend UI

Puppeteer for scraping

Morgan, Winstor for logging?

Web scraping - Puppeteer

• Puppeteer: an NPM library which provides a high-level API to control Chrome or Chromium over the DevTools Protocol

  • We chose Puppeteer as it can be used to scrape the data from our data source, which we need to do as part of our API

UI design - Material UI

• Material UI (MUI): an open-source frontend framework for React components based on Google’s Material Design

  • We decided to use MUI as it would make the development of our app quicker and easier. Since MUI defines its own components, we can use them to quickly design the UI of our app, instead of spending time writing our own CSS classes

  • By using MUI, we can keep our frontend UI theme consistent, and even customise it to suit our needs

  • The extensive documentation also means it will be easy to pick up, as well as debug