...
Initially we sign to put everything query parameter in the request body, in order to make the URL string more readable for users, but soon we realize that GET request doesn’t contain request body, therefore we have two options: to put query in query string, or to use POST request for the usage of request body, but to keep request format in standard, we have finally choose the query string as our selection.
Accessing Swagger Documentation
In local machine: http://localhost:4000/api-docs
In deployed public URL: https://news-website-seng3011.herokuapp.com/api-docs
...
API usage
Return all matching reports that match the parameters given
| Code Block |
|---|
GET/report/?country=''&city=''&start_date=''&end_date=''&key_term=''&key_term='' |
...
terms='' |
Parameters:
country - the country to find reports for
city - the city to find reports for
start_date - the starting date and time to search for
end_date - the ending date and time to search for
key_terms - comma-separated list of key terms to search for e.g. disease names, symptoms. Can be left empty if the user wants to get all reports
If a report cannot be extracted, a message saying how many reports could not be extracted is provided, as well as a list of each article url and publish date of the article.
Example: getting all reports that mention “Outbreak” or are about “Hantavirus“ from Canberra, Australia between 2011-04-19T11:48:00 and 2022-03-16T09:38:00
Request
| Code Block |
|---|
curl -X 'GET' \
'http://localhost:4000/report?start_date=2011-04-19T11%3A48%3A00&end_date=2022-03-16T09%3A38%3A00&city=Canberra&country=AU&key_terms=Outbreak%2CHantavirus' \
-H 'accept: appplication/json' |
Response
| Code Block | ||
|---|---|---|
| ||
[
{
"url": "http://www.upi.com/Top_News/World-News/2013/05/15/Salmonella-poisoning-outbreak-in-Australia/UPI-30421368619035/ ",
"headline": "Salmonella poisoning outbreak in Australia",
"main_text": "DICKSON, Australia, May 15 (UPI) -- At least 100 people have been diagnosed with salmonella poisoning after eating at a new restaurant in Australia, health officials said.",
"reports": []
},
{
"url": "https://www.foodsafetynews.com/2021/11/salmonella-top-cause-of-foodborne-outbreaks-in-australia/ ",
"headline": "",
"main_text": "Salmonella dominated reported outbreaks in Australia in 2016 causing several large incidents, according to a study published recently.",
"reports": []
},
{
"url": "https://www.foodsafetynews.com/2021/07/salmonella-dominates-outbreaks-in-australia/ ",
"headline": "",
"main_text": "More than 450 foodborne outbreaks were reported over a three-year period in Australia, according to a new study.",
"reports": []
},
"1 article(s) could not be scraped from: [{\"url\":\"http://www.healthcanal.com/infections/36035-Deadly-virus-discovered-bats-also-jumps-species.html \"}]"
] |
User can query a single report by providing
| Code Block |
|---|
GET/report/?{:reportId} |
...
User (frontend) related Actions
Each user will be login to our web APP by username and password. After login, user have ability to save/like the report as they wish. The saved report will be written in the database and can be Unsave as they wish. The potential usesage of this will be
...
Return all user saved report by
...
username
Request:
| Code Block | ||
|---|---|---|
| ||
GET/report/:userId/ |
Save report to corresponding user, return success report saved
| Code Block |
|---|
POST/report/:userId/:reportId |
...
user/:username/ |
Response:
| Code Block |
|---|
[
{
"url": "http://www.upi.com/Top_News/World-News/2013/05/15/Salmonella-poisoning-outbreak-in-Australia/UPI-30421368619035/ ",
"headline": "Salmonella poisoning outbreak in Australia",
"main_text": "DICKSON, Australia, May 15 (UPI) -- At least 100 people have been diagnosed with salmonella poisoning after eating at a new restaurant in Australia, health officials said.",
"reports": []
},
{
"url": "https://www.foodsafetynews.com/2021/11/salmonella-top-cause-of-foodborne-outbreaks-in-australia/ ",
"headline": "",
"main_text": "Salmonella dominated reported outbreaks in Australia in 2016 causing several large incidents, according to a study published recently.",
"reports": []
},
{
"url": "https://www.foodsafetynews.com/2021/07/salmonella-dominates-outbreaks-in-australia/ ",
"headline": "",
"main_text": "More than 450 foodborne outbreaks were reported over a three-year period in Australia, according to a new study.",
"reports": []
},
"1 article(s) could not be scraped from: [{\"url\":\"http://www.healthcanal.com/infections/36035-Deadly-virus-discovered-bats-also-jumps-species.html \"}]"
] |
User can Save report to corresponding usernameand reportId return success saved report.
Request:
| Code Block |
|---|
POST/report/user/:username
Requset Body: {
Reports:[
{
"url": "http://www.upi.com/Top_News/World-News/2013/05/15/Salmonella-poisoning-outbreak-in-Australia/UPI-30421368619035/ ",
"headline": "Salmonella poisoning outbreak in Australia",
"main_text": "DICKSON, Australia, May 15 (UPI) -- At least 100 people have been diagnosed with salmonella poisoning after eating at a new restaurant in Australia, health officials said.",
"reports": []
},
]
} |
Response:
| Code Block |
|---|
{
"url": "http://www.upi.com/Top_News/World-News/2013/05/15/Salmonella-poisoning-outbreak-in-Australia/UPI-30421368619035/ ",
"headline": "Salmonella poisoning outbreak in Australia",
"main_text": "DICKSON, Australia, May 15 (UPI) -- At least 100 people have been diagnosed with salmonella poisoning after eating at a new restaurant in Australia, health officials said.",
"reports": []
}, |
User can Unsave/Remove report by usernameand reportId, return success deleted report
Request:
| Code Block | ||
|---|---|---|
| ||
DELETE/api/report/:userId/report/user/:username/:reportId |
Response:
| Code Block |
|---|
{
"url": "http://www.upi.com/Top_News/World-News/2013/05/15/Salmonella-poisoning-outbreak-in-Australia/UPI-30421368619035/ ",
"headline": "Salmonella poisoning outbreak in Australia",
"main_text": "DICKSON, Australia, May 15 (UPI) -- At least 100 people have been diagnosed with salmonella poisoning after eating at a new restaurant in Australia, health officials said.",
"reports": []
}, |
Sample disease report
| Code Block | ||
|---|---|---|
| ||
report = { id: 'sample_id', url: 'http "url": "https://outbreakswww.globalincidentmap.com/xxx'who.int/csr/don/17-january-2020-novel-coronavirus-japan-ex-china/en/", "date_of_publication": "20222020-0201-2317 16xx:07xx:00xx", country "headline": "United KingdomNovel Coronavirus – Japan (ex-China)", city: "London", eventType: "Salmonella Outbreak (Suspected or Confirmed)", location: { latitude: 51.507, longitude: -0.128 } } |
The database design
...
"main_text": "On 15 January 2020, the Ministry of Health, Labour and Welfare, Japan (MHLW) reported an imported case of laboratory-confirmed 2019-novel coronavirus (2019-nCoV) from Wuhan, Hubei Province, China. The case-patient is male, between the age of 30-39 years, living in Japan. The case-patient travelled to Wuhan, China in late December and developed fever on 3 January 2020 while staying in Wuhan. He did not visit the Huanan Seafood Wholesale Market or any other live animal markets in Wuhan. He has indicated that he was in close contact with a person with pneumonia. On 6 January, he traveled back to Japan and tested negative for influenza when he visited a local clinic on the same day.",
"reports": [
{
"event_date": "2020-01-03 xx:xx:xx to 2020-01-15",
"locations": [
{
"country": "China",
"location": "Wuhan, Hubei Province"
},
{
"country": "Japan",
"location": ""
}
],
"diseases": [
"2019-nCoV"
],
"syndromes": [
"Fever of unknown Origin"
]
}
]
} |
We design a light weight database with 4 fields, the country and city are storing for fast enumerate purpose in query suggestions.
...
Postman: API testing application
We chose Postman over other tools such as Insomnia to test our API as a few of our members had experience using it
Postman also provides the ability to collaborate as a team to write, store and run test cases
Specific Libraries:
Web scraping - Puppeteer
Puppeteer: an NPM library that provides a high-level API to control Chrome or Chromium over the DevTools Protocol
...
Library - PuppeteerJS, Morgan(logger), Winston(Logger), Nodemon, CORS (TODO)
...