-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* document schema in main README * document libcal * document nss
- Loading branch information
1 parent
1eb9d5a
commit 6da8491
Showing
5 changed files
with
264 additions
and
190 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,52 @@ | ||
# Freerooms Scrapers | ||
|
||
This repository contains the scrapers for the underlying buildings, rooms and bookings data used by Freerooms. | ||
|
||
For documentation on the internals of each scraper, please see the READMEs in the respective directories for each scraper. | ||
|
||
For instructions on how you can access this data, see the [DevSoc GraphQL API](https://github.com/devsoc-unsw/graphql-api). | ||
|
||
## Schema | ||
|
||
### Buildings | ||
|
||
| **Field** | **Description** | **Example** | | ||
|-----------|----------------------------------------------|-----------------| | ||
| `id` | Building ID in the format `CAMPUS-GRID_REF`. | "K-F8" | | ||
| `name` | Name of the building. | "Law Building" | | ||
| `lat` | Latitude of the building. | -33.91700 | | ||
| `long` | Longitude of the building. | 151.227791 | | ||
| `aliases` | List of alternative names for the building. | ["Law Library"] | | ||
|
||
### Rooms | ||
|
||
| **Field** | **Description** | **Example** | | ||
|--------------|---------------------------------------------------|---------------------| | ||
| `id` | Room ID in the format `CAMPUS-GRID_REF-ROOM_NUM`. | "K-J17-305" | | ||
| `name` | Name of the room. | "Brass Lab J17 305" | | ||
| `abbr` | Shortened name, as seen on timetable. | "BrassME305" | | ||
| `usage` | Room type - see below for list. | "CMLB" | | ||
| `capacity` | Number of people the room is suitable for. | 36 | | ||
| `school` | School that manages the room - `" "` if none. | "CSE" | | ||
| `buildingId` | ID of building that room is in. | "K-J17" | | ||
|
||
Mapping of room usages can be found [here](https://github.com/devsoc-unsw/freerooms/blob/dev/common/roomUsages.ts). Mapping of school codes can be found [here](https://github.com/devsoc-unsw/freerooms/blob/dev/common/schools.ts). | ||
|
||
### Bookings | ||
|
||
| **Field** | **Description** | **Example** | | ||
|---------------|------------------------------------------------------|-----------------------------| | ||
| `bookingType` | Type of booking - see below. | "SOCIETY" | | ||
| `name` | Name of the booking (usually related to the booker). | "SOFTWAREDEV" | | ||
| `roomId` | ID of the room the booking is for. | "K-E19-G05" | | ||
| `start` | Start time of the booking. | "2024-01-27T04:00:00+00:00" | | ||
| `end` | End time of the booking. | "2024-01-27T08:00:00+00:00" | | ||
|
||
Full list of current booking types is: "CLASS", "SOCIETY", "INTERNAL", "LIB", "BLOCK", "MISC". | ||
|
||
### Relationships | ||
The following relationships exist between tables. These relationships are tracked by Hasura and can be followed in GraphQL queries. | ||
- Every **building** contains 1 or more **rooms** | ||
- Every **room** belongs to a **building** | ||
- Every **room** has 0 or more **bookings** | ||
- Every **booking** is for a specific **room** |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,11 @@ | ||
# libcal-scraper | ||
Scraper for https://unswlibrary-bookings.libcal.com/ | ||
Scraper for https://unswlibrary-bookings.libcal.com/. This scrapes data on study rooms in the Main Library and Law Library, as well as the bookings for these rooms. | ||
|
||
## Data Sources | ||
|
||
The booking data is found on this page: | ||
https://unswlibrary-bookings.libcal.com/r/new/availability?lid=6581&zone=0&gid=0&capacity=0 | ||
|
||
However, the data is not scraped from the HTML, but instead fetched by mimicking the API call made by this page to https://unswlibrary-bookings.libcal.com/spaces/availability/grid. This returns a list of "slots" with item (room) IDs and start/end times, some of which are booked. | ||
|
||
The list of item IDs from the above JSON is then used to scrape the data for each room from the individual room pages (e.g. https://unswlibrary-bookings.libcal.com/space/46481). This is scraped from the HTML using [cheerio](https://cheerio.js.org/). |
Oops, something went wrong.