Sample Application
The front-end of this application consists of a dashboard that presents data via counters, graphs, and lists, along with displaying images and relevant information about them. The project can be accessed by: Front-end.
The front-end side of this application was made using the following technologies:
-
Vue.js
- Vue.js is a JavaScript-based framework that has as main features: a light learning curve, good documentation and good integration capabilities
-
Vuetify
- It is a semantic User Interface (UI) framework focused on gathering resources to facilitate and optimize the development of graphical components through the design system Material Design
-
ApexCharts
- It is a library for building interactive graphics for Web solutions
-
Axios
- Axios is a library based on communication over HTTP protocol through promises. It works on both the back-end and the front-end using the same code base, using HTTP module with Node.js, while on the front-end it uses XMLHttpRequests, i.e. it is isomorphic.
Figure 1: Homepage screen
The home page consists of 6 cards, a side menu and a button. The cards are distributed among categories: count, list and chart. Thus, the application contains 3 count cards, 2 chart cards and 1 list card.
Detailing each of the components of this screen:
-
- It aims to be the central control between the main screens of the application, so it contains 2 buttons, one that directs the user to the main screen and another that directs to the cameras analysis screen.
-
- This is the component responsible for changing the entire application language, giving the user the option to choose between Portuguese and English.
-
- It displays the data referring to the number of people with objects that have been detected as hostile by the model
-
- The data regarding the number of persons with firearms detected by the template is displayed
-
- The data regarding the number of persons wearing helmets detected by the template is displayed
-
- This component returns a list containing the readings of people, weapons and helmets in a monthly format over the 12 months of the year.
-
- This component composes a line graph containing a series of data regarding the number of people detected per hour
-
- This component composes a line graph containing three sets of data regarding temperature (in Celsius), pressure (in mmHg) and humidity (in g/m^3) of the environment where the camera is located
Figure 2: Camera exhibition screen
This page is responsible for displaying the cameras registered by the user, for this, there is a carousel with the registered cameras, a controller to move between cameras, a side scroll to fully display the list of registered cameras, and a button that redirects the user to the camera details page indicated in the controller.
Detailing each of the components of this screen:
-
- The camera carousel presents 3 different states that are based on the presence of cameras in the database and images saved in the cameras
- When there are cameras registered and that camera contains images, the application displays the last image present in the highlighted camera. As shown in the main image of this section.
- The camera carousel presents 3 different states that are based on the presence of cameras in the database and images saved in the cameras
-
No cameras registered
- When there are no cameras registered, the application displays a message indicating that there are no cameras registered by the user.
Figure 3: Camera carousel component when there is no camera registered
-
No images registered in the camera
- When there is a registered camera, but that camera has no images added, the application displays a message indicating that there is no image saved by the camera.
Figure 4: Camera carousel component when there are no images registered in the camera
-
- The purpose of this button is to control the movement between the registered cameras so that the user can have control over which camera they want to access the details.
-
- The functionality of this slider is to scroll through a list of all registered cameras and has the objective of showing the collection of cameras to the user without having to check each one of them.
-
- It gives access to a camera's details page, i.e. it redirects the user to a screen containing data about the camera's images, heatmap and counts
Figure 5: Camera detail screen
This screen focuses on displaying the images captured by the camera, as well as heat reading and counting the entities (Crowds, people, guns and helmets) in each image. To be able to display this, the screen contains 3 components distributed between carousel, image and data card.
-
- This component presents all the images captured by the camera and arranges them with the help of a controller located just below it. Each time this controller is used to move the images forward or backwards, the heat readout and the data are updated to respect the highlighted image. In addition, the highlighted image carries boxes and labels to highlight entities detected by the model with their respective colours and confidence value. This component has two states, when there are images and when there are not.
-
Image carousel with registered images
- This state is what is happening in the highlighted image in this section.
-
Image Carousel with no images registered
- When there are no images registered, consequently, there are no detections registered, a component displays a message notifying you that this has happened.
Figure 6: Image carousel component when there are no images registered
-
- The heat reading is related to the cluster analysis of each image. Like any heat map, colours closer to red indicate higher crowding, while colours closer to the blue spectrum indicate low crowding. This component also has two states, when there is a heatmap and when there is not.
-
Image display with heat mapping registered
- This state is what is happening in the highlighted image in this section
-
Display image with no heat map registered
- When there is no heat reading associated with the highlighted image by the carousel, a message is displayed indicating that no reading was found for that image
Figure 7: Heatmap image component when there is no heatmap associated with an image
-
- This card presents different information regarding the featured image in the carousel
-
Featured image URL:
- Basically, it displays a URL that directs to the image in a separate browser tab
-
Count of people in the image:
- Displays an estimate of the number of crowds present in the image, and there is a metric to check what a crowd is that can be checked by the heatmap
-
Number of people detected:
- Displays the value for the number of people detected in the image
-
Number of guns detected:
- Displays the value referring to the number of weapons detected in the image
-
Number of helmets detected:
- Displays the value referring to the number of helmets detected in the image
The back-end of this application seeks to receive the data from the object detection models, store them in a database and send them to the front-end in a suitable format.
To create the back-end of the example application, we looked for open source tools that facilitate development in a popular programming language. Therefore, we chose to use the web frameworks in python, Django and [Django REST](https://www.django-rest-framework.org /).
The database chosen for this application was PostgreSQL, an open-source object-relational database with easy integration with Django.
A model is the single, definitive source of information about application data, and it contains the essential fields and behavior of the data that will be stored. Thus, in this application, five models were developed, namely: Camera, Frame, Detection, Agglomeration e Measurement.
This model is the representation of the cameras of our application. It has the following attributes:
- Creation date;
- Update date;
- Name;
- URL.
This model represents a camera's frames, so each Frame object is linked to a Camera object. It has the following attributes:
- Creation date;
- Update date;
- Image data (binary);
- Image width;
- Image height;
- Related camera.
This model represents a frames' detection, so each Detection object is linked to a Frame object. It has the following attributes:
- Creation date;
- Update date;
- Detection type (person, gun, or helmet);
- X-coordinate of the bounding box;
- Y-coordinate of the bounding box;
- Bounding box width;
- Bounding box height;
- Detection confidence;
- Related frame.
This model represents the agglomeration of persons present in each frame, so each Agglomeration object is linked to a Frame object. It has the following attributes:
- Creation date;
- Update date;
- People count;
- Heatmap data (binary);
- Heatmap width;
- Heatmap height;
- Related frame.
This model is the representation of the measurements made by the hardware sensors. It has the following attributes:
- Creation date;
- Measurement type (temperature, pressure, or humidity);
- Measurement value;
- Measurement unit;
A view, in Django, is a python function that receives a web request and returns a web response. Thus, it is the hotspot from the detection model to the backend and backend to the UI. For this, seven views were developed, they were: DashboardView, CameraFramesView, CameraView, SubmissionView, CameraDetailsView, DetectionsHeatmapView e MeasurementSubmissionView.
A view that receives a GET request and returns the data needed for the application's home page.> #### URL
- /system/api/dashboard/
- num_of_persons (integer)
- Number of "person" detections that occurred on the day.
- num_of_guns (integer)
- Number of "gun" detections that occurred on the day.
- num_of_helmets (integer)
- Number of "helmet" detections that occurred on the day.
- num_of_persons_by_hour (list)
- List containing the number of "person" detections by time of day.
- data_history (dictionary)
- Dictionary containing the number of detections of the type "person", "gun" and "helmet" per month.
- temperature_each_hour (list)
- List containing the average temperature by the hour of the day.
- pression_each_hour (list)
- List containing the average pression by the hour of the day.
- humidity_each_hour (list)
- List containing the average humidity by the hour of the day.
A view that when receiving a GET request returns the ID and the URL of the cameras registered in the application, and when receiving a POST request, it enters a new camera.
- /system/api/cameras/
- name (string)
- Name of the camera to be registered.
- url (string)
- URL of the camera to be registered.
- name (string)
- Name of registered cameras.
- url (string)
- URL of registered cameras.
A view receives a GET request and returns the last frame of each registered camera.
- /system/api/cameras/frames/
- cameras (dictionary)
- Dictionary containing the camera's names (name), the respective frame data in base64 string form (src), and the frame IDs in integer form (frame_id).
A view that receives a POST request from a frame and its detections (optional), executes the cluster model for the sent frame and saves everything in the database.
- /system/api/cameras/
- camera_name (integer)
- Name of the camera from which the frame originates.
- img_data (string)
- Base64 encoded image data.
- img_width (integer)
- Image width.
- img_height (integer)
- Image height.
- detections (list) [optional]
- List with dictionaries containing the data of each frame detection.
- label (integer): detection type (1- person, 2- gun 3- helmet).
- x (integer): x coordinate of the detection bounding box origin.
- y (integer): y coordinate of the detection bounding box origin.
- w (integer): width of the detection bounding box.
- h (integer): height of the detection bounding box.
- confidence (float): confidence of the detection.
A view that receives a GET request with the camera ID, and the number of frames required (10 by default) returns the name, URL, and the last frames of the camera.
- /system/api/cameras/view/int:camID/int:frAmount/
- camera_name (string)
- Name of the camera.
- camera_url (string)
- URL of the camera.
- frames (list) [optional]
- List with dictionaries containing the data of each frame.
- id (integer): frame id number.
- src (string): base64 encoded image data.
A view that receives a GET request with the frame ID and returns the image of the frame with the detections drawn, the detection data, and the agglomeration data.
- /system/api/frames/int:frameID/detections/
- src (string)
- Base64 encoded frame image data.
- heatmap (string)
- Base64 encoded heatmap image data.
- num_of_persons (integer)
- Number of person detections.
- num_of_guns (integer)
- Number of gun detections.
- num_of_helmets (integer)
- Number of helmet detections.
- agg_count (integer)
- People count made by the agglomeration model.
A view that receives a POST request and from a sensor measurement.
- /system/api/sensors/submission/
- type (integer)
- Type of measurement the sensor captured. (0- Temperature, 1- Pressure or 2- Humidity)
- value (integer)
- Value measured by the sensor.
- unit (integer)
- Unit of measurement captured by the sensor.