example_notebook.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# UClick\n",
    "\n",
    "`UClick` is an open source software written in Python to register Control Points on images for further use in [UCalib](https://github.com/Ulises-ICM-UPC/UCalib) and [UDrone](https://github.com/Ulises-ICM-UPC/UDrone).\n",
    "\n",
    "### Description\n",
    "\n",
    "The program is aimed to assist the registration of pixel coordinates corresponding to Control Points for subsequent image calibration. Two types of control points are distinguished: points located on the horizon (Horizon Points, HP) and points located in known coordinates (Ground Control Points, GCP), for which a file with their coordinates will be supplied. The output files can then be used afterwards for manual calibration of images in [UCalib](https://github.com/Ulises-ICM-UPC/UCalib) and [UDrone](https://github.com/Ulises-ICM-UPC/UDrone). A code to verify the quality of the GCPs used in the manual calibration of the images is also provided\n",
    "\n",
    "### Requirements and project structure\n",
    "To run the software it is necessary to have Python (3.8) and install the following dependencies:\n",
    "- cv2 (4.2.0)\n",
    "- numpy (1.19.5)\n",
    "- matplotlib (3.1.2)\n",
    "\n",
    "In parenthesis we indicate the version with which the software has been tested. It is possible that it works with older versions. \n",
    "\n",
    "The structure of the project is the following:\n",
    "* `example.py`\n",
    "* `example_notebook.py`\n",
    "* **`uclick`**\n",
    "  * `uclick.py`\n",
    "  * `ulises_uclick.py`\n",
    "* **`example`**\n",
    "  * `GCPs.txt`\n",
    "  * `GCPs.jpg`\n",
    "  * **`basis`**\n",
    "    * `image000001.jpg`\n",
    "    * `image000001cdg.txt`\n",
    "    * `image000001cdh.txt`\n",
    "    * . . .\n",
    "  * **`TMP`**\n",
    "    * `image000001cdg_check.jpg`\n",
    "    * `image000001cdh_check.jpg`\n",
    "    * . . .\n",
    "\n",
    "The local modules of `UClick` are located in the **`uclick`** folder.\n",
    "\n",
    "To run a demo with the video in folder **`example`** and a set of frames in **`basis`** using a Jupyter Notebook we provide the file `example_notebook.ipynb`. For experienced users, the `example.py` file can be run in a terminal. `UClick` handles `PNG` and `JPEG` image formats.\n",
    "\n",
    "## Registration of Control Points\n",
    "\n",
    "Import modules:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import sys\n",
    "sys.path.insert(0, 'uclick')\n",
    "import matplotlib\n",
    "matplotlib.use('TkAgg')\n",
    "import uclick as uclick"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Set the main path and the path of the folder where the images are placed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "pathFolderMain = 'example'\n",
    "pathFolderBasis = pathFolderMain + os.sep + 'basis'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Horizon Points\n",
    "\n",
    "Each of the images `<image>.jpg` in the folder **`basis`** will be offered to register the HPs. In the case that for an `<image>.jpg` image located in the folder **`basis`** has already been registered, set `overwrite = True` to register it again and `False` if you want to keep it. To facilitate the verification that the HPs have been correctly registered, set parameter `verbosePlot = True` to generate an image `<image>cdh_check.jpg` on a **`TMP`** folder showing the HPs, and to `False` otherwise. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "overwrite = True\n",
    "verbosePlot = True"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Recording of pixel coordinates\n",
    "The recording of a pixel in an image corresponding to a Control Point is done in two steps. Using the left mouse button, a first click zooms the image region in which the point is located and a second click records the pixel coordinates (column and row). Press the middle button or the `Return` or `Esc` keys to end point recording of an image.\n",
    "\n",
    "Run the code to click HP:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "uclick.ClickHorizon(pathFolderBasis, overwrite, verbosePlot)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As a result, for each images a file `<image>cdh.txt` containing the pixel coordinates of the HPs is generated. The structure of this file is the following:\n",
    "* `<basisFrame>cdh.txt`: For each HP one line with\n",
    ">`pixel-column`, `pixel-row`\n",
    "\n",
    "### Ground Control Points\n",
    "\n",
    "The available GCPs are listed in the file `GCPs.txt`. The location of these points is shown in the reference image `GCPs.jpg`.The structure of this file the following:\n",
    "* `GCPs.txt`: For each GCP one line with\n",
    ">`code`, `x-coordinate`, `y-coordinate`, `z-coordinate`, `switch`\n",
    "\n",
    "Quantities must be separated by at least one blank space between them and the last record should not be continued with a newline (return).\n",
    "\n",
    "For each of the images `<image>.jpg` in the folder **`basis`**, the GCPs with `switch = on` will be offered to be registered. In the case that an `<image>.jpg` has already been registered, set `overwrite = True` to register it again and `False` if you want to keep the  points that have already been registered. To verificate the GCPs have been registered, set parameter `verbosePlot = True` to generate an image `<image>cdg_check.jpg` on a **`TMP`**, and to `False` otherwise. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "overwrite = True\n",
    "verbosePlot = True"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Recording of pixel coordinates\n",
    "The recording of a pixel in an image follows the procedure explained above. Now, the middle button or the `Return` or `Esc` keys are used to skip a point. If you do so, its pixel-coordinates will be recorded with values `-999`.\n",
    "\n",
    "Run the code to click GCP:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "uclick.ClickGCPs(pathFolderMain, pathFolderBasis, overwrite, verbosePlot)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As a result, for each images a file `<image>cdg.txt` containing the coordinates of the GCPs is generated. The structure of this file is the following:\n",
    "* `<image>cdg.txt`: For each GCP one line with\n",
    ">`pixel-column`, `pixel-row`, `x-coordinate`, `y-coordinate`, `z-coordinate`, `code`\n",
    "\n",
    "Real-world coordinates are given in the same coordinate system as the GCPs.\n",
    "\n",
    "New GCPs can be added to `<image>cdg.txt` by changing the value of `switch = off` to `on` in the file `GCPs.txt` and then re-running the code to click GCP. In case that `switch = on` changes to `off` the corresponding GCP will be deleted in `<image>cdg.txt`.\n",
    "\n",
    "As the points that were excluded during the registration process are saved with coordinates `-999`, if you want to register them again in mode `overwrite = False`, you will have to delete them manually from the file.\n",
    "\n",
    "## GCP check\n",
    "\n",
    "To verify the quality of the GCPs used in the manual calibration of the images, a RANSAC (RANdom SAmple Consensus) is performed. Points of the files `<image>cdg.txt` located at the **`basis`** folder will be tested. The calibration of the points (minimum 6) is done assuming a _parabolic_ camera model and requires maximum reprojection pixel error `eCritical` of the GCPs. Set `eCritical` and run the RANSAC algorithm:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "eCritical = 5.\n",
    "uclick.CheckGCPs(pathFolderBasis, eCritical)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For each file `<image>cdg.txt`, the GCPs that should be revised or excluded will be reported.\n",
    "\n",
    "## Contact us\n",
    "\n",
    "Are you experiencing problems? Do you want to give us a comment? Do you need to get in touch with us? Please contact us!\n",
    "\n",
    "To do so, we ask you to use the [Issues section](https://github.com/Ulises-ICM-UPC/UClick/issues) instead of emailing us.\n",
    "\n",
    "## Contributions\n",
    "\n",
    "Contributions to this project are welcome. To do a clean pull request, please follow these [guidelines](https://github.com/MarcDiethelm/contributing/blob/master/README.md)\n",
    "\n",
    "## License\n",
    "\n",
    "UClick is released under a [AGPL-3.0 license](https://github.com/Ulises-ICM-UPC/UClick/blob/master/LICENSE). If you use UClick in an academic work, please cite:\n",
    "\n",
    "    @Online{ulisesclick, \n",
    "      author = {Simarro, Gonzalo and Calvete, Daniel},\n",
    "      title = {UClick},\n",
    "      year = 2021,\n",
    "      url = {https://github.com/Ulises-ICM-UPC/UClick}\n",
    "      }"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}