main_tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "482ae900",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": [
     "title"
    ]
   },
   "source": [
    "<img src=\"images/picto_transparent.png\" alt=\"CARS logo\" width=\"200\" align=\"right\"/>\n",
    "\n",
    "# CARS OTB Tutorial with AI\n",
    "\n",
    "This tutorial show the CARS OTB version usage through docker and adds AI MCCNN advanced tutorial through pandora configuration. \n",
    "\n",
    "CARS Team\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0fedc9b2",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Outline\n",
    "\n",
    "* Tutorial preparation\n",
    "* CARS context\n",
    "* 3D photogrammetry basics\n",
    "* CARS high level design\n",
    "* CARS quickstart\n",
    "* CARS CLI examples\n",
    "* CARS Step by step framework manipulation\n",
    "* CARS stereo matching with AI improvement\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "275d703e",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<img src=\"images/picto_transparent.png\" alt=\"CARS logo\" width=\"100\" align=\"right\"/>\n",
    "\n",
    "# Tutorial preparation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a31013ca",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Docker preparation\n",
    "\n",
    "Check Docker install: \n",
    "    \n",
    "    docker -v\n",
    "\n",
    "Get CARS tutorial Docker:\n",
    "\n",
    "    docker pull cnes/cars\n",
    "    docker pull cnes/cars-jupyter\n",
    "    docker pull cnes/cars-tutorial\n",
    "\n",
    "Show docker images:\n",
    "\n",
    "    docker images\n",
    "\n",
    "You should get **cnes/cars** , **cnes/cars-jupyter**, **cnes/cars-tutorial** docker images. \n",
    " \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2e2329f3",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Cars tutorial and jupyter dockers run\n",
    "\n",
    "Run tutorial :\n",
    "    \n",
    "    docker run -p 8000:8000 cnes/cars-tutorial\n",
    "\n",
    "Go to [http://localhost:8000](http://localhost:8000) for this tutorial.\n",
    "\n",
    "Run jupyter notebook:\n",
    "\n",
    "    docker run -p 8888:8888 cnes/cars-jupyter\n",
    "\n",
    "Go to the provided output URL and navigate into CARS tutorials notebooks. \n",
    "The URL must be of this kind with a token: [http://localhost:8888](http://localhost:8888) \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a634cbdb",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<img src=\"images/picto_transparent.png\" alt=\"CARS logo\" width=\"100\" align=\"right\"/>\n",
    "\n",
    "# CARS context"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a7af090c-9e82-49c9-bfd3-4e5c3c87f2db",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## CARS, a satellite multi view stereo pipeline\n",
    "\n",
    "CARS produces Digital Surface Models from satellite imaging by photogrammetry.\n",
    "\n",
    "Main goals: \n",
    "\n",
    "- robust and distributed tool for operational pipelines.\n",
    "- capitalizing 3D developments\n",
    "- prototyping, tests, r&d evaluation\n",
    "\n",
    "*Be aware that CARS is new and evolving to maturity with CNES roadmaps*\n",
    "\n",
    "License:  Apache-2.0\n",
    "\n",
    "<img src=\"images/picto_transparent.png\" alt=\"CARS logo\" width=\"200\" align=\"right\"/>\n",
    "\n",
    "Web sites:\n",
    "\n",
    "- [https://github.com/cnes/cars/](https://github.com/cnes/cars/)\n",
    "- [https://cars.readthedocs.io/](https://cars.readthedocs.io/)\n",
    "\n",
    "\n",
    "\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2eda6177-96c1-4919-aff5-ebefdb69f54a",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## CARS ?\n",
    "\n",
    "<table><tr>\n",
    "<td> \n",
    "  <p align=\"center\" style=\"padding: 10px\">\n",
    "    <img alt=\"Sat sensors\" src=\"images/animation_sat.gif\" width=\"300\">\n",
    "  </p> \n",
    "</td>\n",
    "<td> \n",
    "  <p align=\"center\">\n",
    "    <img alt=\"Overview elevation\" src=\"images/overview_dsm_3d.gif\" width=\"300\">\n",
    "  </p> \n",
    "</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>\n",
    "    <em>Sensors images</em>\n",
    "</td>\n",
    "<td>\n",
    "    <em>Elevation model</em>\n",
    "</td> \n",
    "</tr>\n",
    "</table>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b1212de7-f6cd-41ae-9501-0836e75fe39a",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Projects context\n",
    "\n",
    "- [CO3D project](https://co3d.cnes.fr/en/co3d-0): four small satellites in the CO3D constellation to map the whole globe in 3D\n",
    "- [AI4GEO](https://www.ai4geo.eu/) : production of automatic 3D geospatial information based on IA technologies.\n",
    "- Internal studies, internships, phd, ...\n",
    "\n",
    "\n",
    "<img src=\"images/logo_co3D_cnes.jpg\" alt=\"CO3D logo\" width=\"200\" align=\"left\"/>\n",
    "<img src=\"images/logo_ai4geo.png\" alt=\"AI4GEO logo\" width=\"200\" align=\"right\" style=\"background-color:black;\"/>\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eaf3389d-3391-4405-a29b-5c15d222676f",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## CARS main dependencies\n",
    "\n",
    "<table><tr>\n",
    "<td> \n",
    "  <p align=\"center\" style=\"padding: 10px\">\n",
    "    <img alt=\"Pandora logo\" src=\"images/logo_pandora.png\" width=\"200\">\n",
    "  </p> \n",
    "</td>\n",
    "<td> \n",
    "  <p align=\"center\">\n",
    "    <img alt=\"OTB Logo\" src=\"images/logo-orfeo-toolbox.png\" width=\"100\">\n",
    "  </p> \n",
    "</td>\n",
    "<td> \n",
    "  <p align=\"center\">\n",
    "    <img alt=\"Vlfeat\" src=\"images/logo_vlfeat.png\" width=\"200\">\n",
    "  </p> \n",
    "</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>\n",
    "    <em><a href=\"https://github.com/cnes/pandora/\">Pandora tool, dense matching tool</a></em>\n",
    "</td>\n",
    "<td>\n",
    "    <em><a href=\"https://www.orfeo-toolbox.org/\" alt=\"OTB\">Orfeo tool box, global image library</a></em>\n",
    "</td>\n",
    "<td>\n",
    "    <em><a href=\"https://www.vlfeat.org/\" alt=\"VLFeat\">VLfeat, sparse matching SIFT</a></em>\n",
    "</td> \n",
    "</tr>\n",
    "</table>\n",
    "    \n",
    "    \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c187b81-f966-47c5-aec3-04e815373758",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## CARS ecosystem\n",
    "\n",
    "<table><tr>\n",
    "<td> \n",
    "  <p align=\"center\" style=\"padding: 10px\">\n",
    "    <img alt=\"Demcompare logo\" src=\"images/demcompare_picto.png\" width=\"200\">\n",
    "  </p> \n",
    "</td>\n",
    "<td> \n",
    "  <p align=\"center\" style=\"padding: 10px\">\n",
    "    <img alt=\"Shareloc Logo\" src=\"images/shareloc_picto.png\" width=\"100\">\n",
    "  </p> \n",
    "</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>\n",
    "    <em><a href=\"https://github.com/cnes/demcompare/\" alt=\"Demcompare\">Demcompare, to compare DEM</a></em>\n",
    "</td>\n",
    "<td>\n",
    "    <em><a href=\"https://github.com/cnes/shareloc\" alt=\"Shareloc\">Shareloc, a simple geometry library</a></em>\n",
    "</td>\n",
    "</tr>\n",
    "</table>\n",
    "\n",
    "and others to come :)\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d5462fc9-b9b8-4c9f-8799-d29d4ad8a66b",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### Authors \n",
    "\n",
    "* David Youssefi <david.youssefi@cnes.fr>\n",
    "* Emmanuel Dubois <emmanuel.dubois@cnes.fr>\n",
    "* Emmanuelle Sarrazin <emmanuelle.sarrazin@cnes.fr>\n",
    "* Yoann Steux <yoann.steux@csgroup.eu>\n",
    "* Florian Douziech <florian.douziech@csgroup.eu>\n",
    "* Mathis Roux <mathis.roux@csgroup.eu>\n",
    "\n",
    "\n",
    "See [Authors.md](https://raw.githubusercontent.com/CNES/cars/master/AUTHORS.md) for full contributions in Github repository.\n",
    "\n",
    "### Copyright\n",
    "\n",
    "- CNES Copyright to ease maintenance with [Contributor License Aggrement](https://raw.githubusercontent.com/CNES/cars/master/docs/source/CLA/CCLA-CARS.doc)\n",
    "\n",
    "### Contributions\n",
    "\n",
    "See [Contributing.md](https://raw.githubusercontent.com/CNES/cars/master/CONTRIBUTING.md)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "420f0722-6acb-4bc2-987a-0264c101480f",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### Glossary\n",
    "\n",
    "DSM : Digital Surface Model\n",
    "\n",
    "DEM: Digital Elevation Model. Usually means all elevation models in raster: DSM, DTM,…\n",
    "\n",
    "DSM: Digital Surface Model. Represents the earth’s surface and includes all objects on it. CARS generates DSMs. \n",
    "\n",
    "DTM: Digital Terrain Model. Represents bare ground surface without any objects like plants and buildings.\n",
    "\n",
    "ROI: Region of Interest means a subpart of the DSM raster in CARS.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e8b26e35",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "<img src=\"images/picto_transparent.png\" alt=\"CARS logo\" width=\"100\" align=\"right\"/>\n",
    "\n",
    "# 3D photogrammetry basics"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a33196f-0f0a-4d12-8fe1-91b464f68cb4",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## 3D ?\n",
    "\n",
    "<img alt=\"3D\" src=\"images/Coord_planes_color.svg\" width=\"150\" align=\"right\">\n",
    "\n",
    "3D: geometric setting in which three values are required to determine the position of an element (typically a point)\n",
    "\n",
    "A lot of applications in a lot of fields: 3D printing, biology, architecture, ... \n",
    "\n",
    "Our application here: \n",
    "\n",
    "**GIS 3D Earth cartography with Digital Surface Models !**\n",
    "\n",
    "<img alt=\"DSM\" src=\"images/dsm_clr.png\" width=\"300\" align=\"center\">\n",
    "\n",
    "DSM = a raster grid image with each point containing an elevation information.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "41af5cdc-0b0f-48c0-b535-95249c614406",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## DSM Production methods\n",
    "\n",
    "Several methods: \n",
    "- Eyes ! \n",
    "- GPS : direct on fied measure.\n",
    "- Lidar/Radar : active direct measure.\n",
    "- **Photogrammetry** : indirect measure (same as eyes) by passive observation \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2cdc0b14-88c9-40f1-8c5d-b67d5f4340a7",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Photogrammetry Production principle\n",
    "<table style=\"white-space:nowrap;\">\n",
    "<tr>\n",
    "<td>\n",
    "    &nbsp;\n",
    "</td>\n",
    "<td> \n",
    "    <img alt=\"sat1\" src=\"images/sat.png\" width=\"50\" style=\"transform:rotate(270deg)\">\n",
    "</td>\n",
    "<td> \n",
    "  <p align=\"center\">\n",
    "    <img alt=\"sat2\" src=\"images/sat.png\" width=\"50\" style=\"transform:rotate(310deg)\">\n",
    "  </p> \n",
    "</td>\n",
    "<td> \n",
    "  <p align=\"center\">\n",
    "    <img alt=\"sat3\" src=\"images/sat.png\" width=\"50\">\n",
    "  </p> \n",
    "</td>    \n",
    "</tr>\n",
    "<tr>\n",
    "<td>\n",
    "    Sensor Inputs: </br>\n",
    "    N raster images </br>\n",
    "    + geometric models: altitude, satellite position !\n",
    "</td>\n",
    "<td> \n",
    "    <img alt=\"Pyramid first view\" src=\"images/img1.png\" width=\"200\">\n",
    "</td>\n",
    "<td> \n",
    "  <p align=\"center\">\n",
    "    <img alt=\"img2\" src=\"images/img2.png\" width=\"180\">\n",
    "  </p> \n",
    "</td>\n",
    "<td> \n",
    "  <p align=\"center\">\n",
    "    <img alt=\"img3\" src=\"images/img3.png\" width=\"200\">\n",
    "  </p> \n",
    "</td>\n",
    "</tr>\n",
    "<tr>\n",
    " <td>\n",
    "    Output: </br>\n",
    "     Raster terrain DSM </br>\n",
    "     \n",
    "</td>\n",
    "<td>\n",
    "    &nbsp;\n",
    "</td>\n",
    "<td> \n",
    "  <p align=\"center\" style=\"padding: 10px\">\n",
    "    <img alt=\"DSM\" src=\"images/dsm.png\" width=\"200\">\n",
    "  </p> \n",
    "</td>\n",
    "<td>\n",
    "    &nbsp;\n",
    "</td>\n",
    "</tr>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3913ca6c-87e3-4328-9a51-c9e3d5243d9d",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## One Line of Sight \n",
    "From one image point and geometric direction, how to get altitude ?\n",
    "Needs at least 2 !\n",
    "<img alt=\"img1_los\" src=\"images/img1_los.png\" width=\"300\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ddc7fa3a-d6de-48d9-a6fc-f8fcef960ccc",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Matching\n",
    "For each point in one image, find the correspondent point in the other image.\n",
    "<img alt=\"img1_los\" src=\"images/img1_img2_matching1.png\" width=\"600\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b6bf34f5-9900-4abb-8a2b-554e5513573e",
   "metadata": {
    "tags": []
   },
   "source": [
    "- lot of matching methods !"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7405a498-352f-47c2-b909-567c2d9879c2",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Epipolar matching\n",
    "\n",
    "- Performance driven\n",
    "- Images in same \"eyes\" geometry : lines are aligned and 1 dimension research only\n",
    "\n",
    "<img alt=\"img1_los\" src=\"images/img1_img2_matching2.png\" width=\"600\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f0b0cd9f-e386-47de-9f00-8b7c267f1824",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Triangulation\n",
    "- Images line of sights + matching \n",
    "- = intersection by triangulation \n",
    "- = (x, y, z) point position \n",
    "- => altitude height\n",
    "<img alt=\"img1_los\" src=\"images/img1_img2_triangulation.png\" width=\"600\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "15647e27-1f2a-4b7e-b57c-27b5eb2d1709",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Rasterization\n",
    "Triangulation generates point clouds. \n",
    "\n",
    "To be use as a raster image, a rasterization process project each point in 2D grids to generate DSM.\n",
    "\n",
    "Many methods possibilities here also."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a63da69-2348-4c7a-8085-52feb7c9d856",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "<img src=\"images/picto_transparent.png\" alt=\"CARS logo\" width=\"100\" align=\"right\"/>\n",
    "\n",
    "# CARS high level design"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "25b5aebe-6206-4ff9-a8a7-2f4001a4b286",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## CARS characteristics\n",
    "\n",
    "Objectives: \n",
    "- robust and performant methods for mass production.\n",
    "- state of the art algorithms\n",
    "- satellite data\n",
    "- distributed design\n",
    "- python3 and co ecosystem when possible\n",
    "\n",
    "Technologies used :\n",
    "- Epipolar geometry\n",
    "- Input DTM\n",
    "- Scale Invariant Feature Transform (SIFT) sparse matching [1]\n",
    "- Semi Global Matching(SGM) matching optimization [2]\n",
    "\n",
    "[1] D. G. Lowe. Distinctive image features from scale-invariant keypoints. IJCV, 2(60):91-110, 2004.\n",
    "\n",
    "[2] H. Hirschmuller, \"Stereo Processing by Semiglobal Matching and Mutual Information,\" in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 2, pp. 328-341, Feb. 2008. doi: 10.1109/TPAMI.2007.1166\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "897caa4f-96e4-479b-98d1-15fd0c8a6f36",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## CARS main dependencies\n",
    "\n",
    "- 3D tools: \n",
    " - Matching: Pandora and its libs: libsgm, libmccn,...\n",
    " - Geometry: OTB, shareloc (TO COME) \n",
    " - Sparse matching: Vlfeat\n",
    "- Image libraries: rasterio, pyproj, Fiona, Shapely, NetCDF4\n",
    "- Data libraries: Numpy, Scipi, pandas, Affine, matplotlib.\n",
    "- Distributed and structure libraries: xarray, DASK, numba,\n",
    "- Python packaging,  code quality, documentation: setuptools, pylint, flake8, black, isort, pre-commit, sphinx, ..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0f91dbe-86e7-46e2-b7ed-714d54fbcf7a",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## CARS 3D Framework\n",
    "\n",
    "Composed of:\n",
    "\n",
    "- a python 3D API as \"applications\"\n",
    "- pipelines based on this API \n",
    "\n",
    "\n",
    "<img src=\"images/cars_framework_diagram.png\" alt=\"cars_framework_diagram\" width=\"100%\" align=\"center\"/>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55006f14-d191-4769-b84f-582efb976d93",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Dataset \n",
    "\n",
    "a generic \"CarsDataset\" has been set as internal data structure. \n",
    "\n",
    "- can contain array (ex: for images) or list of point (ex: for sparse matches).\n",
    "\n",
    "- can contains georeference, geometry information.\n",
    "\n",
    "- can be used for parallel/distributed computation, i.e tiled data.\n",
    "\n",
    "- can contained overlaps between tiled data;\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12c564de-3e06-49ec-adc6-11c21c1ea8f7",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## CarsDataset and Orchestrator\n",
    "\n",
    "<img src=\"images/design_overview.png\" alt=\"design_overview\" width=\"100%\" align=\"center\"/>\n",
    "\n",
    "An orchestration framework has been set to manage computing distribution.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50901d78-3048-49ac-8029-699c9633ef12",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Orchestrator and distributed computing\n",
    "\n",
    "DASK framework can be used locally (local_dask) or through PBS on a HPC (pbs_dask).\n",
    "The orchestrator framework separates 3D pipeline from computing distribution. \n",
    "Features:\n",
    "- Memory dependent automatic computing of a tile size \n",
    "- Epipolar tiles and terrain tiles graph creation to distribute tile on nodes.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84ddf140-0524-4135-9be1-3f8680a7e80b",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## CARS CLI\n",
    "\n",
    "<img src=\"images/diagram_cars_overview.png\" alt=\"diagram_cars_overview\" width=\"100%\" align=\"center\"/>\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71dacab4-59ae-438c-9261-7bd9e76c7ab4",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## CARS CLI\n",
    "\n",
    "    cars -h\n",
    "    \n",
    "        usage: cars [-h] [--loglevel {DEBUG,INFO,WARNING,ERROR,CRITICAL}] [--version] conf\n",
    "\n",
    "    CARS: CNES Algorithms to Reconstruct Surface\n",
    "\n",
    "    positional arguments:\n",
    "      conf                  Inputs Configuration File\n",
    "\n",
    "    optional arguments:\n",
    "      -h, --help            show this help message and exit\n",
    "      --loglevel {DEBUG,INFO,WARNING,ERROR,CRITICAL}\n",
    "                            Logger level (default: WARNING. Should be one of (DEBUG, INFO, WARNING, ERROR, CRITICAL)\n",
    "      --version, -v         show program's version number and exit\n",
    "\n",
    "----\n",
    "    cars configfile.json\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5ccc1307-d3b5-46f9-bd76-3fc664d73070",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## CARS Configuration : JSON\n",
    "\n",
    "        {\n",
    "                \"pipeline\": ...,\n",
    "                \"inputs\": {\n",
    "                    ...\n",
    "                },\n",
    "\n",
    "                \"applications\":{\n",
    "                  ...\n",
    "                }\n",
    "\n",
    "                \"orchestrator\": {\n",
    "                        ...\n",
    "                },\n",
    "\n",
    "                \"output\": {\n",
    "                      ...\n",
    "                }\n",
    "        }\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "97d7b83f-7c63-45e8-8123-c991dd0d86ad",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Pipelines \n",
    "\n",
    "Two possibilities: \n",
    "- **sensor_to_dense_dsm** : main pipeline for a dense high resolution DSM (see details after) (default)\n",
    "- **sensor_to_sparse_dsm** : produce a sparse low resolution DSM based on SIFT"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7091019f-60a3-406b-9dc5-4436c54dba76",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Inputs \n",
    "Set sensors, geometric models, pairing, initial_elevation.\n",
    "\n",
    "       {\n",
    "        \"inputs\": {\n",
    "            \"sensors\" : {\n",
    "                \"one\": {\n",
    "                    \"image\": \"img1.tif\",\n",
    "                    \"geomodel\": \"img1.geom\",\n",
    "                    \"no_data\": 0\n",
    "                },\n",
    "                \"two\": {\n",
    "                    \"image\": \"img2.tif\",\n",
    "                    \"geomodel\": \"img2.geom\",\n",
    "                    \"no_data\": 0\n",
    "\n",
    "                }\n",
    "            },\n",
    "            \"pairing\": [[\"one\", \"two\"]],\n",
    "            \"initial_elevation\": \"srtm_dir\"\n",
    "        },"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "477c5d80-547f-494f-bc93-015d3079fbba",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Applications\n",
    "\n",
    "Allows to redefine default parameters for each application used by pipeline and parameter the pipeline."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f80709e6-c3c7-428e-9bf2-da82bd391d41",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Orchestrator\n",
    "Define orchestrator parameters that control the distributed computations:\n",
    "\n",
    "- mode: Parallelization mode “**local_dask**”, “pbs_dask” or “mp”\n",
    "\n",
    "- nb_workers: Number of workers \n",
    "\n",
    "- walltime: dependent on the mode.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fdf961c1-149f-4d8f-af42-6ea9e4c5036e",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Output\n",
    "\n",
    "dependent on the pipeline. For main pipeline example:\n",
    "\n",
    "    \"output\": {\n",
    "          \"out_dir\": \"myoutputfolder\",\n",
    "          \"dsm_basename\": \"mydsm.tif\"\n",
    "    }\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef66890d-c714-4a75-8ed8-877e7bc4c419",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Main simplified 3D pipeline\n",
    "\n",
    "Follows general 3D concepts\n",
    "<table style=\"white-space:nowrap;\">\n",
    "<tr>\n",
    "<td> \n",
    "    <img alt=\"Pyramid first view\" src=\"images/img1.png\" width=\"200\">\n",
    "    </br>\n",
    "    <img alt=\"img2\" src=\"images/img2.png\" width=\"180\">\n",
    "</td>\n",
    "<td>\n",
    "  \n",
    "  </em> Epipolar resampling</em>\n",
    "  <img alt=\"right-arrow\" src=\"images/arrow-right.svg\" align=\"right\">\n",
    "</td>\n",
    "<td> \n",
    "  </em> Dense Matching</em>\n",
    "  <img alt=\"right-arrow\" src=\"images/arrow-right.svg\" align=\"right\">\n",
    "</td>\n",
    "<td> \n",
    "  </em> Triangulation </em>\n",
    "  <img alt=\"right-arrow\" src=\"images/arrow-right.svg\" align=\"right\">\n",
    "</td>\n",
    "<td> \n",
    "  </em> Rasterization</em>\n",
    "</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td> \n",
    "    &nbsp;\n",
    "</td>\n",
    "<td> \n",
    " <img alt=\"OTB Logo\" src=\"images/logo-orfeo-toolbox.png\" width=\"75\">\n",
    "</td>\n",
    "<td> \n",
    "    <img alt=\"Pandora logo\" src=\"images/logo_pandora.png\" width=\"100\">\n",
    "</td>\n",
    "<td> \n",
    " <img alt=\"OTB Logo\" src=\"images/logo-orfeo-toolbox.png\" width=\"75\">\n",
    "</td>\n",
    "<td> \n",
    "<img src=\"images/picto_transparent.png\" alt=\"CARS logo\" width=\"75\"/>\n",
    "</td>\n",
    "</tr>\n",
    "</table>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27426288-e974-4b44-8723-83647b4e217f",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "<img alt=\"sparse_matching_corrections\" src=\"images/sparse_matching_corrections.png\" align=\"right\" width=\"300\">\n",
    "\n",
    "## CARS 3D specifics\n",
    "\n",
    "- First SIFT Sparse matching steps for each pair:\n",
    "   - get vertical epipolar distribution to correct resampling\n",
    "   - get horizontal disparity distribution for dense matching step\n",
    "\n",
    "- use an adapted epipolar geometry : null disparity is based on a reference DTM (SRTM typically)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b84dc580-de64-4640-9288-87c105b87ea4",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "<img alt=\"Pandora logo\" src=\"images/logo_pandora.png\" align=\"right\" width=\"100\"><br>\n",
    "\n",
    "## Pandora dense matching pipeline\n",
    "\n",
    "- Independent toolbox inspired by [1]\n",
    "- Python implementation, except SGM C++ implementation\n",
    "- API or CLI\n",
    "\n",
    "Web site: [https://github.com/CNES/pandora](https://github.com/CNES/pandora)\n",
    "\n",
    "[1] A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms, D. Scharstein and R. Szeliski, vol. 47, International Journal of Computer Vision, 2002\n",
    "\n",
    "<img alt=\"pandora_overview\" src=\"images/pandora_overview.png\" align=\"left\" width=\"80%\"></br>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "06d85cab-4d04-4865-b53e-f0f8d62cca8c",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "<img alt=\"Pandora logo\" src=\"images/logo_pandora.png\" align=\"right\" width=\"100\"><br>\n",
    "\n",
    "## Pandora dense matching pipeline details\n",
    "\n",
    "<img alt=\"pandora_pipeline\" src=\"images/pandora_pipeline.png\" align=\"right\" width=\"150\">\n",
    "\n",
    "<img alt=\"pandora_methods\" src=\"images/pandora_methods.png\" align=\"left\" width=\"50%\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b553112a-8144-412e-bc6e-8b25c28a7325",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Global Dense DSM pipeline\n",
    "<img alt=\"pipeline_dense_dsm\" src=\"images/pipeline_full_res_dsm.png\" width=\"100%\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "87be9630-d1c7-4c63-bb07-87cb2b23d5dd",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Global dense dsm pipeline details\n",
    "\n",
    "For each stereo sensors pair: \n",
    "\n",
    "1. Compute the stereo-rectification grids of the input pair’s images.\n",
    "\n",
    "2. Resample the images pairs in epipolar geometry.\n",
    "\n",
    "3. Compute sift matches between the left and right images in epipolar geometry.\n",
    "\n",
    "4. Predict an optimal disparity range from the filtered point cloud resulting from the sift matches triangulation.\n",
    "\n",
    "5. Create a bilinear correction model of the right image's stereo-rectification grid in order to minimize the epipolar error. Apply the estimated correction to the right grid.\n",
    "\n",
    "6. Resample again the stereo pair in epipolar geometry (using corrected grid for the right image) by using input :term:`DTM` (such as SRTM) in order to reduce the disparity intervals to explore.\n",
    "\n",
    "7. Compute disparity for each image pair in epipolar geometry.\n",
    "\n",
    "8. Triangule the matches and get for each pixel of the reference image a latitude, longitude and altitude coordinate.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30024119-deaf-40bb-baa7-b1b69b4e5806",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Global dense dsm pipeline details\n",
    "\n",
    "For all pairs:\n",
    "\n",
    "9. Merge point clouds coming from each stereo pairs.\n",
    "\n",
    "10. Filtering the 3D points cloud via two consecutive filters. The first one removes the small groups of 3D points. The second filters the points which have the most scattered neighbors.\n",
    "\n",
    "11. Projecting these altitudes on a regular grid as well as the associated color.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78cf8135",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<img src=\"images/picto_transparent.png\" alt=\"CARS logo\" width=\"100\" align=\"right\"/>\n",
    "\n",
    "# CARS quickstart"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "74f37c6d-de1b-4cf4-9281-0a7290441a9d",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Docker quickstart\n",
    "\n",
    "Download CARS Quick Start\n",
    "\n",
    "        mkdir /tmp/cars-tuto/\n",
    "        cd /tmp/cars-tuto/\n",
    "        wget https://raw.githubusercontent.com/CNES/cars/master/tutorials/quick_start.sh\n",
    "\n",
    "Warning: Internet needed to download demo data."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "98908a26-1c98-466b-b06f-c9bcaabfa9c6",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "Run quick_start.sh script\n",
    "```\n",
    "     ./quick_start.sh\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a30ebf8-8c21-4964-aaea-f1ec31bfd7ee",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "```\n",
    "==== Demo CARS (with Docker) =====\n",
    "- Docker must be installed:\n",
    "  # docker -v\n",
    "\n",
    "Docker version 20.10.17, build 100c701\n",
    " \n",
    "- Get CARS dockerfile image:\n",
    " # docker pull cnes/cars\n",
    " \n",
    "- Get and extract data samples from CARS repository:\n",
    "  ! File data_gizeh.tar.bz2 already exists.\n",
    "  # md5sum --status -c data_gizeh.tar.bz2.md5sum\n",
    "md5sum: data_gizeh.tar.bz2.md5sum: Aucun fichier ou dossier de ce type\n",
    "  ! Md5sum check: KO. Exit.\n",
    "  # tar xvfj data_gizeh.tar.bz2\n",
    "data_gizeh/srtm_dir/N29E031_KHEOPS.tif\n",
    "data_gizeh/configfile.json\n",
    "data_gizeh/img1.geom\n",
    "data_gizeh/img2.geom\n",
    "data_gizeh/color1.geom\n",
    "data_gizeh/img3.tif\n",
    "data_gizeh/img2.tif\n",
    "data_gizeh/open-licence-etalab-v2.0-fr.pdf\n",
    "data_gizeh/img3.geom\n",
    "data_gizeh/color1.tif\n",
    "data_gizeh/open-licence-etalab-v2.0-en.pdf\n",
    "data_gizeh/img1.tif\n",
    "data_gizeh/srtm_dir/\n",
    "data_gizeh/\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aa624934",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "Run quick_start.sh script\n",
    "```\n",
    "     ./quick_start.sh\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f10d7e5-f145-4f01-a72f-bb75f5c4cce7",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "```\n",
    "- Launch CARS with sensor_to_full_resolution_dsm pipeline for img1+img2 and img1+img3 pairs:\n",
    "  # cars configfile.json\n",
    "Processing Tiles : [ epi_matches_left ] ...: 100%|█| 16/16 [00:13<00:00,  1.20it\n",
    "Processing Tiles : [ epi_matches_left ] ...: 100%|██████████████████████████████████████████████████████████| 16/16 [00:13<00:00,  1.22it/s]\n",
    "Processing Tiles : [ color , dsm ] ...: 100%|█████████████████████████████████████████████████████████████████| 4/4 [01:12<00:00, 18.13s/it]\n",
    " \n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "678b31e4",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "Run quick_start.sh script\n",
    "```\n",
    "     ./quick_start.sh\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2351eaa6-a688-4cf6-9411-b4863d675d30",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "```\n",
    "- Show resulting DSM:\n",
    "  # ls -al outresults/\n",
    "total 37556\n",
    "-rw-rw-r-- 1 duboise duboise        0 juin  16 15:36 23-06-16_15h36m_sensor_to_dense_dsm.log\n",
    "-rw-rw-r-- 1 duboise duboise 25166744 juin  16 15:38 clr.tif\n",
    "-rw-rw-r-- 1 duboise duboise     7806 juin  16 15:37 content.json\n",
    "-rw-rw-r-- 1 duboise duboise     9621 juin  16 15:36 dask_config_unknown.yaml\n",
    "-rw-rw-r-- 1 duboise duboise 16778119 juin  16 15:38 dsm.tif\n",
    "drwxrwxr-x 2 duboise duboise     4096 juin  16 15:38 one_three\n",
    "drwxrwxr-x 2 duboise duboise     4096 juin  16 15:38 one_two\n",
    "-rw-rw-r-- 1 duboise duboise     8616 juin  16 15:37 used_conf.json\n",
    "drwxrwxr-x 2 duboise duboise     4096 juin  16 15:36 workers_log\n",
    "\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f86df17-3e56-49cc-b907-fb97058780da",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Quick start results\n",
    "\n",
    "<table><tr>\n",
    "<td> \n",
    "  <p align=\"center\" style=\"padding: 10px\">\n",
    "    <img alt=\"DSM\" src=\"images/dsm.png\" width=\"200\">\n",
    "  </p> \n",
    "</td>\n",
    "<td> \n",
    "  <p align=\"center\" style=\"padding: 10px\">\n",
    "    <img alt=\"Color\" src=\"images/clr.png\" width=\"200\">\n",
    "  </p> \n",
    "</td>\n",
    "<td> \n",
    "  <p align=\"center\" style=\"padding: 10px\">\n",
    "    <img alt=\"MIX\" src=\"images/dsm_clr.png\" width=\"200\">\n",
    "  </p> \n",
    "</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>\n",
    "    <em>dsm.tif</em>\n",
    "</td>\n",
    "<td>\n",
    "    <em>clr.tif</em>\n",
    "</td>\n",
    "<td>\n",
    "    <em>clr and dsm colored composition </em>\n",
    "</td> \n",
    "</tr>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30844f38-90ac-4eaa-8de4-e37ad6f6c49c",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Quick start details\n",
    "\n",
    "1. See input data\n",
    "   - sensor images + geometric models\n",
    "   - initial DTM (SRTM tile)\n",
    "\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "caecadfe-a751-48dd-a95f-a2280a78eeeb",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Quick start details\n",
    "\n",
    "2. See configuration\n",
    "\n",
    "    cat configfile.json\n",
    "       \n",
    "       {\n",
    "        \"inputs\": {\n",
    "            \"sensors\" : {\n",
    "                \"one\": {\n",
    "                    \"image\": \"img1.tif\",\n",
    "                    \"geomodel\": \"img1.geom\",\n",
    "                    \"color\": \"color1.tif\",\n",
    "                    \"no_data\": 0\n",
    "                },\n",
    "                \"two\": {\n",
    "                    \"image\": \"img2.tif\",\n",
    "                    \"geomodel\": \"img2.geom\",\n",
    "                    \"no_data\": 0\n",
    "\n",
    "                },\n",
    "                \"three\": {\n",
    "                    \"image\": \"img3.tif\",\n",
    "                    \"geomodel\": \"img3.geom\",\n",
    "                    \"no_data\": 0\n",
    "                }\n",
    "            },\n",
    "            \"pairing\": [[\"one\", \"two\"],[\"one\", \"three\"]],\n",
    "             \"initial_elevation\": \"srtm_dir\"\n",
    "        },\n",
    "\n",
    "        \"output\": {\n",
    "              \"out_dir\": \"outresults\"\n",
    "        }\n",
    "       }\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c249410-54d5-4867-9f12-8228fdde4889",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Quick start advanced\n",
    "\n",
    "When installing CARS directly (but needs OTB, VLFeat installation), a quick_start_advanced.sh runs the same example but without Docker\n",
    "\n",
    "```\n",
    "    ./quick_start_advanced.sh\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7a4207ca-c913-485f-98f6-479124dbf9b4",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Direct installation (if needed)\n",
    "\n",
    "- Install OTB and Vlfeat and cmake\n",
    "- Clone CARS github repo\n",
    "- make install (or make install-dev for pip install -e)\n",
    "- source venv/bin/activate\n",
    "- source venv/bin/env_cars.sh # to set all needed CARS env \n",
    "\n",
    "For more details, see Dockerfile on Github repo."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f636f15-a3c8-47cc-a430-0b2215ee876e",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Jupyter notebook\n",
    "\n",
    "From cars-jupyter docker:\n",
    "\n",
    "    docker run -p 8888:8888 cnes/cars-jupyter\n",
    "    \n",
    "This runs a jupyter notebook directly to *https://localhost:8888/*\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aca77c08",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<img src=\"images/picto_transparent.png\" alt=\"CARS logo\" width=\"100\" align=\"right\"/>\n",
    "\n",
    "# CARS CLI examples"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0fe24115-1d0d-4ff2-bce0-4b6be416c2de",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Logging\n",
    "Run CARS with more information: \n",
    "\n",
    "    cars --loglevel INFO configfile.json"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d6647f2-d2da-4662-bc1b-2ef02aa848f1",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## CARS orchestration modification : nb_workers\n",
    "\n",
    "- Add orchestration configuration in input json file:\n",
    "\n",
    "        \"orchestrator\": {\n",
    "                \"mode\": \"local_dask\",\n",
    "                \"nb_workers\": 4\n",
    "        },\n",
    "\n",
    "\n",
    "- Run CARS again to see 4 workers : cars --loglevel INFO configfile.json \n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "669b6795-a717-4ea9-9dca-ed05333c58c5",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## CARS orchestration modification: sequential mode\n",
    "\n",
    "- Add orchestration configuration in input json file:\n",
    "\n",
    "        \"orchestrator\": {\n",
    "                \"mode\": \"sequential\"\n",
    "        },\n",
    "\n",
    "\n",
    "- Run CARS again : cars --loglevel INFO configfile.json\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d85f1e48-40e7-42e0-ab2b-4fe1e157098b",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Application configuration: save disparity maps\n",
    "\n",
    "- Add application configuration in input json file and define parameters for dense matching application\n",
    "\n",
    "        \"applications\": {\n",
    "                \"dense_matching\":{\n",
    "                        \"method\": \"census_sgm\",\n",
    "                        \"loader\": \"pandora\",\n",
    "                        \"save_disparity_map\": true\n",
    "                      }\n",
    "        },\n",
    "\n",
    "\n",
    "- Run CARS again : cars --loglevel INFO configfile.json\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e83d43cd",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Application configuration: save disparity maps\n",
    "\n",
    "- Show resulting disparity maps\n",
    "\n",
    "\n",
    "```\n",
    "  # ls -l data_gizeh/outresults/\n",
    "     total 44580\n",
    "      -rw-r--r-- 1 carcars carcars        0 août   6 00:42 22-08-05_22h42m_sensor_to_full_res_dsm.log\n",
    "      -rw-r--r-- 1 carcars carcars 33555362 août   6 00:46 clr.tif\n",
    "      -rw-r--r-- 1 carcars carcars     9120 août   6 00:43 content.json\n",
    "      -rw-r--r-- 1 carcars carcars     7864 août   6 00:42 dask_config_unknown.yaml\n",
    "      -rw-r--r-- 1 carcars carcars 16778119 août   6 00:46 dsm.tif\n",
    "      drwxr-xr-x 2 carcars carcars     4096 août   6 00:46 one_three\n",
    "      drwxr-xr-x 2 carcars carcars     4096 août   6 00:46 one_two\n",
    "\n",
    "  # ls -l data_gizeh/outresults/one_two\n",
    "      -rw-r--r-- 1 carcars carcars     9120 août   6 00:43 epi_disp_color_left.tif\n",
    "      -rw-r--r-- 1 carcars carcars     7864 août   6 00:42 epi_disp_left.tif\n",
    "      -rw-r--r-- 1 carcars carcars 16778119 août   6 00:46 epi_disp_mask_left.tif\n",
    "  \n",
    "```\n",
    "  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b0028585",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Application configuration: rasterization parameters\n",
    "\n",
    "- Add application configuration in input json file and define parameters for rasterization application\n",
    "\n",
    "        \"applications\": {\n",
    "             \"point_cloud_rasterization\": { \n",
    "                \"method\": \"simple_gaussian\",\n",
    "                \"dsm_radius\": 3,\n",
    "                \"sigma\": 0.3\n",
    "             }\n",
    "        },\n",
    "\n",
    "\n",
    "- Run CARS again : cars --loglevel INFO configfile.json\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "48765336",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<img src=\"images/picto_transparent.png\" alt=\"CARS logo\" width=\"100\" align=\"right\"/>\n",
    "\n",
    "# CARS Step by step 3D framework manipulation\n",
    "\n",
    "Step by step tutorial of **sensor_to_dense_dsm_pipeline** for one pair.\n",
    "\n",
    "Tutorial can be run through \"sensor_to_dense_dsm_step_by_step.ipynb\" notebook directly (no presentation mode)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b96c72c0-0069-4328-9019-4c72d8e77b0e",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e314f727-80b4-4658-a718-590070d5d8ba",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import math\n",
    "###\n",
    "# Silent OTB info logs\n",
    "os.environ['OTB_LOGGER_LEVEL']='WARNING'\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\",category=UserWarning)\n",
    "# import pretty print\n",
    "import pprint \n",
    "pp = pprint.PrettyPrinter(indent=2)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "714087ba-cbcc-44d2-8e09-6e9ec2a20700",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# import external notebooks helpers function for tutorial\n",
    "from notebook_helpers import get_full_data, show_data, save_data\n",
    "from notebook_helpers import get_dir_path, set_up_demo_inputs\n",
    "\n",
    "# CARS imports\n",
    "from cars.applications.application import Application\n",
    "from cars.applications.grid_generation import grid_correction\n",
    "from cars.applications.sparse_matching import sparse_matching_tools\n",
    "\n",
    "import cars.pipelines.sensor_to_dense_dsm.sensor_dense_dsm_constants as sens_cst\n",
    "from cars.pipelines.sensor_to_dense_dsm import sensors_inputs\n",
    "from cars.pipelines.sensor_to_dense_dsm import dsm_output\n",
    "\n",
    "from cars.core import cars_logging\n",
    "from cars.core import inputs, preprocessing\n",
    "from cars.core.utils import safe_makedirs\n",
    "from cars.orchestrator import orchestrator\n",
    "from cars.core.utils import make_relative_path_absolute\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "18f79ed3-b569-4f44-9805-d9dcb3bf4241",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from cars import __version__\n",
    "#print(\"CARS version used : {}\".format(__version__))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f4f3c924-a3bd-4745-a113-cb200d7b590a",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Define outputs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "52c3f997-8489-414f-88ed-c302f006cc06",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Modify with your own output path if needed\n",
    "output_dir = os.path.join(get_dir_path(), \"output_tutorial\")\n",
    "#print(output_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fc371b22-12f3-4c3c-b938-b616835f5a8c",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Define inputs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9241c6d5-beea-4647-af87-2b3a323373f6",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# By default, the tutorial use data_gizeh_small.tar.bz2\n",
    "input_dir_path = set_up_demo_inputs(\"data_gizeh_small\")\n",
    "\n",
    "inputs_conf = {\n",
    "    \"sensors\": {\n",
    "        \"left\": {\n",
    "            \"image\": os.path.join(input_dir_path, \"img1.tif\"),\n",
    "            \"geomodel\": os.path.join(input_dir_path, \"img1.geom\"),\n",
    "            \"color\": os.path.join(input_dir_path, \"color1.tif\"),\n",
    "            \"no_data\": 0,\n",
    "\n",
    "\n",
    "        },\n",
    "        \"right\": {\n",
    "            \"image\": os.path.join(input_dir_path, \"img2.tif\"),\n",
    "            \"geomodel\": os.path.join(input_dir_path, \"img2.geom\"),\n",
    "            \"no_data\": 0,\n",
    "        },   \n",
    "    },\n",
    "    \"pairing\": [[\"left\", \"right\"]],\n",
    "    \"initial_elevation\": os.path.join(input_dir_path, \"srtm_dir/N29E031_KHEOPS.tif\")\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "288280d9-f08c-4123-b5f5-c8027ecb329b",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Check and complete inputs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c8207287-f23e-400d-b4ba-74c12c9b2b8e",
   "metadata": {},
   "outputs": [],
   "source": [
    "inputs = sensors_inputs.sensors_check_inputs(inputs_conf)\n",
    "#pp.pprint(inputs)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fd0534c4-8413-4bae-938c-c97133541ff2",
   "metadata": {},
   "source": [
    "    {'check_inputs': False,\n",
    "    'default_alt': 0,\n",
    "    'epsg': None,\n",
    "    'geoid': 'PATH_CARS_GEOID',\n",
    "    'initial_elevation': 'PATH_TUTORIAL/data_gizeh_small/srtm_dir',\n",
    "    'pairing': [['left', 'right']],\n",
    "    'roi': None,\n",
    "    'sensors': { \n",
    "        'left': { 'color': 'PATH_TUTORIAL/data_gizeh_small/color1.tif',\n",
    "                 'geomodel': 'PATH_TUTORIAL/data_gizeh_small/img1.geom',\n",
    "                 'image': 'PATH_TUTORIAL/data_gizeh_small/img1.tif',\n",
    "                 'mask': None,\n",
    "                 'classification': None,\n",
    "                 'no_data': 0},\n",
    "       'right': { 'color': 'PATH_TUTORIAL/data_gizeh_small/img2.tif',\n",
    "                  'geomodel': 'PATH_TUTORIAL/data_gizeh_small/img2.geom',\n",
    "                  'image': 'PATH_TUTORIAL/data_gizeh_small/img2.tif',\n",
    "                  'mask': None,\n",
    "                  'classification': None,\n",
    "                  'no_data': 0}}}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "160d1c83-1e30-4e44-b2be-a22886aba8ef",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Applications init"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4237bc30-3ab2-4e95-883f-8174cc536a47",
   "metadata": {
    "tags": []
   },
   "source": [
    "### GridGeneration\n",
    "\n",
    "This application generates epipolar grids corresponding to sensor pair"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fe5b8dd1-040f-49ab-9238-7d023f8ca54f",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "epipolar_grid_generation_application = Application(\"grid_generation\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fc2af174-530d-4cd9-ad82-cbb62e78080c",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "### Resampling\n",
    "\n",
    "This application generates epipolar images from epipolar grids"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e17a964d-a5e5-4be6-af55-b2a8ad44072b",
   "metadata": {},
   "outputs": [],
   "source": [
    "resampling_application = Application(\"resampling\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59b82349-7f32-4680-881a-4f748307aacc",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "### SparseMatching\n",
    "\n",
    "This application generates sparse matches of stereo images pairs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6baf0da9-6c55-4e28-8479-73ca43541406",
   "metadata": {},
   "outputs": [],
   "source": [
    "sparse_matching_application = Application(\"sparse_matching\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe377618",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Applications init"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ccea8668-65e4-4291-b0f6-3ef010c1785a",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "### DenseMatching\n",
    "\n",
    "This application generates dense matches of stereo images pairs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "57fc90f3-ddfe-428f-859b-4bf3a0583a73",
   "metadata": {},
   "outputs": [],
   "source": [
    "dense_matching_application = Application(\"dense_matching\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fcff5123-509c-4ec4-99f8-1a65b68c257f",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "### Triangulation\n",
    "\n",
    "This application triangulates matches, in order to get each (X, Y, Z) point position"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "36819a36-c1df-4b09-850b-ef5e3e3c1c9d",
   "metadata": {},
   "outputs": [],
   "source": [
    "triangulation_application = Application(\"triangulation\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d18ee567-f2b3-43ce-9574-17ff17b1fd28",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "### PointCloudFusion\n",
    "\n",
    "This application performs the fusion of epipolar points from pairs to a terrain point cloud"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c4b96ae1-bf88-4115-b6f6-903f3785c8c5",
   "metadata": {},
   "outputs": [],
   "source": [
    "pc_fusion_application = Application(\"point_cloud_fusion\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1ac58f9-ec44-4969-b259-549d5bc633de",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Applications init"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "82f1694b-c351-476a-b5f3-27a9d46bd8cd",
   "metadata": {
    "tags": []
   },
   "source": [
    "### PointCloudOutliersRemoving : small components\n",
    "\n",
    "This application removes outliers points. The method used is the \"small components removing\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aa228145-c753-4dbc-ac5e-3f855d47dbcc",
   "metadata": {},
   "outputs": [],
   "source": [
    "conf_outlier_removing_small_components = {\"method\": \"small_components\", \"activated\": True}\n",
    "\n",
    "(\n",
    "    pc_outlier_removing_small_comp_application\n",
    ") = Application(\"point_cloud_outliers_removing\",\n",
    "                cfg=conf_outlier_removing_small_components)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a7b69093-9b7f-464d-aa62-43a7b653f0d4",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Applications init"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6d314e8e-077a-476c-ba40-02d769107554",
   "metadata": {
    "tags": []
   },
   "source": [
    "### PointCloudOutliersRemoving : statistical\n",
    "\n",
    "This application removes outliers points. The method used is the \"statistical removing\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "db6db4d4-1285-40e8-8fc6-e296a071a489",
   "metadata": {},
   "outputs": [],
   "source": [
    "conf_outlier_removing_small_statistical = {\"method\": \"statistical\", \"activated\": True}\n",
    "\n",
    "pc_outlier_removing_stats_application = Application(\n",
    "                    \"point_cloud_outliers_removing\",\n",
    "                    cfg=conf_outlier_removing_small_statistical,\n",
    "                    )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "389140f1-cd67-4ccb-a922-396b98051c73",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Applications init"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "beda8c9b-abb6-4eed-a06f-9a14e465fc69",
   "metadata": {
    "tags": []
   },
   "source": [
    "### PointCloudRasterization\n",
    "\n",
    "This application performs the rasterization of a terrain point cloint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7d83e6af-6bbe-49dd-af3e-5a027a87c6a4",
   "metadata": {},
   "outputs": [],
   "source": [
    "conf_rasterization = { \n",
    "    \"method\": \"simple_gaussian\",\n",
    "    \"dsm_radius\": 3,\n",
    "    \"sigma\": 0.3\n",
    "}\n",
    "rasterization_application = Application(\"point_cloud_rasterization\", \n",
    "                                        cfg=conf_rasterization)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34ef92b9-a0d8-40d1-8914-a87684329d52",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Show used application configuration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8475f8f9-b86a-472b-bb81-8456e46120cc",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Example with dense matching application\n",
    "dense_matching_application.print_config()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "24f12e30-e8d9-46f9-bda7-2c2f4ed33e6f",
   "metadata": {
    "tags": []
   },
   "source": [
    "```\n",
    "  {'epipolar_tile_margin_in_percent': 60,\n",
    "    'loader': 'pandora',\n",
    "    'loader_conf': {   \n",
    "    'input': {'nodata_left': -9999, 'nodata_right': -9999},\n",
    "       'pipeline': {   'disparity': {   'disparity_method': 'wta',\n",
    "                                        'invalid_disparity': nan},\n",
    "                       'filter': {   'filter_method': 'median',\n",
    "                                     'filter_size': 3},\n",
    "                       'matching_cost': {   'matching_cost_method': 'census',\n",
    "                                            'subpix': 1,\n",
    "                                            'window_size': 5},\n",
    "                       'optimization': {   'P1': 8,\n",
    "                                           'P2': 32,\n",
    "                                           'min_cost_paths': False,\n",
    "                                           'optimization_method': 'sgm',\n",
    "                                           'overcounting': False,\n",
    "                                           'p2_method': 'constant',\n",
    "                                           'penalty_method': 'sgm_penalty',\n",
    "                                           'piecewise_optimization_layer': 'None',\n",
    "                                           'sgm_version': 'c++',\n",
    "                                           'use_confidence': False},\n",
    "                       'refinement': {   'refinement_method': 'vfit'},\n",
    "                       'right_disp_map': {'method': 'accurate'},\n",
    "                       'validation': {   'cross_checking_threshold': 1.0,\n",
    "                                         'validation_method': 'cross_checking'}}},\n",
    "    'max_elevation_offset': None,\n",
    "    'max_epi_tile_size': 1500,\n",
    "    'method': 'census_sgm',\n",
    "    'min_elevation_offset': None,\n",
    "    'min_epi_tile_size': 300,\n",
    "    'save_disparity_map': False}\n",
    "    \n",
    " ```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "df8560f1-b2a3-40df-ac0e-d811b457b18f",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Create orchestrator\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "885a0d34-c29c-454b-aa6b-fd380a22c69d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Use sequential mode in notebook\n",
    "orchestrator_conf = {\"mode\": \"sequential\"}\n",
    "cars_orchestrator = orchestrator.Orchestrator(\n",
    "                        orchestrator_conf=orchestrator_conf, \n",
    "                        out_dir=output_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "01cd9420-a482-43bf-8e63-79a76ee906d0",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Run pipeline step by step from sensors to DSM"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1b38907-801e-456f-bd51-7143eb13891b",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### Sensors images generation\n",
    "\n",
    "From input configuration \"inputs\" seen before"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c20b4172-b960-424f-bbba-48297727b3c9",
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    _, \n",
    "    sensor_image_left,\n",
    "    sensor_image_right\n",
    ") = sensors_inputs.generate_inputs(inputs)[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3905819a-3d91-4122-acbc-7a9cf32bf39e",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### Grid Generation : epipolar grid generation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2643a59f-a091-4e35-99f9-9f9cca042a94",
   "metadata": {},
   "outputs": [],
   "source": [
    "grid_left, grid_right = epipolar_grid_generation_application.run(\n",
    "    sensor_image_left,\n",
    "    sensor_image_right,\n",
    "    orchestrator=cars_orchestrator,\n",
    "    srtm_dir=inputs[sens_cst.INITIAL_ELEVATION],\n",
    "    default_alt=inputs[sens_cst.DEFAULT_ALT],\n",
    "    geoid_path=inputs[sens_cst.GEOID],\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "796b637c-2ad4-4018-b913-6e95d5d9dcab",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### Resampling : epipolar images generation\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1e7aa8f6-deee-43e8-9b93-f719f03cb1a8",
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    epipolar_image_left, \n",
    "    epipolar_image_right \n",
    ") =  resampling_application.run(\n",
    "        sensor_image_left,\n",
    "        sensor_image_right,\n",
    "        grid_left,\n",
    "        grid_right,\n",
    "        orchestrator=cars_orchestrator,\n",
    "        margins=sparse_matching_application.get_margins()\n",
    "        )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f5ce108-edf4-4dbd-bb06-8eaf0523730a",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### Show epipolar image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b49eb2e1-f728-4fe7-8375-29c04ccd2a27",
   "metadata": {},
   "outputs": [],
   "source": [
    "data_image_left = get_full_data(epipolar_image_left, \"im\")\n",
    "show_data(data_image_left, mode=\"image\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a88972e-fcc7-4ba5-bf6c-bca393353108",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "###  Sparse matching: compute sifts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f9608e45-e1e8-43eb-820d-df007937af40",
   "metadata": {},
   "outputs": [],
   "source": [
    "epipolar_matches_left, _ = sparse_matching_application.run(\n",
    "    epipolar_image_left,\n",
    "    epipolar_image_right,\n",
    "    grid_left.attributes[\"disp_to_alt_ratio\"],\n",
    "    orchestrator=cars_orchestrator\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c063c29e-7578-4ee5-986f-ca7d7a6fa191",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### Grid correction: correct epipolar grids from sparse matches\n",
    "Find correction to apply"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0fac31b0-3725-451e-bfd0-c653894dc461",
   "metadata": {
    "tags": []
   },
   "source": [
    "#### Filter matches"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6b7a916f-2555-466e-99c1-c738fd8a5633",
   "metadata": {},
   "outputs": [],
   "source": [
    "matches_array = sparse_matching_application.filter_matches(\n",
    "                            epipolar_matches_left,\n",
    "                            orchestrator=cars_orchestrator)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4201202b-c91f-48e7-af56-b22311e70e36",
   "metadata": {},
   "source": [
    "#### Estimate grid correction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9b70cb92-da0c-4be4-b34a-61eb721b5a8d",
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    grid_correction_coef, \n",
    "    corrected_matches_array, _, _, _\n",
    ") = grid_correction.estimate_right_grid_correction(\n",
    "                    matches_array, grid_right)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27eb5194-a83c-4435-8096-c01eebaa203e",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### Grid correction: correct epipolar grids from sparse matches\n",
    "Generate new right epipolar grid from correction"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59391f91-322c-40c0-b310-f8074e66a4e0",
   "metadata": {
    "tags": []
   },
   "source": [
    "#### Correct right grid"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "30c2ae48-0189-473d-bf5d-8adfc269453e",
   "metadata": {},
   "outputs": [],
   "source": [
    "corrected_grid_right = grid_correction.correct_grid(\n",
    "                        grid_right, grid_correction_coef)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "353b87bd-3525-4d36-9bf8-e8137fd51634",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### Estimate disp min and disp max from sparse matches"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "246e95dc-21f6-431a-8e5f-d965519e1f8a",
   "metadata": {},
   "outputs": [],
   "source": [
    "dmin, dmax = sparse_matching_tools.compute_disp_min_disp_max(\n",
    "                    sensor_image_left,\n",
    "                    sensor_image_right,\n",
    "                    grid_left,\n",
    "                    corrected_grid_right,\n",
    "                    grid_right,\n",
    "                    corrected_matches_array,\n",
    "                    orchestrator=cars_orchestrator,\n",
    "                    disp_margin=(\n",
    "                        sparse_matching_application.get_disparity_margin()\n",
    "                    ),\n",
    "                    disp_to_alt_ratio=grid_left.attributes[\"disp_to_alt_ratio\"],\n",
    "                    geometry_loader=triangulation_application.get_geometry_loader(),\n",
    "                    srtm_dir=inputs[sens_cst.INITIAL_ELEVATION],\n",
    "                    default_alt=inputs[sens_cst.DEFAULT_ALT],\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "778c37f3-f820-45e8-9ca0-f49a40c03c51",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### Compute margins used in dense matching, with corresponding disparity min and max"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0772b5a9-614e-4046-a6ac-c1b7bf1afde6",
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    dense_matching_margins, \n",
    "    disp_min, \n",
    "    disp_max\n",
    ") = dense_matching_application.get_margins(\n",
    "            grid_left, disp_min=dmin, disp_max=dmax)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b0020ac-7dff-44ae-8534-ab019031580a",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "###  Resampling: generate epipolar images with corrected grids and new margins"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2878c0e4-b3b3-4505-8025-6adfbcfb45ca",
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    new_epipolar_image_left, \n",
    "    new_epipolar_image_right\n",
    ") = resampling_application.run(\n",
    "        sensor_image_left,\n",
    "        sensor_image_right,\n",
    "        grid_left,\n",
    "        corrected_grid_right,\n",
    "        orchestrator=cars_orchestrator,\n",
    "        margins=dense_matching_margins,\n",
    "        optimum_tile_size=(\n",
    "        dense_matching_application.get_optimal_tile_size(\n",
    "            disp_min, \n",
    "            disp_max, \n",
    "            cars_orchestrator.cluster.checked_conf_cluster[\n",
    "                \"max_ram_per_worker\"\n",
    "            ],\n",
    "        )\n",
    "        ),\n",
    "        add_color=True,\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be39e1e3-7749-4b03-8cbd-9d9c1ae029ed",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### Dense Matching: compute disparities with pandora"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cb4e9562-7872-491d-9501-b79c45017d4c",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "epipolar_disparity_map = dense_matching_application.run(\n",
    "        new_epipolar_image_left,\n",
    "        new_epipolar_image_right,\n",
    "        orchestrator=cars_orchestrator,\n",
    "        disp_min=disp_min,\n",
    "        disp_max=disp_max,\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14cd7de6",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### Dense Matching: compute disparities with pandora"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "00ccd849-4aa3-45af-8abf-4774c2267e5e",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "source": [
    "### Show full disparity map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7dbc7816-0e71-440e-ab54-82b3504f380a",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "data_disparity = get_full_data(epipolar_disparity_map, \"disp\")\n",
    "show_data(data_disparity)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e5f211d5-630e-4418-98b7-39fafcca5b81",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### Compute epsg"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7a61ef25-d497-4518-8299-89387e5f5c78",
   "metadata": {},
   "outputs": [],
   "source": [
    "epsg = preprocessing.compute_epsg(\n",
    "    sensor_image_left, \n",
    "    sensor_image_right,\n",
    "    grid_left,\n",
    "    corrected_grid_right,\n",
    "    triangulation_application.get_geometry_loader(),\n",
    "    orchestrator=cars_orchestrator,\n",
    "    srtm_dir=inputs[sens_cst.INITIAL_ELEVATION],\n",
    "    default_alt=inputs[sens_cst.DEFAULT_ALT],\n",
    "    disp_min=disp_min,\n",
    "    disp_max=disp_max\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c4085b5-16fc-45db-b2a1-35463e7d9647",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### Triangulation : triangulate matches"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "26be2442-24bc-47a1-b586-6d6608665d6e",
   "metadata": {},
   "outputs": [],
   "source": [
    "epipolar_points_cloud = triangulation_application.run(\n",
    "        sensor_image_left,\n",
    "        sensor_image_right,\n",
    "        new_epipolar_image_left,\n",
    "        grid_left,\n",
    "        corrected_grid_right,\n",
    "        epipolar_disparity_map,\n",
    "        epsg,\n",
    "        orchestrator=cars_orchestrator,\n",
    "        uncorrected_grid_right=grid_right,\n",
    "        geoid_path=inputs[sens_cst.GEOID],\n",
    "        disp_min=disp_min,\n",
    "        disp_max=disp_max,\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "65035861-2307-4071-82a3-be3ab36a4230",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    " ### Compute terrain bounding box"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cb6c799d-1a87-460a-9d10-e4c68fab7dc4",
   "metadata": {},
   "outputs": [],
   "source": [
    "current_terrain_roi_bbox = preprocessing.compute_terrain_bbox(\n",
    "    inputs[sens_cst.INITIAL_ELEVATION],\n",
    "    inputs[sens_cst.DEFAULT_ALT],\n",
    "    inputs[sens_cst.GEOID],\n",
    "    sensor_image_left,\n",
    "    sensor_image_right,\n",
    "    new_epipolar_image_left,\n",
    "    grid_left,\n",
    "    corrected_grid_right,\n",
    "    epsg,\n",
    "    triangulation_application.get_geometry_loader(),\n",
    "    resolution=rasterization_application.get_resolution(),\n",
    "    disp_min=disp_min,\n",
    "    disp_max=disp_max,\n",
    "    orchestrator=cars_orchestrator\n",
    ")\n",
    "(\n",
    "    terrain_bounds, \n",
    "    optimal_terrain_tile_width\n",
    ") = preprocessing.compute_terrain_bounds(\n",
    "        [current_terrain_roi_bbox],\n",
    "        resolution=rasterization_application.get_resolution()\n",
    "        )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "07f44cef-0496-4afc-bc5d-a20d27738f2f",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### Transform point cloud to terrain point cloud"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b4e36f36-c978-4cc9-90dc-f5970f175a5b",
   "metadata": {},
   "outputs": [],
   "source": [
    "merged_points_clouds = pc_fusion_application.run(\n",
    "    [epipolar_points_cloud],\n",
    "    terrain_bounds,\n",
    "    epsg,\n",
    "    orchestrator=cars_orchestrator,\n",
    "    margins=rasterization_application.get_margins(),\n",
    "    optimal_terrain_tile_width=optimal_terrain_tile_width\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "025df05c-d05f-49a1-bfd1-f3f2a87ccdb3",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### Point Cloud Outlier Removing : remove points with small components removing method"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "740c1ccb-8568-43aa-bcec-fc7cd42b10d8",
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    filtered_sc_merged_points_clouds\n",
    ") = pc_outlier_removing_small_comp_application.run(\n",
    "        merged_points_clouds,\n",
    "        orchestrator=cars_orchestrator)\n",
    "\n",
    "         "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e98b3d7f-b162-4cda-aa74-89d2c1971b00",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### Point Cloud Outlier Removing: remove points with statistical removing method"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4f35ea51-3756-4c19-9ba2-b0e5b30fa74f",
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    filtered_stats_merged_points_clouds\n",
    ") = pc_outlier_removing_stats_application.run(\n",
    "        filtered_sc_merged_points_clouds,\n",
    "        orchestrator=cars_orchestrator)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d41b16bb-f32d-4eb4-8817-293443a00eb2",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### Rasterization : rasterize point cloud"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25439ab1-a682-4195-b7ed-4076a7bbced3",
   "metadata": {},
   "outputs": [],
   "source": [
    "dsm = rasterization_application.run(\n",
    "    filtered_stats_merged_points_clouds,\n",
    "    epsg,\n",
    "    orchestrator=cars_orchestrator\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a09252e-26dc-4088-bf5d-273430819272",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### Show DSM\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1e14151a-cd86-47a4-85b4-3c82926f4b7c",
   "metadata": {},
   "outputs": [],
   "source": [
    "data_dsm = get_full_data(dsm, \"hgt\")\n",
    "show_data(data_dsm, mode=\"dsm\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20fb1223-9135-4755-ad46-38de40d644a6",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### Show ortho image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1931bcf3-c341-4329-9791-63f439c9215a",
   "metadata": {},
   "outputs": [],
   "source": [
    "data_ortho = get_full_data(dsm, \"img\")[..., 0:3]\n",
    "show_data(data_ortho, mode='image')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c812d589-8c9f-4424-b456-188c4e9a102c",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### Save DSM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7004c028-d256-41c6-bf8d-6932afe89504",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "save_data(dsm, os.path.join(output_dir, \"dsm.tif\"), \"hgt\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cad006ba",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<img src=\"images/picto_transparent.png\" alt=\"CARS logo\" width=\"100\" align=\"right\"/>\n",
    "\n",
    "# CARS stereo matching with AI improvement"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2897f78f-71f7-4896-89f6-cce3499ff874",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## New method presentation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e333656e",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<img alt=\"Pandora logo\" src=\"images/logo_pandora.png\" align=\"right\" width=\"100\"><br>\n",
    "\n",
    "## Pandora dense matching pipeline\n",
    "\n",
    "- Independent toolbox inspired by [1]\n",
    "- Python implementation, except SGM C++ implementation\n",
    "- API or CLI\n",
    "\n",
    "Web site: [https://github.com/CNES/pandora](https://github.com/CNES/pandora)\n",
    "\n",
    "[1] A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms, D. Scharstein and R. Szeliski, vol. 47, International Journal of Computer Vision, 2002\n",
    "\n",
    "<img alt=\"pandora_overview\" src=\"images/pandora_overview.png\" align=\"left\" width=\"80%\"></br>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c280349b",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<img alt=\"Pandora logo\" src=\"images/logo_pandora.png\" align=\"right\" width=\"100\"><br>\n",
    "\n",
    "## Pandora dense matching pipeline details\n",
    "\n",
    "<img alt=\"pandora_pipeline\" src=\"images/pandora_pipeline.png\" align=\"right\" width=\"150\">\n",
    "\n",
    "<img alt=\"pandora_methods\" src=\"images/pandora_methods.png\" align=\"left\" width=\"50%\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91f1fbdb",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## MC-CNN similarity measure\n",
    "\n",
    "- mc-cnn [3] is a neural network which computes a similarity measure on pair of small image patches\n",
    "\n",
    "- Implemented in <a href=\"https://github.com/CNES/Pandora_MCCNN\">MC-CNN project</a> used by <a  href=\"https://github.com/CNES/Pandora\"> Pandora</a> as <a href=\"https://github.com/CNES/Pandora_plugin_mccnn\">plugin</a>\n",
    "\n",
    "- Pretrained weights for mc-cnn neural networks are available in <a href=\"https://github.com/CNES/Pandora_plugin_mccnn\">mc-ccnn plugin repository</a>\n",
    "\n",
    "[3] Zbontar, J., & LeCun, Y. (2016). Stereo matching by training a convolutional neural network to compare image patches. J. Mach. Learn. Res., 17(1), 2287-2318."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "92b531ab-852d-4b2e-8e93-62e56a0489fa",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Run new method with CARS and results\n",
    "\n",
    "Tutorial can be run through \"sensor_to_dense_dsm_matching_methods_comparison.ipynb\" notebook directly (no presentation mode)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "feec5f48",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7f7b6fe6-b60c-4f99-948b-2b7a1af4f52a",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "# Notebook local imports\n",
    "\n",
    "import os\n",
    "import math\n",
    "###\n",
    "# Silent OTB info logs\n",
    "os.environ['OTB_LOGGER_LEVEL']='WARNING'\n",
    "import warnings\n",
    "# Filter warnings\n",
    "warnings.filterwarnings(\"ignore\",category=UserWarning)\n",
    "# import pretty print\n",
    "import pprint \n",
    "pp = pprint.PrettyPrinter(indent=4)\n",
    "\n",
    "# import external function for notebook\n",
    "from notebook_helpers import get_full_data, show_data, save_data, get_dir_path, set_up_demo_inputs\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "72c0030d-8b0d-4ca9-84e7-370af1ecc8f4",
   "metadata": {},
   "outputs": [],
   "source": [
    "# CARS imports\n",
    "\n",
    "# Applications\n",
    "from cars.applications.application import Application\n",
    "from cars.applications.grid_generation import grid_correction\n",
    "from cars.applications.sparse_matching import sparse_matching_tools\n",
    "\n",
    "# Pipelines\n",
    "import cars.pipelines.sensor_to_dense_dsm.sensor_dense_dsm_constants as sens_cst\n",
    "from cars.pipelines.sensor_to_dense_dsm import sensors_inputs\n",
    "from cars.pipelines.sensor_to_dense_dsm import dsm_output\n",
    "\n",
    "# Conf, core, orchestrator\n",
    "from cars.core import cars_logging\n",
    "from cars.core import inputs, preprocessing\n",
    "from cars.core.utils import safe_makedirs\n",
    "from cars.orchestrator import orchestrator\n",
    "from cars.core.utils import make_relative_path_absolute"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e345f961-0e80-453f-afe6-b15a7f026f28",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "---------\n",
    "\n",
    "## Inputs/Outputs"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b35429a0-0a8e-4a83-ae4f-506777fbfdb9",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "### Define outputs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "750644f6-96d5-45d9-979c-980183ad4e9a",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "# Modify with your own output path if needed\n",
    "output_dir = os.path.join(get_dir_path(), \"output_notebook\")\n",
    "print(output_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e87bf76c-08c6-4e89-84a5-0657fe6f08c6",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "### Define inputs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2f138b48-7fb7-4286-9cba-779cd6246212",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "# By default, the notebook use data_gizeh_small.tar.bz2, data_gizeh is available also (performance dependent).\n",
    "# For you own data: Modify input_dir_path and modify all images, geometric models and color file names below\n",
    "input_dir_path = set_up_demo_inputs(\"data_gizeh_small\")\n",
    "\n",
    "inputs_conf = {\n",
    "    \"sensors\": {\n",
    "        \"left\": {\n",
    "            \"image\": os.path.join(input_dir_path, \"img1.tif\"),\n",
    "            \"geomodel\": os.path.join(input_dir_path, \"img1.geom\"),\n",
    "            \"color\": os.path.join(input_dir_path, \"color1.tif\"),\n",
    "            \"no_data\": 0,\n",
    "\n",
    "\n",
    "        },\n",
    "        \"right\": {\n",
    "            \"image\": os.path.join(input_dir_path, \"img2.tif\"),\n",
    "            \"geomodel\": os.path.join(input_dir_path, \"img2.geom\"),\n",
    "            \"no_data\": 0,\n",
    "        },   \n",
    "    },\n",
    "    \"pairing\": [[\"left\", \"right\"]],\n",
    "    \"initial_elevation\": os.path.join(input_dir_path, \"srtm_dir/N29E031_KHEOPS.tif\")\n",
    "}\n",
    "\n",
    "inputs = sensors_inputs.sensors_check_inputs(inputs_conf)\n",
    "pp.pprint(inputs)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d7b6424b-09a6-4c5c-ac55-3f9972a0d5c8",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "### Create orchestrator\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9df071c0-5478-464f-bec4-a08b46d3eeb3",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "# Use sequential mode in notebook\n",
    "orchestrator_conf = {\"mode\": \"sequential\"}\n",
    "cars_orchestrator = orchestrator.Orchestrator(orchestrator_conf=orchestrator_conf, out_dir=output_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ced933d0-15a0-448e-b426-cd3171a609d7",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "---------\n",
    "\n",
    "## First, compute epipolar rectified images "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aa3ea1de",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "epipolar_grid_generation_application = Application(\"grid_generation\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "491611da",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "resampling_application = Application(\"resampling\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fcb9681b",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "sparse_matching_application = Application(\"sparse_matching\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "02b096ba-cdd3-47e5-9778-ce4fa8f3a3c6",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Sensors images generation\n",
    "\n",
    "From input configuration \"inputs\" seen before"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1fc74dcd-eaa7-4bed-b1c9-ae5369e1a0bc",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "_, sensor_image_left, sensor_image_right = sensors_inputs.generate_inputs(inputs)[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "35fa45cc-8969-444a-a524-f51acda84322",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Grid Generation : epipolar grid generation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "604e0309-5615-4320-ba1b-fc0e5f018c39",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "grid_left, grid_right = epipolar_grid_generation_application.run(\n",
    "    sensor_image_left,\n",
    "    sensor_image_right,\n",
    "    orchestrator=cars_orchestrator,\n",
    "    srtm_dir=inputs[sens_cst.INITIAL_ELEVATION],\n",
    "    default_alt=inputs[sens_cst.DEFAULT_ALT],\n",
    "    geoid_path=inputs[sens_cst.GEOID],\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bdc39b9a-b16a-40cb-b7a7-bbe6a9f15127",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Resampling : epipolar images generation\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "249c01e1-7969-4059-9085-64d02ce5ba0a",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "epipolar_image_left, epipolar_image_right = resampling_application.run(\n",
    "    sensor_image_left,\n",
    "    sensor_image_right,\n",
    "    grid_left,\n",
    "    grid_right,\n",
    "    orchestrator=cars_orchestrator,\n",
    "    margins=sparse_matching_application.get_margins()\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "48c814dd-91b9-4274-af24-4b54de1fd52a",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "###  Sparse matching: compute sifts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ca76004f-257e-490e-b9db-3e4b8bc8beae",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "epipolar_matches_left, _ = sparse_matching_application.run(\n",
    "    epipolar_image_left,\n",
    "    epipolar_image_right,\n",
    "    grid_left.attributes[\"disp_to_alt_ratio\"],\n",
    "    orchestrator=cars_orchestrator\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fdd5aece-cacb-4d06-8421-4d0719c9ed18",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Grid correction: correct epipolar grids from sparse matches\n",
    "Find correction to apply, and generate new right epipolar grid"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ddebc80e-0e6e-4701-9aac-b80454b8fe97",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "#### Filter matches"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2a484184-b526-4828-ae47-b950aa387623",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "matches_array = sparse_matching_application.filter_matches(epipolar_matches_left, orchestrator=cars_orchestrator)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3b1edad-5099-4eaa-b0f7-a8180c9e7c51",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "#### Estimate grid correction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8428c2ca-29e7-4651-be90-96ac507f0608",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "grid_correction_coef, corrected_matches_array,_, _, _ = grid_correction.estimate_right_grid_correction(matches_array, grid_right)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ff3538a-8925-450e-a7a1-431ad8ef5dd6",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "#### Correct right grid"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1cf32b1f-d1d8-4df4-aff1-650f33d8cf3f",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "corrected_grid_right = grid_correction.correct_grid(grid_right, grid_correction_coef)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fdadcfb0-0e07-4913-a538-a9131043641a",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Estimate disp min and disp max from sparse matches"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "82c134d6-ca65-4273-84a7-47dbe47a6a22",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "dmin, dmax = sparse_matching_tools.compute_disp_min_disp_max(\n",
    "                    sensor_image_left,\n",
    "                    sensor_image_right,\n",
    "                    grid_left,\n",
    "                    corrected_grid_right,\n",
    "                    grid_right,\n",
    "                    corrected_matches_array,\n",
    "                    orchestrator=cars_orchestrator,\n",
    "                    disp_margin=(\n",
    "                        sparse_matching_application.get_disparity_margin()\n",
    "                    ),\n",
    "                    disp_to_alt_ratio=grid_left.attributes[\"disp_to_alt_ratio\"],\n",
    "                    geometry_loader=triangulation_application.get_geometry_loader(),\n",
    "                    srtm_dir=inputs[sens_cst.INITIAL_ELEVATION],\n",
    "                    default_alt=inputs[sens_cst.DEFAULT_ALT],\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84f3bbb1-20af-43df-8ba6-cf078224fea2",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "#### Compute margins used in dense matching, with corresponding disparity min and max"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c58aaccb-92ca-4f5d-b9b4-7ee5503bee0b",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "dense_matching_application = Application(\"dense_matching\")\n",
    "dense_matching_margins, disp_min, disp_max = dense_matching_application.get_margins(\n",
    "    grid_left, disp_min=dmin, disp_max=dmax)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e8a03294-f604-42b6-9090-d4067b55fd43",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "###  Resampling: generate epipolar images with corrected grids and new margins"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "492093fb-9817-4f58-a7fb-03f6ae7cc31f",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "new_epipolar_image_left, new_epipolar_image_right = resampling_application.run(\n",
    "    sensor_image_left,\n",
    "    sensor_image_right,\n",
    "    grid_left,\n",
    "    corrected_grid_right,\n",
    "    orchestrator=cars_orchestrator,\n",
    "    margins=dense_matching_margins,\n",
    "    optimum_tile_size=(\n",
    "        dense_matching_application.get_optimal_tile_size(\n",
    "            disp_min, \n",
    "            disp_max,\n",
    "            cars_orchestrator.cluster.checked_conf_cluster[\n",
    "                \"max_ram_per_worker\"\n",
    "            ]\n",
    "        )\n",
    "    ),\n",
    "    add_color=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "89073ed3-231f-4386-928d-92c40ede20e1",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Dense Matching: compute disparities with pandora by using two differents"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b1bb4ca-36c9-490c-b20f-eb4987183e45",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Census similarity measure with semi-global matching (default method)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9187b0a1-ffda-445d-a2ca-6305a55a1852",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "dense_matching_census_application = Application(\"dense_matching\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2e2fb226-ce1b-44a0-aa6b-3fce976a3106",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "epipolar_disparity_map_census = dense_matching_census_application.run(\n",
    "    new_epipolar_image_left,\n",
    "    new_epipolar_image_right,\n",
    "    orchestrator=cars_orchestrator,\n",
    "    disp_min=disp_min,\n",
    "    disp_max=disp_max,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "81d8ab7a",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Census similarity measure with semi-global matching (default method)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "03a61233-bdb9-48b5-ba28-1ddf8a63c9f5",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "#### Show full disparity map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f6617882",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "data_disparity_census = get_full_data(epipolar_disparity_map_census, \"disp\")\n",
    "show_data(data_disparity_census)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e08b981e",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### MC-CNN, the similarity measure produced by mc-cnn neural network\n",
    "\n",
    "<a href=\"https://github.com/CNES/Pandora_MCCNN\">MC-CNN algorithm</a> used by <a  href=\"https://github.com/CNES/Pandora\"> Pandora</a> as <a href=\"https://github.com/CNES/Pandora_plugin_mccnn\">plugin</a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f0f8d84f",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "dense_matching_mccnn_application = Application(\"dense_matching\", cfg={\"method\": \"mccnn_sgm\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "31802e6d",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "epipolar_disparity_map_mccnn = dense_matching_mccnn_application.run(\n",
    "    new_epipolar_image_left,\n",
    "    new_epipolar_image_right,\n",
    "    orchestrator=cars_orchestrator,\n",
    "    disp_min=disp_min,\n",
    "    disp_max=disp_max,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5b84779c",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### MC-CNN, the similarity measure produced by mc-cnn neural network\n",
    "\n",
    "<a href=\"https://github.com/CNES/Pandora_MCCNN\">MC-CNN algorithm</a> used by <a  href=\"https://github.com/CNES/Pandora\"> Pandora</a> as <a href=\"https://github.com/CNES/Pandora_plugin_mccnn\">plugin</a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b2d570f7",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "#### Show full disparity map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9f8f32cc",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "data_disparity_mccnn = get_full_data(epipolar_disparity_map_mccnn, \"disp\")\n",
    "show_data(data_disparity_mccnn)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "edf6d076",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Compute two DSM and compare them\n",
    "\n",
    "One  from disparity map computed by Census similarity measure and the other from disparity map from MC-CNN similarity measure"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ef135c4a",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "triangulation_application = triangulation_application = Application(\"triangulation\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "62f918a5",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "conf_outlier_removing_small_components = {\"method\": \"small_components\", \"activated\": True}\n",
    "pc_outlier_removing_small_comp_application = Application(\"point_cloud_outliers_removing\", cfg=conf_outlier_removing_small_components)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a84155d2",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "conf_outlier_removing_small_statistical = {\"method\": \"statistical\", \"activated\": True}\n",
    "pc_outlier_removing_stats_application = Application(\"point_cloud_outliers_removing\", cfg=conf_outlier_removing_small_statistical)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ab5ad725",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "pc_fusion_application = Application(\"point_cloud_fusion\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5f7e60d3",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "conf_rasterization = { \n",
    "    \"method\": \"simple_gaussian\",\n",
    "    \"dsm_radius\": 3,\n",
    "    \"sigma\": 0.3\n",
    "}\n",
    "rasterization_application = Application(\"point_cloud_rasterization\", cfg=conf_rasterization)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "86b31a15-634f-4098-a59e-7759d0a94681",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    " Compute epsg"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c447031-3cf3-487c-802a-676b7e11f68a",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "epsg = preprocessing.compute_epsg(\n",
    "    sensor_image_left, \n",
    "    sensor_image_right,\n",
    "    grid_left,\n",
    "    corrected_grid_right,\n",
    "    triangulation_application.get_geometry_loader(),\n",
    "    orchestrator=cars_orchestrator,\n",
    "    srtm_dir=inputs[sens_cst.INITIAL_ELEVATION],\n",
    "    default_alt=inputs[sens_cst.DEFAULT_ALT],\n",
    "    disp_min=disp_min,\n",
    "    disp_max=disp_max\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "15413f23-f082-4d62-85a9-474b58505bff",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Triangulation : triangulate matches"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71bea67a",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "From census disparity map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2f43a59c-3cea-4ca7-9de5-eab30324211c",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "epipolar_points_cloud_census = triangulation_application.run(\n",
    "    sensor_image_left,\n",
    "    sensor_image_right,\n",
    "    new_epipolar_image_left,\n",
    "    grid_left,\n",
    "    corrected_grid_right,\n",
    "    epipolar_disparity_map_census,\n",
    "    epsg,\n",
    "    orchestrator=cars_orchestrator,\n",
    "    uncorrected_grid_right=grid_right,\n",
    "    geoid_path=inputs[sens_cst.GEOID],\n",
    "    disp_min=disp_min,\n",
    "    disp_max=disp_max,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "485bee63",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Triangulation : triangulate matches"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "691538e3",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "From mccnn disparity map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b39fa548",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "epipolar_points_cloud_mccnn = triangulation_application.run(\n",
    "    sensor_image_left,\n",
    "    sensor_image_right,\n",
    "    new_epipolar_image_left,\n",
    "    grid_left,\n",
    "    corrected_grid_right,\n",
    "    epipolar_disparity_map_mccnn,\n",
    "    epsg,\n",
    "    orchestrator=cars_orchestrator,\n",
    "    uncorrected_grid_right=grid_right,\n",
    "    geoid_path=inputs[sens_cst.GEOID],\n",
    "    disp_min=disp_min,\n",
    "    disp_max=disp_max,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "19e815a1-7c88-44ca-9c16-96ca332e094e",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    " #### Compute terrain bounding box"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bda732cc-b761-4389-a14a-096e76fa89e0",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "current_terrain_roi_bbox = preprocessing.compute_terrain_bbox(\n",
    "    inputs[sens_cst.INITIAL_ELEVATION],\n",
    "    inputs[sens_cst.DEFAULT_ALT],\n",
    "    inputs[sens_cst.GEOID],\n",
    "    sensor_image_left,\n",
    "    sensor_image_right,\n",
    "    new_epipolar_image_left,\n",
    "    grid_left,\n",
    "    corrected_grid_right,\n",
    "    epsg,\n",
    "    triangulation_application.get_geometry_loader(),\n",
    "    resolution=rasterization_application.get_resolution(),\n",
    "    disp_min=disp_min,\n",
    "    disp_max=disp_max,\n",
    "    orchestrator=cars_orchestrator\n",
    ")\n",
    "terrain_bounds, optimal_terrain_tile_width = preprocessing.compute_terrain_bounds(\n",
    "    [current_terrain_roi_bbox],\n",
    "    resolution=rasterization_application.get_resolution()\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ff0dc69-3db3-4e09-bd99-dc8042abfbad",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Transform point cloud to terrain point cloud"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d6db2ff5",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "From census disparity map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "20f2fc50-4ed8-48e7-b324-4dc8f71627fc",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "merged_points_clouds_census = pc_fusion_application.run(\n",
    "    [epipolar_points_cloud_census],\n",
    "    terrain_bounds,\n",
    "    epsg,\n",
    "    orchestrator=cars_orchestrator,\n",
    "    margins=rasterization_application.get_margins(),\n",
    "    optimal_terrain_tile_width=optimal_terrain_tile_width\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7e5f0327",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Transform point cloud to terrain point cloud"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9cbcf479",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "From mccnn disparity map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d9250b9c",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "merged_points_clouds_mccnn = pc_fusion_application.run(\n",
    "    [epipolar_points_cloud_mccnn],\n",
    "    terrain_bounds,\n",
    "    epsg,\n",
    "    orchestrator=cars_orchestrator,\n",
    "    margins=rasterization_application.get_margins(),\n",
    "    optimal_terrain_tile_width=optimal_terrain_tile_width\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9eb9052b-e0d7-4b63-8152-dd8541e23d0f",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Point Cloud Outlier Removing : remove points with small components removing method"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d0e49614",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "From census disparity map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dd0f821b-21df-45b8-85cc-b0550ffe73bb",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "filtered_sc_merged_points_clouds_census = pc_outlier_removing_small_comp_application.run(\n",
    "    merged_points_clouds_census,\n",
    "    orchestrator=cars_orchestrator,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7beb87df",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "From mccnn disparity map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b113e4cb",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "filtered_sc_merged_points_clouds_mccnn = pc_outlier_removing_small_comp_application.run(\n",
    "    merged_points_clouds_mccnn,\n",
    "    orchestrator=cars_orchestrator,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ae05e50b-ac50-4047-b90d-cd3459103fd3",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Point Cloud Outlier Removing: remove points with statistical removing method"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d7db9e12",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "From census disparity map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "688b93bb-7400-424e-837b-82a75e5efb71",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "filtered_stats_merged_points_clouds_census = pc_outlier_removing_stats_application.run(\n",
    "    filtered_sc_merged_points_clouds_census,\n",
    "    orchestrator=cars_orchestrator,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14551b8a",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "From mccnn disparity map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ca26b00c",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "filtered_stats_merged_points_clouds_mccnn = pc_outlier_removing_stats_application.run(\n",
    "    filtered_sc_merged_points_clouds_mccnn,\n",
    "    orchestrator=cars_orchestrator,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5eb943f0-cea6-4774-b159-0c63e8a2512f",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Rasterization : rasterize point cloud"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "19a9231f",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "From census disparity map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2064961e-b868-4007-b8fc-3f5e1af614a4",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "dsm_census = rasterization_application.run(\n",
    "    filtered_stats_merged_points_clouds_census,\n",
    "    epsg,\n",
    "    orchestrator=cars_orchestrator\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ce1420e",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "From mccnn disparity map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "00332236",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "dsm_mccnn = rasterization_application.run(\n",
    "    filtered_stats_merged_points_clouds_mccnn,\n",
    "    epsg,\n",
    "    orchestrator=cars_orchestrator\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16fdb7f8-93e0-46a1-9075-6a453c1ad734",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Show DSM\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "53ea4039",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "From census disparity map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "24f618b3",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "data_dsm_census = get_full_data(dsm_census, \"hgt\")\n",
    "show_data(data_dsm_census, mode=\"dsm\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "38b1fa72",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Show DSM\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ced3b826",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "From mccnn disparity map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "34dd7d4d-d40d-4ac4-8da9-c1e8d8fa886e",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "data_dsm_mccnn = get_full_data(dsm_mccnn, \"hgt\")\n",
    "show_data(data_dsm_mccnn, mode=\"dsm\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db85381f-4a62-40a4-915c-933043f22491",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Show ortho image"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e6c3eb7",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "From census disparity map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c4f00e06-08c0-4c55-ac23-5d29359bdd07",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "data_ortho_census = get_full_data(dsm_census, \"img\")[..., 0:3]\n",
    "show_data(data_ortho_census, mode='image')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27708039",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Show ortho image"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4fe28a8c",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "From mccnn disparity map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4fae5419",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "data_ortho_mccnn = get_full_data(dsm_mccnn, \"img\")[..., 0:3]\n",
    "show_data(data_ortho_mccnn, mode='image')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7f27dbd6",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Thanks for your attention ! \n",
    "\n",
    "### Questions ?"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Diaporama",
  "kernelspec": {
   "display_name": "cars-0.7.0",
   "language": "python",
   "name": "cars-0.7.0"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.4"
  },
  "toc-showtags": false,
  "vscode": {
   "interpreter": {
    "hash": "41cef69d03cc8776cafbc952f48c668dc1c9a7aacaa94fd78ea743f1bb88c986"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}