Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate tile2net task #91

Open
wants to merge 11 commits into
base: master
Choose a base branch
from
Open

Integrate tile2net task #91

wants to merge 11 commits into from

Conversation

annehaley
Copy link
Collaborator

@annehaley annehaley commented Dec 2, 2024

This PR adds an orthoimagery (3.6 GB raster) dataset to the Boston floods use case and adds a new simulation type which runs tile2net on a selected imagery dataset. The "Segment Curbs" task downloads tiles of raster data in a format expected by tile2net and then runs the tile2net generate and inference functions. The output files generated by the inference step are then interpreted as new Datasets to add to the current project.

The newly-created vector datasets are sent through our existing vector conversion process and are made available to view on the map. The first Dataset is the "polygons" generated by the inference, which represent the segmentation of curbs, crosswalks, and other walkable paths in the imagery. The second Dataset is the "network" generated by the inference, which is a processed version of the "polygons" set, consisting of semi-connected line segments. An avenue for future work would be to interpret the "network" set into our own Network representation. However, due to the nature of walkable urban systems, the resulting Network would be very disconnected and not viable for meaningful GCC analysis.

Results

  • Clifton Park set got poor results, took 2 mins for a 144 MB file (not trained for this area, not very walkable)
    image

  • Boston set got good results, took 2 hours 10 mins for 3.6 GB file (trained for this area, semi-walkable)
    image

  • Capitol Hill set got fair results, took 8 mins for a 1 GB file (trained for this area, highly walkable park)
    image

@annehaley annehaley requested a review from jjnesbitt December 10, 2024 21:08
const colormapRange = options?.colormap?.range || defaultStyle?.data_range;
const nodataValue = options.nodata || defaultStyle?.transparency_threshold;

const tileParams: Record<string, string> = {
projection: "EPSG:3857",
band: "1",
// band: "1",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What should be done with this?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pointing this out. I had commented this out for the (now default) case of using no colormap for a raster, and I didn't test out the colormap functionality. We do need this argument when applying a colormap, so I added this back in for the colormap case in 5511b25.

@@ -48,6 +48,7 @@ def add_styling(geojson_data, style_options):

def create_raster_map_layer(file_item, style_options):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function is now very large, with lots of code nested under context managers, try/catch, etc. I think this should be split up a bit.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotcha, I split this function in f493c8d

new_map_layer.cloud_optimized_geotiff.save(
cog_raster_path, ContentFile(cog_raster_file.read())
try:
source = large_image.open(raw_data_path)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this function could use some comments as well. It's not really clear to someone reading the code, what the intent of these code blocks are.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, I added comments in f493c8d

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is this file invoked? I don't see it used anywhere.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, I don't invoke this code anywhere. This is a script that I invoked manually to composite separate raster files into one before using the output file for ingest. I want to keep this code somewhere, though, in case we want to incorporate it into the ingest process more formally later. But that will be dependent on our modelling decisions relating to composite rasters.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants