Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Log intersection errors #122

Merged
merged 5 commits into from
Nov 1, 2021

Conversation

jpolchlo
Copy link
Contributor

Pull request checklist

Please check if your PR fulfills the following requirements:

  • Make sure you are requesting to pull a topic/feature/bugfix branch (right side). Don't request your master!
  • Make sure you are making a pull request against the develop branch (left side). Also you should start your branch off our develop.
  • Check the commit's or even all commits' message styles matches our requested structure.
  • Check your code additions will fail neither code linting checks nor unit test.

Pull request type

Please check the type of change your PR introduces:

  • Bugfix
  • Feature
  • Code style update (formatting, renaming)
  • Refactoring (no functional changes, no api changes)
  • Build related changes
  • Documentation content changes
  • Other (please describe):

What is the current behavior?

Current behavior is to crash upon encountering a failing geometry intersection. Many instances of this were solved by #117, but in the event that there are other classes of geometry error that are not covered by buffering the input, crashes will still occur.

Issue Number: #118

What is the new behavior?

This PR introduces a new pathway wherein geometries are wrapped in a Validated structure upon import, and errors in geometry intersection can be propagated down the pipeline and reported in the output. It's not clear if there are any other places in the code that can generate JTS errors, but there are not currently explicit catches for other (currently unknown) risky operations.

Does this introduce a breaking change?

  • Yes
  • No

Other information

@echeipesh echeipesh force-pushed the feature/log-intersection-errors branch from cfef1fe to 0de0fa5 Compare October 28, 2021 17:53
jpolchlo and others added 4 commits October 28, 2021 16:35
…try using UDFs; may be useful refernence if buffer(0) is deemed inappropriate
… errors in the output; this runs, but we have already cleaned out bad geoms in the first instance using buffer(0)

Handle bad WKB inputs; must discard because some feature spark sql expressions can reference geometry
@echeipesh echeipesh force-pushed the feature/log-intersection-errors branch from 8f4d731 to 9424457 Compare October 28, 2021 20:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants