Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better error handling in tom_alerts #1060

Open
15 tasks
Tracked by #832
jchate6 opened this issue Oct 1, 2024 · 1 comment
Open
15 tasks
Tracked by #832

Better error handling in tom_alerts #1060

jchate6 opened this issue Oct 1, 2024 · 1 comment
Labels
Data Services Data Services maintenance Non-user facing updates to clean code and make developer life easier. User Issue Raised by a user

Comments

@jchate6
Copy link
Contributor

jchate6 commented Oct 1, 2024

Some of these issues may have been resolved separately
see #640

As a TOM user, I would like errors in the various broker forms to be displayed with an error message, rather than a 500 / stacktrace on the page. Right now, if you type in the wrong values, or if there is an error in your query to the broker service, it will result in a stacktrace/500 page rather than simple showing you the error on your current page. We need to engineer it better so that these errors are captured and reported back to the user in a helpful way. Some of the errors we may be able to just protect against in the first place, like different formatting errors within some of the fields.

Some common error correcting we should do in all brokers:

  1. Accept multiple date formats, and convert to the one needed by the broker (usually MJD)
  2. Add validation range to all numeric fields, either from broker specific ranges, or just sane values.
  3. Ensure we check if cleaned data fields exist before referencing them, since they will not be present if the field failed validation.

Tasks:

  • tom_fink: max window of 180 minutes should be validated in form
  • tom_fink: max radius updated to 18,000 degrees in help text and validation
  • tom_fink: validate datesearch format
  • alerce: fix clean to check if field exists before referencing (radius)
  • alerce: accept other date formats and convert to MJD on the query
  • alerce: validation on ra, dec, radius ranges
  • anteres: accept other date formats and convert to MJD on the query
  • anteres: add validation ranges to all numeric fields and protect against missing fields in clean
  • anteres: fix cleaned data to only reference if fields exist
  • lasair: add more in depth example of SQL query, so users can understand format
  • lasair: set the default 'limit' : 1000 in the sql_query (see Below)
  • lasair: validate ra/dec/radius ranges
  • tns: validate ra/dec/radius ranges
  • tns: validate target name to ensure no prefix
  • Add error handling such that if the alert query fails for whatever reason, you get an error message rather than a stacktrace
@jchate6 jchate6 added Data Services Data Services maintenance Non-user facing updates to clean code and make developer life easier. labels Oct 1, 2024
@github-project-automation github-project-automation bot moved this to Triage in TOM Toolkit Oct 1, 2024
@talister
Copy link

talister commented Oct 1, 2024

I would add one for lasair: with the default 'limit' : 1000 in the sql_query part of lasair.py, queries will result in too many objects and when the joined, comma-separated list is fed to the api/objects API endpoint further down, it dies in an (uncaught) HTTP 414 error which then produces the following stacktrace:

JSONDecodeError at /alerts/query/1/run/

Expecting value: line 1 column 1 (char 0)

Request Method: 	GET
Request URL: 	http://127.0.0.1:8000/alerts/query/1/run/
Django Version: 	4.2.16
Exception Type: 	JSONDecodeError
Exception Value: 	

Expecting value: line 1 column 1 (char 0)

Exception Location: 	C:\Users\liste\venv\fomo_venv\lib\site-packages\requests\models.py, line 978, in json
Raised during: 	tom_alerts.views.RunQueryView
Python Executable: 	C:\Users\liste\venv\fomo_venv\Scripts\python.exe
Python Version: 	3.10.11
Python Path: 	

Server time: 	Tue, 01 Oct 2024 10:21:34 +0000

Traceback (most recent call last):
  File "C:\Users\liste\venv\fomo_venv\lib\site-packages\requests\models.py", line 974, in json
    return complexjson.loads(self.text, **kwargs)
  File "C:\Users\liste\AppData\Local\Programs\Python\Python310\lib\json\__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "C:\Users\liste\AppData\Local\Programs\Python\Python310\lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Users\liste\AppData\Local\Programs\Python\Python310\lib\json\decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None

During handling of the above exception (Expecting value: line 1 column 1 (char 0)), another exception occurred:
  File "C:\Users\liste\venv\fomo_venv\lib\site-packages\django\core\handlers\exception.py", line 55, in inner
    response = get_response(request)
  File "C:\Users\liste\venv\fomo_venv\lib\site-packages\django\core\handlers\base.py", line 199, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "C:\Users\liste\venv\fomo_venv\lib\site-packages\django\core\handlers\base.py", line 362, in process_exception_by_middleware
    response = middleware_method(request, exception)
  File "C:\Users\liste\venv\fomo_venv\lib\site-packages\tom_common\middleware.py", line 29, in process_exception
    raise exception
  File "C:\Users\liste\venv\fomo_venv\lib\site-packages\django\core\handlers\base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "C:\Users\liste\venv\fomo_venv\lib\site-packages\django\views\generic\base.py", line 104, in view
    return self.dispatch(request, *args, **kwargs)
  File "C:\Users\liste\venv\fomo_venv\lib\site-packages\django\views\generic\base.py", line 143, in dispatch
    return handler(request, *args, **kwargs)
  File "C:\Users\liste\venv\fomo_venv\lib\site-packages\django\views\generic\base.py", line 226, in get
    context = self.get_context_data(**kwargs)
  File "C:\Users\liste\venv\fomo_venv\lib\site-packages\tom_alerts\views.py", line 232, in get_context_data
    alert_query_results = broker_class.fetch_alerts(deepcopy(query.parameters))
  File "C:\Users\liste\venv\fomo_venv\lib\site-packages\tom_alerts\brokers\lasair.py", line 189, in fetch_alerts
    obj_results = obj_response.json()
  File "C:\Users\liste\venv\fomo_venv\lib\site-packages\requests\models.py", line 978, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)

Exception Type: JSONDecodeError at /alerts/query/1/run/
Exception Value: Expecting value: line 1 column 1 (char 0)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Data Services Data Services maintenance Non-user facing updates to clean code and make developer life easier. User Issue Raised by a user
Projects
Status: Backlog
Development

No branches or pull requests

2 participants