Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance UX and add features to the Callback pages #1855

Merged
merged 29 commits into from
Sep 24, 2024

Conversation

whabanks
Copy link
Contributor

@whabanks whabanks commented May 29, 2024

Summary | Résumé

This PR contains UX and feature enhancements to the callbacks pages.

  • Additional validation:
    1. That the URL entered is valid (is https:// and passes the URL format check in validators)
    2. That the URL is reachable and returns http 200 by performing a small ping to their callback service
  • Instead of clearing the form and saving to delete a callback config, a dedicated delete link is now available with a confirmation prompt before deletion
  • A "Test response time" button was added so users can easily gauge how long their service takes to respond
  • Footer form controls are sticky, so users do not need to scroll down to the bottom of the page to test response times.

Test instructions | Instructions pour tester la modification

  1. Create a temporary callback url: https://webhook.site/ or https://beeceptor.com/
  2. Navigate to the api callbacks page

New validation

Test 1 - Valid callback

  1. Use the URL you created earlier and create an API callback
  2. Note that when you save a callback configuration a message with the response time is included
  3. Send some notifications and make sure callbacks are working

Test 2 - Invalid URL

  1. Use the URL you created earlier but make it http:// instead of https://
  2. Note the error message: Enter a URL that starts with https://
  3. Note the hint text: Must start with https://

Test 3 - Valid URL but server doesn't exist

  1. Modify the callback URL so it points to an invalid endpoint
  2. Note the error message: Check your service is running and not using a proxy we cannot access
  3. Note the hint text: Your service must be running and reachable from the internet.

Delete button

  1. Delete callback you created
  2. Note the confirmation message before deletion
  3. Note that during confirmation the Submission controls are not present (Save, Test response time, delete)
  4. Confirm the deletion
  5. Note you are returned to the API page that links to API keys, callbacks, etc.

Test response time button

Test 1 - Button visibility

  1. Navigate to the callback config page
  2. Note that there is no Test response time button
  3. Create a callback again then navigate back to the callback page
  4. Note that when returning to the callback config page, the Test response time button is visible

Test 2 - Response time button

  1. With a callback configured - click the Test response time button
  2. Note the message indicating the response time
  3. Note that you aren't taken back to the API integration page but remained on the Callback config page

Test 3 - Differing URLs

  1. With a callback already configured - change the URL of the callback but do not click save
  2. Test the response time
  3. Note that the URL input into the form was tested

Test 4 - Validation

  1. All the validation cases from the above tests should also apply to the Test response time button.

Inbound SMS callbacks page

To be able to access the Inbound SMS Callbacks page we first need to enable the Receive inbound SMS setting for the service.

  1. Add a row to the inbound_numbers table:
INSERT INTO inbound_numbers (id,"number",provider,service_id,active,created_at,updated_at) VALUES
	 ('2a7b8671-3b71-4ba0-abcf-df880802b0f9' ,'<any phone number>','pinpoint','<your service id',true,'2024-09-09 00:00:00.000',NULL);

  1. In the service_permissions add a row to give your service the inbound_sms permission
INSERT INTO service_permissions (service_id,"permission",created_at) VALUES
	 ('<your service id>','inbound_sms','2024-09-09 15:57:18.958');

  1. Clear all service-.* keys in Redis
  2. Navigate to the service settings and verify that the Receive inbound SMS setting is turned on
  3. Navigate back to API integration -> Callbacks
  4. Note you can choose between Delivery receipts and Received text messages
  5. Choose Received text messages and repeat the validation & test response time button tests to ensure feature parity between the two pages.

- Added the [validators](https://yozachar.github.io/pyvalidators/stable/) lib to the project
- Callback URLs are now checked for validity
- Callback URLs are now tested to determine if they are reachable before saving the callback url
- Validate that callback urls return 200 responses
app/main/views/api_keys.py Fixed Show fixed Hide fixed
Copy link

whabanks and others added 3 commits June 6, 2024 14:21
- Update error messages
- Add logic for catching 5xx and 4xx errors and raise the correct validation error + logging
- Fixed ServiceReceiveCallbackMessagesForm validation
("http://not_https.com", "1234567890", "Must be a valid https URL"),
("https://test.com", "123456789", "Must be at least 10 characters"),
("https://example.com", "", None, "This cannot be empty"),
("http://not_https.com", "1234567890", None, "Enter a URL that starts with https://"),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thoughts on testing these validation messages in Cypress, and instead test that we're logging appropriately here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah its a good question. We could definitely test the error messages in Cypress, although you do already have it done here. I think if we were building up this feature from scratch we would probably have created a cypress test suite, but where we are just tacking something on, maybe this is fine?

I like the idea of testing the logging, especially if those logs' formats are tied to other parts of the system like alarms. Testing logging is pretty easy, I had to do it for system status here.

@whabanks whabanks marked this pull request as ready for review June 11, 2024 16:26
app/main/views/api_keys.py Fixed Show fixed Hide fixed
@whabanks whabanks requested review from andrewleith and jzbahrai and removed request for andrewleith June 11, 2024 17:26
Copy link
Member

@andrewleith andrewleith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this looks great overall, just the question on if we can actually enforce a particular response at setup time without advising the clients of this in advance.

validate_callback_url(field.data, form.bearer_token.data)


def validate_callback_url(service_callback_url, bearer_token):
Copy link
Member

@andrewleith andrewleith Jun 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed in slack, this might be an issue since here it expects the callback servers to respond with a 200 when we call them with {"health_check": "true"}. If we want to do this we would probably have to advertise this in the docs so users are aware.

("http://not_https.com", "1234567890", "Must be a valid https URL"),
("https://test.com", "123456789", "Must be at least 10 characters"),
("https://example.com", "", None, "This cannot be empty"),
("http://not_https.com", "1234567890", None, "Enter a URL that starts with https://"),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah its a good question. We could definitely test the error messages in Cypress, although you do already have it done here. I think if we were building up this feature from scratch we would probably have created a cypress test suite, but where we are just tacking something on, maybe this is fine?

I like the idea of testing the logging, especially if those logs' formats are tied to other parts of the system like alarms. Testing logging is pretty easy, I had to do it for system status here.

Copy link

whabanks and others added 6 commits August 12, 2024 11:38
- We only want to verify that there's a service running at the specified callback URL.
- Additionally we are sending a payload the service won't understand how to response to so a 5xx is likely
- There is now a dedicated delete button to remove a callback configuration instead of needing to empty the form and click save to delete a config
- Users will receive a confirmation banner before deletion occurs
- Saving, creating, and deleting a callback url now provide the user with a confirmation message that the operation succeeded
- Added a button to the callback config page so users can see the response times of their callback services
- Added some translation stubs
- The ping to their service takes place as part of the validation
app/main/views/api_keys.py Fixed Show fixed Hide fixed
app/main/views/api_keys.py Fixed Show fixed Hide fixed
@whabanks whabanks changed the title Add basic validation to callbacks URLs Enhance UX on Callback pages Sep 10, 2024
@whabanks whabanks changed the title Enhance UX on Callback pages Enhance UX and add features to the Callback pages Sep 10, 2024
whabanks and others added 2 commits September 11, 2024 16:11
- Added placeholder FR translations
- Removed validation that allowed the form to be submitted while empty, this was how callbacks were deleted previously
- Added additional check to the make format task
- Updated EN translations
- Formatting
application.py Outdated
@@ -16,7 +16,7 @@

application = Flask("app")
application.wsgi_app = ProxyFix(application.wsgi_app) # type: ignore
xray_recorder.configure(service='Notify-Admin')
xray_recorder.configure(service="Notify-Admin")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a formatting change here.

Copy link
Member

@andrewleith andrewleith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - works as expected! Will test some scenarios in staging once its there.

timeout=2,
)

g.callback_response_time = response.elapsed.total_seconds()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was going to say I was concerned that using the flask global object might leak the value here to other users, but just learned that values stored here are request-specific and last for only one request. Good to know!

- Add new message content
app/main/views/api_keys.py Fixed Show fixed Hide fixed
app/main/views/api_keys.py Fixed Show fixed Hide fixed
@whabanks whabanks merged commit 5d71ce6 into main Sep 24, 2024
10 checks passed
@whabanks whabanks deleted the task/verify-callback-url branch September 24, 2024 20:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants