Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prepend HTTPS to Crawl Start URL #2177

Merged
merged 7 commits into from
Nov 24, 2024
Merged

Prepend HTTPS to Crawl Start URL #2177

merged 7 commits into from
Nov 24, 2024

Conversation

ikreymer
Copy link
Member

@ikreymer ikreymer commented Nov 23, 2024

Part of fix for #2167:

  • Updates validation regex to accept URLs without prefix, not just if starting with www.
  • prepends https:// if URL is considered valid but missing scheme
  • prepend https:// to each URL in URL list, also run validation onblur for URL list

@ikreymer
Copy link
Member Author

ikreymer commented Nov 23, 2024

Might be good enough for 1.12.2, as this addresses most common issue, can handle list of urls in a follow-up.

Copy link
Member

@tw4l tw4l left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works well (with caveat about list of pages), just one inline comment to address

@@ -996,6 +1008,18 @@ https://archiveweb.page/guide`}
const text = msg("Please enter a valid URL.");
inputEl.helpText = text;
inputEl.setCustomValidity(text);
inputEl.setCustomValidity(text);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This appears to be a duplicate of the line above - maybe a mistake?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, good catch!

@ikreymer
Copy link
Member Author

ikreymer commented Nov 23, 2024

Decided to go ahead and just add URL list prepending as well for completion!
Also checks URL list onblur, which previously did not, so copy&pasting a URL list should result in validation.

@ikreymer ikreymer requested a review from tw4l November 23, 2024 18:27
@ikreymer ikreymer merged commit ca012a4 into 1.12.2-release Nov 24, 2024
2 checks passed
@ikreymer ikreymer deleted the prepend-https branch November 24, 2024 19:53
@ikreymer ikreymer mentioned this pull request Nov 25, 2024
ikreymer added a commit that referenced this pull request Nov 26, 2024
Merge 1.12.2 release changes into main, includes:
- Collection replay full refresh on metadata / archived items (#2176)
- Fix for self-registration default org (#2178)
- Prepend missing https in start URL (#2177)
- Updated billing to support free trial messaging (#2179)

---------

Co-authored-by: sua yoo <[email protected]>
Co-authored-by: Henry Wilkinson <[email protected]>
Co-authored-by: sua yoo <[email protected]>
Co-authored-by: SuaYoo <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants