Web Exploration Engine 3 - Redis Cache

Features in Third Release

What's Changed

• Implemented integration for on-page SEO. PR #260 by @MignonErasmus
• Updated Help documentation. PR #259 by @Johane-B
• Applied hotfix for integration tests. PR #265 by @siyamthandandlovu
• Merged development into the zero-shot classification branch. PR #269 by @siyamthandandlovu
• Merged development into the service time branch. PR #270 by @siyamthandandlovu
• Added zero-shot industry classification feature. PR #264 by @Johane-B
• Improved service performance. PR #256 by @katnicole14
• Updated PDF download feature. PR #273 by @Johane-B
• Fixed UI issues related to on-page SEO. PR #271 by @MignonErasmus

Full Changelog: v1.1.0...v1.2.0

Web Exploration Engine 2 - Pipe and Filter

Features in Second Release

What's Changed

The latest version of the Web Exploration Engine 2, Cache-free Version, includes several significant updates and improvements. Here's a summary of the key changes:

• Improved Mobile View for Piechart Legend: The mobile view for the piechart legend has been enhanced by @MignonErasmus. This improvement aims to provide a better user experience when accessing the Web Exploration Engine on mobile devices.

• Configuration and Integration Tests: @siyamthandandlovu has contributed configuration and integration tests to ensure the robustness and reliability of the Web Exploration Engine.

• Enhanced Export Functionality: @CaitMS has added an export dropdown to the results pages, offering users more flexibility in managing and exporting their data.

• New Features: Several new features have been introduced, including an address and contact info scraper, a screenshot homepage, the ability to download individual reports, a summary report, a website comparison page, and more. These features have been developed by various contributors, such as @Johane-B, @katnicole14, and @MignonErasmus.

• Infrastructure and Configuration Updates: @MignonErasmus has made significant contributions to the Web Exploration Engine, including the configuration of Docker images for the frontend and webscraper, the addition of a footer, the integration of address contact screenshot, and more.

• Code Refactoring and Development: @CaitMS, @Johane-B, and @katnicole14 have been involved in code refactoring, development, and the addition of new functionalities to the Web Exploration Engine.

For a comprehensive list of all the changes and updates, you can refer to the Full Changelog.

These updates reflect the ongoing commitment to enhancing the functionality and user experience of the Web Exploration Engine 2, ensuring that it remains a cutting-edge tool for web exploration and analysis.

Web Exploration Engine 1

Features in First Release

1. Advanced Industry Classification and Analysis:

   • Verify if domain names are classified in the intended industry to reach the correct target market.
   • View a summary of correctly classified domain names for an overview of accuracy.
   • See a list of incorrectly classified domains to identify necessary corrections.
   • API users can check if industry classification matches the intended target market.

2. Filterable Results of Multiple URLs:

   • Enter multiple URLs to be scraped simultaneously.
   • View results as they become available for quick access.
   • Search for specific URLs within the results page for easy retrieval.
   • Apply filters to refine results based on specific criteria, aiding analysis.

3. Comprehensive Summary Reports for Multiple URLs:

   • Enter multiple URLs to be scraped simultaneously.
   • View a summary report of all results and analysis.
   • View individual reports for each URL for detailed information.
   • View a summary of industry classifications for all scraped domains.
   • View a summary of the status of all scraped domains.
   • View a domain-match summary for all scraped domains.

Development

• Configuration and Repository Updates:

  • Multiple updates to configuration and repository settings by @siyamthandandlovu #17 #18

• New Features:

  • Authentication backend implemented by @CaitMS #29
  • Splash UI added by @MignonErasmus #36
  • Robots.txt feature added by @katnicole14 #24
  • Industry classification scraping feature by @katnicole14 #44
  • Image scraping feature by @Johane-B #43
  • Scrape status feature by @Johane-B #42
  • Web status integration by @CaitMS #50
  • Error message frontend by @MignonErasmus #55
  • Filter UI on the home page by @MignonErasmus #59
  • Authentication integration by @CaitMS #58
  • Unit tests added by @katnicole14 #66
  • Multiple URLs scraping for industry by @Johane-B #101
  • Summary industry classifications by @Johane-B #113
  • Scrape result page by @MignonErasmus #114
  • Pagination UI improvements by @MignonErasmus #118
  • Domain match feature by @Johane-B #123
  • Results filtering by @Johane-B #169
  • Info popups by @CaitMS #175

• UI/UX Improvements:

  • Improved login and create account responsiveness by @MignonErasmus #77
  • Added text to the hero section by @siyamthandandlovu #125
  • Result cards converted to table format by @MignonErasmus #127
  • Improved summary reports by @Johane-B #152
  • Improved scrape results loading by @MignonErasmus #153
  • Fixed transition bug by @MignonErasmus #173

• Documentation and Guides:

  • Contributing guide created by @siyamthandandlovu #20
  • Updated README.md by @CaitMS #61
  • Demo 1 documentation by @siyamthandandlovu #64

• Bug Fixes and Hotfixes:

  • Fixed classification hotfix by @katnicole14 #92
  • Fixed false parked issue by @Johane-B #167
  • Reporting bug fixes by @CaitMS #163

• Miscellaneous Updates:

  • Initial commit and added email verification to the signup page by @MignonErasmus #51
  • Development updates by @CaitMS #52 #62
  • Code coverage updates by @siyamthandandlovu #63 #65
  • Updated main.yaml to add codecov job by @siyamthandandlovu #63
  • Hotfix for the FAQ page by @siyamthandandlovu #126
  • Improved dashboard and standardization by @MignonErasmus #150
  • Metadata added by @CaitMS #151
  • Integration tests by @katnicole14 #170
  • Merge of dev to master for Demo 2 by @CaitMS #164

Contributors

• @siyamthandandlovu made their first contribution in #17
• @CaitMS made their first contribution in #29
• @MignonErasmus made their first contribution in #36
• @katnicole14 made their first contribution in #24
• @Johane-B made their first contribution in #43

Full Changelog: View the full changelog