Crawling Websites with Infinite Scroll

[n-ohbayashi]

Hi.

We are conducting website crawling using NORCONEX.
As more websites adopt infinite scroll, we are looking for ways to crawl such sites.

For example, would it be possible to use a web driver like Selenium to scroll down to the bottom of the page before crawling?
Do you have any suggestions for effective methods?

Thanks.