Am using Selenium + python to scrap a page which has infinite scroll (basically scroll till max first 500 results are shown) Using below code, am able to scroll to bottom of the page. Now i want to stop when further scrolling doesn't fetches any content. (say, page only has 200 results, i don't want to keep on scrolling assuming max 500 result) driver = webdriver.Firefox() driver.get(url) driver.execute_script("window.scrollTo(0, document.body.scrollHeight);") I tried accessing window.pageYOffset but it's coming as None always. Continue reading...
