[Python] How to get/crawl all the reviews of apps on play store or app store, I am just...

I am using python for this. I am going to play store url for an app then parse the body into tree, extracting data with xpath

reviews = parsed.xpath('//div[@class="single-review"]//div[@class="review-body with-review-wrapper"]')


and then wrting it in a file

reviewFile = rev.sub('[^A-Za-z0-9.,\' ]+', '', reviewFile.text_content().replace(' Full Review ','').strip())
print('writing reviewFile'+reviewFile)


But with this I am able to get only first 40 reviews on the google play store instead of all reviews.

After first 40 reviews are displayed Google Play starts to load more reviews by pinging http://play.google.com/store/getreviews

Continue reading...