can anybody help me? I am trying to use Scrapy to collect the results for searching for a keyword (eg. 朝鲜）on this forum. When I use the search bar, I get a list of results, but then trying to scrape these with Scrapy based on the result page url (http://bbs1.people.com.cn/quickSearch.do?field=title&threadtype=1&content=朝鲜) doesn't seem to work. It usually gives me a time out. Is there any way to resolve this? If I try to scrape the whole page of search results, using the address at the top. Page looks like: But then when I return to it: I can't figure out why....Huge thanks in advance to anybody who can help! Continue reading...
