[Python] Stopping Scrapy from fetching enqueued requests after timout or Keyboard Interrupt

I am trying to make a web crawler with Scrapy which fetches some html pages and saves them via default Request callback i.e. parse()

The thing is, I want the spider to stop crawling pending or enqueued requests and run the close() method shortly after the set timeout or Keyboard interrupt to save the crawled data.

But, as seen in this particular log, the timeout was set to 1min, but it ran 7min to crawl pending requests.

Is there any way I can stop the spider from crawling pending requests?

Continue reading...