asyncio

New in version 2.0.

Scrapy has partial support asyncio. After you install the asyncio reactor, you may use asyncio and asyncio-powered libraries in any coroutine.

Warning

asyncio support in Scrapy is experimental. Future Scrapy versions may introduce related changes without a deprecation period or warning.

Installing the asyncio reactor

To enable asyncio support, set the TWISTED_REACTOR setting to 'twisted.internet.asyncioreactor.AsyncioSelectorReactor'.

If you are using CrawlerRunner, you also need to install the AsyncioSelectorReactor reactor manually. You can do that using install_reactor():

install_reactor('twisted.internet.asyncioreactor.AsyncioSelectorReactor')