Skip to main content
Here’s the clean Markdown version: Scrapy is a powerful web scraping library that supports proxy integration through meta parameters or custom middleware.

Method 1: Using Meta Parameters

Method 2: Custom Middleware

Update the middlewares.py with:
Add the middleware to DOWNLOADER_MIDDLEWARES in settings.py:
Note that, when using proxies with the Scrapy, always use host http and port 65534, which will work perfectly.