Spider closed (finished) 沒完成
2020-12-29 01:45:47 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 309977,
'downloader/request_count': 609,
'downloader/request_method_count/GET': 609,
'downloader/response_bytes': 1549878,
'downloader/response_count': 609,
'downloader/response_status_count/200': 333,
'downloader/response_status_count/302': 276,
'elapsed_time_seconds': 303.197538,
'finish_reason': 'finished',
'finish_time': datetime.datetime(2020, 12, 28, 17, 45, 47, 574530),
'item_scraped_count': 275,
'log_count/INFO': 13,
'request_depth_max': 58,
'response_received_count': 609,
'scheduler/dequeued': 609,
'scheduler/dequeued/memory': 609,
'scheduler/enqueued': 609,
'scheduler/enqueued/memory': 609,
36%|███▌ | 36/100 [00:00<00:00, 358.73it/s]
2020-12-29 01:45:47 [scrapy.core.engine] INFO: Spider closed (finished)
我猜測是scrapy的robot協議沒關閉吧?
settings.py中
1 2 |
|
相關文章
- Scrapy框架-Spider框架IDE
- PlayMaker的特殊事件FINISHED事件
- [求助]SQLException: ResultSet is closedSQLException
- 解決okhttp報java.lang.IllegalStateException: closed,java.lang.IllegalStateException: closedHTTPJavaException
- 3.2.2 Opening a Closed DatabaseDatabase
- java.lang.IllegalStateException: closedJavaException
- 33-TTL Controller for Finished ResourcesController
- svn“Previous operation has not finished; run 'cleanup' if it was interrupted
- NBU恢復Oracle通道完成後RMAN沒有進度Oracle
- Scrapy框架的使用之Spider的用法框架IDE
- svn報錯“Previous operation has not finished; run 'cleanup' if it was int
- Scrapy框架-Spider和CrawlSpider的區別框架IDE
- 爬蟲框架Webmagic原始碼分析之Spider爬蟲框架Web原始碼IDE
- scrapy 單檔案啟動單個spiderIDE
- 「蜘蛛」來了!耶魯大學11名學生標註完成大規模複雜跨域Text-to-SQL資料集Spider跨域SQLIDE
- python程式碼錯誤RuntimeError: Session is closedPythonErrorSession
- Screaming Frog SEO Spider常見問題解答IDE
- 開源網路爬蟲程式(spider)一覽爬蟲IDE
- ORACLE軟體克隆完成後sysdba登入提示沒有許可權Oracle
- Oracle 12c RMAN Cross-Platform Transport of a Closed PDBOracleROSPlatform
- svn報錯cleanup failed–previous operation has not finished; run cleanup if it was interrupte...AI
- Spider pool 蜘蛛池的概念和原理白狐公羊seoIDE
- svn“Previous operation has not finished; run ‘cleanup‘ if it was interrupted“報錯的解決方法
- Process finished with exit code -1073740791 (0xC0000409)
- MariaDB Spider 資料庫分庫分表實踐IDE資料庫
- Storm在spider.io應用的經驗教訓ORMIDE
- HiSpider(Hitask) is a fast and high performance spider with high speedIDEASTORM
- 沒有返回值的建構函式是怎麼完成賦值的?函式賦值
- Idea啟動專案報{dataSource-1} closed錯誤Idea
- Unable to unwrap data, invalid status [CLOSED]-服務端webSocket報錯服務端Web
- android java.lang.IllegalStateException: trying to requery an already closed cursorAndroidJavaException
- 解決Hibernate session is closed的一種可行性方案Session
- ssh問題:ssh_exchange_identification: Connection closed by remote hostIDEREM
- 網路爬蟲開發工具:Screaming Frog SEO Spider 「Mac」爬蟲IDEMac
- 尖叫青蛙網路爬蟲軟體:Screaming Frog SEO Spider爬蟲IDE
- 探索網路世界的得力助手——Screaming Frog SEO SpiderIDE
- 安裝系統時沒有網路無法完成設定進入系統
- 設計模式筆記:開閉原則(OCP,The Open-Closed Principle)設計模式筆記