[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"similar-apify--crawlee-python":3,"tool-apify--crawlee-python":61},[4,18,26,36,44,53],{"id":5,"name":6,"github_repo":7,"description_zh":8,"stars":9,"difficulty_score":10,"last_commit_at":11,"category_tags":12,"status":17},4358,"openclaw","openclaw\u002Fopenclaw","OpenClaw 是一款专为个人打造的本地化 AI 助手，旨在让你在自己的设备上拥有完全可控的智能伙伴。它打破了传统 AI 助手局限于特定网页或应用的束缚，能够直接接入你日常使用的各类通讯渠道，包括微信、WhatsApp、Telegram、Discord、iMessage 等数十种平台。无论你在哪个聊天软件中发送消息，OpenClaw 都能即时响应，甚至支持在 macOS、iOS 和 Android 设备上进行语音交互，并提供实时的画布渲染功能供你操控。\n\n这款工具主要解决了用户对数据隐私、响应速度以及“始终在线”体验的需求。通过将 AI 部署在本地，用户无需依赖云端服务即可享受快速、私密的智能辅助，真正实现了“你的数据，你做主”。其独特的技术亮点在于强大的网关架构，将控制平面与核心助手分离，确保跨平台通信的流畅性与扩展性。\n\nOpenClaw 非常适合希望构建个性化工作流的技术爱好者、开发者，以及注重隐私保护且不愿被单一生态绑定的普通用户。只要具备基础的终端操作能力（支持 macOS、Linux 及 Windows WSL2），即可通过简单的命令行引导完成部署。如果你渴望拥有一个懂你",349277,3,"2026-04-06T06:32:30",[13,14,15,16],"Agent","开发框架","图像","数据工具","ready",{"id":19,"name":20,"github_repo":21,"description_zh":22,"stars":23,"difficulty_score":10,"last_commit_at":24,"category_tags":25,"status":17},3808,"stable-diffusion-webui","AUTOMATIC1111\u002Fstable-diffusion-webui","stable-diffusion-webui 是一个基于 Gradio 构建的网页版操作界面，旨在让用户能够轻松地在本地运行和使用强大的 Stable Diffusion 图像生成模型。它解决了原始模型依赖命令行、操作门槛高且功能分散的痛点，将复杂的 AI 绘图流程整合进一个直观易用的图形化平台。\n\n无论是希望快速上手的普通创作者、需要精细控制画面细节的设计师，还是想要深入探索模型潜力的开发者与研究人员，都能从中获益。其核心亮点在于极高的功能丰富度：不仅支持文生图、图生图、局部重绘（Inpainting）和外绘（Outpainting）等基础模式，还独创了注意力机制调整、提示词矩阵、负向提示词以及“高清修复”等高级功能。此外，它内置了 GFPGAN 和 CodeFormer 等人脸修复工具，支持多种神经网络放大算法，并允许用户通过插件系统无限扩展能力。即使是显存有限的设备，stable-diffusion-webui 也提供了相应的优化选项，让高质量的 AI 艺术创作变得触手可及。",162132,"2026-04-05T11:01:52",[14,15,13],{"id":27,"name":28,"github_repo":29,"description_zh":30,"stars":31,"difficulty_score":32,"last_commit_at":33,"category_tags":34,"status":17},1381,"everything-claude-code","affaan-m\u002Feverything-claude-code","everything-claude-code 是一套专为 AI 编程助手（如 Claude Code、Codex、Cursor 等）打造的高性能优化系统。它不仅仅是一组配置文件，而是一个经过长期实战打磨的完整框架，旨在解决 AI 代理在实际开发中面临的效率低下、记忆丢失、安全隐患及缺乏持续学习能力等核心痛点。\n\n通过引入技能模块化、直觉增强、记忆持久化机制以及内置的安全扫描功能，everything-claude-code 能显著提升 AI 在复杂任务中的表现，帮助开发者构建更稳定、更智能的生产级 AI 代理。其独特的“研究优先”开发理念和针对 Token 消耗的优化策略，使得模型响应更快、成本更低，同时有效防御潜在的攻击向量。\n\n这套工具特别适合软件开发者、AI 研究人员以及希望深度定制 AI 工作流的技术团队使用。无论您是在构建大型代码库，还是需要 AI 协助进行安全审计与自动化测试，everything-claude-code 都能提供强大的底层支持。作为一个曾荣获 Anthropic 黑客大奖的开源项目，它融合了多语言支持与丰富的实战钩子（hooks），让 AI 真正成长为懂上",148568,2,"2026-04-09T23:34:24",[14,13,35],"语言模型",{"id":37,"name":38,"github_repo":39,"description_zh":40,"stars":41,"difficulty_score":32,"last_commit_at":42,"category_tags":43,"status":17},2271,"ComfyUI","Comfy-Org\u002FComfyUI","ComfyUI 是一款功能强大且高度模块化的视觉 AI 引擎，专为设计和执行复杂的 Stable Diffusion 图像生成流程而打造。它摒弃了传统的代码编写模式，采用直观的节点式流程图界面，让用户通过连接不同的功能模块即可构建个性化的生成管线。\n\n这一设计巧妙解决了高级 AI 绘图工作流配置复杂、灵活性不足的痛点。用户无需具备编程背景，也能自由组合模型、调整参数并实时预览效果，轻松实现从基础文生图到多步骤高清修复等各类复杂任务。ComfyUI 拥有极佳的兼容性，不仅支持 Windows、macOS 和 Linux 全平台，还广泛适配 NVIDIA、AMD、Intel 及苹果 Silicon 等多种硬件架构，并率先支持 SDXL、Flux、SD3 等前沿模型。\n\n无论是希望深入探索算法潜力的研究人员和开发者，还是追求极致创作自由度的设计师与资深 AI 绘画爱好者，ComfyUI 都能提供强大的支持。其独特的模块化架构允许社区不断扩展新功能，使其成为当前最灵活、生态最丰富的开源扩散模型工具之一，帮助用户将创意高效转化为现实。",108111,"2026-04-08T11:23:26",[14,15,13],{"id":45,"name":46,"github_repo":47,"description_zh":48,"stars":49,"difficulty_score":32,"last_commit_at":50,"category_tags":51,"status":17},6121,"gemini-cli","google-gemini\u002Fgemini-cli","gemini-cli 是一款由谷歌推出的开源 AI 命令行工具，它将强大的 Gemini 大模型能力直接集成到用户的终端环境中。对于习惯在命令行工作的开发者而言，它提供了一条从输入提示词到获取模型响应的最短路径，无需切换窗口即可享受智能辅助。\n\n这款工具主要解决了开发过程中频繁上下文切换的痛点，让用户能在熟悉的终端界面内直接完成代码理解、生成、调试以及自动化运维任务。无论是查询大型代码库、根据草图生成应用，还是执行复杂的 Git 操作，gemini-cli 都能通过自然语言指令高效处理。\n\n它特别适合广大软件工程师、DevOps 人员及技术研究人员使用。其核心亮点包括支持高达 100 万 token 的超长上下文窗口，具备出色的逻辑推理能力；内置 Google 搜索、文件操作及 Shell 命令执行等实用工具；更独特的是，它支持 MCP（模型上下文协议），允许用户灵活扩展自定义集成，连接如图像生成等外部能力。此外，个人谷歌账号即可享受免费的额度支持，且项目基于 Apache 2.0 协议完全开源，是提升终端工作效率的理想助手。",100752,"2026-04-10T01:20:03",[52,13,15,14],"插件",{"id":54,"name":55,"github_repo":56,"description_zh":57,"stars":58,"difficulty_score":32,"last_commit_at":59,"category_tags":60,"status":17},4721,"markitdown","microsoft\u002Fmarkitdown","MarkItDown 是一款由微软 AutoGen 团队打造的轻量级 Python 工具，专为将各类文件高效转换为 Markdown 格式而设计。它支持 PDF、Word、Excel、PPT、图片（含 OCR）、音频（含语音转录）、HTML 乃至 YouTube 链接等多种格式的解析，能够精准提取文档中的标题、列表、表格和链接等关键结构信息。\n\n在人工智能应用日益普及的今天，大语言模型（LLM）虽擅长处理文本，却难以直接读取复杂的二进制办公文档。MarkItDown 恰好解决了这一痛点，它将非结构化或半结构化的文件转化为模型“原生理解”且 Token 效率极高的 Markdown 格式，成为连接本地文件与 AI 分析 pipeline 的理想桥梁。此外，它还提供了 MCP（模型上下文协议）服务器，可无缝集成到 Claude Desktop 等 LLM 应用中。\n\n这款工具特别适合开发者、数据科学家及 AI 研究人员使用，尤其是那些需要构建文档检索增强生成（RAG）系统、进行批量文本分析或希望让 AI 助手直接“阅读”本地文件的用户。虽然生成的内容也具备一定可读性，但其核心优势在于为机器",93400,"2026-04-06T19:52:38",[52,14],{"id":62,"github_repo":63,"name":64,"description_en":65,"description_zh":66,"ai_summary_zh":67,"readme_en":68,"readme_zh":69,"quickstart_zh":70,"use_case_zh":71,"hero_image_url":72,"owner_login":73,"owner_name":74,"owner_avatar_url":75,"owner_bio":76,"owner_company":77,"owner_location":77,"owner_email":78,"owner_twitter":73,"owner_website":79,"owner_url":80,"languages":81,"stars":108,"forks":109,"last_commit_at":110,"license":111,"difficulty_score":32,"env_os":112,"env_gpu":113,"env_ram":114,"env_deps":115,"category_tags":123,"github_topics":124,"view_count":32,"oss_zip_url":77,"oss_zip_packed_at":77,"status":17,"created_at":140,"updated_at":141,"faqs":142,"releases":171},6136,"apify\u002Fcrawlee-python","crawlee-python","Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.","crawlee-python 是一款专为 Python 开发者打造的网页抓取与浏览器自动化库，旨在帮助用户快速构建稳定可靠的网络爬虫。无论是为人工智能、大语言模型（LLM）或检索增强生成（RAG）系统提取训练数据，还是批量下载网站上的 HTML、PDF 及图片文件，它都能轻松胜任。\n\n传统爬虫开发常面临反机器人机制拦截、动态网页渲染复杂以及数据存储繁琐等痛点。crawlee-python 通过内置的智能策略，让爬虫行为在默认配置下即可高度模拟人类操作，有效绕过现代网站的防护系统。它无缝整合了 Playwright、BeautifulSoup、Parsel 及原生 HTTP 等多种技术栈，支持有头和无头两种浏览器模式，并提供自动代理轮换功能，极大降低了处理复杂网页的技术门槛。\n\n该工具特别适合需要高效采集公开数据的软件工程师、数据科学家及 AI 研究人员使用。其独特的亮点在于“开箱即用”的可靠性设计与灵活的扩展能力：既提供了简单的 CLI 命令和模板供新手快速启动项目，又允许资深开发者深度定制各项参数以满足特定需求。配合完善的文档与活跃的社区支持，crawlee-python 能让用户专","crawlee-python 是一款专为 Python 开发者打造的网页抓取与浏览器自动化库，旨在帮助用户快速构建稳定可靠的网络爬虫。无论是为人工智能、大语言模型（LLM）或检索增强生成（RAG）系统提取训练数据，还是批量下载网站上的 HTML、PDF 及图片文件，它都能轻松胜任。\n\n传统爬虫开发常面临反机器人机制拦截、动态网页渲染复杂以及数据存储繁琐等痛点。crawlee-python 通过内置的智能策略，让爬虫行为在默认配置下即可高度模拟人类操作，有效绕过现代网站的防护系统。它无缝整合了 Playwright、BeautifulSoup、Parsel 及原生 HTTP 等多种技术栈，支持有头和无头两种浏览器模式，并提供自动代理轮换功能，极大降低了处理复杂网页的技术门槛。\n\n该工具特别适合需要高效采集公开数据的软件工程师、数据科学家及 AI 研究人员使用。其独特的亮点在于“开箱即用”的可靠性设计与灵活的扩展能力：既提供了简单的 CLI 命令和模板供新手快速启动项目，又允许资深开发者深度定制各项参数以满足特定需求。配合完善的文档与活跃的社区支持，crawlee-python 能让用户专注于数据价值本身，而非纠缠于底层技术细节。","\u003Ch1 align=\"center\">\n    \u003Ca href=\"https:\u002F\u002Fcrawlee.dev\">\n        \u003Cpicture>\n          \u003Csource media=\"(prefers-color-scheme: dark)\" srcset=\"https:\u002F\u002Fraw.githubusercontent.com\u002Fapify\u002Fcrawlee-python\u002Fmaster\u002Fwebsite\u002Fstatic\u002Fimg\u002Fcrawlee-dark.svg?sanitize=true\">\n          \u003Cimg alt=\"Crawlee\" src=\"https:\u002F\u002Fraw.githubusercontent.com\u002Fapify\u002Fcrawlee-python\u002Fmaster\u002Fwebsite\u002Fstatic\u002Fimg\u002Fcrawlee-light.svg?sanitize=true\" width=\"500\">\n        \u003C\u002Fpicture>\n    \u003C\u002Fa>\n    \u003Cbr>\n    \u003Csmall>A web scraping and browser automation library\u003C\u002Fsmall>\n\u003C\u002Fh1>\n\n\u003Cp align=center>\n    \u003Ca href=\"https:\u002F\u002Ftrendshift.io\u002Frepositories\u002F11169\" target=\"_blank\">\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fapify_crawlee-python_readme_4a68feb902da.png\" alt=\"apify%2Fcrawlee-python | Trendshift\" style=\"width: 250px; height: 55px;\" width=\"250\" height=\"55\"\u002F>\u003C\u002Fa>\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n  \u003Ca href=\"https:\u002F\u002Fbadge.fury.io\u002Fpy\u002Fcrawlee\" rel=\"nofollow\">\u003Cimg src=\"https:\u002F\u002Fbadge.fury.io\u002Fpy\u002Fcrawlee.svg\" alt=\"PyPI package version\">\u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Fpypi.org\u002Fproject\u002Fcrawlee\u002F\" rel=\"nofollow\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fpypi\u002Fdm\u002Fcrawlee\" alt=\"PyPI package downloads\">\u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Fcodecov.io\u002Fgh\u002Fapify\u002Fcrawlee-python\">\u003Cimg src=\"https:\u002F\u002Fcodecov.io\u002Fgh\u002Fapify\u002Fcrawlee-python\u002Fgraph\u002Fbadge.svg?token=cCju61iPQG\" alt=\"Codecov report\">\u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Fpypi.org\u002Fproject\u002Fcrawlee\u002F\" rel=\"nofollow\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fpypi\u002Fpyversions\u002Fcrawlee\" alt=\"PyPI Python version\">\u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Fdiscord.gg\u002FjyEM2PRvMU\" rel=\"nofollow\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fdiscord\u002F801163717915574323?label=discord\" alt=\"Chat on Discord\">\u003C\u002Fa>\n\u003C\u002Fp>\n\nCrawlee covers your crawling and scraping end-to-end and **helps you build reliable scrapers. Fast.**\n\nYour crawlers will appear almost human-like and fly under the radar of modern bot protections even with the default configuration. Crawlee gives you the tools to crawl the web for links, scrape data and persistently store it in machine-readable formats, without having to worry about the technical details. And thanks to rich configuration options, you can tweak almost any aspect of Crawlee to suit your project's needs if the default settings don't cut it.\n\n> 👉 **View full documentation, guides and examples on the [Crawlee project website](https:\u002F\u002Fcrawlee.dev\u002Fpython\u002F)** 👈\n\nWe also have a TypeScript implementation of the Crawlee, which you can explore and utilize for your projects. Visit our GitHub repository for more information [Crawlee for JS\u002FTS on GitHub](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee).\n\n## Installation\n\nWe recommend visiting the [Introduction tutorial](https:\u002F\u002Fcrawlee.dev\u002Fpython\u002Fdocs\u002Fintroduction) in Crawlee documentation for more information.\n\nCrawlee is available as [`crawlee`](https:\u002F\u002Fpypi.org\u002Fproject\u002Fcrawlee\u002F) package on PyPI. This package includes the core functionality, while additional features are available as optional extras to keep dependencies and package size minimal.\n\nTo install Crawlee with all features, run the following command:\n\n```sh\npython -m pip install 'crawlee[all]'\n```\n\nThen, install the [Playwright](https:\u002F\u002Fplaywright.dev\u002F) dependencies:\n\n```sh\nplaywright install\n```\n\nVerify that Crawlee is successfully installed:\n\n```sh\npython -c 'import crawlee; print(crawlee.__version__)'\n```\n\nFor detailed installation instructions see the [Setting up](https:\u002F\u002Fcrawlee.dev\u002Fpython\u002Fdocs\u002Fintroduction\u002Fsetting-up) documentation page.\n\n### With Crawlee CLI\n\nThe quickest way to get started with Crawlee is by using the Crawlee CLI and selecting one of the prepared templates. First, ensure you have [uv](https:\u002F\u002Fpypi.org\u002Fproject\u002Fuv\u002F) installed:\n\n```sh\nuv --help\n```\n\nIf [uv](https:\u002F\u002Fpypi.org\u002Fproject\u002Fuv\u002F) is not installed, follow the official [installation guide](https:\u002F\u002Fdocs.astral.sh\u002Fuv\u002Fgetting-started\u002Finstallation\u002F).\n\nThen, run the CLI and choose from the available templates:\n\n```sh\nuvx 'crawlee[cli]' create my-crawler\n```\n\nIf you already have `crawlee` installed, you can spin it up by running:\n\n```sh\ncrawlee create my-crawler\n```\n\n## Examples\n\nHere are some practical examples to help you get started with different types of crawlers in Crawlee. Each example demonstrates how to set up and run a crawler for specific use cases, whether you need to handle simple HTML pages or interact with JavaScript-heavy sites. A crawler run will create a `storage\u002F` directory in your current working directory.\n\n### BeautifulSoupCrawler\n\nThe [`BeautifulSoupCrawler`](https:\u002F\u002Fcrawlee.dev\u002Fpython\u002Fapi\u002Fclass\u002FBeautifulSoupCrawler) downloads web pages using an HTTP library and provides HTML-parsed content to the user. By default it uses [`HttpxHttpClient`](https:\u002F\u002Fcrawlee.dev\u002Fpython\u002Fapi\u002Fclass\u002FHttpxHttpClient) for HTTP communication and [BeautifulSoup](https:\u002F\u002Fpypi.org\u002Fproject\u002Fbeautifulsoup4\u002F) for parsing HTML. It is ideal for projects that require efficient extraction of data from HTML content. This crawler has very good performance since it does not use a browser. However, if you need to execute client-side JavaScript, to get your content, this is not going to be enough and you will need to use [`PlaywrightCrawler`](https:\u002F\u002Fcrawlee.dev\u002Fpython\u002Fapi\u002Fclass\u002FPlaywrightCrawler). Also if you want to use this crawler, make sure you install `crawlee` with `beautifulsoup` extra.\n\n```python\nimport asyncio\n\nfrom crawlee.crawlers import BeautifulSoupCrawler, BeautifulSoupCrawlingContext\n\n\nasync def main() -> None:\n    crawler = BeautifulSoupCrawler(\n        # Limit the crawl to max requests. Remove or increase it for crawling all links.\n        max_requests_per_crawl=10,\n    )\n\n    # Define the default request handler, which will be called for every request.\n    @crawler.router.default_handler\n    async def request_handler(context: BeautifulSoupCrawlingContext) -> None:\n        context.log.info(f'Processing {context.request.url} ...')\n\n        # Extract data from the page.\n        data = {\n            'url': context.request.url,\n            'title': context.soup.title.string if context.soup.title else None,\n        }\n\n        # Push the extracted data to the default dataset.\n        await context.push_data(data)\n\n        # Enqueue all links found on the page.\n        await context.enqueue_links()\n\n    # Run the crawler with the initial list of URLs.\n    await crawler.run(['https:\u002F\u002Fcrawlee.dev'])\n\n\nif __name__ == '__main__':\n    asyncio.run(main())\n```\n\n### PlaywrightCrawler\n\nThe [`PlaywrightCrawler`](https:\u002F\u002Fcrawlee.dev\u002Fpython\u002Fapi\u002Fclass\u002FPlaywrightCrawler) uses a headless browser to download web pages and provides an API for data extraction. It is built on [Playwright](https:\u002F\u002Fplaywright.dev\u002F), an automation library designed for managing headless browsers. It excels at retrieving web pages that rely on client-side JavaScript for content generation, or tasks requiring interaction with JavaScript-driven content. For scenarios where JavaScript execution is unnecessary or higher performance is required, consider using the [`BeautifulSoupCrawler`](https:\u002F\u002Fcrawlee.dev\u002Fpython\u002Fapi\u002Fclass\u002FBeautifulSoupCrawler). Also if you want to use this crawler, make sure you install `crawlee` with `playwright` extra.\n\n```python\nimport asyncio\n\nfrom crawlee.crawlers import PlaywrightCrawler, PlaywrightCrawlingContext\n\n\nasync def main() -> None:\n    crawler = PlaywrightCrawler(\n        # Limit the crawl to max requests. Remove or increase it for crawling all links.\n        max_requests_per_crawl=10,\n    )\n\n    # Define the default request handler, which will be called for every request.\n    @crawler.router.default_handler\n    async def request_handler(context: PlaywrightCrawlingContext) -> None:\n        context.log.info(f'Processing {context.request.url} ...')\n\n        # Extract data from the page.\n        data = {\n            'url': context.request.url,\n            'title': await context.page.title(),\n        }\n\n        # Push the extracted data to the default dataset.\n        await context.push_data(data)\n\n        # Enqueue all links found on the page.\n        await context.enqueue_links()\n\n    # Run the crawler with the initial list of requests.\n    await crawler.run(['https:\u002F\u002Fcrawlee.dev'])\n\n\nif __name__ == '__main__':\n    asyncio.run(main())\n```\n\n### More examples\n\nExplore our [Examples](https:\u002F\u002Fcrawlee.dev\u002Fpython\u002Fdocs\u002Fexamples) page in the Crawlee documentation for a wide range of additional use cases and demonstrations.\n\n## Features\n\nWhy Crawlee is the preferred choice for web scraping and crawling?\n\n### Why use Crawlee instead of just a random HTTP library with an HTML parser?\n\n- Unified interface for **HTTP & headless browser** crawling.\n- Automatic **parallel crawling** based on available system resources.\n- Written in Python with **type hints** - enhances DX (IDE autocompletion) and reduces bugs (static type checking).\n- Automatic **retries** on errors or when you’re getting blocked.\n- Integrated **proxy rotation** and session management.\n- Configurable **request routing** - direct URLs to the appropriate handlers.\n- Persistent **queue for URLs** to crawl.\n- Pluggable **storage** of both tabular data and files.\n- Robust **error handling**.\n\n### Why to use Crawlee rather than Scrapy?\n\n- **Asyncio-based** – Leveraging the standard [Asyncio](https:\u002F\u002Fdocs.python.org\u002F3\u002Flibrary\u002Fasyncio.html) library, Crawlee delivers better performance and seamless compatibility with other modern asynchronous libraries.\n- **Type hints** – Newer project built with modern Python, and complete type hint coverage for a better developer experience.\n- **Simple integration** – Crawlee crawlers are regular Python scripts, requiring no additional launcher executor. This flexibility allows to integrate a crawler directly into other applications.\n- **State persistence** – Supports state persistence during interruptions, saving time and costs by avoiding the need to restart scraping pipelines from scratch after an issue.\n- **Organized data storages** – Allows saving of multiple types of results in a single scraping run. Offers several storing options (see [datasets](https:\u002F\u002Fcrawlee.dev\u002Fpython\u002Fapi\u002Fclass\u002FDataset) & [key-value stores](https:\u002F\u002Fcrawlee.dev\u002Fpython\u002Fapi\u002Fclass\u002FKeyValueStore)).\n\n## Running on the Apify platform\n\nCrawlee is open-source and runs anywhere, but since it's developed by [Apify](https:\u002F\u002Fapify.com), it's easy to set up on the Apify platform and run in the cloud. Visit the [Apify SDK website](https:\u002F\u002Fdocs.apify.com\u002Fsdk\u002Fpython\u002F) to learn more about deploying Crawlee to the Apify platform.\n\n## Support\n\nIf you find any bug or issue with Crawlee, please [submit an issue on GitHub](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fissues). For questions, you can ask on [Stack Overflow](https:\u002F\u002Fstackoverflow.com\u002Fquestions\u002Ftagged\u002Fapify), in GitHub Discussions or you can join our [Discord server](https:\u002F\u002Fdiscord.com\u002Finvite\u002FjyEM2PRvMU).\n\n## Contributing\n\nYour code contributions are welcome, and you'll be praised for eternity! If you have any ideas for improvements, either submit an issue or create a pull request. For contribution guidelines and the code of conduct, see [CONTRIBUTING.md](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fblob\u002Fmaster\u002FCONTRIBUTING.md).\n\n## License\n\nThis project is licensed under the Apache License 2.0 - see the [LICENSE](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fblob\u002Fmaster\u002FLICENSE) file for details.\n","\u003Ch1 align=\"center\">\n    \u003Ca href=\"https:\u002F\u002Fcrawlee.dev\">\n        \u003Cpicture>\n          \u003Csource media=\"(prefers-color-scheme: dark)\" srcset=\"https:\u002F\u002Fraw.githubusercontent.com\u002Fapify\u002Fcrawlee-python\u002Fmaster\u002Fwebsite\u002Fstatic\u002Fimg\u002Fcrawlee-dark.svg?sanitize=true\">\n          \u003Cimg alt=\"Crawlee\" src=\"https:\u002F\u002Fraw.githubusercontent.com\u002Fapify\u002Fcrawlee-python\u002Fmaster\u002Fwebsite\u002Fstatic\u002Fimg\u002Fcrawlee-light.svg?sanitize=true\" width=\"500\">\n        \u003C\u002Fpicture>\n    \u003C\u002Fa>\n    \u003Cbr>\n    \u003Csmall>一个用于网络爬取和浏览器自动化的小程序库\u003C\u002Fsmall>\n\u003C\u002Fh1>\n\n\u003Cp align=center>\n    \u003Ca href=\"https:\u002F\u002Ftrendshift.io\u002Frepositories\u002F11169\" target=\"_blank\">\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fapify_crawlee-python_readme_4a68feb902da.png\" alt=\"apify%2Fcrawlee-python | Trendshift\" style=\"width: 250px; height: 55px;\" width=\"250\" height=\"55\"\u002F>\u003C\u002Fa>\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n  \u003Ca href=\"https:\u002F\u002Fbadge.fury.io\u002Fpy\u002Fcrawlee\" rel=\"nofollow\">\u003Cimg src=\"https:\u002F\u002Fbadge.fury.io\u002Fpy\u002Fcrawlee.svg\" alt=\"PyPI 包版本\">\u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Fpypi.org\u002Fproject\u002Fcrawlee\u002F\" rel=\"nofollow\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fpypi\u002Fdm\u002Fcrawlee\" alt=\"PyPI 包下载量\">\u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Fcodecov.io\u002Fgh\u002Fapify\u002Fcrawlee-python\">\u003Cimg src=\"https:\u002F\u002Fcodecov.io\u002Fgh\u002Fapify\u002Fcrawlee-python\u002Fgraph\u002Fbadge.svg?token=cCju61iPQG\" alt=\"Codecov 报告\">\u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Fpypi.org\u002Fproject\u002Fcrawlee\u002F\" rel=\"nofollow\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fpypi\u002Fpyversions\u002Fcrawlee\" alt=\"PyPI Python 版本\">\u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Fdiscord.gg\u002FjyEM2PRvMU\" rel=\"nofollow\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fdiscord\u002F801163717915574323?label=discord\" alt=\"在 Discord 上交流\">\u003C\u002Fa>\n\u003C\u002Fp>\n\nCrawlee 覆盖了您的爬取和数据抓取全流程，并且**帮助您快速构建可靠的爬虫**。\n\n即使使用默认配置，您的爬虫也会表现得几乎像真人一样，轻松躲过现代反爬机制的检测。Crawlee 提供了丰富的工具，让您能够抓取网页上的链接、提取数据并以机器可读的格式持久化存储，而无需担心底层的技术细节。同时，借助其强大的配置选项，您可以根据项目需求调整 Crawlee 的几乎所有方面，以满足特定场景的要求。\n\n> 👉 **请访问 [Crawlee 项目官网](https:\u002F\u002Fcrawlee.dev\u002Fpython\u002F) 查看完整的文档、指南和示例** 👈\n\n我们还提供了 TypeScript 版本的 Crawlee，您也可以将其用于自己的项目中。更多信息请访问我们的 GitHub 仓库：[Crawlee for JS\u002FTS on GitHub](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee)。\n\n## 安装\n\n建议您阅读 Crawlee 文档中的[入门教程](https:\u002F\u002Fcrawlee.dev\u002Fpython\u002Fdocs\u002Fintroduction)，以获取更多详细信息。\n\nCrawlee 作为 [`crawlee`](https:\u002F\u002Fpypi.org\u002Fproject\u002Fcrawlee\u002F) 包发布在 PyPI 上。该包包含了核心功能，而其他附加特性则以可选扩展的形式提供，以便保持依赖项和包体积的最小化。\n\n要安装包含所有功能的 Crawlee，请运行以下命令：\n\n```sh\npython -m pip install 'crawlee[all]'\n```\n\n然后安装 [Playwright](https:\u002F\u002Fplaywright.dev\u002F) 的依赖项：\n\n```sh\nplaywright install\n```\n\n验证 Crawlee 是否成功安装：\n\n```sh\npython -c 'import crawlee; print(crawlee.__version__)'\n```\n\n详细的安装步骤请参阅[设置](https:\u002F\u002Fcrawlee.dev\u002Fpython\u002Fdocs\u002Fintroduction\u002Fsetting-up)文档页面。\n\n### 使用 Crawlee CLI\n\n开始使用 Crawlee 最快捷的方式是通过 Crawlee CLI 并选择一个预设模板。首先确保您已安装 [uv](https:\u002F\u002Fpypi.org\u002Fproject\u002Fuv\u002F)：\n\n```sh\nuv --help\n```\n\n如果尚未安装 [uv](https:\u002F\u002Fpypi.org\u002Fproject\u002Fuv\u002F)，请按照官方[安装指南](https:\u002F\u002Fdocs.astral.sh\u002Fuv\u002Fgetting-started\u002Finstallation\u002F)进行安装。\n\n然后运行 CLI 并从可用模板中选择：\n\n```sh\nuvx 'crawlee[cli]' create my-crawler\n```\n\n如果您已经安装了 `crawlee`，可以直接运行以下命令启动：\n\n```sh\ncrawlee create my-crawler\n```\n\n## 示例\n\n以下是一些实用示例，帮助您快速上手 Crawlee 中不同类型的爬虫。每个示例都展示了如何针对特定用例设置和运行爬虫，无论您是需要处理简单的 HTML 页面，还是与大量 JavaScript 的网站交互。运行爬虫后，会在当前工作目录下创建一个 `storage\u002F` 目录。\n\n### BeautifulSoupCrawler\n\n[`BeautifulSoupCrawler`](https:\u002F\u002Fcrawlee.dev\u002Fpython\u002Fapi\u002Fclass\u002FBeautifulSoupCrawler) 使用 HTTP 库下载网页，并将解析后的 HTML 内容提供给用户。默认情况下，它使用 [`HttpxHttpClient`](https:\u002F\u002Fcrawlee.dev\u002Fpython\u002Fapi\u002Fclass\u002FHttpxHttpClient) 进行 HTTP 通信，并用 [BeautifulSoup](https:\u002F\u002Fpypi.org\u002Fproject\u002Fbeautifulsoup4\u002F) 解析 HTML。它非常适合需要高效提取 HTML 内容的项目。由于不使用浏览器，这种爬虫性能非常出色。然而，如果您需要执行客户端 JavaScript 才能获取内容，则此方法无法满足需求，这时就需要使用 [`PlaywrightCrawler`](https:\u002F\u002Fcrawlee.dev\u002Fpython\u002Fapi\u002Fclass\u002FPlaywrightCrawler)。另外，如果您打算使用此爬虫，请务必安装带有 `beautifulsoup` 扩展的 `crawlee`。\n\n```python\nimport asyncio\n\nfrom crawlee.crawlers import BeautifulSoupCrawler, BeautifulSoupCrawlingContext\n\n\nasync def main() -> None:\n    crawler = BeautifulSoupCrawler(\n        # 限制爬取的最大请求数。移除或增加此值可爬取所有链接。\n        max_requests_per_crawl=10,\n    )\n\n    # 定义默认请求处理器，它将被调用以处理每一条请求。\n    @crawler.router.default_handler\n    async def request_handler(context: BeautifulSoupCrawlingContext) -> None:\n        context.log.info(f'正在处理 {context.request.url} ...')\n\n        # 从页面中提取数据。\n        data = {\n            'url': context.request.url,\n            'title': context.soup.title.string if context.soup.title else None,\n        }\n\n        # 将提取的数据推送到默认数据集中。\n        await context.push_data(data)\n\n        # 将页面中找到的所有链接加入队列。\n        await context.enqueue_links()\n\n    # 使用初始 URL 列表运行爬虫。\n    await crawler.run(['https:\u002F\u002Fcrawlee.dev'])\n\n\nif __name__ == '__main__':\n    asyncio.run(main())\n```\n\n### PlaywrightCrawler\n\n[`PlaywrightCrawler`](https:\u002F\u002Fcrawlee.dev\u002Fpython\u002Fapi\u002Fclass\u002FPlaywrightCrawler) 使用无头浏览器下载网页，并提供用于数据提取的 API。它基于 [Playwright](https:\u002F\u002Fplaywright.dev\u002F) 构建，Playwright 是一个专为管理无头浏览器而设计的自动化库。该爬虫特别擅长抓取那些依赖客户端 JavaScript 生成内容的页面，或需要与 JavaScript 驱动内容交互的任务。如果在某些场景中无需执行 JavaScript 或需要更高性能，则可以考虑使用 [`BeautifulSoupCrawler`](https:\u002F\u002Fcrawlee.dev\u002Fpython\u002Fapi\u002Fclass\u002FBeautifulSoupCrawler)。此外，若要使用此爬虫，请确保安装 `crawlee` 时带有 `playwright` 附加组件。\n\n```python\nimport asyncio\n\nfrom crawlee.crawlers import PlaywrightCrawler, PlaywrightCrawlingContext\n\n\nasync def main() -> None:\n    crawler = PlaywrightCrawler(\n        # 限制爬取的最大请求数。移除或增加此值以爬取所有链接。\n        max_requests_per_crawl=10,\n    )\n\n    # 定义默认请求处理器，它将被调用处理每一个请求。\n    @crawler.router.default_handler\n    async def request_handler(context: PlaywrightCrawlingContext) -> None:\n        context.log.info(f'正在处理 {context.request.url} ...')\n\n        # 从页面中提取数据。\n        data = {\n            'url': context.request.url,\n            'title': await context.page.title(),\n        }\n\n        # 将提取的数据推送到默认数据集。\n        await context.push_data(data)\n\n        # 将页面上找到的所有链接加入队列。\n        await context.enqueue_links()\n\n    # 使用初始请求列表运行爬虫。\n    await crawler.run(['https:\u002F\u002Fcrawlee.dev'])\n\n\nif __name__ == '__main__':\n    asyncio.run(main())\n```\n\n### 更多示例\n\n请访问 Crawlee 文档中的 [示例](https:\u002F\u002Fcrawlee.dev\u002Fpython\u002Fdocs\u002Fexamples) 页面，了解更广泛的使用场景和演示。\n\n## 功能特性\n\n为什么 Crawlee 是网络爬取和抓取任务的首选工具？\n\n### 为什么选择 Crawlee 而不是简单的 HTTP 库结合 HTML 解析器？\n\n- 统一的 **HTTP 和无头浏览器** 抓取接口。\n- 基于系统可用资源的自动 **并行抓取**。\n- 使用 Python 编写并配备 **类型提示**，提升开发体验（IDE 自动补全）并减少错误（静态类型检查）。\n- 在遇到错误或被封禁时自动进行 **重试**。\n- 集成 **代理轮换** 和会话管理功能。\n- 可配置的 **请求路由**，可将不同 URL 路由到相应的处理器。\n- 持久化的待抓取 **URL 队列**。\n- 可插拔的 **存储系统**，支持表格数据和文件的存储。\n- 强大的 **错误处理机制**。\n\n### 为什么选择 Crawlee 而不是 Scrapy？\n\n- 基于 **Asyncio** —— Crawlee 利用标准的 Asyncio 库，提供了更好的性能，并与其他现代异步库无缝兼容。\n- **类型提示** —— 新项目采用现代 Python 编写，全面覆盖类型提示，从而提升开发者体验。\n- **简单集成** —— Crawlee 爬虫是普通的 Python 脚本，无需额外的启动器或执行器。这种灵活性使得爬虫可以直接集成到其他应用程序中。\n- **状态持久化** —— 支持在中断后恢复状态，避免因问题导致爬取流程需要从头开始，从而节省时间和成本。\n- **结构化的数据存储** —— 允许在一次爬取过程中保存多种类型的结果。提供多种存储选项（参见 [数据集](https:\u002F\u002Fcrawlee.dev\u002Fpython\u002Fapi\u002Fclass\u002FDataset) 和 [键值存储](https:\u002F\u002Fcrawlee.dev\u002Fpython\u002Fapi\u002Fclass\u002FKeyValueStore)）。\n\n## 在 Apify 平台上的运行\n\nCrawlee 是开源的，可以在任何地方运行，但由于它由 [Apify](https:\u002F\u002Fapify.com) 开发，因此在 Apify 平台上设置并云端运行非常方便。请访问 [Apify SDK 官网](https:\u002F\u002Fdocs.apify.com\u002Fsdk\u002Fpython\u002F) 了解更多关于将 Crawlee 部署到 Apify 平台的信息。\n\n## 支持\n\n如果您发现 Crawlee 中存在任何 bug 或问题，请在 [GitHub 上提交 issue](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fissues)。如有疑问，您可以在 [Stack Overflow](https:\u002F\u002Fstackoverflow.com\u002Fquestions\u002Ftagged\u002Fapify)、GitHub 讨论区提问，或者加入我们的 [Discord 社区](https:\u002F\u002Fdiscord.com\u002Finvite\u002FjyEM2PRvMU)。\n\n## 贡献\n\n我们欢迎您的代码贡献，您的努力将永载史册！如果您有任何改进建议，可以提交 issue 或创建 pull request。有关贡献指南和行为准则，请参阅 [CONTRIBUTING.md](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fblob\u002Fmaster\u002FCONTRIBUTING.md)。\n\n## 许可证\n\n本项目采用 Apache License 2.0 许可证授权——详情请参阅 [LICENSE](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fblob\u002Fmaster\u002FLICENSE) 文件。","# Crawlee-Python 快速上手指南\n\nCrawlee 是一个专为 Python 打造的网页爬虫和浏览器自动化库。它能帮助你快速构建稳定、高效的爬虫，自动处理反爬机制、代理轮换、请求重试和数据持久化，让爬虫行为更像真实用户。\n\n## 环境准备\n\n在开始之前，请确保你的开发环境满足以下要求：\n\n*   **操作系统**：Windows, macOS 或 Linux\n*   **Python 版本**：Python 3.9 或更高版本\n*   **包管理工具**：推荐使用 `pip` 或现代化的 `uv` (用于 CLI 快速初始化)\n*   **网络环境**：能够访问 PyPI 和 GitHub（若下载 Playwright 浏览器内核较慢，建议配置国内镜像或代理）\n\n## 安装步骤\n\n### 方式一：使用 pip 安装（推荐）\n\n1.  **安装 Crawlee 核心库及所有功能组件**：\n    ```sh\n    python -m pip install 'crawlee[all]'\n    ```\n    > **提示**：国内用户若下载缓慢，可添加清华或阿里镜像源：\n    > `python -m pip install 'crawlee[all]' -i https:\u002F\u002Fpypi.tuna.tsinghua.edu.cn\u002Fsimple`\n\n2.  **安装 Playwright 浏览器内核**：\n    Crawlee 依赖 Playwright 进行浏览器自动化，需单独下载浏览器二进制文件：\n    ```sh\n    playwright install\n    ```\n    > **注意**：此步骤会下载 Chrome、Firefox 等浏览器内核，请确保网络连接通畅。\n\n3.  **验证安装**：\n    ```sh\n    python -c 'import crawlee; print(crawlee.__version__)'\n    ```\n\n### 方式二：使用 CLI 快速创建项目\n\n如果你已安装 [`uv`](https:\u002F\u002Fpypi.org\u002Fproject\u002Fuv\u002F) 工具，可以使用官方 CLI 一键生成项目模板：\n\n```sh\nuvx 'crawlee[cli]' create my-crawler\n```\n或者在已安装 `crawlee` 的情况下运行：\n```sh\ncrawlee create my-crawler\n```\n\n## 基本使用\n\nCrawlee 提供了多种爬虫类型，最常用的是 **BeautifulSoupCrawler**（高性能，适合静态页面）和 **PlaywrightCrawler**（支持 JavaScript 渲染，适合动态页面）。\n\n### 示例 1：使用 BeautifulSoupCrawler 抓取静态页面\n\n此模式不启动浏览器，通过 HTTP 请求获取 HTML 并解析，速度极快。\n\n```python\nimport asyncio\n\nfrom crawlee.crawlers import BeautifulSoupCrawler, BeautifulSoupCrawlingContext\n\n\nasync def main() -> None:\n    crawler = BeautifulSoupCrawler(\n        # 限制最大请求数，生产环境中可移除或调大此参数\n        max_requests_per_crawl=10,\n    )\n\n    # 定义默认请求处理器，每个请求都会触发此函数\n    @crawler.router.default_handler\n    async def request_handler(context: BeautifulSoupCrawlingContext) -> None:\n        context.log.info(f'Processing {context.request.url} ...')\n\n        # 提取数据\n        data = {\n            'url': context.request.url,\n            'title': context.soup.title.string if context.soup.title else None,\n        }\n\n        # 将数据保存到默认数据集\n        await context.push_data(data)\n\n        # 将页面中发现的所有链接加入队列\n        await context.enqueue_links()\n\n    # 启动爬虫，传入初始 URL 列表\n    await crawler.run(['https:\u002F\u002Fcrawlee.dev'])\n\n\nif __name__ == '__main__':\n    asyncio.run(main())\n```\n\n### 示例 2：使用 PlaywrightCrawler 抓取动态页面\n\n如果目标网站依赖 JavaScript 加载内容（如 React\u002FVue 应用），请使用此模式。它会启动无头浏览器进行渲染。\n\n```python\nimport asyncio\n\nfrom crawlee.crawlers import PlaywrightCrawler, PlaywrightCrawlingContext\n\n\nasync def main() -> None:\n    crawler = PlaywrightCrawler(\n        # 限制最大请求数\n        max_requests_per_crawl=10,\n    )\n\n    # 定义默认请求处理器\n    @crawler.router.default_handler\n    async def request_handler(context: PlaywrightCrawlingContext) -> None:\n        context.log.info(f'Processing {context.request.url} ...')\n\n        # 提取数据（此时页面已完成 JS 渲染）\n        data = {\n            'url': context.request.url,\n            'title': await context.page.title(),\n        }\n\n        # 保存数据\n        await context.push_data(data)\n\n        # 将页面链接加入队列\n        await context.enqueue_links()\n\n    # 启动爬虫\n    await crawler.run(['https:\u002F\u002Fcrawlee.dev'])\n\n\nif __name__ == '__main__':\n    asyncio.run(main())\n```\n\n### 运行结果\n\n执行上述脚本后，Crawlee 会在当前工作目录下自动生成一个 `storage\u002F` 文件夹。抓取到的数据将以机器可读的格式（如 JSON\u002FCSV）保存在 `storage\u002Fdatasets\u002Fdefault\u002F` 目录中。","某电商数据团队需要每日监控竞品网站的价格变动与新品上架情况，以支撑动态定价策略。\n\n### 没有 crawlee-python 时\n- 自行编写的脚本极易被目标网站的反爬虫机制识别并封禁 IP，导致数据采集频繁中断。\n- 面对大量动态渲染的 JavaScript 页面，需手动整合 Selenium 或 Playwright，代码维护成本极高且运行缓慢。\n- 缺乏内置的代理轮换与请求重试机制，网络波动或单点故障常导致整个采集任务失败，数据完整性难以保证。\n- 原始数据散落在临时文件中，清洗与结构化存储需额外编写大量胶水代码，严重拖慢分析进度。\n\n### 使用 crawlee-python 后\n- 默认配置即可模拟人类浏览行为，自动绕过主流反爬防护，配合内置代理轮换确保持续稳定采集。\n- 无缝集成 Playwright 与 BeautifulSoup，轻松处理动态加载内容，用简洁的代码逻辑实现复杂页面的高效抓取。\n- 内置强大的容错机制与自动重试策略，即使遇到网络异常也能自动恢复，确保每日全量数据零丢失。\n- 直接输出机器可读的结构化数据（如 JSON、CSV），并可一键持久化存储，让数据分析师能立即投入模型训练。\n\ncrawlee-python 将原本需要数天构建和维护的脆弱爬虫系统，转变为几小时内即可部署的高可靠性数据管道。","https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fapify_crawlee-python_0a543ce0.png","apify","Apify","https:\u002F\u002Foss.gittoolsai.com\u002Favatars\u002Fapify_81fc5119.png","Let's put the web to work.",null,"hello@apify.com","https:\u002F\u002Fapify.com\u002F","https:\u002F\u002Fgithub.com\u002Fapify",[82,86,90,94,97,101,104],{"name":83,"color":84,"percentage":85},"Python","#3572A5",76,{"name":87,"color":88,"percentage":89},"MDX","#fcb32c",17.7,{"name":91,"color":92,"percentage":93},"JavaScript","#f1e05a",4,{"name":95,"color":96,"percentage":32},"CSS","#663399",{"name":98,"color":99,"percentage":100},"Lua","#000080",0.2,{"name":102,"color":103,"percentage":100},"Dockerfile","#384d54",{"name":105,"color":106,"percentage":107},"Shell","#89e051",0,8742,705,"2026-04-10T02:08:09","Apache-2.0","Linux, macOS, Windows","非必需。若使用 PlaywrightCrawler 进行浏览器自动化，依赖系统显卡驱动，无特定 CUDA 版本要求；BeautifulSoupCrawler 无需 GPU。","未说明（根据并发量和是否启用浏览器动态调整）",{"notes":116,"python":117,"dependencies":118},"1. 核心功能通过 'crawlee' 包安装，可选功能（如 beautifulsoup, playwright）需作为 extras 安装。2. 若使用 PlaywrightCrawler，安装后必须运行 'playwright install' 下载浏览器内核。3. 推荐使用 'uv' 工具配合 CLI 快速创建项目模板。4. 该库基于 Asyncio 构建，支持自动代理轮换和会话管理。","3.9+",[119,120,121,122],"httpx","beautifulsoup4","playwright","uv",[13,52,16],[73,125,126,127,128,129,130,131,121,132,133,134,135,136,137,138,139],"automation","beautifulsoup","crawler","crawling","headless","headless-chrome","pip","python","scraper","scraping","web-crawler","web-crawling","web-scraping","hacktoberfest","parsel","2026-03-27T02:49:30.150509","2026-04-10T15:51:24.575231",[143,148,153,157,162,167],{"id":144,"question_zh":145,"answer_zh":146,"source_url":147},27780,"如何在不同的处理器（handlers）之间传递数据？","在 Crawlee Python 中，若需在列表页处理器和详情页处理器之间共享数据（例如先抓取部分数据，再在详情页补充），目前官方尚未提供直接的内置机制。常见的变通方案包括：\n1. 使用 Pydantic 模型进行序列化：利用 `model_dump_json` 和 `model_validate_json` 处理包含 datetime、set 等复杂类型的数据，避免手动编写 `json.dumps` 的 `default` 参数。\n2. 利用请求上下文：虽然 `enqueue_links` 主要用于添加新请求，但可以通过自定义请求头或用户数据字段（如果支持）来携带少量状态信息。\n3. 外部存储：对于大量数据，建议暂时存入 Redis 或本地文件，通过唯一 ID 关联。\n注意：直接使用 `multiprocessing` 可能会导致序列化错误（segmentation faults），因为需要反序列化自定义类型。","https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fissues\u002F524",{"id":149,"question_zh":150,"answer_zh":151,"source_url":152},27781,"Crawlee Python 支持哪些 HTML 解析器？是否推荐使用 Scrapling？","Crawlee Python 默认支持 BeautifulSoup 和 Parsel。社区用户推荐使用 [Scrapling](https:\u002F\u002Fgithub.com\u002FD4Vinci\u002FScrapling)，因为它基于 lxml 构建，速度更快（基准测试显示比 BS4+lxml 快约 698 倍），且支持更强大的选择器功能（如按文本内容搜索、正则匹配、自动修复蜘蛛等）。\n若要在项目中使用 Scrapling，需注意其 API 已更新：移除了 `xpath_first`\u002F`css_first` 方法，现在应使用类似 Scrapy\u002FParsel 的语法，例如 `page.css('title')[0].text` 或 `page.css('title::text').get()`。此外，新版 Scrapling 已通过 `mypy` 和 `pyright` 的类型检查。","https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fissues\u002F1392",{"id":154,"question_zh":155,"answer_zh":156,"source_url":152},27782,"在使用 Scrapling 进行类型检查时，遇到 'Item has no attribute' 的错误怎么办？","当使用 `mypy` 或 `pyright` 检查 Scrapling 代码时，如果直接访问 `xpath_first` 或 `css_first` 返回结果的属性（如 `.text`），可能会报类型错误，因为这些方法可能返回 `None`。\n错误示例：\n```python\ntitle_el = page.xpath_first('\u002F\u002Ftitle')\ntitle_el.text  # 报错：NoneType 没有 text 属性\n```\n解决方案是显式处理 `None` 情况：\n```python\ntitle_el = page.xpath_first('\u002F\u002Ftitle')\nif title_el:\n    print(title_el.text)\n# 或者使用三元表达式\nresult = title_el.text if title_el else None\n```\n尽管 Scrapling 库本身已类型安全，但在用户代码中仍需遵循 Python 的类型检查规则来处理可选值。",{"id":158,"question_zh":159,"answer_zh":160,"source_url":161},27783,"如何从 `poetry` 和 `pipx` 迁移到 `uv` 和 `uvx`？","Crawlee 生态正在逐步迁移到 `uv` 工具链以提升性能。迁移步骤如下：\n1. 安装 `uv`：确保系统已安装 uv（支持 Windows）。\n2. 替换命令：将原有的 `pipx run crawlee create` 替换为更简洁的 `uvx crawlee create`。\n3. 项目依赖：在现有项目中，可以使用 `uv add` 替代 `poetry add` 来管理依赖。\n4. 文档更新：官方文档首页的命令示例已更新为 `uvx` 形式。\n目前该迁移已在 `apify-sdk-python`、`crawlee-python` 等核心库中完成，模板项目也在同步更新中。","https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fissues\u002F628",{"id":163,"question_zh":164,"answer_zh":165,"source_url":166},27784,"为什么 `AutoscaledPool` 打印的平均内存使用率会突然从 0 跳到 1.0？","这是一个已知的日志显示问题。在 Apify 平台或本地运行时，`AutoscaledPool` 输出的内存使用率（mem）有时会显示不准确，表现为初始值为 0.0，然后突然跳变为 1.0（即 100%），而实际 CPU 和其他指标变化是渐进的。\n这通常是由于内存采样或计算逻辑在特定时间窗口的聚合方式导致的视觉误差，并不代表实际内存瞬间耗尽。如果任务未因内存不足被终止，可忽略此日志跳跃现象。建议结合操作系统的监控工具（如 `top` 或云平台的监控面板）来确认真实的内存使用情况。","https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fissues\u002F662",{"id":168,"question_zh":169,"answer_zh":170,"source_url":147},27785,"Pydantic 在处理爬虫数据序列化时有什么优势？","在 Crawlee 中处理包含非标准 JSON 类型（如 `datetime`、`set`、`decimal`）的数据时，使用 Pydantic 可以避免手动编写繁琐的序列化逻辑。\n优势包括：\n1. 自动序列化：使用 `model_dump_json()` 可直接将包含日期时间的对象转换为 ISO 格式字符串，无需自定义 `json.dumps` 的 `default` 函数。\n2. 自动反序列化：使用 `model_validate_json()` 可将 JSON 字符串还原为包含正确类型（如 datetime 对象）的模型实例。\n3. 类型检查：提供编译时的类型安全保障。\n虽然对于小型爬虫可能显得厚重，但在处理复杂数据结构时，它能显著减少样板代码并提高健壮性。",[172,177,182,187,192,197,202,207,212,217,222,227,232,237,242,247,252,257,262,267],{"id":173,"version":174,"summary_zh":175,"released_at":176},188684,"v1.6.2","## [1.6.2](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Freleases\u002Ftag\u002Fv1.6.2)（2026-04-08）\n\n### 🐛 问题修复\n\n- **file-system:** 在 RQ 恢复时回收孤立的进行中请求（[#1825](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1825)）（[e86794a](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fe86794a6e5605432c9331c7cd99edf885527a3eb)），由 @vdusek 提供\n- 防止多个爬虫共享 `EventManager` 时过早关闭它（[#1810](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1810)）（[2efb668](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F2efb668ad54fb3e8d740066446563d1e8a39d2e8)），由 @Mantisus 提供\n- 将 SQLite 优化应用于 `SqlStorageClient` 中的自定义 `connection_string`（[#1837](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1837)）（[8b53e27](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F8b53e273067e27b4ef4b2b4bb40277b15ef6b058)），由 @Mantisus 提供\n- 将 `SharedTimeout` 应用于导航后钩子（[#1839](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1839)）（[88bd05a](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F88bd05a2127ebfe3cd4eb78c514a63fc9e2cd079)），由 @vdusek 提供\n\n","2026-04-08T13:06:07",{"id":178,"version":179,"summary_zh":180,"released_at":181},188685,"v1.6.1","## [1.6.1](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Freleases\u002Ftag\u002Fv1.6.1)（2026-03-30）\n\n### 🐛 问题修复\n\n- 处理 `RequestList` 中的无效 URL（[#1803](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1803)）（[0b2e3fc](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F0b2e3fc5cbca371131b54085e052a6cda6361b0f)），由 @Mantisus 提供\n- **playwright:** 在持久化浏览器中过滤不支持的上下文选项（[#1796](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1796)）（[69ad22e](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F69ad22e60ef558d8c26e84e2bd165fe03f116b7f)），由 @sushant-mutnale 提供\n- 移除 `Session.retire()` 中的重复 `usage_count` 增量（[#1816](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1816)）（[c40d411](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fc40d411b024ba2aae531a3c97609f78ad2c2757e)），由 @vdusek 提供\n- 延迟页面对象的清理，使其在错误处理程序中可用（[#1814](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1814)）（[7eeb500](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F7eeb5007cfb911901203ea21e1fd40127641feb1)），由 @janbuchar 提供\n\n### ⚡ 性能优化\n\n- 通过 `asyncio.to_thread` 将 BeautifulSoup 解析卸载到线程中（[#1817](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1817)）（[d612ffa](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fd612ffa1730f2aacfb7a28ae2b0ce2f4eda77692)），由 @vdusek 提供\n","2026-03-30T12:51:46",{"id":183,"version":184,"summary_zh":185,"released_at":186},188686,"v1.6.0","## [1.6.0](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Freleases\u002Ftag\u002Fv1.6.0) (2026-03-20)\n\n### 🚀 新特性\n\n- 允许提取并入队非 `href` 链接（[#1781](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1781)）（[6db365d](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F6db365d1625206d8d691256c9cd4b44a821238bb)）由 @kozlice 提供\n- 为爬虫添加 `post_navigation_hooks`（[#1795](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1795)）（[38ceda6](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F38ceda635a18cb2f14efc7c8e8b67f3adb7e53fd)）由 @Mantisus 提供\n- 为 `BrowserPool` 添加页面生命周期钩子（[#1791](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1791)）（[6f2ac13](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F6f2ac13fea4cfa8a65e6e41430d3e8d28cc3a787)）由 @Mantisus 提供\n- 暴露 `BrowserType` 和 `CrawleePage`（[#1798](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1798)）（[b50b9f2](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fb50b9f2a8396dcee2bd7eaf76c94d24912c2bc5f)）由 @Mantisus 提供\n- 在 `BasicCrawler` 中暴露 `use_state`（[#1799](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1799)）（[d121873](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fd121873a7f5902b911dd04b4aa9eaf75a8449323)）由 @Mantisus 提供\n\n### 🐛 修复\n\n- **redis:** 不从请求队列中移除已处理的请求数据（[#1787](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1787)）（[3008c61](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F3008c61dcbe07ccdf3c43f198b37582cc1356c9a)）由 @kozlice 提供\n- **redis:** 更新请求队列 Redis 存储客户端中的实际 `Request` 状态（[#1789](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1789)）（[787231c](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F787231cebeb863ee2b4395964a79a37053dbec01)）由 @Mantisus 提供\n","2026-03-20T08:12:02",{"id":188,"version":189,"summary_zh":190,"released_at":191},188687,"v1.5.0","## [1.5.0](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Freleases\u002Ftag\u002Fv1.5.0) (2026-03-06)\n\n### 🚀 新特性\n\n- 在模板中使用专用的 Playwright Docker 镜像 ([#1757](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1757)) ([747c0cf](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F747c0cf4a82296a2e3ea5cac5ef4c9578ea62a0c)) 由 @Pijukatel 提供\n- 添加 `discover_valid_sitemaps` 工具函数 ([#1777](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1777)) ([872447b](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F872447b60bbdb3926068064a971492807b1bdfbb)) 由 @Mantisus 提供\n\n### 🐛 修复缺陷\n\n- 防止在 BrowserPool 迭代过程中修改列表 ([#1703](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1703)) ([70309d9](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F70309d9bf568d268a26b3ba6392be2b6ff284c65)) 由 @vdusek 提供\n- 修复 `max_requests_per_crawl` 不排除失败请求的问题 ([#1766](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1766)) ([d6bb0b4](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fd6bb0b4a9dc5dd6668d076fbfa1b5e748deaee0d)) 由 @Pijukatel 提供\n- **playwright:** 为 `send_request` 释放 `APIResponse` 的响应体 ([#1771](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1771)) ([29d301b](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F29d301bf9d7795f2fbaddb99235a7157b880f60c)) 由 @kozlice 提供\n- 当存储客户端无法将请求入队时，`add_request` 返回 `None` ([#1775](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1775)) ([944753a](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F944753a71956c30f3ce0896ffa24be7de5348933)) 由 @Mantisus 提供\n- 在 `PlaywrightBrowserController` 中复用已存在的浏览器上下文 ([#1778](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1778)) ([4487543](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F44875433df83d433aa69ada458b91df3ad569f5e)) 由 @Pijukatel 提供\n\n","2026-03-06T13:21:14",{"id":193,"version":194,"summary_zh":195,"released_at":196},188688,"v1.4.0","## [1.4.0](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Freleases\u002Ftag\u002Fv1.4.0) (2026-02-17)\n\n### 🚀 新特性\n\n- 动态内存快照  ([#1715](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1715)) ([568a7b1](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F568a7b186dedda19ad814ee8af3cd8e256cc4ad9)) 由 @Pijukatel 提供\n- 为 `SqlStorageClient` 添加 `MySQL` 和 `MariaDB` 支持  ([#1749](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1749)) ([202b500](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F202b5009ea5d35ea779eb5b8db1fc575f90ca7bb)) 由 @Mantisus 提供\n\n### 🐛 修复缺陷\n\n- 统一 `ServiceLocator` 中的日志级别  ([#1746](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1746)) ([4163413](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F4163413049485b035c38efd6a4a7d41502a44cfc)) 由 @janbuchar 提供\n- 修复 `PlaywrightCrawler` 不小心设置全局配置的问题  ([#1747](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1747)) ([fa58438](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Ffa58438026eb72a6002c8d494725bf4e48b4407e)) 由 @Pijukatel 提供\n- 修复 `Snapshotter` 对乱序样本的处理问题  ([#1735](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1735)) ([387c712](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F387c712306055d901b1c0df4a9666967f039aefd)) 由 @Pijukatel 提供\n\n### ⚡ 性能优化\n\n- 优化 `SqlStorageClient` 中元数据记录的处理  ([#1551](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1551)) ([df1347a](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fdf1347aacf05c05980000d15b36b65996119ea86)) 由 @Mantisus 提供\n\n","2026-02-17T11:30:46",{"id":198,"version":199,"summary_zh":200,"released_at":201},188689,"v1.3.2","## [1.3.2](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Freleases\u002Ftag\u002Fv1.3.2)（2026-02-09）\n\n### 🐛 问题修复\n\n- 对 `request_max_duration` 统计使用 `max()` 而不是 `min()`（[#1701](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1701)）（[85c4335](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F85c43351a05ada1369b720061f6f1a7e158340b6)），由 @vdusek 提供\n- 防止在 `block_requests` 中修改默认的 URL 模式列表（[#1702](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1702)）（[fcf9adb](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Ffcf9adb6a0cfeaa87ca482372d4e066584eb28d6)），由 @vdusek 提供\n- 在 `Request` 中保留 `user_data` 的 `None` 值（[#1707](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1707)）（[3c575bc](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F3c575bc2b0f1c89c99d134ad3a3fa7455ccc6910)），由 @Mantisus 提供\n- 在并发调用 `new_page` 时，让 `PlaywrightBrowserController` 尊重 `max_open_pages_per_browser` 限制（[#1712](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1712)）（[2e5534b](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F2e5534b98913d5cbd6b721b2423d063772024417)），由 @Mantisus 提供\n","2026-02-09T14:08:47",{"id":203,"version":204,"summary_zh":205,"released_at":206},188690,"v1.3.1","## [1.3.1](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Freleases\u002Ftag\u002Fv1.3.1)（2026-01-30）\n\n### 🐛 修复缺陷\n\n- 使用 `purge` 方法重置 `RequestQueue` 元数据中的所有计数器（[#1686](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1686)）（[ee09260](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fee0926084589f1b6e15840b6185ec5433be3b72f)），由 @Mantisus 提供\n- 为 `ImpitHttpClient` 设置默认值 `http3=False`（[#1685](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1685)）（[3f390f6](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F3f390f677540a3905038d7db6a6d1efad32fd045)），由 @Mantisus 提供\n- 防止 `get_request` 永久阻塞请求（[#1684](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1684)）（[da416f9](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fda416f98fb453904d62e7d29d8f24611ffb3ba8d)），由 @Mirza-Samad-Ahmed-Baig 提供\n- 除非明确请求，否则不同爬虫之间不共享状态（[#1669](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1669)）（[64c246b](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F64c246bedea14f86e607d23adc5bec644c578364)），由 @Pijukatel 提供\n\n","2026-01-30T08:18:41",{"id":208,"version":209,"summary_zh":210,"released_at":211},188691,"v1.3.0","## [1.3.0](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Freleases\u002Ftag\u002Fv1.3.0) (2026-01-20)\n\n### 🚀 新特性\n\n- 为 `AdaptivePlaywrightCrawler` 暴露 `AdaptivePlaywrightCrawlerStatisticState` ([#1635](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1635)) ([1bb4bcb](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F1bb4bcb4ccbec347ad9c14f70e9e946d48e3c38e))，由 @Mantisus 提供\n\n### 🐛 修复 bug\n\n- 防止并发存储创建中的竞态条件 ([#1626](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1626)) ([7f17a43](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F7f17a4347d5884962767e757a92ec173688fed7b))，由 @Mantisus 提供\n- 在使用自定义解析器初始化时，为 `AdaptivePlaywrightCrawler` 创建正确的统计信息 ([#1637](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1637)) ([bff7260](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fbff726055dd0d7e07a2c546b15cbee22abd85960))，由 @Mantisus 提供\n- 修复在使用 `limit` 参数时，`EnqueueLinksFunction` 添加额外链接的问题 ([#1674](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1674)) ([71d7867](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F71d7867b14f7f07cac06899f5da006091af4a954))，由 @Mantisus 提供\n","2026-01-20T09:18:10",{"id":213,"version":214,"summary_zh":215,"released_at":216},188692,"v1.2.1","## [1.2.1](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Freleases\u002Ftag\u002Fv1.2.1)（2025-12-16）\n\n### 🐛 问题修复\n\n- 修复错误摘要过短的问题（[#1605](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1605)）（[b751208](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fb751208d9a56e9d923e4559baeba35e2eede0450)），由 @Pijukatel 提供\n- 冻结核心 `Request` 字段（[#1603](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1603)）（[ae6d86b](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fae6d86b8c82900116032596201d94cd7875aaadc)），由 @Mantisus 提供\n- 在 `enqueue_links` 中，重定向后仍尊重 `enqueue_strategy` 配置（[#1607](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1607)）（[700df91](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F700df91bc9be1299388030a3e48e4dbc6f5b85a0)），由 @Mantisus 提供\n- 在请求处理器失败时，保护 `Request` 对象免受部分修改（[#1585](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1585)）（[a69caf8](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fa69caf87edecc755287c53c8cc0ca4725af5d411)），由 @Mantisus 提供\n\n","2025-12-16T17:22:26",{"id":218,"version":219,"summary_zh":220,"released_at":221},188693,"v1.2.0","## [1.2.0](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Freleases\u002Ftag\u002Fv1.2.0) (2025-12-08)\n\n### 🚀 新特性\n\n- 为 Crawler 的 export_data 方法添加额外的关键字参数 ([#1597](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1597)) ([5977f37](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F5977f376b93a7c0d4dd53f0d331a4b04fedba2c6))，由 @vdusek 提供\n- 为 `PlaywrightCrawler` 添加 `goto_options` 参数 ([#1599](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1599)) ([0b82f3b](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F0b82f3b6fb175223ea2aa5b348afcd5fdb767972))，由 @Mantisus 提供\n\n### 🐛 修复 bug\n\n- 仅将 requestHandlerTimeout 应用于请求处理器 ([#1474](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1474)) ([0dfb6c2](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F0dfb6c2a13b6650736245fa39b3fbff397644df7))，由 @janbuchar 提供\n- 处理 `error_handler` 返回 `Request` 的情况 ([#1595](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1595)) ([8a961a2](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F8a961a2b07d0d33a7302dbb13c17f3d90999d390))，由 @Mantisus 提供\n- 使 `Request.state` 的状态转换与 `Request` 的生命周期保持一致 ([#1601](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1601)) ([383225f](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F383225f9f055d95ffb1302b8cf96f42ec264f1fc))，由 @Mantisus 提供\n","2025-12-08T13:59:51",{"id":223,"version":224,"summary_zh":225,"released_at":226},188694,"v1.1.1","## [1.1.1](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Freleases\u002Ftag\u002Fv1.1.1) (2025-12-02)\n\n### 🐛 Bug Fixes\n\n- Unify separators in `unique_key` construction ([#1569](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1569)) ([af46a37](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Faf46a3733b059a8052489296e172f005def953f7)) by @vdusek\n- Fix `same-domain` strategy ignoring public suffix  ([#1572](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1572)) ([3d018b2](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F3d018b21a28a4bee493829783057188d6106a69b)) by @Pijukatel\n- Make context helpers work in `FailedRequestHandler` and `ErrorHandler` ([#1570](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1570)) ([b830019](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fb830019350830ac33075316061659e2854f7f4a5)) by @Pijukatel\n- Fix non-ASCII character corruption in `FileSystemStorageClient` on systems without UTF-8 default encoding ([#1580](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1580)) ([f179f86](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Ff179f8671b0b6af9264450e4fef7e49d1cecd2bd)) by @Mantisus\n- Respect `<base>` when enqueuing ([#1590](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1590)) ([de517a1](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fde517a1629cc29b20568143eb64018f216d4ba33)) by @Mantisus\n\n","2025-12-02T10:26:32",{"id":228,"version":229,"summary_zh":230,"released_at":231},188695,"v1.1.0","## [1.1.0](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Freleases\u002Ftag\u002Fv1.1.0) (2025-11-18)\n\n### 🚀 Features\n\n- Add `chrome` `BrowserType` for `PlaywrightCrawler` to use the Chrome browser ([#1487](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1487)) ([b06937b](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fb06937bbc3afe3c936b554bfc503365c1b2c526b)) by @Mantisus\n- Add `RedisStorageClient` based on Redis v8.0+ ([#1406](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1406)) ([d08d13d](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fd08d13d39203c24ab61fe254b0956d6744db3b5f)) by @Mantisus\n- Add support for Python 3.14 ([#1553](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1553)) ([89e9130](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F89e9130cabee0fbc974b29c26483b7fa0edf627c)) by @Mantisus\n- Add `transform_request_function` parameter for `SitemapRequestLoader` ([#1525](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1525)) ([dc90127](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fdc901271849b239ba2a947e8ebff8e1815e8c4fb)) by @Mantisus\n\n### 🐛 Bug Fixes\n\n- Improve indexing of the `request_queue_records` table for `SqlRequestQueueClient` ([#1527](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1527)) ([6509534](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F65095346a9d8b703b10c91e0510154c3c48a4176)) by @Mantisus\n- Improve error handling for `RobotsTxtFile.load` ([#1524](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1524)) ([596a311](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F596a31184914a254b3e7a81fd2f48ea8eda7db49)) by @Mantisus\n- Fix `crawler_runtime` not being updated during run and only in the end ([#1540](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1540)) ([0d6c3f6](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F0d6c3f6d3337ddb6cab4873747c28cf95605d550)) by @Pijukatel\n- Ensure persist state event emission when exiting `EventManager` context ([#1562](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1562)) ([6a44f17](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F6a44f172600cbcacebab899082d6efc9105c4e03)) by @Pijukatel\n\n","2025-11-18T09:36:58",{"id":233,"version":234,"summary_zh":235,"released_at":236},188696,"v1.0.4","## [1.0.4](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Freleases\u002Ftag\u002Fv1.0.4) (2025-10-24)\n\n### 🐛 Bug Fixes\n\n- Respect `enqueue_strategy` in `enqueue_links` ([#1505](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1505)) ([6ee04bc](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F6ee04bc08c50a70f2e956a79d4ce5072a726c3a8)) by @Mantisus\n- Exclude incorrect links before checking `robots.txt` ([#1502](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1502)) ([3273da5](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F3273da5fee62ec9254666b376f382474c3532a56)) by @Mantisus\n- Resolve compatibility issue between `SqlStorageClient` and `AdaptivePlaywrightCrawler` ([#1496](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1496)) ([ce172c4](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fce172c425a8643a1d4c919db4f5e5a6e47e91deb)) by @Mantisus\n- Fix `BasicCrawler` statistics persistence ([#1490](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1490)) ([1eb1c19](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F1eb1c19aa6f9dda4a0e3f7eda23f77a554f95076)) by @Pijukatel\n- Save context state in result for `AdaptivePlaywrightCrawler` after isolated processing in `SubCrawler` ([#1488](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1488)) ([62b7c70](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F62b7c70b54085fc65a660062028014f4502beba9)) by @Mantisus\n\n","2025-10-24T09:50:23",{"id":238,"version":239,"summary_zh":240,"released_at":241},188697,"v1.0.3","## [1.0.3](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Freleases\u002Ftag\u002Fv1.0.3) (2025-10-17)\n\n### 🐛 Bug Fixes\n\n- Add support for Pydantic v2.12 ([#1471](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1471)) ([35c1108](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F35c110878c2f445a2866be2522ea8703e9b371dd)) by @Mantisus\n- Fix database version warning message ([#1485](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1485)) ([18a545e](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F18a545ee8add92e844acd0068f9cb8580a82e1c9)) by @Mantisus\n- Fix `reclaim_request` in `SqlRequestQueueClient` to correctly update the request state ([#1486](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1486)) ([1502469](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F150246957f8f7f1ceb77bb77e3a02a903c50cae1)) by @Mantisus\n- Fix `KeyValueStore.auto_saved_value` failing in some scenarios ([#1438](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1438)) ([b35dee7](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fb35dee78180e57161b826641d45a61b8d8f6ef51)) by @Pijukatel\n\n","2025-10-17T08:54:56",{"id":243,"version":244,"summary_zh":245,"released_at":246},188698,"v1.0.2","## [1.0.2](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Freleases\u002Ftag\u002Fv1.0.2) (2025-10-08)\n\n### 🐛 Bug Fixes\n\n- Use Self type in the open() method of storage clients ([#1462](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1462)) ([4ec6f6c](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F4ec6f6c08f81632197f602ff99151338b3eba6e7)) by @janbuchar\n- Add storages name validation ([#1457](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1457)) ([84de11a](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F84de11a3a603503076f5b7df487c9abab68a9015)) by @Mantisus\n- Pin pydantic version to <2.12.0 to avoid compatibility issues ([#1467](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1467)) ([f11b86f](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Ff11b86f7ed57f98e83dc1b52f15f2017a919bf59)) by @vdusek\n\n","2025-10-08T07:58:24",{"id":248,"version":249,"summary_zh":250,"released_at":251},188699,"v1.0.1","## [1.0.1](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Freleases\u002Ftag\u002Fv1.0.1) (2025-10-06)\n\n### 🐛 Bug Fixes\n\n- Fix memory leak in `PlaywrightCrawler` on browser context creation ([#1446](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1446)) ([bb181e5](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fbb181e58d8070fba38e62d6e57fe981a00e5f035)) by @Pijukatel\n- Update templates to handle optional httpx client ([#1440](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1440)) ([c087efd](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fc087efd39baedf46ca3e5cae1ddc1acd6396e6c1)) by @Pijukatel\n\n","2025-10-06T08:27:57",{"id":253,"version":254,"summary_zh":255,"released_at":256},188700,"v1.0.0","## [1.0.0](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Freleases\u002Ftag\u002Fv1.0.0) (2025-09-29)\r\n\r\n- Check out the [Release blog post](https:\u002F\u002Fcrawlee.dev\u002Fblog\u002Fcrawlee-for-python-v1) for more details.\r\n- Check out the [Upgrading guide](https:\u002F\u002Fcrawlee.dev\u002Fpython\u002Fdocs\u002Fupgrading\u002Fupgrading-to-v1) to ensure a smooth update.\r\n\r\n### 🚀 Features\r\n\r\n- Add utility for load and parse Sitemap and `SitemapRequestLoader` ([#1169](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1169)) ([66599f8](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F66599f8d085f3a8622e130019b6fdce2325737de)) by @Mantisus\r\n- Add periodic status logging and `status_message_callback` parameter for customization ([#1265](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1265)) ([b992fb2](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fb992fb2a457dedd20fc3014d7a4a8afe14602342)) by @Mantisus\r\n- Add crawlee-cli option to skip project installation ([#1294](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1294)) ([4d5aef0](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F4d5aef05613d10c1442fe449d1cf0f63392c98e3)) by @Pijukatel\r\n- Improve `Crawlee` CLI help text ([#1297](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1297)) ([afbe10f](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fafbe10f15d93353f5bc551bf9f193414179d0dd7)) by @Pijukatel\r\n- Add basic `OpenTelemetry` instrumentation ([#1255](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1255)) ([a92d8b3](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fa92d8b3f843ee795bba7e14710bb1faa1fdbf292)) by @Pijukatel\r\n- Add `ImpitHttpClient` http-client client using the `impit` library ([#1151](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1151)) ([0d0d268](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F0d0d2681a4379c0e7ba54c49c86dabfef641610f)) by @Mantisus\r\n- Prevent overloading system memory when running locally ([#1270](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1270)) ([30de3bd](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F30de3bd7722cbc34db9fc582b4bda7dc2dfa90ff)) by @janbuchar\r\n- Expose `PlaywrightPersistentBrowser` class ([#1314](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1314)) ([b5fa955](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fb5fa95508d7c099ff3a342577f338439283a975f)) by @Mantisus\r\n- Add `impit` option for Crawlee CLI ([#1312](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1312)) ([508d7ce](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F508d7ce4d998f37ab2adcf9c057c3c635a69f863)) by @Mantisus\r\n- Persist RequestList state ([#1274](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1274)) ([cc68014](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fcc680147ba3cc8b35b9da70274e53e6f5dd92434)) by @janbuchar\r\n- Persist `DefaultRenderingTypePredictor` state ([#1340](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1340)) ([fad4c25](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Ffad4c25fc712915c4a45b24e3290b6f5dbd8a683)) by @Mantisus\r\n- Persist the `SitemapRequestLoader` state ([#1347](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1347)) ([27ef9ad](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F27ef9ad194552ea9f1321d91a7a52054be9a8a51)) by @Mantisus\r\n- Add support for NDU storages ([#1401](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1401)) ([5dbd212](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F5dbd212663e7abc37535713f4c6e3a5bbf30a12e)) by @vdusek\r\n- Add RQ id, name, alias args to `add_requests` and `enqueue_links` methods ([#1413](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1413)) ([1cae2bc](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F1cae2bca0b1508fcb3cb419dc239caf33e20a7ef)) by @Mantisus\r\n- Add `SqlStorageClient` based on `sqlalchemy` v2+ ([#1339](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1339)) ([07c75a0](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F07c75a078b443b58bfaaeb72eb2aa1439458dc47)) by @Mantisus\r\n\r\n### 🐛 Bug Fixes\r\n\r\n- Fix memory estimation not working on MacOS ([#1330](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1330)) ([ab020eb](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fab020eb821a75723225b652d64babd84c368183f)) by @Pijukatel\r\n- Fix retry count to not count the original request ([#1328](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1328)) ([74fa1d9](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F74fa1d936cb3c29cf62d87862a96b4266694af2f)) by @Pijukatel\r\n- [**breaking**] Remove unused "stats" field from RequestQueueMetadata ([#1331](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1331)) ([0a63bef](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F0a63bef514b0bdcd3d6f208b386f706d0fe561e6)) by @vdusek\r\n- Ignore unknown parameters passed in cookies ([#1336](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1336)) ([50d3ef7](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F50d3ef7540551383d26d40f3404b435bde35b47d)) by @Mantisus\r\n- Fix `timeout` for `stream` method in `ImpitHttpClient` ([#1352](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1352)) ([54b693b](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F54b693b838f135a596e1e9493b565bc558b19a3a)) by @Mantisus\r\n- Include reason in the session rotation warning logs ([#1363](https:\u002F","2025-09-29T07:11:22",{"id":258,"version":259,"summary_zh":260,"released_at":261},188701,"v0.6.12","## [0.6.12](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Freleases\u002Ftag\u002Fv0.6.12) (2025-07-30)\r\n\r\n### 🚀 Features\r\n\r\n- Add `retire_browser_after_page_count` parameter for `BrowserPool` ([#1266](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1266)) ([603aa2b](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F603aa2b192ef4bc42d88244bd009fffdb0614c06)) by @Mantisus\r\n\r\n### 🐛 Bug Fixes\r\n\r\n- Use `perf_counter_ns` for request duration tracking ([#1260](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1260)) ([9e92f6b](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F9e92f6b54400ce5004fbab770e2e4ac42f73148f)) by @Pijukatel, closes [#1256](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fissues\u002F1256)\r\n- Fix memory estimation not working on MacOS (#1330) ([8558954](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F8558954feeb7d5e91378186974a29851fedae9c8)) by @Pijukatel, closes [#1329](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fissues\u002F1329)\r\n- Fix retry count to not count the original request (#1328) ([1aff3aa](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F1aff3aaf0cdbe452a3731192449a445e5b2d7a63)) by @Pijukatel, closes [#1326](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fissues\u002F1326)\r\n- Ignore unknown parameters passed in cookies (#1336) ([0f2610c](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F0f2610c0ee1154dc004de60fc57fe7c9f478166a)) by @Mantisus, closes [#1333](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fissues\u002F1333)","2025-07-30T11:45:06",{"id":263,"version":264,"summary_zh":265,"released_at":266},188702,"v0.6.11","## [0.6.11](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Freleases\u002Ftag\u002Fv0.6.11) (2025-06-23)\n\n### 🚀 Features\n\n- Add `stream` method for `HttpClient` ([#1241](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1241)) ([95c68b0](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F95c68b0b2d0bf9e093c1d0ee1002625172f7a868)) by @Mantisus\n\n### 🐛 Bug Fixes\n\n- Fix `ClientSnapshot` overload calculation ([#1228](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1228)) ([a4fc1b6](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fa4fc1b6e83143650666108c289c084ea0463b80c)) by @Pijukatel\n- Use `PSS` instead of `RSS` to estimate children process memory usage on Linux ([#1210](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1210)) ([436032f](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002F436032f2de5f7d7fa1016033f1bb224159a8e6bf)) by @Pijukatel\n- Do not raise an error to check 'same-domain' if there is no hostname in the url ([#1251](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1251)) ([a6c3aab](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fa6c3aabf5f8341f215275077b6768a56118bc656)) by @Mantisus\n\n","2025-06-23T08:49:06",{"id":268,"version":269,"summary_zh":270,"released_at":271},188703,"v0.6.10","## [0.6.10](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Freleases\u002Ftag\u002Fv0.6.10) (2025-06-02)\n\n### 🐛 Bug Fixes\n\n- Allow config change on `PlaywrightCrawler` ([#1186](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1186)) ([f17bf31](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Ff17bf31456b702631aa7e0c26d4f07fd5eb7d1bd)) by @mylank\n- Add `payload` to `SendRequestFunction` to support `POST` request ([#1202](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1202)) ([e7449f2](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fe7449f206c580cb8383a66e4c9ff5f67c5ce8409)) by @Mantisus\n- Fix match check for specified enqueue strategy for requests with redirect ([#1199](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1199)) ([d84c30c](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fd84c30cbd7c94d6525d3b6e8e86b379050454c0e)) by @Mantisus\n- Set `WindowsSelectorEventLoopPolicy` only for curl-impersonate template without `playwright` ([#1209](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1209)) ([f3b839f](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Ff3b839ffc2ccc1b889b6d5928f35f57b725e27f1)) by @Mantisus\n- Add support non-GET requests for `PlaywrightCrawler` ([#1208](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1208)) ([dbb9f44](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fdbb9f44c71af15e1f86766fa0ba68281dd85fd9e)) by @Mantisus\n- Respect `EnqueueLinksKwargs` for `extract_links` function ([#1213](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fpull\u002F1213)) ([c9907d6](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python\u002Fcommit\u002Fc9907d6ff4c3a4a719b279cea77694c00a5a963d)) by @Mantisus\n\n","2025-06-02T12:09:38"]