[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"similar-philippgille--chromem-go":3,"tool-philippgille--chromem-go":64},[4,17,27,35,43,56],{"id":5,"name":6,"github_repo":7,"description_zh":8,"stars":9,"difficulty_score":10,"last_commit_at":11,"category_tags":12,"status":16},3808,"stable-diffusion-webui","AUTOMATIC1111\u002Fstable-diffusion-webui","stable-diffusion-webui 是一个基于 Gradio 构建的网页版操作界面，旨在让用户能够轻松地在本地运行和使用强大的 Stable Diffusion 图像生成模型。它解决了原始模型依赖命令行、操作门槛高且功能分散的痛点，将复杂的 AI 绘图流程整合进一个直观易用的图形化平台。\n\n无论是希望快速上手的普通创作者、需要精细控制画面细节的设计师，还是想要深入探索模型潜力的开发者与研究人员，都能从中获益。其核心亮点在于极高的功能丰富度：不仅支持文生图、图生图、局部重绘（Inpainting）和外绘（Outpainting）等基础模式，还独创了注意力机制调整、提示词矩阵、负向提示词以及“高清修复”等高级功能。此外，它内置了 GFPGAN 和 CodeFormer 等人脸修复工具，支持多种神经网络放大算法，并允许用户通过插件系统无限扩展能力。即使是显存有限的设备，stable-diffusion-webui 也提供了相应的优化选项，让高质量的 AI 艺术创作变得触手可及。",162132,3,"2026-04-05T11:01:52",[13,14,15],"开发框架","图像","Agent","ready",{"id":18,"name":19,"github_repo":20,"description_zh":21,"stars":22,"difficulty_score":23,"last_commit_at":24,"category_tags":25,"status":16},1381,"everything-claude-code","affaan-m\u002Feverything-claude-code","everything-claude-code 是一套专为 AI 编程助手（如 Claude Code、Codex、Cursor 等）打造的高性能优化系统。它不仅仅是一组配置文件，而是一个经过长期实战打磨的完整框架，旨在解决 AI 代理在实际开发中面临的效率低下、记忆丢失、安全隐患及缺乏持续学习能力等核心痛点。\n\n通过引入技能模块化、直觉增强、记忆持久化机制以及内置的安全扫描功能，everything-claude-code 能显著提升 AI 在复杂任务中的表现，帮助开发者构建更稳定、更智能的生产级 AI 代理。其独特的“研究优先”开发理念和针对 Token 消耗的优化策略，使得模型响应更快、成本更低，同时有效防御潜在的攻击向量。\n\n这套工具特别适合软件开发者、AI 研究人员以及希望深度定制 AI 工作流的技术团队使用。无论您是在构建大型代码库，还是需要 AI 协助进行安全审计与自动化测试，everything-claude-code 都能提供强大的底层支持。作为一个曾荣获 Anthropic 黑客大奖的开源项目，它融合了多语言支持与丰富的实战钩子（hooks），让 AI 真正成长为懂上",138956,2,"2026-04-05T11:33:21",[13,15,26],"语言模型",{"id":28,"name":29,"github_repo":30,"description_zh":31,"stars":32,"difficulty_score":23,"last_commit_at":33,"category_tags":34,"status":16},2271,"ComfyUI","Comfy-Org\u002FComfyUI","ComfyUI 是一款功能强大且高度模块化的视觉 AI 引擎，专为设计和执行复杂的 Stable Diffusion 图像生成流程而打造。它摒弃了传统的代码编写模式，采用直观的节点式流程图界面，让用户通过连接不同的功能模块即可构建个性化的生成管线。\n\n这一设计巧妙解决了高级 AI 绘图工作流配置复杂、灵活性不足的痛点。用户无需具备编程背景，也能自由组合模型、调整参数并实时预览效果，轻松实现从基础文生图到多步骤高清修复等各类复杂任务。ComfyUI 拥有极佳的兼容性，不仅支持 Windows、macOS 和 Linux 全平台，还广泛适配 NVIDIA、AMD、Intel 及苹果 Silicon 等多种硬件架构，并率先支持 SDXL、Flux、SD3 等前沿模型。\n\n无论是希望深入探索算法潜力的研究人员和开发者，还是追求极致创作自由度的设计师与资深 AI 绘画爱好者，ComfyUI 都能提供强大的支持。其独特的模块化架构允许社区不断扩展新功能，使其成为当前最灵活、生态最丰富的开源扩散模型工具之一，帮助用户将创意高效转化为现实。",107662,"2026-04-03T11:11:01",[13,14,15],{"id":36,"name":37,"github_repo":38,"description_zh":39,"stars":40,"difficulty_score":23,"last_commit_at":41,"category_tags":42,"status":16},3704,"NextChat","ChatGPTNextWeb\u002FNextChat","NextChat 是一款轻量且极速的 AI 助手，旨在为用户提供流畅、跨平台的大模型交互体验。它完美解决了用户在多设备间切换时难以保持对话连续性，以及面对众多 AI 模型不知如何统一管理的痛点。无论是日常办公、学习辅助还是创意激发，NextChat 都能让用户随时随地通过网页、iOS、Android、Windows、MacOS 或 Linux 端无缝接入智能服务。\n\n这款工具非常适合普通用户、学生、职场人士以及需要私有化部署的企业团队使用。对于开发者而言，它也提供了便捷的自托管方案，支持一键部署到 Vercel 或 Zeabur 等平台。\n\nNextChat 的核心亮点在于其广泛的模型兼容性，原生支持 Claude、DeepSeek、GPT-4 及 Gemini Pro 等主流大模型，让用户在一个界面即可自由切换不同 AI 能力。此外，它还率先支持 MCP（Model Context Protocol）协议，增强了上下文处理能力。针对企业用户，NextChat 提供专业版解决方案，具备品牌定制、细粒度权限控制、内部知识库整合及安全审计等功能，满足公司对数据隐私和个性化管理的高标准要求。",87618,"2026-04-05T07:20:52",[13,26],{"id":44,"name":45,"github_repo":46,"description_zh":47,"stars":48,"difficulty_score":23,"last_commit_at":49,"category_tags":50,"status":16},2268,"ML-For-Beginners","microsoft\u002FML-For-Beginners","ML-For-Beginners 是由微软推出的一套系统化机器学习入门课程，旨在帮助零基础用户轻松掌握经典机器学习知识。这套课程将学习路径规划为 12 周，包含 26 节精炼课程和 52 道配套测验，内容涵盖从基础概念到实际应用的完整流程，有效解决了初学者面对庞大知识体系时无从下手、缺乏结构化指导的痛点。\n\n无论是希望转型的开发者、需要补充算法背景的研究人员，还是对人工智能充满好奇的普通爱好者，都能从中受益。课程不仅提供了清晰的理论讲解，还强调动手实践，让用户在循序渐进中建立扎实的技能基础。其独特的亮点在于强大的多语言支持，通过自动化机制提供了包括简体中文在内的 50 多种语言版本，极大地降低了全球不同背景用户的学习门槛。此外，项目采用开源协作模式，社区活跃且内容持续更新，确保学习者能获取前沿且准确的技术资讯。如果你正寻找一条清晰、友好且专业的机器学习入门之路，ML-For-Beginners 将是理想的起点。",84991,"2026-04-05T10:45:23",[14,51,52,53,15,54,26,13,55],"数据工具","视频","插件","其他","音频",{"id":57,"name":58,"github_repo":59,"description_zh":60,"stars":61,"difficulty_score":10,"last_commit_at":62,"category_tags":63,"status":16},3128,"ragflow","infiniflow\u002Fragflow","RAGFlow 是一款领先的开源检索增强生成（RAG）引擎，旨在为大语言模型构建更精准、可靠的上下文层。它巧妙地将前沿的 RAG 技术与智能体（Agent）能力相结合，不仅支持从各类文档中高效提取知识，还能让模型基于这些知识进行逻辑推理和任务执行。\n\n在大模型应用中，幻觉问题和知识滞后是常见痛点。RAGFlow 通过深度解析复杂文档结构（如表格、图表及混合排版），显著提升了信息检索的准确度，从而有效减少模型“胡编乱造”的现象，确保回答既有据可依又具备时效性。其内置的智能体机制更进一步，使系统不仅能回答问题，还能自主规划步骤解决复杂问题。\n\n这款工具特别适合开发者、企业技术团队以及 AI 研究人员使用。无论是希望快速搭建私有知识库问答系统，还是致力于探索大模型在垂直领域落地的创新者，都能从中受益。RAGFlow 提供了可视化的工作流编排界面和灵活的 API 接口，既降低了非算法背景用户的上手门槛，也满足了专业开发者对系统深度定制的需求。作为基于 Apache 2.0 协议开源的项目，它正成为连接通用大模型与行业专有知识之间的重要桥梁。",77062,"2026-04-04T04:44:48",[15,14,13,26,54],{"id":65,"github_repo":66,"name":67,"description_en":68,"description_zh":69,"ai_summary_zh":69,"readme_en":70,"readme_zh":71,"quickstart_zh":72,"use_case_zh":73,"hero_image_url":74,"owner_login":75,"owner_name":76,"owner_avatar_url":77,"owner_bio":78,"owner_company":79,"owner_location":80,"owner_email":81,"owner_twitter":81,"owner_website":81,"owner_url":82,"languages":83,"stars":88,"forks":89,"last_commit_at":90,"license":91,"difficulty_score":92,"env_os":93,"env_gpu":94,"env_ram":94,"env_deps":95,"category_tags":98,"github_topics":99,"view_count":10,"oss_zip_url":81,"oss_zip_packed_at":81,"status":16,"created_at":115,"updated_at":116,"faqs":117,"releases":133},1203,"philippgille\u002Fchromem-go","chromem-go","Embeddable vector database for Go with Chroma-like interface and zero third-party dependencies. In-memory with optional persistence.","chromem-go是一个专为Go语言设计的嵌入式向量数据库，提供类似Chroma的接口，但没有第三方依赖。它在内存中运行，支持可选持久化，让你在Go应用中轻松实现检索增强生成（RAG）、文本搜索或推荐系统等功能，无需额外部署数据库服务。\n\n它解决了开发者在集成向量数据库时的常见痛点：传统方案需要运行独立数据库（如Chroma），增加了运维复杂度和资源开销。chromem-go通过嵌入式设计，让RAG功能直接集成到应用中，就像使用SQLite一样简单。\n\n适合Go开发者使用，尤其适用于需要快速构建知识库问答、代码搜索或实时推荐功能的场景。技术亮点在于零依赖、高性能——在普通笔记本电脑上，查询1000条文档仅需0.3毫秒，10万条仅需40毫秒，且内存占用极小。项目目前处于beta阶段，但已提供开箱即用的嵌入模型支持，让开发者能快速上手。","# chromem-go\n\n[![Go Reference](https:\u002F\u002Fpkg.go.dev\u002Fbadge\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go.svg)](https:\u002F\u002Fpkg.go.dev\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go)\n[![Build status](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Factions\u002Fworkflows\u002Fgo.yml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Factions\u002Fworkflows\u002Fgo.yml)\n[![Go Report Card](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fphilippgille_chromem-go_readme_2b4a70945b89.png)](https:\u002F\u002Fgoreportcard.com\u002Freport\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go)\n[![GitHub Releases](https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Frelease\u002Fphilippgille\u002Fchromem-go.svg)](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Freleases)\n\nEmbeddable vector database for Go with Chroma-like interface and zero third-party dependencies. In-memory with optional persistence.\n\nBecause `chromem-go` is embeddable it enables you to add retrieval augmented generation (RAG) and similar embeddings-based features into your Go app *without having to run a separate database*. Like when using SQLite instead of PostgreSQL\u002FMySQL\u002Fetc.\n\nIt's *not* a library to connect to Chroma and also not a reimplementation of it in Go. It's a database on its own.\n\nThe focus is not scale (millions of documents) or number of features, but simplicity and performance for the most common use cases. On a mid-range 2020 Intel laptop CPU you can query 1,000 documents in 0.3 ms and 100,000 documents in 40 ms, with very few and small memory allocations. See [Benchmarks](#benchmarks) for details.\n\n> ⚠️ The project is in beta, under heavy construction, and may introduce breaking changes in releases before `v1.0.0`. All changes are documented in the [`CHANGELOG`](.\u002FCHANGELOG.md).\n\n## Contents\n\n1. [Use cases](#use-cases)\n2. [Interface](#interface)\n3. [Features + Roadmap](#features)\n4. [Installation](#installation)\n5. [Usage](#usage)\n6. [Benchmarks](#benchmarks)\n7. [Development](#development)\n8. [Motivation](#motivation)\n9. [Related projects](#related-projects)\n\n## Use cases\n\nWith a vector database you can do various things:\n\n- Retrieval augmented generation (RAG), question answering (Q&A)\n- Text and code search\n- Recommendation systems\n- Classification\n- Clustering\n\nLet's look at the RAG use case in more detail:\n\n### RAG\n\nThe knowledge of large language models (LLMs) - even the ones with 30 billion, 70 billion parameters and more - is limited. They don't know anything about what happened after their training ended, they don't know anything about data they were not trained with (like your company's intranet, Jira \u002F bug tracker, wiki or other kinds of knowledge bases), and even the data they *do* know they often can't reproduce it *exactly*, but start to *hallucinate* instead.\n\nFine-tuning an LLM can help a bit, but it's more meant to improve the LLMs reasoning about specific topics, or reproduce the style of written text or code. Fine-tuning does *not* add knowledge *1:1* into the model. Details are lost or mixed up. And knowledge cutoff (about anything that happened after the fine-tuning) isn't solved either.\n\n=> A vector database can act as the up-to-date, precise knowledge for LLMs:\n\n1. You store relevant documents that you want the LLM to know in the database.\n2. The database stores the *embeddings* alongside the documents, which you can either provide or can be created by specific \"embedding models\" like OpenAI's `text-embedding-3-small`.\n   - `chromem-go` can do this for you and supports multiple embedding providers and models out-of-the-box.\n3. Later, when you want to talk to the LLM, you first send the question to the vector DB to find *similar*\u002F*related* content. This is called \"nearest neighbor search\".\n4. In the question to the LLM, you provide this content alongside your question.\n5. The LLM can take this up-to-date precise content into account when answering.\n\nCheck out the [example code](examples) to see it in action!\n\n## Interface\n\nOur original inspiration was the [Chroma](https:\u002F\u002Fwww.trychroma.com\u002F) interface, whose core API is the following (taken from their [README](https:\u002F\u002Fgithub.com\u002Fchroma-core\u002Fchroma\u002Fblob\u002F0.4.21\u002FREADME.md)):\n\n\u003Cdetails>\u003Csummary>Chroma core interface\u003C\u002Fsummary>\n\n```python\nimport chromadb\n# setup Chroma in-memory, for easy prototyping. Can add persistence easily!\nclient = chromadb.Client()\n\n# Create collection. get_collection, get_or_create_collection, delete_collection also available!\ncollection = client.create_collection(\"all-my-documents\")\n\n# Add docs to the collection. Can also update and delete. Row-based API coming soon!\ncollection.add(\n    documents=[\"This is document1\", \"This is document2\"], # we handle tokenization, embedding, and indexing automatically. You can skip that and add your own embeddings as well\n    metadatas=[{\"source\": \"notion\"}, {\"source\": \"google-docs\"}], # filter on these!\n    ids=[\"doc1\", \"doc2\"], # unique for each doc\n)\n\n# Query\u002Fsearch 2 most similar results. You can also .get by id\nresults = collection.query(\n    query_texts=[\"This is a query document\"],\n    n_results=2,\n    # where={\"metadata_field\": \"is_equal_to_this\"}, # optional filter\n    # where_document={\"$contains\":\"search_string\"}  # optional filter\n)\n```\n\n\u003C\u002Fdetails>\n\nOur Go library exposes the same interface:\n\n\u003Cdetails>\u003Csummary>chromem-go equivalent\u003C\u002Fsummary>\n\n```go\npackage main\n\nimport \"github.com\u002Fphilippgille\u002Fchromem-go\"\n\nfunc main() {\n    \u002F\u002F Set up chromem-go in-memory, for easy prototyping. Can add persistence easily!\n    \u002F\u002F We call it DB instead of client because there's no client-server separation. The DB is embedded.\n    db := chromem.NewDB()\n\n    \u002F\u002F Create collection. GetCollection, GetOrCreateCollection, DeleteCollection also available!\n    collection, _ := db.CreateCollection(\"all-my-documents\", nil, nil)\n\n    \u002F\u002F Add docs to the collection. Update and delete will be added in the future.\n    \u002F\u002F Can be multi-threaded with AddConcurrently()!\n    \u002F\u002F We're showing the Chroma-like method here, but more Go-idiomatic methods are also available!\n    _ = collection.Add(ctx,\n        []string{\"doc1\", \"doc2\"}, \u002F\u002F unique ID for each doc\n        nil, \u002F\u002F We handle embedding automatically. You can skip that and add your own embeddings as well.\n        []map[string]string{{\"source\": \"notion\"}, {\"source\": \"google-docs\"}}, \u002F\u002F Filter on these!\n        []string{\"This is document1\", \"This is document2\"},\n    )\n\n    \u002F\u002F Query\u002Fsearch 2 most similar results. You can also get by ID.\n    results, _ := collection.Query(ctx,\n        \"This is a query document\",\n        2,\n        map[string]string{\"metadata_field\": \"is_equal_to_this\"}, \u002F\u002F optional filter\n        map[string]string{\"$contains\": \"search_string\"},         \u002F\u002F optional filter\n    )\n}\n```\n\n\u003C\u002Fdetails>\n\nInitially `chromem-go` started with just the four core methods, but we added more over time. We intentionally don't want to cover 100% of Chroma's API surface though.  \nWe're providing some alternative methods that are more Go-idiomatic instead.\n\nFor the full interface see the Godoc: \u003Chttps:\u002F\u002Fpkg.go.dev\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go>\n\n## Features\n\n- [X] Zero dependencies on third party libraries\n- [X] Embeddable (like SQLite, i.e. no client-server model, no separate DB to maintain)\n- [X] Multithreaded processing (when adding and querying documents), making use of Go's native concurrency features\n- [X] Experimental WebAssembly binding\n- Embedding creators:\n  - Hosted:\n    - [X] [OpenAI](https:\u002F\u002Fplatform.openai.com\u002Fdocs\u002Fguides\u002Fembeddings\u002Fembedding-models) (default)\n    - [X] [Azure OpenAI](https:\u002F\u002Flearn.microsoft.com\u002Fen-us\u002Fazure\u002Fai-services\u002Fopenai\u002Fhow-to\u002Fembeddings)\n    - [X] [GCP Vertex AI](https:\u002F\u002Fcloud.google.com\u002Fvertex-ai\u002Fgenerative-ai\u002Fdocs\u002Fembeddings)\n    - [X] [Cohere](https:\u002F\u002Fcohere.com\u002Fmodels\u002Fembed)\n    - [X] [Mistral](https:\u002F\u002Fdocs.mistral.ai\u002Fplatform\u002Fendpoints\u002F#embedding-models)\n    - [X] [Jina](https:\u002F\u002Fjina.ai\u002Fembeddings)\n    - [X] [mixedbread.ai](https:\u002F\u002Fwww.mixedbread.ai\u002F)\n  - Local:\n    - [X] [Ollama](https:\u002F\u002Fgithub.com\u002Follama\u002Follama)\n    - [X] [LocalAI](https:\u002F\u002Fgithub.com\u002Fmudler\u002FLocalAI)\n  - Bring your own (implement [`chromem.EmbeddingFunc`](https:\u002F\u002Fpkg.go.dev\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go#EmbeddingFunc))\n  - You can also pass existing embeddings when adding documents to a collection, instead of letting `chromem-go` create them\n- Similarity search:\n  - [X] Exhaustive nearest neighbor search using cosine similarity (sometimes also called exact search or brute-force search or FLAT index)\n- Filters:\n  - [X] Document filters: `$contains`, `$not_contains`\n  - [X] Metadata filters: Exact matches\n- Storage:\n  - [X] In-memory\n  - [X] Optional immediate persistence (writes one file for each added collection and document, encoded as [gob](https:\u002F\u002Fgo.dev\u002Fblog\u002Fgob), optionally gzip-compressed)\n  - [X] Backups: Export and import of the entire DB to\u002Ffrom a single file (encoded as [gob](https:\u002F\u002Fgo.dev\u002Fblog\u002Fgob), optionally gzip-compressed and AES-GCM encrypted)\n    - Includes methods for generic `io.Writer`\u002F`io.Reader` so you can plug S3 buckets and other blob storage, see [examples\u002Fs3-export-import](examples\u002Fs3-export-import) for example code\n- Data types:\n  - [X] Documents (text)\n\n### Roadmap\n\n- Performance:\n  - Use SIMD for dot product calculation on supported CPUs (draft PR: [#48](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F48))\n  - Add [roaring bitmaps](https:\u002F\u002Fgithub.com\u002FRoaringBitmap\u002Froaring) to speed up full text filtering\n- Embedding creators:\n  - Add an `EmbeddingFunc` that downloads and shells out to [llamafile](https:\u002F\u002Fgithub.com\u002FMozilla-Ocho\u002Fllamafile)\n- Similarity search:\n  - Approximate nearest neighbor search with index (ANN)\n    - Hierarchical Navigable Small World (HNSW)\n    - Inverted file flat (IVFFlat)\n- Filters:\n  - Operators (`$and`, `$or` etc.)\n- Storage:\n  - JSON as second encoding format\n  - Write-ahead log (WAL) as second file format\n  - Optional remote storage (S3, PostgreSQL, ...)\n- Data types:\n  - Images\n  - Videos\n\n## Installation\n\n`go get github.com\u002Fphilippgille\u002Fchromem-go@latest`\n\n## Usage\n\nSee the Godoc for a reference: \u003Chttps:\u002F\u002Fpkg.go.dev\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go>\n\nFor full, working examples, using the vector database for retrieval augmented generation (RAG) and semantic search and using either OpenAI or locally running the embeddings model and LLM (in Ollama), see the [example code](examples).\n\n### Quickstart\n\nThis is taken from the [\"minimal\" example](examples\u002Fminimal):\n\n```go\npackage main\n\nimport (\n \"context\"\n \"fmt\"\n \"runtime\"\n\n \"github.com\u002Fphilippgille\u002Fchromem-go\"\n)\n\nfunc main() {\n  ctx := context.Background()\n\n  db := chromem.NewDB()\n\n  \u002F\u002F Passing nil as embedding function leads to OpenAI being used and requires\n  \u002F\u002F \"OPENAI_API_KEY\" env var to be set. Other providers are supported as well.\n  \u002F\u002F For example pass `chromem.NewEmbeddingFuncOllama(...)` to use Ollama.\n  c, err := db.CreateCollection(\"knowledge-base\", nil, nil)\n  if err != nil {\n    panic(err)\n  }\n\n  err = c.AddDocuments(ctx, []chromem.Document{\n    {\n      ID:      \"1\",\n      Content: \"The sky is blue because of Rayleigh scattering.\",\n    },\n    {\n      ID:      \"2\",\n      Content: \"Leaves are green because chlorophyll absorbs red and blue light.\",\n    },\n  }, runtime.NumCPU())\n  if err != nil {\n    panic(err)\n  }\n\n  res, err := c.Query(ctx, \"Why is the sky blue?\", 1, nil, nil)\n  if err != nil {\n    panic(err)\n  }\n\n  fmt.Printf(\"ID: %v\\nSimilarity: %v\\nContent: %v\\n\", res[0].ID, res[0].Similarity, res[0].Content)\n}\n```\n\nOutput:\n\n```text\nID: 1\nSimilarity: 0.6833369\nContent: The sky is blue because of Rayleigh scattering.\n```\n\n## Benchmarks\n\nBenchmarked on 2024-03-17 with:\n\n- Computer: Framework Laptop 13 (first generation, 2021)\n- CPU: 11th Gen Intel Core i5-1135G7 (2020)\n- Memory: 32 GB\n- OS: Fedora Linux 39\n  - Kernel: 6.7\n\n```console\n$ go test -benchmem -run=^$ -bench .\ngoos: linux\ngoarch: amd64\npkg: github.com\u002Fphilippgille\u002Fchromem-go\ncpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz\nBenchmarkCollection_Query_NoContent_100-8          13164      90276 ns\u002Fop     5176 B\u002Fop       95 allocs\u002Fop\nBenchmarkCollection_Query_NoContent_1000-8          2142     520261 ns\u002Fop    13558 B\u002Fop      141 allocs\u002Fop\nBenchmarkCollection_Query_NoContent_5000-8           561    2150354 ns\u002Fop    47096 B\u002Fop      173 allocs\u002Fop\nBenchmarkCollection_Query_NoContent_25000-8          120    9890177 ns\u002Fop   211783 B\u002Fop      208 allocs\u002Fop\nBenchmarkCollection_Query_NoContent_100000-8          30   39574238 ns\u002Fop   810370 B\u002Fop      232 allocs\u002Fop\nBenchmarkCollection_Query_100-8                    13225      91058 ns\u002Fop     5177 B\u002Fop       95 allocs\u002Fop\nBenchmarkCollection_Query_1000-8                    2226     519693 ns\u002Fop    13552 B\u002Fop      140 allocs\u002Fop\nBenchmarkCollection_Query_5000-8                     550    2128121 ns\u002Fop    47108 B\u002Fop      173 allocs\u002Fop\nBenchmarkCollection_Query_25000-8                    100   10063260 ns\u002Fop   211705 B\u002Fop      205 allocs\u002Fop\nBenchmarkCollection_Query_100000-8                    30   39404005 ns\u002Fop   810295 B\u002Fop      229 allocs\u002Fop\nPASS\nok   github.com\u002Fphilippgille\u002Fchromem-go 28.402s\n```\n\n## Development\n\n- Build: `go build .\u002F...`\n- Test: `go test -v -race -count 1 .\u002F...`\n- Benchmark:\n  - `go test -benchmem -run=^$ -bench .` (add `> bench.out` or similar to write to a file)\n  - With profiling: `go test -benchmem -run ^$ -cpuprofile cpu.out -bench .`\n    - (profiles: `-cpuprofile`, `-memprofile`, `-blockprofile`, `-mutexprofile`)\n- Compare benchmarks:\n  1. Install `benchstat`: `go install golang.org\u002Fx\u002Fperf\u002Fcmd\u002Fbenchstat@latest`\n  2. Compare two benchmark results: `benchstat before.out after.out`\n\n## Motivation\n\nIn December 2023, when I wanted to play around with retrieval augmented generation (RAG) in a Go program, I looked for a vector database that could be embedded in the Go program, just like you would embed SQLite in order to not require any separate DB setup and maintenance. I was surprised when I didn't find any, given the abundance of embedded key-value stores in the Go ecosystem.\n\nAt the time most of the popular vector databases like Pinecone, Qdrant, Milvus, Chroma, Weaviate and others were not embeddable at all or only in Python or JavaScript\u002FTypeScript.\n\nThen I found [@eliben](https:\u002F\u002Fgithub.com\u002Feliben)'s [blog post](https:\u002F\u002Feli.thegreenplace.net\u002F2023\u002Fretrieval-augmented-generation-in-go\u002F) and [example code](https:\u002F\u002Fgithub.com\u002Feliben\u002Fcode-for-blog\u002Ftree\u002Feda87b87dad9ed8bd45d1c8d6395efba3741ed39\u002F2023\u002Fgo-rag-openai) which showed that with very little Go code you could create a very basic PoC of a vector database.\n\nThat's when I decided to build my own vector database, embeddable in Go, inspired by the ChromaDB interface. ChromaDB stood out for being embeddable (in Python), and by showing its core API in 4 commands on their README and on the landing page of their website.\n\n## Related projects\n\n- Shoutout to [@eliben](https:\u002F\u002Fgithub.com\u002Feliben) whose [blog post](https:\u002F\u002Feli.thegreenplace.net\u002F2023\u002Fretrieval-augmented-generation-in-go\u002F) and [example code](https:\u002F\u002Fgithub.com\u002Feliben\u002Fcode-for-blog\u002Ftree\u002Feda87b87dad9ed8bd45d1c8d6395efba3741ed39\u002F2023\u002Fgo-rag-openai) inspired me to start this project!\n- [Chroma](https:\u002F\u002Fgithub.com\u002Fchroma-core\u002Fchroma): Looking at Pinecone, Qdrant, Milvus, Weaviate and others, Chroma stood out by showing its core API in 4 commands on their README and on the landing page of their website. It was also putting the most emphasis on its embeddability (in Python).\n- The big, full-fledged client-server-based vector databases for maximum scale and performance:\n  - [Pinecone](https:\u002F\u002Fwww.pinecone.io\u002F): Closed source\n  - [Qdrant](https:\u002F\u002Fgithub.com\u002Fqdrant\u002Fqdrant): Written in Rust, not embeddable in Go\n  - [Milvus](https:\u002F\u002Fgithub.com\u002Fmilvus-io\u002Fmilvus): Written in Go and C++, but not embeddable as of December 2023\n  - [Weaviate](https:\u002F\u002Fgithub.com\u002Fweaviate\u002Fweaviate): Written in Go, but not embeddable in Go as of March 2024 (only in Python and JavaScript\u002FTypeScript and that's experimental)\n- Some non-specialized SQL, NoSQL and Key-Value databases added support for storing vectors and (some of them) querying based on similarity:\n  - [pgvector](https:\u002F\u002Fgithub.com\u002Fpgvector\u002Fpgvector) extension for [PostgreSQL](https:\u002F\u002Fwww.postgresql.org\u002F): Client-server model\n  - [Redis](https:\u002F\u002Fgithub.com\u002Fredis\u002Fredis) ([1](https:\u002F\u002Fredis.io\u002Fdocs\u002Finteract\u002Fsearch-and-query\u002Fquery\u002Fvector-search\u002F), [2](https:\u002F\u002Fredis.io\u002Fdocs\u002Finteract\u002Fsearch-and-query\u002Fadvanced-concepts\u002Fvectors\u002F)): Client-server model\n  - [sqlite-vss](https:\u002F\u002Fgithub.com\u002Fasg017\u002Fsqlite-vss) extension for [SQLite](https:\u002F\u002Fwww.sqlite.org\u002F): Embedded, but the [Go bindings](https:\u002F\u002Fgithub.com\u002Fasg017\u002Fsqlite-vss\u002Ftree\u002F8fc44301843029a13a474d1f292378485e1fdd62\u002Fbindings\u002Fgo) require CGO. There's a [CGO-free Go library](https:\u002F\u002Fgitlab.com\u002Fcznic\u002Fsqlite) for SQLite, but then it's without the vector search extension.\n  - [DuckDB](https:\u002F\u002Fgithub.com\u002Fduckdb\u002Fduckdb) has a function to calculate cosine similarity ([1](https:\u002F\u002Fduckdb.org\u002Fdocs\u002Fsql\u002Ffunctions\u002Fnested)): Embedded, but the Go bindings use CGO\n  - [MongoDB](https:\u002F\u002Fgithub.com\u002Fmongodb\u002Fmongo)'s cloud platform offers a vector search product ([1](https:\u002F\u002Fwww.mongodb.com\u002Fproducts\u002Fplatform\u002Fatlas-vector-search)): Client-server model\n- Some libraries for vector similarity search:\n  - [Faiss](https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Ffaiss): Written in C++; 3rd party Go bindings use CGO\n  - [Annoy](https:\u002F\u002Fgithub.com\u002Fspotify\u002Fannoy): Written in C++; Go bindings use CGO ([1](https:\u002F\u002Fgithub.com\u002Fspotify\u002Fannoy\u002Fblob\u002F2be37c9e015544be2cf60c431f0cccc076151a2d\u002FREADME_GO.rst))\n  - [USearch](https:\u002F\u002Fgithub.com\u002Funum-cloud\u002Fusearch): Written in C++; Go bindings use CGO\n- Some orchestration libraries, inspired by the Python library [LangChain](https:\u002F\u002Fgithub.com\u002Flangchain-ai\u002Flangchain), but with no or only rudimentary embedded vector DB:\n  - [LangChain Go](https:\u002F\u002Fgithub.com\u002Ftmc\u002Flangchaingo)\n  - [LinGoose](https:\u002F\u002Fgithub.com\u002Fhenomis\u002Flingoose)\n  - [GoLC](https:\u002F\u002Fgithub.com\u002Fhupe1980\u002Fgolc)\n","# chromem-go\n\n[![Go 参考文档](https:\u002F\u002Fpkg.go.dev\u002Fbadge\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go.svg)](https:\u002F\u002Fpkg.go.dev\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go)\n[![构建状态](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Factions\u002Fworkflows\u002Fgo.yml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Factions\u002Fworkflows\u002Fgo.yml)\n[![Go Report Card](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fphilippgille_chromem-go_readme_2b4a70945b89.png)](https:\u002F\u002Fgoreportcard.com\u002Freport\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go)\n[![GitHub 发布](https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Frelease\u002Fphilippgille\u002Fchromem-go.svg)](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Freleases)\n\n适用于 Go 语言的可嵌入向量数据库，提供与 Chroma 类似的接口，并且零第三方依赖。支持内存存储及可选持久化。\n\n由于 `chromem-go` 是可嵌入的，它使您能够在 Go 应用程序中添加检索增强生成（RAG）以及类似的基于嵌入的功能，而无需运行单独的数据库。这就像使用 SQLite 而不是 PostgreSQL\u002FMySQL 等数据库一样。\n\n它并不是用于连接到 Chroma 的库，也不是在 Go 中对其的重新实现。它本身就是一个独立的数据库。\n\n该项目的重点并非大规模（如数百万文档）或功能数量，而是针对最常见的用例提供简单性和高性能。在一台中端 2020 年款 Intel 笔记本电脑上，您可以以 0.3 毫秒查询 1,000 条文档，以 40 毫秒查询 100,000 条文档，且内存分配极少、体积很小。详情请参阅 [基准测试](#benchmarks)。\n\n> ⚠️ 该项目目前处于 beta 阶段，开发仍在进行中，在 `v1.0.0` 之前发布的版本中可能会引入破坏性更改。所有更改均记录在 [`CHANGELOG`](.\u002FCHANGELOG.md) 中。\n\n## 目录\n\n1. [用例](#use-cases)\n2. [接口](#interface)\n3. [特性 + 路线图](#features)\n4. [安装](#installation)\n5. [使用](#usage)\n6. [基准测试](#benchmarks)\n7. [开发](#development)\n8. [动机](#motivation)\n9. [相关项目](#related-projects)\n\n## 用例\n\n借助向量数据库，您可以实现多种功能：\n\n- 检索增强生成（RAG）、问答系统（Q&A）\n- 文本和代码搜索\n- 推荐系统\n- 分类\n- 聚类\n\n让我们更详细地探讨 RAG 的用例：\n\n### RAG\n\n大型语言模型（LLM）的知识——即使是那些拥有 300 亿、700 亿参数甚至更多参数的模型——也是有限的。它们不了解训练结束之后发生的事情，也不了解未在其训练数据中出现的内容（例如贵公司的内网、Jira\u002F缺陷跟踪器、维基或其他类型的知识库）。即便它们确实知道某些内容，也往往无法精确复现，反而会产生“幻觉”。\n\n对 LLM 进行微调可以在一定程度上有所帮助，但它主要是为了提升模型在特定主题上的推理能力，或者模仿某种文本或代码的风格。微调并不能将知识以“一对一”的方式直接注入模型中。细节往往会丢失或混淆，而且也无法解决知识截止问题（即无法处理微调之后发生的事情）。\n\n因此，向量数据库可以作为 LLM 的最新、精确知识来源：\n\n1. 您将希望 LLM 了解的相关文档存储在数据库中。\n2. 数据库会同时存储文档的嵌入表示，这些嵌入可以由您提供，也可以由特定的“嵌入模型”（如 OpenAI 的 `text-embedding-3-small`）自动生成。\n   - `chromem-go` 可以为您完成这一过程，并开箱即用支持多种嵌入提供商和模型。\n3. 当您需要与 LLM 对话时，首先将问题发送到向量数据库，以查找“相似”或“相关”的内容。这被称为“最近邻搜索”。\n4. 在向 LLM 提出问题时，您将这些相关内容与问题一同提供。\n5. LLM 在回答时可以将这些最新、精确的内容纳入考虑。\n\n请查看 [示例代码](examples)，了解其实际应用！\n\n## 接口\n\n我们的最初灵感来源于 [Chroma](https:\u002F\u002Fwww.trychroma.com\u002F) 的接口，其核心 API 如下（摘自他们的 [README](https:\u002F\u002Fgithub.com\u002Fchroma-core\u002Fchroma\u002Fblob\u002F0.4.21\u002FREADME.md))：\n\n\u003Cdetails>\u003Csummary>Chroma 核心接口\u003C\u002Fsummary>\n\n```python\nimport chromadb\n# 设置内存中的 Chroma，便于快速原型设计。也可轻松添加持久化！\nclient = chromadb.Client()\n\n# 创建集合。还可以获取集合、创建或获取集合、删除集合！\ncollection = client.create_collection(\"all-my-documents\")\n\n# 向集合中添加文档。未来还将支持更新和删除操作。\n# 支持按行操作的 API 即将推出！\ncollection.add(\n    documents=[\"这是文档1\", \"这是文档2\"], # 我们会自动处理分词、嵌入和索引。您也可以跳过这些步骤，自行提供嵌入。\n    metadatas=[{\"source\": \"notion\"}, {\"source\": \"google-docs\"}], # 可以根据这些元数据进行过滤！\n    ids=[\"doc1\", \"doc2\"], # 每个文档都有唯一 ID\n)\n\n# 查询\u002F搜索最相似的 2 个结果。也可以按 ID 获取。\nresults = collection.query(\n    query_texts=[\"这是一份查询文档\"],\n    n_results=2,\n    # where={\"metadata_field\": \"is_equal_to_this\"}, # 可选过滤条件\n    # where_document={\"$contains\":\"search_string\"}  # 可选过滤条件\n)\n```\n\n\u003C\u002Fdetails>\n\n我们的 Go 库暴露了相同的接口：\n\n\u003Cdetails>\u003Csummary>chromem-go 等效接口\u003C\u002Fsummary>\n\n```go\npackage main\n\nimport \"github.com\u002Fphilippgille\u002Fchromem-go\"\n\nfunc main() {\n    \u002F\u002F 设置内存中的 chromem-go，便于快速原型设计。也可轻松添加持久化！\n    \u002F\u002F 我们称之为 DB 而不是 client，因为这里没有客户端-服务器分离。DB 是嵌入式的。\n    db := chromem.NewDB()\n\n    \u002F\u002F 创建集合。还可以获取集合、创建或获取集合、删除集合！\n    collection, _ := db.CreateCollection(\"all-my-documents\", nil, nil)\n\n    \u002F\u002F 向集合中添加文档。未来将支持更新和删除操作。\n    \u002F\u002F 可以使用 AddConcurrently() 方法进行多线程操作！\n    \u002F\u002F 这里展示的是类似 Chroma 的方法，但也有更符合 Go 语言习惯的方法！\n    _ = collection.Add(ctx,\n        []string{\"doc1\", \"doc2\"}, \u002F\u002F 每个文档都有唯一 ID\n        nil, \u002F\u002F 我们会自动处理嵌入。您也可以跳过这一步，自行提供嵌入。\n        []map[string]string{{\"source\": \"notion\"}, {\"source\": \"google-docs\"}}, \u002F\u002F 可以根据这些元数据进行过滤！\n        []string{\"这是文档1\", \"这是文档2\"},\n    )\n\n    \u002F\u002F 查询\u002F搜索最相似的 2 个结果。也可以按 ID 获取。\n    results, _ := collection.Query(ctx,\n        \"这是一份查询文档\",\n        2,\n        map[string]string{\"metadata_field\": \"is_equal_to_this\"}, \u002F\u002F 可选过滤条件\n        map[string]string{\"$contains\": \"search_string\"},         \u002F\u002F 可选过滤条件\n    )\n}\n```\n\n\u003C\u002Fdetails>\n\n最初，`chromem-go` 只实现了四个核心方法，但我们随着时间的推移又增加了更多功能。不过，我们并没有打算完全覆盖 Chroma 的全部 API 表面。相反，我们提供了更具 Go 语言风格的替代方法。\n\n完整的接口请参阅 Godoc：[https:\u002F\u002Fpkg.go.dev\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go](https:\u002F\u002Fpkg.go.dev\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go)\n\n## 特性\n\n- [X] 无第三方库依赖\n- [X] 可嵌入式（类似 SQLite，即无客户端-服务器模式，无需维护独立的数据库）\n- [X] 多线程处理（在添加和查询文档时），充分利用 Go 的原生并发特性\n- [X] 实验性 WebAssembly 绑定\n- 嵌入模型提供者：\n  - 托管服务：\n    - [X] [OpenAI](https:\u002F\u002Fplatform.openai.com\u002Fdocs\u002Fguides\u002Fembeddings\u002Fembedding-models)（默认）\n    - [X] [Azure OpenAI](https:\u002F\u002Flearn.microsoft.com\u002Fen-us\u002Fazure\u002Fai-services\u002Fopenai\u002Fhow-to\u002Fembeddings)\n    - [X] [GCP Vertex AI](https:\u002F\u002Fcloud.google.com\u002Fvertex-ai\u002Fgenerative-ai\u002Fdocs\u002Fembeddings)\n    - [X] [Cohere](https:\u002F\u002Fcohere.com\u002Fmodels\u002Fembed)\n    - [X] [Mistral](https:\u002F\u002Fdocs.mistral.ai\u002Fplatform\u002Fendpoints\u002F#embedding-models)\n    - [X] [Jina](https:\u002F\u002Fjina.ai\u002Fembeddings)\n    - [X] [mixedbread.ai](https:\u002F\u002Fwww.mixedbread.ai\u002F)\n  - 本地运行：\n    - [X] [Ollama](https:\u002F\u002Fgithub.com\u002Follama\u002Follama)\n    - [X] [LocalAI](https:\u002F\u002Fgithub.com\u002Fmudler\u002FLocalAI)\n  - 自定义实现（实现 [`chromem.EmbeddingFunc`](https:\u002F\u002Fpkg.go.dev\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go#EmbeddingFunc)）\n  - 也可以在向集合中添加文档时直接传入已有的嵌入向量，而不是让 `chromem-go` 自动生成\n- 相似度搜索：\n  - [X] 使用余弦相似度进行穷举最近邻搜索（有时也称为精确搜索、暴力搜索或 FLAT 索引）\n- 过滤器：\n  - [X] 文档过滤：`$contains`、`$not_contains`\n  - [X] 元数据过滤：精确匹配\n- 存储：\n  - [X] 内存中\n  - [X] 可选即时持久化（为每个添加的集合和文档写入一个文件，使用 [gob](https:\u002F\u002Fgo.dev\u002Fblog\u002Fgob) 编码，可选 gzip 压缩）\n  - [X] 备份：将整个数据库导出和导入到单个文件中（使用 [gob](https:\u002F\u002Fgo.dev\u002Fblog\u002Fgob) 编码，可选 gzip 压缩和 AES-GCM 加密）\n    - 包括针对通用 `io.Writer`\u002F`io.Reader` 的方法，因此可以对接 S3 存储桶和其他对象存储，示例代码参见 [examples\u002Fs3-export-import](examples\u002Fs3-export-import)\n- 数据类型：\n  - [X] 文档（文本）\n\n### 路线图\n\n- 性能：\n  - 在支持的 CPU 上使用 SIMD 进行点积计算（PR 草案：[#48](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F48)）\n  - 添加 [roaring bitmaps](https:\u002F\u002Fgithub.com\u002FRoaringBitmap\u002Froaring) 以加速全文过滤\n- 嵌入模型提供者：\n  - 添加一个 `EmbeddingFunc`，用于下载并调用 [llamafile](https:\u002F\u002Fgithub.com\u002FMozilla-Ocho\u002Fllamafile)\n- 相似度搜索：\n  - 使用索引的近似最近邻搜索 (ANN)\n    - 分层可导航小世界 (HNSW)\n    - 倒排文件平铺 (IVFFlat)\n- 过滤器：\n  - 操作符（`$and`、`$or` 等）\n- 存储：\n  - 将 JSON 作为第二种编码格式\n  - 将预写日志 (WAL) 作为第二种文件格式\n  - 可选远程存储（S3、PostgreSQL 等）\n- 数据类型：\n  - 图像\n  - 视频\n\n## 安装\n\n`go get github.com\u002Fphilippgille\u002Fchromem-go@latest`\n\n## 使用\n\n参考 Godoc：\u003Chttps:\u002F\u002Fpkg.go.dev\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go>\n\n有关完整可用的示例，包括如何使用向量数据库进行检索增强生成 (RAG) 和语义搜索，以及如何使用 OpenAI 或本地运行的嵌入模型和 LLM（通过 Ollama），请参阅 [示例代码](examples)。\n\n### 快速入门\n\n以下内容摘自 [\"minimal\" 示例](examples\u002Fminimal)：\n\n```go\npackage main\n\nimport (\n \"context\"\n \"fmt\"\n \"runtime\"\n\n \"github.com\u002Fphilippgille\u002Fchromem-go\"\n)\n\nfunc main() {\n  ctx := context.Background()\n\n  db := chromem.NewDB()\n\n  \u002F\u002F 传入 nil 作为嵌入函数会导致使用 OpenAI，并且需要设置 \"OPENAI_API_KEY\" 环境变量。其他提供商也同样支持。\n  \u002F\u002F 例如，传入 `chromem.NewEmbeddingFuncOllama(...)` 可以使用 Ollama。\n  c, err := db.CreateCollection(\"knowledge-base\", nil, nil)\n  if err != nil {\n    panic(err)\n  }\n\n  err = c.AddDocuments(ctx, []chromem.Document{\n    {\n      ID:      \"1\",\n      Content: \"天空之所以是蓝色，是因为瑞利散射。\",\n    },\n    {\n      ID:      \"2\",\n      Content: \"叶子之所以是绿色，是因为叶绿素吸收了红光和蓝光。\",\n    },\n  }, runtime.NumCPU())\n  if err != nil {\n    panic(err)\n  }\n\n  res, err := c.Query(ctx, \"为什么天空是蓝色的？\", 1, nil, nil)\n  if err != nil {\n    panic(err)\n  }\n\n  fmt.Printf(\"ID: %v\\n相似度: %v\\n内容: %v\\n\", res[0].ID, res[0].Similarity, res[0].Content)\n}\n```\n\n输出：\n\n```text\nID: 1\n相似度: 0.6833369\n内容: 天空之所以是蓝色，是因为瑞利散射。\n```\n\n## 基准测试\n\n基准测试于 2024 年 3 月 17 日进行，配置如下：\n\n- 计算机：Framework Laptop 13（第一代，2021 年）\n- CPU：第 11 代 Intel Core i5-1135G7（2020 年）\n- 内存：32 GB\n- 操作系统：Fedora Linux 39\n  - 内核：6.7\n\n```console\n$ go test -benchmem -run=^$ -bench .\ngoos: linux\ngoarch: amd64\npkg: github.com\u002Fphilippgille\u002Fchromem-go\ncpu: 第 11 代 Intel(R) Core(TM) i5-1135G7 @ 2.40GHz\nBenchmarkCollection_Query_NoContent_100-8          13164      90276 ns\u002Fop     5176 B\u002Fop       95 allocs\u002Fop\nBenchmarkCollection_Query_NoContent_1000-8          2142     520261 ns\u002Fop    13558 B\u002Fop      141 allocs\u002Fop\nBenchmarkCollection_Query_NoContent_5000-8           561    2150354 ns\u002Fop    47096 B\u002Fop      173 allocs\u002Fop\nBenchmarkCollection_Query_NoContent_25000-8          120    9890177 ns\u002Fop   211783 B\u002Fop      208 allocs\u002Fop\nBenchmarkCollection_Query_NoContent_100000-8          30   39574238 ns\u002Fop   810370 B\u002Fop      232 allocs\u002Fop\nBenchmarkCollection_Query_100-8                    13225      91058 ns\u002Fop     5177 B\u002Fop       95 allocs\u002Fop\nBenchmarkCollection_Query_1000-8                    2226     519693 ns\u002Fop    13552 B\u002Fop      140 allocs\u002Fop\nBenchmarkCollection_Query_5000-8                     550    2128121 ns\u002Fop    47108 B\u002Fop      173 allocs\u002Fop\nBenchmarkCollection_Query_25000-8                    100   10063260 ns\u002Fop   211705 B\u002Fop      205 allocs\u002Fop\nBenchmarkCollection_Query_100000-8                    30   39404005 ns\u002Fop   810295 B\u002Fop      229 allocs\u002Fop\nPASS\nok   github.com\u002Fphilippgille\u002Fchromem-go 28.402s\n```\n\n## 开发\n\n- 构建：`go build .\u002F...`\n- 测试：`go test -v -race -count 1 .\u002F...`\n- 基准测试：\n  - `go test -benchmem -run=^$ -bench .`（可添加 `> bench.out` 等重定向到文件）\n  - 含性能分析：`go test -benchmem -run ^$ -cpuprofile cpu.out -bench .`\n    - （分析选项：`-cpuprofile`、`-memprofile`、`-blockprofile`、`-mutexprofile`）\n- 对比基准测试结果：\n  1. 安装 `benchstat`：`go install golang.org\u002Fx\u002Fperf\u002Fcmd\u002Fbenchstat@latest`\n  2. 对比两个基准测试结果：`benchstat before.out after.out`\n\n## 动机\n\n2023年12月，当我想要在Go程序中尝试检索增强生成（RAG）时，我寻找一种可以嵌入到Go程序中的向量数据库，就像你可以嵌入SQLite一样，这样就不需要单独的数据库设置和维护了。然而，令我惊讶的是，在Go生态系统中虽然有许多嵌入式键值存储，却几乎找不到这样的向量数据库。\n\n当时，许多流行的向量数据库，如Pinecone、Qdrant、Milvus、Chroma、Weaviate等，要么完全无法嵌入，要么仅支持Python或JavaScript\u002FTypeScript语言。\n\n后来，我发现了[@eliben](https:\u002F\u002Fgithub.com\u002Feliben)的[博客文章](https:\u002F\u002Feli.thegreenplace.net\u002F2023\u002Fretrieval-augmented-generation-in-go\u002F)和[示例代码](https:\u002F\u002Fgithub.com\u002Feliben\u002Fcode-for-blog\u002Ftree\u002Feda87b87dad9ed8bd45d1c8d6395efba3741ed39\u002F2023\u002Fgo-rag-openai)，它们展示了只需少量Go代码就能实现一个非常基础的向量数据库概念验证。\n\n正是在那时，我决定基于ChromaDB的接口，构建一个可嵌入Go的向量数据库。ChromaDB之所以脱颖而出，是因为它支持嵌入式使用（Python），并且在其README文件和官网首页上仅用4个命令就展示了其核心API。\n\n## 相关项目\n\n- 特别感谢[@eliben](https:\u002F\u002Fgithub.com\u002Feliben)，他的[博客文章](https:\u002F\u002Feli.thegreenplace.net\u002F2023\u002Fretrieval-augmented-generation-in-go\u002F)和[示例代码](https:\u002F\u002Fgithub.com\u002Feliben\u002Fcode-for-blog\u002Ftree\u002Feda87b87dad9ed8bd45d1c8d6395efba3741ed39\u002F2023\u002Fgo-rag-openai)，启发了我启动这个项目！\n- [Chroma](https:\u002F\u002Fgithub.com\u002Fchroma-core\u002Fchroma)：对比Pinecone、Qdrant、Milvus、Weaviate等，Chroma以其在README和官网首页上仅用4个命令展示核心API而独树一帜。此外，Chroma还特别强调其嵌入式能力（Python）。\n- 一些大型、功能完善的客户端-服务器架构向量数据库，旨在实现最大规模和最佳性能：\n  - [Pinecone](https:\u002F\u002Fwww.pinecone.io\u002F)：闭源\n  - [Qdrant](https:\u002F\u002Fgithub.com\u002Fqdrant\u002Fqdrant)：使用Rust编写，无法嵌入Go\n  - [Milvus](https:\u002F\u002Fgithub.com\u002Fmilvus-io\u002Fmilvus)：使用Go和C++编写，但截至2023年12月仍不支持嵌入式使用\n  - [Weaviate](https:\u002F\u002Fgithub.com\u002Fweaviate\u002Fweaviate)：使用Go编写，但截至2024年3月仍不支持Go嵌入（仅支持Python和JavaScript\u002FTypeScript，且处于实验阶段）\n- 一些非专门的SQL、NoSQL和键值存储数据库开始支持向量存储以及基于相似度的查询：\n  - [pgvector](https:\u002F\u002Fgithub.com\u002Fpgvector\u002Fpgvector)扩展，适用于[PostgreSQL](https:\u002F\u002Fwww.postgresql.org\u002F)：客户端-服务器模式\n  - [Redis](https:\u002F\u002Fgithub.com\u002Fredis\u002Fredis)（[1](https:\u002F\u002Fredis.io\u002Fdocs\u002Finteract\u002Fsearch-and-query\u002Fquery\u002Fvector-search\u002F)，[2](https:\u002F\u002Fredis.io\u002Fdocs\u002Finteract\u002Fsearch-and-query\u002Fadvanced-concepts\u002Fvectors\u002F)）：客户端-服务器模式\n  - [sqlite-vss](https:\u002F\u002Fgithub.com\u002Fasg017\u002Fsqlite-vss)扩展，适用于[SQLite](https:\u002F\u002Fwww.sqlite.org\u002F)：嵌入式，但其[Go绑定](https:\u002F\u002Fgithub.com\u002Fasg017\u002Fsqlite-vss\u002Ftree\u002F8fc44301843029a13a474d1f292378485e1fdd62\u002Fbindings\u002Fgo)依赖CGO。虽然存在一个无CGO的SQLite Go库[1](https:\u002F\u002Fgitlab.com\u002Fcznic\u002Fsqlite)，但该库并不包含向量搜索扩展。\n  - [DuckDB](https:\u002F\u002Fgithub.com\u002Fduckdb\u002Fduckdb)提供计算余弦相似度的函数（[1](https:\u002F\u002Fduckdb.org\u002Fdocs\u002Fsql\u002Ffunctions\u002Fnested)）：嵌入式，但其Go绑定使用CGO\n  - [MongoDB](https:\u002F\u002Fgithub.com\u002Fmongodb\u002Fmongo)的云平台提供向量搜索服务（[1](https:\u002F\u002Fwww.mongodb.com\u002Fproducts\u002Fplatform\u002Fatlas-vector-search)）：客户端-服务器模式\n- 一些用于向量相似度搜索的库：\n  - [Faiss](https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Ffaiss)：使用C++编写；第三方Go绑定依赖CGO\n  - [Annoy](https:\u002F\u002Fgithub.com\u002Fspotify\u002Fannoy)：使用C++编写；Go绑定依赖CGO（[1](https:\u002F\u002Fgithub.com\u002Fspotify\u002Fannoy\u002Fblob\u002F2be37c9e015544be2cf60c431f0cccc076151a2d\u002FREADME_GO.rst)）\n  - [USearch](https:\u002F\u002Fgithub.com\u002Funum-cloud\u002Fusearch)：使用C++编写；Go绑定依赖CGO\n- 一些编排库，受Python库[LangChain](https:\u002F\u002Fgithub.com\u002Flangchain-ai\u002Flangchain)启发，但内置的向量数据库功能要么缺失，要么非常简陋：\n  - [LangChain Go](https:\u002F\u002Fgithub.com\u002Ftmc\u002Flangchaingo)\n  - [LinGoose](https:\u002F\u002Fgithub.com\u002Fhenomis\u002Flingoose)\n  - [GoLC](https:\u002F\u002Fgithub.com\u002Fhupe1980\u002Fgolc)","# chromem-go 快速上手指南\n\n## 环境准备\n- 需要安装 Go 1.18+ 环境\n- 无第三方依赖（零依赖）\n- 使用 OpenAI 时需设置 `OPENAI_API_KEY` 环境变量\n\n## 安装步骤\n```bash\nexport GOPROXY=https:\u002F\u002Fgoproxy.cn,direct\ngo get -u github.com\u002Fphilippgille\u002Fchromem-go@latest\n```\n\n## 基本使用\n以下是最小化示例（使用 OpenAI 生成嵌入）：\n\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"runtime\"\n\n\t\"github.com\u002Fphilippgille\u002Fchromem-go\"\n)\n\nfunc main() {\n\tctx := context.Background()\n\tdb := chromem.NewDB()\n\n\tc, _ := db.CreateCollection(\"knowledge-base\", nil, nil)\n\n\t_ = c.AddDocuments(ctx, []chromem.Document{\n\t\t{ID: \"1\", Content: \"The sky is blue because of Rayleigh scattering.\"},\n\t\t{ID: \"2\", Content: \"Leaves are green because chlorophyll absorbs red and blue light.\"},\n\t}, runtime.NumCPU())\n\n\tres, _ := c.Query(ctx, \"Why is the sky blue?\", 1, nil, nil)\n\tfmt.Printf(\"ID: %v\\nSimilarity: %v\\nContent: %v\\n\", res[0].ID, res[0].Similarity, res[0].Content)\n}\n```\n\n运行后输出：\n```\nID: 1\nSimilarity: 0.6833369\nContent: The sky is blue because of Rayleigh scattering.\n```","某金融科技公司开发团队为内部知识库构建AI问答系统，帮助员工快速查询产品文档和历史会议记录，但早期方案依赖外部数据库导致效率低下。\n\n### 没有 chromem-go 时\n- 需要额外部署Chroma服务，运维团队每月花费20小时维护数据库实例，增加成本。\n- 查询依赖网络请求，平均延迟150ms，员工提问后需等待，影响工作效率。\n- 开发环境需手动配置数据库连接，新成员上手时间长达1天，阻碍快速迭代。\n- 文档量达5万时，外部数据库资源占用高，响应速度骤降，系统不稳定。\n\n### 使用 chromem-go 后\n- 直接嵌入Go应用，部署从小时级缩短至秒级，运维成本归零。\n- 内存中查询，5万文档响应仅25ms，员工提问即时获得答案。\n- 开发环境零配置，新成员10分钟内即可启动应用，加速功能迭代。\n- 资源占用优化，文档量5万时内存仅占用50MB，系统稳定高效。\n\nchromem-go让Go应用轻松实现轻量级向量检索，显著提升开发效率和用户体验。","https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fphilippgille_chromem-go_bab329aa.png","philippgille","Philipp Gillé","https:\u002F\u002Foss.gittoolsai.com\u002Favatars\u002Fphilippgille_17ec542a.jpg","Software engineer, tech enthusiast","@Heetch","Leipzig, Germany",null,"https:\u002F\u002Fgithub.com\u002Fphilippgille",[84],{"name":85,"color":86,"percentage":87},"Go","#00ADD8",100,907,65,"2026-04-04T01:24:45","MPL-2.0",1,"","未说明",{"notes":96,"python":94,"dependencies":97},"需要设置 OPENAI_API_KEY 环境变量（或其他嵌入模型提供商的密钥），项目处于 beta 阶段可能引入破坏性变更",[],[26,51,54,13],[100,101,102,103,104,105,106,107,108,109,110,111,112,113,114],"embeddings","go","golang","llm","llms","nearest-neighbor","rag","retrieval-augmented-generation","vector-database","vector-search","chroma","chromadb","embedded","in-memory","cosine-similarity","2026-03-27T02:49:30.150509","2026-04-06T07:13:55.822119",[118,123,128],{"id":119,"question_zh":120,"answer_zh":121,"source_url":122},5487,"运行最小示例时出现 401 Unauthorized 错误如何解决？","需要指定 embedding 函数，例如使用 Ollama。修改代码：将 `db.CreateCollection(\"knowledge-base\", nil, nil)` 替换为 `db.CreateCollection(\"knowledge-base\", nil, chromem.NewEmbeddingFuncOllama(\"nomic-embed-text\", \"\"))`。","https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fissues\u002F108",{"id":124,"question_zh":125,"answer_zh":126,"source_url":127},5488,"如何根据 ID 获取文档？","使用 `c.GetByID(ctx, id)` 方法。此功能已添加，可直接调用以获取指定 ID 的文档。","https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fissues\u002F95",{"id":129,"question_zh":130,"answer_zh":131,"source_url":132},5489,"如何使用 io.Writer 和 io.Reader 进行导出和导入？","已添加 `Export` 和 `Import` 的 io.Reader\u002Fio.Writer 版本，支持 S3 或 HTTP 等数据源。示例代码可参考 PR #71、#72 和 #73。","https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fissues\u002F70",[134,139,144,149,154,159,164],{"id":135,"version":136,"summary_zh":137,"released_at":138},104996,"v0.7.0","Highlights in this release are the possibility to export\u002Fimport the DB to\u002Ffrom object storage like S3, a way to run a *negative* query and either filter or subtract the results from the regular query results, and the license change from AGPL to MPL. But many other additions, improvements and important fixes made it into this release as well, all without breaking changes:\r\n\r\n## Added\r\n\r\n- Added `DB.ExportToWriter()` to allow users to pass any `io.Writer` implementation for the DB export, not just a file. This allows for example to export the DB to AWS S3 or compatible services (like Ceph, MinIO etc.). (PR [#71](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F71))\r\n- Added `DB.ImportFromReader()` to allow users to pass any `io.ReadSeeker` implementation for the DB import, not just a file. This allows for example to import the DB from AWS S3 or compatible services (like Ceph, MinIO etc.). (PR [#72](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F72))\r\n- Added example code for S3 export\u002Fimport with the ⬆️ new methods (PR [#73](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F73))\r\n- Added Azure OpenAI compatibility (PR [#74](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F74) by [@iwilltry42](https:\u002F\u002Fgithub.com\u002Fiwilltry42))\r\n- Added lint job in GitHub Action (PR [#82](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F82) by [@erikdubbelboer](https:\u002F\u002Fgithub.com\u002F@erikdubbelboer))\r\n- Added the feature to run a *negative* query and either filter or subtract them from the regular query results (PR [#80](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F80) by [@erikdubbelboer](https:\u002F\u002Fgithub.com\u002F@erikdubbelboer))\r\n  - This PR also added the new `DB.QueryWithOptions` method and related options structs and constants for future extensibility without breaking the parameter list of the query method!\r\n- Added option to only import\u002Fexport selected collections to\u002Ffrom a DB (PR [#88](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F88) by [@iwilltry42](https:\u002F\u002Fgithub.com\u002Fiwilltry42))\r\n- Added Google \u002F GCP Vertex AI embedding function (PR [#91](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F91) by [@iwilltry42](https:\u002F\u002Fgithub.com\u002Fiwilltry42) and [#93](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F93))\r\n- Added new embedding model constants for Jina and Mixedbread (PR [#94](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F94))\r\n- Added `Collection.GetByID()` to get a document for a known ID (PR [#97](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F97), for issue [#95](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fissues\u002F95))\r\n\r\n## Improved\r\n\r\n- Changed license from AGPL to MPL (PR [#87](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F87))\r\n- Updated `golangci-lint` in CI to its latest version (PR [#99](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F99))\r\n- Added Go 1.23 to the CI build matrix (PR [#98](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F98))\r\n- Improved Godoc (PR [#100](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F100))\r\n\r\n## Fixed\r\n\r\n- The `Collection.QueryEmbedding()` call assumed\u002Fexpected the query embedding from the parameter to be normalized already, but it wasn't documented and it's also inconvenient for users who use an embedding model\u002FAPI that doesn't return normalized embeddings. Now we check whether the embedding is normalized and if it's not then we normalize it. (PR [#77](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F77))\r\n  - (Currently `chromem-go` only does cosine similarity, and document embeddings are already being normalized, so the query embedding has to be normalized as well. In the future we might offer other distance functions or allow to inject your own and make the normalization optional)\r\n- Fixed test panic on unexpected pass (PR [#78](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F78))\r\n- Fixed out of range panic on query (PR [#79](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F79))\r\n- Fixed all `golangci-lint` warnings (PR [#82](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F82) by [@erikdubbelboer](https:\u002F\u002Fgithub.com\u002F@erikdubbelboer))\r\n- Fixed grammar and inconsistent receiver names (PR [#85](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F85) by [@codefromthecrypt](https:\u002F\u002Fgithub.com\u002Fcodefromthecrypt))\r\n- Fixed link in Godoc (PR [#89](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F89))\r\n\r\n## New Contributors\r\n\r\n* @codefromthecrypt made their first contribution in https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F83\r\n\r\n**Full Changelog**: https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fcompare\u002Fv0.6.0...v0.7.0","2024-09-01T17:30:26",{"id":140,"version":141,"summary_zh":142,"released_at":143},104997,"v0.6.0","\r\nHighlights in this release are an extended interface, experimental WebAssembly bindings, and the option to use a custom Ollama URL. But also the fact that three people contributed to this release! Thank you so much! 🙇‍♂️\r\n\r\n## Added\r\n\r\n- Added `Collection.Delete()` to delete documents from a collection (PR [#63](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F63) by [@iwilltry42](https:\u002F\u002Fgithub.com\u002Fiwilltry42))\r\n- Added an experimental WebAssembly binding (package `wasm`) and example (PR [#69](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F69))\r\n\r\n## Improved\r\n\r\n- Use prefixes for `nomic-embed-text` model in RAG-Wikipedia-Ollama example (PR [#49](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F49), [#65](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F65))\r\n  - Thanks [@rinor](https:\u002F\u002Fgithub.com\u002Frinor) for pointing out the bug!\r\n- Made Ollama URL configurable (PR [#64](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F64) by [@erikdubbelboer](https:\u002F\u002Fgithub.com\u002F@erikdubbelboer))\r\n- Added building of code examples to CI (PR [#66](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F66))\r\n- Improved RAG template (PR [#67](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F67))\r\n\r\n## Breaking changes\r\n\r\n- `NewEmbeddingFuncOllama` now requires a second parameter for the base URL. But it can be empty to use the default which was also used in the past.\r\n\r\n## New Contributors\r\n\r\n- [@rinor](https:\u002F\u002Fgithub.com\u002Frinor) made their first contribution by suggesting a fix in \u003Chttps:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F49>\r\n- [@iwilltry42](https:\u002F\u002Fgithub.com\u002Fiwilltry42) made their first contribution in \u003Chttps:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F63>\r\n- [@erikdubbelboer](https:\u002F\u002Fgithub.com\u002F@erikdubbelboer) made their first contribution in \u003Chttps:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F64>\r\n\r\n**Full Changelog**: \u003Chttps:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fcompare\u002Fv0.5.0...v0.6.0>\r\n","2024-04-25T21:11:15",{"id":145,"version":146,"summary_zh":147,"released_at":148},104998,"v0.5.0","\r\nHighlights in this release are query performance improvements (5x faster, 98% fewer memory allocations), export\u002Fimport of the entire DB to\u002Ffrom a single file with optional gzip-compression and AES-GCM encryption, optional gzip-compression for the regular persistence, a new code example for semantic search across 5,000 arXiv papers, and an embedding func for [Cohere](https:\u002F\u002Fcohere.com\u002Fmodels\u002Fembed).\r\n\r\n### Added\r\n\r\n- Added arXiv semantic search example (PR [#45](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F45))\r\n- Added basic query benchmark (PR [#46](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F46))\r\n- Added unit test for collection query errors (PR [#51](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F51))\r\n- Added `Collection.QueryEmbedding()` method for when you already have the embedding of your query (PR [#52](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F52))\r\n- Added export and import of the entire DB to\u002Ffrom a single file, with optional gzip-compression and AES-GCM encryption (PR [#58](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F58))\r\n- Added optional gzip-compression to the regular persistence (i.e. the DB from `NewPersistentDB()` which writes a file for each added collection and document) (PR [#59](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F59))\r\n- Added minimal example (PR [#60](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F60), [#62](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F62))\r\n- Added embedding func for [Cohere](https:\u002F\u002Fcohere.com\u002Fmodels\u002Fembed) (PR [#61](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F61))\r\n\r\n### Improved\r\n\r\n- Changed the example link target to directory instead of `main.go` file (PR [#43](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F43))\r\n- Improved query performance (5x faster, 98% fewer memory allocations) (PR [#47](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F47), [#53](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F53), [#54](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F54))\r\n  - \u003Cdetails>\u003Csummary>benchstat output\u003C\u002Fsummary>\r\n\r\n    ```text\r\n    goos: linux\r\n    goarch: amd64\r\n    pkg: github.com\u002Fphilippgille\u002Fchromem-go\r\n    cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz\r\n                                        │    before     │               after                 │\r\n                                        │    sec\u002Fop     │    sec\u002Fop     vs base               │\r\n    Collection_Query_NoContent_100-8      413.69µ ±  4%   90.79µ ±  2%  -78.05% (p=0.002 n=6)\r\n    Collection_Query_NoContent_1000-8     2759.4µ ±  0%   518.8µ ±  1%  -81.20% (p=0.002 n=6)\r\n    Collection_Query_NoContent_5000-8     12.980m ±  1%   2.144m ±  1%  -83.49% (p=0.002 n=6)\r\n    Collection_Query_NoContent_25000-8    66.559m ±  1%   9.947m ±  2%  -85.06% (p=0.002 n=6)\r\n    Collection_Query_NoContent_100000-8   282.41m ±  3%   39.75m ±  1%  -85.92% (p=0.002 n=6)\r\n    Collection_Query_100-8                416.75µ ±  2%   90.99µ ±  1%  -78.17% (p=0.002 n=6)\r\n    Collection_Query_1000-8               2792.8µ ± 23%   595.2µ ± 13%  -78.69% (p=0.002 n=6)\r\n    Collection_Query_5000-8               15.643m ±  1%   2.556m ±  1%  -83.66% (p=0.002 n=6)\r\n    Collection_Query_25000-8               78.29m ±  1%   11.66m ±  1%  -85.11% (p=0.002 n=6)\r\n    Collection_Query_100000-8             338.54m ±  5%   39.70m ± 12%  -88.27% (p=0.002 n=6)\r\n    geomean                                12.97m         2.192m        -83.10%\r\n\r\n                                        │      before      │               after                 │\r\n                                        │       B\u002Fop       │     B\u002Fop      vs base               │\r\n    Collection_Query_NoContent_100-8       1211.007Ki ± 0%   5.030Ki ± 0%  -99.58% (p=0.002 n=6)\r\n    Collection_Query_NoContent_1000-8      12082.16Ki ± 0%   13.24Ki ± 0%  -99.89% (p=0.002 n=6)\r\n    Collection_Query_NoContent_5000-8      60394.23Ki ± 0%   45.99Ki ± 0%  -99.92% (p=0.002 n=6)\r\n    Collection_Query_NoContent_25000-8     301962.1Ki ± 0%   206.7Ki ± 0%  -99.93% (p=0.002 n=6)\r\n    Collection_Query_NoContent_100000-8   1207818.1Ki ± 0%   791.4Ki ± 0%  -99.93% (p=0.002 n=6)\r\n    Collection_Query_100-8                 1211.006Ki ± 0%   5.033Ki ± 0%  -99.58% (p=0.002 n=6)\r\n    Collection_Query_1000-8                12082.11Ki ± 0%   13.25Ki ± 0%  -99.89% (p=0.002 n=6)\r\n    Collection_Query_5000-8                60394.10Ki ± 0%   46.04Ki ± 0%  -99.92% (p=0.002 n=6)\r\n    Collection_Query_25000-8               301962.1Ki ± 0%   206.8Ki ± 0%  -99.93% (p=0.002 n=6)\r\n    Collection_Query_100000-8             1207818.1Ki ± 0%   791.4Ki ± 0%  -99.93% (p=0.002 n=6)\r\n    geomean                                   49.13Mi        54.97Ki       -99.89%\r\n\r\n                                        │    before     │              after                │\r\n                                        │   allocs\u002Fop   │ allocs\u002Fop   vs base               │\r\n    Collection_Query_NoContent_100-8        238.00 ± 0%   94.00 ± 1%  -60.50% (p=0.002 n=6)\r\n    Collection_Query_NoContent_1000-","2024-03-23T11:52:40",{"id":150,"version":151,"summary_zh":152,"released_at":153},104999,"v0.4.0","Highlights in this release are optional persistence, an extended interface, support for creating embeddings with [Ollama](https:\u002F\u002Fgithub.com\u002Follama\u002Follama\u002F), the exporting of the `Document` struct, and more Go-idiomatic methods to add documents to collections.\r\n\r\n### Added\r\n\r\n- Extended the interface:\r\n  - `DB.ListCollections()` (PR [#12](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F12))\r\n  - `DB.GetCollection()` (PR [#13](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F13) + [#19](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F19))\r\n  - `DB.DeleteCollection()` (PR [#14](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F14))\r\n  - `DB.Reset()` (PR [#15](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F15))\r\n  - `DB.GetOrCreateCollection()` (PR [#22](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F22))\r\n  - `Collection.Count()` (PR [#27](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F27))\r\n  - `Document` struct, `NewDocument()` function, `Collection.AddDocument()` and `Collection.AddDocuments()` methods (PR [#34](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F34))\r\n    - More Go-idiomatic alternatives to `Collection.Add()`\r\n- Added various unit tests (PR [#20](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F20), [#39](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F39))\r\n- Added optional persistence! Via multiple PRs:\r\n  - Write in PR [#23](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F23), [#24](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F24), [#31](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F31)\r\n  - Read in PR [#25](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F25)\r\n  - Delete in PR [#26](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F26)\r\n- Added support for creating embeddings with [Ollama](https:\u002F\u002Fgithub.com\u002Follama\u002Follama\u002F) (PR [#32](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F32))\r\n- Added example documentation (PR [#42](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F42))\r\n\r\n### Improved\r\n\r\n- Improved example (PR [#11](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F11), [#28](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F28), [#33](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F33))\r\n- Stop exporting `Collection.Metadata` (PR [#16](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F16))\r\n  - Goal: Prevent direct modifications which could cause data races in case of the user doing a modification while `chromem-go` for example ranges over it during a `Collection.Query()` call.\r\n- Copy metadata in constructors (PR [#17](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F17))\r\n  - Goal: Prevent direct modifications which could cause data races in case of the user doing a modification while `chromem-go` for example ranges over it.\r\n- Improved CI (PR [#18](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F18))\r\n  - Add Go 1.22 to test matrix, update used GitHub Action from v4 to v5, use race detector during tests\r\n- Reorganize code internally (PR [#21](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F21))\r\n- Switched to newer recommended check for file related `ErrNotExist` errors (PR [#29](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F29))\r\n- Added more validations in several existing methods (PR [#30](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F30))\r\n- Internal variable renamed (PR [#37](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F37))\r\n- Fail unit tests immediately (PR [#40](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F40))\r\n\r\n### Fixed\r\n\r\n- Fixed metadatas validation in `Collection.AddConcurrently()` (PR [#35](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F35))\r\n- Fixed Godoc of `Collection.Query()` method (PR [#36](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F36))\r\n- Fixed length of result slice (PR [#38](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F38))\r\n- Fixed filter test (PR [#41](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F41))\r\n\r\n### Breaking changes\r\n\r\n- Because functions can't be (de-)serialized, `GetCollection` requires a new parameter of type `EmbeddingFunc`, in order to set the correct func when using a DB with persistence and it just loaded the collections and documents from storage. (PR [#25](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F25))\r\n- Some methods now return an error (due to file operations when persistence is used)\r\n- Additional validations will return an early error, but most (if not all) prior calls with the invalid parameters probably lead to some errors down the line anyway\r\n- `Collection.Metadata` is not exported anymore\r\n- `Result.Document` field was renamed to `Result.Content`, to avoid confusion with the now exported `Document` struct\r\n","2024-03-06T22:46:41",{"id":155,"version":156,"summary_zh":157,"released_at":158},105000,"v0.3.0","\r\n## Added\r\n\r\n- Added support for more OpenAI embedding models (PR [#6](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F6))\r\n- Added support for more embedding creators\u002Fproviders: (PR [#10](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F10))\r\n  - [Mistral](https:\u002F\u002Fdocs.mistral.ai\u002Fplatform\u002Fendpoints\u002F#embedding-models), [Jina](https:\u002F\u002Fjina.ai\u002Fembeddings), [mixedbread.ai](https:\u002F\u002Fwww.mixedbread.ai\u002F), [LocalAI](https:\u002F\u002Fgithub.com\u002Fmudler\u002FLocalAI)\r\n\r\n## Improved\r\n\r\n- Improve concurrency when adding documents to collection (PR [#2](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F2))\r\n- Rename `Client` to `DB` to better indicate that the database is embedded and there's no client-server separation (PR [#3](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F3))\r\n- Change OpenAPI embedding model from \"text-embedding-ada-002\" to \"text-embedding-3-small\" (PR [#4](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F4))\r\n- Allow custom base URL for OpenAI, enabling the use of [Azure OpenAI](https:\u002F\u002Fazure.microsoft.com\u002Fen-us\u002Fproducts\u002Fai-services\u002Fopenai-service), [LiteLLM](https:\u002F\u002Fgithub.com\u002FBerriAI\u002Flitellm), [ollama](https:\u002F\u002Fgithub.com\u002Follama\u002Follama\u002Fblob\u002Fmain\u002Fdocs\u002Fopenai.md) etc. (PR [#7](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F7))\r\n- Renamed `EmbeddingFunc` constructors to follow best practice (PR [#9](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F9))\r\n\r\n## Fixed\r\n\r\n- Don't allow `nResults` arg \u003C 0 (PR [#5](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F5))\r\n\r\n## Breaking changes\r\n\r\n- Several function names and signatures were changed in this release. This can happen as long as the version is at `v0.x.y`.\r\n","2024-02-10T18:19:12",{"id":160,"version":161,"summary_zh":162,"released_at":163},105001,"v0.2.0","\r\n## Added\r\n\r\n- Added GitHub Actions config ([commit](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fcommits\u002Ffae84f2069ec28bbf9f4e30dca569f447d6aee6a))\r\n- Added `CHANGELOG.md` ([commit](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fcommits\u002Fbb0aa24b95ed19a743b2b5aa60098077bebdea41))\r\n- Exported embedding creation functions ([commit](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fcommits\u002F9d8ce4ae88c08bc975a0ed6b180bc01dcb2a390f))\r\n- Added `Collection.AddConcurrently` to add embeddings concurrently ([commit](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fcommits\u002F50fe3b743696d0209d2e4c617633ba335870ab7d))\r\n\r\n## Improved\r\n\r\n- Improved example code ([commit](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fcommits\u002Fc6437611d2fd48c5458b1932d5df62f90501981f))\r\n- Removed unused field in `Client` ([commit](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fcommits\u002F9c8b01ad386008d09675a26b7eca9c9605af5b1c))\r\n- Improved validation in `Query` method ([commit](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fcommits\u002F0bd196ee7c36164fad123c7b21766c7444de246d))\r\n- Added and improved Godoc ([commit](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fcommits\u002Fc3a4db9563efb270af5aee585a7fca54b2ab08dc))\r\n- Improved locking around a collection's documents ([commit](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fcommits\u002Fcefec66912d2fc96928154a9159ca05bd52c5149))\r\n- Removed dependency on third party library for OpenAI ([commit](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fcommits\u002F1a28e1b89808cb223d67808a8e90bb9c36d2d801))\r\n- Parallelized document querying (PR [#1](https:\u002F\u002Fgithub.com\u002Fphilippgille\u002Fchromem-go\u002Fpull\u002F1))\r\n","2024-01-01T18:20:56",{"id":165,"version":166,"summary_zh":167,"released_at":168},105002,"v0.1.0","Initial release with a minimal Chroma-like interface and a working retrieval augmented generation (RAG) example.","2023-12-29T00:01:06"]