Index and retrieve documents for vector search using Sentence Transformers and DuckDB - Agentic RAG (1/5)
β’
9
David Berenstein
davidberenstein1957
AI & ML interests
Everything data
Recent Activity
upvoted
an
article
about 3 hours ago
Open-source DeepResearch β Freeing our search agents
upvoted
an
article
about 3 hours ago
DABStep: Data Agent Benchmark for Multi-step Reasoning
commented on
an
article
about 3 hours ago
Ο0 and Ο0-FAST: Vision-Language-Action Models for General Robot Control
Articles
Organizations
Posts
46
Post
291
Creating an agentic RAG stack on the Hugging Face Hub - part 1 - retrieval (1/5).
π Web apps and microservices included!
Chunk, embed and index documents at a huge scale without overhead.
Blog: https://huggingface.co./blog/davidberenstein1957/ai-blueprint-agentic-rag-part-1-retrieve
π Web apps and microservices included!
Chunk, embed and index documents at a huge scale without overhead.
Blog: https://huggingface.co./blog/davidberenstein1957/ai-blueprint-agentic-rag-part-1-retrieve
Post
1514
tldr; Parquet is awesome, DuckDB too!
Datasets on the Hugging Face Hub rely on parquet files. We can interact with these files using DuckDB as a fast in-memory database system. One of DuckDBβs features is vector similarity search which can be used with or without an index.
blog:
https://huggingface.co./learn/cookbook/vector_search_with_hub_as_backend
Datasets on the Hugging Face Hub rely on parquet files. We can interact with these files using DuckDB as a fast in-memory database system. One of DuckDBβs features is vector similarity search which can be used with or without an index.
blog:
https://huggingface.co./learn/cookbook/vector_search_with_hub_as_backend
Collections
10
A collection of smoll but mighty models
-
382
Synthetic Data Generator
π§¬Build datasets using natural language
-
7
Transformers Pipeline Playground
πSearch, load and play with transformer pipelines
-
13
Human Feedback Collector | Meta-Llama-3.1-8B-Instruct | (DPO)
π¦ΎLLM, chatbot, human-feedback
-
10
Distilabel Synthetic Data Pipeline Finder
βFind and view synthetic data pipelines on Hugging Face
spaces
17
pinned
Running
62
Vectorsearch Hub Datasets
π₯
Add vectors to Hub datasets and do in memory vector search.
pinned
Running
Vectorsearch Hub Datasets
π₯
Add vectors to Hub datasets and do in memory vector search.
pinned
Sleeping
7
Transformers Pipeline Playground
π
Search, load and play with transformer pipelines
pinned
Running
22
Text To SQL Hub Datasets
π₯
Generate SQL queries from natural language for Hugging Face datasets
pinned
Sleeping
5
Human Feedback Collector | Meta-Llama-3.1-8B-Instruct | (KTO)
π¦Ύ
LLM, chatbot, human-feedback
pinned
Sleeping
13
Human Feedback Collector | Meta-Llama-3.1-8B-Instruct | (DPO)
π¦Ύ
LLM, chatbot, human-feedback
models
3
datasets
45
davidberenstein1957/short-think-making-reasoning-less-chatty
Viewer
β’
Updated
β’
40
β’
56
β’
3
davidberenstein1957/structured-generation-information-extraction-vlms-openbmb-RLAIF-V-Dataset
Viewer
β’
Updated
β’
10
β’
32
davidberenstein1957/my-vicinity-repo
Viewer
β’
Updated
β’
5
β’
42
β’
1
davidberenstein1957/openai_records_0742e002
Updated
β’
10
davidberenstein1957/hf_client_records_778db757
Viewer
β’
Updated
β’
1
β’
25
davidberenstein1957/hf_client_records_104d3f34
Viewer
β’
Updated
β’
1
β’
20
davidberenstein1957/example
Updated
β’
12
davidberenstein1957/follow-the-money-imdb
Viewer
β’
Updated
β’
100
β’
44
davidberenstein1957/sft-dataset
Viewer
β’
Updated
β’
10
β’
63
β’
3
davidberenstein1957/openai_records_a1b9734b
Viewer
β’
Updated
β’
1
β’
35