Building an AI-powered search engine from scratch
โข
8
Astra Clelia Bertelli PRO
as-cle-bert
AI & ML interests
Biology + Artificial Intelligence = โค๏ธ | AI for sustainable development, sustainable development for AI | Researching on Machine Learning Enhancement | I love automation for everyday things | Blogger | Open Source
Recent Activity
updated
a Space
about 22 hours ago
as-cle-bert/pokemon-bot
posted
an
update
about 22 hours ago
Hi HuggingFacers!๐คถ๐ผ
As my last 2024 project, I've dropped a Discord Bot that knows a lot about Pokemons๐ฆ
GitHub ๐ https://github.com/AstraBert/Pokemon-Bot
Demo Space ๐ https://huggingface.co./spaces/as-cle-bert/pokemon-bot
The bot integrates:
- Chat features (Cohere's Command-R) with RAG functionalities (hybrid search and reranking with Qdrant) and chat memory (managed through PostgreSQL) to produce information about Pokemons
- Image-based search to identify Pokemons from their images (via Qdrant)
- Card package random extraction and description
HuggingFace๐ค, as usual, plays the most important role in the application stack, with the following models:
- https://huggingface.co./sentence-transformers/LaBSE
- https://huggingface.co./prithivida/Splade_PP_en_v1
- https://huggingface.co./facebook/dinov2-large
And datasets:
- https://huggingface.co./datasets/Karbo31881/Pokemon_images
- https://huggingface.co./datasets/wanghaofan/pokemon-wiki-captions
- https://huggingface.co./datasets/TheFusion21/PokemonCards
Have fun!๐
liked
a Space
about 24 hours ago
as-cle-bert/pokemon-bot
Articles
Organizations
as-cle-bert's activity
posted
an
update
about 22 hours ago
Post
523
Hi HuggingFacers!๐คถ๐ผ
As my last 2024 project, I've dropped a Discord Bot that knows a lot about Pokemons๐ฆ
GitHub ๐ https://github.com/AstraBert/Pokemon-Bot
Demo Space ๐ as-cle-bert/pokemon-bot
The bot integrates:
- Chat features (Cohere's Command-R) with RAG functionalities (hybrid search and reranking with Qdrant) and chat memory (managed through PostgreSQL) to produce information about Pokemons
- Image-based search to identify Pokemons from their images (via Qdrant)
- Card package random extraction and description
HuggingFace๐ค, as usual, plays the most important role in the application stack, with the following models:
- sentence-transformers/LaBSE
- prithivida/Splade_PP_en_v1
- facebook/dinov2-large
And datasets:
- Karbo31881/Pokemon_images
- wanghaofan/pokemon-wiki-captions
- TheFusion21/PokemonCards
Have fun!๐
As my last 2024 project, I've dropped a Discord Bot that knows a lot about Pokemons๐ฆ
GitHub ๐ https://github.com/AstraBert/Pokemon-Bot
Demo Space ๐ as-cle-bert/pokemon-bot
The bot integrates:
- Chat features (Cohere's Command-R) with RAG functionalities (hybrid search and reranking with Qdrant) and chat memory (managed through PostgreSQL) to produce information about Pokemons
- Image-based search to identify Pokemons from their images (via Qdrant)
- Card package random extraction and description
HuggingFace๐ค, as usual, plays the most important role in the application stack, with the following models:
- sentence-transformers/LaBSE
- prithivida/Splade_PP_en_v1
- facebook/dinov2-large
And datasets:
- Karbo31881/Pokemon_images
- wanghaofan/pokemon-wiki-captions
- TheFusion21/PokemonCards
Have fun!๐
posted
an
update
14 days ago
Post
585
Hi HF Community!
I just published a blog article on building PrAIvateSearch (https://github.com/AstraBert/PrAIvateSearch), a user-owend, local and open-source AI-powered search engine๐:
https://huggingface.co./blog/as-cle-bert/build-an-ai-powered-search-engine-from-scratch
"Own your AI, search the web with it๐๐"
Feel free to try it out and contribute to it on GitHub: let's make OSS AI grown and thrive!๐
I just published a blog article on building PrAIvateSearch (https://github.com/AstraBert/PrAIvateSearch), a user-owend, local and open-source AI-powered search engine๐:
https://huggingface.co./blog/as-cle-bert/build-an-ai-powered-search-engine-from-scratch
"Own your AI, search the web with it๐๐"
Feel free to try it out and contribute to it on GitHub: let's make OSS AI grown and thrive!๐
posted
an
update
23 days ago
Post
1410
Hi HuggingFacers!๐ค
December is here and time has come, for most of us, to wrap up our code projects and take stock of our 2024 contributions๐๏ธ
In order to do this, I made a small Gradio application,
as-cle-bert/what-a-git-year
that scrapes information from your GitHub profile and summarizes them, producing also nice plots๐
Find also the GitHub repo here: https://github.com/AstraBert/what-a-git-year โญ
Hope that everyone had a Git year!๐
December is here and time has come, for most of us, to wrap up our code projects and take stock of our 2024 contributions๐๏ธ
In order to do this, I made a small Gradio application,
what-a-git-year:as-cle-bert/what-a-git-year
that scrapes information from your GitHub profile and summarizes them, producing also nice plots๐
Find also the GitHub repo here: https://github.com/AstraBert/what-a-git-year โญ
Hope that everyone had a Git year!๐
posted
an
update
29 days ago
Post
1035
Hi there!๐ค
I just deployed a Streamlit-based space on HF that fetches your Home Feed on BlueSky and summarizes it with Cohere's CommandR via Langchain๐งช
Find it here:
as-cle-bert/bsky-feedllama-demo
I'm also working on a Gradio local implementation with Llama3.2 that for now only works with source code and doesn't have docs, but that will be soon supported by Docker๐ณ and have a nice README:
https://github.com/AstraBert/bluesky-feedllama
Contributions and feedback are always welcome!๐ค๐ฆ
I just deployed a Streamlit-based space on HF that fetches your Home Feed on BlueSky and summarizes it with Cohere's CommandR via Langchain๐งช
Find it here:
as-cle-bert/bsky-feedllama-demo
I'm also working on a Gradio local implementation with Llama3.2 that for now only works with source code and doesn't have docs, but that will be soon supported by Docker๐ณ and have a nice README:
https://github.com/AstraBert/bluesky-feedllama
Contributions and feedback are always welcome!๐ค๐ฆ
posted
an
update
about 1 month ago
Post
1257
Hi HuggingFacers!๐ค
I'm thrilled to introduce my latest project: ๐ฆ๐ฒ๐ป๐ง๐ฟ๐๐ (๐ฆ๐ฒ๐ปtence ๐ง๐ฟansformers ๐๐aluator), a python package that offers simple customizable evaluation for text retrieval accuracy and time performance of Sentence Transformers-compatible text embedders on PDF data!๐
Learn more in my LinkedIn post: https://www.linkedin.com/posts/astra-clelia-bertelli-583904297_python-embedders-semanticsearch-activity-7266754133557190656-j1e3
And on the GitHub repo: https://github.com/AstraBert/SenTrEv
Have fun!๐
I'm thrilled to introduce my latest project: ๐ฆ๐ฒ๐ป๐ง๐ฟ๐๐ (๐ฆ๐ฒ๐ปtence ๐ง๐ฟansformers ๐๐aluator), a python package that offers simple customizable evaluation for text retrieval accuracy and time performance of Sentence Transformers-compatible text embedders on PDF data!๐
Learn more in my LinkedIn post: https://www.linkedin.com/posts/astra-clelia-bertelli-583904297_python-embedders-semanticsearch-activity-7266754133557190656-j1e3
And on the GitHub repo: https://github.com/AstraBert/SenTrEv
Have fun!๐
posted
an
update
about 2 months ago
Post
1664
Hi HugginfgFacers!๐ค
If you're into biomedical sciences, you will know the pain that, sometimes, searching PubMed can be๐โโ๏ธ
For these purposes, I built a bot that scrapes PubMed for you, starting from the exact title of a publication or key word search - all beautifully rendered through Gradioโ
Find it here: as-cle-bert/BioMedicalPapersBot
And here's the GitHub repository๐ฑ: https://github.com/AstraBert/BioMedicalPapersBot
It's also available as a Docker image!๐ณ
Best of luck with your research!
PS: in the very near future some AI summarization features will be included!
If you're into biomedical sciences, you will know the pain that, sometimes, searching PubMed can be๐โโ๏ธ
For these purposes, I built a bot that scrapes PubMed for you, starting from the exact title of a publication or key word search - all beautifully rendered through Gradioโ
Find it here: as-cle-bert/BioMedicalPapersBot
And here's the GitHub repository๐ฑ: https://github.com/AstraBert/BioMedicalPapersBot
It's also available as a Docker image!๐ณ
docker pull ghcr.io/astrabert/biomedicalpapersbot:mainBest of luck with your research!
PS: in the very near future some AI summarization features will be included!
posted
an
update
about 2 months ago
Post
761
Hi there HuggingFacers!๐ค
Are you working with Streamlit on Spaces and struggling with authentication and user management?๐ง
Well, you can check out my last community article (https://huggingface.co./blog/as-cle-bert/streamlit-supabase-auth-ui) on a new python package I've been working on, that connects Supabase to Streamlit UI, in order to create a seamless authentication for your seamless Streamlit apps!๐
You can find a demo of it on Spaces: as-cle-bert/streamlit-supabase-auth-ui
Have fun!๐
Are you working with Streamlit on Spaces and struggling with authentication and user management?๐ง
Well, you can check out my last community article (https://huggingface.co./blog/as-cle-bert/streamlit-supabase-auth-ui) on a new python package I've been working on, that connects Supabase to Streamlit UI, in order to create a seamless authentication for your seamless Streamlit apps!๐
You can find a demo of it on Spaces: as-cle-bert/streamlit-supabase-auth-ui
Have fun!๐
posted
an
update
2 months ago
Post
3214
Hi HuggingFacers!๐ค
As you may have probably heard, in the past weeks three Tech Giants (Microsoft, Amazon and Google) announced that they would bet on nuclear reactors to feed the surging energy demand of data centers, driven by increasing AI data and computational flows.
I try to explain the state of AI energy consumptions, its environmental impact and the key points of "turning AI nuclear" in my last article on HF community blog: https://huggingface.co./blog/as-cle-bert/ai-is-turning-nuclear-a-review
Enjoy the reading!๐ฑ
As you may have probably heard, in the past weeks three Tech Giants (Microsoft, Amazon and Google) announced that they would bet on nuclear reactors to feed the surging energy demand of data centers, driven by increasing AI data and computational flows.
I try to explain the state of AI energy consumptions, its environmental impact and the key points of "turning AI nuclear" in my last article on HF community blog: https://huggingface.co./blog/as-cle-bert/ai-is-turning-nuclear-a-review
Enjoy the reading!๐ฑ
posted
an
update
2 months ago
Post
1353
Hi there HuggingFacers!
Have you ever dreamt of an improbable books crossover, like Frodo from ๐๐ฐ๐ณ๐ฅ ๐ฐ๐ง ๐ต๐ฉ๐ฆ ๐๐ช๐ฏ๐จ๐ด becoming the main character of the ๐๐ฅ๐บ๐ด๐ด๐ฆ๐บ or Emma Bovary from ๐๐ข๐ฅ๐ข๐ฎ๐ฆ ๐๐ฐ๐ท๐ข๐ณ๐บ acting as a modern-days Shakespearean Juliet?
Well, all of this is now possible! I'm thrilled to introduce my latest opensource product for storytelling: ๐๐จ๐จ๐ค๐ฌ-๐ฆ๐ข๐ฑ๐๐ซ-๐๐ข ๐ฏ๐.๐.๐ !
Built with ReactJS and shipped directly to you on Spaces thanks to Docker, this webapp combines the power of two AI tools:
- gpt-4o-mini by OpenAI, which takes care of cooking new and intriguing plots starting from the user's instructions, the titles and the summaries of the two books to mix (summaries are scraped through Wikipedia)
- text2img realtime API by ModelsLab, which provides a stable diffusion pipeline to create a thumbnail for your newly-generated story
Everything is provided under a simple and intuitive UI, which uses chatscope's React template kit.
Curious of trying? The app is already live at:
as-cle-bert/books-mixer-ai
And you can also have a tour of the GitHub repo (and leave a little โญ while you're there):
https://github.com/AstraBert/books-mixer-ai
The documentation is still under construction, but will become available soon๐
Have fun!๐๐
Have you ever dreamt of an improbable books crossover, like Frodo from ๐๐ฐ๐ณ๐ฅ ๐ฐ๐ง ๐ต๐ฉ๐ฆ ๐๐ช๐ฏ๐จ๐ด becoming the main character of the ๐๐ฅ๐บ๐ด๐ด๐ฆ๐บ or Emma Bovary from ๐๐ข๐ฅ๐ข๐ฎ๐ฆ ๐๐ฐ๐ท๐ข๐ณ๐บ acting as a modern-days Shakespearean Juliet?
Well, all of this is now possible! I'm thrilled to introduce my latest opensource product for storytelling: ๐๐จ๐จ๐ค๐ฌ-๐ฆ๐ข๐ฑ๐๐ซ-๐๐ข ๐ฏ๐.๐.๐ !
Built with ReactJS and shipped directly to you on Spaces thanks to Docker, this webapp combines the power of two AI tools:
- gpt-4o-mini by OpenAI, which takes care of cooking new and intriguing plots starting from the user's instructions, the titles and the summaries of the two books to mix (summaries are scraped through Wikipedia)
- text2img realtime API by ModelsLab, which provides a stable diffusion pipeline to create a thumbnail for your newly-generated story
Everything is provided under a simple and intuitive UI, which uses chatscope's React template kit.
Curious of trying? The app is already live at:
as-cle-bert/books-mixer-ai
And you can also have a tour of the GitHub repo (and leave a little โญ while you're there):
https://github.com/AstraBert/books-mixer-ai
The documentation is still under construction, but will become available soon๐
Have fun!๐๐
posted
an
update
5 months ago
Post
5059
Hi HF Community!๐ค
In the past days, OpenAI announced their search engine, SearchGPT: today, I'm glad to introduce you SearchPhi, an AI-powered and open-source web search tool that aims to reproduce similar features to SearchGPT, built upon microsoft/Phi-3-mini-4k-instruct, llama.cpp๐ฆ and Streamlit.
Although not as capable as SearchGPT, SearchPhi v0.0-beta.0 is a first step toward a fully functional and multimodal search engine :)
If you want to know more, head over to the GitHub repository (https://github.com/AstraBert/SearchPhi) and, to test it out, use this HF space: as-cle-bert/SearchPhi
Have fun!๐ฑ
In the past days, OpenAI announced their search engine, SearchGPT: today, I'm glad to introduce you SearchPhi, an AI-powered and open-source web search tool that aims to reproduce similar features to SearchGPT, built upon microsoft/Phi-3-mini-4k-instruct, llama.cpp๐ฆ and Streamlit.
Although not as capable as SearchGPT, SearchPhi v0.0-beta.0 is a first step toward a fully functional and multimodal search engine :)
If you want to know more, head over to the GitHub repository (https://github.com/AstraBert/SearchPhi) and, to test it out, use this HF space: as-cle-bert/SearchPhi
Have fun!๐ฑ
posted
an
update
5 months ago
Post
2600
Hi HF community!๐ค
Hope y'all are as excited as me for the release of Llama 3.1! ๐ฆ
Following the release, I built a space exploiting HF Inference API, thanks to a recipe you can find in this awesome GitHub repo (https://github.com/huggingface/huggingface-llama-recipes/): you can now run Llama-3.1-405B customizing its system instructions and other parameters, for free! ๐
Follow this link: as-cle-bert/Llama-3.1-405B-FP8 and let the fun begin!๐
Hope y'all are as excited as me for the release of Llama 3.1! ๐ฆ
Following the release, I built a space exploiting HF Inference API, thanks to a recipe you can find in this awesome GitHub repo (https://github.com/huggingface/huggingface-llama-recipes/): you can now run Llama-3.1-405B customizing its system instructions and other parameters, for free! ๐
Follow this link: as-cle-bert/Llama-3.1-405B-FP8 and let the fun begin!๐
- 1 reply
posted
an
update
5 months ago
Post
1402
Hi HuggingFacers!๐ค
Good news concerning as-cle-bert/smolLM-arena, the chat arena where you can compare some of the Small Language Models (<1.7B) on the Hub and cast your vote to choose the best!๐ฑ
The space now has a new interface with chatbots instead of textboxs, it runs faster and it also comes with usage instructions :)
Have fun!๐
Good news concerning as-cle-bert/smolLM-arena, the chat arena where you can compare some of the Small Language Models (<1.7B) on the Hub and cast your vote to choose the best!๐ฑ
The space now has a new interface with chatbots instead of textboxs, it runs faster and it also comes with usage instructions :)
Have fun!๐
replied to
their
post
5 months ago
The SmolLM series is specifically designed to run on devices like smartphones, yes :) And, concerning the arena for models 7 to 20B, I didn't want to spoiler it, but It's coming soon! ;)
replied to
their
post
5 months ago
Thanks for pointing that out, now it should work!
posted
an
update
5 months ago
Post
563
Hi HF Community!๐ค
As you may know, small language models like the SmolLM series have been on the rise recently: although it may not be completely fair to compare them with larger models, my thought was that we could build a space where these SLMs could compete against each other in a chat arena, and here is what came out: as-cle-bert/smolLM-arena ๐
Even though there might be some little issues and hiccups due to GPU resources allocation, this space offers you the possibility to compare and play around with several Small Language Models, coming also with a leaderboard page (make sure to refresh it for the latest updates!)๐
Have fun!๐
As you may know, small language models like the SmolLM series have been on the rise recently: although it may not be completely fair to compare them with larger models, my thought was that we could build a space where these SLMs could compete against each other in a chat arena, and here is what came out: as-cle-bert/smolLM-arena ๐
Even though there might be some little issues and hiccups due to GPU resources allocation, this space offers you the possibility to compare and play around with several Small Language Models, coming also with a leaderboard page (make sure to refresh it for the latest updates!)๐
Have fun!๐
-
- 5 replies
posted
an
update
5 months ago
Post
1076
Responsibly building AI also means knowing its impact on the environment and the hidden carbon costs associated with it๐ฑ
If you're interested in the subject, you can check out my latest community article: https://huggingface.co./blog/as-cle-bert/is-ai-carbon-footprint-worrisome
Where I try to unravel AI's carbon footprint and potential solutions to reduce it๐ป
Enjoy!๐ค
If you're interested in the subject, you can check out my latest community article: https://huggingface.co./blog/as-cle-bert/is-ai-carbon-footprint-worrisome
Where I try to unravel AI's carbon footprint and potential solutions to reduce it๐ป
Enjoy!๐ค
posted
an
update
6 months ago
Post
775
Hi HF community! ๐ค
There's a new space out in the wild! as-cle-bert/self-reviewing-coding-assistant ๐ฆ
It's a self-correcting and self-reviewing python coding assistant based on GPT4-o and LangChain, inspired by Codium-AI's AlphaCodium ๐พ
Have fun! ๐
There's a new space out in the wild! as-cle-bert/self-reviewing-coding-assistant ๐ฆ
It's a self-correcting and self-reviewing python coding assistant based on GPT4-o and LangChain, inspired by Codium-AI's AlphaCodium ๐พ
Have fun! ๐
posted
an
update
6 months ago
Post
1831
๐ค Hi HF community!
๐ค A vital question that every developer may have asked themselves in the last three years is: how can we improve AI code generation?
๐ In my last Community blog post, I talk about Codium AI's AlphaCodium and how they tried to enhance LLMs coding skills with flow engineering: https://huggingface.co./blog/as-cle-bert/repetita-iuvant-how-to-improve-ai-code-generation
๐ Enjoy!
๐ค A vital question that every developer may have asked themselves in the last three years is: how can we improve AI code generation?
๐ In my last Community blog post, I talk about Codium AI's AlphaCodium and how they tried to enhance LLMs coding skills with flow engineering: https://huggingface.co./blog/as-cle-bert/repetita-iuvant-how-to-improve-ai-code-generation
๐ Enjoy!
posted
an
update
6 months ago
Post
3629
๐ค Hi HF Community!
๐งฌ As you may now, Evolutionary Scale recently released EvolutionaryScale/esm3-sm-open-v1 model here on the Hub, "a frontier generative model for biology, able to jointly reason across three fundamental biological properties of proteins: sequence, structure, and function" - as it is described on the dedicated GitHub page.
โก If you are curious about it and you want to try it out, you can do it with a space I built, as-cle-bert/proteins-with-esm
Hope this helps with your research!๐
๐งฌ As you may now, Evolutionary Scale recently released EvolutionaryScale/esm3-sm-open-v1 model here on the Hub, "a frontier generative model for biology, able to jointly reason across three fundamental biological properties of proteins: sequence, structure, and function" - as it is described on the dedicated GitHub page.
โก If you are curious about it and you want to try it out, you can do it with a space I built, as-cle-bert/proteins-with-esm
Hope this helps with your research!๐
posted
an
update
6 months ago
Post
2010
Hi HuggingFacers!๐ค
๐ฅ If you are Bioinformaticians or Biologists, you may be familiar with BLAST, a search algorithm that allows researchers to identify the group of organisms (species, taxa...) from which DNA/Protein sequences come.
๐ฅฑ You may also be familiar with the difficulties to interpret long and multi-parametric results coming out from BLAST searches: here's where we can operate with LLMs, summarizing the outputs and/or replying to queries about them!
๐งฌ You can now run BLAST for 16S rRNA bacterial sequences here on HF, summarizing and/or asking questions about the results, or make sense of your online BLAST searches uploading description tables, using the last space I built: as-cle-bert/BLAST-SummarAIzer
Have fun and may this be helpful to your research!๐ป
๐ฅ If you are Bioinformaticians or Biologists, you may be familiar with BLAST, a search algorithm that allows researchers to identify the group of organisms (species, taxa...) from which DNA/Protein sequences come.
๐ฅฑ You may also be familiar with the difficulties to interpret long and multi-parametric results coming out from BLAST searches: here's where we can operate with LLMs, summarizing the outputs and/or replying to queries about them!
๐งฌ You can now run BLAST for 16S rRNA bacterial sequences here on HF, summarizing and/or asking questions about the results, or make sense of your online BLAST searches uploading description tables, using the last space I built: as-cle-bert/BLAST-SummarAIzer
Have fun and may this be helpful to your research!๐ป