
How vector search works, why I replaced paid embedding and chat calls with a free open model, and what 62 test queries showed.
The AI search on Automate & Tweak finds tools by meaning, not by matching words. For most of this year it depended on two paid AI services: one turned text into vectors, the other wrote a chat answer on top of the results. In November 2026 I replaced both with a free, open embedding model that I run myself. This post explains how vector search works, what changed, and what I measured before and after the switch.
How vector search works
An embedding model reads a piece of text and returns a fixed list of numbers, a vector. Texts with a similar meaning get similar vectors, even when they share no words and even when they are in different languages. "Make a logo for my bakery" and a tool description about brand design end up close together.
Search then takes three steps:
- Index once: every tool description is turned into a vector and stored.
- Embed the query: the visitor's search text is turned into a vector with the same model.
- Compare: cosine similarity scores every stored vector against the query, and the highest scores are the results.
The comparison itself is simple arithmetic:
import math
def cosine(a, b):
dot = sum(x * y for x, y in zip(a, b))
return dot / (math.sqrt(sum(x * x for x in a)) * math.sqrt(sum(y * y for y in b)))
print(round(cosine([0.2, 0.9, 0.1], [0.25, 0.8, 0.0]), 3))
One rule matters a lot: vectors from different models cannot be compared. If you change the model, you embed everything again.
What the first version did
The first version, from early 2026, used a paid embedding API for the vectors and a hosted chat model for the conversation. Every search made at least two paid calls: one to embed the question, one to have the chat model classify the message and write an answer that recommended tools from the top results. A daily job also refreshed the tool vectors through the paid API.
It worked, but every visitor search cost money, and the chat answer added waiting time without making the list of tools any better. When I rebuilt the site in November 2026, I dropped the chat layer and kept plain semantic results.
Why I stopped paying per search
My rule for this project is that content work and routine features should not depend on paid AI calls. Paying for each visitor's query did not fit that rule, so on 14 November 2026 I switched query embedding off. The site fell back to keyword search.
That fallback showed how much the vectors had been doing. In a test with 62 real-style queries, keyword search returned something useful for 2 of them. Natural sentences such as "write meeting notes from a recording" simply found nothing.
Here are three of the test queries, with the top three results from each method (test run of 16 November 2026):
query: make a logo for my startup
keyword search: (nothing)
vector search: Logopony, LogoAI, LogoCreatorAI
query: transcribe meeting recordings and summarize them
keyword search: (nothing)
vector search: Fireflies, Video Highlight, tl;dv
query: video achtergrond verwijderen (Dutch: remove video background)
keyword search: (nothing)
vector search: remove.bg, cutout.pro, Videoleap
Choosing a free model
I tested two open, multilingual embedding models that are free to use, a small one (multilingual-e5-small) and a larger one (BGE-M3), on the same 62 queries: 40 in English, 10 in other European languages and Hindi, and 12 in Korean, Japanese and Chinese.
- Both models answered all 62 queries with semantic results.
- English: about equal, with a few better and a few worse top-three results on each side.
- Other languages: the larger model was clearly better. The small one returned unrelated tools for some short queries outside English, as the examples below show.
- Speed: the median query took about 120 ms with the small model and about 350 ms with the larger one.
инструмент за създаване на лого (Bulgarian: logo maker tool)
small model: Loom, Arvin, Socrat
large model: LogoAI, Logo Diffusion, Logopony
회사 웹사이트용 챗봇 (Korean: chatbot for a company website)
small model: You.com, Jan, Loom
large model: ChatShape, Chat Thing, CustomGPT.ai
会议记录自动生成 (Chinese: generate meeting minutes automatically)
small model: AI2SQL, Shadow, AiSOAP
large model: Noty.ai, Fireflies, Demodesk
The site is available in many languages, so I chose the larger model and accepted the slower queries.
How it runs now
Every tool description, around 1,400 of them in November 2026, has a vector from the new model. A nightly job re-embeds only the tools whose text changed, which takes seconds. New tools are embedded right after they are added.
When a visitor searches, the query is embedded, compared against all tool vectors held in memory, and results above a minimum score are shown. If the model is slow or unavailable, search waits at most three seconds and then falls back to keyword search, so the page never breaks.
Since 18 November 2026 the normal search box uses the same vectors. Exact name matches still come first, so a search for a brand finds that brand, then tools by meaning follow, then tools whose description contains the words. A full sentence such as "meeting recordings and summarize them" now returns meeting tools instead of nothing.
A few lessons from the switch:
- Embed first, then switch: I built the complete set of new vectors while the old ones still served searches, and only then pointed search at the new model.
- Keep the old vectors: the paid vectors are still stored as a backup, so going back is a configuration change, not a rebuild.
- Bulk work competes with visitors: a full rebuild took about four hours. While it ran, live queries slowed down and fell back to keywords, so I now do large runs at night.
- Cap the input length: long inputs cost a lot of memory. Tool texts are short, so a lower maximum length lost nothing.
What I do not know yet
I have not measured whether visitors click more results since the switch; usage of the AI search was low while it ran on keywords, so there is little to compare with. Very short queries in Chinese and Japanese are still the weakest case. I also do not know yet whether a reranking step on top of the vector results would be worth its extra time.
Frequently asked questions
Does a free embedding model give worse results than a paid one?
I did not run a strict side-by-side quality test against the paid model; the two returned different nearest neighbours, which says little about quality on its own. The free larger model did return relevant tools for nearly every test query, in English and in other languages.
Do I need a chat model on top of vector search?
Not for finding tools. A ranked list answered the question directly, and the chat answer mostly added cost and waiting time.
What happens when the model is down?
Search falls back to keyword matching after a short timeout. Results are worse for natural sentences, but the page keeps working.
What to do first
- Write down 30 to 60 real queries, including ones in every language you support.
- Measure keyword search on that list so you know your baseline.
- Test one small and one larger open multilingual model on the same list.
- Embed everything with the new model before you switch search over.
- Keep a fallback for when the model is slow or unavailable.
- Run large re-embeds outside your busiest hours.
Try it yourself: type a full sentence into the search box on Automate & Tweak (opens in a new tab), or read more posts about AI and multilingual sites.