jonas.splinter
Search
Blog

From paid AI calls to a free model: how my tool search finds meaning


A magnifying glass over a grid of app-style tool icons, highlighting a group of design tools while the others fade.

How vector search works, why I replaced paid embedding and chat calls with a free open model, and what 62 test queries showed.

The AI search on Automate & Tweak finds tools by meaning, not by matching words. For most of this year it depended on two paid AI services: one turned text into vectors, the other wrote a chat answer on top of the results. In November 2026 I replaced both with a free, open embedding model that I run myself. This post explains how vector search works, what changed, and what I measured before and after the switch.

How vector search works

An embedding model reads a piece of text and returns a fixed list of numbers, a vector. Texts with a similar meaning get similar vectors, even when they share no words and even when they are in different languages. "Make a logo for my bakery" and a tool description about brand design end up close together.

Search then takes three steps:

  1. Index once: every tool description is turned into a vector and stored.
  2. Embed the query: the visitor's search text is turned into a vector with the same model.
  3. Compare: cosine similarity scores every stored vector against the query, and the highest scores are the results.

The comparison itself is simple arithmetic:

import math

def cosine(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    return dot / (math.sqrt(sum(x * x for x in a)) * math.sqrt(sum(y * y for y in b)))

print(round(cosine([0.2, 0.9, 0.1], [0.25, 0.8, 0.0]), 3))

One rule matters a lot: vectors from different models cannot be compared. If you change the model, you embed everything again.

What the first version did

The first version, from early 2026, used a paid embedding API for the vectors and a hosted chat model for the conversation. Every search made at least two paid calls: one to embed the question, one to have the chat model classify the message and write an answer that recommended tools from the top results. A daily job also refreshed the tool vectors through the paid API.

It worked, but every visitor search cost money, and the chat answer added waiting time without making the list of tools any better. When I rebuilt the site in November 2026, I dropped the chat layer and kept plain semantic results.

My rule for this project is that content work and routine features should not depend on paid AI calls. Paying for each visitor's query did not fit that rule, so on 14 November 2026 I switched query embedding off. The site fell back to keyword search.

That fallback showed how much the vectors had been doing. In a test with 62 real-style queries, keyword search returned something useful for 2 of them. Natural sentences such as "write meeting notes from a recording" simply found nothing.

Here are three of the test queries, with the top three results from each method (test run of 16 November 2026):

query: make a logo for my startup
  keyword search:  (nothing)
  vector search:   Logopony, LogoAI, LogoCreatorAI

query: transcribe meeting recordings and summarize them
  keyword search:  (nothing)
  vector search:   Fireflies, Video Highlight, tl;dv

query: video achtergrond verwijderen   (Dutch: remove video background)
  keyword search:  (nothing)
  vector search:   remove.bg, cutout.pro, Videoleap

Choosing a free model

I tested two open, multilingual embedding models that are free to use, a small one (multilingual-e5-small) and a larger one (BGE-M3), on the same 62 queries: 40 in English, 10 in other European languages and Hindi, and 12 in Korean, Japanese and Chinese.

  • Both models answered all 62 queries with semantic results.
  • English: about equal, with a few better and a few worse top-three results on each side.
  • Other languages: the larger model was clearly better. The small one returned unrelated tools for some short queries outside English, as the examples below show.
  • Speed: the median query took about 120 ms with the small model and about 350 ms with the larger one.
инструмент за създаване на лого   (Bulgarian: logo maker tool)
  small model:  Loom, Arvin, Socrat
  large model:  LogoAI, Logo Diffusion, Logopony

회사 웹사이트용 챗봇   (Korean: chatbot for a company website)
  small model:  You.com, Jan, Loom
  large model:  ChatShape, Chat Thing, CustomGPT.ai

会议记录自动生成   (Chinese: generate meeting minutes automatically)
  small model:  AI2SQL, Shadow, AiSOAP
  large model:  Noty.ai, Fireflies, Demodesk

The site is available in many languages, so I chose the larger model and accepted the slower queries.

How it runs now

Every tool description, around 1,400 of them in November 2026, has a vector from the new model. A nightly job re-embeds only the tools whose text changed, which takes seconds. New tools are embedded right after they are added.

When a visitor searches, the query is embedded, compared against all tool vectors held in memory, and results above a minimum score are shown. If the model is slow or unavailable, search waits at most three seconds and then falls back to keyword search, so the page never breaks.

Since 18 November 2026 the normal search box uses the same vectors. Exact name matches still come first, so a search for a brand finds that brand, then tools by meaning follow, then tools whose description contains the words. A full sentence such as "meeting recordings and summarize them" now returns meeting tools instead of nothing.

A few lessons from the switch:

  • Embed first, then switch: I built the complete set of new vectors while the old ones still served searches, and only then pointed search at the new model.
  • Keep the old vectors: the paid vectors are still stored as a backup, so going back is a configuration change, not a rebuild.
  • Bulk work competes with visitors: a full rebuild took about four hours. While it ran, live queries slowed down and fell back to keywords, so I now do large runs at night.
  • Cap the input length: long inputs cost a lot of memory. Tool texts are short, so a lower maximum length lost nothing.

What I do not know yet

I have not measured whether visitors click more results since the switch; usage of the AI search was low while it ran on keywords, so there is little to compare with. Very short queries in Chinese and Japanese are still the weakest case. I also do not know yet whether a reranking step on top of the vector results would be worth its extra time.

Frequently asked questions

Does a free embedding model give worse results than a paid one?

I did not run a strict side-by-side quality test against the paid model; the two returned different nearest neighbours, which says little about quality on its own. The free larger model did return relevant tools for nearly every test query, in English and in other languages.

Not for finding tools. A ranked list answered the question directly, and the chat answer mostly added cost and waiting time.

What happens when the model is down?

Search falls back to keyword matching after a short timeout. Results are worse for natural sentences, but the page keeps working.

What to do first

  1. Write down 30 to 60 real queries, including ones in every language you support.
  2. Measure keyword search on that list so you know your baseline.
  3. Test one small and one larger open multilingual model on the same list.
  4. Embed everything with the new model before you switch search over.
  5. Keep a fallback for when the model is slow or unavailable.
  6. Run large re-embeds outside your busiest hours.

Try it yourself: type a full sentence into the search box on Automate & Tweak (opens in a new tab), or read more posts about AI and multilingual sites.

Related posts

More to read on the same topics

Related updates

Short notes on the same topics