Search benchmarks against Exa
EU law, company and research-paper searches measured against Exa's search API, with the questions, scoring and scripts published. Where Exa is better, it says so.
| Area | Ours vs Exa | |
|---|---|---|
| EU law | Right article first: 76 / 102 vs 34 / 102 | Right article in top 5: 96 / 102 vs 75 / 102 |
| Companies | Found the company: 10 / 10 vs 10 / 10 | Register ID or LEI in the result: 10 / 10 vs 0 / 10 |
| Research papers | Right paper first: 16 / 20 vs 17 / 20 | Right paper in top 5: 17 / 20 vs 19 / 20 |
In short: where the answer is an official record (an EU article, a register entry) our own sources win clearly. On research papers Exa is slightly ahead. On the open web Exa's own index is larger than anything we run, and we do not claim otherwise.
EU law: does the search return the article itself?
Run on 15 November 2026.
Method. 102 held-out compliance questions across 41 EU acts (GDPR, AI Act, DSA, DMA, NIS2, DORA, MiCA, VAT, PSD2, AML, consumer, product and sustainability law), each with the article a correct answer must cite. They were written, and every expected article checked against its own text, before either search was run; none was used to tune our ranking. Ours counts only when the result's citation is exactly that article of that act. Exa (search type auto, 5 results with text) was scored leniently: a hit when the title, URL or first 1,500 characters name the article number and the act.
| Right article first | Right article in top 5 | Average time | |
|---|---|---|---|
| Our EU law search (own index of 47 acts) | 76 / 102 | 96 / 102 | about 1.3 s |
| Exa search, type auto | 34 / 102 | 75 / 102 | about 1.5 s |
- Exa mostly ranked pages about the law first: national regulators, law firms, explainers. The article itself often came second or third, and for some questions not at all.
- The 20 questions we tuned our ranking on (14 November) give the same picture: ours 10 first and 19 in the top 5, Exa 4 and 13 in this run.
- An agent that has to cite a provision needs the provision. Exa's pages are often still useful background.
Limits. We wrote the questions ourselves; they are checked against the law, not against either search. The questions do not say "EU": Exa searches the whole web and some of its answers were UK or US pages, which a question naming the EU might avoid. Ours searches EU law only. English questions only.
Re-run it: The 122 questions and expected articles · Our run · Exa run · Product: EU law search API
Companies: who is the registered legal entity?
Run on 14 November 2026.
Method. 10 European company names. Exa used category company, 5 results. Ours searched the official registers (France, Sweden, Norway, Finland, Czechia, Estonia) and GLEIF.
| Found the company | Register ID or LEI in the result | Average time | |
|---|---|---|---|
| Our company search (official registers and GLEIF) | 10 / 10 | 10 / 10 | 1.6 to 3.0 s |
| Exa search, category company | 10 / 10 | 0 / 10 | about 1.7 s |
- Both found every company. Exa returns the company's website with facts gathered from the web (founding year, headcount, funding).
- We return the registered entity: register ID, legal form, seat, LEI and a link to the register. That is what KYB, invoicing and supplier checks need. Different questions, both answered well.
Limits. 10 names is a small set. Where GLEIF reports no parent company, a same-named subsidiary can come first.
Re-run it: Benchmark script (companies and papers) · Product: Exa alternative for EU law and company data
Research papers: is the right paper first?
Run on 14 November 2026.
Method. 20 well-known papers: 10 asked by title words, 10 by a description. A hit when the result's title is the paper's title or its URL holds the paper's arXiv ID. Exa used category research paper.
| Right paper first | Right paper in top 5 | Average time | |
|---|---|---|---|
| Exa search, category research paper | 17 / 20 | 19 / 20 | about 0.6 s |
| Our paper search (OpenAlex and Crossref, open data) | 16 / 20 | 17 / 20 | about 1.3 s |
- Exa is ahead here, and faster. Ours is close from open data with no index of our own.
- Our misses were description-style questions and two answered by Crossref when OpenAlex rate-limited the test machine.
Limits. Measured from our machine; in production the free OpenAlex allowance is shared, so more answers come from Crossref.
Re-run it: Benchmark script (companies and papers)
Where Exa has no product to compare
Exa's API reference showed none of these on 14 November 2026, so there is nothing to race. Each answers from the official source.
| Product | What it returns |
|---|---|
| Sanctions screening | OFAC, EU, UN and UK lists in one call, each list's own date |
| KYB check | Register entry, LEI and group parent, EU VAT number and four sanctions lists in one call |
| EU e-invoices | Create, validate and convert XRechnung, ZUGFeRD and Factur-X |
| EU VAT numbers | Checked live in the European Commission's VIES |
How these were run
- Exa was called through its public API on a standard account, with the settings named in each method. Spend: under $0.50 of Exa's free monthly credit.
- Our side ran the same code our API runs, from our test machine.
- Each question set, scorer and script is in the public repository linked above. Send us a question set and we will add the run, whatever it shows.
- Exa's own SDKs (exa-js, exa-py) work against our API: change the base URL to
https://tools.funwords.eu/v1.