Free evaluation tool
Site search relevance scorecard
Compare search engines with the same catalog and query set. Record the expected SKU, copy the first three returned SKUs in rank order, and see which checks pass. This worksheet evaluates your observed results; it does not run queries against your store or claim a vendor benchmark.
How to use the scorecard
- Choose buyer queries before tuning. Keep a separate acceptance set that the vendor has not seen.
- Set a requirement per query: the expected SKU first, in the top three, or no results. Separate alternative acceptable SKUs with a vertical bar (
|). - Run the query on each search implementation using the same catalog snapshot and filters. Enter returned SKUs in result order, separated with
|, then mark the row tested. - Review failures, especially critical queries. Download the CSV to keep a dated record alongside catalog version, prices, stock and response times.
Exact SKU comparison ignores letter case and surrounding spaces, but keeps punctuation. A wrong code with one changed character is still wrong. A zero-result check passes only after you mark the query tested and leave observed results empty.
Use this with a full pilot
This score measures a small part of relevance. The full evaluation also checks filters, data freshness, stock, prices, mobile interaction, latency and business outcomes. Agree on acceptance thresholds for your catalog before comparing vendors.