You do not need cynicism to read this market accurately. You need to know which claims carry information and which are decoration.
Phrases that carry little information
"Powered by AI." Covers everything from a carefully built system to one API call behind a button. Not a specification.
"Trained on your data." Usually means retrieval — your documents are searched and relevant pieces are put into the prompt. Genuine training or fine-tuning on your data is expensive and much rarer than the phrase implies. Worth asking which one is meant, because they behave very differently.
"Understands your business." Almost always retrieval over documents you supplied. Useful, but it is search plus a prompt, not comprehension.
"Autonomous." Ask exactly which actions it takes without approval, and what happens when it is wrong. The answer is usually narrower and more supervised than the word suggests — which is good engineering, and worth knowing.
"Hallucination-free." No system built on a language model can promise this. Grounding in retrieved sources reduces it substantially; it does not eliminate it. Treat the absolute claim as a signal about the marketing, not the product.
What benchmarks actually measure
Benchmark scores are real measurements of specific tasks under specific conditions. The gap between them and your work is where products differ most.
Three things worth knowing:
- Contamination. If benchmark questions appeared in training data, the score measures memorisation rather than capability. This is a known and unresolved problem.
- Selection. A vendor cites the benchmarks it does well on. The absence of a common benchmark is information.
- Narrowness. Excellent performance on graduate exam questions predicts very little about handling your inconsistently formatted spreadsheet.
Costs that appear later
Verification time. If output must be checked, that time belongs in the total. A tool that halves drafting time and doubles review time has saved nothing.
Integration and maintenance. Connecting a tool to real systems is where the schedule goes. Then the API changes.
Prompt and workflow rebuilding. Model updates change behaviour. Prompts tuned to one version can degrade on the next, and you will not be told which day that happened.
Attention. Six tools each saving a little, each needing to be learned and kept in mind, can cost more than they return.
What to actually look for
Concrete, checkable claims: what data it can access, what it does when uncertain, where processing happens, what you can export, what it costs at your volume.
A vendor who answers those precisely is telling you something. A vendor who answers them with adjectives is also telling you something.