The question nobody in this market answers
Every comp tool will hand you a number. An ARV, a value range, a list of comparable sales. The number always looks confident. So ask the vendor a simple follow-up. Across all the estimates you produced last quarter, what was the median miss against the prices those homes actually closed at.
ChatARV sells comp reports from $89 per month. It publishes no accuracy metric. Bricked AI markets model-selected comps, condition analysis, and machine learning repair estimates. It publishes no accuracy metric. DealCheck, a deal calculator, is honest about the division of labor. You supply the value judgment, it runs the math. It has no valuation engine to grade, and it publishes no accuracy metric either.
None of this proves any of those tools is inaccurate. It proves something more basic. Nobody, including the vendors, knows how accurate they are. If the answer to the question is silence, you are the error bar.
What a graded valuation actually looks like
Grading a valuation tool requires one discipline above all others. The prediction must be locked before the outcome is known. We call this freeze-forward, and it works like a chain of custody for a number.
Freeze-forward, step by step
- 1.While a home is listed for sale, Resideline produces an estimate and freezes it. No edits, no do-overs, no quiet updates when the market moves.
- 2.The home closes at a real price, set by a real buyer and a real seller.
- 3.The frozen estimate is graded against that closing price. The gap is the error, and it is recorded whether it flatters us or not.
- 4.The result posts to a public scoreboard at resideline.com/accuracy, which updates as closings land.
Today the scoreboard shows a 2.48% median error across 1,380+ graded closings, with about 92% of estimates landing within 10% of the actual sale price, across 50 states. Those numbers will move as more homes close. That is the point. A live scoreboard is a standing offer to be checked.
Why LLM comp selection cannot produce this number
The newer comp tools are built on large language models. Press describes ChatARV as a ChatGPT spinoff for sizing up deals. Bricked AI leads with model-selected comps. Conversational comp selection is a real convenience. It is also, structurally, the wrong shape for producing an error rate. Three reasons.
First, an error rate needs a single frozen prediction per property. A language model generates a fresh answer on every request. Ask twice and you can get two different comp sets and two different values. Which one gets graded. Until the vendor freezes one canonical estimate per property and stores it, there is nothing to measure.
Second, grading needs a timestamped ledger of predictions made before the outcomes were known. A chat transcript on a user's screen is not a ledger. The vendor would need to record every estimate, wait months for closings, match them, and publish the misses. That is infrastructure and, more importantly, exposure. Publishing a scoreboard means publishing your bad days.
Third, the incentives point the other way. A tool that regenerates its answer on demand can always look right in hindsight, because the answer you remember is the last one it gave you. Freeze-forward removes that comfort. The estimate on our scoreboard is the one we produced while the home was still for sale, not the one we would produce today knowing the close.
To be precise, this is not an argument that language models pick bad comps. They may pick fine ones. It is an argument that a conversational comp picker, as shipped today by these vendors, cannot tell you its own error rate, and the vendors in question do not publish one. A graded AVM can, because grading is built into the product instead of bolted onto the marketing.
The non-disclosure state test
Where the data gets hard
Eleven states do not make sale prices public record: Texas, Utah, Missouri, Louisiana, Mississippi, Kansas, Indiana, Montana, New Mexico, North Dakota, and Wyoming. ChatARV documents that it only works where sale prices are public record, and that it does not work in these states. Credit for the honesty. But note what the limitation reveals. A tool that depends entirely on public-record prices has no independent valuation engine underneath. When the easy data disappears, so does the product.
Resideline covers 50 states, and the scoreboard grades closings across that footprint. If your market is Dallas or Kansas City or Albuquerque, the comparison shopping ends quickly.
Receipts change behavior
A public scoreboard is not just marketing with extra steps. It disciplines the engine that feeds it. Because estimates are frozen and graded, we cannot quietly repair a bad miss after the fact. We have to improve the engine instead. Photo condition grading that separates the As-Is value from the ARV exists because the scoreboard punished us for treating a dated kitchen like a renovated one.
It also changes what we can offer you. Because we know the distribution of our misses, we can put money on it. If a frozen estimate misses the actual sale price by more than 10%, that report is free. No comp tool in this comparison advertises anything like that, and the reason is the same as before. You cannot guarantee a number you have never measured.
Three asks for any vendor
- 1. What is your median error against actual closing prices.
- 2. How many closed sales is that measured on.
- 3. Where can I watch the number update as new homes close.
Our answers: 2.48%, 1,380+ closings, and resideline.com/accuracy.
Competitor descriptions reference public pricing and marketing pages as of July 2026. ChatARV, Bricked AI, and DealCheck are products of their respective owners, and Resideline is not affiliated with any of them.