AI model selection for real estate: test the work before rollout

Choose an AI model by testing the work your real estate team needs to finish. A lease extraction, an operating-statement comparison and an investor narrative have different failure modes. A strong result on a general benchmark does not establish that a model will handle your documents, calculations or access rules.

This guide offers a practical evaluation method for language-model workflows. It uses lease abstraction as an example and focuses on source accuracy, review effort, cost and release controls. It does not rank model brands or prescribe a universal passing score.

Define the task and the evidence of a correct answer

Write down the input, expected output and person who will review it. For a lease abstract, specify the fields, treatment of amendments, required source references and what to do when a term is absent or ambiguous. A useful answer may be “not established by the supplied documents,” with an explanation of the missing evidence.

Microsoft's model-selection guidance recommends matching capabilities to the task and comparing candidates using representative inputs. Use that as a starting point, then define checks that fit the actual deliverable. For example, an option notice deadline needs the correct clause and date logic, while a property summary needs supported facts and clear limitations.

Keep calculations in inspectable formulas or appropriate calculation tools. Test whether the model chooses the right inputs and explains the result. A fluent narrative does not verify the arithmetic behind it.

Apply access and deployment requirements first

Confirm where the candidate processes data, which people and systems can access it, applicable retention settings and the provider's permitted uses. Resolve these questions before sending confidential test documents. Use approved, minimized or appropriately redacted material when the evaluation does not need identifiable records.

Open weights and local hosting do not automatically establish privacy or a lower total cost. The deployment still needs access controls, logging, maintenance and a documented data path. Evaluate those arrangements alongside the model. Exclude a candidate that cannot meet a required boundary even if its average quality score is high.

Build a test set that exposes real mistakes

Use examples that reflect the formats and conditions your team receives. Include clear digital documents, scanned pages, unusual layouts, missing attachments and amendments that change an earlier term. Add cases with similar property names and conflicting dates, where an unsupported assumption would be easy to miss.

Have a qualified reviewer establish the expected answer and its source before scoring candidates. Keep a separate group of examples for prompt development so that the final evaluation tests unfamiliar cases. Record the files and versions used. There is no universal sample count that proves a workflow is ready; the set needs to cover the failures that matter to your scope.

For document preparation and source handling, use the checks in our real estate document-processing guide. Poor extraction or missing pages can make a capable model appear unreliable, so retain enough evidence to identify where an error began.

Score the output and the work required to approve it

Run each candidate against the same test cases and agreed instructions. Save the model version, configuration, source set and output. The following worksheet separates answer quality from the effort required to use it.

Check	What to record	Example failure
Field correctness	Correct, incorrect, omitted and unsupported values	Rent from the original lease survives a later amendment
Source support	Whether each material answer points to the right evidence	A correct-looking date cites an unrelated page
Uncertainty handling	Whether missing or conflicting evidence is identified	An absent notice term becomes an invented deadline
Format and tool use	Whether the output fits the template and required calculations work	A numeric field contains an explanation instead of a value
Review effort	Reviewer time, corrections and unresolved exceptions	The abstract requires checking every field from scratch
Reliability and cost	Completion time, failures, retries and cost per accepted output	A low-cost attempt needs repeated reruns before review

Report critical failures separately from averages. A model can extract many routine fields correctly and still miss the amendment that changes the decision. Agree required checks and acceptance criteria before seeing the results. Use weighted scores only to compare candidates that satisfy those requirements.

Diagnose the cause before changing models

For each failed case, inspect the input that reached the model. Was the correct page present? Did extraction preserve the table? Did retrieval select the current amendment? Did a calculation tool return an error? Separate these causes from a model's incorrect interpretation.

Change one part at a time and rerun the relevant cases. Keep earlier failures in the test set so that an improvement does not hide a regression. If a simple workflow already meets the requirements, additional routing or models need a measurable benefit to justify their maintenance.

Release with a reviewer and a fallback

Run the candidate alongside the existing process on approved material before relying on its output. Compare the results without letting the test send investor communications, change records or trigger a consequential action. Name the person who can approve the change and the conditions that require returning to the previous process.

Repeat the evaluation when the model version, prompts, retrieval, tools or source formats materially change. Track corrections after release and add representative failures to future tests. The NIST AI Risk Management Framework is a voluntary reference for considering trustworthiness throughout AI development and use; a test worksheet alone does not establish compliance or certification.

Keep the firm's knowledge useful as models change

Store your definitions, approved examples, source references and evaluation results so the team can compare a future candidate with the same evidence. Model selection then becomes a repeatable review of the workflow's performance.

Leni's own-your-AI platform is designed around connected real estate data, company knowledge and model choice. Teams can also scope specialist support through Leni's AI-native services to prepare and check agreed work. Choose the model and delivery process against the output your team must review, with people retaining approval of decisions and external communications.