Human Decision Authority in AI-Assisted Underwriting
An anonymous Ampity account of underwriting decision support, where model recommendations and explanations informed human review, without verified outcome claims.
[Overview] Recommendations supported an underwriting decision
This anonymized Ampity engagement concerned loan-underwriting decision support. The historical account describes an AI layer providing recommendations and explanations for underwriters, with humans making the final calls. The selected approach assisted application review without assigning the final underwriting decision to the model.
The account supplies a description of the work, rather than an operating audit. It does not establish processing speed, decision consistency, default reduction, adoption or regulatory compliance. The customer remains unnamed.
[Context] Manual review and the retained decision owner
Underwriters reviewed applications manually. Slow turnaround and variation between reviewers were the reported concerns. Those concerns motivated decision support, but they do not establish that a model would make better decisions or identify the cause of the variation.
Human final-decision authority and explanations for AI recommendations were recorded constraints. They are presented here as requirements of this engagement. No jurisdiction, applicable lending rule or legal determination is identified, so the case makes no claim that all underwriting must follow this design.
[Decision] Assist review rather than replace it
The original alternatives contrasted AI making decisions with people spot-checking them, and AI making suggestions that people reviewed before deciding. The account selected the second approach. That choice places the consequential judgment with the underwriter rather than treating a model score as the final disposition.
The underwriter still needed to inspect the application and consider the recommendation. Adding that output also introduces another source of possible error; retaining a reviewer does not itself prove effective review or faster handling.
[Design] Keep the score, explanation and decision distinct
The recorded design includes application scoring, SHAP feature-attribution explanations and human review of AI suggestions alongside application context. It does not identify a model family, score scale, hosting provider, data pipeline or decision-record implementation. No deployment topology is inferred from those component descriptions.
NIST's explanation principles distinguish explanation accuracy from decision accuracy and consider whether an explanation is meaningful to its recipient. These are general evaluation distinctions, not findings about this engagement's interface.
The SHAP maintainers' causal-interpretation example also shows why explaining a predictive model does not, by itself, identify the effect of changing an input. A feature attribution should not be presented as proof of why an applicant will behave a certain way or what an intervention will achieve.
Historical responsibility view reconstructed from the original account, not a deployment or control audit. The model supplied a recommendation and explanation; human underwriters retained the final decision. Reviewer feedback is mentioned in the account, but its training, evaluation and release path is not established.
Application context and an AI recommendation with a SHAP explanation were inputs to human review in the described design. The final underwriting decision belonged to a person. The account does not establish the receiving system's permission controls, completeness of review context or model-update procedure. Feedback is not depicted as an automatic instruction to change the running model.
[Feedback] Reviewer input and model changes need separate treatment
Reviewer feedback appears in the original description of model updating. The account does not specify how feedback was selected, labelled, approved for training or evaluated before a model change. It therefore does not establish an automatic or continuously improving training-and-deployment loop.
For a comparable implementation, separate an application decision, feedback and permission to change the model. Reviewer disagreement needs investigation before becoming a training label. Model changes need their own authorized evaluation and release decision.
[Delivery boundary] The account describes roles, not a verified rollout
The narrative does not establish pilot validation, production coverage or how stale inputs, unavailable recommendations and overrides were handled. It also does not show whether a receiving system technically prevented a model output from becoming a final decision. The described responsibility split leaves those operating questions open.
[Review worksheet] Inspect one permitted decision path
The following is a proposed read-only review for a comparable system, not an exercise performed for this engagement. Use only authorized, minimized records in their approved environment. Do not copy applicant information into public examples or submit decisions through a live lending integration.
| Boundary | Record to inspect | Question for the owner |
|---|---|---|
| Application and recommendation | Permitted input revision, model reference and corresponding output | Did the suggestion concern the evidence available for this application? |
| Explanation and review | Explanation shown, application context available and review record | Could the reviewer inspect the basis of the suggestion and record a different judgment? |
| Final decision | Authorized decision owner and recorded disposition | Who made the consequential decision, and what established their authority? |
| Feedback and model change | Feedback purpose, permitted use and any separate evaluation/release record | Was reviewer feedback distinguished from an approved model update? |
If a later exercise is needed, first authorize an isolated environment with synthetic or approved sanitized fixtures, test identities and non-delivering destinations. Prevent effects on real applications, applicant communications, credit records and funding. Without that separation, keep the investigation read-only. Stopping new model calls would not reverse a decision already recorded or a message already sent.
[Lessons] Define what human review must establish
For a comparable system, specify the evidence a reviewer needs and the unresolved cases they can hold or escalate. A human decision in a process diagram does not establish improved credit outcomes or effective control over the receiving system.
Start with one permitted decision record and use the worksheet to identify the first missing link. The AI governance framework covers the broader decision and change inventory. The review-queue capacity article addresses the separate question of whether reviewers can handle the work created by an AI workflow.
Technical reference
NISTIR 8312 explanation principles and the SHAP maintainers' causal-interpretation example were checked on 17 October 2026 for the general explanation limits linked above. Neither source establishes this engagement's model version, interface, review effectiveness, deployment, legal obligations or outcomes.