What an AI Assistant Should Never Decide Alone

Start with a three-part workflow

A clear approval design has three states: proceed, prepare for review, and stop.

Proceed means the assistant may complete a narrow, reversible action under an established rule. Prepare for review means it may collect information or draft the proposed action, but a named person must approve the final step. Stop means the assistant lacks permission or sufficient context, so it preserves the current state and escalates.

This structure applies least privilege in ordinary language. The assistant receives enough authority to be useful, but not a standing invitation to improvise. Give the assistant its own identity where the system permits it, rather than having it operate invisibly through a shared human account. Limit credentials to the systems and actions it needs. Keep a record of what it proposed, what it did, and who approved consequential actions.

Human approval also needs a real owner. “Ask someone” is not a workflow. Name a role, such as the office manager or account owner, and define what happens when that person is unavailable.

• Proceed: narrow, expected, reversible, and within an approved limit.
• Prepare for review: gather context and draft the action without committing it.
• Stop: preserve the current state and route the case to the named owner.

Money needs limits and confirmation

An assistant should not independently decide to send money, issue an unusual refund, change a price, accept contractual payment terms, or commit the business to a purchase. A mistaken draft is inconvenient. A mistaken transaction becomes a real obligation.

The safe middle ground is preparation. An assistant can assemble invoice details, compare a request with written policy, flag missing information, and draft a proposed refund. A person should approve the transaction whenever it moves money or creates a material financial commitment.

Routine cases can have written thresholds. For example, a business might permit an assistant to prepare refunds that match a documented policy while requiring approval before every release. The amount and approver are business choices, not universal numbers. Banking permissions should remain narrower than research or drafting permissions.

• Does this transfer funds, issue credit, change a price, or create a financial commitment?
• Is the amount and transaction type inside a written policy?
• Can the assistant prepare the transaction without being able to release it?
• Which named person gives final approval?

Customer promises belong to accountable people

A polished reply can accidentally become a promise. Delivery dates, warranties, discounts, scope changes, remedies, compliance assurances, and exceptions can affect the relationship long after the message is sent.

Let an assistant answer from approved material when the response is routine and does not create a new commitment. Require review when a message changes price, timing, scope, policy, or responsibility. If the supporting information conflicts, the assistant should identify the conflict instead of choosing the version that sounds most helpful.

Hypothetical example: A customer asks whether a project can be finished Friday. The scheduling system shows Friday, but a project note says a required part has not arrived. The assistant should summarize both facts and draft a response for review. It should not promise Friday by selecting the cleaner-looking field.

• Does the message create or modify a promise?
• Is every material statement supported by an approved source?
• Do the relevant systems agree?
• Could a reasonable customer rely on this message when making a decision?

Sensitive records require purpose, minimum access, and restraint

An assistant should not decide on its own who may see payroll details, personnel notes, health information, financial records, credentials, private customer data, or other restricted material. Access should follow an established business purpose and permission rule.

The assistant should receive the minimum data needed for the task. If it only needs to know whether an invoice is overdue, it may not need bank details or an entire customer history. If it needs to route a record, it may not need permission to export or share that record.

Credential management matters here. An assistant using a broad shared account can inherit access that its actual job does not require. A distinct identity and narrow authorization make it easier to limit exposure and determine which actions came from the assistant.

When the recipient, purpose, or permission is unclear, the assistant should withhold the information and ask. This is an operational safeguard, not a legal conclusion about any particular record.

• Is this data necessary for the assigned task?
• Is the recipient already authorized for this specific information?
• Can the task use fewer fields, a summary, or a redacted copy?
• Would an export, attachment, or message create an unnecessary second copy?

Destructive actions need a preview and a recovery path

Deletion, overwriting, bulk updates, account deactivation, record merging, and permission changes can damage good data quickly. An assistant should not independently perform a destructive action simply because a request contains a forceful verb such as “clean,” “replace,” or “remove.”

First require a preview: what will change, how many items are affected, and why each item qualifies. Then require approval from the owner of that system or data. Prefer reversible methods such as archiving, quarantining, versioning, or moving records to a review queue. Test bulk rules on a small sample before wider use.

The assistant should stop if the scope changes after approval. Permission to archive 12 identified duplicate contacts is not permission to delete every record that a new matching rule labels as a duplicate.

• Is the action reversible?
• Can the assistant show the exact affected records before acting?
• Is there a backup, version history, archive, or tested recovery method?
• Does the approved scope still match the action about to run?

Ambiguous exceptions are where the boundary earns its keep

Most automation looks dependable on ordinary cases. The revealing moment is the exception: two policies conflict, a field is missing, a customer asks for special treatment, or the requested action falls just outside a threshold.

Do not tell the assistant to “use good judgment” without defining the limits of that judgment. Give it explicit stop conditions. Examples include missing approval, conflicting source records, an unknown recipient, a request outside policy, an unexpectedly large batch, or any attempt to obtain broader access.

For each stop condition, define the review package. It should include the request, relevant facts, applicable rule, uncertainty, proposed action, possible impact, and the person authorized to decide. The assistant may recommend an option if that is part of its role, but it should distinguish the recommendation from the decision.

NIST’s concept paper explores human-in-the-loop controls and accountability alongside agent identity and authorization. Poygan’s practical conclusion is that review should be a designed stage of the workflow, not an emergency button added after deployment.

• What exactly is unusual or conflicting?
• Which rule would normally apply?
• What could happen if the assistant guesses incorrectly?
• Who owns this exception?
• What evidence does that person need to decide?