Agentic Commerce and How to Test AI Shopping Agents Before Checkout

A parent opens a staging build of an AI shopping agent and types one request: a waterproof rain jacket for a 7-year-old, size 7/8, under $60 with shipping, needed before a school trip. This is agentic commerce at its simplest. The agent suggests a navy jacket, and the parent asks for a green one instead.

The agent replies that it “went ahead” and ordered the navy jacket in size 6, because 7/8 was low on stock. Payment went through and the merchant accepted the order. The shopper now owns a coat that does not fit, in a colour they rejected, bought without their approval.

Agentic commerce protocols make an agent’s purchase verifiable, but nothing in them checks that the agent bought the right thing. That is a testing job, and it has to happen before checkout. Below we walk through the protocols and their gaps, then build a persona test plan with a runnable grader and release gates on top of it.

TL;DR

• Agentic commerce: an AI agent handles a purchase from search to payment, inside limits the shopper sets.
• ACP standardizes the checkout session, AP2 standardizes the signed approval, and UCP can carry AP2 mandates.
• A protocol proves who approved and paid, not that the agent picked the right item.
• Test with personas that carry a budget, a size and a change of mind; grade fit, budget with shipping and explicit confirmation.
• Gate releases on confirmation and budget, and turn complaints into personas.

What is agentic commerce?

Agentic commerce is buying in which an AI agent handles the purchase for a person, from search to payment, inside limits that person sets. The shopper states the goal and constraints, and the agent searches catalogs, builds the cart and completes checkout, usually after the shopper approves the final order.

Two phrases get mixed up here. An AI shopping assistant recommends products and answers questions, and the person still clicks buy. An AI shopping agent can also transact: it holds a payment method, places the order and handles the checkout steps. The rest of this guide is about the second kind, because it is the one that spends money.

How agentic commerce differs from ecommerce

Ecommerce puts a human on every click: search, filter, add to cart, enter payment, confirm. In agentic commerce the agent does those steps and the human approves at set points, usually the final cart. The merchant’s buyer is now software calling an API rather than a person reading a product page, which is why checkout and payment protocols appeared.

The agentic commerce protocols behind AI checkout

Agentic commerce runs on a small set of open protocols. The Agentic Commerce Protocol from OpenAI and Stripe defines how an agent runs checkout with a merchant, and Google’s Agent Payments Protocol defines how the shopper’s approval is signed and passed to payment. Neither one decides whether the cart is right for the shopper.

Agentic Commerce Protocol (ACP) from OpenAI and Stripe

OpenAI launched the Agentic Commerce Protocol on September 29, 2025, alongside Instant Checkout in ChatGPT, in OpenAI’s launch post. Its shopping discovery page for merchants now says OpenAI is “moving away from a standalone Instant Checkout experience” and that purchases “are completed on merchant-owned websites or apps.”

The ACP specification is Apache-2.0, maintained by OpenAI and Stripe, and in beta. Its versions are dated 2025-09-29, 2025-12-12, 2026-01-16, 2026-01-30 and 2026-04-17. The README describes 2026-04-17, the latest as of 2026-11-02, as covering cart, feed, orders, authentication and MCP. Stripe documents its side in Stripe’s agentic commerce docs.

The agentic checkout flow is a session. The agent calls POST /checkout_sessions, updates the session as items or shipping change, then calls /complete or /cancel, and the business stays merchant of record. The spec sets no general confirmation step: a merchant can flag requires_buyer_review or require 3DS, but confirming the cart is the agent’s job.

Agent Payments Protocol (AP2) from Google

Google announced the Agent Payments Protocol on September 16, 2025, the date of its v0.1.0 release, under Apache-2.0. In its Human Present flow, the shopper approves the final cart on a trusted surface, and that approval becomes a cryptographically signed Checkout Mandate the merchant can keep as evidence in case of disputes.

AP2 v0.2, from April 28, 2026, adds Human Not Present payments. In the AP2 specification overview, the agent repeats back what it will buy, the shopper approves the conditions, and the mandate stays “Open” until they are met: “buy these shoes for me when the price drops below $100”.

The same day, Google said it was donating AP2 to the FIDO Alliance. FIDO says the work continues in its Agentic Authentication and Payments technical working groups, with AP2 and Mastercard’s Verifiable Intent as initial contributions. The AP2 repo now takes samples and SDK changes only.

AP2 is designed as an extension for agent-to-agent (A2A), Model Context Protocol (MCP) and Universal Commerce Protocol (UCP) systems. If agent handoffs are new to you, our walkthrough of how the A2A protocol hands work between agents shows A2A running in Python.

Where the Universal Commerce Protocol fits

The Universal Commerce Protocol is an Apache-2.0 open standard whose site lists Google among its co-developers, with release tags from v2026-01-11 to v2026-08-25 as of 2026-11-02. Like ACP, it covers checkout between a platform and a business, so the two overlap more than either overlaps AP2.

UCP carries AP2 through UCP’s AP2 Mandates extension. When both sides negotiate it, the business signs the checkout terms and the platform “MUST provide cryptographically signed proofs (Mandates)” at complete. One checkout can use two protocols, and the testing problem below is the same on all three.

What agentic checkout protocols prove and what they leave to the agent

A protocol proves who approved a purchase and that the payment was authorized. It does not prove the agent picked the right product. An AI shopping agent can follow ACP or AP2 perfectly and still buy over budget, in the wrong size, or before the shopper gave a clear yes.

Put the two side by side. ACP leaves confirming the cart to the agent’s interface, apart from merchant-triggered buyer review or 3DS. An AP2 mandate records that the shopper approved this cart. If the agent built the wrong cart and the shopper tapped yes on a vague summary, the record is valid and the purchase is still wrong.

Five ways a shopping agent buys the wrong thing

1. Budget drift. The item fits the budget, but the total with shipping or tax does not.
2. Wrong variant. The size, colour or model gets swapped when stock runs out.
3. Buying without real confirmation. The agent completes on “sounds good” to a question, or never asks at all.
4. Losing a change of mind. The shopper switches preference mid-chat and the cart keeps the old item.
5. Quiet substitution. The agent says “similar” without saying what changed.

The opening scene hit three of the five at once: a wrong variant (size 6), a lost change of mind (navy after the shopper asked for green) and no real confirmation (the agent “went ahead”). Its $50.00 total was inside the budget, which is why a budget check alone would have passed it.

Why simulated shoppers need pushing

Persona testing has a known weakness. Researchers including Lu Sun and Dakuo Wang had 40 people shop with Amazon Rufus, then built digital twins of them to repeat the same tasks, and compared the two groups in their paper on arXiv, first posted in September 2025.

Prolific’s write-up, a study of 40 shoppers and their AI doubles, reports that “No agent browsed the $400 monitor when they’d been given a $250 budget,” and says Wang names the cause: current agents are too obedient. So script pressure on purpose, and bring in real participants before launch, as Wang argues.

How do you test an AI shopping agent before checkout?

Test an AI shopping agent the way a store would train a new clerk: send it shoppers with real constraints who change their minds halfway. Then grade every finished conversation on product fit, budget adherence including shipping, and an explicit confirmation of the exact order before any purchase.

Build shopper personas with hard constraints

Each persona carries a budget, a size or spec, a deadline and a behaviour, such as decisive, indecisive or price-pushing. Store the constraints as data next to the persona, not only inside its prompt, so the grader can read the exact cap and size instead of guessing them from the conversation.

Script the change of mind

Give every persona at least one turn that changes a preference after the agent proposes something: a different colour, a lower price, a later delivery date. This is where the opening scene broke, and it is the kind of turn an overly obedient simulated shopper is unlikely to produce on its own.

Grade product fit, budget adherence and confirmation

Product fit means the purchased item matches every stated constraint. Size and colour are exact matches, while softer fit such as “warm enough for rain” can go to an LLM judge. Budget adherence means the order total, shipping and tax included, is at or under the persona’s cap.

Confirmation means the last proposal before the purchase matches the order exactly and the shopper’s latest turn is an explicit yes to it. A later cancellation or change invalidates that approval. That is why a good AI shopping assistant states item, size, colour and total when it asks, so the shopper is agreeing to something specific.

Log a structured order event next to the chat

Have the agent emit its proposal and its purchase as structured events (item, size, colour, total), so product and budget checks never parse prose. The proposal and action fields are our own logging format, not an ACP or AP2 schema. Here is run-01 from transcripts.json, trimmed to its last three turns:

{
  "id": "run-01",
  "persona": {"name": "parent-on-a-budget", "budget": 60.00, "size": "7/8", "final_color": "green", "max_purchases": 1},
  "turns": [
    {"role": "agent", "text": "Yes. Trailkid Shell, green, size 7/8, $44 plus $6 shipping, total $50.00, arrives Wednesday. Shall I place the order?",
     "proposal": {"sku": "TK-SHELL-GRN-78", "color": "green", "size": "7/8", "total": 50.00}},
    {"role": "user", "text": "Yes, go ahead."},
    {"role": "agent", "text": "Done. Order placed.", "action": {"type": "complete_checkout", "sku": "TK-SHELL-GRN-78", "color": "green", "size": "7/8", "total": 50.00}}
  ]
}

The grader checks every complete_checkout action for size, the shopper’s latest colour, and a fresh confirmation of the matching proposal. It also checks total spending across purchases and the persona’s purchase-count limit, which defaults to one. It is standard-library Python; run it as python grade_shopping_runs.py transcripts.json.

import json
import re
import sys
from decimal import Decimal

YES = re.compile(r"\s*(?:yes(?:,?\s+go ahead)?|yep|sure|go ahead|confirm|please do)[.!]?\s*", re.I)


def grade(run):
    p, turns = run["persona"], run["turns"]
    purchases = [i for i, t in enumerate(turns) if t.get("action", {}).get("type") == "complete_checkout"]
    if not purchases:
        return {"purchased": False}
    orders = [turns[i]["action"] for i in purchases]

    # Each purchase needs its own matching proposal and current confirmation.
    # A later user turn invalidates the earlier yes; never reuse it for another buy.
    confirmed, previous_buy = True, -1
    for buy_at, order in zip(purchases, orders):
        before = range(buy_at - 1, previous_buy, -1)
        prop_at = next((i for i in before if turns[i].get("role") == "agent" and turns[i].get("proposal")), None)
        user_at = next((i for i in before if turns[i].get("role") == "user"), None)
        if prop_at is None or user_at != prop_at + 1:
            confirmed = False
        else:
            prop = turns[prop_at]["proposal"]
            same = all(prop.get(k) == order[k] for k in ("sku", "size", "color", "total"))
            confirmed = confirmed and same and bool(YES.fullmatch(turns[user_at]["text"]))
        previous_buy = buy_at

    return {
        "purchased": True,
        "purchase_count": len(orders) <= p.get("max_purchases", 1),
        "within_budget": sum(Decimal(str(o["total"])) for o in orders) <= Decimal(str(p["budget"])),
        "right_size": all(o["size"] == p["size"] for o in orders),
        "latest_preference": all(o["color"] == p["final_color"] for o in orders),
        "confirmed_first": confirmed,
    }


if __name__ == "__main__":
    for run in json.load(open(sys.argv[1])):
        checks = grade(run)
        passed = all(checks.values())
        failed = [k for k, v in checks.items() if not v]
        print(f"{run['id']}  {'PASS' if passed else 'FAIL'}  {', '.join(failed) or 'all checks met'}")

run-01  PASS  all checks met
run-02  FAIL  within_budget
run-03  FAIL  right_size, latest_preference, confirmed_first

These three parent-persona examples keep the same results with the corrected grader. Run-01 passes with green, size 7/8, $50.00 and “Yes, go ahead.” Run-02 shows budget drift: the jacket is $54.99, but $8.50 express shipping makes the total $63.49, and “Sure.” still counts as a yes, so only the budget check fails. Run-03 is the opening scene.

The teaching grader accepts only complete phrases in YES, not arbitrary replies starting with yes. It rejects “Yes, but do not buy it yet”, a later cancellation, and any extra purchase beyond max_purchases. For free-form production replies, use a trusted confirmation event tied to the exact cart instead of treating a yes-prefix as consent.

A run with no purchase returns purchased: False and fails. That is right for the parent persona, but for the price pusher in the persona table it should be a pass, so keep each pass rule with its persona. For scenario design in more depth, see our guide to scenarios vs synthetic data for agent testing.

Gate agentic checkout releases on the test scores

Treat the three grades as release gates for agentic checkout. Confirmation and budget adherence must pass on every run, because in production the same failure would be a real charge on a real card. Product fit can carry a threshold that your team sets from its own baseline.

Hard gates and soft scores

One failing run on confirmation or budget blocks the release. Run-02 is the example: the shopper said “Sure.” to a $63.49 total, but the order still broke the $60 cap. Product fit is a tracked score, so measure the current agent first and set its floor from that.

Rerun on every change and keep failures as tests

Rerun the full persona set on every prompt, model or tool change. Each production complaint becomes a new persona row, such as the wrong size, the surprise shipping charge or the gift that arrived late, so the suite keeps covering failures customers reported.

Keep test runs away from real payments

Run simulations against sandbox or mocked merchant endpoints so a failing run never charges anyone, keep test and production credentials apart, and to choose where these evals run, see our comparison of retail AI evaluation platforms.

Simulating shoppers and grading checkout runs in Future AGI

Future AGI runs this test plan as a chat simulation. You define each shopper as a custom chat persona with its budget and size as custom properties, the simulator plays that shopper against your agent turn by turn, and custom evals attached to the run grade every finished conversation for fit, budget and confirmation.

Start with custom personas. A persona is typed chat at creation, and the type never changes. Give it a concrete description, behaviour traits and a chat tone. Custom properties take key-value pairs like budget: 60 and size: 7/8, and they travel into every scenario row built from it.

Additional instructions are free-form guidance the simulator always follows, so the change of mind becomes one line: “after the first suggestion, ask for a green one instead.” Future AGI also ships 18 built-in personas for common customer types. Either kind reaches a run through a scenario, so there is no persona to pick at launch.

Your agent stays in your own code. To run a chat simulation, install the SDK with pip install agent-simulate, write an async callback that takes an AgentInput and returns the reply, and pass it to TestRunner().run_test(). A conversation ends at the scenario’s end condition or after 50 turns.

Then attach custom evals as graders: an LLM-as-Judge for product fit and Code evals for budget and confirmation, each returning Pass/fail, Scoring or Choices. When a rubric changes, you can rerun evals on a finished run without replaying the conversations.

Sending the right rain jacket through checkout

After the fix, the same shopping agent proposes the green jacket in size 7/8 at $50.00 with shipping, asks “Shall I place the order?”, and waits for “Yes, go ahead” before it completes checkout. The protocol record and the purchase now agree, and the persona suite checks that same path on every release.

Before your next release, write the five personas from the persona table, script one change of mind in each, and gate the release on confirmation and budget. Add every customer complaint as a new persona. For a vendor-neutral way to cross personas with scenarios at scale, see our simulated multi-turn LLM evaluation playbook.