Visa launched the Visa Agentic Commerce Benchmark on Nov. 5 to test AI models in realistic shopping scenarios. The public can take the test as well.
The first round put 22 models from 12 labs through 15 challenges, including choosing a TV, laptop or personal care appliance. Models had to identify a shopper’s needs through conversation and choose not to buy if no product met the requirements.
The models achieved an overall pass rate of 57.2%. The top performers passed 82.2% of attempts, while the lowest-scoring models passed 28.9%. The five models that uncovered the most hidden customer requirements had an average pass rate of 70.2%.
