AI Commerce Watch glossary
AI crawler
A bot operated by an AI company or answer engine that reads web pages to collect training data or ground live answers. Distinct from search crawlers in purpose and behavior.
GPTBot
OpenAI's declared web crawler for training data collection. Controllable via robots.txt, though compliance is voluntary.
Answer engine
An AI product that responds to questions with synthesized answers and citations - a growing product-discovery channel for ecommerce.
Agentic commerce
Purchases researched and executed by AI agents acting on a user's behalf, compressing the shopping journey into machine-speed sessions.
robots.txt
A sitewide text file stating which crawlers may access which paths. A statement of policy, not an enforcement mechanism - hostile or indifferent bots ignore it.
Edge enforcement
Applying crawler and bot policy (allow, throttle, block) at the network edge in front of the storefront, where decisions execute before requests consume origin resources.