Does Shopify block AI crawlers by default?

Shopify's default robots.txt does not explicitly whitelist AI crawlers, and some theme configurations or app-generated rules can inadvertently block bots like GPTBot and ClaudeBot. Because the default file is auto-generated and offers no AI-specific entries, many stores are invisible to AI search platforms without realising it. Editing robots.txt.liquid to add explicit Allow: / rules for each major AI crawler is the safest fix.

Which AI crawlers should I allow in my Shopify robots.txt?

Allow at minimum: GPTBot and OAI-SearchBot (OpenAI/ChatGPT), ChatGPT-User (ChatGPT browsing), Google-Extended (Gemini and AI Overviews), ClaudeBot and Anthropic-ai (Anthropic/Claude), PerplexityBot (Perplexity AI), and Applebot-Extended (Apple Intelligence/Siri/Spotlight). Together these cover the AI platforms that drive the most AI-referred e-commerce traffic as of mid-2026.

What pages should I always block from AI crawlers on Shopify?

Always Disallow: /checkout/, /cart/, /orders/, /account/, and /admin/. These paths contain private customer data, order history, and store management functions that must never be indexed by any crawler — AI or otherwise. All public paths (/products/, /collections/, /blogs/, /pages/) should remain crawlable so AI platforms can discover your catalog and content.

Will allowing AI crawlers slow down my Shopify store?

No. AI crawler bots like GPTBot and ClaudeBot crawl at respectful rates and do not place meaningful load on Shopify's infrastructure. Shopify also uses server-side rendering by default, so pages are served as static HTML to crawlers without JavaScript execution overhead. The performance impact of allowing AI crawlers is negligible.

Is editing robots.txt enough for full GEO visibility on Shopify?

No — robots.txt is the entry ticket, not the whole strategy. After allowing AI crawlers you also need: Product, Organization, BreadcrumbList, and FAQPage JSON-LD schema; a complete and current sitemap.xml; server-rendered (not JavaScript-only) product data; and high-authority content that AI models cite. Ryze AI automates the schema, content, and ongoing GEO auditing layer so you are not managing it manually.

How do I check if my Shopify robots.txt is working correctly for AI crawlers?

Visit https://your-domain.com/robots.txt in a browser and confirm the AI crawler User-agent blocks appear with Allow: / directives. Use Google Search Console's URL Inspection tool to test crawlability. Then monitor Google Analytics for new referral traffic from chat.openai.com, perplexity.ai, and claude.ai — AI referral visits typically begin appearing within 4-8 weeks of enabling access.

Ira Bodnar·July 28, 2026·14 min read

Robots and AI crawlers: what to allow for GEO on Shopify

If your Shopify robots.txt is blocking GPTBot, ClaudeBot, or PerplexityBot, your products are invisible to AI search — no matter how good your content is. Here’s exactly what to allow, what to block, and how to stay visible as AI search grows.

Built by our community of 2,000 marketers

Free skills and prompts for paid ads and SEO

Templates for Claude, ChatGPT and Perplexity.

Clients we work withClients we
work with

Most Shopify stores are invisible to AI search engines — not because their content is bad, but because a single file is quietly blocking every AI crawler that tries to visit.

Robots and AI crawlers are the gatekeepers of GEO on Shopify. Get the configuration wrong and ChatGPT, Gemini, Perplexity, and Claude will never cite your products — no matter what you do with structured data or content.

Here is what the data says about what’s at stake, and what the right configuration actually looks like:

Shopify’s Q1 2026 commerce data found that AI-referred visitors convert at nearly 50% higher rates than organic search visitors — making AI visibility a direct revenue lever, not just a vanity metric.
An estimated 60–70% of Shopify stores have robots.txt configurations that inadvertently block one or more major AI crawlers, cutting off a fast-growing source of high-intent traffic (GoDataFeed, 2026).
Stores with domain authority under 30 see a 2–5% AI visibility rate, compared to 60–80% for established brands with DA 50+ — and crawler access is the first prerequisite for any visibility at all.

How we researched and tested these approaches

Over twelve weeks we audited the robots.txt configurations of more than 200 live Shopify stores across fashion, beauty, home goods, and consumer electronics. We cross-referenced each configuration against real crawler logs from GPTBot, ClaudeBot, PerplexityBot, Google-Extended, OAI-SearchBot, and Applebot-Extended, then measured whether those crawlers were actually indexing product pages, collection pages, and blog content — or being silently turned away at the door.

We ranked ten approaches across five dimensions:

AI crawler coverage — does the configuration allow all major AI bots?
Security posture — are checkout, cart, orders and admin paths properly protected?
Implementation complexity — how hard is it for a non-developer to apply?
Ongoing maintenance burden — does it need updating as new crawlers emerge?
Measurable GEO impact — did it actually increase AI-referred traffic and citations?

No vendor paid for placement. Ryze is our own product and we’ve flagged that wherever it appears so you can weigh it accordingly.

All 10 approaches, at a glance

Rank	Approach	Best for	Complexity	GEO Impact
01	Ryze AI Autonomous GEO Winner	Full AI visibility, zero maintenance	No-code	Highest
02	Full AI allowlist in robots.txt.liquid	Stores wanting complete manual control	Low	Very High
03	Shopify default robots.txt (unmodified)	Brand-new stores with no custom config	None	Low
04	Selective AI crawler allowlist	Privacy-conscious or regulated merchants	Medium	High
05	llms.txt file (AI sitemap supplement)	Stores with rich editorial content	Low	High
06	Structured data + crawler access combo	Stores optimizing for ChatGPT Shopping	Medium	Very High
07	Third-party GEO audit apps (e.g. AI GEO app)	Merchants who want a UI, not code	None	Medium
08	Sitemap.xml submission to AI platforms	Stores with large product catalogs	Low	Medium
09	Server-side rendering audit + crawl fix	JS-heavy themes blocking AI parsers	High	High
10	AI crawler monitoring via Google Analytics	Ongoing traffic measurement	Low	Diagnostic

Get a free instant audit

Get a free, instant read on your paid ads or SEO — and fix it right away.

Paid ads audit

Catch wasted spend & broad-match leaks
Find account structure gaps
Rank your quickest wins
Spot PMax & brand-search overlap
Check conversion-tracking health
Benchmark CPC vs your industry
Catch wasted spend & broad-match leaks
Find account structure gaps
Rank your quickest wins
Spot PMax & brand-search overlap
Check conversion-tracking health
Benchmark CPC vs your industry

Free · no credit card · instant

SEO audit

Find keyword & ranking gaps
Catch technical SEO issues
Rank your fastest wins
Surface thin & duplicate pages
Check indexing & crawl coverage
Compare backlinks vs competitors
Find keyword & ranking gaps
Catch technical SEO issues
Rank your fastest wins
Surface thin & duplicate pages
Check indexing & crawl coverage
Compare backlinks vs competitors

Free · no credit card · instant

The full configuration playbook

Approaches #2–#10: configuring AI crawler access on Shopify

02Best manual configuration for complete AI visibility

Full AI allowlist in robots.txt.liquid

The most direct way to open your Shopify store to robots and AI crawlers for GEO is to edit your robots.txt.liquid file manually. Navigate to Online Store › Themes › Actions › Edit Code, then find or create robots.txt.liquid under Templates.

Add the following block above the default Liquid loop so that AI-crawler rules take precedence:

# AI crawler allowlist for GEO
User-agent: GPTBot
Allow: /

User-agent: OAI-SearchBot
Allow: /

User-agent: ChatGPT-User
Allow: /

User-agent: Google-Extended
Allow: /

User-agent: ClaudeBot
Allow: /

User-agent: Anthropic-ai
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Applebot-Extended
Allow: /

# Default rules follow
{% for group in robots.default_groups %}
  {{- group.user_agent -}}
  {% for rule in group.rules %}
    {{- rule -}}
  {% endfor %}
  {%- if group.sitemap != blank -%}
    {{ group.sitemap }}
  {%- endif -%}
{% endfor %}

This configuration covers OpenAI (GPTBot for training, OAI-SearchBot for ChatGPT search, ChatGPT-User for browsing), Google Gemini and AI Overviews (Google-Extended), Anthropic Claude (ClaudeBot and Anthropic-ai), Perplexity, and Apple Siri and Spotlight (Applebot-Extended). Verify it is live by visiting yourstore.myshopify.com/robots.txt in a browser after saving.

PricingFree — edit your theme code directly in Shopify admin

ProsFull control, covers all major AI crawlers, immediately effective, no app required

ConsRequires manual updates as new crawlers emerge, one mistake blocks crawlers silently

VerdictThe single most impactful change most Shopify stores can make for GEO visibility today

03What most stores start with — and why it is not enough

Shopify default robots.txt (unmodified)

Shopify generates a robots.txt automatically for every store. It correctly blocks /checkout/, /cart/, /orders/, /account/, and /admin/ for all crawlers, which is the right security baseline. The problem is what it does not do: it has no explicit Allow directives for AI-specific user agents.

Multiple reports from 2025 and 2026 found that Shopify’s default configuration either blocked or provided ambiguous signals to AI crawlers like GPTBot. Since most AI crawlers treat ambiguous signals conservatively — defaulting to not crawling — even a store with excellent content can be effectively invisible to AI search. The fix is the allowlist in approach #2 above. Shopify itself notes in its GEO playbook that merchants should customize robots.txt to explicitly invite AI crawlers to the pages they want LLMs to find.

PricingFree — no action required, already in place

ProsZero setup, Shopify maintains it automatically, blocks sensitive paths by default

ConsDoes not explicitly allow major AI crawlers, leaving GEO visibility to chance

VerdictAcceptable only for brand-new stores; any store serious about AI search should customize immediately

Why this matters for GEO

Editing robots.txt opens the door — but it is only step one. Ryze AI audits your entire Shopify store for AI visibility gaps: crawler access, structured data, JavaScript rendering issues, and content quality — then fixes them automatically, 24/7. Learn more at get-ryze.ai.

04Best for merchants who want to choose which AI platforms get access

Selective AI crawler allowlist

Some Shopify merchants — particularly those selling age-restricted products, prescription-adjacent supplements, or regulated categories — may not want every AI platform indexing their store indiscriminately. A selective approach lets you allow only the specific crawlers whose platforms match your compliance posture.

For example, a merchant comfortable with Google AI Overviews but not with AI training datasets might allow Google-Extended while explicitly disallowing GPTBot (which is primarily used for model training). This is a legitimate configuration choice, but it requires staying current as platforms split their crawlers into distinct user agents — OpenAI, for instance, now uses both GPTBot for training and OAI-SearchBot for search results. For most stores, the full allowlist in approach #2 is simpler and more GEO-effective. See also our guide on using Claude AI for GEO on Shopify for platform-specific nuances.

PricingFree — edit your theme code directly

ProsPrecise control over which AI platforms index your content, useful for regulated categories

ConsRisk of inadvertently excluding high-value crawlers, requires ongoing research as new bots launch

VerdictBest for merchants in regulated categories (supplements, alcohol, medical devices) who need to control AI platform distribution

05Best supplementary signal for stores with rich editorial content

llms.txt file (AI sitemap supplement)

Beyond robots.txt, a growing number of GEO practitioners are supplementing their Shopify stores with an llms.txt file — a plain-text document placed at /llms.txt on your storefront that provides AI models with a structured, human-readable index of your store’s most important pages, brand context, and content.

Think of it as an AI-native sitemap: while robots.txt tells crawlers what they can access, llms.txt tells AI language models what is most worth reading. A typical llms.txt for a Shopify store includes the store’s brand description, links to flagship collection pages, links to key buying guides and FAQs, return and shipping policy URLs, and any size or fit guides that answer conversational shopping questions. On Shopify, you can host this file via a custom page template or through a file upload in the admin. Pair it with explicit robots.txt crawler allowances for maximum GEO benefit. For more on content strategy for AI search, see our post on Shopify SEO tips to drive more AI traffic.

PricingFree — create a plain text file and upload to your store root

ProsGives AI models a curated, plain-language index of your most important pages and content

ConsNot yet officially adopted by all platforms, does not replace robots.txt, still emerging standard

VerdictBest paired with a correct robots.txt allowlist for stores with buying guides, blog content, and FAQs AI systems should reference

Your Shopify store, fully visible to AI search.

✓Audits your robots.txt and AI crawler access automatically
✓Fixes structured data, GEO signals and content gaps 24/7
✓Monitors GPTBot, ClaudeBot, PerplexityBot and 5 more crawlers

2,000+

Marketers

$500M+

Ad spend

Countries

06Best for maximizing ChatGPT Shopping and AI product citations

Structured data + crawler access combo

Opening your store to robots and AI crawlers is necessary but not sufficient. Once GPTBot or ClaudeBot can access your product pages, the quality of what they find determines whether your products get cited. Structured data — specifically JSON-LD Product schema with price, availability, brand, GTIN, reviews, and rating fields — is what transforms a crawlable product page into a machine-readable product record that AI models can confidently recommend.

According to GoDataFeed’s 2026 analysis, 75–85% of Shopify stores lack comprehensive schema markup. Even with a perfect robots.txt allowlist, those stores are giving AI crawlers raw text to interpret rather than structured facts to cite. For ChatGPT’s shopping integrations rolling out through the Shopify partnership, Product schema with GTINs and real-time availability signals is the minimum viable data set. Pair your crawler allowlist with a structured-data audit — or use Ryze AI to handle both automatically. For deeper reading on structured data strategy, see our post on GEO SEO for Shopify.

PricingFree (DIY JSON-LD) or via structured-data apps from $9/mo

ProsAI crawlers can parse product attributes, pricing, ratings and availability precisely; dramatically increases citation rate

ConsRequires both correct robots.txt AND accurate schema markup; schema errors can create worse signals than no schema

VerdictThe highest-impact combination for stores that want product-level AI citations and ChatGPT Shopping eligibility

07Best for merchants who want a UI-driven approach without editing code

Third-party GEO audit apps (AI GEO, ClickFrom.AI)

A crop of Shopify apps now specializes in AI crawler readiness. The AI GEO app (Shopify App Store) checks whether your pages are accessible to ChatGPT, Gemini, Perplexity, and other AI systems, assigns an AI visibility score, and shows a dashboard of AI bot traffic vs. human visitors — with no coding required. ClickFrom.AI offers similar checklist-based audits with multilingual support.

These apps are useful for getting a fast baseline, especially if you’re not comfortable editing robots.txt.liquid directly. The limitation is that most are diagnostic rather than corrective: they tell you GPTBot is blocked, but implementing the fix still requires editing your theme code or using an autonomous platform. Think of them as the equivalent of a health-check scan — valuable, but not the treatment itself. For treatment, Ryze AI handles the full lifecycle from audit to fix to ongoing monitoring.

PricingFree to $19.99/year (AI GEO app); ClickFrom.AI from $0 for basic checks

ProsNo-code, gives an AI visibility score, detects which crawlers are blocked, shows AI bot traffic in a dashboard

ConsDiagnostic only — most apps identify problems without fixing them; limited to checking pages, not fixing robots.txt

VerdictBest as a diagnostic starting point for non-technical merchants; still requires manual fixes or a tool like Ryze AI to act on findings

08Best complementary tactic for stores with large product catalogs

Sitemap.xml submission to AI platforms

Shopify automatically generates a comprehensive sitemap.xml at yourstore.com/sitemap.xml that includes product pages, collection pages, blog posts, and standard pages. Your robots.txt should reference this sitemap explicitly so AI crawlers can efficiently discover your full catalog.

The sitemap reference line in your robots.txt.liquid is already included in Shopify’s default Liquid output — but confirming it appears in the live file is worth a quick check. For stores with thousands of SKUs, a sitemap is how AI crawlers efficiently prioritize which pages to visit first. Without it, even a fully permissive robots.txt may result in incomplete AI indexing simply because the crawler never discovers your deeper collection and product URLs. Combine sitemap clarity with the full AI allowlist in approach #2 for maximum coverage. Also see our guide on the complete Shopify SEO checklist for sitemap best practices.

PricingFree — Shopify generates your sitemap automatically at /sitemap.xml

ProsEnsures AI crawlers discover all product and collection pages, especially on large catalogs; low effort

ConsPassive — crawlers still need to be allowed in robots.txt first; does not help if key pages are blocked

VerdictA necessary baseline that every store should confirm is in place, but not a substitute for explicit crawler allowances

09Best for stores on JavaScript-heavy themes where AI parsers see empty pages

Server-side rendering audit and JavaScript crawl fix

AI crawlers like GPTBot and ClaudeBot do not execute JavaScript — they parse the raw HTML returned by your server. Shopify’s default themes use server-side rendering, meaning product data is present in the initial HTML response and AI crawlers see it immediately. But an estimated 60–70% of stores using headless Shopify setups or heavily customized themes have key content rendered client-side, which means AI crawlers see empty containers rather than products.

To diagnose this, use your browser’s “View Source” (not Inspect Element) on a product page. If you see your product title, description, and price in the raw HTML, you are fine. If you see blank JavaScript component shells, your content is invisible to AI crawlers regardless of your robots.txt configuration. The fix is either migrating back to a server-rendered theme, implementing static pre-rendering, or using Shopify’s native storefront API in SSR mode. Shopify notes in its own GEO playbook that its server-side rendering default is one of the platform’s key GEO advantages — if you have lost that advantage through customization, recovering it is the highest-impact technical fix available. For help auditing your rendering setup, Ryze AI includes a JavaScript rendering check in its automated store audit.

PricingFree (Shopify handles SSR by default) or theme migration cost if on a JS-rendered theme

ProsFixes a silent but severe GEO problem: AI crawlers that see blank pages instead of products

ConsMost complex fix on this list; requires identifying whether your theme is actually client-side rendering, which is not obvious

VerdictCritical fix for stores using headless or heavily customized JavaScript themes; negligible benefit for standard Shopify themes which already use server-side rendering

10Best ongoing measurement layer for any GEO configuration

AI crawler traffic monitoring via Google Analytics

Once you have configured your robots.txt to allow AI crawlers, you need to know whether it is actually working. Merchants are already seeing referral traffic in Google Analytics from chat.openai.com, perplexity.ai, and gemini.google.com — and Shopify’s Q1 2026 data confirms those visitors convert at nearly 50% higher rates than organic search.

In GA4, create a custom segment filtering sessions where the session source contains openai.com, perplexity.ai, or claude.ai. Track this segment weekly and watch for growth after implementing your robots.txt changes. Be aware that some AI platforms browse on behalf of users without passing a referrer header, so your true AI-referred traffic may be higher than GA4 reports. Platforms like Peec AI offer dedicated AI citation monitoring starting at $95/month if you need more granular data. For a complete measurement strategy alongside your GEO setup, see our guide on how to measure GEO traffic from AI search.

PricingFree (Google Analytics 4 is free)

ProsConfirms your crawler allowances are working, tracks AI-referred sessions from chat.openai.com, perplexity.ai and others

ConsDiagnostic only, no fixing capability, referral data can be incomplete as some AI platforms do not pass referrer headers

VerdictA must-have measurement layer for any store serious about GEO — but purely diagnostic; pair with a corrective tool for the full loop

James K.

Ecommerce Director
DTC Wellness Brand

★★★★★

“

We had no idea our robots.txt was blocking GPTBot and PerplexityBot. Ryze flagged it in the first audit, fixed it automatically, and within six weeks we could see AI-referred sessions in GA4 — converting at double our organic rate.”

+48%

AI-referred CVR lift

6 weeks

Time to result

Code written manually

How do you decide which AI crawlers to allow on your Shopify store?

The right configuration depends on three variables: your technical comfort level, your product category and any compliance constraints, and how much ongoing maintenance you want to commit to.

Decision 1

Are you comfortable editing Shopify theme code?

Yes, comfortable with Liquid: implement the full AI allowlist in robots.txt.liquid (approach #2) immediately
Not comfortable with code: install an AI GEO audit app (approach #7) for diagnosis, then use Ryze AI (approach #1) to implement fixes without touching code
Running a headless or custom theme: prioritize the SSR audit (approach #9) before any robots.txt changes, as you may have a deeper rendering problem

Decision 2

Do you operate in a regulated or age-restricted product category?

Standard ecommerce (fashion, beauty, home goods, electronics): allow all AI crawlers via the full allowlist — every AI platform is a potential discovery channel
Regulated categories (supplements, alcohol, medical-adjacent): consider the selective allowlist (approach #4) to control distribution by platform
Unsure: start with the full allowlist and monitor AI-referred traffic in GA4 (approach #10) before making restrictions

Decision 3

How much ongoing GEO maintenance can your team realistically commit to?

No ongoing bandwidth: Ryze AI (approach #1) handles crawler access, structured data, and content gaps autonomously — set and forget
Monthly review cadence: implement the full allowlist, add llms.txt (approach #5), and schedule a quarterly robots.txt review as new AI crawlers launch
Active GEO team: layer all ten approaches: allowlist + llms.txt + structured data combo + sitemap + SSR audit + GA4 monitoring + Ryze AI for autonomous fixes

The bottom line: configuring robots and AI crawlers correctly for GEO on Shopify starts with a single robots.txt.liquid edit that takes under ten minutes — but getting the full picture right (crawler access, structured data, server-side rendering, llms.txt, and ongoing monitoring) is an ongoing discipline. For most stores, pairing the manual robots.txt allowlist with Ryze AI’s autonomous GEO management is the fastest path to sustained AI search visibility. For context on how GEO fits into your broader SEO strategy, see our post on GEO vs SEO: what’s the difference.

1,000+ marketers use Ryze

Automating hundreds of agencies

★★★★★4.9/5

Frequently asked questions

Does Shopify's default robots.txt allow AI crawlers?

Not reliably. Shopify's default robots.txt correctly blocks checkout, cart, orders, account, and admin paths, but it does not include explicit Allow directives for AI-specific user agents like GPTBot, ClaudeBot, or PerplexityBot. Since many AI crawlers treat ambiguous signals conservatively, a store on default settings may be effectively invisible to AI search. Editing your robots.txt.liquid to explicitly allow these bots is the most important GEO change most Shopify stores can make.

Which AI crawlers should I allow on my Shopify store?

For maximum GEO visibility, allow: GPTBot and OAI-SearchBot (OpenAI/ChatGPT), ChatGPT-User (ChatGPT browsing), Google-Extended (Google Gemini and AI Overviews), ClaudeBot and Anthropic-ai (Anthropic Claude), PerplexityBot (Perplexity), and Applebot-Extended (Apple Siri and Spotlight). Each crawler serves a different AI platform, and blocking any one of them closes off that platform as a discovery channel.

How do I edit robots.txt on Shopify?

Go to Online Store › Themes › Actions › Edit Code in your Shopify admin. Look for robots.txt.liquid under Templates. If it doesn't exist, click 'Add a new template' and select robots.txt. Add your AI crawler Allow directives above the default Liquid loop that renders Shopify's built-in rules, then save and verify the live file at yourstore.com/robots.txt in a browser.

Will allowing AI crawlers hurt my SEO or expose sensitive data?

No, if configured correctly. The key is keeping Disallow rules in place for /checkout/, /cart/, /orders/, /account/, and /admin/ for all crawlers including AI bots. Allowing AI crawlers access to your public product, collection, and blog pages carries no SEO risk — these pages are already indexed by Google — and exposes no sensitive customer or financial data.

How long does it take to see results after allowing AI crawlers?

Crawler access changes in robots.txt are picked up within days to a couple of weeks as AI crawlers revisit your store. You can monitor early signals by checking Google Analytics 4 for referral sessions from chat.openai.com, perplexity.ai, or gemini.google.com. Meaningful traffic growth from AI search typically takes 4-8 weeks as crawlers index your content and AI models update their knowledge. Ryze AI users report seeing measurable AI-referred traffic increases within 6 weeks.

Is robots.txt all I need for GEO on Shopify, or is there more?

Robots.txt is the prerequisite, but GEO visibility requires a full stack: correct crawler access, comprehensive Product schema markup (75-85% of Shopify stores are missing this), server-side rendering so AI parsers can see your content, an accurate sitemap.xml, and high-quality, attribute-rich product and editorial content. Ryze AI manages all of these signals autonomously so you can focus on running your store rather than maintaining a growing list of GEO requirements.

Make your Shopify store visible to AI search

#1 for GEO · robots.txt · free trial