US perspectives: AI breaks the internet (and helps rebuild it)

Listen to this article

This audio is generated using AI

As with any investment, your capital is at risk.

Maya runs a small US travel site. Last year, her Plan a Weekend in… guides paid the bills. This year, she wakes to a strange pattern: website crawls are up, but click-throughs are down. AI answer engines skim her pages to construct a reply inside their users’ chat boxes, so fewer people ever reach her site. She still pays to create content and host the website. The audience just doesn’t arrive anymore.

Speaking to Matthew Prince, chief executive of Cloudflare, this summer provides context: Compared with the ‘Old Google’ search model, a site visit is about 750 times less likely using OpenAI and about 30,000 times less likely via Anthropic. The bargain of “we let search engines crawl, you send us readers” has cracked.

What’s breaking

For 25 years, the web ran on a simple swap: ‘crawl → click’. AI answer engines are rewiring it to ‘crawl → answer’, often with no visit at all. More and more of the internet is machine traffic. This is muddying measurement, ad spend and trust. Analytics are noisy, and revenue patterns are changing.

About 20 per cent of the web’s traffic goes through Cloudflare’s network. It is responding to this shift by blocking many AI crawlers by default and piloting a way for sites to charge per crawl so creators aren’t left footing the bill. That’s an early sketch of a new deal.

So what might the AI-era internet look like?

1. Pay what you use

If an AI system like OpenAI’s ChatGPT uses Maya’s work to craft an answer, it could pay for that use, just as a cafe pays for the electricity that keeps the lights on.

With Cloudflare’s pilot, when a bot asks for a page, Maya’s server can reply with “payment required” and a tiny price. If the bot pays, it gets the content. If not, it doesn’t. That keeps instant answers for users but restores a fairer exchange for creators.

Circle, the company behind the USDC stablecoin, and others have outlined a blockchain-based internet protocol, x402. The intent is for AI agents to pay these micro‑fees in USDC and other stablecoins automatically and get back to work in seconds. Cloudflare is partnering with Coinbase to encourage the adoption of this protocol. This model keeps the convenience of instant answers and adds a ‘meter’ so value flows back to the source.

This is a democratic vision. An alternative is that the AI answer platform pays creators a monthly sum based on their content’s engagement. Here, value accrues to the demand aggregators – the AI crawler – in a similar way to Google and search. Alternatively, platforms strike licensing deals (eg with Reddit and The New York Times). 

2. Provenance: show where it came from

AI-generated content is exploding. Therefore, origin increasingly matters. Maya can attach Content Credentials to her work. These tamper‑evident labels say who made a photo or article and how it was edited. 

The Coalition for Content Provenance and Authenticity (C2PA)’s open technical standard is being baked into infrastructure so those labels survive across the web. Providers now preserve Content Credentials through their image pipelines, helping original work stand out. Combining this with blockchain technology to record this metadata creates an immutable record.

Provenance also needs people, not just pixels. Privacy-preserving tools like Private Access Tokens (used in Apple’s ecosystem) and Cloudflare’s Turnstile help services tell real visitors from bot swarms without intrusive tracking. 

When Maya turns these on, her audience quality improves, and advertisers on her site have more confidence in what they’re paying for.

3. Accountability: hold AI agents to a standard

As AI agents learn to browse, summarise and act, large enterprises are likely to insist on guardrails. That means saying what an agent is allowed to fetch and why – recording what it actually did – and enforcing consequences when it ignores rules, including payment rules. For Maya, who hosts her site on Amazon Web Services’ cloud, this turns ‘black boxes’ into ‘glass boxes’ that are inspectable and correctable. AI answer engines that play by these rules will be easier to trust and do business with.

The future: agentic commerce?

This all leads to agentic commerce, which could accelerate Maya’s business. Instead of waiting for people to click her website, Maya lets AI helpers find and buy her trips directly. A shopper tells their AI: “Plan a 10‑day Croatia trip for two under $4,000.” The AI then pulls up Maya’s pre-made package, asks her bot to tweak dates or hotels, and then pays, all inside the AI chat box.

Maya runs her business on Shopify’s ecommerce platform, and its software makes Maya’s trips easy for AIs to read and buy, allowing her to set simple rules. For example, what can be swapped, how much can be discounted, etc. For payment, the AI can again use stablecoins like USDC to split money instantly between Maya and Croatian suppliers, cutting card fees and currency hassles.

Potential losers in this vision are free-search-dependent sites – is this the end of the ‘listicle’? – non-paying AI aggregators and bot-ridden social platforms. Advertising may also keep shifting towards trusted walled gardens and away from the open internet.

Meanwhile, datacentre and power suppliers benefit regardless of who wins, and emerging compute marketplaces could ease capacity and cost pressures.

Back to Maya

In the future, Maya opens her dashboard. Crawls are up, but now a meter ticks. Answers pay a sliver, credentials travel with her assets, and agentic commerce converts guides to bookings.

Perhaps this is how AI breaks and rebuilds the internet.

Risk factors 

The views expressed should not be considered as advice or a recommendation to buy, sell or hold a particular investment. They reflect opinion and should not be taken as statements of fact nor should any reliance be placed on them when making investment decisions.

This communication was produced and approved in October 2025 and has not been updated subsequently. It represents views held at the time of writing and may not reflect current thinking.

This communication contains information on investments which does not constitute independent research. Accordingly, it is not subject to the protections afforded to independent research, but is classified as advertising under Art 68 of the Financial Services Act (‘FinSA’) and Baillie Gifford and its staff may have dealt in the investments concerned.

All information is sourced from Baillie Gifford & Co and is current unless otherwise stated.

The images used in this communication are for illustrative purposes only.

Important information 

Baillie Gifford & Co and Baillie Gifford & Co Limited are authorised and regulated by the Financial Conduct Authority (FCA). Baillie Gifford & Co Limited is an Authorised Corporate Director of OEICs.

Baillie Gifford Overseas Limited provides investment management and advisory services to non-UK Professional/Institutional clients only. Baillie Gifford Overseas Limited is wholly owned by Baillie Gifford & Co. Baillie Gifford & Co and Baillie Gifford Overseas Limited are authorised and regulated by the FCA in the UK.

Persons resident or domiciled outside the UK should consult with their professional advisers as to whether they require any governmental or other consents in order to enable them to invest, and with their tax advisers for advice relevant to their own particular circumstances.

Financial Intermediaries

This communication is suitable for use of financial intermediaries. Financial intermediaries are solely responsible for any further distribution and Baillie Gifford takes no responsibility for the reliance on this document by any other person who did not receive this document directly from Baillie Gifford.

Europe

Baillie Gifford Investment Management (Europe) Ltd (BGE) is authorised by the Central Bank of Ireland as an AIFM under the AIFM Regulations and as a UCITS management company under the UCITS Regulation. BGE also has regulatory permissions to perform Individual Portfolio Management activities. BGE provides investment management and advisory services to European (excluding UK) segregated clients. BGE has been appointed as UCITS management company to the following UCITS umbrella company; Baillie Gifford Worldwide Funds plc. BGE is a wholly owned subsidiary of Baillie Gifford Overseas Limited, which is wholly owned by Baillie Gifford & Co. Baillie Gifford Overseas Limited and Baillie Gifford & Co are authorised and regulated in the UK by the Financial Conduct Authority.

China

Baillie Gifford Investment Management (Shanghai) Limited 柏基投资管理(上海)有限公司(‘BGIMS’) is wholly owned by Baillie Gifford Overseas Limited and may provide investment research to the Baillie Gifford Group pursuant to applicable laws. BGIMS is incorporated in Shanghai in the People’s Republic of China (‘PRC’) as a wholly foreign-owned limited liability company with a unified social credit code of 91310000MA1FL6KQ30. BGIMS is a registered Private Fund Manager with the Asset Management Association of China (‘AMAC’) and manages private security investment fund in the PRC, with a registration code of P1071226.

Baillie Gifford Overseas Investment Fund Management (Shanghai) Limited

柏基海外投资基金管理(上海)有限公司(‘BGQS’) is a wholly owned subsidiary of BGIMS incorporated in Shanghai as a limited liability company with its unified social credit code of 91310000MA1FL7JFXQ. BGQS is a registered Private Fund Manager with AMAC with a registration code of P1071708. BGQS has been approved by Shanghai Municipal Financial Regulatory Bureau for the Qualified Domestic Limited Partners (QDLP) Pilot Program, under which it may raise funds from PRC investors for making overseas investments.

Hong Kong

Baillie Gifford Asia (Hong Kong) Limited 柏基亞洲(香港)有限公司 is wholly owned by Baillie Gifford Overseas Limited and holds a Type 1 license from the Securities & Futures Commission of Hong Kong to market and distribute Baillie Gifford’s range of collective investment schemes to professional investors in Hong Kong. Baillie Gifford Asia (Hong Kong) Limited 柏基亞洲(香港)有限公司 can be contacted at Suites 2713-2715, Two International Finance Centre, 8 Finance Street, Central, Hong Kong. Telephone +852 3756 5700.

South Korea

Baillie Gifford Overseas Limited is licensed with the Financial Services Commission in South Korea as a cross border Discretionary Investment Manager and Non-discretionary Investment Adviser.

Japan

Mitsubishi UFJ Baillie Gifford Asset Management Limited (‘MUBGAM’) is a joint venture company between Mitsubishi UFJ Trust & Banking Corporation and Baillie Gifford Overseas Limited. MUBGAM is authorised and regulated by the Financial Conduct Authority.

Australia

Baillie Gifford Overseas Limited (ARBN 118 567 178) is registered as a foreign company under the Corporations Act 2001 (Cth) and holds Foreign Australian Financial Services Licence No 528911. This material is provided to you on the basis that you are a “wholesale client” within the meaning of section 761G of the Corporations Act 2001 (Cth) (“Corporations Act”). Please advise Baillie Gifford Overseas Limited immediately if you are not a wholesale client. In no circumstances may this material be made available to a “retail client” within the meaning of section 761G of the Corporations Act.

This material contains general information only. It does not take into account any person’s objectives, financial situation or needs.

South Africa

Baillie Gifford Overseas Limited is registered as a Foreign Financial Services Provider with the Financial Sector Conduct Authority in South Africa.

North America

Baillie Gifford International LLC is wholly owned by Baillie Gifford Overseas Limited; it was formed in Delaware in 2005 and is registered with the SEC. It is the legal entity through which Baillie Gifford Overseas Limited provides client service and marketing functions in North America. Baillie Gifford Overseas Limited is registered with the SEC in the United States of America.

The Manager is not resident in Canada, its head office and principal place of business is in Edinburgh, Scotland. Baillie Gifford Overseas Limited is regulated in Canada as a portfolio manager and exempt market dealer with the Ontario Securities Commission ('OSC'). Its portfolio manager licence is currently passported into Alberta, Quebec, Saskatchewan, Manitoba and Newfoundland & Labrador whereas the exempt market dealer licence is passported across all Canadian provinces and territories. Baillie Gifford International LLC is regulated by the OSC as an exempt market and its licence is passported across all Canadian provinces and territories. Baillie Gifford Investment Management (Europe) Limited (‘BGE’) relies on the International Investment Fund Manager Exemption in the provinces of Ontario and Quebec.

Israel

Baillie Gifford Overseas Limited is not licensed under Israel’s Regulation of Investment Advising, Investment Marketing and Portfolio Management Law, 5755-1995 (the Advice Law) and does not carry insurance pursuant to the Advice Law. This material is only intended for those categories of Israeli residents who are qualified clients listed on the First Addendum to the Advice Law.

Singapore

Baillie Gifford Asia (Singapore) Private Limited is wholly owned by Baillie Gifford Overseas Limited and is regulated by the Monetary Authority of Singapore as a holder of a capital markets services licence to conduct fund management activities for institutional investors and accredited investors in Singapore. Baillie Gifford Overseas Limited, as a foreign related corporation of Baillie Gifford Asia (Singapore) Private Limited, has entered into a cross-border business arrangement with Baillie Gifford Asia (Singapore) Private Limited, and shall be relying upon the exemption under regulation 4 of the Securities and Futures (Exemption for Cross-Border Arrangements) (Foreign Related Corporations) Regulations 2021 which enables both Baillie Gifford Overseas Limited and Baillie Gifford Asia (Singapore) Private Limited to market the full range of segregated mandate services to institutional investors and accredited investors in Singapore.

170821 10057469