The AI Search Rabbit Hole: What I Know & If It Matters to You

Hello Reader,

One of the issues I've been monitoring and keeping an ear to the ground on is the impact of Google's AI summaries at the top of the search engine results pages (aka SERPs), as well as searches in chatbot Large Language Models in general (think ChatGPT etc). This is what those AI generated search results look like.

Now, before I get more into this, what surprised me when I was trying to find a screenshot to include was that I actually had to do several searches to get an AI summary. I searched "what's the best trick to teach your dog," "how to teach my dog tricks," "what's the cutest dog trick," and a couple others I can't recall before this simple search pulled an AI summary up.

No one really knows for sure, but between 20 percent and 74 percent of different types of searches receive these bot-generated summaries, which in some cases replace featured snippets in the results.

This is heavily topic and query (aka search) format dependent. Generally, "what is" type searches have more, while "how to," or more niche searches have fewer or none at all. For example one of my phrases, "SEO for Coaches" doesn't have one (usually) but "What are some keywords for coaches" usually does have an AI summary. (Pro tip, please don't use these bot-generated keywords, they're not properly sourced—yes, I've checked.)

This distinction feels to me like important information to sock away as we try to understand the landscape of how or if these searches are impacting traffic to websites.

I went to a webinar awhile back that broke down a ton of the data we do know about Generative AI's impact on search, and it was quite interesting and some of the info surprised me.

I'll get into the numbers in a second, but the complexity of this is why when someone asks me, as they did in my Squarespace SEO webinar last week, "Isn't Google dead because of ChatGPT," my answer is, "We need to take a step back and ask a different question."

(More on that question in a second...)

Before we get into that, here's some important context about LLMs/chatbots/generative AI that I believe all of you need to know:

(And, of course, this is a very dynamic topic, so keep in mind that this may change six months from now.)

1. Generative AI or LLMs get their information from somewhere—it's not creating answers from scratch, websites and other sources provide the data it remixes! I know this may seem obvious, but I'm stuck by the number of people who don't know this.
2. Google searches grew 20%+ in 2024 (this is according to research conducted by Datos for SparkToro); ChatGPT's search volume is similar to Pinterest's, which is something but still niche.
3. Most of the activity on chatbots are conversational or transactional, not searches.
4. Thus far, chatbots like ChatGPT have not impacted the market share of searches in places like Google, Bing, DuckDuckGo, and on voice search tools like Siri.
5. We know there are overall MORE searches because of this new software, but not as many as you'd think! It's not driving the market in a statistically meaningful way.

Think of queries on the internet as a pie that keeps getting bigger, of which AI software now has its own slice.

So, with that context it kind of makes sense that Google's AI summaries in search engine results are very much those where there are large numbers of searches, common questions (a fun dog trick to teach is less common than an easy dog trick to teach, like in my example above), and more basic information.

You still with me? I know this is a lot!

What I've observed in my testing is that repetitive information finds its way into the AI summaries, just like they do AI chatbots. For example, there are several essentially duplicative directories of Squarespace experts on the internet, and the AI generated summary around Squarespace experts basically pulls that repeated information up into the top of the search results.

Now, you may think this is great for the websites summarized, right? Think again! In fact, it results in a 34 percent decline in click-throughs. (According to AHrefs, a trusted source on SEO data.)

So, if you find yourself in one of these summaries, you may be disappointed versus if you find yourself in the first few positions after the ads on the results page. As a result, I don't love being cited in those summaries, except for the potential benefit of brand awareness.

As a result, if you're not aiming for basic searches about basic information, I'm leaning towards this currently being a non-issue for many expertise businesses. It's quite likely this software can't really pull enough repetitive data from the internet to help or hurt you. (Of course, that may change.)

So, back to my earlier point, what is the right question to ask about search behavior and Generative AI/Large Language Models?

The better question is:

Is the rise of LLMs/Generative AI going to impact my marketing ecosystem? If so, what do I need to adjust to avoid any problems because of that?

It's a less dramatic, more pragmatic question than, "Isn't SEO dead," but it'll get you headed in the right direction much more effectively. The fun part is that what works well in traditional content-first search strategies are the exact techniques that solve that equation.

Onward & upward,

Sarah