Generative AI continues to transform search engine optimization (SEO), and as it does, it’s become more important than ever to understand how tools like ChatGPT interpret website content.
With this in mind, the team at Eastern Standard, a leader in digital strategy and design and a WP Engine agency partner, undertook an insightful experiment to uncover how ChatGPT processes and responds to various types of website content.
In addition to their website, the Eastern Standard team analyzed content from various B2B, healthcare, higher education, and nonprofit websites to evaluate ChatGPT’s ability to understand and interpret different content formats and identify areas for improvement.
The results revealed significant insights you can use in your own SEO strategies as you navigate the evolving age of AI.
To find out more, we sat down with Eastern Standard Co-founder and Chief Digital & Technology Officer Jim Keller, who provided us with a closer look at the findings.
Read on for a recap of our conversation.
Thank you, Jim. Before discussing the findings, can you tell us more about Eastern Standard and the projects you specialize in?
Eastern Standard is a branding and digital agency originally based out of Philadelphia but now operating a fully remote team. We help a variety of clients get the most out of their digital presence through audience research and messaging, SEO and content strategy, UX and design, web development and CMS implementation, and ongoing site optimization.
Our client base is quite varied, but we have a strong portfolio in higher education, healthcare, B2B & professional services, and nonprofit organizations.
With regard to the exercise you ran using ChatGPT, what exactly did you do, and what were you hoping to find?
We wanted to better understand how AI tools read and interpret content. Our clients were asking how AI should influence their content strategy, and we needed a way to offer concrete guidance instead of just high-level intuitions.
We conducted an experiment where we ingested content from many websites into an AI-readable format, then fed it into an OpenAI language model. We started asking simple questions: for a university we might ask, “How much does it cost to apply?” or, “How do I schedule a visit?”
At first, we would get pretty unsatisfactory answers, so we refined our prompts and updated our approach. Then something interesting happened.
We would still occasionally get unsatisfactory answers, but it wasn’t because the code was buggy; it was because of actual deficiencies in the content or the content structure. This led us to start using the tool to make specific content strategy improvements.
In your findings, you note that generative AI doesn’t just ingest keywords but interprets content, making it crucial to create clear, complete, well-structured content. Can you elaborate on the specific nuances content creators should focus on to ensure their work is well-interpreted by AI?
The first thing we noticed is that the AI liked descriptive, fully formed text it could easily read and interpret. Long-form paragraphs and other clear, declarative sentences allowed it to provide the most accurate and confident results.
Obviously, we don’t want to simply write giant walls of text, but content creators should take every opportunity to use clear and complete text phrases to answer specific questions.
For example, instead of writing, “We offer a variety of paid media and digital marketing services”, go a bit further and give the AI something to really chew on: “Our agency provides pay-per-click ad campaign management, content strategy and copywriting, landing page creation, technical SEO and link building services.”
If you’re using lists, grids, or other visual elements to break up text content, that doesn’t pose a problem, but it’s critical that your site uses the best semantic HTML markup for the job.
A lot of sites fall back on
In one of your tests, ChatGPT incorrectly inferred that a hospital did not offer medical services due to its legal disclaimer. How can organizations ensure that AI accurately interprets their essential content while still maintaining necessary legal language?
Ultimately, I don’t think it’s a problem to maintain legal language. We didn’t build any special conditions for legal language into our tool, but Google is smart enough to know that certain content falls into a special category.
I think the takeaway here is that there’s sort of a “relative strength” of the language used on the site that can influence AI. To the point above, the legal language was clear, complete, and made definitive and declarative statements.
When compared against other areas of the site that may have been relevant to the same query, the clearer answer “won.” So again, it’s a matter of ensuring your text content is chock full of specific answers and isn’t pure marketing jargon.
You found that old content can confuse ChatGPT, leading to outdated or incorrect responses. What best practices do you recommend for maintaining and updating web content to avoid such issues?
You shouldn’t be afraid to remove old, stale content. Content is still king, but that doesn’t mean that more content is always better.
For a site relaunch we completed last year, we cut down the number of blog posts by about 70%. There were too many posts that had little or nothing to do with our lead generation or conversion strategy, so after some internal debate, we opted to just eliminate them.
We felt confident based on past experience, crawl data, and analytics that it was the right move. We used a “410 Gone” code for those pages (which isn’t as common as 301 or other codes) to indicate “Yes, we removed these on purpose.”
The strategy paid off: the remaining, highly relevant blog posts were elevated in many cases to top positions, including featured Google snippets.
“You shouldn’t be afraid to remove old, stale content. Content is still king, but that doesn’t mean that more content is always better.”
That said, AI bots and search engines are smart enough to weigh newer content vs older content as long as there are clear signals about what’s newer and what’s not.
However, we commonly see that those signals—proper meta tags or even a date on a press release—either aren’t there, aren’t accurate, or are tucked away so it’s not clear to a text parser, “This date means this is the date the page was published.”
So if you won’t be removing content, make sure your page meta tags are present and accurate, including those that specify publish dates.
How has the increasing use of generative AI in search engines influenced your overall SEO strategy for clients in different sectors?
Knowing that content will be read and interpreted by AI has influenced us to add more text blocks than we might have before, but ultimately most of the tactics we extracted from our AI experiment aren’t new.
Proper page structure and markup, clear and direct content, and effective metatags are practices that should’ve been in place prior to AI. However, they’re more important than ever since AI will be interpreting content in a different way than previous crawlers and parsers.
After AI ingests the content, it’s like having access to a person whose only knowledge of the world comes from the website. We find ourselves asking, “Given the content on the site, how would that person answer this question or that question?”
If we’re not convinced that our proto-human would have the answer, we have more content work to do.
Given your insights, what proactive steps can organizations take to audit their existing content and align it better with the interpretive capabilities of generative AI?
- Clear out old content that may be providing outdated, irrelevant, or conflicting answers
- Don’t miss the opportunity to clearly and completely answer questions within the text content on your site.
- Take this opportunity to review your site for practices that aren’t good for SEO regardless of generative AI. For example:
- Content or important information embedded exclusively in graphics or images
- Using broad messaging instead of answering specific questions
- Combining too much disparate content on a single page
- Failing to use semantic HTML tags
How do you foresee the integration of AI tools like ChatGPT evolving in the context of website management and content creation over the next few years?
There’s enough on this topic to fill its own article, but I think it’s safe to say that AI will provide augmentation to people in many different roles. It’s likely that it will become embedded within our workflows and processes the same way that something like Slack has become completely intertwined with how we get work done.
In terms of content creation, AI is already a great tool for content, especially if you’re relying on it for ideas, drafts, and revisions rather than final copy.
And while every software tool seems to be rushing to incorporate it regardless of need, there are already helpful productivity adds such as Jira’s AI tool that allows you to craft queries in plain language instead of query language. So once the initial hype train slows down, we’ll be left with some practical and highly useful augmentation to our existing tools.
Thank you, Jim!
Find out more about WP Engine’s Agency Partner Program—the largest WordPress agency ecosystem—here, or visit WP Engine to learn more about our fully managed WordPress platform.
