Sep 11, 2026
By Victor Teran
Google Had Never Crawled 20 of Our 27 Pages. We Only Found Out Because We Looked
We audited our own site the way we audit a client's. Google had never crawled 20 of its 27 pages. The sitemap was valid, declared in robots.txt, and had never been submitted. Nobody would have noticed for another year.
Analytics
SEO
Measurement
Every founder has a version of this problem. Something in the acquisition path has been quietly broken since launch, it produces no error, and the only symptom is a number that stays flat for reasons everyone explains differently.
Ours was search. We ran our own audit in September 2026 and the result was not a ranking problem or a content problem. Google had simply never looked at most of the site. Twenty of twenty-seven URLs came back as unknown, with a last-crawl time of never.
We are publishing it because the failure mode generalises, and because it is more useful than another post about tags.
Key takeaways
Twenty of our twenty-seven pages had never been crawled. The seven that had were exactly the ones the homepage linked to.
The sitemap existed, was valid, and was declared in robots.txt. It had never been submitted, and existing is not the same as submitted.
Fixing discovery moved indexed pages from 7 to 10 within 24 hours, and pulled three money pages from never-crawled to indexed.
Requesting indexing on the blog index was the unlock. Google crawled it, followed its links, and took the blog from zero posts indexed to nine of eleven in three days.
The first search impressions that arrived were all for former client brand names, which is a diagnosis in itself: the site was findable only to people already looking for someone else.
How does a whole site go uncrawled?
Because discovery and quality are different problems, and everyone spends their time on the second one.
The mental model most people carry is that Google finds everything and then decides how to rank it. The first half is not true. Google finds what it is led to, and then decides whether to spend crawl budget on the rest.
Our seven crawled pages were the seven the homepage linked to. That is the entire explanation. Everything else, the case studies, the service pages, the blog, existed and was reachable by a human clicking around, but had nothing pointing at it that Google had followed.
The sitemap should have solved that. It was generated, valid, listed in robots.txt, and contained every URL. It had never been submitted in Search Console, and nothing about the site's behaviour indicated that. There is no error state for this. The file returns 200 to anyone who asks.
Google's own documentation is clear that a sitemap is a hint about which pages you consider important, not an instruction to crawl them (Google Search Central). It also spells out that crawl capacity is finite and allocated by demand (crawl budget documentation), which is the mechanism that left twenty of our pages untouched.
What did fixing discovery actually change?
Indexed pages went from 7 to 10 in 24 hours, and the three pages that sell went from never-crawled to indexed.
Two changes did it. Submitting the sitemap, and adding real crawlable links in the site footer to the pages that had been orphaned.
Before | After 24 hours |
|---|---|
7 pages indexed | 10 pages indexed |
20 URLs unknown to Google | 9 unknown |
Money pages never crawled | All three indexed |
Sitemap never submitted | Submitted, 0 errors, 0 warnings |
The footer link matters more than it sounds. A link a human reaches through a menu is not necessarily a link a crawler follows (how Google's crawlers work). Money pages that only exist at the end of a funnel tend to be exactly the pages nothing points at.
The underlying lesson is the one we keep relearning: a metric that stays flat is not the same as a metric that is being measured, which is also why a redesign can ship and change nothing.
FREE GUIDE
The 4 product leaks costing you growth
A short audit guide for founders. Find the four places your product leaks revenue, and what to fix first.
What finally moved the blog?
Requesting indexing on one page, the blog index, and letting Google do the rest.
The sitemap and the footer links fixed the money pages within a day. The blog did not move. A week later the index page still read discovered but not indexed, and every single one of the eleven posts came back as unknown to Google. Written, live, in the sitemap, never looked at.
The fix took five minutes. In Search Console's URL Inspection, we requested indexing on /blog and on three individual posts. That puts a URL in a priority crawl queue and bypasses the crawl budget question entirely.
Before the request | Three days later | |
|---|---|---|
| Discovered, never crawled | Indexed |
Blog posts indexed | 0 of 11 | 9 of 11 |
Posts unknown to Google | 11 | 1 |
Only four URLs were requested by hand. The other five indexed themselves, because once Google crawled the index page it followed the links on it. That is the whole lesson in one line: you do not need to push every page, you need to get the crawler to the page that points at the rest.
Two posts are still outstanding, which is a useful reminder that this is a queue and not a switch.
What did the first search data actually say?
Every impression we got was for a former client's brand name, not for anything we sell.
Once data started arriving, the picture was unambiguous. The impressions were for queries like the names of products we had worked on, landing on case study pages. Zero impressions for activation, retention, onboarding, or growth audits.
That is a useful diagnosis rather than a disappointment. It says the site is currently discoverable to people already searching for someone else, and invisible to the people it was built for. No amount of on-page work changes that. It is a demand and authority problem, and it takes longer than a technical fix.
It also killed a comfortable assumption. We had been treating flat search traffic as a content problem, which would have meant writing more. It was a discovery problem, which meant writing more would have produced more uncrawled pages.
Why does this keep happening to good teams?
Because a silent failure produces output that looks like success.
Nothing about an uncrawled site looks broken. The pages load. The sitemap validates. Analytics shows traffic, because the traffic is coming from direct and referral. Every dashboard is green and the thing you actually wanted, being found by strangers, was never happening.
This is the same shape as the automation failures we wrote about in why automations fail silently: the expensive failure is not the one that crashes, it is the one that keeps running while wrong. A crash gets investigated. A quiet zero gets explained.
The general form is worth internalising, because it applies well beyond search:
1. A step nobody owns. Submitting a sitemap belongs to no role. It gets generated by the platform and assumed by everyone. 2. No error surface. The failure produces a valid file, a 200 response, and no alert. 3. A plausible alternative story. Flat traffic gets attributed to content, or the market, or timing. All of those are more interesting than "nobody looked".
What should you check on your own product?
Anything where the absence of a result could be mistaken for a slow result.
Search is one instance. The same audit question applies to your onboarding emails, your event tracking, your paid pixels and your integrations. In each case, ask what a total failure would look like, and whether you could tell it apart from normal-but-slow.
Concretely, for a founder reading this today, three checks worth twenty minutes:
Search Console: open URL Inspection on five pages you care about. If they say discovered or unknown rather than indexed, your problem is upstream of content.
Analytics: verify a real event fires by triggering it yourself and watching it arrive, not by trusting the dashboard's presence.
Lifecycle email: sign up as a new user with a real address and see what actually arrives, in what order, over a week.
We found ours by running the audit we sell on ourselves. That is not a coincidence. A diagnosis is worth paying for because it looks at the parts nobody has a reason to look at, and it does that before the flat quarter rather than after it.
WHAT NEXT
Want this fixed in your product, not just explained?
Sep 11, 2026
By Victor Teran
Google Had Never Crawled 20 of Our 27 Pages. We Only Found Out Because We Looked
We audited our own site the way we audit a client's. Google had never crawled 20 of its 27 pages. The sitemap was valid, declared in robots.txt, and had never been submitted. Nobody would have noticed for another year.
Analytics
SEO
Measurement
Every founder has a version of this problem. Something in the acquisition path has been quietly broken since launch, it produces no error, and the only symptom is a number that stays flat for reasons everyone explains differently.
Ours was search. We ran our own audit in September 2026 and the result was not a ranking problem or a content problem. Google had simply never looked at most of the site. Twenty of twenty-seven URLs came back as unknown, with a last-crawl time of never.
We are publishing it because the failure mode generalises, and because it is more useful than another post about tags.
Key takeaways
Twenty of our twenty-seven pages had never been crawled. The seven that had were exactly the ones the homepage linked to.
The sitemap existed, was valid, and was declared in robots.txt. It had never been submitted, and existing is not the same as submitted.
Fixing discovery moved indexed pages from 7 to 10 within 24 hours, and pulled three money pages from never-crawled to indexed.
Requesting indexing on the blog index was the unlock. Google crawled it, followed its links, and took the blog from zero posts indexed to nine of eleven in three days.
The first search impressions that arrived were all for former client brand names, which is a diagnosis in itself: the site was findable only to people already looking for someone else.
How does a whole site go uncrawled?
Because discovery and quality are different problems, and everyone spends their time on the second one.
The mental model most people carry is that Google finds everything and then decides how to rank it. The first half is not true. Google finds what it is led to, and then decides whether to spend crawl budget on the rest.
Our seven crawled pages were the seven the homepage linked to. That is the entire explanation. Everything else, the case studies, the service pages, the blog, existed and was reachable by a human clicking around, but had nothing pointing at it that Google had followed.
The sitemap should have solved that. It was generated, valid, listed in robots.txt, and contained every URL. It had never been submitted in Search Console, and nothing about the site's behaviour indicated that. There is no error state for this. The file returns 200 to anyone who asks.
Google's own documentation is clear that a sitemap is a hint about which pages you consider important, not an instruction to crawl them (Google Search Central). It also spells out that crawl capacity is finite and allocated by demand (crawl budget documentation), which is the mechanism that left twenty of our pages untouched.
What did fixing discovery actually change?
Indexed pages went from 7 to 10 in 24 hours, and the three pages that sell went from never-crawled to indexed.
Two changes did it. Submitting the sitemap, and adding real crawlable links in the site footer to the pages that had been orphaned.
Before | After 24 hours |
|---|---|
7 pages indexed | 10 pages indexed |
20 URLs unknown to Google | 9 unknown |
Money pages never crawled | All three indexed |
Sitemap never submitted | Submitted, 0 errors, 0 warnings |
The footer link matters more than it sounds. A link a human reaches through a menu is not necessarily a link a crawler follows (how Google's crawlers work). Money pages that only exist at the end of a funnel tend to be exactly the pages nothing points at.
The underlying lesson is the one we keep relearning: a metric that stays flat is not the same as a metric that is being measured, which is also why a redesign can ship and change nothing.
FREE GUIDE
The 4 product leaks costing you growth
A short audit guide for founders. Find the four places your product leaks revenue, and what to fix first.
What finally moved the blog?
Requesting indexing on one page, the blog index, and letting Google do the rest.
The sitemap and the footer links fixed the money pages within a day. The blog did not move. A week later the index page still read discovered but not indexed, and every single one of the eleven posts came back as unknown to Google. Written, live, in the sitemap, never looked at.
The fix took five minutes. In Search Console's URL Inspection, we requested indexing on /blog and on three individual posts. That puts a URL in a priority crawl queue and bypasses the crawl budget question entirely.
Before the request | Three days later | |
|---|---|---|
| Discovered, never crawled | Indexed |
Blog posts indexed | 0 of 11 | 9 of 11 |
Posts unknown to Google | 11 | 1 |
Only four URLs were requested by hand. The other five indexed themselves, because once Google crawled the index page it followed the links on it. That is the whole lesson in one line: you do not need to push every page, you need to get the crawler to the page that points at the rest.
Two posts are still outstanding, which is a useful reminder that this is a queue and not a switch.
What did the first search data actually say?
Every impression we got was for a former client's brand name, not for anything we sell.
Once data started arriving, the picture was unambiguous. The impressions were for queries like the names of products we had worked on, landing on case study pages. Zero impressions for activation, retention, onboarding, or growth audits.
That is a useful diagnosis rather than a disappointment. It says the site is currently discoverable to people already searching for someone else, and invisible to the people it was built for. No amount of on-page work changes that. It is a demand and authority problem, and it takes longer than a technical fix.
It also killed a comfortable assumption. We had been treating flat search traffic as a content problem, which would have meant writing more. It was a discovery problem, which meant writing more would have produced more uncrawled pages.
Why does this keep happening to good teams?
Because a silent failure produces output that looks like success.
Nothing about an uncrawled site looks broken. The pages load. The sitemap validates. Analytics shows traffic, because the traffic is coming from direct and referral. Every dashboard is green and the thing you actually wanted, being found by strangers, was never happening.
This is the same shape as the automation failures we wrote about in why automations fail silently: the expensive failure is not the one that crashes, it is the one that keeps running while wrong. A crash gets investigated. A quiet zero gets explained.
The general form is worth internalising, because it applies well beyond search:
1. A step nobody owns. Submitting a sitemap belongs to no role. It gets generated by the platform and assumed by everyone. 2. No error surface. The failure produces a valid file, a 200 response, and no alert. 3. A plausible alternative story. Flat traffic gets attributed to content, or the market, or timing. All of those are more interesting than "nobody looked".
What should you check on your own product?
Anything where the absence of a result could be mistaken for a slow result.
Search is one instance. The same audit question applies to your onboarding emails, your event tracking, your paid pixels and your integrations. In each case, ask what a total failure would look like, and whether you could tell it apart from normal-but-slow.
Concretely, for a founder reading this today, three checks worth twenty minutes:
Search Console: open URL Inspection on five pages you care about. If they say discovered or unknown rather than indexed, your problem is upstream of content.
Analytics: verify a real event fires by triggering it yourself and watching it arrive, not by trusting the dashboard's presence.
Lifecycle email: sign up as a new user with a real address and see what actually arrives, in what order, over a week.
We found ours by running the audit we sell on ourselves. That is not a coincidence. A diagnosis is worth paying for because it looks at the parts nobody has a reason to look at, and it does that before the flat quarter rather than after it.
WHAT NEXT
Want this fixed in your product, not just explained?
Sep 11, 2026
By Victor Teran
Google Had Never Crawled 20 of Our 27 Pages. We Only Found Out Because We Looked
We audited our own site the way we audit a client's. Google had never crawled 20 of its 27 pages. The sitemap was valid, declared in robots.txt, and had never been submitted. Nobody would have noticed for another year.
Analytics
SEO
Measurement
Every founder has a version of this problem. Something in the acquisition path has been quietly broken since launch, it produces no error, and the only symptom is a number that stays flat for reasons everyone explains differently.
Ours was search. We ran our own audit in September 2026 and the result was not a ranking problem or a content problem. Google had simply never looked at most of the site. Twenty of twenty-seven URLs came back as unknown, with a last-crawl time of never.
We are publishing it because the failure mode generalises, and because it is more useful than another post about tags.
Key takeaways
Twenty of our twenty-seven pages had never been crawled. The seven that had were exactly the ones the homepage linked to.
The sitemap existed, was valid, and was declared in robots.txt. It had never been submitted, and existing is not the same as submitted.
Fixing discovery moved indexed pages from 7 to 10 within 24 hours, and pulled three money pages from never-crawled to indexed.
Requesting indexing on the blog index was the unlock. Google crawled it, followed its links, and took the blog from zero posts indexed to nine of eleven in three days.
The first search impressions that arrived were all for former client brand names, which is a diagnosis in itself: the site was findable only to people already looking for someone else.
How does a whole site go uncrawled?
Because discovery and quality are different problems, and everyone spends their time on the second one.
The mental model most people carry is that Google finds everything and then decides how to rank it. The first half is not true. Google finds what it is led to, and then decides whether to spend crawl budget on the rest.
Our seven crawled pages were the seven the homepage linked to. That is the entire explanation. Everything else, the case studies, the service pages, the blog, existed and was reachable by a human clicking around, but had nothing pointing at it that Google had followed.
The sitemap should have solved that. It was generated, valid, listed in robots.txt, and contained every URL. It had never been submitted in Search Console, and nothing about the site's behaviour indicated that. There is no error state for this. The file returns 200 to anyone who asks.
Google's own documentation is clear that a sitemap is a hint about which pages you consider important, not an instruction to crawl them (Google Search Central). It also spells out that crawl capacity is finite and allocated by demand (crawl budget documentation), which is the mechanism that left twenty of our pages untouched.
What did fixing discovery actually change?
Indexed pages went from 7 to 10 in 24 hours, and the three pages that sell went from never-crawled to indexed.
Two changes did it. Submitting the sitemap, and adding real crawlable links in the site footer to the pages that had been orphaned.
Before | After 24 hours |
|---|---|
7 pages indexed | 10 pages indexed |
20 URLs unknown to Google | 9 unknown |
Money pages never crawled | All three indexed |
Sitemap never submitted | Submitted, 0 errors, 0 warnings |
The footer link matters more than it sounds. A link a human reaches through a menu is not necessarily a link a crawler follows (how Google's crawlers work). Money pages that only exist at the end of a funnel tend to be exactly the pages nothing points at.
The underlying lesson is the one we keep relearning: a metric that stays flat is not the same as a metric that is being measured, which is also why a redesign can ship and change nothing.
FREE GUIDE
The 4 product leaks costing you growth
A short audit guide for founders. Find the four places your product leaks revenue, and what to fix first.
What finally moved the blog?
Requesting indexing on one page, the blog index, and letting Google do the rest.
The sitemap and the footer links fixed the money pages within a day. The blog did not move. A week later the index page still read discovered but not indexed, and every single one of the eleven posts came back as unknown to Google. Written, live, in the sitemap, never looked at.
The fix took five minutes. In Search Console's URL Inspection, we requested indexing on /blog and on three individual posts. That puts a URL in a priority crawl queue and bypasses the crawl budget question entirely.
Before the request | Three days later | |
|---|---|---|
| Discovered, never crawled | Indexed |
Blog posts indexed | 0 of 11 | 9 of 11 |
Posts unknown to Google | 11 | 1 |
Only four URLs were requested by hand. The other five indexed themselves, because once Google crawled the index page it followed the links on it. That is the whole lesson in one line: you do not need to push every page, you need to get the crawler to the page that points at the rest.
Two posts are still outstanding, which is a useful reminder that this is a queue and not a switch.
What did the first search data actually say?
Every impression we got was for a former client's brand name, not for anything we sell.
Once data started arriving, the picture was unambiguous. The impressions were for queries like the names of products we had worked on, landing on case study pages. Zero impressions for activation, retention, onboarding, or growth audits.
That is a useful diagnosis rather than a disappointment. It says the site is currently discoverable to people already searching for someone else, and invisible to the people it was built for. No amount of on-page work changes that. It is a demand and authority problem, and it takes longer than a technical fix.
It also killed a comfortable assumption. We had been treating flat search traffic as a content problem, which would have meant writing more. It was a discovery problem, which meant writing more would have produced more uncrawled pages.
Why does this keep happening to good teams?
Because a silent failure produces output that looks like success.
Nothing about an uncrawled site looks broken. The pages load. The sitemap validates. Analytics shows traffic, because the traffic is coming from direct and referral. Every dashboard is green and the thing you actually wanted, being found by strangers, was never happening.
This is the same shape as the automation failures we wrote about in why automations fail silently: the expensive failure is not the one that crashes, it is the one that keeps running while wrong. A crash gets investigated. A quiet zero gets explained.
The general form is worth internalising, because it applies well beyond search:
1. A step nobody owns. Submitting a sitemap belongs to no role. It gets generated by the platform and assumed by everyone. 2. No error surface. The failure produces a valid file, a 200 response, and no alert. 3. A plausible alternative story. Flat traffic gets attributed to content, or the market, or timing. All of those are more interesting than "nobody looked".
What should you check on your own product?
Anything where the absence of a result could be mistaken for a slow result.
Search is one instance. The same audit question applies to your onboarding emails, your event tracking, your paid pixels and your integrations. In each case, ask what a total failure would look like, and whether you could tell it apart from normal-but-slow.
Concretely, for a founder reading this today, three checks worth twenty minutes:
Search Console: open URL Inspection on five pages you care about. If they say discovered or unknown rather than indexed, your problem is upstream of content.
Analytics: verify a real event fires by triggering it yourself and watching it arrive, not by trusting the dashboard's presence.
Lifecycle email: sign up as a new user with a real address and see what actually arrives, in what order, over a week.
We found ours by running the audit we sell on ourselves. That is not a coincidence. A diagnosis is worth paying for because it looks at the parts nobody has a reason to look at, and it does that before the flat quarter rather than after it.
WHAT NEXT