Is Google Crawling Your Site a Lot? That Could Be a Bad Sign

Gary Illyes, a Google Analyst, recently warned in a LinkedIn post that if Google starts aggressively crawling your website, it might indicate underlying issues rather than being a purely positive sign.

Increased Crawling: A Double-Edged Sword

While more crawling can sometimes mean Google is finding your site valuable, Illyes advises caution. He mentions two common problems associated with increased crawling: infinite spaces and website hacks.

Infinite Spaces: A Common Culprit

Illyes highlights that websites with “infinite spaces” like calendar modules or endlessly filterable product listings can create unlimited URLs. This can cause a spike in crawling activity.

“If your site generally has pages that search users find helpful, crawlers will get excited about these infinite spaces for a time,” Illyes explains.

To prevent this, he recommends using the robots.txt file to block crawlers from accessing these areas.

Hacked Sites: Another Cause for Concern

A sudden increase in crawling can also indicate a security breach where hackers inject spam into your site. Initially, crawlers may treat this as new content before recognizing it as malicious.

Illyes states, “If a no-good-doer somehow managed to get access…they might flood your site with, well, crap… crawlers will get excited about these new pages for a time and happily crawl them.”

Be Skeptical of Crawling Spikes

Illyes advises treating unexpected increases in crawling as potential problems until the root cause is identified. He notes, “Treat unexpected sharp increases in crawling as a symptom…until you can prove otherwise.”

Fixing Hacked Sites: Google’s Guidance

For those dealing with hacked sites, Illyes refers to a helpful page and video from Google outlining the recovery process.

Key Steps in Google’s Recovery Process

  1. Identify the Vulnerability: The first step is finding how the hacker gained access. Tools like Google’s Webmaster Tools can help detect issues.
  2. Fix the Vulnerability: Once identified, close the security hole to prevent future unauthorized access. This might involve updating software or changing passwords.
  3. Clean the Hacked Content: Remove any spam, malware, or other malicious content. Security plugins like Wordfence can assist in this process.
  4. Harden Security: Take additional measures to secure the site, such as enabling firewalls, limiting user permissions, and regularly updating software.
  5. Request a Review: After cleaning the site and fixing vulnerabilities, request Google to review the site to remove any security warnings or blacklists.

The review process is typically faster for malware issues than for spam issues, as spam cleanup requires more thorough inspection.

Additional Tips from Google’s John Mueller

John Mueller has also offered advice for recovering from the SEO impact of hacked pages:

  • Use the URL removal tool to quickly deindex hacked pages.
  • Improve the overall quality of the site beyond just removing hacked content.
  • Understand that lingering impacts may persist for months as the site works to regain Google’s trust.

By following these guidelines and being vigilant about sudden changes in crawling activity, you can better protect your site from potential issues and maintain a healthy SEO profile.

Recent Articles

Google Alerts How Soft 404 Errors Hurt Your SEO and How to Fix Them
WordPress Plugin Supply Chain Attacks Are Increasing
Google’s Struggles with Two Types of Searches
Google Completes June 2024 Spam Update Rollout
Google Ends Continuous Scroll in Search Results What It Really Means
Untitled design (7)

🚀🚀Enrollment Closing Today, Batch will start on July 26, 2024. Seats Left: 04/24


Fill The below information and our team will get back to you.

Coming soon....

This feature is currently not available for everyone. Our team is working on it and it will be updated shortly.