With the exception of Halloween, a spooky-looking house isn’t the most inviting place. You may be picturing a tall, rusted wrought iron gate with a “BEWARE” sign in front of a decrepit Victorian manor riddled with cobwebs. While looks may be deceiving, we’re going to stay far away.
You want content on your website to be as inviting as possible so people will find it and benefit from it. It’s crucial that before visitors can find you, search engines must be able to crawl through your site and do the same. SEMrush is showing us 11 crawlability problems and how to fix them so you can tear down those cobwebs and spruce up your site to help search engines lead visitors right to your front door.
- Pages Blocked in Robots.txt: This is where search engines look to see which pages they can and can’t crawl. Check your robots.txt file to ensure that you’re not blocking them from accessing your website. Replace “disallow” directives with “allow” to grant access.
- Nofollow links: The “nofollow” tag (<meta name=”robots” content=”nofollow”>) tells search engines not to crawl specific links. Check your site for these links and remove the tag if it’s not necessary.
- Bad Site Architecture: Put simply, organization is key. Improving your site architecture involves organizing pages in a logical hierarchy with internal links. This will help crawlers easily access all pages.
- Lack of Internal Links: Check to ensure you don’t have pages with no internal links pointing to them. A page without any links pointing to it is like an uncharted island – it’s isolated from the rest of the site and there’s no clear path to get there.
- “Noindex Tags”: Google treats “noindex” tags (<meta name=”robots” content=”noindex”>)as it does “nofollow” tags, so leaving these tags on pages you want the crawler to access will render your page off-limits to the crawler.
If you’ve noticed pages of yours aren’t showing up in Google when you’re trying to earn a top-ranking SERP position, these issues may be to blame. SEMrush lists another 6 potential problems your pages may be facing, so read on and stay diligent.
