Don’t Let Crawlers Get Lost on Your Website

Designing navigation, menus and filters are the points where the paths of web design, UX and SEO intersect. Navigation must be designed for users. The goal is to make searching for services and products comfortable. It also must be SEO friendly to enable Search Engine robots to efficiently crawl your website.

Different types of navigation are designed to make moving through the website easy for users. At the same time, SEO professionals use navigation and internal linking to control a robot’s journey across your website.

Let’s take a closer look at the different types of navigation and things that you have to consider in the optimization process.

Global Navigation

The main purpose of global navigation is to make the most important subpages available from every part of your website. It is also a key attribute of the global navigation – it generates site-wide links. It means that they are repeated on each page because the header and footer menus are displayed on every page.

Remember, every link to a subpage disperses the Page Rank within your website. Think twice before adding any links to your navigation. It’s natural that smaller websites should have simpler navigation. It’s much more complicated in larger e-commerce or news websites. The best choice for large websites is to indicate the most valuable categories to link from global navigation.

I know that it’s not easy to indicate pages with the biggest value. Here are things that you should take into consideration:

6 steps to plan your global navigation

  • Make a segmentation of your customers – specify who buys what.

  • Create logical groups of the products.

  • Try to divide them into subcategories.

  • Check your competitors and make note of what seems to be a good and bad solution. You can use those ideas in your website.

  • Undertake keyword research and find the best keywords for each landing page. If some category has a small volume of searches, consider if it is a valuable category. On the other hand, check if any of the subcategories have a significant volume of searches – maybe it deserves to be an independent category?

  • Prepare labels for your categories. On the one hand, they should be based on your keyword research, and on the other hand, they must be intuitive for users.

Here are examples of how big brands have dealt with categories in their navigation.

Nike decided to use an extremely powerful mega menu.

In the main navigation, you will find categories: Men, Women, Boys, Girls and Customize – very intuitive. Subcategories divide products into types, activities and brands. Users and robots gain direct access to the most important sections within the website.

But it’s not difficult to find a messy navigation. Here is an example:

This is neither user-friendly nor for robots – too many links and many of them should be moved to local or subsidiary navigation.

Menu implementation

While planning your main navigation you have to consider the technology that should be used. Google still prefers HTML links rather than fancy AJAX or JavaScript in the navigation. HTML is easy to crawl and, let’s say, natural to robots. Google does its best to crawl and understand AJAX but we can’t be sure that it will be properly crawled and if the link juice will be passed correctly.

AJAX and JavaScript are not the only bad ideas. Buttons or images in the navigation should also be changed to HTML. Moreover, HTML5 allows you to semantically markup the navigation with element.

  • In the head section indicate “next” and “previous” pages.

  • Consider if the content in the following paginated pages is unique. If it’s unique and valuable to users – let them be indexed. If not, block them with “noindex, follow”.

  • If you have a “view all” page (listing of all the products) add rel=canonical on the paginated pages pointing to the “view all” page.

  • Important: if you don’t have a “view all page”, never add rel=”canonical” on the paginated pages pointing to the first page! It may deindex the important pages!

  • Do not add rel=”nofollow” to the links to the paginated pages.

Related articles / products

Presenting thematically related content may increase user engagement. Moreover, it is also an opportunity to build strong, thematic, internal links. You can link to the articles from the same category as “Related posts”. If you have an e-commerce, you can present similar or complementary products.

However, do not include too many related links, because it can disperse link juice and can be distracting to users as well.

Also, try to eliminate useless links to the same subpage. Sometimes every clickable element on “related product” are a link to the same subpage. It also disperses link juice.

Breadcrumbs navigation

Breadcrumbs navigation was designed to show the click path from the current page to the home page. It’s located above the main content and presents previous categories in the website’s structure.

Breadcrumbs are a perfect solution, both for robots and users. Users receive an instant answer to the question: Where am I and how do I step back to the previous categories?

Robots gain natural links to related categories. It’s very useful while crawling complex website structures. If the robot directly accesses your product page it can easily crawl pages that are higher in the structure.

While optimizing your breadcrumbs navigation, remember not to add a link in the last breadcrumb to the current page. It wastes the link juice.

It is also worth implementing markups for breadcrumbs. It will change the appearance of your URLs in the search engine results. It will look like a click path.

Contextual navigation

I would say that it’s the lowest level of navigation, but very powerful for SEO. Contextual navigation consists of inserting links in the content to the thematically connected pages. It provides a high value for users (point additional, important content) and can increase an engagement to your website. Additionally, robots can navigate between connected articles. The links inserted in these articles are powerful, because they are surrounded by context. However, be careful because it’s easy to cross the line into natural linking. It’s easy to transform a valuable article into a spammy creature.


  • Always be focused on the user experience.
  • Check if all the pages in your navigation are crawlable.
  • Check if all the sections are linked (avoid orphaned pages).
  • Make sure unwanted URLs won’t be indexed – search results, filter results etc.
  • Double check your canonicals.
  • Keep your global navigation updated – remove redirected or broken URLs.
  • 29 June 2016

See all articles by Maria Cieślak

Did you like this article?

Why not share this article:

Subscribe to Elephate's newsletter to receive fresh SEO and Content Marketing updates and more!

Thanks! We are happy to have you on our list!

Expect some tasty news from the Elephate team soon.


more blog posts

Back to Blog list