URL optimization in 2013 and beyond

You use them every day when you go online.  Marketers use them in all sorts of promotions. Yet why is the creation and optimization of URLs a topic that often gets overlooked?  You would think that people would spend more time on such an integral component of their web pages.  Before we discuss some URL optimization practices, let’s take a look at the history of the URL.

URLs were invented in 1994 by Tim Berners-Lee and the Internet Engineering Task Force (IETF) URI working group based on some documentation from meetings that began in 1992.  (Yes people, the Internet is that young).  Every URL that exists consists of the following elements:

  • The protocol (i.e. http)
  • a host or domain name (in some cases an IP address)
  • a resource name or path of the resource

URL defined

A URL, is defined as a Uniform (or universal) resource locator. Simply put, a URL is the address of a web page.  A typical URL might look something like this:

example URL

The purpose of a URL is really to produce a text-based name for the IP addresses that computers use to identify with servers and vice versa.  URLs were created for humans and were designed, to provide a user friendly, semantically sensible name for a webpage. URLs make it easier for us to determine the name of a web page and potentially the topicality of that page. In fact, a properly formatted URL should allow users to identify the subject matter of a page by simply looking at the URL.  This is beneficial to not only human visitors but search engine crawlers as well.

To me URLs are one of the most valuable elements on a webpage that we can create.  Think about the important uses of a URL:

  • To identify the topic of a given page
  • For navigation purposes
  • For linking purposes
  • For marketing and promotional purposes

Search engines crawl URLs and will index a web page based in part of the composition of the URL.  Although they may deny it, having relevant keywords in your URLs can help with your search efforts as search engine place some authority on the “search engine friendliness” of the URL.  While the engines use hundreds of elements in their ranking algorithms a strong, relevant and yes keyword-rich URL can go a long way in gaining visibility within Search results.  Over at Moz (formerly SEOmoz) they produced a great graphic on the Anatomy of a URL which we feel is worth sharing here:

anatomy of a URL

This illustrates the key components of an SEO friendly URL.

Here are some of the typical questions about URL that we get asked.

Ten questions about URL optimization aka URL best practices

Do I optimize URLs for users or for search?

The answer is for both.  A URL should be clean, concise and be reflective of the site’s architecture.  Keep in mind that a site’s URL structure should be as simple as possible.  URLs should be logical and according to Google  be created “… in a manner that is most intelligible to humans.”  Creating URLs that are logical for humans will mean that you are also creating a search engine friendly URL in the process.  URLs should be optimized for the user and visitors of your site.  Doing so will indirectly create a search engine friendly URL as well.

Should I use underscores or hyphens in my URLs?

We have always recommended that you use hyphens as word separators in URLs.  Google also recommends this as they state that URLs such as http://example.com/red-widgets is more useful to them than http://www.example.com/redwidgets. Let’s make this very clear:  USE HYPHENS AS WORD SEPARATORS IN YOUR URLs.  Whatever you do be consistent (i.e. do not use hyphens and underscores… sheesh).

Should I include keywords in my URLs?

Yes when it makes senses to do so.  Use common sense and do not spam your URLs with keywords.  Having a primary key phrase in the URL can help communicate the topicality of your page, why wouldn’t you include it in your URL as it helps describe what that page is all about.  Include relevant keywords and do not spam your URLs.

Should I repeat keywords in URLs?

No, this can and will be considered as keyword stuffing by the Engines and should be avoided.  Look for Google to come down on sites that keyword spam their URLs in the future.

Should I use dynamically generated URLs?

When possible, we recommend using “old-fashioned” static URLs as opposed to dynamically generated URLs.  URLs that are dynamically generated can cause issues for the search engine crawlers and append the URL with unnecessary parameters and long ID numbers.  In addition, static URLs are more appealing to users.  Which URL would you rather click on: http://www.example.com/red-widgets or http://www.example.com/index.php?id_col=123&sid=4567abcd8910def987654431/?  Furthermore dynamically generating URLs can also cause multiple versions of the same URL (read: duplicate content) to be created as things like timestamps are added to the URL.  Session IDs, for example, can create massive amounts of duplication and a greater number of URLs. Excessive duplicate content can cause issues with efficient crawling and indexing of your site’s content.  If your CMS produces dynamically generated URLs you might want to look into leveraging URL re-write rules to produce cleaner, shorter and more static appearing URLs.

Can I use upper and lowercase words in my URLs?

You can, but we recommend against it.  We recommend that “no” do not use upper and lowercase words in URL.  In fact we recommend that URLs be in lowercase for consistency.  Inconsistencies in the URLs can cause issues with linking and crawling.  Not to mention they just look bad.  As a best practice we recommend that you keep your URLs consistent and use lowercase in your URL creation.

Can URLs be too long?

Yes and look for this to change even further in 2014 and beyond.  Previously and perhaps even currently a URL should not be longer than 2,048 characters.  The main reason is that browsers such as Internet Explorer won’t be able to load the page if they are longer.  In addition, moving forward with Google algorithm updates such as Penguin and Panda, we would expect that Google (if they haven’t already started) will use long URLs as a negative ranking factor. Others out there seem to agree.  Consider this, the sitemaps protocol (used by Google and other major engines), which allows a site to inform search engines about available pages, has a limit of 2,048 characters in a URL.  Some have stated that the Google SERP tool does not support URLs longer than 1,855.  This post also shares some insight that the “RI producers should use names that conform to the DNS syntax, even when use of DNS is not immediately apparent, and should limit these names to no more than 255 characters in length.”  While this is specific to the hostname it does hint that shorter URLs are preferred for a number of reasons.

Should I simply put everything in a subfolder and base my URL structure off of that?

This question is an interesting one.  Traditionally this is what is recommended however there is a “limit” as to how many folders deep you should push your content.  One thing that we have seen in all of our years of analyzing sites and URLs is that URLs that are closer to the root of the domain tend to perform better in Search.  Some of the observations that we have noticed with URLs that are closer to the root:

  • Are crawled more actively and efficiently than deeper pages
  • Have an easier time getting indexed
  • Inherit PageRank easier
  • Tend to perform better and have greater visibility within the search results
  • Tend to receive higher quality back links
  • Tend to receive higher page authority from the engines

We understand the need to create a hierarchy that is reflective of your offering (especially for ecommerce sites with categories and sub-cats) but there is something to be said for keeping your content as close to the root as possible.  While Google suggests that “A site’s URL structure should be as simple as possible” we know that Google has placed additional value on pages that are closer to the root.  So if you are planning a redesign you might want to factor this in.  Ideally your URL should be reflective of your site hierarchy but if you can keep your content as close to the root as possible you will be on the right path to search success.  Examples of site that do a good job of keeping their content close to the root include:

http://www.ehealthinsurance.com/

http://www.zappos.com/

http://www.apple.com

Should my URLs be http or https?

The answer is that it depends on the type of site you have. You may use both.  Having said that, the majority of sites use the standard HTTP version.  HTTPS signify that the site is a secure site.  HTTPS uses Secure Socket Layer (SSL) as a sub layer under its regular HTTP application layering.  In the past search engines such as Google have had an easier time indexing http versions of sites however they will index https as well.  The fact of the matter is that HTTPS should be used with sites where it makes sense to do so (i.e. banking sites, payment page, shopping websites, login pages etc.).  More information on http vs https.

Should I use vanity URLs?

A vanity URL is a cleaner, shorter URL that is usually used in marketing promotions of the print, TV or radio variety.  These URLs are typically redirected to point to something that is related to those URLs.  The purpose of vanity URLs is to make them more memorable for the viewer ideally so that a user will type in the URL when they are online. More on vanity URLs atWikipedia You can use vanity URLs, but better yet is to simply create user friendly URLs on your site or microsites so you won’t need vanity URLs.

As a bonus 11th question that we often get asked is around the use of redirection and if it is ok to redirect an “old page” to a “new” page.

Redirection is fine providing that you are using permanent 301 redirects and that you are not creating redirect chains where the redirect jumps from one page to another to another to another etc.  Google has stated that they are ok if you have three redirects or less in a chain but the fact is that where possible you should not leverage redirects chains if they can be avoided.  We understand that there are times when a redirect is going to be required.  For example you have changed the URL of an existing page to have a new filename.  Ideally the less times you redirect a page the better.  The main reason is that the page has built up authority and you want to preserve as much of that authority over time as possible.  As a best practice we recommend that if redirects are required that you:

  1. Always use permanent 301 redirects
  2. Avoid the creation of redirect chains where possible.

There you have it, 11 URL FAQs that we typically get from clients, webmasters and web teams.  When you create URLs try to think of the big picture and think of longevity.  You don’t want to rewrite your URLs every couple of years.  Keep your URLs simple, short and relevant.  Your site visitors will appreciate your URLs not to mention the search engines who will be crawling your URLs.  Having consistent URLs is great for navigation, interlinking and for indexing.  Pay attention to length and include relevant keywords when it makes sense to do so.  Do not change your URLs for the sake of changing your URLs.  Plan out your naming conventions carefully.

We are expecting Google to make some changes with how they leverage URLs as part of their ranking algorithms so following the best practices outlined above will help ensure that your URLs will be both user and search friendly.  Most importantly create URLs that will resonate with your audience.  Your URLs are the address as to how people will find your content.  Make it easy for them to find your content and make it easy for them to remember where they found it.