HTTP Headers and SEO: What you should know

Reviewing your HTTP response headers can provide you valuable assets regarding your technical SEO performances. Whether you are checking not-matching canonical URLs or looped redirects, having a clear understanding of each returned HTTP header is really important. Our article is focusing on 5 HTTP response header fields that can have an impact on your SEO strategy.

1# X-Robots-Tag

The X-Robots-Tag HTTP header field indicates robots directives (not unlike those specified in the HTML tag). If a page isn’t indexed, and you can’t seem to figure out why, remember to check for the X-Robots-Tag header field. For more information about this tag, Google has produced a handy resource.

Examples:
X-Robots-Tag: noindex
X-Robots-Tag: googlebot: nofollow
X-Robots-Tag: unavailable_after: 12 DEC 2017 11:00:00 CET

2# Server

The server header field offers interesting information about the name of the server from which the HTTP response has been sent. Server details are used when you have to implement some server-side redirects or in an audit process.
Regarding each server type, you will have to provide different types of instructions for setting up mapped redirects. An Apache server might call for some .htaccess edits/additions, while an IIS server might call for some work with the URL Rewrite Module. Overall, it’s important to understand what types of servers your clients are using.

Examples:
Server: Apache/2.2.22
Server: nginx
Server: Microsoft-IIS/7.0
others…

3# Location

Once a requested resource has been redirected to a new URL, the location field is the next information to check. From there, you can spot the absolute URL that is redirected to and identify patterns in chained redirects for instance. But also, be sure to identify concurrent responses caused by redirects.

4# Link

The HTTP header link field shows that the requested resource has a relationship with other resources. Types of relationship you can see is the specification of a canonical URL in the link header field. While this is not frequent, Google has announced it was supporting this back in 2011.

Examples:
Link: <http://www.mywebsite.com/blue-shirt.html>; rel=”canonical”
Link: ; rel=”alternate”

Moz has written a great post about implementing rel=”canonical” in HTTP headers.

5# Status

Response status are not specifically a header field but they are part of the status-line in a response. They are still pertinent to analyze as they give informations about a status of the requested resource. They are better known as HTTP status code.

From 100s to 500s, status codes are split within the following categories:

  • 100s: informational. The request has been received and the process is going on.
  • 200s: success: the request has been received and works well.
  • 300s: redirect: the request has been received but needs a extra step to be completed.
  • 400s: client error: the request has been asked by a client but the destination page is not correct.
  • 500s: server error: the request asked by the client was correct but the server failed to deliver it.

We have written a detailed article about HTTP status codes on our blog, don’t hesitate to check it out.
To sum up, mentioned HTTP header fields are just an extract of what you can encountered when doing technical SEO audits. Feel free to share in the comment section, HTTP header fields you are dealing with.

You can access a complete list of the HTTP header fields here

Emma

About Emma

Emma is Communication & Marketing Manager at @Cogniteev and write about SEO and search engine updates.

Entries by Emma