Beyond Google: the universe of search

When we talk about search, we often talk about Google: Google and Google properties have the lion’s share of search visibility. There’s a reason we say we’re going to “google” something. But Google isn’t the only search engine out there, and depending on your audience, you stand to gain a lot of traction by adapting your SEO for the rest of the universe of search.

Search engines, search engines everywhere

Despite Google’s world-wide dominance, there are pockets where other search engines have a strong–and growing–foothold. In fact, Wikipedia lists approximately 20 or so current, general search engines, along with many more subject-specific search engines..

The biggest competitors to Google are Bing (and Yahoo!), Yandex, Baidu, and the small-but-growing DuckDuckGo:

Bing and Yahoo!

In 2009, Bing and Yahoo! Drew up a ten-year agreement to share search technologies, which means that their ranking, indexes, and search use are the same, and can be combined as a single search engine with two faces. We’ll treat them as one for the rest of this article.
Bing is particularly interesting for SEO and marketers but only because it holds over a third of the US market share, but also because there’s some evidence that Bing traffic converts better than Google traffic.

Optimizing for Bing

Bing employs many of the same technologies and works with many of the same ranking signals as Google. However, there are differences.

Some ranking signals count for more on Bing than on Google:

• Domain age and extension
• Social signals

Social media plays a role in today’s effort to rank well in search results. The most obvious part it plays is via influence. If you are a social influencer, your followers tend to share your information widely, which in turn results in Bing seeing these positive signals. These positive signals can have an impact on how your site ranks organically in the long run.

• Page load time

On-page SEO for Bing is also slightly different from Google. Bing explains:

• Only one <h1> per page is expected.
• Internal links are used to determine how content is related.
• The keyword/phrase you are targeting should be used a few times in the content, along with variations of the keyword or phrase.
• Fresh content, like on Google, is an advantage.

Like Google, Bing also evaluates mobile-friendliness. The following features are taken into account when determining mobile friendliness:

• Viewport configuration
• Zoom configuration
• Content width
• Readability of text
• Spacing of links and other content
• Important resources blocked from Bingbot

You can target local searches by geo-targeting all or part of your site in the Webmaster Tools. Geo-targeting in Bing is stated explicitly by the webmaster and can apply to the page, the directory, the subdomain, or to the whole domain.

Finally, Bing also uses entities to improve its understanding of content, and supports a variety of entities through Schema.org markup:

[Ebook] Technical SEO for non-technical thinkers

Technical SEO is one of the growing SEO fields today. It involves finding SEO solutions based on the how and why of how search engines–and websites–work. This ebook is everything you’ve always wanted to share with your clients, your friends, and your marketing teammates.

Read the ebook

Crawlability for Bing

Like Google, Bing’s index is built and updated principally through crawling the web. This means that websites must be crawlable in order to rank correctly on Bing.

According to Bing’s documentation, the following elements facilitate crawling:

Like, Google, you can also submit URLs for indexing. Bing will crawl URLs it finds in sitemaps–but you can also mass-submit URLs through their indexing tool and push them directly to the index via their API.

Bing’s tools and technology

Tools

• Webmaster tools
  • SEO Reports
  • SEO Analyzer
  • Markup validator
  • Keyword research tool
  • Fetch as Bingbot
• Webmaster blog

Technology

• Azure (BERT)

Yandex

Yandex is the Russian equivalent to Google. In addition to its search engine, it has many other web properties similar to Google’s: email, photos, products… Yandex can claim over 50% of the search market share in Russian, and is well-known and used in many of the surrounding countries.

Ranking in Yandex

It is important to understand that, as a Russian (and primarily Russian-language) search engine, Yandex prioritizes Russian geography in local searches, and uses separate ranking algorithms for .ru domains compared to the rest of the web.

Compared to Google, Yandex’s indexing process is very different:

• Slower indexing process
• Pages larger than 10MB are not indexed
• URLs over 1024 characters are not indexed
• Language filtering:
  • “Automatic” indexing for sites containing pages in Russian, Ukrainian, and Belarusian
  • Content evaluation for sites in English, German, and French
  • Other languages not supported

Finally, not all indexed pages can appear in search. In fact, 3xx, 4xx and 5xx status codes, or redirections targeted only at robots can exclude a page from search results, even if it’s indexed.

To find out which of your pages are available, you can consult “Pages in search” in the Webmaster tools.

The ranking signals with the most weight for Yandex include the following:

• User behavior: “user behavior is a HUGE ranking factor and plays a much bigger role than other signals, such as link building.”
• Technical optimization
• Content completeness and uniqueness (Note that Yandex’s duplicate content filter is stricter than Google’s.)
• Localization (region)
• Commercial ranking factors, such as detailed information about the company, prices, conditions of service, or reviews
• Domain age and content freshness
• Keyword use, including meta keyword tags and presence of the keyword in the URL. However, the Palekh algorithm update for long-tail queries also introduced a neural network into Yandex’s query analysis.

If this begins to look like large, well-established e-commerce sites have a leg up, you’re not far from the mark.

Crawlability for Yandex

While a website needs to be crawled to be indexed and ranked on Yandex, keywords play a more important role than internal links. Consequently, internal site structure is less important than for Google.

Webmasters are encouraged to:

• Submit sitemaps
• Use static content, as the Yandex crawler does not support Javascript

Yandex’s bot examines the following HTML elements:

• Meta keywords: used to determine page relevance
• Meta description: used as the description in snippets (page text can also be used)
• Meta viewport: indicates responsive design compatibility
• Meta Content-Type : used to determine type and content of a URL
• Meta Refresh: can be used to redirect pages. This meta tag will redirect a visitor after N seconds
• Meta robots
• <noindex> tags for specific content
• Link rel=nofollow
• Link rel=canonical. This can be ignored if the canonical is not accessible, if the canonical is on a different domain or subdomain, or if multiple canonical URLs are provided.

Optimizing for Yandex

There are a few aspects where Yandex differs significantly from Google.

Hreflang

Hreflangs should be used for Yandex in the following cases:

• Page versions “differ only in the template language”, though the content, such as blog posts or forum comments, remains the same
• Page versions are in the same language but target different countries or regions
• Pages are “complete and accurate” translations
• Pages that redirect automatically by location or language should use x-default hreflang

Hreflang tags only work in the <head> or HTTP response; sitemaps are not supported.

Thematic Index Citation (TIC)

Yandex’s TIC used to rank sites directly; now, it has become a reflection of quality but doesn’t correlate perfectly to how well a site ranks. The TIC is a score from 0-150 000 that measures how well the site link to something that adds value to the user. Value is determined by high-quality content and subject-matter relevance.

Geo-targeting

Yandex encourages webmasters to specify the region their websites are targeting in the Webmaster tools.

Geo-dependent and geo-independent queries are treated differently and results not mixed on the SERPs. This means that if your business could profit from local search, geo-targeting is required.

To geo-target a page, you will need to:

• Publish your business’s name, address, postal code, telephone number, and area code on your website.
• Publish the same information for every regional branch of your business.
• Make every page of your website available to Yandex’s robot regardless of the robot’s IP address.

Anti-linkspam

Yandex has invested significant energy in fighting linkspam, which they define as using links created to influence search algorithms that may be bought or sold in an attempt to artificially increase a site’s credibility without improving the quality of the site.

The “Minusinsk” algorithm detects linkspam and can lead to severe penalties.

Mobile Search

Mobile search is growing rapidly in Russia. Yandex’s ranking criteria for mobile search are similar to Google’s.

However, it’s important to place a mobile version of the site on a separate subdomain or a separate domain:

Yandex treats the main version and the mobile version of the site as separate sites and indexes them separately from each other.
To help the robot identify the mobile version correctly, specify the URLs of the mobile pages on the corresponding pages of the main site. For example, you can use the <link> element.
[…] If the mobile version of the site is located in the main site directory, the Yandex indexing robot can’t index it correctly. We recommend that you create a mobile version in a separate domain or in the subdomain of the main site.

— Yandex Support

Yandex’s tools and technologies

Tools

• Keyword statistics
• Analytics
• Webmaster tools
  • Robots.txt analysis
  • Semantic markup validator
• Webmaster blog
• Log Analysis (recommended)

Technologies

Yandex has a number of operators that can be used to create more specific search queries:

• Check what is indexed for a site using the url query operator. In the Yandex search box, enter the url: query operator, site address and *. For example: url:www.example.com*.
• Title[keyword] – Search for a given keyword within a title tag, works the same way as Google’s intitle: operator.
• Inurl=”keyword” – Works the same way as Google’s inurl: operator.
• Mime=”html/pdf/doc/ppt/xls/rtf/swf” – Searches for specific file types, for example: seo << mime=”ppt” would return results that are PowerPoint files related to SEO.

Baidu (百度)

Baidu is China’s principal search engine. China’s communist regime means that this implies several important points: first, Google is not available in China. Second, Baidu is restricted to websites that are both in Chinese. Finally, government censorship does apply.

Ranking in Baidu

Like other search engines, Baidu’s index is created through crawling by the Baidu spider. Instead of simply following links, though URLs received through Baidu’s link submission tool are prioritized for crawling.

Baidu’s ranking algorithms do no discriminate against foreign sites. However, they only rank sites that can match queries in simplified Mandarin. This means content in Chinese is a requirement.

The simplified script and Mandarin grammar can be a handicap for regions that use the traditional script (Taiwan, Hong Kong) as well as regions that use other Chinese languages with the simplified script–though Baidu is still used by websites and users in both cases.

Other ranking factors include:

• Freshness
• Flat site structure (Pages closer to the home page are seen as more important, and internal link related elements such as broken links, redirections, and anchor text can have a direct impact on ranking. Orphan pages, for example, are often not indexed.)
• Few subdomains
• No Javascript (not supported by Baidu spider)
• Page speed
• Content quality
• Mobile friendliness
• Exact keyword matches in the title

Finally, like in Google’s SERPs, Baidu products are often placed before organic results. This includes ads and Baidu-owned pages.

Optimizing for Baidu

Page speed

Baidu’s algorithms give an enormous weight to page speed, to the point that websites need to be hosted in China to be competitive. Hosting in China is extremely complicated for foreign websites (you need an Internet Content Provider License, which in turn requires a Chinese business license…), so most foreign websites are required to rely on CDNs.

No matter how you host your website, though, page speed is essential, and the cutoffs are clear: the first screen of pages must load within 1.5 seconds.

Content quality

Content quality was not particularly important until recently; however, quality is a rising factor.

Page titles are extremely important. Titles should be clear, concise (irrelevant words can lead to penalties), and contain the page’s keywords (but stuffing is penalized).

Because of the local, Communist context, it is extremely important to check and validate all content before it is published.

On-page SEO

On-page SEO for Baidu looks at many of the same elements as on-page SEO for other search engines, but prioritizes different aspects.

Baidu’s instructions to webmasters include the following on-page rules:

• The first screen of your page should be loaded within 1.5 seconds
• Content should use a font size no smaller than 10pt
• The core content (the main part that excludes the navigation bar and any ads) of your website should take up at least 50% of the first screen.
• For mobile search results, the landing page should be mobile-friendly
• Mobile and UX rules should be respected
• The site should use HTTPS
• The page should include the following HTML tags: <title>, <meta name=”description”>, <meta name=”keywords”>, and headers (<h1>…)
• The page should link back to home page

Home page

Extra weight is given to the website’s home page, both in ranking algorithms and in on-page rules. In fact, home pages turn up more often in search results. Consequently, many Chinese sites that are optimized for Baidu contain few pages, preferring to place much of their content on the home page itself.

Baidu’s tools

Tools

• Baidu Webmaster Tools

DuckDuckGo

DuckDuckGo differs enormously from other search engines in both its principles and its technology. It was founded based on the idea that privacy should matter, and therefore does not adapt its results based on user profiling. It also does not use a crawler for indexing and ranking, but bases its SERPs on “sources”.

As a consequence, local searches tend to yield less pertinent results on DuckDuckGo. Even queries including locations are often not very accurate, but the search engine attracts a growing number of users with privacy and data concerns.

Ranking in DuckDuckGo

DuckDuckGo explains:

DuckDuckGo gets its results from over four hundred sources. These include hundreds of vertical sources delivering niche Instant Answers, DuckDuckBot (our crawler) and crowd-sourced sites (like Wikipedia, stored in our answer indexes). We also, of course, have more traditional links in the search results, which we source from Bing, Yahoo, and Yandex.
— DuckDuckGo Sources

This means that if your site is indexed on Bing, Yandex or Google, it will also be available on DuckDuckGo.

Because DuckDuckGo does not build its own index of sites from scratch but its own ranking algorithms, “the best way to get good rankings (in pretty much all search engines) is to get links from high quality sites.”

However, very little additional information on its ranking algorithms is available.

Optimizing for DuckDuckGo

Suggestions for optimizing for DuckDuckGo aren’t very different from optimization for other search engines. However, it has been suggested that repurposing content on multiple sources may help ranking on DuckDuckGo.

Other strategies for standing on on DuckDuckGo include the following:

• Suggest that DuckDuckGo add a bang for you website. Bangs (!websitename) provide quick direct access to your site
• Use structured data (Schema.org)
• Reinforce your backlink strategy
• Make sure your website is crawlable and indexable for other search engines

In conclusion

Depending on your target audience, Google may not be the be-all and end-all of your visibility needs. If you look beyond Google, though, you can find a broad universe of search engines, each with their own specificities.

Many share significant similarities with Google, either because they are competing for the same users, or because they are evaluating the same websites. In the case of Bing, it’s the little technical details that can make a big difference in rankings. Where this is not the case, however, search engines may function differently–and in turn, website creators and SEOs must adapt.

This is the case with Yandex and its regional settings, with Baidu and its China-hosted home pages, or with DuckDuckGo and its index built from indexes of many other sources.