Crawlability: Crawling like it’s 2019


What’s inside?

OnCrawl teamed up with experienced SEOs to bring you the ebook Crawling like it’s 2019: How to run a site that search engines love to crawl (Ensuring websites meet search engine requirements for crawlability to boost SEO performance).


At the heart of how search engines do their work, crawlability is one of the major–but often overlooked–issues in SEO. This ebook discusses why a website’s crawlability matters, the SEO elements that have an impact on whether or not a site can be crawled by search engines, and the tools you need to measure and optimize crawlability.


Learn what search engines need, and take advantage of our contributors’ years of SEO experience covering contexts including in-house, agency, and freelance posts, industry participation as speakers and judges, and work with well-known giants, from Google to Ebay to Canon.


Download the ebook


  • Introduction
    • The Experts
  • The basics: why talk about crawlability?
    • How search engines work
    • How websites adapt to search engines
  • Elements that influence crawlability
    • Site architecture and internal link structure
    • Page depth
    • Bot accessibility
    • Page speed
    • Use of JS, Flash, and dynamic content
    • Redirects
  • How to measure crawlability on your site
    • URL Inspection Tools
    • Crawlers
  • OnCrawl
    • Using OnCrawl to optimize crawlability
    • About OnCrawl


The experts

In alphabetical order, these SEOs have shared their experience and knowledge on crawlability.


 Aysun Akarsu  @aysunakarsu

Aysun Akarsu is a trilingual data scientist specialized in machine intelligence for digital marketing wanting to help companies in making data driven decisions for reaching a broader, qualified audience.

Her experience includes managing long term global SEO projects for internationally well-known brands such as Nokia or SFR. She has proficiency in analysis of the internal and external search data. Internally: crawls, web server log files, analytics tools e.g. Google Analytics, Search Console, PPC, internal search data and databases; externally: links, crawls (of other websites), SERPs, APIs. She has expertise not only analyzing each of them but also crossing all these data to get powerful insights. She is skilled at analyzing big search data on huge web sites: Ebay, Viadeo, Kelkoo, Homeaway, Izlesene, among others.

Aysun is a technical SEO speaker at leading events e.g. TechSEOBoost USA, BrightonSEO UK. She writes regularly about search data analysis at SearchDatalogy.


murat Murat Yatağan @muratyatagan

Murat Yatağan is an SEO Consultant with more than 9 years of expertise on SEO and UX. He currently works at Brainly as VP of Growth.

His qualifications include 5 years as a Senior Product Analyst at Google Search Quality Team (now Trust & Safety), 2 years as Senior SEO Manager at Global Savings Group, his role as an Advisory Board member at San Francisco-based startup Indisera, and his participation as Judge for UK Search Awards, MENA Search Awards and European Search Awards and US Search Awards.


 Omi Sido  @OmiSido

Omi is a seasoned international speaker and is known in the industry for his humour and ability to deliver actionable insights that audiences can immediately start using. From SEO consulting with some of the world’s largest telecommunications and travel companies to managing in-house SEO at HostelWorld and Daily Mail, Omi loves diving into complex data and finding the bright spots. Currently, Omi is a Senior Technical SEO at Canon Europe.





Download the ebook