Why did we build Oncrawl?

October 15, 2014 - 2  min reading time - by François Goube
Home > Product updates > Why did we build Oncrawl?

As an in-house SEO guy or an SEO consultant, I’ve always been surprised about how badly we were looking at OnPage SEO focusing on building quality link profiles. However we can do a lot more with what we have if we have the right data. We can do even more monitoring your data about onsite SEO and understanding what’s good and what can hurt your rankings. Before digging more about what’s Oncrawl, let me give you my thoughts before we built it.

We are weak… so is our onpage optimisations.

We are just humans, tracking our onpage changes is really a pain. When it comes to large websites it’s barely impossible, most of all if you run websites with user generated content (hello marketplaces !). We don’t have an exhaustive overview of our content – product pages, listings, how do they perform inside my website? Are they popular? Do they get the right amount of juice? – It’s not humanly possible. You can just guess.

And Google Webmaster Tools sucks.

A few years back you could still rely on Google Webmaster Tool, but we all know it does not give you an exhaustive overview, neither access to an up-to-date dataset. My problem was always to go further than just « ok I have duplicate title tags, but what else? ». How all my Hn tags are performing? How can I be sure that my top products pages are well crawled, indexed and ranked by Google? GWT doesn’t tell me that. So I deeply need something more « actionnable ».
When I first met my partner @Tuxnco who was working at Exalead (a French competitor of Google) he explains me all the data they used to understand a website, to score its quality. We decided these data should be widely available to all web workers. As we were really good at crawling the web, we tried to figure out how we could help the seosphere. So we came up with Oncrawl.

Oncrawl will make it easy to track your onpage SEO

We are about to unleash a kind of super-power for SEOs out there. We will give you access to the most comprehensive website crawler for SEO. Oncrawl is built on top of Nutch and ElasticSearch, and with all our love and a 10 years experience in Natural Language Processing. If you want to join our beta please do! We will be happy to showcase you our work and talk about our next steps.

All the best (rankings)

@FrancoisGoube – Founder @Oncrawl.

François Goube See all their articles
@FrancoisGoube is the founder of @Oncrawl. He has over 15 years industry experience and has a weird passion about semantic algorithms. More about him here.