Accueil > Oncrawl’s Technology & Data

Oncrawl’s Technology & Data

Oncrawl uses reliable, secure and auto-scalable technology to meet your SEO needs.

rocket

Auto-scalable and cloud-based storage

Oncrawl has an unlimited storage capacity thanks to the most powerful cloud in the world: the Google cloud. Our solution also has access to many data centers across the world to ensure robust and reliable storage.
Data hosting in the Google cloud allows us to adjust our storage according to the volume of data and the number of customers using Oncrawl. In other words, Oncrawl adapts to your needs for storage limits and processing power.
Our solution is auto-scalable: our clusters grow depending on their activity and the allocated resources vary according to the data handled in real time

row

Unlimited crawling capacity

The technology used by Oncrawl allows us to quickly crawl millions of URLs. The only limit is the efficiency of our clients’ infrastructure. Oncrawl is able to adapt its capacity to the desired speed by our customers and adopt a “safe” mode. Currently, we collect up to 300 millions of URLs per day and per website, and 500 million log lines per day. There’s no crawl we can’t handle and we’ve been able to analyze very large JavaScript websites where our competitors didn’t manage to complete these crawls.

heart-care

Secure data system, CCPA & GDPR compliance

Oncrawl is designed to match enterprise-level security requirements, respect user privacy, and comply with GDPR and CCPA requirements. For our European clients, we can ensure data storage on the European continent. We have set up a policy for restricting internal access to data. Every log file that we collect is stored in our isolated and secure FTP space. Our service is protected by the HTTPS protocol and by isolated software components. It also profits from extra protection from a software stack that we maintain up to date.

clock

Real-time crawling and log monitoring

With Oncrawl, you can check the progress of your crawls and the analysis of your logs in real time. In order to facilitate your work, we have a notification system which keeps you informed of the progress of your SEO analysis. You can check on the status of your data at any moment.

Oncrawl Developer picto

Go further with the access to our API

Oncrawl provides a SaaS app which offers a complete analysis of your data. We know that some of our users would like to build their own dashboards with raw data from Oncrawl and others sources. That is why we offer complete access to our API. Our solution gives total freedom to users who are interested in integrating Oncrawl’s data into their own ecosystem. Thanks to the API, you can go even further with the Oncrawl data!

Learn more about Oncrawl Developer →

The technologies behind the scenes at Oncrawl

Oncrawl is built and run on a set of reliable technologies.

And other secret ingredients…