Our crawl over crawl last feature allows you to compare two different crawls and stresses out crawl evolutions.
It follows our last release about ‘Trends’ that gave you the opportunity to spot global trends between different crawls. Here, you can access complete views of your SEO improvements and highlight differences between crawls on a given theme. We have thus included new types of graphs to read your data.
How to read a sunburst?
You read a sunburst like a traditional pie. These graphics are very useful to follow the evolution of a website, crawl after crawl or to check differences between two versions of a website (between a live version and during a restructuration for instance).
This multi-level pie chart lets you compare two crawls depending on a given theme:
- The first level and inner circle : shows the pages belonging to the first crawl (the older crawl).
- The second level and outside circle : shows the pages of the second crawl (the newest one) according to each segment of the inner circle.
So you can easily find, for example, indexable pages in the first crawl that are not anymore in the second one and vice versa.
In this chart, the inner circle shows the repartition of pages from the first crawl point of view (the older one). You can see that there are indexable pages, no indexable pages and pages that were not in the first crawl but appear in the second (the grey section).
Then, for each section of the inner circle, you can see the pages’ repartition of a given section in the second crawl. The inner grey section means that those pages did not exist in the first crawl but appear in the second one (the outside green and red section belonging to the inner grey one).
Grey sections mean pages are whether new or not existing in the structure depending on which cercle they belong.
By clicking on the legend, you can decide which data you want to display or focus on. Crawl 2 offers a more global view.
Let’s have a look at the inner circle.
The distribution of pages in the first crawl according to their indexability
The first crawl contains 10 854 indexable pages and 177 no indexable pages. 1 661 pages have only been found in the second crawl.
Now have a look at the outside circle. For each segment of the first circle, we find the distribution of these pages in the second crawl.
Within the 10 854 indexable pages in the first crawl, only 10 104 are still indexable in the second one, 560 are now non indexable in the second one and 190 are not in the second crawl anymore.
Let’s focus on a small section : non indexable pages in the first crawl
Within the 177 non indexable pages from the first crawl, 94 are now indexable in the second one and 13 remain indexable.
Within the 177 non indexable pages in the first crawl, 70 are not located in the second crawl. 94 + 13 + 70 = 177. We find as expected the repartition of the 177 non indexable pages found in the first crawl.
Focus on new pages : pages only found in the second crawl
It allows you to see the status of new pages in terms of indexation.
All new pages: 1 661 pages.
Within the 1 661 newly created pages, 709 are not indexable.
Within the 1661 newly created pages, 952 are indexable.
Summary : all pages from second crawl
10 104 pages were indexable in the first crawl. 11 150 are now indexable in the second one. 177 pages were non indexable in the first crawl but 1 282 are now non indexable in the second one.
1661 pages have been created. 260 pages have been deleted from the structure.
Crawl over crawl available data
This new feature is divided by business expertises and between the following tabs:
- Internal linking
- Status Codes
For instance, in the ‘Content’ section, you acccess a strong focus on your duplications’ evolution between crawls:
Also, you can analyze how your page depth has evolved crawl over crawl. In the graph below, you can see depth movements:
For example, if we look at depth 5, we can see pages that are gone to shallower or deeper depth or pages that come from shallower or deeper depth between crawl 1 and 2. Here, 264 pages that were in crawl 1 and in depth 5 have gone to a shallower depth (depth 4, 3 or 2).
This is just a snapshot of what is available. Our Data Explorer also lets you dig into more than 300 metrics to run crawl comparisons.
How to get our Crawl over Crawl feature?
You need to have at least a Business plan to enjoy our Crawl over Crawl data. You can do unlimited comparaison inside a same project and between a crawl and its previous one with identical start URLs.
By ticking the dedicated checkbox in your crawl parameters, you can compare the crawl you are about the launch with the last one.
You can also check that your crawl configuration is the same by clicking on ‘crawl config comparison’:
Also, when running a crawl over crawl, you should do complete crawls if you want to get accurate results. You can’t compare crawls and get reliable results if you don’t compare all URLs.
For the moment, this feature is in beta so you will need us to activate it manually to start using it. So just ask!