If you use Oncrawl every day, or at least every week, you know that we have a native connector for Majestic. This allows us to retrieve via their API all the backlink information for each URL, as well as the Trust Flow and Citation Flow of the site. Every time you launch a crawl, the data is updated.
However, if you’re more of an Ahrefs user, it’s still possible to integrate their data into Oncrawl using our Data Ingestion feature.
What is Data Ingestion?
Data Ingestion is a tool that allows you to add any data into Oncrawl via simple CSV files. The only requirement is to aggregate the data by URL. This is a tool that is underestimated by our customers, but whose possibilities for integrating third-party data are enormous. It is possible to integrate for example:
- Turnover data generated by URL
- Margin data
- Conversion data/fulfilled targets
- Backlinks data
- Google Ads spend data
- Internal scoring metrics
Integration of Ahrefs backlink data
Step 1: Export Ahrefs data
Here we will focus on integrating backlinks data from Ahrefs. To do so, we will export the data from the “Inbound Links” report. Simply click on the “Export” button, choose your export (quick and limited to 1000 rows, or full), then click on “Start export”. You get a CSV file.
Step 2: Prepare the CSV file
Now that we have the Ahrefs export, it is time to prepare the file for integration in Oncrawl. Here are the steps to prepare your file:
- Import the CSV file into Excel with UTF-8 encoding
- Rename the ‘Link URL’ column to ‘URL’. This column is mandatory
- Delete the columns you are not interested in. In our example, we will keep :
- Link URL
- Total Backlinks
- Domain Rating
- URL Rating
- Keywords
- Traffic
As it is impossible to have several distinct values for each URL, I will create a pivot table with the data to keep. I will calculate for each URL the sum of Total_Backlinks and Traffic, the average for URL_Rating, Domain_Rating and Keyword. You can do it differently depending on your needs. Here is what my excel file looks like:
Once the file is ready and cleared, save it with only data in CSV format, encoded in UTF-8, then compress your CSV file in .zip. Feel free to read this article to learn all the intricacies of using CSV files with Oncrawl. You must respect all elements to avoid errors when importing into Oncrawl.
Step 3: Import into Oncrawl
To import your file into Oncrawl, go to the “Add data sources” section, then to the Data Ingestion tab. You can upload your file, it will be analyzed and processed by Oncrawl.
The second step is to launch a crawl by linking to this file. To do this, go to the settings of your crawl, then to the “Data Ingestion” tab. Turn on the option and select your file dropped above.
Everything is now ready and Oncrawl will cross-reference the data from the crawl with the data in your CSV file. At the end of the crawl, you will have new data that you can use via the Data Explorer or via the segmentations. If you want to update your Data Ingest data, you just have to upload a new file with the updated data, and to restart the crawl with this new file.
[Ebook] How your internal linking scheme affects Inrank
How to use this new data?
Use the Data Explorer
The easiest way to use this data is to go to the Data Explorer. There you will find new fields related to the data you have just ingested into Oncrawl.
Use an OQL (Oncrawl Query Language) to access the data. For example, list the pages that have backlinks and that respond in 3xx.
Use segmentations
If you prefer to visualize the data with our graphs, then using segmentations with ingested data is preferable.
So create new segmentations based on your new fields. For example, I’ll create a segmentation with the “Total_Backlinks” field that lists pages based on the backlinks they have.
I can now use this segmentation, which categorizes pages based on the backlinks they have, in any part of the app, like here with the distribution of status codes by backlink range.
I see that I have 13 pages that respond in 301 and have between 25 and 50 backlinks. This will help me prioritize SEO actions.
Another example, do we have pages with backlinks, which make few internal links in follow? This allows me to check if the pages that receive backlinks distribute the juice or keep it.
Here is the link of our drive to download these segmentations: https://drive.google.com/file/d/102Ncrs0rc8vi-OskveaHzicXmwagpAdA/view?usp=sharing
Use custom dashboards
You will also be able to use our custom dashboards in order to have graphs only dedicated to Ahrefs data. You will find data on status code, orphaned pages that have backlinks, and the impact of backlinks on your SEO traffic.
If you followed the procedure in this article to integrate the Ahrefs data, if you used the same field names, then the segmentations and custom dashboards will work without modifications.
Here is the link to our drive to download the segmentations and custom dashboards: https://drive.google.com/file/d/102Ncrs0rc8vi-OskveaHzicXmwagpAdA/view?usp=sharing
Takeaways
As you can see, using Data Ingestion is quite simple and relies on the use of CSV files. In our example, we have seen how to add data from Ahrefs, but it is quite possible to integrate any external data source with Oncrawl. The advantage is to cross-reference the data with all other data sources such as crawl, logs, Google Analytics and Google Search Console. Don’t hesitate to use this free Oncrawl feature, which is very underestimated, but has enormous potential!
