- Use cases
- Customer Success
- LOG IN
- Start free trial
OnCrawl recently released a Splunk integration to facilitate log monitoring for Splunk users. We’ve found that companies use our Splunk integration for two main purposes: process automation, and elevated security control. But the advantages of the tool don’t stop there. Here are five ways you can use the OnCrawl Splunk integration to improve your technical SEO.
Your log files represent all activity on your website as recorded by the website’s server itself. It is the most complete and most reliable source of information regarding what happens on your website. This includes the number and frequency of bot hits, the number and frequency of SEO organic hits coming from SERPs, breakdown by device type (desktop vs. mobile) or by URL type (page vs. resource), precise page sizes and actual HTTP status codes.
Some of the many advantages provided by SEO log analysis:
Splunk is an enterprise solution for machine data aggregation. Capable of indexing and managing data from multiple sources at scale, it includes functions for server log processing for site security and reporting purposes.
Some of the advantages of Splunk:
Log monitoring in Splunk
Splunk users benefit from an OnCrawl integration to connect server log data managed in Splunk with SEO data in the OnCrawl platform.
Splunk proves the ability to aggregate, search, monitor, and set up alerting for log data. It parses and reindexes content in server logs. Using powerful search and filters, it provides answers to questions about the data in your log files. This processing step provides statistics for trends revealed in log data.
When you want to apply this to SEO data, however, it’s best to start with the raw data. And that’s exactly what the OnCrawl Splunk integration does.
Instead of just showing separate statistics for log data, this allows you to integrate information from logs with all of the other data sources in the OnCrawl platform. In turn, this allows you to examine relationships between SEO metrics and information on user and bot behavior in your log files.
Number of organic visits by page click depth.
This cross-data analysis can include axes useful to SEO:
Whether or not you need automation or finer controls for data security, if you’re a Splunk user, you’ll
like how simple it is to set up.
If you’re not a system administrator, setting up log monitoring for SEO can seem like a complicated task.
Our suggestion is to simply skip the hard parts. You can now set everything up directly in Splunk and use the key you generate to create the connection with OnCrawl.
That’s it. You’re ready to go. It couldn’t be easier.
The manual process of using log data collected in Splunk requires multiple steps:
This process must be repeated regularly to avoid gaps in your log data. Often this becomes a daily task.
If you choose to use the Splunk integration for OnCrawl, you no longer need to launch the task regularly. You only need to set up the process (and, as mentioned earlier, it couldn’t be easier). You no longer need to worry about launching a script–or, worse, a series of manual actions–every day; the integration handles that for you.
Protect yourself from losing data when there’s a problem. Because log monitoring relies on a continuous stream of data, gaps can lead to incorrect conclusions. You should never have to ask questions like: is the apparent absence of organic visits this morning due to something that happened on Google, or am I just missing the data?
The OnCrawl Splunk integration protects you in case your server is down or the connection is lost, and prevents human errors when you don’t have time or simply forget to upload data. If we can’t connect to the server, it won’t cause gaps in your data; we’ll just collect it a little later. If you find a set of data from an earlier date that you forgot to add to Splunk, the OnCrawl integration will also automatically pick it up.
At OnCrawl, we take the security of your data extremely seriously.
As always, sensitive data in your logs are kept where you put them in your private, secure FTP space and are never made available elsewhere. For example, the only personal data we process are IP addresses when validating the authenticity of Googlebot visits. We keep no record of the IP addresses used–only the result of the validation. If necessary, you can remove sensitive information made available for analysis at any time by removing files from your FTP space.
The integration for Splunk goes even further. We make sure you maintain control of your data throughout the process. You define the access rights, the data to be shared, and the frequency of the updates in OnCrawl. When you share data with OnCrawl through the Splunk integration, we use standard, secure protocols to communicate with Splunk, protected by a password and a key that you set up.
Because the setup is done in Splunk, OnCrawl never sees anything you don’t allow us to see. You choose what information you share with OnCrawl. Not only that, but since you manage the setup, if there are changes in your logging process or in your company’s standards, you can make changes at any time.