Dark Data the Good, the Bad and the Useful

Big Data continues to be a major driving force in organisational growth. The ability for organisations to harvest and harness the potential of Big Data can lead to the development of otherwise unforeseen innovation and process which can be a major resource in an organisations battle for market share.

Within the world of Big data lies a subset called “Dark Data”. Dark Data is acquired in vast quantities across organisational networks during the course of everyday business operations. It is generally unstructured, un-tagged and is found in repositories that have not been analysed or processed

Great, but what does this mean for your company?!

Dark Data is risky:

Dark Data brings with it risk and reward. AI and machine learning tools collect huge amounts of information most of which can be untagged and is positioned outside the realms of structured databases. Because of this, it is fertile hunting ground for hackers seeking to exploit personal data that is present on organisational networks. This lack of visibility on dark data means that in the event this information becomes compromised, the chances are you won’t have even realize until it’s too late.

In the darkness lies opportunity:

In spite of the very real risks that are associated with retention of dark data. More and more corporate are starting to see that this form of data has the potential to be harnessed and used as a valuable resource. We are seeing major organisations look at how this unstructured and often random data can be analysed to bring forth meaningful insights. We are starting to see sales teams for example use dark data to find critical information such as customer preferences and regional specific criteria,  helping companies deliver more targeted marketing campaigns and sales strategies. It is believed that adopting a strategy to use dark data and derive meaning from it can lead to CAGR of 20%.


So what now:

It is important that organisations realize that, yes, dark data is a risk to organisations and retention/redundancy  strategies need to be implemented in order to guard against these dangers. However, it also needs to be seen as a major opportunity. Data that lies out of sight in an unstructured manner, it IS still data and can have many potential uses.

Organisations need to first concentrate on visibility, its only once this information is surfaced and we know what unstructured data we have, can we begin to then decide whether it can be used or disregarded. Tools need to be in place in organisations that find this data, once this occurs the next step is analyzing and finding trends. Cost benefit analysis is paramount in establishing the potential benefits in evaluation of the data as opposed to the cost of the tools and man hours required.

Finally, once the information has been gathered data analysis tools needs to be applied to establish the tangible leanings and presented to departmental decision makers.


Check out PixAlert’s Critical Data Audit Tool to understand how we can help you  find “Dark Data”

Niall Kelly