Big Data Gets Bigger Every Minute

Big Data Gets Bigger

A lot can happen in a minute!

Peter Sweeney – Waterford Technologies

In the last 60 seconds around the world, there were 2.5 million posts on Facebook, 2 million searches on Google, 278,000 tweets on twitter and 204 million emails sent.

As more and more communication is done digitally, it all has to be stored somewhere. As cameras and smartphones get better resolution sensors, emails themselves with their photo (and other) attachments are growing at an exponential rate in terms of storage size.

Most people don’t think about this too much day to day but those attachments could be job applications, contracts for purchases, legal agreements and key company financial information as well as the company Christmas party photos.

From an IT perspective, you could say that as discs are relatively cheap now, let’s just keep adding more and more space for our email archive to grow.  But, there is a bigger picture and that involves not just the cost of managing that data and the impact on the performance of your email system as it gets bigger.

The increasing impact of global and national legislation on data retention and destruction must also be considered.  Backing up all of that data is one of the largest IT costs in any organisation.

So, there is a conflict. The safest route is to keep everything some may say.  Using email archiving can compress and deduplicate emails and attachments.  But the cost of responding to a freedom of information request where there is 20 years of data to access can be huge.

Big Data & Archiving

What companies need to do is to look at the numerous obligations on them in respect of data around compliance, data protection and regulatory body obligations and draw up an effective data management strategy that covers electronic data like emails.

To do that, you need to know what you have and decide on retention strategies and time-scales and an effective automatic implementation strategy, but one that has the flexibility to exclude items for the purposes of legal hold, should a matter come to litigation.

It’s a minefield that needs careful treading, but with an effective email archiving and reporting tool it’s one that all companies can manage.

If you would like to know more contact us.