By Darren Hockley is MD of eLearning provider, DeltaNet International
It can be hard to understand exactly what ‘Big Data’ is since the term, by definition, refers to very complex data sets so vast in scope that traditional data systems (e.g. data warehouses and databases) fail to support it.
In reality, big data is exactly what it sounds like: a whole lot of data – more than you can probably imagine – and it’s growing exponentially.
Today, each and every internet user creates increasingly large amounts of data – reportedly combining into a massive 2.5 quintillion bytes of data every day. To put such an unthinkable number into perspective, this includes over 204 million emails per minute, 65 billion WhatsApp messages per day, and 1.2 trillion Google searches per year.
Every day, internet connected cars produce 4 terabytes of data (one terabyte = 1,000,000,000,000 bytes) and online wearables generate 28 petabytes (that’s 28,000,000,000,000,000 bytes). Indeed, it’s estimated that the digital universe will reach 44 zettabytes in total in 2020 – one zettabyte is one sextillion bytes … suffice to say, we’re talking very big data here.
Think about your own use of data-driven technology; it’s likely you own a smartphone or use GPS, for example. Perhaps you track your fitness levels with a wearable or log sleep patterns or other healthcare on an app. Maybe you browse social media or make use of mobile-banking, contactless payments, or online food delivery.
Many of us use video streaming platforms, order taxis online, or share photos with friends and family using a messaging service – the list seems endless, but all of these have one thing in common: the accumulation of data.
The more data digital service providers have on us, the better or more ‘intelligent’ their product generally is. Each time apps are updated, or new devices released, the new version includes some sort of increased functionality. Perhaps it’s more intuitive or easier to navigate, or maybe it just seems to know what you like, making helpful and relevant suggestions.
Of course, all of this comes at a price: data. Your data.
The 3Vs of Big Data
The sheer amount of data gathered across multiple internet-enabled devices known, collectively, as the Internet of Things (IoT) is a result of three elements. These are the ‘3Vs’ that are said to characterise Big Data:
- Volume – The IoT is growing exponentially year on year, as are the amount of internet users and, subsequently, the amount of data available for analysis.
- Variety – Data doesn’t just refer to numbers and statistics as it did once upon a time. Big Data is made up of structured, semi-structured, and unstructured data, including (but not limited to) non-numerical data like emails, pictures, and voicemails.
- Velocity – velocity refers to both the speed at which data is generated (which, we know, is enormous) and the speed at which this data may be processed. It’s worth noting that advancements in technology mean data can now be analysed pretty much in real time.
Using Big Data Ethically
Recent media reports focusing on the use of our data, e.g. the Facebook/Cambridge Analytica scandal, have lifted the lid somewhat on the ways our data is used and how valuable it is to all sorts of organisations.
Whilst Facebook took a hit following the exposé (15 million Americans are reported to have left the platform since 2017), many, many more remained on the platform, and the company’s profits haven’t really been dented. The truth is many people understand – feel comfortable even – with the knowledge that what we do online is useful ‘business intelligence’. We’re used to cookies tracking our browsing history, or third-party apps asking for access to our social media profiles. It’s nothing new.
Still, the big data revolution raises hard questions relating to privacy, confidentiality, transparency, and identity.
We may feel comfortable sharing our photographs and memories with Facebook, but many were (rightfully) outraged at the platform’s involvement and influence upon election campaigns. It’s great to use banking apps and other fintech on our phones, but how do you feel being profiled based on what magazines you buy or what you like to eat? Even who you associate with? What if this meant you were refused a loan or given higher interest rates on your credit card?
With so much digital information available for firms to use and analyse, it’s imperative that regulatory bodies and data protection legislation keep up. New directives, like the EU’s General Data Protection Regulation (GDPR) in 2018, are necessary to update data protection legislation for an increasingly digital, increasingly automated world.
Set to become the global standard for data protection, GDPR significantly enhanced and clarified the rights of the individual when it came to automated data processing and profiling, stressing the need for transparency about how companies plan to use our data, and placing the burden of accountability and compliance firmly with data controllers (the organisations that own data).
Moving beyond regulatory requirements, organisations that store and process personal data would do well to think of using big data ethically as the competitive advantage you didn’t know you had.
Maintaining strict transparency about your collection and use of data sends trust signals to new and existing customers, letting them know you take their privacy seriously. It also lets your employees know they work for a respectful, trustworthy company with a sound data privacy culture. Employees who feel proud to work at their organisation tend to stick around longer, recommend your services, and treat their work seriously.
GDPR laid out seven principles for data protection that are useful for any organisations looking to use big data in a safe, ethical way. These include:
- Processing data in a manner which is lawful, fair, and transparent and which maintains the data subject’s rights.
- Processing data onlyfor the purpose it was collected – if your purposes change over time, or you have a new purpose which you did not originally anticipate, you may need to seek new consent for processing data.
- Limiting the storage of data only to that which is strictly necessary and relevant. In the case that excessive data is (or has been) collected, the data should not be used and should be destroyed securely.
- Maintaining data records that are accurate and up to date. Where any personal data is found to be inaccurate, reasonable steps must be taken to ensure that such inaccurate data is deleted or rectified without delay.
- Storing personal data only for as long as is necessary. Companies must not hold on to personal data ‘just in case’.
- Processing and storing data with integrity. Every reasonable measure should be taken to maintain the security and confidentiality of data and to prevent unlawful processing, loss, destruction, or damage of data.
- Maintaining a culture of accountability. Data controllers are responsible for and must be able to demonstrate compliance with, data protection laws.
About the author
Darren Hockley is Managing Director at DeltaNet International. The company specialises in engaging compliance and health and safety eLearning designed to mitigate risks and improve employee performance.
DeltaNet International work with organisations around the globe to help them reach their key business objectives, offering off-the-shelf eLearning packages, as well as tailored and bespoke projects.Our in-house designers and developers use a mixture of cutting-edge design techniques to bring important legislation and best working practices to life.
Building upon more than twenty years’ experience in the industry, DeltaNet International was formed to re-invigorate how eLearning content is designed and delivered with the goal of improving both its effectiveness and usability.