Big Data

Big Data refers to data that is too large or complex to be analyzed by traditional software or human analysis and needs to use Artificial Intelligence and Machine Learning. Big Data is characterized by 3 Vs:

• Volume: The size of the data captured, stored and analyzed, typically running into Terabytes and Petabytes.

• Velocity: The speed or frequency at which the data is received, a factor that is especially relevant if the data needs to be analyzed in real-time.

• Variability: The number of data types the data may have and the number of different structures it may take.

The volume of data captured by computing systems today is growing exponentially with the increasing number of computing devices (including smart phone and sensors) all connected and constantly exchanging data over the Internet of Things.

It is not possible for humans to analyze this volume of data and AI and ML solutions are very widely used to not only analyze the huge volume of data in a short time but also derive more insights from the data than human analysis is capable of.

There are two main sources for Big Data: the Internet of Things and Social Media. IoT has billions of (and growing) sensors continuously capturing data about the environment around them and sending them to servers for analysis and decisions while Social Media has billions of people exchanging messages and information using text, images, video and audio, which can be stored for analysis wherever possible to get insights into consumer behavior.