Amity University, Lucknow, Feb 2019
Mamta Swaroop
Director (R&D, Operations)
Sapphire Application Integrators Pvt Ltd, Delhi
Corporate world changed its advertising bylines
From
“Enterprise Solutions”
to
“Analytics driven Enterprise Solutions”
Idea is to fuel
Risk Reduction
Product / Service Betterment
Assist Researches
Predict Results
What makes us to move towards Big Data
Ever increasing Volumes
Mixed Up Variety
Unstoppable Velocity
Datafication of World
Uber owns no Vehicles
FB generates no Content
Alibaba has no Inventory
AirBnb owns no Real Estate
How the Ecosystem has evolved over a period of time
Statistical Concepts
IT Computing
Data
Which layer is your Cup of Tea ??
Data Layers – (Storage)
Quantitative-Qualitative, Structured-Semi-Structured-Unstructured, Machine readable/Not readable, Digital-Analog, Personal-Public
Log files, Social Media data, Audio, Video, Word docs, PDFs, Medical Records
HDFS – Data File System, MongoDB – Document based Database, Amazon S3 – Data Buckets
Ingestion & Integration Layers – (Integration)
Data Warehouses
Pull Data from multiple sources
Bring some sort of uniformity
Stich, Blendo, Kafka….. etc are the tools
Data Processing Layer – (Fetching Information)
Platform that allows firing Queries On
Unstructured data
Semi Structured data
Structured data
Map Reduce – break a bigger problem into atomic problems; fetch atomic results & integrate
Document based DB – Document holds unstructured data
Spark, SQl, Redshift etc… are the tools
Analytics Layer – (Business Intelligence)
Using generated data in Decision Making
Query – Milliseconds Vs Analysis - Minutes
Patterns and Correlations can be established between random events in the world, which might not seem correlated at all
Testing hypotheses established
Causality can be understood
In other words –
Predictions can be made
Deep insights can be gained
Machine learning can be applied
Artificial Intelligence can be implemented
Excel and Crystal Reports fail here.
Tableau, Chartio, Looker etc are the tools
Does the Analysis changed / affected / impacted your Decision?