• Sapphire

Functional Layers in the Data Eco System

Amity University, Lucknow, Feb 2019



Mamta Swaroop

Director (R&D, Operations)

Sapphire Application Integrators Pvt Ltd, Delhi




Corporate world changed its advertising bylines


From

“Enterprise Solutions”

to

“Analytics driven Enterprise Solutions”




Idea is to fuel


Risk Reduction

Product / Service Betterment

Assist Researches

Predict Results




What makes us to move towards Big Data


Ever increasing Volumes

Mixed Up Variety

Unstoppable Velocity




Datafication of World


Uber owns no Vehicles

FB generates no Content

Alibaba has no Inventory

AirBnb owns no Real Estate



How the Ecosystem has evolved over a period of time


Statistical Concepts

IT Computing

Data



Which layer is your Cup of Tea ??





Data Layers – (Storage)

Quantitative-Qualitative, Structured-Semi-Structured-Unstructured, Machine readable/Not readable, Digital-Analog, Personal-Public

Log files, Social Media data, Audio, Video, Word docs, PDFs, Medical Records

HDFS – Data File System, MongoDB – Document based Database, Amazon S3 – Data Buckets



Ingestion & Integration Layers – (Integration)


Data Warehouses

Pull Data from multiple sources

Bring some sort of uniformity

Stich, Blendo, Kafka….. etc are the tools


Data Processing Layer – (Fetching Information)


Platform that allows firing Queries On

Unstructured data

Semi Structured data

Structured data

Map Reduce – break a bigger problem into atomic problems; fetch atomic results & integrate

Document based DB – Document holds unstructured data

Spark, SQl, Redshift etc… are the tools


Analytics Layer – (Business Intelligence)

Using generated data in Decision Making

Query – Milliseconds Vs Analysis - Minutes

Patterns and Correlations can be established between random events in the world, which might not seem correlated at all

Testing hypotheses established

Causality can be understood

In other words –

Predictions can be made

Deep insights can be gained

Machine learning can be applied

Artificial Intelligence can be implemented

Excel and Crystal Reports fail here.

Tableau, Chartio, Looker etc are the tools

Does the Analysis changed / affected / impacted your Decision?




Recent Posts

See All

TELCO's Pie

Tonse Blog, Sep 2015 Mamta Swaroop mswaroop@hotmail.com “Human beings in a surveyed urban environment are each surrounded by 1000 to 5000 track-able objects.” You might not like to disclose your locat

©2019 by Sapphire Application Integrators Pvt. Ltd.