• Sapphire

Functional Layers in the Data Eco System

Amity University, Lucknow, Feb 2019

Mamta Swaroop

Director (R&D, Operations)

Sapphire Application Integrators Pvt Ltd, Delhi


Corporate world changed its advertising bylines


“Enterprise Solutions”


“Analytics driven Enterprise Solutions”

Idea is to fuel

Risk Reduction

Product / Service Betterment

Assist Researches

Predict Results

What makes us to move towards Big Data

Ever increasing Volumes

Mixed Up Variety

Unstoppable Velocity

Datafication of World

Uber owns no Vehicles

FB generates no Content

Alibaba has no Inventory

AirBnb owns no Real Estate

How the Ecosystem has evolved over a period of time

Statistical Concepts

IT Computing


Which layer is your Cup of Tea ??


Data Layers – (Storage)

Quantitative-Qualitative, Structured-Semi-Structured-Unstructured, Machine readable/Not readable, Digital-Analog, Personal-Public

Log files, Social Media data, Audio, Video, Word docs, PDFs, Medical Records

HDFS – Data File System, MongoDB – Document based Database, Amazon S3 – Data Buckets

Ingestion & Integration Layers – (Integration)

Data Warehouses

Pull Data from multiple sources

Bring some sort of uniformity

Stich, Blendo, Kafka….. etc are the tools

Data Processing Layer – (Fetching Information)

Platform that allows firing Queries On

Unstructured data

Semi Structured data

Structured data

Map Reduce – break a bigger problem into atomic problems; fetch atomic results & integrate

Document based DB – Document holds unstructured data

Spark, SQl, Redshift etc… are the tools

Analytics Layer – (Business Intelligence)

Using generated data in Decision Making

Query – Milliseconds Vs Analysis - Minutes

Patterns and Correlations can be established between random events in the world, which might not seem correlated at all

Testing hypotheses established

Causality can be understood

In other words –

Predictions can be made

Deep insights can be gained

Machine learning can be applied

Artificial Intelligence can be implemented

Excel and Crystal Reports fail here.

Tableau, Chartio, Looker etc are the tools

Does the Analysis changed / affected / impacted your Decision?

2 views0 comments

Recent Posts

See All

Tonse Blog, Sep 2015 Mamta Swaroop mswaroop@hotmail.com “Human beings in a surveyed urban environment are each surrounded by 1000 to 5000 track-able objects.” You might not like to disclose your locat