AWS CSA
  • Introduction
  • 1. AWS Overview
    • 1.1 AWS global Infrastructure
    • 1.2 Networking and Content Delivery
    • 1.3 Computing
    • 1.4 Storage
    • 1.5 Databases
    • 1.6 Migration Services
    • 1.7 Analytics
    • 1.8 Security and Identity
    • 1.9 Management Tools
    • 1.10 Application Services
    • 1.11 Developer Tools
    • 1.12 Mobile Services
    • 1.13 Desktop and App Streaming
    • 1.14 Artificial Intelligence
    • 1.15 Messaging
  • 2. IAM - Identity Access Management
    • 2.1 IAM Introduction
    • 2.2 IAM Labs
    • 2.3 Create a billing alart
    • 2.4 IAM Conclusion
  • 3. S3 - Simple Storage Service
    • 3.1 S3 Introduction
    • 3.2 S3 Labs
    • 3.3 Version Control Labs
    • 3.4 Cross Region Replication
    • 3.5 Life Cycle Management
    • 3.6 CloudFront
    • 3.7 CloudFront Labs
    • 3.8 S3 Security and Encryption
    • 3.9 Storage Gateway
    • 3.10 Snowball
    • 3.11 Snowball Labs
    • 3.12 S3 Transfer Acceleration
    • 3.13 Create a static web page using S3
    • 3.14 Storage Summary
  • 4. EC2 - Elastic Compute Cloud
    • 4.1 EC2 Introduction I
    • 4.2 EC2 Introduction II
    • 4.3 EC2 Lab 1
    • 4.4 EC2 Lab 2
    • 4.5 Security Groups Lab
    • 4.6 Upgrade EBS Volume Types - Lab1
    • 4.7 Upgrade EBS Volume Types - Lab2
    • 4.8 RAID, Volumes & Snapshots
    • 4.9 Create An AMI - Lab
    • 4.10 AMI's EBS Root Volumes vs. Instance Store
    • 4.11 Elastic Load Balancer & Health Check - Lab
    • 4.12 CloudWatch EC2 - Lab
    • 4.13 The AWS Command Line & EC2
    • 4.14 Using IAM Roles with EC2
    • 4.15 S3 CLI & Regions
    • 4.16 Using Bootstrap Scripts
    • 4.17 EC2 Instance Metadata
    • 4.18 Launch Configurations & Auto Scaling Groups
    • 4.19 EC2 Placement Groups - Exam Must Know
    • 4.20 Elastic File System (EFS) Lab
    • 4.21 Lambda Concepts
    • 4.22 Build A Serverless Webpage Using API Gateway & Lambda
    • 4.23 Use Polly to Pass Your Exam - Lab of a Serverless Application
    • 4.24 Use Polly to Pass Your Exam - Lab of a Serverless Application
    • 4.25 EC2 - Summary & Exam Tips
  • 5. Route53
    • 5.1 DNS 101
    • 5.2 Route53 - Register A Domain Name - Lab
    • 5.3 Setup Our EC2 Instances - Lab
    • 5.4 Routing Policies - Lab
  • 6. Databases on AWS
    • 6.1 Databases 101
    • 6.2 Launching an RDS Instance - Lab
    • 6.3 RDS - Backups, Multi-AZ & Read Replicas
    • 6.4 DynamoDB
    • 6.5 Redshift
    • 6.6 ElastiCache
    • 6.7 Aurora
    • 6.8 Database Summary
  • 7. VPC
    • 7.1 VPC Overview
    • 7.2 VPC Lab - Part 1
    • 7.3 VPC Lab - Part 2
    • 7.4 NAT Instances & NAT Gateways
    • 7.5 Network Access Control Lists vs. Security Groups
    • 7.6 Load Balancers & Custom VPCs
    • 7.7 VPC Flow Logs
    • 7.8 NATs vs. Bastions
    • 7.9 VPC End Points
    • 7.10 VPC Clean Up
    • 7.11 VPC Summary
  • 8. Application Services
    • 8.1 SQS - Simple Queue Service
    • 8.2 SWF - Simple Workflow Service
    • 8.3 SNS - Simple Notification Service
    • 8.4 Elastic Transcoder
    • 8.5 API Gateway
    • 8.6 Kinesis 101
    • 8.7 Kinesis Streams Lab
    • 8.8 Application Services Summary
  • 9. White Paper Reviews
    • 9.1 Overview of AWS
    • 9.2 Overview of Security Processes - Part 1
    • 9.3 Overview of Security Processes - Part 2
    • 9.4 Risk & Compliance Whitepaper
    • 9.5 Storage Options in the Cloud Whitepaper
    • 9.6 Architecting For The Cloud Best Practices Whitepaper
  • 10. Well Architected Framework
    • 10.1 Introduction of Well Architected Framework
    • 10.2 Pillar One - Security
    • 10.3 Pillar Two - Reliability
    • 10.4 Pillar Three - Performance
    • 10.5 Pillar Four - Cost Optimization
    • 10.6 Pillar Five - Operational Excellence
    • 10.7 Well Architected Framework - Summary
  • 11. Additional Exam Tips
    • 11.1 Exam Tips Based On Student Feedback
    • 11.2 Consolidated Billing
    • 11.3 AWS Organizations - Lab
    • 11.4 Cross Accounts Access
    • 11.5 Resource Groups & Tagging
    • 11.6 VPC Peering & ClassicLink
    • 11.7 Direct Connect
    • 11.8 Security Token Service
    • 11.9 Active Directory Integration
    • 11.10 Workspaces
    • 11.11 ECS - Part 1 - What is Docker
    • 11.12 ECS - Part 2 - What is ECS
    • 11.13 CloudFormation - A Brief Introduction
    • 11.14 Step Functions - A Brief Introduction
  • 12. Practice Tests Questions Summary
    • 12.1 Whizlabs - Free Test
    • 12.2 Whizlabs - Diagnosis Test
    • 12.3 Whizlabs - Practice Test I
    • 12.4 Whizlabs - Practice Test II
    • 12.5 Whizlabs - Practice Test III
    • 12.6 Whizlabs - Practice Test IV
    • 12.7 Whizlabs - Practice Test V
    • 12.8 Whizlabs - Practice Test VI
    • 12.9 Whizlabs - Practice Test VII
    • 12.10 Whizlabs - Section Tests - Part 1
    • 12.11 Whizlabs - Section Tests - Part 2
    • 12.12 Udemy - Final Quiz
  • 13. The Real World - Creating a fault tolerant Word Press Site
    • 13.1 Getting Setup
Powered by GitBook
On this page
  • What is streaming data
  • What is Kinesis
  • Core Kinesis Services
  • What is Kinesis Streams
  • What is Kinesis Firehose
  • Kinesis Analytics
  • Exam Tips

Was this helpful?

  1. 8. Application Services

8.6 Kinesis 101

What is streaming data

Streaming data is data that is generated continuously by thousands of data sources, which typically send in the data records simultaneously, and in small sizes (order of Kilobytes).

  • Purchases from online stores (amazon.com)

  • Stock prices.

  • Game data (as the gamer plays)

  • Social network data

  • Geospatial data (uber.com)

  • IOT sensor data

What is Kinesis

Amazon Kinesis is a platform on AWS to send your streaming data too. Kinesis makes it easy to load and analyze streaming data, and also providing the ability for you to build your own custom application for you business needs.

Amazon Kinesis is a fully managed service for real-time processing of streaming data at massive scale. You can configure hundreds of thousands of data producers to continuously put data into an Amazon Kinesis stream.

For example, data from website click streams, application logs, and social media feeds. Within less than a second, the data will be available for your Amazon Kinesis Applications to read and process from the stream.

Kinesis actually can be used to (when you should use it):

  • Consume big data

  • Stream large amounts of social media, news feeds logs, etc, into the cloud

  • Process large amounts of data:

    • Redshift for business intelligence

    • Elastic MapReduce for big data processing

Core Kinesis Services

  • Kinesis Streams

  • Kinesis Firehose

  • Kinesis Analytics

What is Kinesis Streams

All of the EC2 instances, mobile phones, laptops or IOT are streaming data sources, they send data to Kinesis Steams. The data will be stored for 24 hours by default, but you can increase this for 7 days. These data is actually stored in Shards. Once the data is stored in the shards, you could have a fleet of EC2 instances which are called Consumers and they take the data from the shards and do some calculation or processing on it and turn it into something useful (e.g. doing data aggregating, sentiment analysis on social media feeds, stock prediction algorithms, etc). Once the consumers done their calculations, they can then send the processed data to be stored (e.g. S3, DynamoDB, EMR, Redshift, RDS, etc).

Key parts of Kinesis:

  • Kinesis Streams consist of shards

    • Each shard can support up to 5 transactions per second for reads, up to a maximum total data read rate of 2 MB per second and up to 1,000 records per second for writes, up to a maximum total data write rate of 1 MB per second (including partition keys).

    • The data capacity of your stream is a function of the number of shards that you specify for the stream. The total capacity of the stream is the sum of the capacities of its shards.

What is Kinesis Firehose

All of the EC2 instances, mobile phones, laptops or IOT are streaming data sources, they send data to Kinesis Firehose. For Kinesis Firehose, you don't need to worry about shards, streams or manually adding in shards to keep up with your data, it is automated. You even don't need to worry about consumers go in and mining the data. You can query/analyze that data using Lambda in real time. Once that data has been analyzed, you can send it over directly to S3. In Kinesis Firehose, there is a automatic data retention window as well. In Kinesis Streams, you can retain data by default for 24 hours, but you can extend it up to 7 days, Firehose doesn't work like that. As soon as the data comes into Firehose, it is either analyzed using Lambda or it is sent directly to S3 or other locations. For example, you can send the data to S3 firstly and then copy the data into Redshift. You can also send the data to ElasticSearch Cluster and write the data into it.

Kinesis Analytics

When we have Kinesis Firehose and Kinesis Streams, the Kinesis Analytics allows you to run the SQL queries of that data as it exist within Firehose or Streams and then you can use that SQL queries to store that data in S3 or Redshift or ElasticSearch Cluster. It is a way of analyzing the data that is inside Kinesis using SQL languages, and you can use it both on Kinesis Streams and Kinesis Firehose.

Exam Tips

  • Know the difference between Kinesis Streams and Firehose. You will be given scenario questions and you must choose the most relevant service.

  • Understand what Kinesis Analytics is.

Previous8.5 API GatewayNext8.7 Kinesis Streams Lab

Last updated 5 years ago

Was this helpful?