IOPS types and its differences

Overview of IOPS (Input/Output Operations per Second):

  1. IOPS Basics:

    • IOPS is a measure of the number of read or write operations that a storage volume can perform per second.
    • IOPS performance can vary depending on the storage volume type and size.

  2. Amazon EBS (Elastic Block Store) IOPS:

    • Amazon EBS volumes come in different types, such as General Purpose SSD (gp2/gp3) and Provisioned IOPS SSD (io1/io2).
    • General Purpose SSD volumes have a baseline IOPS that scales with the volume size, up to 16,000 IOPS, and can also burst above the baseline when needed.
    • Provisioned IOPS SSD volumes allow you to provision the IOPS performance you need, up to 64,000 IOPS per volume.

  3. Monitoring and Troubleshooting IOPS:

    • You can use Amazon CloudWatch metrics to monitor the IOPS and throughput of your EBS volumes.
    • To calculate the maximum IOPS and throughput for an EBS volume, you can use the following AWS CLI command:
    aws ec2 describe-instance-types --instance-types m6i.2xlarge --query InstanceTypes[*].[InstanceType, EbsInfo.EbsOptimizedSupport, EbsInfo.EbsOptimizedInfo]


Key differences between General-Purpose IOPS and Provisioned IOPS:

  1. General Purpose SSD (gp2/gp3) Volumes:

    • General Purpose SSD volumes provide a balance of price and performance.
    • They have a baseline IOPS that scales with the volume size, up to 16,000 IOPS.
    • You can also burst above the baseline IOPS when needed, up to 3,000 IOPS.

  2. Provisioned IOPS SSD (io1/io2) Volumes:

    • Provisioned IOPS SSD volumes are designed for I/O-intensive workloads, such as databases.
    • You can provision the IOPS performance you need, up to 64,000 IOPS per volume.
    • Provisioned IOPS volumes provide consistent and predictable performance.
    • They are available on all EC2 instance types, except for those that support io2 Block Express volumes.

To achieve the maximum IOPS and throughput limits, you should use EBS-optimized EC2 instances. You can use the following AWS CLI command to find the EBS optimization details for a specific instance type:

aws ec2 describe-instance-types --instance-types m6i.2xlarge --query InstanceTypes[*].[InstanceType, EbsInfo.EbsOptimizedSupport, EbsInfo.EbsOptimizedInfo]


Key differences between Baseline IOPS and Burst IOPS:

  1. Baseline IOPS:

    • Baseline IOPS refers to the minimum guaranteed performance level for an EBS volume.
    • The baseline IOPS depends on the EBS volume type and size. For example, a 100 GB General Purpose SSD (gp2) volume has a baseline of 300 IOPS.
    • The baseline IOPS provides a consistent level of performance for your workload.

  2. Burst IOPS:

    • Burst IOPS allows EBS volumes to exceed their baseline IOPS for short periods of time when needed.
    • Volumes smaller than 1 TB can burst up to 3,000 IOPS, regardless of their baseline IOPS.
    • The ability to burst is governed by I/O credits. The volume accumulates I/O credits when the workload is below the baseline, and it can use these credits to burst above the baseline when needed.
    • The burst duration depends on the available I/O credits and the difference between the burst IOPS and the baseline IOPS.

To find the specific baseline and maximum burst IOPS for your EBS volume, you can use the AWS CLI or the AWS Management Console. For example, using the AWS CLI:

aws ec2 describe-instance-types --instance-types m6i.2xlarge --query InstanceTypes[*].[InstanceType, EbsInfo.EbsOptimizedSupport, EbsInfo.EbsOptimizedInfo]

This command will provide the baseline and maximum burst IOPS for the specified instance type.



Comments

Post a Comment

Popular posts from this blog

Difference b/w General-purpose and a Directory bucket in Amazon S3

  The key differences between a general-purpose S3 bucket and a directory bucket in Amazon S3 are: Purpose: General purpose S3 buckets are the standard type of S3 buckets used for storing objects. Directory buckets are a specialized type of S3 bucket designed for use with the AWS Application Discovery Service. Naming Conventions: General purpose S3 bucket names must be unique across all AWS accounts and regions. Directory bucket names must follow a specific naming convention:  <name>--azid--x-s3 , where  <name>  is a unique identifier. Inactive State: Directory buckets that have no request activity for at least 90 days can transition to an inactive state, where they become temporarily inaccessible for reads and writes. General purpose S3 buckets do not have this inactive state behaviour. Access and Permissions: Access to directory buckets is typically restricted to users with permissions to the AWS Application Discovery Service and Migration Hub. General purpose S3 buckets
Read more

Difference between full snapshots and incremental snapshots

The differences between full snapshots and incremental snapshots in Amazon EBS are: Snapshot Type: Incremental snapshots: These snapshots only store the changed data blocks since the last snapshot, making them more efficient and cost-effective for frequent backups. Full snapshots: When you archive an incremental snapshot, it is converted to a full snapshot that includes all the blocks written to the volume at the time the snapshot was created. Storage Tier: Incremental snapshots are stored in the Standard tier by default. Full snapshots are stored in the Archive tier after being converted from an incremental snapshot. Retrieval and Cost: Incremental snapshots in the Standard tier can be retrieved quickly and without additional charges. Full snapshots in the Archive tier have lower storage costs but incur additional charges for data retrieval, typically around $0.03 per GB. Use Cases: Incremental snapshots are suitable for frequent backups and fast recovery. Full snapshots in the Archi
Read more

Snapshot storage tier types and cost difference

The key points on the different storage tiers and cost differences for EBS snapshots: Standard Tier: This is the default storage tier for EBS snapshots. Snapshots stored in the standard tier are incremental, meaning only the changed data blocks are stored. The pricing for the standard tier varies by AWS region, but is typically around $0.05 per GB-month. Archive Tier: The archive tier is a lower-cost storage option for rarely accessed snapshots. When you archive a snapshot, it is converted to a full snapshot (not incremental) and moved to the archive tier. The pricing for the archive tier is significantly lower, typically around $0.0125 per GB-month. However, there is an additional charge for retrieving data from the archive tier, around $0.03 per GB. Cost Savings: Archiving infrequently accessed snapshots to the lower-cost archive tier can result in significant cost savings, up to 75% compared to the standard tier. The savings are more substantial at larger scales, where there are mul
Read more

Differences between an IAM User and Role

The main differences between an IAM User and an IAM Role are: Identity Association : An IAM User is uniquely associated with a single person or application. An IAM Role is not associated with a specific person, but can be assumed by anyone who needs it. Credentials : IAM Users have long-term credentials like a password or access keys. IAM Roles do not have long-term credentials. Instead, they provide temporary security credentials when the role is assumed. Use Cases : IAM Users are typically used for individual users or applications that require long-term access to AWS resources. IAM Roles are used for scenarios where you want to grant temporary access to AWS resources, such as when an EC2 instance needs to access other AWS services. Permissions : Both IAM Users and IAM Roles can be granted specific permissions using IAM policies. The permissions for an IAM Role are defined by the policies attached to the role, and can be assumed by any entity that has been granted permission to do so.
Read more

AWS Storage types

  The main types of storage available on AWS are: Object Storage : Amazon S3 (Simple Storage Service) is the primary object storage service on AWS. It is designed for internet-scale, highly durable, and highly available object storage. S3 is suitable for many use cases, including backup, archiving, web hosting, and big data analytics. Block Storage : Amazon EBS (Elastic Block Store) provides block-level storage volumes that can be attached to EC2 instances. EBS is designed for low-latency, high-throughput access from a single EC2 instance. EBS volumes are suitable for workloads that require persistent data storage, such as databases, file systems, and applications. File Storage : Amazon EFS (Elastic File System) provides a simple, serverless, and scalable file system for AWS compute services. Amazon FSx for Lustre, Amazon FSx for NetApp ONTAP, and Amazon FSx for Windows File Server are other file storage services offered by AWS. These file storage services are suitable for applicatio
Read more

How to reduce the EBS snapshots storage cost

Here are some ways to reduce the storage costs for Amazon EBS snapshots: Leverage the EBS Snapshots Archive storage tier: The EBS Snapshots Archive tier offers significantly lower storage costs, typically around $0.0125 per GB-month, compared to the standard tier at around $0.05 per GB-month. Archive infrequently accessed snapshots, such as monthly, quarterly, or yearly backups, to the lower-cost archive tier. Keep in mind that there is an additional charge for retrieving data from the archive tier, around $0.03 per GB. Optimize snapshot retention: Regularly review and delete snapshots that are no longer needed, especially older snapshots that are not part of the current volume lineage. Consolidate multiple incremental snapshots into a single full snapshot to reduce the overall storage footprint. Disable and archive snapshots for unused AMIs: If you have Amazon Machine Images (AMIs) that are no longer in use, you can disable them to prevent new instances from being launched. Archive th
Read more

How burstable CPU performance works in T series EC2 instances

  Burstable CPU performance refers to the ability of certain Amazon EC2 instance types, such as the T-series instances, to provide a baseline level of CPU performance with the ability to temporarily burst above that baseline when needed. Here's how burstable CPU performance works: Baseline Performance : Burstable instances are designed to provide a consistent baseline level of CPU performance, which is suitable for many general-purpose workloads that don't require sustained high CPU usage. CPU Credits : Burstable instances earn "CPU credits" over time when they are idle or underutilized. These CPU credits can be used to burst above the baseline CPU performance when the workload requires it. Bursting : When the instance's CPU utilization exceeds the baseline, it can use the accumulated CPU credits to burst above the baseline for a limited period. The amount of time the instance can burst depends on the number of CPU credits it has earned. Unlimited Mode : Some burs
Read more

Key features of Amazon S3

Here are some of the key features of Amazon S3: Storage Classes: Amazon S3 offers different storage classes to optimize for different use cases, such as S3 Standard, S3 Intelligent-Tiering, S3 Standard-IA, S3 One Zone-IA, and S3 Glacier. Storage Management: S3 provides features like Lifecycle policies, Replication, and Object Versioning to help manage your data throughout its lifecycle. Access Management and Security: S3 supports access control through IAM policies, bucket policies, and ACLs. It also provides encryption options, including server-side encryption and client-side encryption. Data Processing: You can use S3 as a data source for various AWS services like Amazon Athena, Amazon Redshift, and AWS Glue for data processing and analytics. Storage Logging and Monitoring: S3 provides access logging and metrics to help you monitor and audit your S3 usage. Analytics and Insights: S3 offers features like S3 Analytics and S3 Inventory to help you analyze and gain insights into you
Read more

Pros and cons of On-Demand, Reserved, Spot and Saving Plans EC2 instances

The pros and cons of the on-demand pricing model for EC2 instances are: Pros: Flexibility: On-demand instances allow you to scale your compute resources up or down as needed, without any long-term commitments. No upfront costs: With on-demand, you only pay for the compute capacity you use, without any upfront payments or long-term contracts. Suitable for unpredictable workloads: On-demand is well-suited for applications with short-term, spiky, or unpredictable workloads that cannot be interrupted. Ideal for development and testing: On-demand instances are a good choice for workloads like pre-production environments, experiments, and proofs of concept. Cons: Higher costs for long-term, steady-state workloads: Compared to other pricing models like Reserved Instances or Savings Plans, on-demand instances can be more expensive for long-term, steady-state workloads. No long-term discounts: On-demand instances do not offer any long-term discounts or commitments, unlike Reserved Instances
Read more

Where the EBS snapshot stores and snapshot charges

Your Amazon EBS snapshots are stored in Amazon S3. When you create an EBS snapshot, it is automatically stored in the S3 bucket associated with your AWS account. The specific location where the snapshot is stored is not directly accessible to you, as AWS manages the underlying storage infrastructure. Here are a few key points about where your EBS snapshots are saved: EBS snapshots are stored in the AWS region where you created them. Each AWS region has its own set of S3 buckets where the snapshots are stored. The snapshots are stored in a secure, durable, and highly available manner within the S3 infrastructure. AWS manages the storage and replication of your snapshots to ensure data durability and availability. You can view the list of your EBS snapshots in the AWS Management Console, but you cannot directly access the S3 bucket or location where they are stored. AWS manages the storage and retrieval of your snapshots on your behalf. If you need to restore an EBS volume from a snapsho
Read more