Day-9-Aws
A Comprehensive Guide to AWS S3: Features, Characteristics, and Practical Demos
TLDR: This blog post provides an in-depth exploration of AWS S3, detailing its features, characteristics, and practical demonstrations on how to create buckets, manage permissions, and host static websites. It emphasizes the importance of S3 in solving storage problems for organizations and highlights its reliability, scalability, and cost-effectiveness.
Today marks Day 9 of the AWS Zero to Hero series, where we will dive deep into the concept of AWS S3 (Simple Storage Service). In this post, we will explore what S3 is, its properties, characteristics, and features, and we will also conduct two practical demonstrations. Make sure to read till the end to grasp both the theoretical and practical aspects of AWS S3.
s3 → can store lots of data (lots of objs) |
99.99999999999 %→ 99.(11 →9 %) —> 11 9’s → almost always up , no down time → your file stored in s3 never get deleted |
Advantage: HP - CS (High Performance Computer Software) |
store → application log files, database backups, and configuration files |
access data using HTTP protocol → so s3 is called Globally accessible service |
Scalability (single filesize < 5TB): multi part upload → upload files of huge size → when we re-upload it will continue where it is left |
Security : |
Cost Effective : depends on storage class |
Create s3 bucket |
search → s3 → bucket (card) → create bucket |
General Config (card) → [name: <team>-<feature>-<branch>-<domain> [app1-payment-pod-example.com] → region : → ] |
Bucket ownership (card) → |
Block public access setting (card) → block all public access (checked) |
Bucket versioning (card)→ enable |
Tags (card) → edit → add tag → [key : → value: ] → save changes —> identify this file belong to which project |
Default Encryption (card) → [ type : SSE-S3 → bucket key : enable ] |
Advance Setting (card) |
—> create bucket |
bucket name → should be unique across the glob |
region → resides in a region, but access globally (bucket in us-east → but access globally) |
Upload information → created s3 |
click on created s3 → object (tab)→ object (card) → upload |
upload (page) → files and folder (card) → add files / add folder → select file / folder from laptop → upload |
like git → s3 also has versioning concept → keep track of files stored → so we should enable it |
click on created s3 → properties (tab)→ bucket versioning (card) → edit → enable → save changes |
click on created s3 → object (tab)→ object (card) → click on uploaded object (file) → version (tab) → see list of version |
—> we can download old version , current version → some one |
properties (tab) → explore |
click on created s3 → properties (tab) |
bucket versioning (card) |
Tag (card) |
Default Encryption (card) |
Intelligent-tiering Archive Configuration (card) |
Server access logging (card) → enable → target bucket : → save changes |
Aws cloudTrail data events (card) |
event notifications (card) |
amazon eventbridge (card) |
transfer acceleration (card) |
obj lock (card) → if i am using the obj, the other user can’t update the obj |
requester pay (card) |
static web hosting (card) |
manage permissions to restrict access to sensitive data in an S3 bucket
create an IAM user and manage permissions to restrict access to sensitive data in an S3 bucket |
search→ IAM → Access Management (sidebar) → Users (option) → add users |
→[name : demo-s3-bucket-user → (checked) access to Aws management console → I want to create IAM user → custom password: — → next → permission option(card) → attach policy directly →AmazonS3FullAccess ] → create user |
login → IAM user →[ account_id : -- → next → IAM user name :demo-s3-bucket-user → password: — ] → sign in |
search→ s3 →click on created s3 → permissions (tab) → bucket policy (card)→ edit → add statement |
→ now policy is automatically generated → edit policy |
id | name | “blockallpublicaccess” |
principal | against whom to perform —→ | “*” |
effect | allow / deny | “deny”” |
action | [] → chose service → s3 → all action —> | [“s3:*” ] |
resource | [] → add resource → (service: s3 → resource type: bucket → resource ARN : arn:aws:s3::: app1-payment-prod-example.com/* ) → add resource | “arn:aws:s3::: app1-payment-prod-example.com/*” |
[] → add resource → (service: s3 → resource type: bucket → resource ARN : arn:aws:s3::: app1-payment-prod-example.com ) → add resource | “arn:aws:s3::: app1-payment-prod-example.com” |
the above setting block all user including you → so above statement should be ignored for owner of bucket (you) |
Add a condition → [condition key : aws:PrincipalArn→ operator: StringNotEqual → value : "aws:PrincipalArn": "arn:aws:iam::AWS_ACCOUNT_ID:root" ] → add condition |
{
"Version": "2012-10-17",
"Id": "RestrictBucketToIAMUsersOnly",
"Statement": [
{
"Sid": "AllowOwnerOnlyAccess",
"Effect": "Deny",
"Principal": "*",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::app1-payment-prod-example.com/*",
"arn:aws:s3:::app1-payment-prod-example.com"
],
"Condition": {
"StringNotEquals": {
"aws:PrincipalArn": "arn:aws:iam::AWS_ACCOUNT_ID:root"
}
}
}
]
}
static web hosting |
click on created s3 → properties (tab) → static web hosting (card) → edit |
static website hosting : enable → host type : host a static website → index document : index.html → save changes |
click on created s3 → properties (tab) → static web hosting (card) → url → click on url → forbidden error → 403 |
click on created s3 → permission (tab) → Block public access (card) → (unchecked) Block public access → confirm |
static web hosting (card) → url → click on url → forbidden error → 403 —→ still |
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "PublicReadGetObject",
"Effect": "Allow",
"Principal": "*",
"Action": [
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::app1-payment-prod-example.com/*"
]
}
]
}
What is AWS S3?
AWS S3 stands for Simple Storage Service, and it is one of the most user-friendly services offered by AWS. Many individuals begin their AWS journey with S3 due to its simplicity and ease of execution. The primary problem that S3 addresses is storage, which is a common issue faced by both individuals and organizations.
The Storage Problem
In our daily lives, we often run out of storage on our devices, prompting us to purchase external hard drives or USB drives. However, organizations deal with much larger data sets, including databases, backups, application logs, and various file types. As organizations transition to public cloud solutions, they require a robust storage solution, which is where AWS S3 comes into play.
Characteristics of AWS S3
AWS S3 is popular for several reasons, primarily due to its reliability, which is often referred to as "11 nines" (99.999999999%). This reliability is a key factor in its success. Here are the five main characteristics of AWS S3:
Highly Scalable: S3 can handle vast amounts of data without any restrictions on the amount of information stored.
High Availability: S3 ensures that data is accessible whenever needed.
Security: S3 provides robust security features to protect stored data.
Cost Efficiency: S3 offers a cost-effective solution for storing large amounts of data.
Performance: S3 is designed for high performance, allowing for quick data retrieval and storage.
Types of Files Stored in S3
S3 does not impose restrictions on the types of files that can be stored. Users can store images, videos, application logs, databases, and more. As a DevOps engineer, you might typically deal with application log files, database backups, and configuration files.
Creating an S3 Bucket
Creating an S3 bucket is straightforward. Here’s how you can do it:
Navigate to the S3 service in the AWS Management Console.
Click on "Create Bucket" and provide a unique name for your bucket. Remember, bucket names must be globally unique across all AWS accounts.
Select the appropriate AWS region for your bucket to minimize latency.
Configure the bucket settings, including public access settings and encryption options.
Click on "Create Bucket" to finalize the process.
Uploading and Managing Objects in S3
Once your bucket is created, you can upload files (referred to as objects in S3) easily. Here’s how:
Click on your bucket name in the S3 console.
Click on the "Upload" button and select the files you want to upload.
After uploading, you can manage these objects by deleting them or modifying their properties.
Reliability of S3
AWS S3 is designed to be highly reliable. For instance, if you upload one billion objects over a period of 100 years, AWS guarantees that only one object may be lost. This reliability is achieved through a replication mechanism that creates multiple copies of your data across different availability zones.
Security Features of S3
Security is a critical aspect of AWS S3. Here are some key security features:
Encryption: S3 supports encryption at rest and in transit.
Access Control: You can configure bucket policies and access control lists (ACLs) to manage who can access your data.
Logging: S3 allows you to enable logging to track access to your buckets.
Cost Efficiency of S3
AWS S3 is known for its cost-effectiveness. For example, storing 1 TB of data can cost as little as four to five dollars per month, depending on the storage class selected. S3 offers various storage classes, each designed for different use cases and cost considerations.
Practical Demonstrations
Demo 1: Managing Bucket Permissions
In this demonstration, we will create an IAM user and manage permissions to restrict access to sensitive data in an S3 bucket. Here’s a brief overview of the steps:
Create an IAM user without any permissions.
Attempt to access the S3 bucket with this user to confirm access is denied.
Grant the user S3 permissions and verify access.
Implement bucket policies to restrict access to sensitive information.
Demo 2: Hosting a Static Website
In this demonstration, we will host a static website using S3. The steps include:
Create an S3 bucket for the website.
Upload the static website files (e.g., index.html).
Enable static website hosting in the bucket properties.
Configure bucket policies to allow public access to the website.
Conclusion
AWS S3 is a powerful and versatile storage solution that addresses the storage needs of individuals and organizations alike. Its reliability, scalability, security, and cost-effectiveness make it a popular choice for cloud storage. By understanding how to create and manage S3 buckets, as well as implementing security measures, you can effectively utilize AWS S3 for your storage needs.
I hope you found this guide helpful. If you have any questions or feedback, feel free to leave a comment. Thank you for reading!