S3 Bucket Policy Fundamentals

1. From Identity-Based to Resource-Based Policies

While an IAM policy is attached to an identity (a "principal"), a bucket policy is a JSON document attached directly to an S3 bucket. This shifts the focus from "What can this user do?" to "Who can access this resource?".

A bucket policy has a structure very similar to the IAM policies we saw previously, but with one crucial difference: you must explicitly state the Principal—the user, role, service, or account that the policy statement applies to.

Let's watch a short video that introduces the main access control mechanisms for S3 and clearly defines the role of bucket policies.

2. When to Use a Bucket Policy vs. an IAM Policy

Since both IAM policies and bucket policies can grant access to S3, a common exam question is knowing when to use which. The choice depends on your management and security goals.

The most important distinction for data engineering scenarios is cross-account access. If a data processing job in Account B needs to read data from your data lake in Account A, a bucket policy is often the most direct way to grant that permission.

This video does an excellent job of summarizing these practical differences.

3. Policy Evaluation: The Golden Rule

When both an identity-based policy and a resource-based policy apply to a request, AWS evaluates all of them to make a decision. The logic can be complex, but the most important rule is simple:

An explicit Deny in any applicable policy always overrides any Allow.

For access to be granted, there must be an explicit Allow from either the IAM policy, the bucket policy (or both), and there must be no explicit Deny statements in any policy.

4. Implementing Granular Control with Bucket Policies

Now, let's get practical. A key task in managing a data lake is granting access not to the entire bucket, but to specific "folders" or prefixes within it. For example, you might want an ingestion service to only be able to write to /raw and an analytics service to only read from /curated.

Understanding S3 "Folders"

First, it's crucial to understand that S3 doesn't have folders in the traditional sense. It has a flat structure of objects within a bucket. The "folders" you see in the AWS Console are a visual representation created by using a shared prefix in the object keys.

Crafting a Granular Policy: A Walkthrough

Now that we understand prefixes, let's see how to use them in a bucket policy to grant specific permissions. The following video demonstrates creating a bucket policy that grants a user full permissions, but only within a specific confidential/ folder.

Here is an example of what such a policy looks like. This policy allows the role DataProcessingRole to read (GetObject) from the raw-data/ prefix and write (PutObject) to the processed-data/ prefix, but nowhere else in the bucket.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowReadFromRawData",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::111122223333:role/DataProcessingRole"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::my-data-lake-bucket/raw-data/*"
        },
        {
            "Sid": "AllowWriteToProcessedData",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::111122223333:role/DataProcessingRole"
            },
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::my-data-lake-bucket/processed-data/*"
        }
    ]
}

Notice a few key details:

• Two Statements: The policy uses two separate statements, each with a clear Sid (Statement ID), to handle the different permissions for each prefix.
• Principal: The same principal (the DataProcessingRole) is specified in both statements.
• Action: The actions are specific (s3:GetObject, s3:PutObject), following the principle of least privilege.
• Resource: The Resource ARN uses a wildcard (/*) to apply the permissions to all objects within that prefix.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "GrantPartnerReadAccessToSharedDataset",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::444455556666:root"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::customer-data-lake-789/shared-datasets/quarterly-reports/*"
        }
    ]
}

Conclusion

In this lesson, we explored S3 bucket policies as a powerful tool for resource-based permissions. By attaching a policy directly to your S3 data lake, you gain fine-grained, centralized control over who can access your data, which is essential for security and governance.

Key Takeaways:

• Bucket policies are resource-based policies attached directly to an S3 bucket.
• They are the preferred method for granting cross-account access to S3 resources.
• The Principal element is mandatory and specifies who the policy statement applies to.
• You can achieve granular control by scoping the Resource ARN to specific prefixes (folders) and using least-privilege Actions.
• Remember the evaluation logic: an explicit Deny in any policy always wins.

So far in this module, we've focused on controlling who can access data. In our next lesson, we'll shift our focus to protecting the data itself. We will configure server-side encryption for S3 objects using KMS and describe envelope encryption, ensuring that your data remains secure even if it falls into the wrong hands.