Skip to main content
Back to path

Welcome to a Grasp lesson

Create a free account to keep studying and create your own study path.

Lesson 51

Granting Permissions with Lake Formation

Recommended resources

Hello! Let's continue our exploration of AWS Lake Formation.

In our last session, we laid the essential groundwork for data lake governance. We configured a Data Lake Administrator, upgraded the Data Catalog's permission model to enable fine-grained controls, registered an S3 location, and created a database. Your data lake is now "known" to Lake Formation, but so far, no one has been granted access to it.

Today, we will build directly on that foundation. This lesson focuses on the core task of managing access by implementing policies. By the end of this session, you will be able to grant database- and table-level permissions to IAM principals using Lake Formation. This is a central skill for any data engineer working with AWS and a frequent topic on the certification exam.

How Lake Formation Manages Permissions

Before we grant permissions, let's quickly solidify our understanding of how they work. As we've discussed, for a query from a service like Athena to succeed, it must pass checks from both IAM and Lake Formation.

When a principal (like an IAM user or role) queries data through Athena, the request flow looks like this:

  1. Metadata Request: Athena asks the AWS Glue Data Catalog for information about the table being queried.
  2. Permission Check: The Data Catalog consults Lake Formation to see if the principal has the necessary permissions (e.g., SELECT on that table). Lake Formation only returns the metadata for the databases, tables, and columns the principal is allowed to see.
  3. Credential Vending: If the metadata check passes, Athena requests temporary credentials from Lake Formation to access the underlying data in S3.
  4. Data Retrieval: Using these temporary, scoped-down credentials, Athena reads the data from S3 and returns the results to the user.

This "credential vending" process is key: you don't grant principals direct S3 permissions. Instead, Lake Formation acts as a centralized broker, granting temporary S3 access on a per-query basis.

AWS Lake Formation Table-Level Permissions Enforcement
This diagram shows how Lake Formation acts as a gatekeeper. When an IAM role requests tables via the `GetTables` API, Lake Formation checks its policies. Here, the policy only allows access to the 'Customers' table, so only that table's metadata is returned, effectively hiding the 'Orders' table from the principal.

There are two primary methods for granting permissions in Lake Formation, and you need to be familiar with both for the exam. We'll cover them in order.

Method 1: Named Resource Access Control

This is the most direct and traditional method. You grant specific permissions on specific resources (identified by name) to specific principals. It follows a classic database GRANT/REVOKE model.

The structure is simple: GRANT [permission] ON [resource] TO [principal].

For example:

  • Grant SELECT on table_A to user_analyst_1.
  • Grant CREATE_TABLE on database_B to role_etl_job.

The following video provides an excellent demonstration of this process. It shows a Data Lake Administrator granting permissions first on a database and then on specific tables to different roles.

Deep Dive Into AWS Lake Formation - Level 300 (United States)

In this segment from 'Deep Dive Into AWS Lake Formation', watch how to grant permissions using the named resource method. The demo covers granting database-level permissions and then more granular table-level permissions.

Watch from 04:21 to 07:32. Observe how the presenter grants CREATE_TABLE permission on a database to an ETL role. Then, pay close attention (starting at 16:40) as they grant SELECT permission on tables to a 'retail manager' role. Notice the 'grantable permissions' option, which allows the grantee to pass on their permissions to others.

For a step-by-step textual guide, the official AWS documentation provides a clear walkthrough.

Managing access to a table or database with Lake Formation

This documentation outlines the console steps for granting permissions on a database or table.

Read the section 'Granting Lake Formation permission on a table or database' under the 'Console' tab. This provides a concise, step-by-step procedure that matches the video demonstration.

The named resource method is straightforward and effective for simple use cases or data lakes with a small number of resources and users. However, as your data lake grows, managing hundreds or thousands of individual GRANT statements can become complex and error-prone. This leads us to a more scalable approach.

Test your understanding!

You grant an IAM role named etl-role the Lake Formation permission CREATE_TABLE on a database named processed_data. However, the IAM policy attached to etl-role does not include the glue:CreateTable permission.

What will happen when a Glue ETL job assuming etl-role tries to create a table in the processed_data database?

Show answer

The operation will fail. For any action to succeed, it must be permitted by both Lake Formation and IAM. Even though Lake Formation grants permission to create a table, the principal also needs the underlying IAM permission to call the glue:CreateTable API action. Lake Formation provides the fine-grained authorization on the specific resource, while IAM provides the coarse-grained permission to call the API itself.

Method 2: Tag-Based Access Control (LF-TBAC)

For larger, more complex data lakes, Tag-Based Access Control (LF-TBAC) is the recommended best practice. Instead of granting permissions on named resources, you use a system of tags.

The process involves three steps:

  1. Define LF-Tags: A Data Lake Administrator creates LF-Tags, which are simple key-value pairs (e.g., Confidentiality:PII, Project:Phoenix).
  2. Assign Tags to Resources: These tags are assigned to databases, tables, or even individual columns.
  3. Grant Tags to Principals: You grant principals permissions on the LF-Tags themselves.

A principal receives access to a resource if the LF-Tags they have been granted match the LF-Tags assigned to the resource.

This approach dramatically simplifies management. Instead of managing thousands of grants, you manage a handful of tag-based policies. If a new table is created, you simply assign it the appropriate tag (e.g., Department:Sales), and all principals with the Department:Sales tag automatically get access.

The following video gives a clear explanation and comparison of the two methods.

Lake Formation: Managing Permissions Through LF-Tags

This video from DataEng Uncomplicated provides a fantastic overview of LF-Tags and why they are a superior method for managing permissions at scale.

Watch from the beginning to 03:58, and then from 05:11 to 06:09. Focus on understanding: The core difference between named-resource and tag-based access. The benefits of using tags for organization and scalability. The example of how tags are assigned to databases and users to control access. The comparison illustrating how complicated named-resource grants can become.

To reinforce this, the article below provides an excellent summary of both named resource and tag-based methods.

AWS Lake Formation For Data Governance and Security

This article provides a structured overview of the different ways to grant permissions in Lake Formation. It's a great textual summary of what we've just discussed.

Read the section titled '1) Managing Metadata access control'. Pay attention to the descriptions of '“Named resource” access control' and '“Tag-based” access control'. This will solidify the distinction between the two methods.

For the Data Engineer exam, you should be able to evaluate a scenario and determine whether the direct named-resource method is sufficient or if the more scalable LF-TBAC approach is required.

Conclusion

In this lesson, you've learned the practical steps for securing your data lake by granting permissions to your users and services. We've moved from a registered but inaccessible data lake to one where access is explicitly and securely managed.

Key Takeaways:

  • Dual Permission Model: Access to data in a Lake Formation-governed data lake requires permissions from both IAM (for API access) and Lake Formation (for resource access).
  • Named Resource Method: A direct approach for granting permissions on specific databases and tables to specific principals. It's simple for small-scale use cases.
  • Tag-Based Access Control (LF-TBAC): A scalable and recommended method where permissions are granted on tags, which are then applied to resources and principals. This simplifies governance in large, complex environments.
  • Integrated Enforcement: Services like Athena, Redshift Spectrum, and EMR are "Lake Formation-aware" and will automatically enforce the permissions you define.

Next Up:
We've now covered database and table-level security. The next logical step is to get even more granular. In our next lesson, we will focus on how to implement fine-grained access control using Lake Formation's column-level security, allowing you to protect sensitive columns within a table.