How to set up Unity Catalog
From metastore to three-level namespace: the Unity Catalog setup decisions you do not get to undo later.
- 01
Decide on your metastore
The metastore is the top-level securable object in Unity Catalog and it holds your metadata about data and AI assets, plus the permissions that apply to them. You set up one per region and attach the workspaces in that region to it. Multiple workspaces can share the same metastore, and that is exactly the point: your permission model and your lineage then live in one place instead of being redone per workspace.
Workspaces created after 9 November 2023 are automatically enabled for Unity Catalog by Databricks. If you have an older workspace, do not assume it is already in place.
Decisionone metastore per region, not one per team. You separate teams with catalogs, not with metastores. If you work in multiple regions you have multiple metastores, and sharing data between regions becomes an explicit choice you make deliberately. - 02
Choose your catalog layout
Below the metastore sits the three-level namespace:
catalog.schema.table. The catalog is the first level and therefore where you make your coarsest split. Permissions granted on a catalog are inherited by every schema and table below it, so this choice determines how many individual grants you hand out for the rest of your life.There are two common layouts and they do not exclude each other:
- By environment:
dev,tst,prd. Easy to understand, and promotion between environments is a rename rather than a move. - By domain:
sales,finance,operations. Matches ownership and how people talk about data.
Decisionpick one axis as the main split and use the other as the schema. Trying to do both at catalog level gives you `sales_dev` next to `sales_prd` and loses the permission inheritance that makes the catalog useful in the first place. - By environment:
- 03
Pick the right table type per table
Unity Catalog has three kinds of tables and each has its own reason to exist:
- Managed tables: the recommendation for most new tables. Databricks manages the lifecycle, the file layout and the optimization. You get cleanup and compaction without writing a policy for it yourself.
- External tables: for data that already sits somewhere, for data that has to be reached by path from tools outside Databricks, and for file formats managed tables do not support.
- Foreign tables: for Lakehouse Federation. These are read-only references to data in an external system, so do not count on writing to them.
Decisionmanaged by default, external as an exception with a reason. Making external the default "because we already have the storage" also takes back the maintenance Databricks would otherwise have done. - 04
Migrate away from the Hive metastore
If you have an existing workspace, the legacy Hive metastore appears in the three-level namespace as the
hive_metastorecatalog. It works, which is exactly why it stays. But whatever sits in it falls outside your Unity Catalog permission model and outside your lineage.Decisionput an end date on `hive_metastore` and measure progress in number of tables. Without a date this becomes the permanent leftover area your audit will start asking about. - 05
Grant access at the level where it belongs
Grant permissions as high in the hierarchy as you can and as low as you must. A catalog grant to a security group is one action that carries through to everything below it; the same access as individual table permissions is a hundred actions nobody keeps up with.
Decisiongroups, not people. And record who owns each catalog, because the owner is the one who can hand out permissions. Without a named owner that lands on the platform team by default, and that does not scale.
What to do next
Connect Unity Catalog to Microsoft Purview once the layout is settled, so your lineage and classification sit in the same picture as the rest of your estate. See How to connect Azure Databricks to Microsoft Purview.
The process at a glance
Click a step for its key decision
Decide on your metastore
one metastore per region, not one per team. You separate teams with catalogs, not with metastores. If you work in multiple regions you have multiple metastores, and sharing data between regions becomes an explicit choice you make deliberately.
Read next
Related patterns
Related use cases
Related best practices
