Introduction
In this blog post we’re going to learn about managing Terraform state remotely, by leveraging Azure Blob Storage.
Prerequisites
Before you can follow this tutorial, you will need the following:
- An Azure subscription
- The Azure CLI installed on your computer and logged in to your Azure subscription
- The Terraform CLI installed on your computer
Terraform
What is Terraform?
Terraform is an open-source infrastructure as code (IAC) software tool that enables you to safely and predictably create, change, and destroy infrastructure. Terraform not only allows you to manage cloud infrastructure as code, but also allows you to configure tools such as Kubernetes, F5 Firewalls, Aquasec, and more.
In fact, if there’s has an API that can be used to deploy or configure a resource, Terraform can probably be used to manage it.
Terraform Providers
Providers are the plugins that Terraform uses to interact with various platforms and resources, there are over 3400 providers currently and that number is only increasing. Go over to the Terraform registry and check out the providers that are available, you may be surprised to find out what you can find.
Terraform State
Terraform state is the information that Terraform needs to successfully manage infrastructure. It is used to map resources to configuration, which is then used to track any drift between the two over time.
Terraform state is stored in a file called terraform.tfstate by default.
If this doesn’t make sense, don’t worry, by the end of this tutorial you will know what Terraform state is, what it is made up of and how to manage it.
Managing state
Create a working directory
To follow along you will need a working Terraform configuration, if you already have that then you can skip this part and move onto the next section.
so let’s create a working directory and initialise Terraform:
mkdir managing-terraform-state-demo
cd managing-terraform-state-demo
Create a file called main.tf and add the following to it:
# Azure provider
provider "azurerm" {
features {}
}
The above code will configure the AzureRM provider, which will allow us to create resources in Azure.
Deep dive into Terraform state
You can now initialise Terraform by running the following command:
terraform init
You should get a similar output to the below, meaning your initialisation was successful:
Initializing the backend...
Initializing provider plugins...
- Finding latest version of hashicorp/azurerm...
- Installing hashicorp/azurerm v3.71.0...
- Installed hashicorp/azurerm v3.71.0 (signed by HashiCorp)
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
Notice that Terraform has pulled down the AzureRM provider and installed it into a directory called .terraform.
If we run a terraform plan now, we should see that there are no changes to be made as we haven’t defined any resources yet:
terraform plan
output:
No changes. Your infrastructure matches the configuration.
Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.
Exploring the terraform state file
Now that we have initialised Terraform, we can take a look at the state file that has been created.
terraform state pull
You should see something similar to the below:
{
"version": 4,
"terraform_version": "1.5.2",
"serial": 1,
"lineage": "9205abdc-9fa3-3cbd-3760-d4faa61b71b0",
"outputs": {},
"resources": [],
"check_results": null
}
There are some important fields here, below is a table explaining all of them:
| field | description |
|---|---|
| version | The version of the Terraform state file format. |
| terraform_version | The version of Terraform that was used to create the state file. |
| serial | The serial number of the state file. This is incremented each time the state file is updated. |
| lineage | A unique identifier for the state file. This always stays the same once created. |
| outputs | The outputs of the Terraform configuration. This is currently empty, as we have not created any outputs. |
| resources | The resources that have been created by the Terraform configuration. This is currently empty, as we have not created any resources. |
| check_results | The results of the last terraform plan. This is currently empty, as we have not run a terraform plan. |
Adding resources
Let’s add some sample code to our Terraform configuration, so we can see what happens to our state file when we add resources.
Add the following to your main.tf file:
resource "azurerm_resource_group" "this" {
count = 2
location = "uksouth"
name = "resource-group-demo-${count.index}"
tags = {
builtBy = "terraform"
}
}
This will create two resource groups called resource-group-demo-0 and resource-group-demo-1 in the uksouth region.
Now let’s run a terraform plan and see what happens to our state file:
terraform plan
output:
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# azurerm_resource_group.this[0] will be created
+ resource "azurerm_resource_group" "this" {
+ id = (known after apply)
+ location = "uksouth"
+ name = "resource-group-demo-0"
+ tags = {
+ "builtBy" = "terraform"
}
}
# azurerm_resource_group.this[1] will be created
+ resource "azurerm_resource_group" "this" {
+ id = (known after apply)
+ location = "uksouth"
+ name = "resource-group-demo-1"
+ tags = {
+ "builtBy" = "terraform"
}
}
Plan: 2 to add, 0 to change, 0 to destroy.
As expected, Terraform is going to create two resource groups called resource-group-demo-0 and resource-group-demo-1.
Let’s apply that and get the two resource groups created:
terraform apply -auto-approve
At the end of your output you should see something similar to the below:
azurerm_resource_group.this[1]: Creating...
azurerm_resource_group.this[0]: Creating...
azurerm_resource_group.this[0]: Creation complete after 0s [id=/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/resource-group-demo-0]
azurerm_resource_group.this[1]: Creation complete after 0s [id=/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/resource-group-demo-1]
Apply complete! Resources: 2 added, 0 changed, 0 destroyed.
Cool! We’ve just created two resource groups. Let’s see what has happened to the state after that apply:
terraform state pull
You should get similar output to this:
{
"version": 4,
"terraform_version": "1.5.2",
"serial": 2,
"lineage": "9205abdc-9fa3-3cbd-3760-d4faa61b71b0",
"outputs": {},
"resources": [
{
"mode": "managed",
"type": "azurerm_resource_group",
"name": "this",
"provider": "provider[\"registry.terraform.io/hashicorp/azurerm\"]",
"instances": [
{
"index_key": 0,
"schema_version": 0,
"attributes": {
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/resource-group-demo-0",
"location": "uksouth",
"managed_by": "",
"name": "resource-group-demo-0",
"tags": {
"builtBy": "terraform"
},
"timeouts": null
},
"sensitive_attributes": [],
"private": "xxxxxxx"
},
{
"index_key": 1,
"schema_version": 0,
"attributes": {
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/resource-group-demo-1",
"location": "uksouth",
"managed_by": "",
"name": "resource-group-demo-1",
"tags": {
"builtBy": "terraform"
},
"timeouts": null
},
"sensitive_attributes": [],
"private": "xxxxx"
}
]
}
],
"check_results": null
}
There are a couple of things to note here, first of all our state is starting to look like real state, we can now see the two resource groups that we created. Secondly the serial number has been incremented to 2, this is because we have updated the state file by adding two resources. The next time we add resources and apply them, the serial number will be incremented to 3 and so on and so forth.
The serial is important, as it is used to prevent two people from updating the state file at the same time, which could cause issues in the long run.
Now the serial number has been incremented, if someone tried to apply an old version of the state file, Terraform would throw an error and prevent them from doing so.
╷
│ Error: Saved plan is stale
│
│ The given plan file can no longer be applied because the state was changed by another operation after the plan was created.
╵
Managing state
Now we understand what terraform state is, how it looks and how it works, let’s go through some of the ways in which we can manage the resources within it.
Viewing state
We’ve already seen how to view the state file using the terraform state pull command, but there are other ways to view the state file.
You can use the terraform state list command to list all the resources in your state file:
terraform state list
output:
azurerm_resource_group.this[0]
azurerm_resource_group.this[1]
You can also use the terraform state show command to show the details of a specific resource:
terraform state show 'azurerm_resource_group.this[0]'
output:
# azurerm_resource_group.this[0]:
resource "azurerm_resource_group" "this" {
id = "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/resource-group-demo-0"
location = "uksouth"
name = "resource-group-demo-0"
tags = {
"builtBy" = "terraform"
}
}
Importing resources
Using the CLI
If you have resources that have been created outside of Terraform, you can import them into your state file using the terraform import command.
For example, if you had a resource group called resource-group-demo-2 that was created outside of Terraform, you could import it like this:
terraform import 'azurerm_resource_group.this[0]' /subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/resource-group-demo-0
output:
Import successful!
The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.
Using import blocks
You can also import resources using import blocks, which are blocks of code that you can add to your Terraform configuration to import resources into your state file.
For example, if you wanted to import the first resource group that we created in the previous step, you could add the following to your Terraform configuration in a file called imports.tf:
import {
to = azurerm_resource_group.this[0]
id = "/subscriptions//resourceGroups/resource-group-demo-0"
}
This has exactly the same effect as using the terraform import command,
but it can be committed to source control, and you can follow the same workflow of running a plan to make sure it all looks good and then an apply once you’re happy with it.
Moving resources
There may be a time when you need to move resources in your state.
An example of this would be when you first create a resource but as you improve your terraform code you may decide to move that creation into a module so that others can take advantage of your work.
When you carry out a piece of work like this it means that the reference to that resource will change, which will cause Terraform to think that it is a new resource and try to create it again.
To stop that from happening you can move the resource to its new reference using the terraform state mv command, or you also have the option of using move blocks.
Say you wanted to create a module to create resource groups, you could create a module called resource-group and move the resource groups that we created earlier into it.
Using terraform state mv to move resource-group-demo-0 into the module:
terraform state mv 'azurerm_resource_group.this[0]' 'module.resource-group.azurerm_resource_group.this[0]'
Using move blocks to achieve the same thing:
moved {
from = azurerm_resource_group.this[0]
to = module.resource-group.azurerm_resource_group.this[0]
}
Deleting resources
There may be a rare occasion where you need to delete a resource from your state file, you can do this using the terraform state rm command.
Note: be very careful using this command as it cause confusion and issues. Removing state from your state file doesn’t remove it from the platform or cloud provider.
Let’s attempt to remove a resource from our state file:
terraform state rm 'azurerm_resource_group.this[0]'
output:
Removed azurerm_resource_group.this[0]
Successfully removed 1 resource instance(s).
If you run terraform state list again you should see that the resource group has been removed from the state file and is no longer listed.
Let’s import it back into our configuration again:
terraform import 'azurerm_resource_group.this[0]' /subscriptions//resourceGroups/resource-group-demo-0
Conclusion
In this tutorial we have gone through what Terraform state is, what it looks like and how to manage the resources within it. Managing state via the cli is almost now a last resort as you can use moved and import blocks to move and import resources, with the added benefit of being able to plan beforehand. Hopefully you now have a better understanding of Terraform state and found this tutorial useful.
If you have any questions please feel free to ask below! Thanks :)
