Consul
Mutual TLS (mTLS) Encryption
This topics describes how to enable Mutual TLS (mTLS) encryption on a Consul datacenter.
The content in this page is not referred to Consul service mesh mTLS encryption. For documentation about Consul service mesh Certification authority refer to Service Mesh Certificate Authority documentation.
Benefits of TLS encryption
Consul supports the use of TLS certificates to secure the traffic related to the RCP
, gRPC
, and HTTP
communications and to verify the authenticity of servers and clients.
Implementing TLS encryption in Consul provides several security features.
Encrypt data in transit: TLS encrypts data transmitted between Consul datacenter nodes and user interfaces like the UI or CLI, ensuring that sensitive information are not exposed to unauthorized parties.
Authenticate: TLS ensures that only verified parties can communicate with each other by using certificates. This prevents unauthorized nodes, services, or users from interacting with your Consul datacenter.
Prevent man-in-the-middle (MITM) attacks: Without TLS, attackers can intercept and change communications within your Consul deployments. TLS mitigates this risk by encrypting data and verifying the identity of the communication participants.
Comply with regulations or standards: Compliance frameworks and regulations like PCI-DSS and HIPAA mandate the encryption of data in transit, making TLS a requirement for Consul deployments in regulated environments.
To enable TLS encryption, Consul requires that all clients and servers have key pairs that are generated by a single Certification Authority (CA). This should be a private CA, used only internally.
Client and server authenticity is verified using the following parameters:
Workflow
To enable TLS encryption in Consul deployments, complete the following steps:
- Initialize the built-in CA that will be used to sign all certificates.
- Create server certificates to secure Consul servers and distribute them to server nodes.
- Configure server agents for TLS encryption.
- Start server agents.
- Configure Consul client agents.
- Start client agents.
Initialize the built-in CA
The first step for configuring TLS for Consul is generating certificates. To prevent unauthorized datacenter access, Consul requires all certificates to be signed by the same Certification Authority (CA). This should be a private CA and not a public one as any certificate signed by this CA will be allowed to communicate with the datacenter.
There are a variety of tools for creating and managing your own CA, like the PKI secrets engine in Vault or the Terraform TLS provider, but Consul ships with built-in TLS tools to create and manage certificates.
You can create the CA and certificates before starting Consul, as long as you have the Consul binary installed in your path.
$ consul tls ca create -domain=consul
==> Saved consul-agent-ca.pem
==> Saved consul-agent-ca-key.pem
The command generates two files, <domain>-agent-ca.pem
and <domain>-agent-ca-key.pem
. In this example the domain used to generate the certificates is the default one, consul
.
The CA certificate, consul-agent-ca.pem
, contains the public key necessary to validate Consul certificates and therefore must be distributed to every node
that runs a consul agent.
The CA key, consul-agent-ca-key.pem
, will be used to sign certificates for Consul nodes and must be kept private. Possession of this key allows anyone to run Consul as a trusted server or generate new valid certificates for the datacenter and obtain access to all Consul data, including ACL tokens.
Create server certificates
In order to authenticate Consul servers, servers are provided with a special certificate that contains server.<domain>.<datacenter>
in the Common Name
and in the Subject Alternative Name
. If you enable verify_server_hostname
, only agents that provide such certificate are allowed to boot as a server.
Without verify_server_hostname = true
an attacker could compromise a Consul client agent and restart the agent as a server in order to get access to all the
data in your datacenter! This is why server certificates are special, and only servers should have them provisioned.
Note
Server keys, like the CA key, must be kept private - they effectively allow access to all Consul data.
Create a server certificate for datacenter dc1
in the consul
domain. If your datacenter or domain is different, remember to use the appropriate flag values.
$ consul tls cert create -server -dc=dc1 -domain=consul
==> WARNING: Server Certificates grants authority to become a
server and access all state in the cluster including root keys
and all ACL tokens. Do not distribute them to production hosts
that are not server nodes. Store them as securely as CA keys.
==> Using consul-agent-ca.pem and consul-agent-ca-key.pem
==> Saved dc1-server-consul-0.pem
==> Saved dc1-server-consul-0-key.pem
Repeat this process on the same node where you created the CA, until there is an individual certificate for each server. The command can be called over and over again, it will automatically increase the certificate and key numbers. You will need to distribute the certificates to the servers.
Distribute the server certificates
After generating the server certificates, you will need to distribute them to the Consul servers and put their on-disk location in the agent configuration file.
The following files need to be copied to your Consul server:
consul-agent-ca.pem
: CA public certificate.dc1-server-consul-0.pem
: Consul server node public certificate for first server node in thedc1
datacenter in theconsul
domain.dc1-server-consul-0-key.pem
: Consul server node private key for first server node in thedc1
datacenter in theconsul
domain.
Configure server agents
There are two methods for configuring Consul server agents depending on the way you want to distribute certificates to the client agents:
- auto encryption method, that uses the Consul Connect CA to generate client certificates, which are automatically distributed to all Consul clients.
- operator method, that requires you to manually generate client certificates and distribute them to the single client agents.
We recommend the auto-encryption method to alleviate the client certificate generation and distribution step for operators. Using auto-encryption the client certificates will also be automatically rotated without the need for an operator intervention.
Use the operator method if you need to use a third-party CA or need more fine-grained control over certificate management.
HCL
addresses = {
https = "0.0.0.0"
}
ports {
https = 8501
}
tls {
defaults {
ca_file = "consul-agent-ca.pem"
cert_file = "dc1-server-consul-0.pem"
key_file = "dc1-server-consul-0-key.pem"
verify_incoming = true
verify_outgoing = true
verify_server_hostname = true
}
}
auto_encrypt {
allow_tls = true
}
JSON
{
"addresses": {
"https" : "0.0.0.0"
},
"ports": {
"https" : 8501
},
"tls" : {
"defaults": {
"ca_file": "consul-agent-ca.pem",
"cert_file": "dc1-server-consul-0.pem",
"key_file": "dc1-server-consul-0-key.pem",
"verify_incoming": true,
"verify_outgoing": true,
"verify_server_hostname": true
}
},
"auto_encrypt" : {
"allow_tls" : true
}
}
Consul will not enable TLS for the HTTP unless the https
port has been assigned a port number > 0
. We recommend using 8501
for https
as this default will automatically work with some tooling.
Start Consul servers
Now that you have configured your servers, you can start the Consul process.
$ systemctl start consul
Your servers will only be communicating with TLS encryption for RPC and consensus.
Configure client agents
There are two methods for distributing client certificates: auto encryption and operator. We recommend the auto-encryption method to alleviate the client certificate generation and distribution step for operators. This method uses the Connect CA to generate client certificates, which are automatically distributed to all Consul clients.
Use the operator method if you need to use a third-party CA or need more fine-grained control over certificate management.
To configure the Consul client agents when using auto-encryption you only need to distribute the CA certificate that will be used to validate the certificates exposed by other Consul agents.
HCL
addresses = {
https = "0.0.0.0"
}
ports {
https = 8501
}
tls {
defaults {
ca_file = "consul-agent-ca.pem"
verify_incoming = true
verify_outgoing = true
verify_server_hostname = true
}
}
auto_encrypt = {
tls = true
}
JSON
{
"addresses": {
"https": "0.0.0.0"
},
"ports": {
"https": 8501
},
"tls": {
"defaults": {
"ca_file": "consul-agent-ca.pem",
"verify_incoming": true,
"verify_outgoing": true,
"verify_server_hostname": true
}
},
"auto_encrypt" : {
"tls" : true
}
}
Start Consul clients
Now that you have configured your clients nodes, you can start the Consul process.
$ systemctl start consul
Your client agents' communications will now be encrypted using mutual TLS encryption.
Rotate TLS certificates
To maintain the security offered by TLS encryption we recommend to rotate TLS certificates often.
TLS certificates are part of Consul's reloadable configuration so you do not need to restart the Consul agents to renew certificates and there is no risk of downtime.
To rotate certificates for Consul server agents complete the following steps:
- Generate new certificates for all server agents to replace the old ones and distribute them to server nodes.
- Reload Consul configuration on all servers using
consul reload
.
To rotate certificates for Consul client agents complete the following steps:
When using the auto-encryption method, the client certificates will be automatically rotated without the need for an operator intervention.
Examples
The configuration snippets provided in this page are valid to configure complete mTLS for your Consul datacenter. This means that all interfaces require the client to provide a valid certificate in order to communicate with the Consul agent. This is valid for all requests, API, CLI, and UI.
Since Consul 1.12
it is possible to have different settings for the HTTP protocol, used for the Consul's REST API, the CLI integration, and the UI, and the RPC protocol, used for internal communications between the Consul agents.
Interact with Consul without a client certificate
If you want to avoid the need to present a valid client certificate every time you need to interact with Consul API, CLI, or UI, it is possible to configure Consul to trust all incoming HTTPs connections. Setting tls.https.verify_incoming
to false
removes the need to present a client certificate. RCP communications are still mTLS encrypted.
addresses = {
https = "0.0.0.0"
}
ports {
https = 8501
http = -1
}
tls {
https {
ca_file = "/etc/consul.d/consul-agent-ca.pem"
cert_file = "/etc/consul.d/consul-agent.pem"
key_file = "/etc/consul.d/consul-agent-key.pem"
verify_incoming = false
verify_outgoing = true
}
internal_rpc {
ca_file = "/etc/consul.d/consul-agent-ca.pem"
cert_file = "/etc/consul.d/consul-agent.pem"
key_file = "/etc/consul.d/consul-agent-key.pem"
verify_incoming = true
verify_outgoing = true
verify_server_hostname = true
}
}
Use HTTP for local client interaction
If you want to avoid the need to present a valid client certificate or to use the CA certificate every time you need to interact with your local Consul agent, it is possible to configure Consul to keep the HTTP listener only active on the localhost
interface. External requests to the API or the UI will still be TLS protected but setting tls.https.verify_incoming
to false
removes the need to present a client certificate. RCP communications are still mTLS encrypted.
addresses = {
https = "0.0.0.0"
http = "127.0.0.1"
}
ports {
https = 8501
http = 8500
}
tls {
https {
ca_file = "/etc/consul.d/consul-agent-ca.pem"
cert_file = "/etc/consul.d/consul-agent.pem"
key_file = "/etc/consul.d/consul-agent-key.pem"
verify_incoming = false
verify_outgoing = true
}
internal_rpc {
ca_file = "/etc/consul.d/consul-agent-ca.pem"
cert_file = "/etc/consul.d/consul-agent.pem"
key_file = "/etc/consul.d/consul-agent-key.pem"
verify_incoming = true
verify_outgoing = true
verify_server_hostname = true
}
}
Next steps
After completing the steps in this topic, your agents will be configured with TLS for encrypted communication. With the auto encryption method, your client certificates are managed by the server. With the operator method, you distributed all the certificates, but have a more flexible configuration.
Documentation for the commands used in this topic is available at Consul agent configuration - TLS configuration reference documentation and at the consul tls
CLI command reference.
To learn how to automate TLS certificate generation and rotation, refer to the Generate mTLS Certificates for Consul with Vault tutorial.
The other prerequisites for a secure Consul deployment are gossip encryption and Access Control Lists (ACLs) with default deny policy.