Auto-renew Cloudflare origin CA with Terraform and Helm

Photo by Maxim Abramov on Unsplash

First, Why Cloudflare?

According to data from W3Techs, 81.2% of all websites that use a CDN or reverse proxy rely on Cloudflare.

Cloudflare’s services are growing in popularity for two reasons. First, its CDN caches copies of digital media on servers that are located closer to a website’s visitors. This enables media-heavy websites to load much faster.

Second, websites have been increasingly targeted by bots, DDoS attacks, and data breaches. Instead of requiring a website’s host and visitors to rely on updating their own cybersecurity software, Cloudflare’s platform acts as an online “filtration system” that proactively filters out malicious attacks.

Second, What is OriginIssuer?

Managing TLS certificates and keys within K8s can be challenging and error prone. The secret resources have to be constructed correctly, as components expect secrets with specific fields. Some forms of domain verification require manually rotating secrets to pass. Once you’re successful, don’t forget to renew before the certificate expires.

K8s lets you automate the infrastructure required to run containerized applications as well as management and operational tasks. This level of automation is a must if you want to run containerized applications at scale in a production environment.

Automating an IT environment means converting human knowledge and effort (usually a painful effort) into software. This is what K8s operators (In this case, cert-manager) do, taking all known information and the application lifecycle and then systematizing the info into an automated form. K8s can then handle these tasks for you.

Below diagram (from another amazing post here) shows you the relationship between cert-manager and an external cert-manager issuer

There are two controllers we are going to create here

The OriginIssuer controller watches for creation and modification of OriginIssuer custom resources. The controllers create a Cloudflare API client using the details and credentials referenced. This client API instance will later be used to sign certificates through the API. The controller will periodically retry to create an API client; once it is successful, it updates the OriginIssuer’s status to be ready.

The CertificateRequest controller watches for the creation and modification of cert-manager’s CertificateRequest resources. These resources are created automatically by cert-manager as needed during a certificate’s lifecycle.

The controller looks for Certificate Requests that reference a known OriginIssuer, this reference is copied by cert-manager from the origin Certificate resource, and ignores all resources that do not match. The controller then verifies the OriginIssuer is in the ready state, before transforming the certificate request into an API request using the previously created clients.

On a successful response, the signed certificate is added to the certificate request, and which cert-manager will use to create or update the secret resource. On an unsuccessful request, the controller will periodically retry.

Third, why this post?

For Automatic DNS, SSL, and Load Balancing we see amazing terraform tutorial and Helm chart for external-dns and aws-load-balancer-controller. One of the missing tutorial I couldn’t find when I write this post is terraform code for origin-ca-issuer.

Since Cloudflare proxying all external traffic to Load Balancer (Ingress controlled Load Balancer) with default browser side SSL certificate

we only need Origin Server TLS certificate for a full SSL tunnel

As Cloudflare mentioned in End-to-end HTTPS with Cloudflare — Part 3: SSL options, you can provide your self-signed certificate for Full mode or you can provide a Cloudflare Origin CA certificate (or a valid certificate purchased from a CA) to enable Full (strict)mode

Origin CA Issuer is a cert-manager CertificateRequest controller for Cloudflare’s Origin CA feature. Here is a perfect post Create Automatic SSL and DNS from Cloudflare for Kubernetes Ingress Resources on how to do it on YAML way and on this post we are going to do it through Terraform and Helm.

Below are the steps break down

1. Create cert-manager
2. Create origin-ca-issuer-crds
3. Create origin-ca-issuer-rbac
4. Create origin-ca-issuer-manifests (namespace, deployment, serviceaccount)
5. Create kubernetes_secret — origin-ca-key
6. Create origin-ca-issuer
7. Create k8s-wildcard-cert
8. Create k8s-istio-gateway
9. Create kiali_virtualservice

Here we go!

1. First, we setup cert-manager

2. Second, create origin-ca-issuer-crds

3. Third, create namespace

4. Since orginal-ca’s helm chart wasn’t on any helm repo you kinda have to download it into your repo folder, this helm chart will take care of rbac, deployment and serviceaccount

5. And then get your Origin CA Key from API Token section of your Cloudflare dashboard and create a kubernetes_secret resource

6. This OriginIssuer resource creates a binding between cert-manager and the Cloudflare API for an account.

One tricky part about creating original issuer is that you can’t use Terraform kubernetes_manifest because when custom resource definitions have already been created, configuration can be deployed without any problems, but when they are not present —meaning when you rely on the configuration to create them, this is when the GroupVersionResource error appears. If you would like to avoid this problem. use kubectl_manifest instead. you can find the issuer.yaml from here.

7. After creating an OriginIssuer, we can now create a Certificate with cert-manager. This defines the domains, including wildcards, that the certificate should be issued for, how long the certificate should be valid, and when cert-manager should renew the certificate.

Once created, cert-manager begins managing the lifecycle of this certificate, including creating the key material, crafting a certificate signature request (CSR), and constructing a certificate request that will be processed by the origin-ca-issuer.

8. Your setup for Istio gateway

9. We use Kiali as an example here to show how to setup virtual service

Now we got:

• DNS record for *.test.com proxied by Cloudflare
• SSL certificate for your origin server signed by Cloudflare
• Full (strict) mode enabled
• Orange Cloud enabled

and you’re ready to go. All done!