Why does the Terraform script that contains the Kubernetes cluster namespaces fails as connection refused?

When you Generate Plan for the Terraform script that contains cluster, used to work properly three months ago, but now the same Terraform script gets the following error.

"Error: Get "http://localhost/api/v1/namespaces/external-secrets": 
dial tcp [::1]:80: connect: connection refused".

When compared the logs between successful and failed Generate Plan execution. Observed new versions picked up of providers, forced the script to the same version just to ensure a version change did not cause the error. Still the error persists.

This issue is from Kubernetes provider, there are many discussion around this error by different Cloud providers, but there is no definite solution.

Check the following references and solutions provided to fix the issue

References

• Terraform don’t use Kubernetes provider with your cluster resource
• Many open issues on Kubernetes provider
• Stack overflow issue

Based on previous experience following are the two workaround solution to fix.

Solution 1

If you are using provider authentication to Kubernetes as shown in the codeblock.

provider "kubernetes" {
  config_path = data.ibm_container_cluster_config.cluster_config.config_file_path
}

Change the template as follows.

provider "kubernetes" {
  host                   = data.ibm_container_cluster_config.cluster_config.host
  client_certificate     = data.ibm_container_cluster_config.cluster_config.admin_certificate
  client_key             = data.ibm_container_cluster_config.cluster_config.admin_key
  cluster_ca_certificate = data.ibm_container_cluster_config.cluster_config.ca_certificate
}

Solution 2

You can remove the state related to cluster config from the statefile by using IBM Cloud Schematics CLI command and re-run the Terraform plan or Terraform apply command.

ibmcloud schematics workspace state rm --id --address <enter the workspace ID and the address of the resource to mark as taint>