Planning app deployments

What kind of apps can I run in IBM Cloud Kubernetes Service?

Stateless apps

Stateless apps are preferred for cloud-native environments like Kubernetes. They are simple to migrate and scale because they declare dependencies, store configurations separately from the code, and treat backing services such as databases as attached resources instead of coupled to the app. The app pods don't require persistent data storage or a stable network IP address, and as such, pods can be terminated, rescheduled, and scaled in response to workload demands. The app uses a Database-as-a-Service for persistent data, and NodePort, load balancer, or Ingress services to expose the workload on a stable IP address.

Stateful apps

Stateful apps are more complicated than stateless apps to set up, manage, and scale because the pods require persistent data and a stable network identity. Stateful apps are often databases or other distributed, data-intensive workloads where processing is more efficient closer to the data itself. If you want to deploy a stateful app, you need to set up persistent storage and mount a persistent volume to the pod that is controlled by a StatefulSet object. You can choose to add file, block, or object storage as the persistent storage for your stateful set. You can also install Portworx on your bare metal worker nodes and use Portworx as a highly available software-defined storage solution to manage persistent storage for your stateful apps. For more information about how stateful sets work, see the Kubernetes documentation.

What are some guidelines for developing stateless, cloud-native apps?

Check out the Twelve-Factor App, a language-neutral methodology for considering how to develop your app across 12 factors, summarized as follows.

1. Code base: Use a single code base in a version control system for your deployments. When you pull an image for your container deployment, specify a tested image tag instead of using latest.
2. Dependencies: Explicitly declare and isolate external dependencies.
3. Configuration: Store deployment-specific configuration in environment variables, not in the code.
4. Backing services: Treat backing services, such as data stores or message queues, as attached or replaceable resources.
5. App stages: Build in distinct stages such as build, release, run, with strict separate among them.
6. Processes: Run as one or more stateless processes that share nothing and use persistent storage for saving data.
7. Port binding: Port bindings are self-contained and provide a service endpoint on well-defined host and port.
8. Concurrency: Manage and scale your app through process instances such as replicas and horizontal scaling. Set resource requests and limits for your deployments. Note that Calico network policies can't limit bandwidth. Instead, consider Istio.
9. Disposability: Design your app to be disposable, with minimal startup, graceful shutdown, and toleration for abrupt process terminations. Remember, containers, pods, and even worker nodes are meant to be disposable, so plan your app accordingly.
10. Dev-to-prod parity: Set up a continuous integration and continuous delivery pipeline for your app, with minimal difference between the app in development and the app in prod.
11. Logs: Treat logs as event streams: the outer or hosting environment processes and routes log files. Important: In IBM Cloud Kubernetes Service, logs are not turned on by default. To enable, see Configuring log forwarding.
12. Admin processes: Keep any one-time admin scripts with your app and run them as a Kubernetes Job object to ensure that the admin scripts run with the same environment as the app itself. For orchestration of larger packages that you want to run in your Kubernetes clusters, consider using a package manager such as Helm.

What about serverless apps?

You can run serverless apps and jobs through the IBM Cloud Code Engine service. Code Engine can also build your images for you. Code Engine is designed so that you don't need to interact with the underlying technology it is built upon. However, if you have existing tooling that is based upon Kubernetes or Knative, you can still use it with Code Engine. For more information, see Using Kubernetes to interact with your application.

I already have an app. How can I migrate it to IBM Cloud Kubernetes Service?

You can take some general steps to containerize your app as follows.

1. Use the Twelve-Factor App as a guide for isolating dependencies, separating processes into separate services, and reducing the statefulness of your app as much as possible.
2. Find an appropriate base image to use. You can use publicly available images from Docker Hub, public IBM images, or build and manage your own in your private IBM Cloud Container Registry.
3. Add to your Docker image only what is necessary to run the app.
4. Instead of relying on local storage, plan to use persistent storage or cloud database-as-a-service solutions to back up your app's data.
5. Over time, refactor your app processes into microservices.

I thought that I needed to put my app in a container. Now what's all this stuff about pods?

A pod is the smallest deployable unit that Kubernetes can manage. You put your container (or a group of containers) into a pod and use the pod configuration file to tell the pod how to run the container and share resources with other pods. All containers that you put into a pod run in a shared context, which means that they share the virtual or physical machine.

What to put in a container

As you think about your application's components, consider whether they have significantly different resource requirements for things like CPU and memory. Could some components run at a best effort, where going down for a little while to divert resources to other areas is acceptable? Is another component customer-facing, so it's critical for it to stay up? Split them up into separate containers. You can always deploy them to the same pod so that they run together in sync.

What to put in a pod

The containers for your app don't always have to be in the same pod. In fact, if you have a component that is stateful and difficult to scale, such as a database service, put it in a different pod that you can schedule on a worker node with more resources to handle the workload. If your containers work correctly if they run on different worker nodes, then use multiple pods. If they need to be on the same machine and scale together, group the containers into the same pod.

So if I can use a pod, why do I need all these different types of objects?

Creating a pod YAML file is easy. You can write one with just a few lines as follows.

apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
  - name: nginx
    image: nginx
    ports:
    - containerPort: 80

But you don't want to stop there. If the node that your pod runs on goes down, then your pod goes down with it and isn't rescheduled. Instead, use a deployment to support pod rescheduling, replica sets, and rolling updates. A basic deployment is almost as easy to make as a pod. Instead of defining the container in the spec by itself, however, you specify replicas and a template in the deployment spec. The template has its own spec for the containers within it, such as follows.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
        ports:
        - containerPort: 80

You can keep adding features, such as pod anti-affinity or resource limits, all in the same YAML file.

For a more detailed explanation of different features that you can add to your deployment, see Making your app deployment YAML file.

What type of Kubernetes objects can I make for my app?

When you prepare your app YAML file, you have many options to increase the app's availability, performance, and security. For example, instead of a single pod, you can use a Kubernetes controller object to manage your workload, such as a replica set, job, or daemon set. For more information about pods and controllers, view the Kubernetes documentation. A deployment that manages a replica set of pods is a common use case for an app.

For example, a kind: Deployment object is a good choice to deploy an app pod because with it, you can specify a replica set for more availability for your pods.

The following table describes why you might create different types of Kubernetes workload objects.

What if I want my app configuration to use variables? How do I add these variables to the YAML?

To add variable information to your deployments instead of hardcoding the data into the YAML file, you can use a Kubernetes ConfigMap or Secret object.

To consume a ConfigMap or secret, you need to mount it to the pod. The ConfigMap or secret is combined with the pod just before the pod is run. You can reuse a deployment spec and image across many apps, but then swap out the customized configmaps and secrets. Secrets in particular can take up a lot of storage on the local node, so plan accordingly.

Both resources define key-value pairs, but you use them for different situations.

Configmap

Provide non-sensitive configuration information for workloads that are specified in a deployment. You can use configmaps in three main ways.

• File system: You can mount an entire file or a set of variables to a pod. A file is created for each entry based on the key name contents of the file that are set to the value.
• Environment variable: Dynamically set the environment variable for a container spec.
• Command-line option: Set the command-line option that is used in a container spec.

Secret

Provide sensitive information to your workloads, such as follows. Other users of the cluster might have access to the secret, so be sure that you know the secret information can be shared with those users.

• Personally identifiable information (PII: Store sensitive information such as email addresses or other types of information that are required for company compliance or government regulation in secrets.
• Credentials: Put credentials such as passwords, keys, and tokens in a secret to reduce the risk of accidental exposure. For example, when you bind a service to your cluster, the credentials are stored in a secret.

Want to make your secrets even more secured? Ask your cluster admin to enable a key management service provider in your cluster to encrypt new and existing secrets.

How can I make sure that my app has the correct resources?

When you specify your app YAML file, you can add Kubernetes functionalities to your app configuration that help your app get the correct resources. In particular, set resource limits and requests for each container that is defined in your YAML file.

Additionally, your cluster admin might set up resource controls that can affect your app deployment, such as the following.

• Resource quotas
• Pod priority

How can I add capabilities to my app configuration?

See Specifying your app requirements in your YAML file for descriptions of what you might include in a deployment. The example includes the following options.

• Replica sets
• Labels
• Affinity
• Image policies
• Ports
• Resource requests and limits
• Liveness and readiness probes
• Services to expose the app service on a port.
• Configmaps to set container environment variables.
• Secrets to set container environment variables.
• Persistent volumes that are mounted to the container for storage.

How can I add IBM services to my app, such as Watson?

See Adding services to apps.

How can I increase the availability of my app?

Consider the following options to increase availability of your app.

Use deployments and replica sets to deploy your app and its dependencies

A deployment is a Kubernetes resource that you can use to declare all the components of your app and its dependencies. With deployments, you don't have to write down all the steps and instead can focus on your app. When you deploy more than one pod, a replica set is automatically created for your deployments that monitors the pods and assures that the specified number of pods is up and running. When a pod goes down, the replica set replaces the unresponsive pod with a new one. You can use a deployment to define update strategies for your app, including the number of pods that you want to add during a rolling update and the number of pods that can be unavailable at a time. When you perform a rolling update, the deployment checks whether the revision is working and stops the rollout when failures are detected. With deployments, you can concurrently deploy multiple revisions with different options. For example, you can test a deployment first before you decide to push it to production. By using Deployments, you can track any deployed revisions. You can use this history to roll back to a previous version if you encounter that your updates are not working as expected.

Include enough replicas for your app's workload, plus two

To make your app even more highly available and more resilient to failure, consider including extra replicas than the minimum to handle the expected workload. Extra replicas can handle the workload in case a pod crashes and the replica set did not yet recover the crashed pod. For protection against two simultaneous failures, include two extra replicas. This setup is an N+2 pattern, where N is the number of replicas to handle the incoming workload and +2 is an extra two replicas. As long as your cluster has enough space, you can have as many pods as you want.

Spread pods across multiple nodes (anti-affinity)

When you create your deployment, each pod can be deployed to the same worker node. This is known as affinity, or colocation. To protect your app against worker node failure, you can configure your deployment to spread your pods across multiple worker nodes by using the podAntiAffinity option with your standard clusters. You can define two types of pod anti-affinity: preferred or required. For more information, see the Kubernetes documentation on Assigning Pods to Nodes.

For an example of affinity in an app deployment, see Making your app deployment YAML file.

Distribute pods across multiple zones or regions

To protect your app from a zone failure, you can create multiple clusters in separate zones or add zones to a worker pool in a multizone cluster. Multizone clusters are available only in certain classic or VPC multizones, such as Dallas. If you create multiple clusters in separate zones, you must set up a global load balancer. When you use a replica set and specify pod anti-affinity, Kubernetes spreads your app pods across the nodes. If your nodes are in multiple zones, the pods are spread across the zones, increasing the availability of your app. If you want to limit your apps to run only in one zone, you can configure pod affinity, or create and label a worker pool in one zone.

In a multizone cluster deployment, are my app pods distributed evenly across the nodes?

The pods are evenly distributed across zones, but not always across nodes. For example, if you have a cluster with one node in each of three zones and deploy a replica set of six pods, then each node gets two pods. However, if you have a cluster with two nodes in each of three zones and deploy a replica set of six pods, each zone schedules two pods, and might schedule one pod per node or might not. For more control over scheduling, you can set pod affinity.

If a zone goes down, how are pods rescheduled onto the remaining nodes in the other zones?

It depends on your scheduling policy that you used in the deployment. If you included node-specific pod affinity, your pods are not rescheduled. If you did not, pods are created on available worker nodes in other zones, but they might not be balanced. For example, the two pods might be spread across the two available nodes, or they might both be scheduled onto one node with available capacity. Similarly, when the unavailable zone returns, pods are not automatically deleted and rebalanced across nodes. If you want the pods to be rebalanced across zones after the zone is back up, configure the Kubernetes descheduler. In multizone clusters, try to keep your worker node capacity at 50% per zone so that enough capacity remains to protect your cluster against a zonal failure.

What if I want to spread my app across regions?

To protect your app from a region failure, create a second cluster in another region, set up a global load balancer to connect your clusters, and use a deployment YAML to deploy a duplicate replica set with pod anti-affinity for your app.

What if my apps need persistent storage?

Use a cloud service such as IBM Cloudant or IBM Cloud Object Storage.

How can I scale my app?

If you want to dynamically add and remove apps in response to workload usage, see Scaling apps for steps to enable horizontal pod autoscaling.

How can I organize my deployments to make them easier to update and manage?

Now that you have a good idea of what to include in your deployment, you might wonder how are you going to manage all these different YAML files? Not to mention the objects that they create in your Kubernetes environment!

The following tips can help you organize your deployment YAML files.

• Use a version-control system, such as Git.
• Group closely related Kubernetes objects within a single YAML file. For example, if you are creating a deployment, you might also add the service file to the YAML. Separate objects with --- such as in the following example.

  apiVersion: apps/v1
  kind: Deployment
  metadata:
  ...
  ---
  apiVersion: v1
  kind: Service
  metadata:
  ...
  

• You can use the kubectl apply -f command to apply to an entire directory, not just a single file.
• Try out the kustomize project that you can use to help write, customize, and reuse your Kubernetes resource YAML configurations.

Within the YAML file, you can use labels or annotations as metadata to manage your deployments.

Labels

Labels are key:value pairs that can be attached to Kubernetes objects such as pods and deployments. They can be whatever you want, and are useful for selecting objects based on the label information. Labels provide the foundation for grouping objects. See the following examples for ideas for labels.

• app: nginx
• version: v1
• env: dev

Annotations

Annotations are similar to labels in that they are also key:value pairs. They are better for non-identifying information that can be leveraged by tools or libraries, such as holding extra information about where an object came from, how to use the object, pointers to related tracking repos, or a policy about the object. You don't select objects based on annotations.

What app update strategies can I use?

To update your app, you can choose from various strategies such as the following. You might start with a rolling deployment or instantaneous switch before you progress to a more complicated canary deployment.

Rolling deployment

You can use Kubernetes-native functionality to create a v2 deployment and to gradually replace your previous v1 deployment. This approach requires that apps are compatible with earlier version so that users who are served the v2 app version don't experience any breaking changes. For more information, see Managing rolling deployments to update your apps.

Instantaneous switch

Also referred to as a blue-green deployment, an instantaneous switch requires double the compute resources to have two versions of an app running at once. With this approach, you can switch your users to the newer version in near real time. Make sure that you use service label selectors (such as version: green and version: blue) to make sure that requests are sent to the correct app version. You can create the new version: green deployment, wait until it is ready, and then delete the version: blue deployment. Or you can perform a rolling update, but set the maxUnavailable parameter to 0% and the maxSurge parameter to 100%.

Canary or A/B deployment

A more complex update strategy, a canary deployment is when you pick a percentage of users such as 5% and send them to the new app version. You collect metrics in your logging and monitoring tools on how the new app version performs, do A/B testing, and then roll out the update to more users. As with all deployments, labeling the app (such as version: stable and version: canary) is critical. To manage canary deployments, you might install the managed Istio add-on service mesh, set up Monitoring for your cluster, and then use the Istio service mesh for A/B testing as described in this blog post.

How can I automate my app deployment?

If you want to run your app in multiple clusters, public and private environments, or even multiple cloud providers, you might wonder how you can make your deployment strategy work across these environments. With IBM Cloud and other open source tools, you can package your application to help automate deployments.

Set up a continuous integration and delivery (CI/CD) pipeline

With your app configuration files organized in a source control management system such as Git, you can build your pipeline to test and deploy code to different environments, such as test and prod. Work with your cluster administrator to set up continuous integration and delivery.

Package your app configuration files

Package your app with tools like Kustomize or Helm.

• With the kustomize project, you can write, customize, and reuse your Kubernetes resource YAML configurations.
• With the Helm Kubernetes package manager, you can specify all Kubernetes resources that your app requires in a Helm chart. Then, you can use Helm to create the YAML configuration files and deploy these files in your cluster. You can also integrate IBM Cloud-provided Helm charts to extend your cluster's capabilities, such as with a block storage plug-in.

Are you looking to create YAML file templates? Some people use Helm to do just that, or you might try out other community tools such as ytt.

How can I make sure that my services are connected to the correct deployments and ready to go?

For most services, add a selector to your service .yaml file so that it applies to pods that run your apps by that label. Many times when your app first starts up, you don't want it to process requests immediately. Add a readiness probe to your deployment so that traffic is only sent to a pod that is considered ready. For an example of a deployment with a service that uses labels and sets a readiness probe, check out this NGINX YAML.

Sometimes, you don't want the service to use a label. For example, you might have an external database or want to point the service to another service in a different namespace within the cluster. When this happens, you have to manually add an endpoints object and link it to the service.

How can I expose my services on the Internet?

You can create three types of services for external networking: NodePort, LoadBalancer, and Ingress.

You have different options that depend on your cluster type. For more information, see Planning networking services.

• Standard cluster: You can expose your app by using a NodePort, load balancer, or Ingress service.
• Cluster that is made private by using Calico: You can expose your app by using a NodePort, load balancer, or Ingress service. You also must use a Calico preDNAT network policy to block the public node ports.
• Private VLAN-only standard cluster: You can expose your app by using a NodePort, load balancer, or Ingress service. You also must open the port for the service's private IP address in your firewall.

As you plan how many Service objects you need in your cluster, keep in mind that Kubernetes uses iptables to handle networking and port forwarding rules. If you run many services in your cluster, such as 5000, performance might be impacted.

How can I control who has access to my app deployments?

The account and cluster administrators can control access on many different levels: the cluster, Kubernetes namespace, pod, and container.

With IBM Cloud IAM, you can assign permissions to individual users, groups, or service accounts at the cluster-instance level. You can scope cluster access down further by restricting users to particular namespaces within the cluster. For more information, see Assigning cluster access.

To control access at the pod level, you can configure pod security policies (PSPs).

Within the app deployment YAML, you can set the security context for a pod or container. For more information, review the Kubernetes documentation.

After I deploy my app, how can I monitor its health?

You can set up IBM Cloud logging and monitoring for your cluster. You might also choose to integrate with a third-party logging or monitoring service.