IBM Cloud Docs
Combining IBM Cloudant with other IBM services

Combining IBM Cloudant with other IBM services

Each database technology has its own strengths and weaknesses. Some are built for high availability and data durability (at the expense of more hardware and extra cost). Others favor speed and can churn out blazingly fast queries (but might lose data in a sudden power failure).

In this tutorial, you combine two IBM® services, IBM® Cloudant® for IBM Cloud® and IBM Cloud® Databases for Redis to optimize for speed and cost by implementing a caching system for database queries.

This tutorial takes less than an hour to complete. It is not entirely cost-free because the Databases for Redis service does not come with a free tier. However, if you deprovision the services after you finish with them, you should not have to pay more than a few dollars.

The Project - Team Directory

Team Directory is a web app that contains the details of all divisional employees. These employees are assigned to six different color-coded teams (Red, Orange, Green, Blue, Yellow, and Purple). When you visit the app, you can select one team and see the names of all the members of that team. The app also shows you whether this list of team members was obtained from the IBM Cloudant database or from the Databases for Redis cache and how long the query took. You can see just how much quicker cached data is to retrieve.

Objectives

  1. Learn how to provision multiple services on the IBM Cloud® by using Terraform.
  2. Learn how to use the IBM Cloudant NodeJS SDK to access your IBM Cloud® services.

Prerequisites

You need the following items:

  1. An IBM Cloud® pay-as-you-go account
  2. Access to a Mac or Linux™ terminal
  3. Git
  4. Node.js and npm
  5. Terraform
  6. Jq, a command-line tool to process JSON data

Step 1. Obtain an API key to deploy infrastructure to your account

Follow the steps in this document to create an API key and make a note of it.

Step 2. Clone the code repo and cd into the Terraform directory

Now, you get the code and create a credentials file.

  1. In a terminal window, type the following command:

    git clone https://github.com/IBM-Cloud/team-directory.git
    cd team-directory
    
  2. Create a document called terraform.tfvars with the following fields:

    ibmcloud_api_key = "<your_api_key_from_step_1>"
    region = "eu-gb"
    redis_password  = "<make_up_a_password>"
    

Step 3. Create the infrastructure

  1. Create all the infrastructure that you need by running the Terraform script:

    terraform apply --auto-approve
    

    The Terraform folder contains a number of simple scripts, which you can see in the following list:

    main.tf
    Tells Terraform to use the IBM Cloud.
    variables.tf
    Contains the variable definitions whose values are populated from terraform.tfvars.
    redis.tf
    Creates a Databases for Redis instance and some credentials to access it.
    cloudant.tf
    Creates an IBM Cloudant free-tier instance and some credentials to access it.

    It takes several minutes for the resources to be ready. Now you have an IBM Cloudant instance and a Databases for Redis instance that you can access.

    You can have only one free-tier instance per account. If you already have one, you must delete it, or change the plan variable to standard.

  2. Check out these instances by visiting the Resources section of your IBM Cloud® account.

The Terraform script outputs several bits of information that you use in the next steps.

Step 4. Run the service

In this step, you run a script that creates all the environment variables that your service needs and then runs the service.

In the terminal, type the following command:

./build.sh

When you run this script, it takes the data output from the Terraform script and uses the jq facility to parse the content and create the environment variables needed. It then runs the script (server.js) that initializes the database and populates it with data.

Step 5. Visit your website

  1. Open a browser and visit https://localhost:8080. See the button for each team in the following screen capture:

    {Click the button for each team.
    Team Directory team selectors

  2. Click any of the colored buttons and obtain a list of team members.

  3. Look at where the data came from, IBM Cloudant (cache = false) or Databases for Redis (cache = true) and how long it took.

  4. Click the same button that you clicked in step two. You can see the data that comes from the Databases for Redis cache and takes a lot less time to execute in the following screen capture:

    Click the team button and see the members of the team.
    Team Directory team list and cache information

    In a real cloud-based application, the application server and the Databases for Redis instance would be close to each other (in the same data center) so latency between the two would be only a few milliseconds. In this example, extra network hops exist between your locally hosted application server and the cloud-hosted Databases for Redis cache so latency gains aren't as good as in production.

    The cached data is set to expire within 60 seconds. So if you return to one of the teams after 60 seconds, the data is again retrieved from the database and not the cache.

  5. Use the Clear Cache button to remove all cached data from Databases for Redis if necessary.

About the Code

The application is a simple Node.js application. It uses three main packages:

  • @ibm-cloud/cloudant to connect to IBM Cloudant and read/write data.
  • redis to connect to the Databases for Redis instance and read/write data.
  • Express to enable a simple web server that allows users to interact with the data.

The following sections describe the two main files.

server.js

This server.js file runs the web server and communicates with IBM Cloudant and Databases for Redis. When the front end submits a team selection to the team route (see index.html), the app.route function first checks the cache to see whether it has the data already. If it does, then it returns that. Otherwise, it makes a query to IBM Cloudant to retrieve team data, store it in the cache, and return it to the front end.

The read operation uses an IBM Cloudant design document and a MapReduce view to select documents. This selection is beyond the scope of this tutorial, but you can read more about views and design documents in the documentation.

This script also contains some code that uploads test data (contained in the directorydata.json file) to the database the first time it runs.

index.html

The index.html page is the only page of the application that is using the Vue.js framework. When it loads, it shows you the available teams.

When you select a team, it makes an HTTP POST request with your choice to the /team route of the application (see redis.js). A successful return from the application contains all the data for the team members. We are displaying only their names and town for simplicity.

Summary

Today, we combined two IBM Cloud services to optimize cost and user experience: IBM Cloudant as a document store and query engine and Databases for Redis as a content cache. Cached documents can be retrieved more quickly and more cheaply, but the tradeoff is that your application might be showing old data to your users for a time.

If you followed this tutorial, you must deprovision your resources to stop incurring charges. You can deprovision from the terraform directory on your terminal by typing the following command, terraform destroy --auto-approve.