Combining IBM Cloudant with other IBM services
Each database technology has its own strengths and weaknesses. Some are built for high availability and data durability (at the expense of more hardware and extra cost). Others favor speed and can churn out blazingly fast queries (but might lose data in a sudden power failure).
In this tutorial, you combine two IBM® services, IBM® Cloudant® for IBM Cloud® and IBM Cloud® Databases for Redis to optimize for speed and cost by implementing a caching system for database queries.
This tutorial takes less than an hour to complete. It is not entirely cost-free because the Databases for Redis service does not come with a free tier. However, if you deprovision the services after you finish with them, you should not have to pay more than a few dollars.
The Project - Team Directory
Team Directory is a web app that contains the details of all divisional employees. These employees are assigned to six different color-coded teams (Red, Orange, Green, Blue, Yellow, and Purple). When you visit the app, you can select one team and see the names of all the members of that team. The app also shows you whether this list of team members was obtained from the IBM Cloudant database or from the Databases for Redis cache and how long the query took. You can see just how much quicker cached data is to retrieve.
Objectives
- Learn how to provision multiple services on the IBM Cloud® by using Terraform.
- Learn how to use the IBM Cloudant NodeJS SDK to access your IBM Cloud® services.
Prerequisites
You need the following items:
- An IBM Cloud® pay-as-you-go account
- Access to a Mac or Linux™ terminal
- Git
- Node.js and npm
- Terraform
- Jq, a command-line tool to process JSON data
Step 1. Obtain an API key to deploy infrastructure to your account
Follow the steps in this document to create an API key and make a note of it.
Step 2. Clone the code repo and cd into the Terraform directory
Now, you get the code and create a credentials file.
-
In a terminal window, type the following command:
git clone https://github.com/IBM-Cloud/team-directory.git cd team-directory
-
Create a document called
terraform.tfvars
with the following fields:ibmcloud_api_key = "<your_api_key_from_step_1>" region = "eu-gb" redis_password = "<make_up_a_password>"
Step 3. Create the infrastructure
-
Create all the infrastructure that you need by running the Terraform script:
terraform apply --auto-approve
The Terraform folder contains a number of simple scripts, which you can see in the following list:
main.tf
- Tells Terraform to use the IBM Cloud.
variables.tf
- Contains the variable definitions whose values are populated from terraform.tfvars.
redis.tf
- Creates a Databases for Redis instance and some credentials to access it.
cloudant.tf
- Creates an IBM Cloudant free-tier instance and some credentials to access it.
It takes several minutes for the resources to be ready. Now you have an IBM Cloudant instance and a Databases for Redis instance that you can access.
You can have only one free-tier instance per account. If you already have one, you must delete it, or change the
plan
variable tostandard
. -
Check out these instances by visiting the Resources section of your IBM Cloud® account.
The Terraform script outputs several bits of information that you use in the next steps.
Step 4. Run the service
In this step, you run a script that creates all the environment variables that your service needs and then runs the service.
In the terminal, type the following command:
./build.sh
When you run this script, it takes the data output from the Terraform script and uses the jq
facility to parse the content and create the environment variables needed. It then runs the script (server.js
) that initializes
the database and populates it with data.
Step 5. Visit your website
-
Open a browser and visit
https://localhost:8080
. See the button for each team in the following screen capture:Team Directory team selectors -
Click any of the colored buttons and obtain a list of team members.
-
Look at where the data came from, IBM Cloudant (cache = false) or Databases for Redis (cache = true) and how long it took.
-
Click the same button that you clicked in step two. You can see the data that comes from the Databases for Redis cache and takes a lot less time to execute in the following screen capture:
Team Directory team list and cache information In a real cloud-based application, the application server and the Databases for Redis instance would be close to each other (in the same data center) so latency between the two would be only a few milliseconds. In this example, extra network hops exist between your locally hosted application server and the cloud-hosted Databases for Redis cache so latency gains aren't as good as in production.
The cached data is set to expire within 60 seconds. So if you return to one of the teams after 60 seconds, the data is again retrieved from the database and not the cache.
-
Use the Clear Cache button to remove all cached data from Databases for Redis if necessary.
About the Code
The application is a simple Node.js application. It uses three main packages:
@ibm-cloud/cloudant
to connect to IBM Cloudant and read/write data.redis
to connect to the Databases for Redis instance and read/write data.- Express to enable a simple web server that allows users to interact with the data.
The following sections describe the two main files.
server.js
This server.js
file runs the web server and communicates with IBM Cloudant and Databases for Redis. When the front end submits a team selection to the team
route (see index.html
),
the app.route
function first checks the cache to see whether it has the data already. If it does, then it returns that. Otherwise, it makes a query to IBM Cloudant to retrieve team data, store it in the cache, and return it
to the front end.
The read operation uses an IBM Cloudant design document and a MapReduce view to select documents. This selection is beyond the scope of this tutorial, but you can read more about views and design documents in the documentation.
This script also contains some code that uploads test data (contained in the directorydata.json
file) to the database the first time it runs.
index.html
The index.html page is the only page of the application that is using the Vue.js framework. When it loads, it shows you the available teams.
When you select a team, it makes an HTTP POST request with your choice to the /team
route of the application (see redis.js
). A successful return from the application contains all the
data for the team members. We are displaying only their names and town for simplicity.
Summary
Today, we combined two IBM Cloud services to optimize cost and user experience: IBM Cloudant as a document store and query engine and Databases for Redis as a content cache. Cached documents can be retrieved more quickly and more cheaply, but the tradeoff is that your application might be showing old data to your users for a time.
If you followed this tutorial, you must deprovision your resources to stop incurring charges. You can deprovision from the terraform
directory on your terminal by typing the following command, terraform destroy --auto-approve
.