Best practices for Databases for Redis

What is Databases for Redis?

Databases for Redis is a managed Redis OSS service offered on IBM Cloud. It is an in-memory data structure store used as a database, cache, message broker, and streaming engine. Unlike traditional databases, which store data on disk, Redis stores data in memory (RAM) enabling low latency. The data is ephemeral, which enables Databases for Redis to offer top performance and high throughput. You can also configure Redis to store data persistently on disk by trading off performance with data availability, because this is achieved by enabling AOF (Append Only File) sync with RBD snapshots.

RBD snapshots are enabled for backup and high availability, even if persistence is disabled.

Best practices for instance capacity planning

You must plan Databases for Redis instance capacity based on your application and architectural design, understand hardware requirements, and prepare for the increase or decrease in demand. The following recommendations can help you to optimize the size your instance.

Understand your data

You are expected to know the data types, size, and lifetime of your data. This helps you understand the duration of data available in-memory before it gets deleted or moved.

Estimate memory requirements

It is important to calculate your memory requirements. Remember to factor in not only your data, but also the memory required for replication, client connections, max-memory buffers, and Redis metadata.

Understand the read/write loads

Identifying your read/write loads helps you prepare for auto-scaling needs, time your application requests, and maintain master-follower sync.

Understand your IOPS needs

Input/output operations per second is a key factor that you should consider in instance capacity planning. Databases for Redis takes RDB snapshots periodically in keeping with the default Redis configuration, which engages disk even if your instance is set up for cache. It's possible for very busy databases to exceed the IOPS for the disk size, and increasing disk size can alleviate a performance bottleneck.

Plan for redundancy and reconnects

There are multiple components that are involved in cloud computation, which can cause momentary failures. However, at IBM® Cloud Databases, we offer 99.99% high availability, and encourage you to plan for connection blips in your application design using retry and reconnect logic.

Consider network bandwidth

Your network bandwidth might significantly impact your Databases for Redis performance. Ensure that you have sufficient network bandwidth to handle your database loads.

Monitoring and adjustments

Our recommendation is to monitor your Databases for Redis instance performance and usage to determine the evolving usage pattern of your database instance and resize as needed.

Best practices for performance

Disable persistence

By default, Databases for Redis has persistance enabled. This writes AOF sync and increases IOPS load. If your application doesn’t need to persist data, disable this using the command Set appendonly = no. For more information, see
Setting an example cache

RAM vs cores

Redis is a single-threaded, in-memory database. Inherently, it needs more RAM than CORES, unlike other persistent databases. Even though it is single-threaded, it uses ‘multiplexing’ to process the requests, but all the requests are processed by a thread. However, other cores are needed to maintain database integrity and stability for its internal processes. You are encouraged to focus more on RAM and disk (for IOPS) for Databases for Redis. For more information, see Best practices for instance capacity planning.

Downsizing memory

You are advised to take care when you reduce the memory of your Databases for Redis instance. Because Redis is an in-memory database, its memory holds your data for storage, processing, and retrieval purposes. Drastically reducing the memory can temporarily stop your instance returning an error, because there is not sufficient space available to complete the operations. For example, consider 20 GB of data trying to load in 15 GB of memory, this case is bound to return an error.

Avoid expensive commands

Certain commands in Redis are expensive to run. For example, the KEYS command, which is used frequently, but should be avoided. Instead, use the SCAN command, which spreads the iteration over many calls and does not tie up your whole server at one time.

Choose an eviction policy

You should choose an eviction policy that works for your application. By default, deployments are configured with a noeviction policy. Use eviction policies like allkeys-lru, volatile-lru, allkeys-random, volatile-random, volatile-ttl. For more details, see memory policy.

Set maxmemory values

You can adjust the maxmemory value. However, set a reasonable limit, otherwise your data can consume all the available memory and your deployment can run out of resources. By default, we set it at 80% of the data node’s available memory.

Set TTL (Time-To-Live) policies

TTL is a great feature where keys are deleted from the database after a defined time. This is extremely helpful if you are using Redis a cache. However, be careful about setting a very short or a very long value because a very short value can create recomputation of values and a very long value can create unnecessary memory use. See TTL command for more details.

Best practices for high availability

Retry and reconnect logic

Systems are prone to disruptions. You are highly encouraged to implement retry and reconnect logic into your application architecture to avoid disruptions. Use IOREDIS, NODEREDIS or any other package of your choice to ensure continuity of your application.

For more details, see Error detection and handling with Redis blog post.

Best practices for monitoring

You must monitor your LogDNA for the following common errors and take corrective measures:

• AOF sync
• Read-only available
• Connection to master lost
• For guidance about preventing errors, see How can I avoid common Databases for Redis errors?.

General best practices

Connection pooling

Creating or closing connections is costly. Managing connections efficiently is important and connection pooling is helpful to minimize the overhead associated with opening and closing connections.

Connection limits

Utilize connections efficiently. Overloading Databases for Redis with too many connections will return errors and you will face application disruptions. Leave some connections available, as a number of them are reserved internally to maintain the state and integrity of your database. You are encouraged to use connection pooling.

Connection timeouts

Setting appropriate timeout values for your connections is also important to prevent resources being tied up indefinitely. However, be careful with setting short timeouts because this can lead to connection churn and increased latency. Align timeouts with the operational expectations of your application.

Use the Redis pipeline feature

Redis pipelining is a technique for improving performance by issuing multiple commands at once without waiting for the response to each individual command. For more information, see Redis pipelining.

Use the Redis Streams feature

Redis Streams is a data type that provides a super fast in-memory abstraction of an append-only log.

Split large data

You are recommended to split large datasets into smaller chunks with more keys, that is, split your data over multiple keys.

Batch schedule

Databases for Redis is scheduled to create automated backups everyday at a scheduled time. During this time, your databases IOPS are utilized. It is recommended that you should not run your own batch jobs at this time.

Set up notification channels

We recommend that Databases for Redis you set up email IDs in IBM Accounts to receive periodic updates about version changes, end-of-life, or maintenance schedules. You can also monitor your IBM Account notification icon to receive these updates.

The Best Practices for Redis on the IBM Cloud blog post provides further best practice information.