Redis as a sidekiq queue in docker

Redis by default sets `appendonly no`, `maxmemory-policy noeviction`, and `save 3600 1 300 100 60 10000`. Many clients I work for need to learn about this. The default Redis configuration can grow out of memory or stop writing. 

In docker, for Sidekiq, this becomes especially painful during heavy load as it crashes and requires health checks and a restart policy. If you process money transactions, you might lose people's money.

The great thing about Redis is that it doesn't have a configuration file; it runs preconfigured with sensible defaults.

This means we only need to provide the configuration we want to override, like a docker-compose override.

Redis has two persistence modes that can be individually configured: AOF and RDB.

RDB - Redis Data Persistence
Let's talk about RDB first. The default:

save 3600 1 300 100 60 10000

The numbers are pairs. The first number (3600) is the frequency in seconds (every 1 hour) we save if we have one key change. Then, we save every 300 seconds (five minutes) if we have 60 key changes. Lastly, we save every minute if we have 10,000 key changes.

RDB is a snapshot in time containing all the data at that point. This is a potentially lossy approach. It should be considered the same as a PostgreSQL backup.

AOF - Append Only

AOF can be thought of as event sourcing or log stream. It writes all commands to the log, which can be read back from the beginning. Writing every command immediately might be a little extreme (performance-wise), but Redis already has our back with sensible defaults.

First, we need to turn on append-only mode, and that's done with `appendonly yes`. Next, it is essential to know about `appendfsync everysec`. It can be configured in different ways, but for processing Sidekiq queues, this should be a great start.

Data directory

This is important; we want a super-fast volume for the Redis database. Preferably, both here and for the PostgreSQL database, we want NVMe disks. SSDs work, too, but they are a little slower.

The data directory is configured with `dir /data` or `dir /redis-data`.

Final redis.conf (1st attempt)

# /usr/local/etc/redis/redis.conf

appendonly yes
appendfsync everysec
save 1500 1 300 100 60 1000
dir /data

This is combining what we talked about before.


We have several ways to use this configuration override. Let's inspect what redis-server says. 

➜  redis-server --help                                                                                                                                                                                                                      01/05/24 - 12:18 PM
Usage: ./redis-server [/path/to/redis.conf] [options] [-]
       ./redis-server - (read config from stdin)
       ./redis-server -v or --version
       ./redis-server -h or --help
       ./redis-server --test-memory <megabytes>
       ./redis-server --check-system

       ./redis-server (run the server with default conf)
       echo 'maxmemory 128mb' | ./redis-server -
       ./redis-server /etc/redis/6379.conf
       ./redis-server --port 7777
       ./redis-server --port 7777 --replicaof 8888
       ./redis-server /etc/myredis.conf --loglevel verbose -
       ./redis-server /etc/myredis.conf --loglevel verbose

Sentinel mode:
       ./redis-server /etc/sentinel.conf --sentinel

The easiest, therefore would be to just att these new settings as command line arguments:

version: '3'
    image: redis
    command: ["redis-server", "--appendonly", "yes", "--appendfsync", "everysec", "--save", "900 1 300 10 60 1000", "--dir", "/data"]
      - redis-data:/data

There are other alternatives, like building our own custom Dockerfile: 

FROM redis:7-alpine
COPY docker/redis/redis.conf /usr/local/etc/redis/redis.conf