Configuration File

The YAML configuration file provides a way to list databases and replicas that Litestream should manage. In addition, there are global variables that can be applied to all replicas.

If the -config flag is not specified, Litestream will attempt to read litestream.yml from the current directory and then fallback to using /etc/litestream.yml.

Variable expansion

By default, Litestream will perform environment variable expansion within the config file before reading it. Any references to $VAR or ${VAR} formatted variables will be replaced with their environment variable values. If no value is set then it will be replaced with an empty string.

This can cause issues if you have a value in a configuration file which has a dollar sign followed by characters—for example, a password. In this case, you can set the -no-expand-env flag on any litestream command to disable expansion.

Global settings

AWS credentials

If you are using AWS S3 replication, it can be useful to specify your credentials in one place instead of for each replica:

access-key-id:     AKIAxxxxxxxxxxxxxxxx
secret-access-key: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/xxxxxxxxx

Metrics

Litestream produces a continuous stream of metrics that are exported as a Prometheus endpoint. These are disabled by default as it requires an HTTP server to start. You can enable it by setting a bind address in the config:

addr: ":9090"

When you start Litestream with this setting enabled, you’ll see metrics at http://localhost:9090/metrics

Database settings

Litestream can monitor one or more database files that are specified in the configuration file. Database files are also referenced in litestream commands by their absolute path.

The path field is required for the database and typically you’ll have one or more replicas listed under the replicas field.

dbs:
  - path: /var/lib/db1
    replicas:
      - url: s3://mybkt.litestream.io/db1

  - path: /var/lib/db2
    replicas:
      - path: /backup/db2
      - url:  s3://mybkt.litestream.io/db2

The database also supports the following settings:

upstream—specifies the primary Litestream instance when using live read replication. See the following section for details.
monitor-delay-interval—specifies the amount of time to wait after a database change before writing to the shadow WAL. This setting allows batching of multiple transactions into a single WAL segment file. Defaults to 10ms.
checkpoint-interval—specifies the interval at which to checkpoint the WAL and start a new WAL index. This determines the granularity at which you can perform point-in-time recovery. Defaults to 1m.
min-checkpoint-page-count—the minimum number of WAL pages before Litestream will perform a PASSIVE checkpoint. Defaults to 1000.
max-checkpoint-page-count—the maximum number of WAL pages before Litestream will perform a RESTART checkpoint. Defaults to 10000.
shadow-retention-count—the number of WAL indexes to retain in the shadow WAL. This is used to retain WAL files on the primary so that read replicas have time to read them. Defaults to 32.

Upstream settings

The upstream field for the database enables read replication from another Litestream instance that acts as the primary. The primary must have the HTTP server enabled by setting the addr config property.

Typically, you only need to specify the url field with the scheme, host, and port of the primary Litestream instance:

dbs:
  - path: /var/lib/db1
    upstream:
      - url: http://${PRIMARY_HOSTNAME}:9090

An upstream.path field is also available if the database path of the primary does not match the database path of the replica.

Replica settings

Litestream currently supports two types of replicas:

"abs" replicates a database to an Azure Blob Storage container.
"file" replicates a database to another local file path.
"s3" replicates a database to an S3-compatible bucket.

All replicas have unique name which is specified by the "name" field. If a name is not specified then the name defaults to the replica type. The name is only needed when using multiple replicas of the same type on a database.

The following replica settings are also available for all replica types:

url—Short-hand form of specifying a replica location. Setting this field will apply changes to multiples fields including bucket, path, region, etc.
retention—The amount of time that snapshot & WAL files will be kept. After the retention period, a new snapshot will be created and the old one will be removed. WAL files that exist before the oldest snapshot will also be removed. Defaults to 24h.
retention-check-interval—Specifies how often Litestream will check if retention needs to be enforced. Defaults to 1h.
snapshot-interval—Specifies how often new snapshots will be created. This is used to reduce the time to restore since newer snapshots will have fewer WAL frames to apply. Retention still applies to these snapshots.

If you do not set a snapshot interval then a new snapshot will be created whenever retention is performed. Retention occurs every 24 hours by default.
sync-interval—Frequency in which frames are pushed to the replica. Defaults to 1s. Increasing frequency can increase cloud storage costs significantly.

S3 replica

The easiest way to configure an S3 replica is to use the url field:

dbs:
  - path: /var/lib/db
    replicas:
      - url: s3://mybkt.litestream.io/db

However, you can break this out into separate fields as well:

dbs:
  - path: /var/lib/db
    replicas:
      - type:   s3
        bucket: mybkt.litestream.io
        path:   db

In addition, you can specify the region and AWS credentials per-replica:

dbs:
  - path: /var/lib/db
    replicas:
      - url: s3://mybkt.litestream.io/db
        region: us-east1
        access-key-id: AKIAxxxxxxxxxxxxxxxx
        secret-access-key: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/xxxxxxxxx

The following settings are specific to S3 replicas:

access-key-id—Replica-specific authentication key. If not specified, the global key or the LITESTREAM_ACCESS_KEY_ID environment variable will be used instead. The AWS_ACCESS_KEY_ID variable can also be used.
secret-access-key—Replica-specific secret key. If not specified, the global secret or the LITESTREAM_SECRET_ACCESS_KEY environment variable will be used instead. The AWS_SECRET_ACCESS_KEY variable can also be used.
bucket—Specifies the name of the remote bucket to replicate to.
path—Specifies the path to use within the bucket.
region—Specifies the bucket’s region. Only used for AWS S3 & Backblaze B2.
endpoint—Specifies the endpoint URL of the S3-compatible service. Only required for non-AWS services.
force-path-style—Uses the path style which is required by non-AWS services. This is automatically enabled if endpoint is set.
skip-verify—Disables TLS verification. This is useful when testing against a local node such as MinIO and you are using self-signed certificates.

File replica

File replicas can be configured using the "path" field:

dbs:
  - path: /var/lib/db
    replicas:
      - path: /backup/db

If no "type" field is specified and a "url" is not used then "file" is assumed.

Multiple replicas

You can specify multiple replicas for a database, however, each one must have a unique name. For example, this configuration below is for replicating to AWS S3 & to DigitalOcean Spaces. Both of these use the “s3” replica type so their default name would be “s3”. Instead, we can name them each unique names, my_aws_replica & my_do_replica.

dbs:
  - path: /local/path/to/db
    replicas:
      - name: my_aws_replica
        url: s3://myawsbucket/db
      - name: my_do_replica
        url: s3://mybkt.nyc3.digitaloceanspaces.com/db

Retention period

Replicas maintain a snapshot of the database as well as a contiguous sequence of SQLite WAL page updates. These updates take up space so new snapshots are created and old WAL files are dropped through a process called “retention”.

The default retention period is 24h. You can change that with the retention field. Retention is enforced periodically and defaults to every 1h. This can be changed with the retention-check-interval field.

dbs:
  - path: /var/lib/db
    replicas:
      - url: s3://mybkt.litestream.io/db
        retention: 4h

Duration values can be specified using second (s), minute (m), or hour (h) but days, weeks, & years are not supported.

Configuration File

Variable expansion#

Global settings#

AWS credentials#

Metrics#

Database settings#

Upstream settings#

Replica settings#

S3 replica#

File replica#

Multiple replicas#

Retention period#