Management PostgreSQL store

PostgreSQL is recommended for production deployments. It supports concurrent access, enabling multiple management instances for high availability.

Configuration

Combined setup (config.yaml)

To use PostgreSQL, update your config.yaml:

server:
  store:
    engine: "postgres"
    dsn: "host=<PG_HOST> user=<PG_USER> password=<PG_PASSWORD> dbname=<PG_DB_NAME> port=<PG_PORT>"

You can also pass the DSN as an NB_ environment variable on the netbird-server container in your docker-compose.yml:

environment:
  - NB_STORE_ENGINE_POSTGRES_DSN=host=<PG_HOST> user=<PG_USER> password=<PG_PASSWORD> dbname=<PG_DB_NAME> port=<PG_PORT>

Restart the server and confirm:

docker compose restart netbird-server
docker compose logs netbird-server

You should see:

INFO management/server/store.go:109: using Postgres store engine

Older multi-container setup (management.json)

To enable Postgres, add to your setup.env:

NETBIRD_STORE_CONFIG_ENGINE=postgres

This sets the following in your management.json:

"StoreConfig": {
    "Engine": "postgres"
}

Create a .env file with the connection string:

NETBIRD_STORE_ENGINE_POSTGRES_DSN="host=<PG_HOST> user=<PG_USER> password=<PG_PASSWORD> dbname=<PG_DB_NAME> port=<PG_PORT>"

Update docker-compose.yml to pass the config to the management container:

environment:
  - NETBIRD_STORE_ENGINE_POSTGRES_DSN=${NETBIRD_STORE_ENGINE_POSTGRES_DSN}
env_file:
  - .env

Migrating from SQLite store to Postgres store

This migration process allows users to seamlessly transition between storage options while maintaining data integrity.

  1. Backup your data store (store.db in datadir - default /var/lib/netbird/)

For the combined setup:

mkdir backup
docker compose cp -a netbird-server:/var/lib/netbird/. backup/

For the older multi-container setup:

mkdir backup
docker compose cp -a management:/var/lib/netbird/. backup/
  1. Import SQLite data to Postgres

For migrating the SQLite data we rely on the pgloader tool. You can install it by running sudo apt-get install pgloader on debian or brew install pgloader on MacOS.

pgloader --type sqlite backup/store.db "postgresql://<PG_USER>:<PG_PASSWORD>@<PG_HOST>:<PG_PORT>/<PG_DB_NAME>"
  1. Resolve any compatibility issues that may arise due to the migration process.

To address this, you can execute the following SQL commands in your PostgreSQL database:

DELETE FROM policy_rules WHERE policy_rules.policy_id NOT IN (SELECT id FROM policies);
DELETE FROM routes WHERE routes.account_id NOT IN (SELECT id FROM accounts);
DELETE FROM name_server_groups WHERE name_server_groups.account_id NOT IN (SELECT id FROM accounts);
DROP TABLE IF EXISTS rules;
  1. Enable Postgres in your configuration.

For the combined setup, update config.yaml:

server:
  store:
    engine: "postgres"
    dsn: "host=<PG_HOST> user=<PG_USER> password=<PG_PASSWORD> dbname=<PG_DB_NAME> port=<PG_PORT>"

For the older multi-container setup, update management.json:

"StoreConfig": {
    "Engine": "postgres"
}

And pass the DSN via environment variable as described in the older multi-container setup section above.

  1. Restart the server and confirm:

For the combined setup:

docker compose restart netbird-server
docker compose logs netbird-server

For the older multi-container setup:

docker compose restart management
docker compose logs management

You should see an entry similar to:

INFO management/server/store.go:109: using Postgres store engine

Rollback to SQLite store

To rollback to the SQLite store, follow these steps:

  1. Restore store.db backup

For the combined setup:

docker compose cp backup/. netbird-server:/var/lib/netbird/

For the older multi-container setup:

docker compose cp backup/. management:/var/lib/netbird/
  1. Switch the store engine back to SQLite.

For the combined setup, update config.yaml:

server:
  store:
    engine: "sqlite"

For the older multi-container setup, update management.json:

"StoreConfig": {
    "Engine": "sqlite"
}
  1. Restart the server and confirm:

For the combined setup:

docker compose restart netbird-server
docker compose logs netbird-server

For the older multi-container setup:

docker compose restart management
docker compose logs management

You should see an entry similar to:

INFO management/server/store.go:109: using SQLite file store engine

Optional Rollback Postgres data to SQLite

This is optional and should be used only if you want to rollback the data from Postgres to SQLite while running the same NetBird version. For migrating the Postgres data, we rely on the pg_dump, sed, and sqlite3 tools. Make sure these are installed before proceeding

  1. Export Postgres data
pg_dump --data-only --column-inserts "postgresql://<PG_USER>:<PG_PASSWORD>@<PG_HOST>:<PG_PORT>/<PG_DB_NAME>" > data.sql
  1. Convert exported Postgres data sql to SQLite format
sed \
-e 's/\\\\:/\:/g' \
-e 's/\\\\//g' \
-e 's/\\\\;/;/g' \
-e '/^SET /d' \
-e '/setval/d' \
-e "s/'true'/1/g" \
-e "s/'false'/0/g" \
-e 's/public\.//' \
-e '/^[[:space:]]*SELECT/d' data.sql > data.sql
  1. Generate database schema from SQLite backup
sqlite3 backup/store.db '.schema' > schema.sql
  1. Create SQLite database with Postgres exported data
sqlite3 store.db '.read schema.sql' && sqlite3 store.db '.read data.sql'
  1. Copy db to the container

For the combined setup:

docker compose cp store.db netbird-server:/var/lib/netbird/store.db

For the older multi-container setup:

docker compose cp store.db management:/var/lib/netbird/store.db
  1. Switch the store engine back to SQLite using the instructions in the Rollback to SQLite store section above.