2019-01-01 20:40:48 +08:00
# Importing an existing Postgres database from another installation (optional)
2020-12-14 07:48:36 +08:00
Run this if you'd like to import your database from a previous installation.
(don't forget to import your Synapse `media_store` files as well - see [the importing-synape-media-store guide ](importing-synapse-media-store.md )).
2019-01-01 20:40:48 +08:00
## Prerequisites
For this to work, **the database name in Postgres must match** what this playbook uses.
2021-01-22 21:50:24 +08:00
This playbook uses a Postgres database name of `synapse` by default (controlled by the `matrix_synapse_database_database` variable).
If your database name differs, be sure to change `matrix_synapse_database_database` to your desired name and to re-run the playbook before proceeding.
2019-01-01 20:40:48 +08:00
The playbook supports importing Postgres dump files in **text** (e.g. `pg_dump > dump.sql` ) or **gzipped** formats (e.g. `pg_dump | gzip -c > dump.sql.gz` ).
2021-12-05 16:41:00 +08:00
Importing multiple databases (as dumped by `pg_dumpall` ) is also supported.
But the migration might be a good moment, to "reset" a not properly working bridge. Be aware, that it might affect all users (new link to bridge, new roomes, ...)
2019-10-05 16:28:26 +08:00
2019-01-01 21:35:33 +08:00
Before doing the actual import, **you need to upload your Postgres dump file to the server** (any path is okay).
2019-01-01 20:40:48 +08:00
## Importing
To import, run this command (make sure to replace `<server-path-to-postgres-dump.sql>` with a file path on your server):
2021-01-22 23:39:08 +08:00
```sh
ansible-playbook -i inventory/hosts setup.yml \
2021-08-02 15:08:24 +08:00
--extra-vars='server_path_postgres_dump=< server-path-to-postgres-dump.sql > ' \
2021-01-22 23:39:08 +08:00
--tags=import-postgres
```
2019-01-01 20:40:48 +08:00
**Note**: `<server-path-to-postgres-dump.sql>` must be a file path to a Postgres dump file on the server (not on your local machine!).
2020-07-13 22:03:24 +08:00
2021-01-22 23:39:08 +08:00
2020-07-13 22:03:24 +08:00
## Troubleshooting
2021-12-05 16:41:00 +08:00
### Table Ownership
2020-07-13 22:03:24 +08:00
A table ownership issue can occur if you are importing from a Synapse installation which was both:
- migrated from SQLite to Postgres, and
- used a username other than 'synapse'
In this case you may run into the following error during the import task:
```
"ERROR: role \"synapse_user\" does not exist"
```
where `synapse_user` is the database username from the previous Synapse installation.
This can be verified by examining the dump for ALTER TABLE statements which set OWNER TO that username:
```Shell
2021-12-01 05:07:04 +08:00
$ grep "ALTER TABLE" homeserver.sql
2020-07-13 22:03:24 +08:00
ALTER TABLE public.access_tokens OWNER TO synapse_user;
ALTER TABLE public.account_data OWNER TO synapse_user;
ALTER TABLE public.account_data_max_stream_id OWNER TO synapse_user;
ALTER TABLE public.account_validity OWNER TO synapse_user;
ALTER TABLE public.application_services_state OWNER TO synapse_user;
...
```
It can be worked around by changing the username to `synapse` , for example by using `sed` :
2020-07-13 22:12:17 +08:00
```Shell
2021-12-01 05:07:04 +08:00
$ sed -i "s/OWNER TO synapse_user;/OWNER TO synapse;/g" homeserver.sql
2020-07-13 22:03:24 +08:00
```
2021-12-01 05:07:04 +08:00
This uses sed to perform an 'in-place' (`-i`) replacement globally (`/g`), searching for `synapse_user` and replacing with `synapse` (`s/synapse_user/synapse`). If your database username was different, change `synapse_user` to that username instead. Expand search/replace statement as shown in example above, in case of old user name like `matrix` - replacing `matrix` only would... well - you can imagine.
2020-07-13 22:03:24 +08:00
Note that if the previous import failed with an error it may have made changes which are incompatible with re-running the import task right away; if you do so it may fail with an error such as:
```
ERROR: relation \"access_tokens\" already exists
```
2021-12-05 16:41:00 +08:00
### Repeat import
2020-07-13 22:03:24 +08:00
In this case you can use the command suggested in the import task to clear the database before retrying the import:
```Shell
# systemctl stop matrix-postgres
# rm -rf /matrix/postgres/data/*
# systemctl start matrix-postgres
```
2021-12-05 16:41:00 +08:00
Now on your local machine run `ansible-playbook -i inventory/hosts setup.yml --tags=setup-postgres` to prepare the database roles etc.
If not, you probably get this error. `synapse` is the correct table owner, but the role is missing in database.
```
"ERROR: role synapse does not exist"
```
Once the database is clear and the ownership of the tables has been fixed in the SQL file, the import task should succeed.
Check, if `--dbname` is set to `synapse` (not `matrix` ) and replace paths (or even better, copy this line from your terminal)
```
/usr/bin/env docker run --rm --name matrix-postgres-import --log-driver=none --user=998:1001 --cap-drop=ALL --network=matrix --env-file=/matrix/postgres/env-postgres-psql --mount type=bind,src=/migration/synapse_dump.sql,dst=/synapse_dump.sql,ro --entrypoint=/bin/sh docker.io/postgres:14.1-alpine -c "cat /synapse_dump.sql | grep -vE '^(CREATE|ALTER) ROLE (matrix)(;| WITH)' | grep -vE '^CREATE DATABASE (matrix)\s' | psql -v ON_ERROR_STOP=1 -h matrix-postgres --dbname=synapse"
```
### Hints
To open psql terminal run `/usr/local/bin/matrix-postgres-cli`