Reporting

DB Replic does not replicate call data

After experiencing a ‘no space left on device’ and restarting containers, it can sometimes happen that the data from XiVO is not replicated anymore. The following error can be found only in docker logs of DB Replic:

xivo-dcomp logs -tf --tail=10 db_replic

Error:

db_replic_1       | liquibase.exception.LockException: Could not acquire change log lock.  Currently locked by fe80:0:0:0:42:acff:fe11:8%eth0 (fe80:0:0:0:42:acff:fe11:8%eth0) since 4/10/17 3:28 PM

This error is caused by a lock in liquibase db. Follow the below steps in order to fix the issue:

Warning

This problem should not happen, so you should know what you are doing and not follow this procedure blindly.

1. Stop db_replic (XiVO), xivo_stats (XiVO CC) and pack_reporting (XiVO CC)

2. On XiVOCC, enter inside the pgxivocc docker container:

   docker exec -it xivocc_pgxivocc_1 /bin/bash
   

3. Log on the database as postgres:

   su - postgres
   psql xivo_stats
   

4. Find if there still is an active lock:

   xivo_stats=# select * from databasechangeloglock;
    id | locked |       lockgranted       |                            lockedby
   ----+--------+-------------------------+-----------------------------------------------------------------
     1 | t      | 2017-04-10 15:28:10.684 | fe80:0:0:0:42:acff:fe11:8%eth0 (fe80:0:0:0:42:acff:fe11:8%eth0)
   

5. If so, delete the lock:

   xivo_stats=# truncate databasechangeloglock;
   xivo_stats=# select * from databasechangeloglock;
    id | locked | lockgranted | lockedby
   ----+--------+-------------+----------
     1 | f      |             |
   

6. Restart the containers:

   xivo-dcomp start db_replic (XiVO)
   xivocc-dcomp start pack_reporting (XiVO CC)
   xivocc-dcomp start xivo_stats (XiVO CC)
   

Now, if xivo_stats is stuck in exit (126), try a docker rm -v {xivo_stats contener id} followed by a xivocc-dcomp up -d.