Poddery - Diaspora, Matrix and XMPP

We run a diaspora service at poddery.com

Environment

We use diaspora-installer-mysql package from https://people.debian.org/~praveen/diaspora-jessie/

See /usr/share/doc/diaspora-common/README for package specific configuration.

System health check:

• There should be a data disk attached (added from cloud.scaleway.com)
• The attached disk (/dev/nbd2) should be an lvm physical volume (pvcreate /dev/nbd2). We cannot use it directly for encryption, so we use lvm.
• /dev/data is an lvm volume group created from /dev/nbd2 (vgcreate data /dev/nbd2).
• /dev/data/diaspora is an lvm logical volume (lvcreate -n diaspora /dev/data -L 93.5G).
• /dev/mapper/diaspora is an encrypted device (cryptsetup luksFormat /dev/data/diaspora; cryptsetup luksOpen /dev/data/diaspora diaspora)
• /dev/mapper/diaspora is an ext4 file system (mkfs.ext4 /dev/mapper/diaspora)
• /var/lib/diaspora should be mounted (mount /dev/mapper/diaspora /var/lib/diaspora)
• all critical data should be on /var/lib/diaspora.
  • /etc/init.d/mysql stop; mv /var/lib/mysql /var/lib/diaspora; ln /var/lib/diaspora/mysql /var/lib/diaspora
  • mkdir /var/lib/diaspora/uploads; chown -R diaspora: /var/lib/diaspora/uploads;ln -s /var/lib/diaspora/uploads /usr/share/diaspora/public/uploads

Homepage

Homepage and other static pages are maintained in our gitlab instance. You can change it directly in the master branch or send pull requests. You can edit it via web as well.

• poddery.com -> https://git.fosscommunity.in/community/poddery.com.

• save.poddery.com -> https://git.fosscommunity.in/community/save.poddery.com

save.poddery.com repo is maintained as a sub module in poddery.com repo. See this tutorial -> https://chrisjean.com/git-submodules-adding-using-removing-and-updating/ for working with git submodules.

SSL/TLS certificates

# letsencrypt certonly -d poddery.com -d www.poddery.com -d test.poddery.com
# cp  -L /etc/letsencrypt/live/test.poddery.com/fullchain.pem /etc/diaspora/ssl/poddery.com-bundle.pem
# cp -L /etc/letsencrypt/live/test.poddery.com/privkey.pem /etc/diaspora/ssl/poddery.com.key

# ls -l /etc/prosody/certs/
total 0
lrwxrwxrwx 1 root root 40 Mar 28 01:16 poddery.com.crt -> /etc/diaspora/ssl/poddery.com-bundle.pem
lrwxrwxrwx 1 root root 33 Mar 28 01:16 poddery.com.key -> /etc/diaspora/ssl/poddery.com.key

Statistics

• live statistics

Coordination

• loomio group - we use this for decision making.
• We also have a LibreSignal group for quick collaboration.
• Some of us also hangout at XMPP room of fosscommunity.in, #fci@conference.diasp.in

Contact

Email: poddery at autistici.org

The following people have their gpg keys in the password file.

Praveen Arimbrathodiyil (piratepin) (ID: 0xCE1F9C674512), Balasankar C (ID: 0x96EDAB9B2E6B7171), Manu Krishnan T V (ID: 0x5D0064186AF037D9), Fayad Fami (fayad) (ID: 0x51C954405D432381), Abhijith PA (ID: 0x863D4DF2ED9C28EF), Syam G Krishnan (sgk) (ID: 0x6EF48CCD865A1FFC)

Server Access

Maintained in a private git repo at -> https://git.fosscommunity.in/community/access

Setting up Backup

Backup is setup on a Scaleway C1 VPS (4 core ARM processor with 2GB RAM)

Hostname (IP): backup.poddery.com (212.47.244.122)

# apt-get install lvm2 cryptsetup

Directly creating luks volume on /dev/nbd1 is not working, so we use a logical volume

# pvcreate /dev/nbd1
# vgcreate data /dev/nbd1
# lvcreate -n diaspora -L 46.5G /dev/data

# cryptsetup luksFormat /dev/data/diaspora
# cryptsetup luksOpen /dev/data/diaspora diaspora

and update /etc/crypttab

# <target name> <source device>         <key file>      <options>
diaspora /dev/data/diaspora none luks

# mkfs.ext4 /dev/mapper/diaspora
# mkdir /var/lib/diaspora

and update /etc/fstab

# UNCONFIGURED FSTAB FOR BASE SYSTEM
/dev/mapper/diaspora /var/lib/diaspora ext4 defaults 0 2

# mount -a
# apt-get install mysql-server

Move mysql data directory to encrypted volume

# /etc/init.d/mysql stop
# mv /var/lib/mysql /var/lib/diaspora/
# ln -s /var/lib/diaspora/mysql /var/lib/mysql

Follow steps in https://dev.mysql.com/doc/refman/5.5/en/replication-howto-masterbaseconfig.html for replication

Follow steps in https://www.howtoforge.com/how-to-set-up-mysql-database-replication-with-ssl-encryption-on-centos-5.4 for ssl (but ssl support is disabled in debian)

Follow steps in http://www.networkcomputing.com/storage/how-set-ssh-encrypted-mysql-replication/1111882674 to use ssh port forwarding to have encrypted replication

# adduser sshtunnel --disabled-login
# su sshtunnel

Generate ssh key pair and copy public key to target system

$ ssh-keygen -t rsa
$ ssh -f sshtunnel@poddery.com -L 7777:127.0.0.1:3306 -N

Test the connectivity

# mysql -u poddery_backup -p -P 7777 -h 127.0.0.1

Uploads are rsynced every hour

# crontab -e
# m h  dom mon dow   command
0 * * * * rsync -av --delete diaspora@poddery.com:diaspora/public/uploads/ /var/lib/diaspora/uploads/ >/var/lib/diaspora/rsync-uploads.log

Maintenance history

This section holds maintenance/issue history for future tracking.

1. Prosody error - Failed to load private key

certmanager error SSL/TLS: Failed to load '/etc/letsencrypt/live/poddery.com/privkey.pem': Previous error (see logs), or other system error. (for poddery.com)
tls error   Unable to initialize TLS: error loading private key (system lib)
certmanager error SSL/TLS: Failed to load '/etc/letsencrypt/live/poddery.com/privkey.pem': Check that the permissions allow Prosody to read this file.

This error is usually when ssl certificate in freshly installed or renewed. Prosody user is unable to access the key file due to lack of privileges.

Note that Poddery uses Letsencrypt for ssl.

Fix:

• Make sure that prosody user is in 'certs' group (this group may also be called ssl-certs as setup by Letencrypt)
• /etc/letsencrypt/ is the ssl directory.
• Prosody user should have permissions to all folders importantly archive and live folders in /etc/letsencrypt. Permissions to each folder must be 750.
• Troubleshoot by checking if you can switch to each folder in /etc/letsencrypt as prosody user and cat the files.