We run decentralized and federated Diaspora social netowrk, XMPP and Matrix instant messaging services at poddery.com. Along with Diaspora, Poddery username and password can be used to access XMPP and Matrix services as well. chat.poddery.com provides Riot client (accessed by a web browser), which can be used to connect to any Matrix server without installing a Riot app/client.

Environment

Hosting

Poddery is hosted at Hetzner with the following specs:

Intel Xeon E3-1246V3 Process - 4 Cores, 3.5GHz
4TB HDD
32GB DDR3 RAM

Operating System

Debian Buster

User Visible Services

Diaspora

Currently installed version is 0.7.6.1 which is available in Debian Buster contrib
For live statistics see https://poddery.com/statistics

Chat/XMPP

Prosody is used as the XMPP server which is modern and lightweight.
Currently installed version is 0.11.2 which is available in Debian Buster.
All XEPs are enabled which the Conversations app support.

Chat/Matrix

Synapse is used as the Matrix server.
Synapse is currently installed directly from the official GitHub repo.

Homepage

Homepage and other static pages are maintained in FSCI GitLab instance.

poddery.com -> https://git.fosscommunity.in/community/poddery.com
save.poddery.com -> https://git.fosscommunity.in/community/save.poddery.com
fund.poddery.com -> https://git.fosscommunity.in/community/fund-poddery

Backend Services

Web Server / Reverse Proxy

Nginx web server which also acts as front-end (reverse proxy) for Diaspora and Matrix.

Database

PostgreSQL for Matrix
MySQL for Diaspora

TODO: Consider migrating to PostgreSQL to optimize resources (We can reduce one service and RAM usage).

Email

Exim

SSL/TLS certificates

Let's Encrypt

Firewall

UFW (Uncomplicated Firewall)

Intrusion Prevention

Fail2ban

Coordination

Loomio group - Mainly used for decision making
Matrix room - #poddery:poddery.com
Issue tracker - Used for tracking progress of tasks

Contact

Email: poddery at autistici.org (alias that reaches Akhilan, Abhijith Balan, Fayad, Balasankar, Julius, Praveen, Prasobh, Sruthi, Shirish, Vamsee and Manukrishnan)
The following people have their GPG keys in the access file:
- ID: 0xCE1F9C674512C22A - Praveen Arimbrathodiyil (piratepin)
- ID: 0xB77D2E2E23735427 - Balasankar C
- ID: 0x5D0064186AF037D9 - Manu Krishnan T V
- ID: 0x51C954405D432381 - Fayad Fami (fayad)
- ID: 0x863D4DF2ED9C28EF - Abhijith PA
- ID: 0x6EF48CCD865A1FFC - Syam G Krishnan (sgk)
- ID: 0xFD49D0BC6FEAECDA - Sagar Ippalpalli
- ID: 0x92FDAB42A95FF20C - Pirate Bady (piratesin)
- ID: 0x0B1955F40C691CCE - Kannan
- ID: 0x32FF6C6F5B7AE248 - Akhil Varkey
- ID: 0xFBB7061C27CB70C1 - Ranjith Siji
- ID: 0xEAAFE4A8F39DE34F - Kiran S Kunjumon (hacksk)
It's recommended to setup Vim GnuPG Plugin for transparent editing. Those who are new to GPG can follow this guide.

Server Access

Maintained in a private git repo at https://git.fosscommunity.in/community/access

Configuration and Maintenance

Disk Partitioning

RAID 1 setup on 2x2TB HDDs (sda and sdb).

mdadm --verbose --create /dev/mdX --level=mirror --raid-devices=2 /dev/sdaY /dev/sdbY

Separate partitions for swap (md0 - 16GB), boot (md1 - 512MB) and root (md2 - 50GB).
LVM on Luks for separate encrypted data partitions for database, static files and logs.

# Setup LUKS (make sure lvm2, udev and cryptsetup packages are installed).
cryptsetup luksFormat /dev/mdX
# Give disk encryption password as specified in the access repo
cryptsetup luksOpen /dev/mdX poddery

# LVM Setup
# Create physical volume named poddery
pvcreate /dev/mapper/poddery
# Create volume group named data
vgcreate data /dev/mapper/poddery
# Create logical volumes named log, db and static
lvcreate -n log /dev/data -L 50G
lvcreate -n db /dev/data -L 500G
# Assign remaining free space for static files
lvcreate -n static /dev/data -l 100%FREE 

# Create directories for mounting the encrypted partitions
mkdir /var/lib/db /var/lib/static /var/log/poddery

# Manually mount encrypted partitions. This is needed on each reboot as Hetzner doesn't provide a web console so that we can't decrypt the partitions during booting.
mount /dev/data/db /var/lib/db
mount /dev/data/static /var/lib/static
mount /dev/data/log /var/log/poddery

Hardening checklist

SSH password based login disabled (allow only key based logins)
SSH login disabled for root user (use a normal user with sudo)

# Check for the following settings in /etc/ssh/sshd_config:
...
PermitRootLogin no
...
PasswordAuthentication no
...

ufw firewall enabled with only the ports that needs to be opened (ufw tutorial):

ufw default deny incoming
ufw default allow outgoing
ufw allow ssh
ufw enable

# Verify everything is setup properly
ufw status

fail2ban configured against brute force attacks:

# Check for the following line /etc/ssh/sshd_config
...
LogLevel VERBOSE
...

# Restart SSH and enable fail2ban
systemctl restart ssh
systemctl enable fail2ban
systemctl start fail2ban

# To unban an IP, first check /var/log/fail2ban.log to get the banned IP and then run the following
# Here sshd is the defaut jail name, change it if you are using a different jail
fail2ban-client set sshd unbanip <banned_ip>

Diaspora

Install diaspora-installer from Debian Buster contrib:

apt install diaspora-installer

Move MySQL data to encrypted partition:

# Make sure /dev/data/db is mounted to /var/lib/db
systemctl stop mysql
systemctl disable mysql
mv /var/lib/mysql /var/lib/db/
ln -s /var/lib/db/mysql /var/lib/
systemctl start mysql

Move static files to encrypted partition:

# Make sure /dev/data/static is mounted to /var/lib/static
mkdir /var/lib/static/diaspora
mv /usr/share/diaspora/public/uploads /var/lib/static/diaspora
ln -s /var/lib/static/diaspora/uploads /usr/share/diaspora/public/
chown -R diaspora: /var/lib/static/diaspora

Modify configuration files at /etc/diaspora and /etc/diaspora.conf as needed (backup of the current configuration files are available in the access repo).
Homepage configuration:

# Make sure git and acl packages are installed
# Grant rwx permissions for the ssh user to /usr/share/diaspora/public
setfacl -m "u:<ssh_user>:rwx" /usr/share/diaspora/public

# Clone poddery.com repo
cd /usr/share/diaspora/public
git clone https://git.fosscommunity.in/community/poddery.com.git
cd poddery.com && mv * .[^.]* .. #Give yes for all files when prompted
cd .. && rmdir poddery.com

Save Poddery repo is maintained as a sub module in poddery.com repo. See this tutorial for working with git submodules.

# Clone save.poddery.com repo
cd /usr/share/diaspora/public/save
git submodule init
git submodule update

Matrix

See the official installation guide of Synapse for installing from source.
Nginx is used as reverse proxy to send requests that has /_matrix/* in URL to Synapse on port 8008. This is configured in /etc/nginx/sites-enabled/diaspora.
Shamil's Synapse Diaspora Auth script is used to authenticate Synapse with Diaspora database.
Move PostgreSQL data to encrypted partition:

# Make sure /dev/data/db is mounted to /var/lib/db
systemctl stop postgresql
systemctl disable postgresql
mv /var/lib/postgres /var/lib/db/
ln -s /var/lib/db/postgres /var/lib/
systemctl start postgresql

Move static files to encrypted partition:

# Make sure /dev/data/static is mounted to /var/lib/static
mkdir /var/lib/static/synapse
mv /var/lib/matrix-synapse/uploads /var/lib/static/synapse/
ln -s /var/lib/static/synapse/uploads /var/lib/matrix-synapse/
mv /var/lib/matrix-synapse/media /var/lib/static/synapse/
ln -s /var/lib/static/synapse/media /var/lib/matrix-synapse/
chown -R matrix-synapse: /var/lib/static/synapse

Install identity server mxisd (deb package available here)

Workers

For scalability, Poddery running workers. Currently all workers specified in that page, expect synapse.app.appservice is running on poddery.com
A new service matrix-synapse@.service is installed for the workers (Save the synape_worker file somewhere like /usr/local/bin/ or something).
The worker config can be found at /etc/matrix-synapse/workers
Synapse needs to be put under a reverse proxy see /etc/nginx/sites-enabled/matrix. A lot of /_matrix/ urls needs to be overridden too see /etc/nginx/sites-enabled/diaspora
These lines must be added to homeserver.yaml as we are running media_repository, federation_sender, pusher, user_dir workers respectively:

 enable_media_repo: False
 send_federation: False
 start_pushers: False
 update_user_directory: false

These services must be enabled, and added to Requires and Before sections of the original matrix-synapse.service:

matrix-synapse@synchrotron.service matrix-synapse@federation_reader.service matrix-synapse@event_creator.service matrix-synapse@federation_sender.service matrix-synapse@pusher.service matrix-synapse@user_dir.service matrix-synapse@media_repository.service matrix-synapse@frontend_proxy.service matrix-synapse@client_reader.service

Synapse Updation

First check synapse/UPGRADE.rst to see if anything extra needs to be done. Then, just run /root/upgrade-synapse

Riot-web Updation

Just run /var/www/get-riot and reload nginx

Chat/XMPP

Steps for setting up Prosody is given at https://wiki.debian.org/Diaspora/XMPP

# Follow steps 1 to 6 from https://wiki.debian.org/Diaspora/XMPP and then run the following:
mysql -u root -p # Enter password from the access repo

CREATE USER 'prosody'@'localhost' IDENTIFIED BY '<passwd_in_repo>';
GRANT ALL PRIVILEGES ON diaspora_production.* TO 'prosody'@'localhost';
FLUSH PRIVILEGES;

systemctl restart prosody

Install plugins

# Make sure mercurial is installed
cd /etc && hg clone https://hg.prosody.im/prosody-modules/ prosody-modules

Set Nginx Conf for BOSH URLS

Add the following in nginx configuration file to enable the BOSH URL to make JSXC Working:

upstream chat_cluster {
  server localhost:5280;
}

location /http-bind {
  proxy_set_header X-Real-IP $remote_addr;
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  proxy_set_header Host $http_host;
  proxy_set_header X-Forwarded-Proto https;
  proxy_redirect off;
  proxy_connect_timeout 5;
  proxy_buffering       off;
  proxy_read_timeout    70;
  keepalive_timeout     70;
  send_timeout          70;
  client_max_body_size 4M;
  client_body_buffer_size 128K;
  proxy_pass http://chat_cluster;
}

See here for more details on nginx configuration. Alternatively, apache settings can be found here.

TLS

Install letsencrypt.
Ensure proper permissions are set for /etc/letsencrypt and its contents.

chown -R root:ssl-cert /etc/letsencrypt
chmod g+r -R /etc/letsencrypt
chmod g+x /etc/letsencrypt/{archive,live}

Generate certificates. For more details see https://certbot.eff.org.
Make sure the certificates used by diaspora are symbolic links to letsencrypt default location:

ls -l /etc/diaspora/ssl
total 0
lrwxrwxrwx 1 root root 47 Apr  2 22:47 poddery.com-bundle.pem -> /etc/letsencrypt/live/poddery.com/fullchain.pem
lrwxrwxrwx 1 root root 45 Apr  2 22:48 poddery.com.key -> /etc/letsencrypt/live/poddery.com/privkey.pem

# If you don't get the above output, then run the following:
cp -L /etc/letsencrypt/live/poddery.com/fullchain.pem /etc/diaspora/ssl/poddery.com-bundle.pem
cp -L /etc/letsencrypt/live/poddery.com/privkey.pem /etc/diaspora/ssl/poddery.com.key

Make sure the certificates used by prosody are symbolic links to letsencrypt default location:

ls -l /etc/prosody/certs/
total 0
lrwxrwxrwx 1 root root 40 Mar 28 01:16 poddery.com.crt -> /etc/letsencrypt/live/poddery.com/fullchain.pem
lrwxrwxrwx 1 root root 33 Mar 28 01:16 poddery.com.key -> /etc/letsencrypt/live/poddery.com/privkey.pem

# If you don't get the above output, then run the following:
cp -L /etc/letsencrypt/live/poddery.com/fullchain.pem /etc/prosody/certs/poddery.com.crt
cp -L /etc/letsencrypt/live/poddery.com/privkey.pem /etc/prosody/certs/poddery.com.key

Cron jobs:

crontab -e
30 2 * * 1 letsencrypt renew  >> /var/log/le-renew.log
32 2 * * 1 /etc/init.d/nginx reload
34 2 * * 1 /etc/init.d/prosody reload

Manually updating TLS certificate:

letsencrypt certonly --webroot -w /usr/share/diaspora/public  -d poddery.com -d www.poddery.com -d test.poddery.com -d groups.poddery.com -d fund.poddery.com -w /usr/share/diaspora/public/save -d save.poddery.com -w /var/www/riot -d chat.poddery.com

History

See here for the archive of Poddery wiki page before the migration to Hetzner.

Navigation menu

Poddery - Diaspora, Matrix and XMPP

Contents

Environment

Hosting

Operating System

User Visible Services

Diaspora

Chat/XMPP

Chat/Matrix

Homepage

Backend Services

Web Server / Reverse Proxy

Database

Email

SSL/TLS certificates

Firewall

Intrusion Prevention

Coordination

Contact

Server Access

Configuration and Maintenance

Disk Partitioning

Hardening checklist

Diaspora

Matrix

Workers

Synapse Updation

Riot-web Updation

Chat/XMPP

Set Nginx Conf for BOSH URLS

TLS

History