Note: Currently new registrations are closed, if you want an account Contact us
Mailing list
Mailing lists are very useful communication and coordination tool. FSCI provides mailing lists to Free Libre Open Source (FLOSS) groups and non profit organizations. If you want to open a new public list, please drop a mail at postmaster(at)lists.fsci.org.in.
Environment
Machine Summary
We are on scaleway virtual cloud server.
| * Cores | : 2 x86 cores |
| * Memory | : 2GB |
| * Disk | : 50GB |
| * OS | : Debian GNU/Linux |
| * Web Server | : Nginx |
| * List manager | : Mailman3 |
| * Spam Filter | : Rspamd |
| * Host name | : lists.fsci.org.in |
Coordination
- Hangout with us in our Matrix room #mailman:poddery.com
- issue tracker - we use this to track progress of tasks
Admins
- Abhijith PA
- Abhinav CK
- Akhil Varkey
emeritus
- Prinz Piuz
- Balasankar C
Admin Documentation
Postfix Aliases
Apart from mailman's aliases. We have some system wide aliases for sending and receiving postfix related issues and warning. Those are kept in /etc/aliases .
Once done editing please run command `newaliases`
Generate DKIM for new domains
We use 'opendkim'[1] to implement DKIM.
To generate DKIM keys, use opendkim-genkey -b 4096 -h rsa-sha256 -r -s <selector_name> -d <domain_name_of_this_list> -v`
We don't have any format on selector naming. Pick something related to this list(for eg: the selector for the domain lists.fsci.org.in is lists what we used).
After generating the DKIM keys, we will get two files.
- <selector_name>.private - which contains the private signing key.
- <selector_name>.txt - which contains the DKIM TXT DNS record for that domain.
Now, create a folder under /etc/opendkim/keys/<domain_name_for_this_list>/ and move the above file to that location.
Create an entry for the newly created domain in /etc/opendkim/KeyTable as well as /etc/opendkim/SigningTable
On /etc/opendkim/KeyTable
<selector_name._domainkey.<domain_name_of_this_list> <domain_name_of_this_list>:<selector_name>:/etc/opendkim/keys/<domain_name_of_this_list>/<selector_name>.private eg: lists._domainkey.lists.fsci.org.in lists.fsci.org.in:lists:/etc/opendkim/lists.private
On /etc/opendkim/SigningTable
<domain_name_of_this_list> <selector_name>._domainkey.<domain_name_of_this_list> eg: lists.fsci.org.in lists._domainkey.lists.fsci.org.in
More on DKIM keys, formating etc: [DomainKeys Identified Mail], Setup DKIM with OpenDKIM on Debian: [How To Install and Configure DKIM with Postfix on Debian]
Verify DKIM and other settings for new domains
Create a text file, dkim-test.txt (replace To address with random address generated by https://dkimvalidator.com or https://www.mail-tester.com/)
From: "Test" <test-dkim@mm.gnu.org.in> To: <random-address>@dkimvalidator.com Subject: Testing DKIM and other settings for this domain Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit This email is to test email settings for a domain is correct
and use sendmail command to send it
/usr/sbin/sendmail -t < dkim-test.txt
Setup Rspamd
See [Rspamd quick start] for the latest installation and setup instructions. Install Rspamd. Redis will be installed automatically(check).
$ sudo apt install rspamd
Check service is running.
$ systemctl status rspamd.service
● rspamd.service - rapid spam filtering system
Loaded: loaded (/lib/systemd/system/rspamd.service; enabled; vendor preset: enabled)
Active: active (running) since Sat 2020-11-07 14:54:03 UTC; 1min 58s ago
Docs: https://rspamd.com/doc/
Main PID: 19339 (rspamd)
Tasks: 5 (limit: 2336)
Memory: 276.2M
CGroup: /system.slice/rspamd.service
├─19339 rspamd: main process
├─19517 rspamd: rspamd_proxy process (localhost:11332)
├─19518 rspamd: controller process (localhost:11334)
├─19519 rspamd: normal process (localhost:11333)
└─19520 rspamd: hs_helper process
Nov 07 14:54:03 lists.fsci.org.in systemd[1]: Started rapid spam filtering system.
Check if redis is working properly using redis-cli
$ redis-cli 127.0.0.1:6379> ping PONG 127.0.0.1:6379> ping Hello[qwe] "Hello[qwe]" 127.0.0.1:6379> exit
Change some redis config to use it as a LRU cache[2]. On /etc/redis/redis.conf ~line 850
maxmemory 500mb maxmemory-policy volatile-ttl bind 127.0.0.1 ::1 //already exists
Configure rspamd
$ sudo rspamadm configwizard
____ _
| _ \ ___ _ __ __ _ _ __ ___ __| |
| |_) |/ __|| '_ \ / _` || '_ ` _ \ / _` |
| _ < \__ \| |_) || (_| || | | | | || (_| |
|_| \_\|___/| .__/ \__,_||_| |_| |_| \__,_|
|_|
Welcome to the configuration tool
We use /etc/rspamd/rspamd.conf configuration file, writing results to /etc/rspamd
Modules enabled: hfilter, phishing, emails, asn, settings, chartable, arc, bayes_expiry, once_received, rbl, fuzzy_check, metadata_exporter, elastic, mid, multimap, spf, dkim_signing, dkim, mime_types, regexp, maillist, dmarc, forged_recipients, milter_headers, whitelist, force_actions, trie
Modules disabled (explicitly): p0f, spamtrap, rspamd_update, mx_check, dcc
Modules disabled (unconfigured): spamassassin, maps_stats, metric_exporter, dynamic_conf, clustering, reputation, antivirus, fuzzy_collect, external_services, ip_score, clickhouse
Modules disabled (no Redis): greylist, url_redirector, replies, neural, ratelimit, history_redis
Modules disabled (experimental):
Modules disabled (failed):
Do you wish to continue?[Y/n]:
Setup WebUI and controller worker:
Controller password is not set, do you want to set one?[Y/n]: Y
Enter passphrase:
Set encrypted password to: <password_hash>
Redis servers are not set:
The following modules will be enabled if you add Redis servers:
* greylist
* url_redirector
* replies
* neural
* ratelimit
* history_redis
Do you wish to set Redis servers?[Y/n]:
Input read only servers separated by `,` [default: localhost]:
Input write only servers separated by `,` [default: localhost]:
Do you have any password set for your Redis?[y/N]:
Do you have any specific database for your Redis?[y/N]:
Do you want to setup dkim signing feature?[y/N]: N
File: /etc/rspamd/local.d/redis.conf, changes list:
write_servers => localhost
read_servers => localhost
File: /etc/rspamd/local.d/worker-controller.inc, changes list: password => <password_hash>
Apply changes?[Y/n]: Y Create file /etc/rspamd/local.d/redis.conf Create file /etc/rspamd/local.d/worker-controller.inc 2 changes applied, the wizard is finished now *** Please reload the Rspamd configuration ***
Now, restart the rspamd service and check status
$ sudo systemctl restart rspamd
$ sudo systemctl status rspamd
● rspamd.service - rapid spam filtering system
Loaded: loaded (/lib/systemd/system/rspamd.service; enabled; vendor preset: enabled)
Active: active (running) since Sat 2020-11-07 15:28:14 UTC; 4s ago
Docs: https://rspamd.com/doc/
Main PID: 21899 (rspamd)
Tasks: 5 (limit: 2336)
Memory: 116.5M
CGroup: /system.slice/rspamd.service
├─21899 rspamd: main process
├─21915 rspamd: rspamd_proxy process (localhost:11332)
├─21916 rspamd: controller process (localhost:11334)
├─21917 rspamd: normal process (localhost:11333)
└─21918 rspamd: hs_helper process
Nov 07 15:28:14 lists.fsci.org.in systemd[1]: Started rapid spam filtering system.
Worker Proxy setup in /etc/rspamd/local.d/worker-proxy.inc
# local.d/worker-proxy.inc
milter = yes; # Enable milter mode
timeout = 120s; # Needed for Milter usually
upstream "local" {
default = yes; # Self-scan upstreams are always default
self_scan = yes; # Enable self-scan
}
count = 4; # Spawn more processes in self-scan mode
max_retries = 5; # How many times master is queried in case of failure
discard_on_reject = false; # Discard message instead of rejection
quarantine_on_reject = false; # Tell MTA to quarantine rejected messages
spam_header = "X-Spam"; # Use the specific spam header
reject_message = "Spam message rejected"; # Use custom rejection message
Disable DKIM Signing on rspamd
To disable DKIM signing (i.e. you use OpenDKIM, or signing is done elsewhere- ie, that's our case) # local.d/dkim_signing.conf enabled = false;
and restart rspamd service after that.
Setting up the WebUI
For using the location, <webroot>/rspamd,
Add the following on /etc/mailman3/nginx.conf
location /rspamd/ {
proxy_pass http://localhost:11334/;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
Now, check nginx syntax, restart the nginx service and check the status.
After restart, you can visit <webroot>/rspamd and login with the previously created password.
Setting up rspamd Worker proxy config on postfix
Rspamd Worker proxy config
On /etc/postfix/main.cf
Don't forget to disable existing configs for the same below
##
## Spam filter and DKIM signatures via Rspamd
##
smtpd_milters = inet:localhost:11332, local:/opendkim/opendkim.sock
non_smtpd_milters = inet:localhost:11332, local:/opendkim/opendkim.sock
milter_protocol = 6
milter_mail_macros = i {mail_addr} {client_addr} {client_name} {auth_authen}
milter_default_action = accept
and then restart postfix.
Finally, enable rspamd service if everything seems to work properly
$ sudo systemctl enable rspamd.
Remove spam confirmation requests
Too many pending subscription requests can slow down[3] list index. Updating postorius is how to fix this. But in case there's no other way, and removing those requests through frontend [4] is impossible, this can be done (GNU Mailman 3.2.2 (La Villa Strangiato)):
$ sudo mailman shell -l listname.lists.fsci.org.in
Welcome to the GNU Mailman shell
The variable 'm' is the listname@lists.fsci.org.in mailing list
>>> from zope.component import getUtility
>>> from mailman.interfaces.pending import IPendings
>>> pendings = getUtility(IPendings)
>>> plist = []
>>> for p in pendings.find(m):
... plist.append(p)
...
>>> len(plist)
2047
>>> # Let us save the spammer email and when they signed up for analysis
>>> import csv
>>> with open('spam.csv', 'w', newline=) as csvfile:
... spamwriter = csv.writer(csvfile, delimiter=',')
... for p in plist:
... spammer = p[1]
... spamwriter.writerow([spammer.get('email'), spammer.get('when'), spammer.get('display_name')])
...
>>> # Now on to actually removing spam
>>> from mailman.interfaces.workflow import IWorkflowStateManager
>>> workflow = getUtility(IWorkflowStateManager)
>>> for p in plist:
... token = p[0]
... pendings.confirm(token)
... workflow.discard(token)
...
>>> # Now, we have to commit this transaction to database
>>> from mailman.config import config
>>> config.db.commit()