51
edits
Note: Currently new registrations are closed, if you want an account Contact us
(add more dkim validator services) |
m (minor markup update) |
||
(8 intermediate revisions by 3 users not shown) | |||
Line 18: | Line 18: | ||
|- | |- | ||
|* List manager||: Mailman3 | |* List manager||: Mailman3 | ||
|- | |||
|* Spam Filter||: Rspamd | |||
|- | |- | ||
|* Host name||: [https://lists.fsci.org.in lists.fsci.org.in] | |* Host name||: [https://lists.fsci.org.in lists.fsci.org.in] | ||
|} | |} | ||
= Coordination = | = Coordination = | ||
Line 28: | Line 31: | ||
== Admins == | == Admins == | ||
Abhijith PA | # Abhijith PA | ||
# Abhinav CK | |||
# Akhil Varkey | |||
====== emeritus ====== | |||
# Prinz Piuz | |||
# Balasankar C | |||
= Admin Documentation = | |||
== Postfix Aliases == | |||
Apart from mailman's aliases. We have some system wide aliases for sending and receiving postfix related issues and warning. Those are kept in ''/etc/aliases'' . | |||
Once done editing please run command `''newaliases''` | |||
== Generate DKIM for new domains == | |||
We use 'opendkim'[http://opendkim.org/] to implement DKIM. | |||
To generate DKIM keys, use <code>opendkim-genkey -b 4096 -h rsa-sha256 -r -s <selector_name> -d <domain_name_of_this_list> -v`</code> | |||
<br /> | |||
We don't have any format on selector naming. Pick something related to this list(for eg: the selector for the domain lists.fsci.org.in is <code>lists</code> what we used). | |||
After generating the DKIM keys, we will get two files. | |||
* <selector_name>.private - which contains the private signing key. | |||
* <selector_name>.txt - which contains the DKIM TXT DNS record for that domain. | |||
<br /> | |||
Now, create a folder under <code>/etc/opendkim/keys/<domain_name_for_this_list>/</code> and move the above file to that location. | |||
<br /> | |||
Create an entry for the newly created domain in /etc/opendkim/KeyTable as well as /etc/opendkim/SigningTable | |||
<br /> | |||
On /etc/opendkim/KeyTable | |||
<selector_name._domainkey.<domain_name_of_this_list> <domain_name_of_this_list>:<selector_name>:/etc/opendkim/keys/<domain_name_of_this_list>/<selector_name>.private | |||
eg: lists._domainkey.lists.fsci.org.in lists.fsci.org.in:lists:/etc/opendkim/lists.private | |||
On /etc/opendkim/SigningTable | |||
<domain_name_of_this_list> <selector_name>._domainkey.<domain_name_of_this_list> | |||
eg: lists.fsci.org.in lists._domainkey.lists.fsci.org.in | |||
More on DKIM keys, formating etc: [[https://en.wikipedia.org/wiki/DomainKeys_Identified_Mail DomainKeys Identified Mail]], | |||
Setup DKIM with OpenDKIM on Debian: [[https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-dkim-with-postfix-on-debian-wheezy How To Install and Configure DKIM with Postfix on Debian]] | |||
== Verify DKIM and other settings for new domains == | == Verify DKIM and other settings for new domains == | ||
Line 47: | Line 85: | ||
/usr/sbin/sendmail -t < dkim-test.txt | /usr/sbin/sendmail -t < dkim-test.txt | ||
== Setup Rspamd == | |||
See [[https://rspamd.com/doc/quickstart.html Rspamd quick start]] for the latest installation and setup instructions. | |||
Install Rspamd. Redis will be installed automatically(check). | |||
$ sudo apt install rspamd | |||
Check service is running. | |||
$ systemctl status rspamd.service | |||
● rspamd.service - rapid spam filtering system | |||
Loaded: loaded (/lib/systemd/system/rspamd.service; enabled; vendor preset: enabled) | |||
Active: active (running) since Sat 2020-11-07 14:54:03 UTC; 1min 58s ago | |||
Docs: https://rspamd.com/doc/ | |||
Main PID: 19339 (rspamd) | |||
Tasks: 5 (limit: 2336) | |||
Memory: 276.2M | |||
CGroup: /system.slice/rspamd.service | |||
├─19339 rspamd: main process | |||
├─19517 rspamd: rspamd_proxy process (localhost:11332) | |||
├─19518 rspamd: controller process (localhost:11334) | |||
├─19519 rspamd: normal process (localhost:11333) | |||
└─19520 rspamd: hs_helper process | |||
Nov 07 14:54:03 lists.fsci.org.in systemd[1]: Started rapid spam filtering system. | |||
Check if redis is working properly using redis-cli | |||
$ redis-cli | |||
127.0.0.1:6379> ping | |||
PONG | |||
127.0.0.1:6379> ping Hello[qwe] | |||
"Hello[qwe]" | |||
127.0.0.1:6379> exit | |||
Change some redis config to use it as a LRU cache[https://redis.io/topics/lru-cache]. | |||
On /etc/redis/redis.conf | |||
~line 850 | |||
maxmemory 500mb | |||
maxmemory-policy volatile-ttl | |||
bind 127.0.0.1 ::1 //already exists | |||
'''Configure rspamd''' | |||
$ sudo rspamadm configwizard | |||
____ _ | |||
| _ \ ___ _ __ __ _ _ __ ___ __| | | |||
| |_) |/ __|| '_ \ / _` || '_ ` _ \ / _` | | |||
| _ < \__ \| |_) || (_| || | | | | || (_| | | |||
|_| \_\|___/| .__/ \__,_||_| |_| |_| \__,_| | |||
|_| | |||
Welcome to the configuration tool | |||
We use /etc/rspamd/rspamd.conf configuration file, writing results to /etc/rspamd | |||
Modules enabled: hfilter, phishing, emails, asn, settings, chartable, arc, bayes_expiry, once_received, rbl, fuzzy_check, metadata_exporter, elastic, mid, multimap, spf, dkim_signing, dkim, mime_types, regexp, maillist, dmarc, forged_recipients, milter_headers, whitelist, force_actions, trie | |||
Modules disabled (explicitly): p0f, spamtrap, rspamd_update, mx_check, dcc | |||
Modules disabled (unconfigured): spamassassin, maps_stats, metric_exporter, dynamic_conf, clustering, reputation, antivirus, fuzzy_collect, external_services, ip_score, clickhouse | |||
Modules disabled (no Redis): greylist, url_redirector, replies, neural, ratelimit, history_redis | |||
Modules disabled (experimental): | |||
Modules disabled (failed): | |||
Do you wish to continue?[Y/n]: | |||
Setup WebUI and controller worker: | |||
Controller password is not set, do you want to set one?[Y/n]: Y | |||
Enter passphrase: | |||
Set encrypted password to: <password_hash> | |||
Redis servers are not set: | |||
The following modules will be enabled if you add Redis servers: | |||
* greylist | |||
* url_redirector | |||
* replies | |||
* neural | |||
* ratelimit | |||
* history_redis | |||
Do you wish to set Redis servers?[Y/n]: | |||
Input read only servers separated by `,` [default: localhost]: | |||
Input write only servers separated by `,` [default: localhost]: | |||
Do you have any password set for your Redis?[y/N]: | |||
Do you have any specific database for your Redis?[y/N]: | |||
Do you want to setup dkim signing feature?[y/N]: N | |||
File: /etc/rspamd/local.d/redis.conf, changes list: | |||
write_servers => localhost | |||
read_servers => localhost | |||
File: /etc/rspamd/local.d/worker-controller.inc, changes list: | |||
password => <password_hash> | |||
Apply changes?[Y/n]: Y | |||
Create file /etc/rspamd/local.d/redis.conf | |||
Create file /etc/rspamd/local.d/worker-controller.inc | |||
2 changes applied, the wizard is finished now | |||
*** Please reload the Rspamd configuration *** | |||
Now, restart the rspamd service and check status | |||
$ sudo systemctl restart rspamd | |||
$ sudo systemctl status rspamd | |||
● rspamd.service - rapid spam filtering system | |||
Loaded: loaded (/lib/systemd/system/rspamd.service; enabled; vendor preset: enabled) | |||
Active: active (running) since Sat 2020-11-07 15:28:14 UTC; 4s ago | |||
Docs: https://rspamd.com/doc/ | |||
Main PID: 21899 (rspamd) | |||
Tasks: 5 (limit: 2336) | |||
Memory: 116.5M | |||
CGroup: /system.slice/rspamd.service | |||
├─21899 rspamd: main process | |||
├─21915 rspamd: rspamd_proxy process (localhost:11332) | |||
├─21916 rspamd: controller process (localhost:11334) | |||
├─21917 rspamd: normal process (localhost:11333) | |||
└─21918 rspamd: hs_helper process | |||
Nov 07 15:28:14 lists.fsci.org.in systemd[1]: Started rapid spam filtering system. | |||
'''Worker Proxy setup''' | |||
in /etc/rspamd/local.d/worker-proxy.inc | |||
# local.d/worker-proxy.inc | |||
milter = yes; # Enable milter mode | |||
timeout = 120s; # Needed for Milter usually | |||
upstream "local" { | |||
default = yes; # Self-scan upstreams are always default | |||
self_scan = yes; # Enable self-scan | |||
} | |||
count = 4; # Spawn more processes in self-scan mode | |||
max_retries = 5; # How many times master is queried in case of failure | |||
discard_on_reject = false; # Discard message instead of rejection | |||
quarantine_on_reject = false; # Tell MTA to quarantine rejected messages | |||
spam_header = "X-Spam"; # Use the specific spam header | |||
reject_message = "Spam message rejected"; # Use custom rejection message | |||
'''Disable DKIM Signing on rspamd''' | |||
To disable DKIM signing (i.e. you use OpenDKIM, or signing is done elsewhere- ie, that's our case) | |||
# local.d/dkim_signing.conf | |||
enabled = false; | |||
and restart rspamd service after that. | |||
'''Setting up the WebUI''' | |||
<br />For using the location, <webroot>/rspamd, | |||
<br />Add the following on /etc/mailman3/nginx.conf | |||
location /rspamd/ { | |||
proxy_pass http://localhost:11334/; | |||
proxy_set_header Host $host; | |||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; | |||
} | |||
Now, check nginx syntax, restart the nginx service and check the status. | |||
<br/> After restart, you can visit <webroot>/rspamd and login with the previously created password. | |||
<br /> | |||
'''Setting up rspamd Worker proxy config on postfix''' | |||
<br />Rspamd Worker proxy config | |||
<br />On /etc/postfix/main.cf | |||
Don't forget to disable existing configs for the same below | |||
## | |||
## Spam filter and DKIM signatures via Rspamd | |||
## | |||
smtpd_milters = inet:localhost:11332, local:/opendkim/opendkim.sock | |||
non_smtpd_milters = inet:localhost:11332, local:/opendkim/opendkim.sock | |||
milter_protocol = 6 | |||
milter_mail_macros = i {mail_addr} {client_addr} {client_name} {auth_authen} | |||
milter_default_action = accept | |||
and then restart postfix. | |||
Finally, enable rspamd service if everything seems to work properly | |||
<code>$ sudo systemctl enable rspamd</code>. | |||
== Remove spam confirmation requests == | |||
Too many pending subscription requests can slow down[https://gitlab.com/mailman/postorius/-/issues/417] list index. Updating postorius is how to fix this. But in case there's no other way, and removing those requests through frontend [https://gitlab.com/mailman/mailman/-/issues/779#note_423132958] is impossible, this can be done (GNU Mailman 3.2.2 (La Villa Strangiato)): | |||
$ sudo mailman shell -l listname.lists.fsci.org.in | |||
Welcome to the GNU Mailman shell | |||
The variable 'm' is the listname@lists.fsci.org.in mailing list | |||
>>> from zope.component import getUtility | |||
>>> from mailman.interfaces.pending import IPendings | |||
>>> pendings = getUtility(IPendings) | |||
>>> plist = [] | |||
>>> for p in pendings.find(m): | |||
... plist.append(p) | |||
... | |||
>>> len(plist) | |||
2047 | |||
>>> # Let us save the spammer email and when they signed up for analysis | |||
>>> import csv | |||
>>> with open('spam.csv', 'w', newline='') as csvfile: | |||
... spamwriter = csv.writer(csvfile, delimiter=',') | |||
... for p in plist: | |||
... spammer = p[1] | |||
... spamwriter.writerow([spammer.get('email'), spammer.get('when'), spammer.get('display_name')]) | |||
... | |||
>>> # Now on to actually removing spam | |||
>>> from mailman.interfaces.workflow import IWorkflowStateManager | |||
>>> workflow = getUtility(IWorkflowStateManager) | |||
>>> for p in plist: | |||
... token = p[0] | |||
... pendings.confirm(token) | |||
... workflow.discard(token) | |||
... | |||
>>> # Now, we have to commit this transaction to database | |||
>>> from mailman.config import config | |||
>>> config.db.commit() | |||
---- | ---- | ||
[[Category: Services]] | [[Category: Services]] |