FSCI Wiki

Difference between revisions of "Mailing list"

Mailing list (view source)

Revision as of 21:54, 12 April 2024

10,183 bytes added ,  12 April
m
minor markup update
(add more dkim validator services)
m (minor markup update)
 
(8 intermediate revisions by 3 users not shown)
Line 18: Line 18:
|-
|-
|* List manager||: Mailman3
|* List manager||: Mailman3
|-
|* Spam Filter||: Rspamd
|-
|-
|* Host name||: [https://lists.fsci.org.in lists.fsci.org.in]
|* Host name||: [https://lists.fsci.org.in lists.fsci.org.in]
|}
|}
= Coordination =
= Coordination =


Line 28: Line 31:
== Admins ==
== Admins ==


Abhijith PA, Prinz Piuz, Balasankar C  
# Abhijith PA  
# Abhinav CK
# Akhil Varkey
 
====== emeritus ======
 
# Prinz Piuz
# Balasankar C  


== Admin Documentation ==
= Admin Documentation =
 
== Postfix Aliases ==
Apart from mailman's aliases. We have some system wide aliases for sending and receiving postfix related issues and warning. Those are kept in ''/etc/aliases'' .
 
Once done editing please run command `''newaliases''`
 
== Generate DKIM for new domains ==
 
We use 'opendkim'[http://opendkim.org/] to implement DKIM.
To generate DKIM keys, use <code>opendkim-genkey -b 4096 -h rsa-sha256 -r -s <selector_name> -d <domain_name_of_this_list> -v`</code>
<br />
We don't have any format on selector naming. Pick something related to this list(for eg: the selector for the domain lists.fsci.org.in  is <code>lists</code> what we used).
After generating the DKIM keys, we will get two files.
* <selector_name>.private - which contains the private signing key.
* <selector_name>.txt - which contains the DKIM TXT DNS record for that domain.
<br />
Now, create a folder under <code>/etc/opendkim/keys/<domain_name_for_this_list>/</code> and move the above file to that location.
<br />
Create an entry for the newly created domain in /etc/opendkim/KeyTable as well as /etc/opendkim/SigningTable
<br />
On /etc/opendkim/KeyTable
<selector_name._domainkey.<domain_name_of_this_list> <domain_name_of_this_list>:<selector_name>:/etc/opendkim/keys/<domain_name_of_this_list>/<selector_name>.private
eg: lists._domainkey.lists.fsci.org.in lists.fsci.org.in:lists:/etc/opendkim/lists.private
On /etc/opendkim/SigningTable
<domain_name_of_this_list> <selector_name>._domainkey.<domain_name_of_this_list>
eg: lists.fsci.org.in lists._domainkey.lists.fsci.org.in
More on DKIM keys, formating etc: [[https://en.wikipedia.org/wiki/DomainKeys_Identified_Mail DomainKeys Identified Mail]],
Setup DKIM with OpenDKIM on Debian: [[https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-dkim-with-postfix-on-debian-wheezy How To Install and Configure DKIM with Postfix on Debian]]


== Verify DKIM and other settings for new domains ==
== Verify DKIM and other settings for new domains ==
Line 47: Line 85:


/usr/sbin/sendmail -t < dkim-test.txt
/usr/sbin/sendmail -t < dkim-test.txt
== Setup Rspamd ==
See [[https://rspamd.com/doc/quickstart.html Rspamd quick start]] for the latest installation and setup instructions.
Install Rspamd. Redis will be installed automatically(check).
$ sudo apt install rspamd
Check service is running.
$ systemctl status rspamd.service
  ● rspamd.service - rapid spam filtering system
  Loaded: loaded (/lib/systemd/system/rspamd.service; enabled; vendor preset: enabled)
  Active: active (running) since Sat 2020-11-07 14:54:03 UTC; 1min 58s ago
    Docs: https://rspamd.com/doc/
    Main PID: 19339 (rspamd)
    Tasks: 5 (limit: 2336)
    Memory: 276.2M
    CGroup: /system.slice/rspamd.service
          ├─19339 rspamd: main process
          ├─19517 rspamd: rspamd_proxy process (localhost:11332)
          ├─19518 rspamd: controller process (localhost:11334)
          ├─19519 rspamd: normal process (localhost:11333)
          └─19520 rspamd: hs_helper process
    Nov 07 14:54:03 lists.fsci.org.in systemd[1]: Started rapid spam filtering system.
Check if redis is working properly using redis-cli
$ redis-cli
  127.0.0.1:6379> ping
  PONG
  127.0.0.1:6379> ping Hello[qwe]
  "Hello[qwe]"
  127.0.0.1:6379> exit
Change some redis config to use it as a LRU cache[https://redis.io/topics/lru-cache].
On /etc/redis/redis.conf
~line 850
maxmemory 500mb
maxmemory-policy volatile-ttl
bind 127.0.0.1 ::1 //already exists
'''Configure rspamd'''
$ sudo rspamadm configwizard
  ____                                    _
|  _ \  ___  _ __    __ _  _ __ ___    __| |
| |_) |/ __|| '_ \  / _` || '_ ` _ \  / _` |
|  _ < \__ \| |_) || (_| || | | | | || (_| |
|_| \_\|___/| .__/  \__,_||_| |_| |_| \__,_|
            |_|
Welcome to the configuration tool
We use /etc/rspamd/rspamd.conf configuration file, writing results to /etc/rspamd
Modules enabled: hfilter, phishing, emails, asn, settings, chartable, arc, bayes_expiry, once_received, rbl, fuzzy_check, metadata_exporter, elastic, mid, multimap, spf, dkim_signing, dkim, mime_types, regexp, maillist, dmarc,  forged_recipients, milter_headers, whitelist, force_actions, trie
Modules disabled (explicitly): p0f, spamtrap, rspamd_update, mx_check, dcc
Modules disabled (unconfigured): spamassassin, maps_stats, metric_exporter, dynamic_conf, clustering, reputation, antivirus, fuzzy_collect, external_services, ip_score, clickhouse
Modules disabled (no Redis): greylist, url_redirector, replies, neural, ratelimit, history_redis
Modules disabled (experimental):
Modules disabled (failed):
Do you wish to continue?[Y/n]:
Setup WebUI and controller worker:
Controller password is not set, do you want to set one?[Y/n]: Y
Enter passphrase:
Set encrypted password to: <password_hash>
Redis servers are not set:
The following modules will be enabled if you add Redis servers:
    * greylist
    * url_redirector
    * replies
    * neural
    * ratelimit
    * history_redis
Do you wish to set Redis servers?[Y/n]:
Input read only servers separated by `,` [default: localhost]:
Input write only servers separated by `,` [default: localhost]:
Do you have any password set for your Redis?[y/N]:
Do you have any specific database for your Redis?[y/N]:
Do you want to setup dkim signing feature?[y/N]: N
File: /etc/rspamd/local.d/redis.conf, changes list:
write_servers => localhost
read_servers => localhost
File: /etc/rspamd/local.d/worker-controller.inc, changes list:
password => <password_hash>
Apply changes?[Y/n]: Y
Create file /etc/rspamd/local.d/redis.conf
Create file /etc/rspamd/local.d/worker-controller.inc
2 changes applied, the wizard is finished now
*** Please reload the Rspamd configuration ***
Now, restart the rspamd service and check status
$ sudo systemctl restart rspamd
$ sudo systemctl status rspamd
  ● rspamd.service - rapid spam filtering system
    Loaded: loaded (/lib/systemd/system/rspamd.service; enabled; vendor preset: enabled)
    Active: active (running) since Sat 2020-11-07 15:28:14 UTC; 4s ago
      Docs: https://rspamd.com/doc/
    Main PID: 21899 (rspamd)
    Tasks: 5 (limit: 2336)
    Memory: 116.5M
    CGroup: /system.slice/rspamd.service
          ├─21899 rspamd: main process
          ├─21915 rspamd: rspamd_proxy process (localhost:11332)
          ├─21916 rspamd: controller process (localhost:11334)
          ├─21917 rspamd: normal process (localhost:11333)
          └─21918 rspamd: hs_helper process
    Nov 07 15:28:14 lists.fsci.org.in systemd[1]: Started rapid spam filtering system.
'''Worker Proxy setup'''
in /etc/rspamd/local.d/worker-proxy.inc   
# local.d/worker-proxy.inc
milter = yes; # Enable milter mode
timeout = 120s; # Needed for Milter usually
upstream "local" {
  default = yes; # Self-scan upstreams are always default
  self_scan = yes; # Enable self-scan
}
count = 4; # Spawn more processes in self-scan mode
max_retries = 5; # How many times master is queried in case of failure
discard_on_reject = false; # Discard message instead of rejection
quarantine_on_reject = false; # Tell MTA to quarantine rejected messages
spam_header = "X-Spam"; # Use the specific spam header
reject_message = "Spam message rejected"; # Use custom rejection message
'''Disable DKIM Signing on rspamd'''
To disable DKIM signing (i.e. you use OpenDKIM, or signing is done elsewhere- ie, that's our case)
# local.d/dkim_signing.conf
enabled = false;
and restart rspamd service after that.
'''Setting up the WebUI'''
<br />For using the location, <webroot>/rspamd,
<br />Add the following on /etc/mailman3/nginx.conf
location /rspamd/ {
    proxy_pass      http://localhost:11334/;
    proxy_set_header Host      $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
Now, check nginx syntax, restart the nginx service and check the status.
<br/> After restart, you can visit <webroot>/rspamd and login with the previously created password.
<br />
'''Setting up rspamd Worker proxy config on postfix'''
<br />Rspamd Worker proxy config
<br />On /etc/postfix/main.cf
Don't forget to disable existing  configs for the same below
##
## Spam filter and DKIM signatures via Rspamd
##
smtpd_milters = inet:localhost:11332, local:/opendkim/opendkim.sock
non_smtpd_milters = inet:localhost:11332, local:/opendkim/opendkim.sock
milter_protocol = 6
milter_mail_macros =  i {mail_addr} {client_addr} {client_name} {auth_authen}
milter_default_action = accept
and then restart postfix.
Finally, enable rspamd service if everything seems to work properly
<code>$ sudo systemctl enable rspamd</code>.
== Remove spam confirmation requests ==
Too many pending subscription requests can slow down[https://gitlab.com/mailman/postorius/-/issues/417] list index. Updating postorius is how to fix this. But in case there's no other way, and removing those requests through frontend [https://gitlab.com/mailman/mailman/-/issues/779#note_423132958] is impossible, this can be done (GNU Mailman 3.2.2 (La Villa Strangiato)):
$ sudo mailman shell -l listname.lists.fsci.org.in
Welcome to the GNU Mailman shell
The variable 'm' is the listname@lists.fsci.org.in mailing list
>>> from zope.component import getUtility
>>> from mailman.interfaces.pending import IPendings
>>> pendings = getUtility(IPendings)
>>> plist = []
>>> for p in pendings.find(m):
...  plist.append(p)
...
>>> len(plist)
2047
>>> # Let us save the spammer email and when they signed up for analysis
>>> import csv
>>> with open('spam.csv', 'w', newline='') as csvfile:
...    spamwriter = csv.writer(csvfile, delimiter=',')
...    for p in plist:
...        spammer = p[1]
...        spamwriter.writerow([spammer.get('email'), spammer.get('when'), spammer.get('display_name')])
...
>>> # Now on to actually removing spam
>>> from mailman.interfaces.workflow import IWorkflowStateManager
>>> workflow = getUtility(IWorkflowStateManager)
>>> for p in plist:
...    token = p[0]
...    pendings.confirm(token)
...    workflow.discard(token)
...
>>> # Now, we have to commit this transaction to database
>>> from mailman.config import config
>>> config.db.commit()


----
----


[[Category: Services]]
[[Category: Services]]
Retrieved from "https://wiki.fsci.in/Special:MobileDiff/10888...11045"