Note: Currently new registrations are closed, if you want an account Contact us

Difference between revisions of "GitLab"

From FSCI Wiki
 
(10 intermediate revisions by 2 users not shown)
Line 3: Line 3:
==Hosting==
==Hosting==


Sponsored by [http://about.gitlab.com GitLab Inc].Hosted at gandi.net in France.
Sponsored by [https://www.infomaniak.com/en/about Infomaniak]. Hosted in Switzerland with infomaniak.com.
 
Previously sponsored by [http://about.gitlab.com GitLab Inc] (2016-2022)


==Setup==
==Setup==


Running on debian gnu/linux 9 stretch with postgresql 9.6. Using gitlab package from https://people.debian.org/~praveen/gitlab/
Running on debian gnu/linux 10 buster with postgresql 11. Using gitlab package from http://fasttrack.debian.net
 
Check https://wiki.debian.org/gitlab for installation/update instructions.
 
Debian package specific documentation -> https://salsa.debian.org/ruby-team/gitlab/raw/buster-fasttrack/debian/README.Debian


Letsencrypt domains:
Letsencrypt domains:


   letsencrypt --expand --webroot --webroot-path /usr/share/gitlab/public -d git.fosscommunity.in -d gitlabce.tk -d gitlab.debian.net -d wiki.fsci.org.in -d git.fsci.org.in  certonly
   letsencrypt --expand --webroot --webroot-path /usr/share/gitlab/public -d git.fosscommunity.in -d gitlab.debian.net -d wiki.fsci.org.in -d git.fsci.org.in  certonly


== Mail Server Setup ==
== Mail Server Setup ==
Line 21: Line 27:
* Using 'inet_interfaces = 127.0.0.1' in main.cf
* Using 'inet_interfaces = 127.0.0.1' in main.cf
* Configured DKIM following https://www.linode.com/docs/email/postfix/configure-spf-and-dkim-in-postfix-on-debian-8/
* Configured DKIM following https://www.linode.com/docs/email/postfix/configure-spf-and-dkim-in-postfix-on-debian-8/
* DMARC is set to reject


==Maintenance==
==Maintenance==


Maintenance discussion at [https://www.loomio.org/g/Qu3O8mSf/fosscommunity-in-git-fosscommunity-in-maintainers this loomio subgroup] and [https://matrix.to/#/#git.fosscommunity.in:matrix.org this matrix chat room]
Maintenance discussion at [https://www.loomio.org/g/Qu3O8mSf/fosscommunity-in-git-fosscommunity-in-maintainers this loomio subgroup] and [https://matrix.to/#/#git.fosscommunity.in:matrix.org this matrix chat room]. Can also use the [https://git.fosscommunity.in/community/management/-/issues git tracker] for development issues.


* [https://git.fosscommunity.in/community/gitlab/ team repo with access details] (private)
* [https://git.fosscommunity.in/community/gitlab/ team repo with access details] (private)
If expecting service disruptions, downtime, or user-facing errors, consider adding an update to [[Status]]
==Gitlab QA - running tests==
<Create a user account for QA>
<code>
$ sudo gem install gitlab-qa
$ GITLAB_USERNAME=<username> GITLAB_PASSWORD=<password> gitlab-qa Test::Instance::Any gitlab/gitlab-ce-qa:<gitlab version> https://git.hacksk.xyz
</code>


==Backup==
==Backup==


Backup server is provided by Manu. Running on debian gnu/linux 9 stretch.
Backup server is provided by Manu (KVM virtual machine with 100 GB storage and 1 GB ram). Running on debian gnu/linux 10 buster.  
 
Documentation: https://linuxhint.com/setup_postgresql_replication/ and https://www.percona.com/blog/2018/09/07/setting-up-streaming-replication-postgresql/


===Slave configuration: Step 1===
===Slave configuration: Step 1===
Install postgresql and rsync
Install postgresql and rsync


  # apt-get install postgresql-contrib-9.6 rsync
  # apt-get install postgresql-contrib-11 rsync
 
Check postgresql server is running
 
# su postgres -c psql


Make sure en_US.UTF-8 locale is available
Make sure en_US.UTF-8 locale is available
Line 44: Line 69:


  # su - postgres
  # su - postgres
  $ ssh-keygen
  $ ssh-keygen -t ed25519


Now copy /var/lib/postgresql/.ssh/id_rsa.pub to master postgres users' /var/lib/postgresql/.ssh/authorized_keys
Now copy /var/lib/postgresql/.ssh/id_ed25519.pub to master postgres users' /var/lib/postgresql/.ssh/authorized_keys


Stop postgresql before changing any configuration
Stop postgresql before changing any configuration
  $ pg_ctlcluster 9.6 main stop
  # systemctl stop postgresql@11-main


  cd /etc/postgresql/9.6/main
as postgres user
  $ su - postgres
$ cd /etc/postgresql/11/main


Open the postgres configuration file
Open the postgres configuration file


  nano postgresql.conf
  $ nano postgresql.conf
 
Set the following configuration options in the postgresql.conf file
Set the following configuration options in the postgresql.conf file


  listen_addresses = 'localhost,192.168.0.115'
  listen_addresses = 'localhost,192.168.0.115'
  port=5432
  port=5432
  wal_level = 'hot_standby'
  wal_level = replica
max_wal_senders = 1
  archive_mode = on
  archive_mode = on
  archive_command = 'cd .'
  archive_command = 'cd .'
max_wal_senders = 1
  hot_standby = on
  hot_standby = on


Line 81: Line 109:
Allow slave to connect to master using the user just created.
Allow slave to connect to master using the user just created.


  $ cd /etc/postgresql/9.6/main
  $ cd /etc/postgresql/11/main


  $ nano pg_hba.conf
  $ nano pg_hba.conf
Line 97: Line 125:
  listen_addresses = 'localhost,213.167.243.152'
  listen_addresses = 'localhost,213.167.243.152'
  port=5432
  port=5432
  wal_level = 'hot_standby'
  wal_level = replica
max_wal_senders = 1
  archive_mode = on
  archive_mode = on
  archive_command = 'cd .'
  archive_command = 'cd .'
max_wal_senders = 1
  hot_standby = on
  hot_standby = on


Now, to activate your changes, reload the postgresql server
Now, to activate your changes, reload the postgresql server


  $ pg_ctlcluster 9.6 main reload
  # systemctl reload postgresql@11-main


You may need to restart it via systemd,
You may need to restart it via systemd,
Line 111: Line 139:
  # systemctl restart postgresql
  # systemctl restart postgresql


Start the backup process,
Open 5432 port in the firewall


  psql -c "select pg_start_backup('initial_backup');"
  # ufw allow from 62.210.83.200 to any port 5432 proto tcp
rsync -cva -e 'ssh -p 12022' --inplace --exclude=*pg_xlog* /var/lib/postgresql/9.6/main/ 62.210.83.200:/var/lib/postgresql/9.6/main/
psql -c "select pg_stop_backup();"


===Slave Configuration: Step 2===
===Slave Configuration: Step 2===
Create a recovery file called  recovery.conf and add following lines.
 
  standby_mode = 'on'
Copy data from master and create recovery.conf
primary_conninfo = 'host=213.167.243.152 port=5432 user=rep password=yourpassword'
  $ pg_basebackup -h git.fosscommunity.in -D /var/lib/postgresql/11/main/ -P -U rep --wal-method=fetch -R
  trigger_file = '/tmp/postgresql.trigger.5432'


Start the slave server
Start the slave server


  $ pg_ctlcluster 9.6 main start
  # systemctl start postgresql@11-main


===Replication Status===
===Replication Status===
Line 132: Line 157:


  $ ps -ef | grep sender
  $ ps -ef | grep sender
$ psql -c "select * from pg_stat_activity where usename='rep';"
On slave server,
On slave server,



Latest revision as of 09:13, 24 November 2022

Our public GitLab instance is https://git.fosscommunity.in

Hosting

Sponsored by Infomaniak. Hosted in Switzerland with infomaniak.com.

Previously sponsored by GitLab Inc (2016-2022)

Setup

Running on debian gnu/linux 10 buster with postgresql 11. Using gitlab package from http://fasttrack.debian.net

Check https://wiki.debian.org/gitlab for installation/update instructions.

Debian package specific documentation -> https://salsa.debian.org/ruby-team/gitlab/raw/buster-fasttrack/debian/README.Debian

Letsencrypt domains:

 letsencrypt --expand --webroot --webroot-path /usr/share/gitlab/public -d git.fosscommunity.in -d gitlab.debian.net -d wiki.fsci.org.in -d git.fsci.org.in  certonly

Mail Server Setup

Maintenance

Maintenance discussion at this loomio subgroup and this matrix chat room. Can also use the git tracker for development issues.

If expecting service disruptions, downtime, or user-facing errors, consider adding an update to Status

Gitlab QA - running tests

<Create a user account for QA>

$ sudo gem install gitlab-qa

$ GITLAB_USERNAME=<username> GITLAB_PASSWORD=<password> gitlab-qa Test::Instance::Any gitlab/gitlab-ce-qa:<gitlab version> https://git.hacksk.xyz

Backup

Backup server is provided by Manu (KVM virtual machine with 100 GB storage and 1 GB ram). Running on debian gnu/linux 10 buster.

Documentation: https://linuxhint.com/setup_postgresql_replication/ and https://www.percona.com/blog/2018/09/07/setting-up-streaming-replication-postgresql/

Slave configuration: Step 1

Install postgresql and rsync

# apt-get install postgresql-contrib-11 rsync

Check postgresql server is running

# su postgres -c psql

Make sure en_US.UTF-8 locale is available

# dpkg-reconfigure locales

Create ssh key for postgres user

# su - postgres
$ ssh-keygen -t ed25519

Now copy /var/lib/postgresql/.ssh/id_ed25519.pub to master postgres users' /var/lib/postgresql/.ssh/authorized_keys

Stop postgresql before changing any configuration

# systemctl stop postgresql@11-main

as postgres user

$ su - postgres
$ cd /etc/postgresql/11/main

Open the postgres configuration file

$ nano postgresql.conf

Set the following configuration options in the postgresql.conf file

listen_addresses = 'localhost,192.168.0.115'
port=5432
wal_level = replica
max_wal_senders = 1
archive_mode = on
archive_command = 'cd .'
hot_standby = on

Master configuration

Create and copy ssh public key to slave like above.

# su - postgres
$ ssh-keygen

Now copy /var/lib/postgresql/.ssh/id_rsa.pub to slave postgres users' /var/lib/postgresql/.ssh/authorized_keys

Create postgresql user for replication.

$ psql -c "CREATE USER rep REPLICATION LOGIN CONNECTION LIMIT 1 ENCRYPTED PASSWORD 'yourpassword';"

Allow slave to connect to master using the user just created.

$ cd /etc/postgresql/11/main
$ nano pg_hba.conf

Add below line to allow rep user to get access to the server

host    replication     rep     62.210.83.200/32   md5

Next , open the postgres configuration file

nano postgresql.conf

Set the following configuration options in the postgresql.conf file

listen_addresses = 'localhost,213.167.243.152'
port=5432
wal_level = replica
max_wal_senders = 1
archive_mode = on
archive_command = 'cd .'
hot_standby = on

Now, to activate your changes, reload the postgresql server

# systemctl reload postgresql@11-main

You may need to restart it via systemd,

# systemctl restart postgresql

Open 5432 port in the firewall

# ufw allow from 62.210.83.200 to any port 5432 proto tcp

Slave Configuration: Step 2

Copy data from master and create recovery.conf

$ pg_basebackup -h git.fosscommunity.in -D /var/lib/postgresql/11/main/ -P -U rep --wal-method=fetch  -R

Start the slave server

# systemctl start postgresql@11-main

Replication Status

On master server ,

$ ps -ef | grep sender
$ psql -c "select * from pg_stat_activity where usename='rep';"

On slave server,

$ ps -ef | grep receiver