Note: Currently new registrations are closed, if you want an account Contact us

Difference between revisions of "GitLab"

From FSCI Wiki
(→‎Backup: update for postgresql 11)
 
(4 intermediate revisions by 2 users not shown)
Line 3: Line 3:
==Hosting==
==Hosting==


Sponsored by [http://about.gitlab.com GitLab Inc]. Hosted at gandi.net in France.
Sponsored by [https://www.infomaniak.com/en/about Infomaniak]. Hosted in Switzerland with infomaniak.com.
 
Previously sponsored by [http://about.gitlab.com GitLab Inc] (2016-2022)


==Setup==
==Setup==
Line 15: Line 17:
Letsencrypt domains:
Letsencrypt domains:


   letsencrypt --expand --webroot --webroot-path /usr/share/gitlab/public -d git.fosscommunity.in -d gitlabce.tk -d gitlab.debian.net -d wiki.fsci.org.in -d git.fsci.org.in  certonly
   letsencrypt --expand --webroot --webroot-path /usr/share/gitlab/public -d git.fosscommunity.in -d gitlab.debian.net -d wiki.fsci.org.in -d git.fsci.org.in  certonly


== Mail Server Setup ==
== Mail Server Setup ==
Line 29: Line 31:
==Maintenance==
==Maintenance==


Maintenance discussion at [https://www.loomio.org/g/Qu3O8mSf/fosscommunity-in-git-fosscommunity-in-maintainers this loomio subgroup] and [https://matrix.to/#/#git.fosscommunity.in:matrix.org this matrix chat room]
Maintenance discussion at [https://www.loomio.org/g/Qu3O8mSf/fosscommunity-in-git-fosscommunity-in-maintainers this loomio subgroup] and [https://matrix.to/#/#git.fosscommunity.in:matrix.org this matrix chat room]. Can also use the [https://git.fosscommunity.in/community/management/-/issues git tracker] for development issues.


* [https://git.fosscommunity.in/community/gitlab/ team repo with access details] (private)
* [https://git.fosscommunity.in/community/gitlab/ team repo with access details] (private)
If expecting service disruptions, downtime, or user-facing errors, consider adding an update to [[Status]]


==Gitlab QA - running tests==
==Gitlab QA - running tests==
Line 47: Line 51:
Backup server is provided by Manu (KVM virtual machine with 100 GB storage and 1 GB ram). Running on debian gnu/linux 10 buster.  
Backup server is provided by Manu (KVM virtual machine with 100 GB storage and 1 GB ram). Running on debian gnu/linux 10 buster.  


Documentation: https://linuxhint.com/setup_postgresql_replication/
Documentation: https://linuxhint.com/setup_postgresql_replication/ and https://www.percona.com/blog/2018/09/07/setting-up-streaming-replication-postgresql/


===Slave configuration: Step 1===
===Slave configuration: Step 1===
Line 134: Line 138:


  # systemctl restart postgresql
  # systemctl restart postgresql
Start the backup process,
psql -c "select pg_start_backup('initial_backup');"
rsync -cva -e 'ssh -p 12022' --inplace --exclude=*pg_xlog* /var/lib/postgresql/11/main/ 62.210.83.200:/var/lib/postgresql/11/main/
psql -c "select pg_stop_backup();"


Open 5432 port in the firewall
Open 5432 port in the firewall
Line 146: Line 144:


===Slave Configuration: Step 2===
===Slave Configuration: Step 2===
Create a recovery file called  recovery.conf and add following lines.
 
  standby_mode = 'on'
Copy data from master and create recovery.conf
primary_conninfo = 'host=213.167.243.152 port=5432 user=rep password=yourpassword'
  $ pg_basebackup -h git.fosscommunity.in -D /var/lib/postgresql/11/main/ -P -U rep --wal-method=fetch -R
  trigger_file = '/tmp/postgresql.trigger.5432'


Start the slave server
Start the slave server
Line 160: Line 157:


  $ ps -ef | grep sender
  $ ps -ef | grep sender
$ psql -c "select * from pg_stat_activity where usename='rep';"
On slave server,
On slave server,



Latest revision as of 09:13, 24 November 2022

Our public GitLab instance is https://git.fosscommunity.in

Hosting

Sponsored by Infomaniak. Hosted in Switzerland with infomaniak.com.

Previously sponsored by GitLab Inc (2016-2022)

Setup

Running on debian gnu/linux 10 buster with postgresql 11. Using gitlab package from http://fasttrack.debian.net

Check https://wiki.debian.org/gitlab for installation/update instructions.

Debian package specific documentation -> https://salsa.debian.org/ruby-team/gitlab/raw/buster-fasttrack/debian/README.Debian

Letsencrypt domains:

 letsencrypt --expand --webroot --webroot-path /usr/share/gitlab/public -d git.fosscommunity.in -d gitlab.debian.net -d wiki.fsci.org.in -d git.fsci.org.in  certonly

Mail Server Setup

Maintenance

Maintenance discussion at this loomio subgroup and this matrix chat room. Can also use the git tracker for development issues.

If expecting service disruptions, downtime, or user-facing errors, consider adding an update to Status

Gitlab QA - running tests

<Create a user account for QA>

$ sudo gem install gitlab-qa

$ GITLAB_USERNAME=<username> GITLAB_PASSWORD=<password> gitlab-qa Test::Instance::Any gitlab/gitlab-ce-qa:<gitlab version> https://git.hacksk.xyz

Backup

Backup server is provided by Manu (KVM virtual machine with 100 GB storage and 1 GB ram). Running on debian gnu/linux 10 buster.

Documentation: https://linuxhint.com/setup_postgresql_replication/ and https://www.percona.com/blog/2018/09/07/setting-up-streaming-replication-postgresql/

Slave configuration: Step 1

Install postgresql and rsync

# apt-get install postgresql-contrib-11 rsync

Check postgresql server is running

# su postgres -c psql

Make sure en_US.UTF-8 locale is available

# dpkg-reconfigure locales

Create ssh key for postgres user

# su - postgres
$ ssh-keygen -t ed25519

Now copy /var/lib/postgresql/.ssh/id_ed25519.pub to master postgres users' /var/lib/postgresql/.ssh/authorized_keys

Stop postgresql before changing any configuration

# systemctl stop postgresql@11-main

as postgres user

$ su - postgres
$ cd /etc/postgresql/11/main

Open the postgres configuration file

$ nano postgresql.conf

Set the following configuration options in the postgresql.conf file

listen_addresses = 'localhost,192.168.0.115'
port=5432
wal_level = replica
max_wal_senders = 1
archive_mode = on
archive_command = 'cd .'
hot_standby = on

Master configuration

Create and copy ssh public key to slave like above.

# su - postgres
$ ssh-keygen

Now copy /var/lib/postgresql/.ssh/id_rsa.pub to slave postgres users' /var/lib/postgresql/.ssh/authorized_keys

Create postgresql user for replication.

$ psql -c "CREATE USER rep REPLICATION LOGIN CONNECTION LIMIT 1 ENCRYPTED PASSWORD 'yourpassword';"

Allow slave to connect to master using the user just created.

$ cd /etc/postgresql/11/main
$ nano pg_hba.conf

Add below line to allow rep user to get access to the server

host    replication     rep     62.210.83.200/32   md5

Next , open the postgres configuration file

nano postgresql.conf

Set the following configuration options in the postgresql.conf file

listen_addresses = 'localhost,213.167.243.152'
port=5432
wal_level = replica
max_wal_senders = 1
archive_mode = on
archive_command = 'cd .'
hot_standby = on

Now, to activate your changes, reload the postgresql server

# systemctl reload postgresql@11-main

You may need to restart it via systemd,

# systemctl restart postgresql

Open 5432 port in the firewall

# ufw allow from 62.210.83.200 to any port 5432 proto tcp

Slave Configuration: Step 2

Copy data from master and create recovery.conf

$ pg_basebackup -h git.fosscommunity.in -D /var/lib/postgresql/11/main/ -P -U rep --wal-method=fetch  -R

Start the slave server

# systemctl start postgresql@11-main

Replication Status

On master server ,

$ ps -ef | grep sender
$ psql -c "select * from pg_stat_activity where usename='rep';"

On slave server,

$ ps -ef | grep receiver