resources/server-setup/git-mirroring.rst

   1 =======================
   2 Git Mirroring to GitLab
   3 =======================
   4
   5 Steps for setting up automatic mirroring cron jobs:
   6
   7 #) Add a new user:
   8
   9    .. code:: bash
  10
  11      sudo adduser --disabled-login --system git-mirroring
  12
  13 #) Start a shell as the :code:`git-mirroring` user:
  14
  15    .. code:: bash
  16
  17      sudo -H -u git-mirroring /bin/bash
  18
  19 #) Switch to the home directory:
  20
  21    .. code:: bash
  22
  23      cd
  24
  25 #) Create an executable file :code:`sync.sh` (replace :code:`nano` with the editor of your choice):
  26
  27    .. code:: bash
  28
  29      touch sync.sh
  30      chmod +x sync.sh
  31      nano sync.sh
  32
  33    a) Type in the following contents:
  34
  35       .. code:: bash
  36
  37         #!/bin/sh
  38         for repo in ~/*.git; do
  39             cd "$repo" && git fetch -q && git push -q gitlab || echo "sync failed for $repo"
  40         done
  41
  42    #) Save and exit the editor.
  43
  44 #) Create a ssh key:
  45
  46    .. code:: bash
  47
  48      ssh-keygen
  49
  50    Press enter for all prompts -- leaving the file names as default and with an empty password.
  51
  52 #) For each repo to be mirrored:
  53
  54    In the following commands, replace :code:`$UPSTREAM_URL` with the url for the upstream repo, :code:`$NAME` with the name selected for the local directory.
  55
  56    a) Create an empty repo using the GitLab website.
  57
  58       .. warning::
  59
  60         Following these steps will overwrite anything that is in the GitLab repo.
  61
  62    In the following commands, replace :code:`$MIRROR_URL` with the ssh url for the mirror, you can get it by clicking the Clone button for the empty repo you just created.
  63
  64    b) Add the ssh key for the :code:`git-mirroring` user to the GitLab repo as a `deploy key with read/write access <https://docs.gitlab.com/ce/ssh/#per-repository-deploy-keys>`_:
  65
  66       Get the ssh public key for the :code:`git-mirroring` user:
  67
  68       .. code:: bash
  69
  70         cat ~/.ssh/id_rsa.pub
  71
  72    #) Clone the upstream repo:
  73
  74       .. code:: bash
  75
  76         git clone --mirror "$UPSTREAM_URL" ~/"$NAME".git
  77
  78    #) Switch to the directory where you just cloned the repo:
  79
  80       .. code:: bash
  81
  82         cd ~/"$NAME".git
  83
  84    #) Add the mirror as a new remote to allow pushing to it:
  85
  86       .. code:: bash
  87
  88         git remote add --mirror=push gitlab "$MIRROR_URL"
  89
  90    #) Try pushing to the new remote to ensure everything is operating correctly:
  91
  92       .. code:: bash
  93
  94         git push gitlab
  95
  96 #) Switch back to the home directory:
  97
  98    .. code:: bash
  99
 100      cd
 101
 102 #) Ensure :code:`sync.sh` works properly:
 103
 104    .. code:: bash
 105
 106      ./sync.sh
 107
 108 #) Set up the cron job for running :code:`sync.sh`:
 109
 110    a) Edit the user's :code:`crontab`:
 111
 112       .. code:: bash
 113
 114         crontab -u git-mirroring -e
 115
 116    #) Add the following line to the end, then save and exit:
 117
 118       .. code::
 119
 120         * * * * * exec ~/sync.sh >> ~/sync.log 2>&1
 121
 122 #) Wait for the cron job to run (2 minutes should be sufficient), then check the log file:
 123
 124    .. code:: bash
 125
 126      less ~/sync.log
 127
 128    The log file should exist and be empty, if it isn't empty, it contains error messages. If it doesn't exist, the cron job didn't run yet for some reason, try waiting a little longer.
 129
 130 #) Try pushing a commit to one of the upstream repos and seeing if it gets properly synced to the corresponding mirror repo. The sync job runs every minute, so wait 2 minutes and check.