docs/ci/bare-metal.rst

   1 Bare-metal CI
   2 =============
   3
   4 The bare-metal scripts run on a system with gitlab-runner and docker,
   5 connected to potentially multiple bare-metal boards that run tests of
   6 Mesa.  Currently only "fastboot" and "ChromeOS Servo" devices are
   7 supported.
   8
   9 In comparison with LAVA, this doesn't involve maintaining a separate
  10 webservice with its own job scheduler and replicating jobs between the
  11 two.  It also places more of the board support in git, instead of
  12 webservice configuration.  On the other hand, the serial interactions
  13 and bootloader support are more primitive.
  14
  15 Requirements (fastboot)
  16 -----------------------
  17
  18 This testing requires power control of the DUTs by the gitlab-runner
  19 machine, since this is what we use to reset the system and get back to
  20 a pristine state at the start of testing.
  21
  22 We require access to the console output from the gitlab-runner system,
  23 since that is how we get the final results back from the tests.  You
  24 should probably have the console on a serial connection, so that you
  25 can see bootloader progress.
  26
  27 The boards need to be able to have a kernel/initramfs supplied by the
  28 gitlab-runner system, since the initramfs is what contains the Mesa
  29 testing payload.
  30
  31 The boards should have networking, so that we can extract the dEQP .xml
  32 results to artifacts on gitlab.
  33
  34 Requirements (servo)
  35 --------------------
  36
  37 For servo-connected boards, we can use the EC connection for power
  38 control to reboot the board.  However, loading a kernel is not as easy
  39 as fastboot, so we assume your bootloader can do TFTP, and that your
  40 gitlab-runner mounts the runner's tftp directory specific to the board
  41 at /tftp in the container.
  42
  43 Since we're going the TFTP route, we also use NFS root.  This avoids
  44 packing the rootfs and sending it to the board as a ramdisk, which
  45 means we can support larger rootfses (for piglit or tracie testing),
  46 at the cost of needing more storage on the runner.
  47
  48 Telling the board about where its TFTP and NFS should come from is
  49 done using dnsmasq on the runner host.  For example, this snippet in
  50 the dnsmasq.conf.d in the google farm, with the gitlab-runner host we
  51 call "servo"::
  52
  53   dhcp-host=1c:69:7a:0d:a3:d3,10.42.0.10,set:servo
  54
  55   # Fixed dhcp addresses for my sanity, and setting a tag for
  56   # specializing other DHCP options
  57   dhcp-host=a0:ce:c8:c8:d9:5d,10.42.0.11,set:cheza1
  58   dhcp-host=a0:ce:c8:c8:d8:81,10.42.0.12,set:cheza2
  59
  60   # Specify the next server, watch out for the double ',,'.  The
  61   # filename didn't seem to get picked up by the bootloader, so we use
  62   # tftp-unique-root and mount directories like
  63   # /srv/tftp/10.42.0.11/jwerner/cheza as /tftp in the job containers.
  64   tftp-unique-root
  65   dhcp-boot=tag:cheza1,cheza1/vmlinuz,,10.42.0.10
  66   dhcp-boot=tag:cheza2,cheza2/vmlinuz,,10.42.0.10
  67
  68   dhcp-option=tag:cheza1,option:root-path,/srv/nfs/cheza1
  69   dhcp-option=tag:cheza2,option:root-path,/srv/nfs/cheza2
  70
  71 Setup
  72 -----
  73
  74 Each board will be registered in fd.o gitlab.  You'll want something
  75 like this to register a fastboot board:
  76
  77 .. code-block:: console
  78
  79   sudo gitlab-runner register \
  80        --url https://gitlab.freedesktop.org \
  81        --registration-token $1 \
  82        --name MY_BOARD_NAME \
  83        --tag-list MY_BOARD_TAG \
  84        --executor docker \
  85        --docker-image "alpine:latest" \
  86        --docker-volumes "/dev:/dev" \
  87        --docker-network-mode "host" \
  88        --docker-privileged \
  89        --non-interactive
  90
  91 For a servo board, you'll need to also volume mount the board's NFS
  92 root dir at /nfs and TFTP kernel directory at /tftp.
  93
  94 The registration token has to come from a fd.o gitlab admin going to
  95 https://gitlab.freedesktop.org/admin/runners
  96
  97 The name scheme for Google's lab is google-freedreno-boardname-n, and
  98 our tag is something like google-freedreno-db410c.  The tag is what
  99 identifies a board type so that board-specific jobs can be dispatched
 100 into that pool.
 101
 102 We need privileged mode and the /dev bind mount in order to get at the
 103 serial console and fastboot USB devices (--device arguments don't
 104 apply to devices that show up after container start, which is the case
 105 with fastboot, and the servo serial devices are actually links to
 106 /dev/pts).  We use host network mode so that we can spin up a nginx
 107 server to collect XML results for fastboot.
 108
 109 Once you've added your boards, you're going to need to add a little
 110 more customization in ``/etc/gitlab-runner/config.toml``.  First, add
 111 ``concurrent = <number of boards>`` at the top ("we should have up to
 112 this many jobs running managed by this gitlab-runner").  Then for each
 113 board's runner, set ``limit = 1`` ("only 1 job served by this board at a
 114 time").  Finally, add the board-specific environment variables
 115 required by your bare-metal script, something like::
 116
 117   [[runners]]
 118     name = "google-freedreno-db410c-1"
 119     environment = ["BM_SERIAL=/dev/ttyDB410c8", "BM_POWERUP=google-power-up.sh 8", "BM_FASTBOOT_SERIAL=15e9e390"]
 120
 121 If you want to collect the results for fastboot you need to add the following
 122 two board-specific environment variables ``BM_WEBDAV_IP`` and ``BM_WEBDAV_PORT``.
 123 These represent the IP address of the docker host and the board specific port number
 124 that gets used to start a nginx server.
 125
 126 Once you've updated your runners' configs, restart with ``sudo service
 127 gitlab-runner restart``