Additionally, the test farm needs to be able to provide a short enough
turnaround time that we can get our MRs through marge-bot without the
pipeline backing up. As a result, we require that the test farm be
-able to handle a whole pipeline's worth of jobs in less than 5 minutes
-(to compare, the build stage is about 10 minutes, if you could get all
-your jobs scheduled on the shared runners in time.).
+able to handle a whole pipeline's worth of jobs in less than 15 minutes
+(to compare, the build stage is about 10 minutes).
If a test farm is short the HW to provide these guarantees, consider
dropping tests to reduce runtime.
to just run 1/10th of the test list.
If a HW CI farm goes offline (network dies and all CI pipelines end up
-stalled) or its runners are consistenly spuriously failing (disk
+stalled) or its runners are consistently spuriously failing (disk
full?), and the maintainer is not immediately available to fix the
issue, please push through an MR disabling that farm's jobs by adding
'.' to the front of the jobs names until the maintainer can bring
report to mesa-dev@lists.freedesktop.org after the fact explaining
what happened and what the mitigation plan is for that failure next
time.
+
+Personal runners
+----------------
+
+Mesa's CI is currently run primarily on packet.net's m1xlarge nodes
+(2.2Ghz Sandybridge), with each job getting 8 cores allocated. You
+can speed up your personal CI builds (and marge-bot merges) by using a
+faster personal machine as a runner. You can find the gitlab-runner
+package in debian, or use gitlab's own builds.
+
+To do so, follow `gitlab's instructions
+<https://docs.gitlab.com/ce/ci/runners/#create-a-specific-runner>`__ to
+register your personal gitlab runner in your Mesa fork. Then, tell
+Mesa how many jobs it should serve (``concurrent=``) and how many
+cores those jobs should use (``FDO_CI_CONCURRENT=``) by editing these
+lines in ``/etc/gitlab-runner/config.toml``, for example::
+
+ concurrent = 2
+
+ [[runners]]
+ environment = ["FDO_CI_CONCURRENT=16"]
+
+
+Docker caching
+--------------
+
+The CI system uses docker images extensively to cache
+infrequently-updated build content like the CTS. The `freedesktop.org
+CI templates
+<https://gitlab.freedesktop.org/freedesktop/ci-templates/>`_ help us
+manage the building of the images to reduce how frequently rebuilds
+happen, and trim down the images (stripping out manpages, cleaning the
+apt cache, and other such common pitfalls of building docker images).
+
+When running a container job, the templates will look for an existing
+build of that image in the container registry under
+``FDO_DISTRIBUTION_TAG``. If it's found it will be reused, and if
+not, the associated `.gitlab-ci/containers/<jobname>.sh`` will be run
+to build it. So, when developing any change to container build
+scripts, you need to update the associated ``FDO_DISTRIBUTION_TAG`` to
+a new unique string. We recommend using the current date plus some
+string related to your branch (so that if you rebase on someone else's
+container update from the same day, you will get a git conflict
+instead of silently reusing their container)
+
+When developing a given change to your docker image, you would have to
+bump the tag on each ``git commit --amend`` to your development
+branch, which can get tedious. Instead, you can navigate to the
+`container registry
+<https://gitlab.freedesktop.org/mesa/mesa/container_registry>`_ for
+your repository and delete the tag to force a rebuild. When your code
+is eventually merged to master, a full image rebuild will occur again
+(forks inherit images from the main repo, but MRs don't propagate
+images from the fork into the main repo's registry).
projects / mesa.git / blobdiff