summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorAntonio Terceiro <terceiro@softwarelivre.org>2018-08-24 16:26:49 -0300
committerAntonio Terceiro <terceiro@softwarelivre.org>2018-08-24 16:59:49 -0300
commit8403d0c7d1df3ace935833cbd4e72ef1127ab6f0 (patch)
treeafe3079d84a651a159ab47f1fc562ed14891d3c4
parentf5abda9a1ee60c24e8a09fca931755b997c30d16 (diff)
linuxdev-br 2018: initial final version ;-)
-rw-r--r--2018/08-26-linuxdev-br-debian-ci/Makefile4
-rw-r--r--2018/08-26-linuxdev-br-debian-ci/ci.debian.net-debci.pngbin0 -> 130999 bytes
-rw-r--r--2018/08-26-linuxdev-br-debian-ci/debian-ci.md154
3 files changed, 150 insertions, 8 deletions
diff --git a/2018/08-26-linuxdev-br-debian-ci/Makefile b/2018/08-26-linuxdev-br-debian-ci/Makefile
index 5814b35..311d5b7 100644
--- a/2018/08-26-linuxdev-br-debian-ci/Makefile
+++ b/2018/08-26-linuxdev-br-debian-ci/Makefile
@@ -10,6 +10,10 @@ HANDOUT = $(PANDOC)
all: $(slideshow) $(handout)
+slideshow: $(slideshow)
+
+handout: $(handout)
+
$(slideshow): $(source)
$(SLIDESHOW) --output $@ $<
diff --git a/2018/08-26-linuxdev-br-debian-ci/ci.debian.net-debci.png b/2018/08-26-linuxdev-br-debian-ci/ci.debian.net-debci.png
new file mode 100644
index 0000000..6e4bf7e
--- /dev/null
+++ b/2018/08-26-linuxdev-br-debian-ci/ci.debian.net-debci.png
Binary files differ
diff --git a/2018/08-26-linuxdev-br-debian-ci/debian-ci.md b/2018/08-26-linuxdev-br-debian-ci/debian-ci.md
index 048bccf..4ecf942 100644
--- a/2018/08-26-linuxdev-br-debian-ci/debian-ci.md
+++ b/2018/08-26-linuxdev-br-debian-ci/debian-ci.md
@@ -1,15 +1,16 @@
% The Debian Continuous Integration project
% Antonio Terceiro
+% August 26th, 2018
# Introduction
## The role of distributions
+* Provide a convenient user experience for managing and using a system
* Be a bridge between upstream developers and users
* Get upstream software to users
* Integrate different upstream components
* Triage and forward feedback and patches back to upstream
-* Provide a convenient user experience for managing and using a system
## How Debian works
@@ -80,7 +81,27 @@
* Regressions: package migrates *slower*
* For the future, the plan is to completely *block* migration on regressions
-## The platform
+## The platform: tools
+
+* `autopkgtest`: the test runner tool
+* `autodep8`: tool to generate test specifications for well-known types of
+ packages
+* `debci`: the CI platform itself
+ * can be installed locally by any user
+ * official instance running at [ci.debian.net](https://ci.debian.net/)
+* `debian-ci-config`: configuration management repository for ci.debian.net
+ * not a package
+* all source code available at
+ [salsa.debian.org/ci-team](https://salsa.debian.org/ci-team)
+
+------------------------------------------------------------------------
+
+## The platform: web UI
+
+* mostly static HTML
+ * pro: no performance concerns
+ * con: not up to date all the time
+* HTTP API
------------------------------------------------------------------------
@@ -88,19 +109,136 @@
------------------------------------------------------------------------
+![https://ci.debian.net/packages/d/debci/unstable/amd64/](ci.debian.net-debci.png)
+
+------------------------------------------------------------------------
+
![Architecture](architecture-hosts.svg)
+## The platform: hosting
+
+
+* 1 master node
+* 10 worker nodes, running tests for the `amd64` architecture
+* ci.debian.net is kindly hosted on AWS on a Debian account with credits
+ sponsored by Amazon.
+
------------------------------------------------------------------------
![Test job queue](debci_queue_size-week.png)
-## Stats
+
+## The platform: test execution environment
+
+* standard autopkgtest operation
+ * developers can easily reproduce the exact same test run on their local
+ machine
+* tests are run in LXC containers
+ * full system container (*vs* application containers)
+ * clean system: nothing installed besides what the test itself specifies
+
+# The Developer Point of view (writing tests)
+
+## Basic concepts
+
+* Tests are part of the source package
+* Specification: DEP-8
+* Tests defined in `debian/tests/control`
+* Source metadata (`debian/control`) must provide a `Testsuite:` field
+ * added automatically if `debian/tests/control` exists by `dpkg` >= 1.17.11
+* Tests must exercise the package *installed* on a system, i.e. it should
+ not test the *source tree*.
+
+## Example
+
+```
+$ grep Source: debian/control
+Source: auto-apt-proxy
+$ cat debian/tests/control
+Tests: apt-integration
+Depends: @, apt-cacher-ng
+
+Test-Command: clitest debian/tests/apt-cacher-ng.txt
+Depends: @, clitest, apt-cacher-ng
+
+[...]
+
+Tests: remove, reinstall
+Restrictions: needs-root
+Depends: @, clitest
+$
+```
+
+## autopkgtest
+
+### virt: LXC
+
+```
+$ autopkgtest --no-built-packages . \
+ -- lxc --sudo autopkgtest-unstable-amd64
+```
+
+Runs the tests from the source package in the current directory (`.`), but
+using binary packages from the repository.
+
+### virt: null
+
+```
+$ autopkgtest --no-built-packages . -- null
+```
+
+Runs the tests from the source package in the current directory (`.`) againt
+the *current system* ("null" virtualization). Requires that the needed packages
+are already installed.
+
+## autodep8
+
+* Solves the problem of several similar packages having identical
+ `debian/tests/control` (e.g. "Ruby libraries", "Kernel modules using DKMS")
+* When a package does not have an explicit `debian/tests/control`,
+ `autopkgtest` calls `autodep8` to produce one.
+* Instead of duplicating `debian/tests/control`, packages declare a special
+ value for their `Testsuite:` field and let autodep8 produce the test
+ specification.
+* `autodep8` can also detect supported package types by looking at the contents
+ of packages
+* Currently supported: Ruby, Perl, Python*, NodeJS*, DKMS, R, Emacs Lisp, Go,
+ Octave
+
+## autodep8: example
+
+
+```
+$ grep Source: debian/control
+Source: ruby-json
+$ cat debian/tests/control
+cat: debian/tests/control: No such file or directory
+$ autodep8
+Test-Command: gem2deb-test-runner --autopkgtest
+ --check-dependencies 2>&1
+Depends: @, ruby-test-unit, gem2deb-test-runner
+```
+
+## More information on writing tests
+
+Debian CI documentation:
+[https://ci.debian.net/doc/](https://ci.debian.net/doc/)
+
+
+Patterns for Writing As-Installed Tests for Debian Packages
+[https://deb.li/pattestdeb](https://deb.li/pattestdeb)
# Conclusions
-## Acknowledgements
+## Lessons
+
+* Adoption *takes time*
+* Improvements to the system are always needed
+* Using the very same tools that developers can use directly makes everyone's
+ lifes easier.
+* Most of the ideas (and maybe even some of the tools) presented here could be
+ applied to other distributions. I would love to chat about that.
+
+## Thank you
-* Ian Jackson created autopkgtest
-* Martin Pitt maintained autopkgtest for a long time
-* Brandon Fairchild: GSOC intern; worked on web UI
-* Lucas Kanashiro: GSOC intern; worked on fixing early test suites
+Contact: terceiro@debian.org