summaryrefslogtreecommitdiff
path: root/2018/08-26-linuxdev-br-debian-ci/debian-ci.md
blob: 048bccfd987f241a9d289229783a9c48f8fb0470 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
% The Debian Continuous Integration project
% Antonio Terceiro

# Introduction

## The role of distributions

* Be a bridge between upstream developers and users
  * Get upstream software to users
  * Integrate different upstream components
  * Triage and forward feedback and patches back to upstream
* Provide a convenient user experience for managing and using a system

## How Debian works

* *unstable*: receives updates directly from developers
* *testing*: updates migrate from *unstable*
  * package has no release-critical bugs
  * all dependencies are also eligible for migration
  * **package has spent at least 2/5/10 days in unstable**
  * migration is automated, and automatic
* *stable*: every ~2 years *testing* becomes *stable*
* Freeze: unstable → testing migration is no longer automatic

## Distribution QA

* "If it builds, it works"
  * Common attitude in large transitions (libraries, toolchains)
  * Doesn't work always
* Sometimes focused on critical paths
  * installation is a big concern
  * less common use cases are not prioritized
* Sometimes general QA is relegated to "QA people"

## Automated tests

* Undeniably useful
* Test-Driven Development (TDD)
* Regression testing
* Different levels
  * Unit testing
  * Integration testing
  * Functional testing

## Putting it all together

* **"package has spent at least 2/5/10 days in unstable"**
  * depends on the package having enough users in *unstable*
  * depends on users finding bugs soon enough
* we want *testing* to be safe for everyday use, and to always be in a
  releaseable state
* Can we find bugs ealier?
* Can we avoid having the most basic bugs from blowing up on users' faces?
* Let's apply automated testing

# Debian Continuous Integration Project

## History

* 2006: first prototype of `autopkgtest`
* early 2014: initial hacking on Debian CI
* mid 2014: two GSOC interns working on Debian CI
* late 2015: distributed architecture
* mid 2017: initial discussions about testing migration integration
* mid 2018: testing migration uses Debian CI test results as input

------------------------------------------------------------------------

![Number of packages tested in *unstable*](packages-unstable.png)

------------------------------------------------------------------------

![Number of packages tested in *testing*](packages-testing.png)

## Testing migration

* No regressions: package migrates *faster*
  * tests for package pass
  * tests for all packages that depend on it also pass
* Regressions: package migrates *slower*
* For the future, the plan is to completely *block* migration on regressions

## The platform

------------------------------------------------------------------------

![https://ci.debian.net/](ci.debian.net.png)

------------------------------------------------------------------------

![Architecture](architecture-hosts.svg)

------------------------------------------------------------------------

![Test job queue](debci_queue_size-week.png)

## Stats

# Conclusions

## Acknowledgements

* Ian Jackson created autopkgtest
* Martin Pitt maintained autopkgtest for a long time
* Brandon Fairchild: GSOC intern; worked on web UI
* Lucas Kanashiro: GSOC intern; worked on fixing early test suites