We will present a framework, called TULIP for "Testing Upgrades of Linux Images Program" whose purpose is to drive upgrade tests of various linux distributions to ensure both fine grained QA at the package level and testing of the standard update mechanism.

Based on fresh installations and day after day upgrades, the framework will detect packets installation failures, interactions leading to binary incompatibility and launch adminstrator unit tests. Various upgrade periodicity will also be tested such as daily, weekly and monthly deltas.

The results will be gathered in a database for every test case and reports showing state and evolution of the packages upgrade quality will be built.

TULIP is based on qemu images (other virtual environments may also be plugged into the system). On defining a new test campain, the base images are generated as well as the rotation policiy. Then, on a daily basis, the upgrade process is exercised. Once upgraded, the images are tested regarding binary dependencies using dependencies testing tools like ldd. Explicit QA scripts can also be run in this phase.

The presentation will focus on describing this architecture and the first results of the upgrade testing campaign on two well-known distributions (Debian and Mandriva). Packages failures as well as binary dependancies problem will be reported.

The possibilities opened by the framework will then be discussed.