Jump to contentJump to page navigation: previous page [access key p]/next page [access key n]

35 Build Constraints

Build constraints provide a way for the user to specify build worker parameters that the Build Service will use to decide which build workers are "qualified" to undertake a given build.

They are intended to be used for defining known, hard requirements for successfully building a given package (for example, disk space, memory, or certain CPU functionality).

35.1 Build Resource Usage and Statistics

Ideally, the build constraints should be set to the minimum values that enable a build to succeed, because any higher setting than the minimum might unnecessarily reduce the number of build workers available to build the package.

Now, in the real world, we do not always have a precise idea of what the minimum values are for all the different build worker parameters that can be controlled using OBS build constraints. That need not deter us from setting build constraints, however. It is not necessary to wait for a build failure before setting minimum memory and disk space constraints, for example, because the OBS can give us reasonable values for memory and disk space based on a successful build.

Each successful build produces a file called _statistics which can be examined to get details of the resources our build consumed. We can then use this information to set appropriate values for the relevant build constraints.

35.1.1 Displaying the build statistics

The information from the _statistics file can be found in the Build Service web UI, by clicking on the build target we are interested in and then clicking on "Show resources". Alternatively, the _statistics file itself can be downloaded from the Build Service, either from the command line or using the OBS API.

35.1.1.1 Downloading the _statistics file using osc

Since the _statistics file is a build artifact produced by every successful build, it is always included among the build artifacts downloaded by osc getbinaries.

35.1.1.2 Downloading the _statistics file using the OBS API

If you are using the OBS API, the relevant call is:

GET /build/{project_name}/{repository_name}/{architecture_name}/{package_name}/_statistics
Important
Important

When reviewing the build statistics, it's important to be aware that the numbers can vary significantly from build to build depending on build parallelism (e.g. make -j).

35.2 Constraint Qualifiers

In general, build constraints are specified in terms of a qualifier and a value. The qualifier expresses "what" - the build worker parameter that is to be constrained - and the semantics of the value depend on the qualifier. If the qualifier takes a numeric value, it generally expresses "how much", or, in other words, the minimum value (of that parameter) that a given build worker must meet in order to fulfill the constraint. But there are some qualifiers, such as hostlabel, that take a simple string value.

In the simplest cases, the qualifier is just a string. In more complicated cases, the qualifier can include subqualifiers and be modified by attributes. For example:

hardware:disk:size unit=G

The string hardware:disk:size in this example means "size, a subqualifier of disk, which is itself a subqualifier of hardware", and unit=G means "the value is expressed in units of Gigabytes.

For a full treatment of constraint syntax, see Section 35.4, “Constraint syntax”.

35.3 Constraint scope and precedence

Depending on the required scope, constraint qualifiers and their values can be set at four different levels: instance, project, package, and build recipe. Setting constraints at the OBS instance level is up to the administrator of the OBS instance and is covered in the OBS Admin Guide. Project-wide constraints are defined in the project configuration. Package constraints are defined in a special file, _constraints, in the packages sources. It is also possible to insert constraints directly in the individual build recipes (RPM spec files, Dockerfiles).

The constraints that are in force for a particular build are determined by merging all constraints defined at all levels: site, project, package, and build recipe. The merging is done in that order, with later settings overwriting earlier ones. That means, for example, that site-wide constraints can be overrided at any of the lower levels (project, package, build recipe), project-level constraints can be overrided at the package and build recipe levels, etc.

35.3.1 Project-scoped constraints

Build constraints for an entire project, or for specific repositories within it, or for specific architectures within those repositories, are defined in the project config by adding lines as so:

Constraint: <QUALIFIER> <VALUE>

The QUALIFIER syntax is the same as used in RPM spec files, documented in Section 35.4, “Constraint syntax”. Within the project configuration, individual Constraint lines can be enclosed in guards to make a constraint apply only to certain architectures or repositories. For example:

%ifarch ppc ppc64 ppc64le
Constraint: hardware:cpu:flag power8
%endif

or

%if "%_repository" == "images"
Constraint: hardware:disk:size unit=M 4000
%endif
Important
Important

Constraints set in project configuration affect not only the project itself, but also all projects that build against it.

For a full treatment of constraint syntax, see Section 35.4, “Constraint syntax”.

35.3.2 Package-scoped constraints

Setting constraints at the package level is achieved by including an XML file called _constraints in the package sources. The Build Service will attempt to validate this file when it is committed (from the osc command line) or saved (in the web UI) to prevent invalid XML from reaching the Build Service.

Here is a simple example showing what a _constraints file might look like:

<?xml version="1.0"?>
<constraints>
  <hardware>
    <physicalmemory>
      <size unit="M">2000</size>
    </physicalmemory>
    <disk>
      <size unit="G">5</size>
    </disk>
  </hardware>
  <sandbox>kvm</sandbox>
  <hostlabel exclude="true">SLOW_CPU</hostlabel>
</constraints>

For details on constraint qualifiers and how to specify them in a _constraints file, see Section 35.4, “Constraint syntax”.

35.3.3 Build recipe-scoped constraints

At the build recipe level, constraints are set by embedding lines containing constraint qualifiers and values directly in RPM spec files or Dockerfiles. Such lines take the form

#!BuildConstraint: <QUALIFIER> <VALUE>

The # character at the beginning of the line causes the build recipe parser (RPM, Docker, podman) to ignore the whole line, but in combination with the ! character signifies an OBS-specific directive that is picked up by a pre-processor within the OBS back-end.

Instead of specifying subqualifiers by nesting directives like in XML, colons are used. For example:

#!BuildConstraint: linux:version:min 3.0

In this example, "linux:version:min" is the constraint qualifier and "3.0" is the value.

As described for project-scoped conditionals, above, #!BuildConstraint lines can be guarded with various conditionals to limit their effect to certain architectures or, e.g., multibuild flavors.

Important
Important

Be careful when setting build constraints. The idea is to use them to express minimum values for the various parameters, below which builds are likely to fail. If you set a constraint too high, you risk reducing the pool of compliant build workers down to a very low number, or even to zero. (A low number of compliant build workers means your build may not start for a long time, and no compliant workers at all will cause the build to fail. See Section 35.5, “Constraint Handling” for details.)

Important
Important

By default, constraints applied to build workers regardless of architecture. However, you may only be interested in certain architectures and not in others. See Section 35.6, “Checking Constraints with osc for how to get architecture-specific information on which workers satisfy your constraints.

35.4 Constraint syntax

This section describes the various constraint qualifiers and their syntax.

35.4.1 hostlabel

The "hostlabel" qualifier is any string which can be assigned to build workers when starting the bs_worker process. Since its intended use is to restrict a build to specific workers, it should only be used after consultation with OBS administrators who have detailed knowledge of the build farm, and ideally as a negative definition, using the exclude=true attribute. Even then, keep in mind that hostlabel settings are not portable, since the always specific to a given OBS instance and should therefore only be used as a last resort in situations that cannot be addressed by any of the other qualifiers.

An example use case is to run benchmarks in a reproducible way.

Example for _constraints file:

<constraints exclude="false">
   <hostlabel>benchmark_runner</hostlabel>
</constraints>

Example for project configuration:

Constraint: hostlabel benchmark_runner

Example for RPM spec file or Dockerfile:

#!BuildConstraint: hostlabel benchmark_runner

35.4.2 sandbox

Defines the "sandbox" which is used for the build. The "sandbox" is the virtual environment in which the build takes place: each build worker is configured with a fixed sandbox type.

The configuration of this build constraint is typically left to OBS administrators, and there is usually no reason for a project or package maintainer to set it.

Example for _constraints file:

<constraints>
   <sandbox exclude="true">kvm</sandbox>
</constraints>

Example for project configuration:

Constraint: sandbox exclude="true" kvm

Example for RPM spec file or Dockerfile:

#!BuildConstraint: sandbox exclude="true" kvm

35.4.3 linux

This is a category of constraints specific to the Linux kernel, applied to the kernel running on the build worker.

35.4.3.1 version

To require a specific Linux kernel version or version range.

Example for the _constraints file:

<constraints>
   <linux><version>
     <min>3.0</min>
     <max>4.0</max>
   </version></linux>
</constraints>

Example for project configuration:

Constraint: linux:version:min 3.0
Constraint: linux:version:max 4.0

Example for RPM spec file or Dockerfile:

#!BuildConstraint: linux:version:min 3.0
#!BuildConstraint: linux:version:max 4.0
35.4.3.1.1 min

Minimum kernel version.

35.4.3.1.2 max

Maximum kernel version.

35.4.3.2 flavor

A specific kernel flavor, such as default or smp (corresponding to the kernel packages kernel-default and kernel-smp, respectively).

Example for _constraints file:

<constraints>
    <linux>
      <flavor>default</flavor>
    </linux>
</constraints>

Example for project configuration:

Constraint: linux:flavor default

Example for RPM spec file or Dockerfile:

#!BuildConstraint: linux:flavor default

35.4.4 hardware

To specify that build workers must meet certain minimum hardware specifications or possess certain hardware features.

35.4.4.1 cpu

To require a specific CPU feature.

35.4.4.1.1 flag

CPU features which are provided by the hardware. On Linux, these can be found in /proc/cpuinfo. The flag element may be used multiple times to require multiple CPU features.

Example for _constraints file:

<constraints>
   <hardware><cpu>
     <flag>mmx</flag>
     <flag>sse2</flag>
   </cpu></hardware>
</constraints>

Example for project configuration:

Constraint: hardware:cpu:flag mmx
Constraint: hardware:cpu:flag sse2

Example for RPM spec file or Dockerfile:

#!BuildConstraint: hardware:cpu:flag mmx
#!BuildConstraint: hardware:cpu:flag sse2

EL0 is a special flag that that can be used on hardware that only supports level-0 exceptions, such as certain armv8l systems. This means that VMs or 32-bit kernels are not supported but userland is supported. This flag can be used to block builds on such hardware if no 64-bit kernel is available for a project.

Example for project configuration:

Constraint: hardware:cpu:flag exclude=true EL0

Example for RPM spec file or Dockerfile:

#!BuildConstraint: hardware:cpu:flag exclude=true EL0

Additional flags are also reported for effective architecture level of the CPU. This includes the following flags:

  • power7

  • power8

  • power9

  • x86-64-v2

  • x86-64-v3

  • x86-64-v4

35.4.4.2 processors

To specify a minimum number of processor cores, virtual or physical (i.e., as reported by /proc/cpuinfo, provided by the build worker and usable for the build

Example for _constraints file:

<constraints>
   <hardware>
     <processors>4</processors>
   </hardware>
</constraints>

Example for project configuration:

Constraint: hardware:processors 4

Example for RPM spec file or Dockerfile:

#!BuildConstraint: hardware:processors 4

35.4.4.3 jobs

Each build worker is configured with a given default for the JOBS environment variable, which expresses how many parallel build processes the worker should be able to sustain. To require a minimal number of pre-confiured parallel jobs for the build, use this qualifier.

Note
Note

This specifies the number of parallel jobs the build tooling should use, so even though it is under hardware it is not actually a hardware requirement.

Example for _constraints file:

<constraints>
   <hardware>
     <jobs>4</jobs>
   </hardware>
</constraints>

Example for project configuration:

Constraint: hardware:jobs 4

Example for RPM spec file or Dockerfile:

#!BuildConstraint: hardware:jobs 4

35.4.4.4 disk

Hard disk specific constraints.

35.4.4.4.1 size

To specify a minimum amount of free disk space, below which a build on the worker will not be attempted.

Example for _constraints file:

<constraints>
  <hardware>
     <disk>
       <size unit="G">4</size>
     </disk>
  </hardware>
</constraints>

Example for project configuration:

Constraint: hardware:disk:size unit=G 4

Example for RPM spec file or Dockerfile:

#!BuildConstraint: hardware:disk:size unit=G 4

35.4.4.5 memory

To specify a minimum amount of RAM memory that the worker must be equipped with.

35.4.4.5.1 size

To require a minimum memory size, including swap space.

Example for _constraints file:

<constraints>
   <hardware>
     <memory>
       <size unit="M">1400</size>
     </memory>
   </hardware>
</constraints>

Example for project configuration:

Constraint: hardware:memory:size unit=M 1400

Example for RPM spec file or Dockerfile:

#!BuildConstraint: hardware:memory:size unit=M 1400

35.4.4.6 physicalmemory

Memory specific.

35.4.4.6.1 size

To require a minimal memory size. Swap space is not taken into account here.

Example for _constraints file:

<constraints>
   <hardware>
     <physicalmemory>
       <size unit=M>1400</size>
     </physicalmemory>
   </hardware>
</constraints>

Example for project configuration:

Constraint: hardware:physicalmemory:size unit=M 1400

Example for RPM spec file or Dockerfile:

#!BuildConstraint: hardware:physicalmemory:size unit=M 1400

35.5 Constraint Handling

What happens when someone sets a constraint so high, that the OBS instance does not have even a single worker that meets it? What happens when just a few workers satisfy all the constraints, but all of them are busy building packages, or have been taken down for maintenance? This section describes how the OBS handles these "low compliant worker" situations.

35.5.1 At least one compliant worker is available

After determining which build workers satisfy the defined constraints for a given build, the scheduler checks if any of them are available to start building. If at least one is available, the build begins. The rest of this section describes the OBS's behavior when no compliant build workers are free to start building a given package.

35.5.2 More than half of existing workers satisfy the constraints

The build will stay in state "scheduled" until one of the compliant workers becomes available. No further notification is set.

35.5.3 Less than half of existing workers satisfy the constraints

The build will stay in state scheduled until one of the compliant workers becomes available. In addition, the dispatch details are set to tell the user that this build might take a long time to complete. The notification looks like this:

waiting for 4 compliant workers (4 down)

In this case, all four compliant workers are down. The notification helps the user understand why the build is not starting. The dispatch details can also be retrieved using the OBS API or, e.g., using the command osc results -v).

35.5.4 No existing workers satisfy the constraints

If no worker can handle the constraints defined by the package or project, the build fails. In such cases, the build log will mention why the build failed:

package build was not possible:

no compliant workers (constraints mismatch hint: hardware:processors sandbox)

Please adapt your constraints.

35.6 Checking Constraints with osc

You can check the constraints of a project or package with the osc tool. You have to be in an osc working directory.

osc checkconstraints [OPTS] [REPOSITORY] [ARCH] [CONSTRAINTSFILE]

Either you give a repository and an arch or osc will check the constraints for all repository / arch pairs for the package. A few examples:

geeko > osc checkconstraints
Repository                Arch                      Worker
----------                ----                      ------
openSUSE_Leap_42.2        x86_64                    1
openSUSE_Leap_42.1        x86_64                    1

If no file is given it takes the local _constraints file. If this file does not exist or the --ignore-file switch is set only the project constraints are used.

geeko > osc checkconstraints openSUSE_Leap_42.1 x86_64
Worker
------
x86_64:worker:1
x86_64:worker:2

If a repository and an arch is given a list of compliant workers is returned.

Another command to verify a worker and display the worker information is osc workerinfo.

geeko > osc workerinfo x86_64:worker:1
<worker hostarch="x86_64" registerserver="http://localhost:5252" workerid="worker:1">
  <hostlabel>MY_WORKER_LABEL_1</hostlabel>
  <sandbox>chroot</sandbox>
  <linux>
    <version>4.1.34-33</version>
    <flavor>default</flavor>
  </linux>
  <hardware>
    <cpu>
      <flag>fpu</flag>
      <flag>vme</flag>
      <flag>de</flag>
    </cpu>
    <processors>2</processors>
    <jobs>1</jobs>
  </hardware>
</worker>

It returns the information of the desired worker.

Print this page