Development process

This document, specifically the file ROADMAP.rst, defines the Repository Service for TUF Roadmap.

Each Roadmap Goal must have a project in the Repository Service for TUF Organization.

The Roadmap contains the desired Repository Service for TUF (RSTUF) features. Each feature has an issue open in the repository-service-tuf/repository-service-tuf (Umbrella repository) issues.

The Component specifies the Milestone (release) that will contain the Roadmap Goal in the ROADMAP.rst (also visible below).

It allows the Component Maintainers to combine Repository Service for TUF Features and internal Component projects during a release.

Working on the Component Issues:

  • If the issue is related or has an impact on the Roadmap Goal, select the Project Goal it to the Milestone.

../_images/development_process.png

Proposing new features

In RSTUF we have two types of Features.

Component-only feature

The component-only features are related and internal to a component (i.e., Repository Service for TUF API or Repository Service for TUF Worker). These features don’t impact the entire RSTUF platform, such as performance, breaking/improving API contracts, user usage/deployment, etc.

We recommend for this type of feature to open a new issue in the specific component.

RSTUF Feature

The RSTUF feature is a large-scope feature that impacts the entire RSTUF platform and requires a more structured proposal to apply/implement cross-components or even user usage change.

We recommend for these changes the following:

  • Always design/document the feature in a new issue.

    • Use as many diagrams as possible (PlantUML or Mermaid in the issues). ❤️

  • After discussing and receiving enough feedback, propose a BDD Feature file (ask for a feature branch).

  • Once the BDD Feature file is approved and merged to a feature branch, open the related issues for the component(s)

Testings

RSTUF uses two test types approach for different levels of tests as RSTUF has multiple components.

Unit tests

We do unit tests (UT) in each component. The intention is to test the low-level components, such as individual methods and functions of the classes.

We use the pytest as a standard framework. Also, to avoid mixed boundaries between RSTUF code and libraries or systems, we use the mocking technique in the UT. For mocking, we use monkeypatch from pytest and pretend to implement the stubbing technic.

To speed up our UT execution time (both locally and on CI) we use the pytest-xdist <https://pypi.org/project/pytest-xdist/> plugin that distributes tests across multiple CPUs. To take advantage of this parallelism we maintain a UT suite that is independent of the order of test execution.

More details will be in each specific component.

Functional Tests using Behavior Development Driven (BDD)

BDD Feature file

For RSTUF Features level, we write the Gherkin feature file in the test features.

The goal is to describe the Scenarios and Behavior for our feature clearly. We try to have clear requirements for implementing, maintaining, and changing the existing features. We also use it as the starting point for proposing RSTUF new Features. From that, we can design, start planning issues, etc.

A new BDD Feature file for an unexisting RSTUF Feature is always merged into a feature branch. After discussing the new feature request, the feature branch is created upon request by the maintainers.

Tooling

We use pytest and pytest-bdd as the framework.

All BDD tests are in the Umbrella repository, inside tests/functional .

The BDD tests have the workflow also in the Umbrella repository , as a reusable GitHub Workflow, and it is triggered by other workflows/components i.e., before releasing.

Running Functional Tests

It is possible to run Functional Tests (FT) from each component, including the Umbrella repository.

  1. Start the development deployment make run-dev

  2. Start the functional tests make ft-<type> (Check Makefile to see the available types of tests).

The functional tests has global environment variables that can be used:

  • PERFORMANCE: bool, it disables the failure in case of low performance timeout. It is used to test only the consistency.

  • METADATA_BASE_URL: to use some custom metatada base url in the TUF Client. Default is http://web:8080

This environment variables needs to be passed to the container that calls the pytest or ft-<name> scripts.

Splitting FT execution

To speed up our FT execution time we use the pytest-split <https://pypi.org/project/pytest-split/> plugin that splits the test suite so that we can run tests across multiple jobs.

We create a job for slow tests (such as performance tests) and another set of jobs that take advantage of the groups created by pytest-split to distribute and run the tests parallelly.

Project organization

The project uses the microservices approach. Each RSTUF components have its own development instructions.

Components Repositories