Workflow integration

Testsuite of project

  • Upstream testsuite for project located in /usr/share/moduleframework/examples/testing-module/
    • You can use it as an inspiration for your tests
    • It contains various types how to schedule tests
    • CI It contains info how it is scheduled in internal-ci or in taskotron or how to do
    • Examples of Manual running of tests on localhost.
    • Example how to run general multi-host tests
    • Every new feature should be covered here - by new Makefile target or by new test run inside every testing module

Taskotron Wokflow

  • Production instance: https://taskotron.fedoraproject.org/resultsdb/results?testcases=dist.modularity-testing-framework
    • Triggered fedmsg via module-stream-version string
    • Triggered by Module Build system done message, list of all: https://apps.fedoraproject.org/datagrepper/raw?topic=org.fedoraproject.prod.mbs.module.state.change
    • There is general runtask.yml taskotron trigger: https://pagure.io/taskotron/task-modularity-testing-framework
      • There is just one for every module and it contains whole logic where to find tests for module.
      • Not needed to duplicate runtash.yml for each component. Scheduler is same (existing Makefile)
      • It run tools/run-them.sh script. It contains whole logic where are tests and how to find them.
    • run-them.sh script for taskotron
      • Test Subject: rpm repositories (tagged koji builds of packages) via systemd-nspawn
      • Located in: /usr/share/moduleframework/tools/run-them.sh
      • Scheduled as: ./run-them.sh testmodule testmodule-master-20170407121558 pdc
      • Example targets: check-run-them-pdc-testmodule, check-run-them-pdc-baseruntime
      • Internal logic
        • Contact PDC (Product definition center) for info about module like koji tags, moduleMD file
        • Try dowload package from modules namespace in dist-git via fedpkg clone
          • checkout to proper version found by PDC (scmurl)
          • Try to find tests there ( if exist Makefile in tests directory)
        • If None: Try to find module dir in MTF project tests in /usr/share/moduleframework/examples directory
        • If None: Run at least general ModuleLinter (/usr/share/moduleframework/tests/modulelint) with general minimal config.yaml located in docs directory

Arbitrary Jenkins Instance

  • Production instance: hidden
    • Triggered via fedmsg file
      • Used tools/run-them.sh script, for same behaviour as Taskotron
    • run-them.sh script for Jenkins based on whole fedmsg
      • Test Subject: Same as Taskotron Workflow
      • Located in: Same as Taskotron Workflow
      • Scheduled as: run-them.sh testmodule /usr/share/moduleframework/tools/example_message_module.yaml fedmsg
      • Example targets: check-run-them-fedmsg-testmodule
      • Internal logic
        • Same as Taskotron Workflow