CI/CD Standards for Code Libraries

[thedrow]

Points for discussion:

CI

• Code Linters
• Test Runner & Required Plugins
• Definition of Unit Tests
• Definition of Functional Tests
• Definition of Integration/Acceptance Tests
• Test Coverage Target
• Build Stages
• Reporting Results
• Required Checks
• Optional Checks
• CI Services

CD

• Nightly Build
• Trigger Releases Using Tags
• Standards for Releases (When to fail to release)

[thedrow]

@luisehk What do you think about taking this on yourself?

I'll shepherd it and be part of the implementation team.
If you can complete this, I think we will all vote to admit you in.

[luisehk]

Sure thing @thedrow, lets do this. What is expected from me? To start working on the CEP for this and start discussing it?

[thedrow]

Yes.
For example, we use pytest. We want to mandate that all our repositories use pytest- because XYZ.

We'd like to introduce new CI services: Drone.io for testing on ARM.
https://cirrus-ci.org/ for FreeBSD.
We also don't test on MacOSX right now.
We'd like to consider dropping AppVeyor and consolidating whatever is possible to one service.
We'd also like to consider if Travis-CI is still our best option.
Maybe cirrus-ci is better.
We have some donations, so we may consider to pay for a CI service if needed or if we wanttto increase our worker concurrency so a price comparison is also required.

[luisehk]

Sounds good, on it.

[matteius]

You mention not having the OSX build, and Technically AppVeyor is a Windows build. As I am a Developer on the Windows platform it is of interest to me that we have a Windows build (such as AppVeyor) and attempt to make its configuration pass where possible if its free to maintain the integration.

[auvipy]

cirrus have all

[thedrow]

cirrus have all

I tested cirrus and it is really really slow.
It's better to use KVM in Travis-CI if we want BSD builds.

[auvipy]

OK great

[graingert]

I'd like to see everything move to Github Actions, as the integration is much cleaner. Certainly for Windows support using a complete microsoft stack makes sense to me

[thedrow]

It's certainly a good direction to take but we need to review the alternatives too before we make a choice.
There might be an advantage to using Azure Pipelines or using both Github Actions and Azure Pipelines together (one can trigger the other).

We need a review and a conclusion to make an informed decision.
You can start by checking out our CEP's template.