Feature request: allow command line arguments instead of an xml

[RoelvandenBerg]

Request

• I would like to be able to a run a test on a docker team engine container by just giving an url without having to mount an xml.
• I would like the script to optionally return an exit 1 when the tests fail.

Usecase
Both requests would make running tests in continuous integration easier (e.g. could create an integration test that shows a service remains ogc compliant), just as it would for regular regression tests that check the validity of services as they are running. We would be able to parametrize the output.

Possible implementation
An example implementation without editing the team engine code: https://github.com/PDOK/ets-ogcapi-features10-docker

[dstenger]

Thank you for providing this feature request.

In general, this repository currently does not provide CLI functionality via Docker but a Tomcat is started containing the webapp of TEAM Engine and a selection of test suites.

So, your proposal is to also provide separate Docker Images which can be used as CLIs.

First question would be if there shall be one Docker Image containing multiple test suites or one Docker Image for each test suite.

Also, we have to consider that there are two different mechanisms of using test suites as CLI.

• TEAM Engine CLI
  • Similar setup to TEAM Engine webapp
  • Multiple test suites can be used by single CLI
  • Works for both, TestNG and CTL test suites
  • Also see https://opengeospatial.github.io/teamengine/users.html
• *-aio.jar
  • Is just supported by TestNG test suites (does not work with CTL test suites)
  • Each test suite has its own *-aio.jar
  • Your referenced approach seems to use the *-aio.jar

Second question would be to decide which of the two options shall be used.

[RoelvandenBerg]

I don't think seperate *-aio.jar images are necessary. We just used the one because we needed that one for our test and found a readme with that particular setup. A larger image that contains all tests would probably even be better if (as I presume) that would make them much bulkier.