Stage2: Unit testing and Component-level testing

[Rorck]

Stage2: Unit testing and Component-level testing

For the descriptions of the testing levels see the D3.1 deliverable. Prerequisites are the responsibility of the BB development team, these must be fulfilled before the validation of the BB. Make sure that the links are pointing to the proper section of the documentation!

Prereq:

• Internal Unit testing
  • Commands to set up test environment/Dockerfile/Dockercompose file
  • Commands to run tests
  • Test execution report/summary (see: template)
• Component-level tests
  • Commands to set up test environment/Dockerfile/Dockercompose file
  • Commands to run tests
  • Test execution report/summary (see: template)
• If you completed the above tasks, mention the @Prometheus-X-association/bme-testing team and ask for a CHECK

Validation:

• Reproduce Internal Unit Tests
  • Rerun Internal Unit test
  • Compare test execution report/summary
• Reproduce Component-level Tests
  • Rerun Component-level test
  • Compare test execution report/summary

[gonczy]

@FelixBole please asnwer this issue

[FelixBole]

Answering this thread which should also answer the sibling thread #4

This component is part of these components which are libraries and meant to be tested directly in the services that use it (in regards to the PTX projects). This means it is tested within the PDC and this also means that it has no purpose of being isolated in a docker container, being a js library to install.

In order to do that, the ptx-docker repository was provided giving a full ecosystem of the core prometheus-x services + some seeded participants + some seeded contracts and configuration to allow triggering exchanges based on DPCP functionality. DPCP functionality based exchanges are done through ecosystem contracts, and the ptx-docker allows to dig into and browse the configuration, participants, data shared, offerings used and contract from which these data exchanges are made.

In addition to that, and to provide more info. A wiki was created on this repository to allow for extensive documentation about the library itself & the README was augmented to provide demonstration examples of generic usage.

It also provides some testing scenarios for the library itself (https://github.com/Prometheus-X-association/data-processing-chain-protocol/wiki/Usage#test-it), although validation for PTX should happen in a controlled ecosystem like the ptx-docker.

It's hard at this stage to provide more information and tools than we are doing right now. I hope this is enough.

@Prometheus-X-association/bme-testing

[micskeiz]

Thanks for the information, @FelixBole !

A quick feedback:

• It would be helpful to add to the main README what tools are needed to build the library (e.g., tsup is only mentioned in the README inside src).
• I was able to run the unit tests.
• There was no mochawesome-report directory after running npm run test. Are any more commands needed?
• I agree that component and integration level tests are better specified in the context of ptx-docker or the dataspace-connector repositories. Those repositories are a good start for having a test framework for component and integration level tests for the core and other building blocks.
• However, those repo-s do not yet contain test plans and test definitions for integration level testing of the protocol.
  • What would be a good place in your opinion to have those tests?
  • For example, some important test scenarios to assess the status of the protocol could be the following (these should be detailed in the test plan to have some acceptance criteria)
    • Happy path, minimal chain is working and consumer receives nonPII data from a provider through an infrastructure service (this is done now).
    • Minimal chain with consent handling PII (there is a scenario for this, but how can we validate that it adheres to the consent?)
    • Testing consent handling where some of the data cannot be processed/imported.
    • Testing adhering to policies in the contract (e.g., some of the data processing is not allowed)
    • Testing error handling and robustness (wrong ids/URL in the chain definition, some node failure during executions...).
    • Service chain with more advanced elements (e.g., parallel executions).
    • Service chain with more than 3 nodes.
    • ...
  • These can be checked both in the sandbox and the ptx-docker environment.