Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 65 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ Add the coverage extension to your `phpunit.xml`:
| `strip_prefixes` | No | `[]` | Comma-separated prefixes to strip from request paths (e.g., `/api`) |
| `specs` | No | `front` | Comma-separated spec names for coverage tracking |
| `output_file` | No | — | File path to write Markdown coverage report (relative paths resolve from `getcwd()`) |
| `console_output` | No | `default` | Console output mode: `default`, `all`, or `uncovered_only` (overridden by `OPENAPI_CONSOLE_OUTPUT` env var) |

*Not required if you call `OpenApiSpecLoader::configure()` manually.

Expand Down Expand Up @@ -163,7 +164,11 @@ For Laravel, set the `max_errors` key in `config/openapi-contract-testing.php`.

## Coverage Report

After running tests, the PHPUnit extension prints a coverage report:
After running tests, the PHPUnit extension prints a coverage report. The output format is controlled by the `console_output` parameter (or `OPENAPI_CONSOLE_OUTPUT` environment variable).

### `default` mode (default)

Shows covered endpoints individually and uncovered as a count:

```
OpenAPI Contract Test Coverage
Expand All @@ -179,6 +184,56 @@ Covered:
Uncovered: 41 endpoints
```

### `all` mode

Shows both covered and uncovered endpoints individually:

```
OpenAPI Contract Test Coverage
==================================================

[front] 12/45 endpoints (26.7%)
--------------------------------------------------
Covered:
✓ GET /v1/pets
✓ POST /v1/pets
✓ GET /v1/pets/{petId}
✓ DELETE /v1/pets/{petId}
Uncovered:
✗ PUT /v1/pets/{petId}
✗ GET /v1/owners
...
```

### `uncovered_only` mode

Shows uncovered endpoints individually and covered as a count — useful for large APIs where you want to focus on missing coverage:

```
OpenAPI Contract Test Coverage
==================================================

[front] 12/45 endpoints (26.7%)
--------------------------------------------------
Covered: 12 endpoints
Uncovered:
✗ PUT /v1/pets/{petId}
✗ GET /v1/owners
...
```

You can set the mode via `phpunit.xml`:

```xml
<parameter name="console_output" value="uncovered_only"/>
```

Or via environment variable (takes priority over `phpunit.xml`):

```bash
OPENAPI_CONSOLE_OUTPUT=uncovered_only vendor/bin/phpunit
```

## CI Integration

### GitHub Actions Step Summary
Expand All @@ -201,6 +256,15 @@ Use the `output_file` parameter to write a Markdown report to a file. This is us
</extensions>
```

You can also use the `OPENAPI_CONSOLE_OUTPUT` environment variable in CI to show uncovered endpoints in the job log:

```yaml
- name: Run tests (show uncovered endpoints)
run: vendor/bin/phpunit
env:
OPENAPI_CONSOLE_OUTPUT: uncovered_only
```

Example GitHub Actions workflow step to post the report as a PR comment:

```yaml
Expand Down
94 changes: 94 additions & 0 deletions src/PHPUnit/ConsoleCoverageRenderer.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
<?php

declare(strict_types=1);

namespace Studio\OpenApiContractTesting\PHPUnit;

use function count;
use function round;
use function str_repeat;

/**
* @phpstan-type CoverageResult array{covered: string[], uncovered: string[], total: int, coveredCount: int}
*/
final class ConsoleCoverageRenderer
{
/**
* @param array<string, CoverageResult> $results
*/
public static function render(array $results, ConsoleOutput $consoleOutput = ConsoleOutput::DEFAULT): string
{
if ($results === []) {
return '';
}

$output = "\n\n";
$output .= "OpenAPI Contract Test Coverage\n";
$output .= str_repeat('=', 50) . "\n";

foreach ($results as $spec => $result) {
$percentage = self::percentage($result['coveredCount'], $result['total']);

$output .= "\n[{$spec}] {$result['coveredCount']}/{$result['total']} endpoints ({$percentage}%)\n";
$output .= str_repeat('-', 50) . "\n";

$output .= self::renderCovered($result, $consoleOutput);
$output .= self::renderUncovered($result, $consoleOutput);
}

$output .= "\n";

return $output;
}

/**
* @param CoverageResult $result
*/
private static function renderCovered(array $result, ConsoleOutput $consoleOutput): string
{
if ($result['covered'] === []) {
return '';
}

if ($consoleOutput === ConsoleOutput::UNCOVERED_ONLY) {
return "Covered: {$result['coveredCount']} endpoints\n";
}

$output = "Covered:\n";

foreach ($result['covered'] as $endpoint) {
$output .= " ✓ {$endpoint}\n";
}

return $output;
}

/**
* @param CoverageResult $result
*/
private static function renderUncovered(array $result, ConsoleOutput $consoleOutput): string
{
if ($result['uncovered'] === []) {
return '';
}

$uncoveredCount = count($result['uncovered']);

if ($consoleOutput === ConsoleOutput::DEFAULT) {
return "Uncovered: {$uncoveredCount} endpoints\n";
}

$output = "Uncovered:\n";

foreach ($result['uncovered'] as $endpoint) {
$output .= " ✗ {$endpoint}\n";
}

return $output;
}

private static function percentage(int $covered, int $total): float|int
{
return $total > 0 ? round($covered / $total * 100, 1) : 0;
}
}
51 changes: 51 additions & 0 deletions src/PHPUnit/ConsoleOutput.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
<?php

declare(strict_types=1);

namespace Studio\OpenApiContractTesting\PHPUnit;

use const STDERR;

use function fwrite;
use function getenv;
use function mb_strtolower;
use function trim;

enum ConsoleOutput: string
{
case DEFAULT = 'default';
case ALL = 'all';
case UNCOVERED_ONLY = 'uncovered_only';

/**
* Resolve the console output mode from environment variable and/or phpunit.xml parameter.
*
* Priority: environment variable > parameter > DEFAULT.
*/
public static function resolve(?string $parameterValue): self
{
$envValue = getenv('OPENAPI_CONSOLE_OUTPUT');

if ($envValue !== false && trim($envValue) !== '') {
$resolved = self::tryFrom(mb_strtolower(trim($envValue)));

if ($resolved === null) {
fwrite(STDERR, "[OpenAPI Coverage] WARNING: Invalid OPENAPI_CONSOLE_OUTPUT value '{$envValue}'. Valid values: default, all, uncovered_only. Falling back to 'default'.\n");
}

return $resolved ?? self::DEFAULT;
}

if ($parameterValue !== null && trim($parameterValue) !== '') {
$resolved = self::tryFrom(mb_strtolower(trim($parameterValue)));

if ($resolved === null) {
fwrite(STDERR, "[OpenAPI Coverage] WARNING: Invalid console_output parameter '{$parameterValue}'. Valid values: default, all, uncovered_only. Falling back to 'default'.\n");
}

return $resolved ?? self::DEFAULT;
}

return self::DEFAULT;
}
}
49 changes: 7 additions & 42 deletions src/PHPUnit/OpenApiCoverageExtension.php
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,6 @@
use function fwrite;
use function getcwd;
use function getenv;
use function round;
use function str_repeat;
use function str_starts_with;

final class OpenApiCoverageExtension implements Extension
Expand Down Expand Up @@ -59,15 +57,20 @@ public function bootstrap(Configuration $configuration, Facade $facade, Paramete
}
}

$consoleOutput = ConsoleOutput::resolve(
$parameters->has('console_output') ? $parameters->get('console_output') : null,
);

$githubSummaryPath = getenv('GITHUB_STEP_SUMMARY') ?: null;

$facade->registerSubscriber(new class ($specs, $outputFile, $githubSummaryPath) implements ExecutionFinishedSubscriber {
$facade->registerSubscriber(new class ($specs, $outputFile, $consoleOutput, $githubSummaryPath) implements ExecutionFinishedSubscriber {
/**
* @param string[] $specs
*/
public function __construct(
private readonly array $specs,
private readonly ?string $outputFile,
private readonly ConsoleOutput $consoleOutput,
private readonly ?string $githubSummaryPath,
) {}

Expand All @@ -80,7 +83,7 @@ public function notify(ExecutionFinished $event): void
return;
}

$this->printReport($results);
echo ConsoleCoverageRenderer::render($results, $this->consoleOutput);

if ($this->outputFile !== null || $this->githubSummaryPath !== null) {
$this->writeMarkdownReport($results);
Expand Down Expand Up @@ -122,39 +125,6 @@ private function computeAllResults(): array
return $results;
}

/**
* @param array<string, array{covered: string[], uncovered: string[], total: int, coveredCount: int}> $results
*/
private function printReport(array $results): void
{
echo "\n\n";
echo "OpenAPI Contract Test Coverage\n";
echo str_repeat('=', 50) . "\n";

foreach ($results as $spec => $result) {
$percentage = self::percentage($result['coveredCount'], $result['total']);

echo "\n[{$spec}] {$result['coveredCount']}/{$result['total']} endpoints ({$percentage}%)\n";
echo str_repeat('-', 50) . "\n";

if ($result['covered'] !== []) {
echo "Covered:\n";

foreach ($result['covered'] as $endpoint) {
echo " ✓ {$endpoint}\n";
}
}

$uncoveredCount = $result['total'] - $result['coveredCount'];

if ($uncoveredCount > 0) {
echo "Uncovered: {$uncoveredCount} endpoints\n";
}
}

echo "\n";
}

/**
* @param array<string, array{covered: string[], uncovered: string[], total: int, coveredCount: int}> $results
*/
Expand All @@ -178,11 +148,6 @@ private function writeMarkdownReport(array $results): void
}
}
}

private static function percentage(int $covered, int $total): float|int
{
return $total > 0 ? round($covered / $total * 100, 1) : 0;
}
});
}
}
Loading