Add cloud provider abstraction for test result reporting

.gitignore

@@ -49,3 +49,4 @@ scripts/
 docs_local/
 research/
 examples/
+PLAN-cloud-provider.md

README.md

@@ -140,6 +140,7 @@ maestro-runner --driver appium --appium-url <HUB_URL> --caps caps.json test flow
 ```
 
 - **[TestingBot](docs/cloud-providers/testingbot.md)** — Setup guide for running on TestingBot's real device cloud
+- **[Sauce Labs](docs/cloud-providers/saucelabs.md)** — Setup guide for running on Sauce Labs Appium cloud
 
 ## Contributing
 

docs/cloud-providers/README.md

@@ -0,0 +1,130 @@
+# Cloud Provider Integration
+
+maestro-runner automatically detects cloud Appium providers from the `--appium-url` and reports test pass/fail after the run completes.
+
+## Supported providers
+
+- [TestingBot](testingbot.md)
+- [Sauce Labs](saucelabs.md)
+
+## How it works
+
+1. **Detect** — after `--appium-url` is parsed, each registered provider checks if the URL matches (e.g., contains "saucelabs")
+2. **Extract metadata** — after the Appium session is created, the provider reads session capabilities and stores provider-specific data (job IDs, session type, etc.) in a `map[string]string`
+3. **Report result** — after all flows and reports complete, the provider receives the full test result and reports pass/fail to the cloud API
+
+No extra flags needed — detection and reporting happen automatically.
+
+## Adding a new provider
+
+All provider code lives in `pkg/cloud/`. To add a new provider:
+
+### 1. Create the file
+
+Copy `pkg/cloud/example_provider.go` to `pkg/cloud/<yourprovider>.go`.
+
+### 2. Implement the Provider interface
+
+```go
+package cloud
+
+type Provider interface {
+    // Name returns the human-readable provider name.
+    Name() string
+
+    // ExtractMeta is called once after the Appium session is created.
+    // Read what you need from sessionID and caps, write to meta.
+    ExtractMeta(sessionID string, caps map[string]interface{}, meta map[string]string)
+
+    // ReportResult is called once after all flows and reports complete.
+    // Use meta for provider-specific data, result for test outcome.
+    ReportResult(appiumURL string, meta map[string]string, result *TestResult) error
+}
+```
+
+### 3. Register via init()
+
+The factory function checks the URL and returns a provider or `nil`:
+
+```go
+func init() {
+    Register(func(appiumURL string) Provider {
+        if !strings.Contains(strings.ToLower(appiumURL), "yourprovider") {
+            return nil
+        }
+        return &yourProvider{}
+    })
+}
+```
+
+### 4. Example skeleton
+
+A complete skeleton is available at `pkg/cloud/example_provider.go`. Copy it, rename, and implement the TODOs.
+
+### 5. Add tests
+
+Create `pkg/cloud/<yourprovider>_test.go` with tests for:
+- URL detection (matches your provider, rejects others)
+- ExtractMeta (correct meta keys)
+- ReportResult (use `httptest.NewServer` to verify endpoint, auth, body)
+
+### 6. Add documentation
+
+Create `docs/cloud-providers/<yourprovider>.md` with:
+- Run command example
+- Example capabilities JSON
+- Any provider-specific notes
+
+Add a link in the main `README.md` under **Cloud Providers** and in this file under **Supported providers**.
+
+## TestResult fields
+
+`ReportResult` receives the full test outcome. Use what your provider's API supports:
+
+```go
+type TestResult struct {
+    Passed      bool           // overall pass/fail
+    Total       int            // total flow count
+    PassedCount int            // flows that passed
+    FailedCount int            // flows that failed
+    Duration    int64          // total duration in milliseconds
+    OutputDir   string         // path to log, reports, screenshots
+    Flows       []FlowResult   // per-flow details
+}
+
+type FlowResult struct {
+    Name     string  // flow name
+    File     string  // source YAML file path
+    Passed   bool    // this flow passed
+    Duration int64   // milliseconds
+    Error    string  // error message (empty if passed)
+}
+```
+
+- Most providers only need `result.Passed` for a simple pass/fail update
+- `result.Flows` is available for providers that support per-test annotations
+- `result.OutputDir` contains `maestro-runner.log`, `report.html`, `report.json`, `junit-report.xml`, and screenshots — providers can upload these if their API supports artifacts
+
+## Meta map
+
+The `meta map[string]string` is owned by the caller and passed through `ExtractMeta` → `ReportResult`. Each provider writes its own keys. Examples:
+
+| Provider | Keys | Description |
+|----------|------|-------------|
+| Sauce Labs | `jobID`, `type` | `type` is "rdc" (real device) or "vms" (emulator/simulator) |
+| (new provider) | `jobID` | Typically the WebDriver session ID |
+
+No naming conflicts since only one provider is active per session.
+
+## Credentials
+
+Each provider handles credentials internally in `ReportResult`. The common pattern is:
+
+1. Extract from `--appium-url` userinfo (e.g., `https://USER:KEY@hub.example.com`)
+2. Fall back to provider-specific environment variables
+
+This keeps credential logic out of the shared interface.
+
+## Error handling
+
+`ReportResult` errors are logged as warnings — they never fail the test run. Local test results and reports are always generated regardless of cloud reporting success.

docs/cloud-providers/saucelabs.md

@@ -0,0 +1,88 @@
+# Sauce Labs (Appium)
+
+Use Appium driver mode with a Sauce Labs URL and provider capabilities.
+
+## Run command
+
+```bash
+maestro-runner \
+  --driver appium \
+  --appium-url "https://$SAUCE_USERNAME:$SAUCE_ACCESS_KEY@ondemand.us-west-1.saucelabs.com:443/wd/hub" \
+  --caps provider-caps.json \
+  test flows/
+```
+
+- Default example uses `us-west-1`. Replace the Sauce Labs endpoints with your region as needed (for example `eu-central-1`, `us-east-4`).
+- The Appium URL should include Sauce credentials (`$SAUCE_USERNAME` and `$SAUCE_ACCESS_KEY`) or be provided via environment variables.
+
+## Example capabilities
+
+Example `provider-caps.json` for Android real device:
+
+```json
+{
+  "platformName": "Android",
+  "appium:automationName": "UiAutomator2",
+  "appium:deviceName": "Samsung.*",
+  "appium:platformVersion": "^1[5-6].*",
+  "appium:app": "storage:filename=mda-2.2.0-25.apk",
+  "sauce:options": {
+    "build": "Maestro Android Run",
+    "appiumVersion": "latest"
+  }
+}
+```
+
+Example `provider-caps.json` for iOS real device:
+
+```json
+{
+  "platformName": "iOS",
+  "appium:automationName": "XCUITest",
+  "appium:deviceName": "iPhone.*",
+  "appium:platformVersion": "^(18|26).*",
+  "appium:app": "storage:filename=SauceLabs-Demo-App.ipa",
+  "sauce:options": {
+    "build": "Maestro iOS Run",
+    "appiumVersion": "latest",
+    "resigningEnabled": true
+  }
+}
+```
+
+Example `provider-caps.json` for Android emulator:
+
+```json
+{
+  "platformName": "Android",
+  "appium:automationName": "UiAutomator2",
+  "appium:deviceName": "Google Pixel 9 Emulator",
+  "appium:platformVersion": "16.0",
+  "appium:app": "storage:filename=mda-2.2.0-25.apk",
+  "sauce:options": {
+    "build": "Maestro Android Emulator Run",
+    "appiumVersion": "2.11.0"
+  }
+}
+```
+
+Example `provider-caps.json` for iOS simulator:
+
+```json
+{
+  "platformName": "iOS",
+  "appium:automationName": "XCUITest",
+  "appium:deviceName": "iPhone Simulator",
+  "appium:platformVersion": "17.0",
+  "appium:app": "storage:filename=SauceLabs-Demo-App.Simulator.zip",
+  "sauce:options": {
+    "build": "Maestro iOS Simulator Run",
+    "appiumVersion": "2.11.3"
+  }
+}
+```
+
+## References
+
+- [Run Maestro Flows on Any Cloud Provider](https://devicelab.dev/blog/run-maestro-flows-any-cloud)
+- [Sauce Labs: Mobile Appium capabilities](https://docs.saucelabs.com/dev/test-configuration-options/#mobile-appium-capabilities)

pkg/cli/test.go

@@ -15,6 +15,7 @@ import (
 	"syscall"
 	"time"
 
+	"github.com/devicelab-dev/maestro-runner/pkg/cloud"
 	"github.com/devicelab-dev/maestro-runner/pkg/config"
 	"github.com/devicelab-dev/maestro-runner/pkg/core"
 	"github.com/devicelab-dev/maestro-runner/pkg/device"
@@ -472,6 +473,10 @@ type RunConfig struct {
 
 	// Flutter
 	NoFlutterFallback bool // Disable automatic Flutter VM Service fallback
+
+	// Cloud provider (detected from AppiumURL, nil if not a cloud provider)
+	CloudProvider cloud.Provider
+	CloudMeta     map[string]string
 }
 
 func hyperlink(url, text string) string {
@@ -844,6 +849,32 @@ func executeTest(cfg *RunConfig) error {
 		fmt.Printf("  %s⚠%s Warning: failed to generate Allure report: %v\n", color(colorYellow), color(colorReset), err)
 	}
 
+	// Report result to cloud provider (if detected)
+	if cfg.CloudProvider != nil {
+		cloudResult := &cloud.TestResult{
+			Passed:      result.Status == report.StatusPassed,
+			Total:       result.TotalFlows,
+			PassedCount: result.PassedFlows,
+			FailedCount: result.FailedFlows,
+			Duration:    result.Duration,
+			OutputDir:   cfg.OutputDir,
+		}
+		for _, f := range result.FlowResults {
+			cloudResult.Flows = append(cloudResult.Flows, cloud.FlowResult{
+				Name:     f.Name,
+				File:     f.SourceFile,
+				Passed:   f.Status == report.StatusPassed,
+				Duration: f.Duration,
+				Error:    f.Error,
+			})
+		}
+		if err := cfg.CloudProvider.ReportResult(cfg.AppiumURL, cfg.CloudMeta, cloudResult); err != nil {
+			logger.Warn("%s result reporting failed: %v", cfg.CloudProvider.Name(), err)
+		} else {
+			logger.Info("%s job updated: passed=%v", cfg.CloudProvider.Name(), cloudResult.Passed)
+		}
+	}
+
 	// Display reports section as a directory tree
 	fmt.Printf("  %sReports:%s %s\n", color(colorBold), color(colorReset), cfg.OutputDir)
 	fmt.Printf("    ├── report.json\n")
@@ -1647,6 +1678,15 @@ func createAppiumDriver(cfg *RunConfig) (core.Driver, func(), error) {
 		return nil, nil, fmt.Errorf("create Appium session: %w", err)
 	}
 	logger.Info("Appium session created successfully: %s", driver.GetPlatformInfo().DeviceID)
+
+	// Detect cloud provider and extract metadata
+	if p := cloud.Detect(cfg.AppiumURL); p != nil {
+		cfg.CloudProvider = p
+		cfg.CloudMeta = make(map[string]string)
+		p.ExtractMeta(driver.SessionID(), driver.SessionCaps(), cfg.CloudMeta)
+		logger.Info("Cloud provider detected: %s", p.Name())
+	}
+
 	printSetupSuccess("Appium session created")
 
 	// Cleanup function

pkg/cloud/example_provider.go

@@ -0,0 +1,62 @@
+// This file is a skeleton for adding a new cloud provider.
+// Copy this file, rename it, and implement the TODOs.
+//
+// Steps:
+//  1. Copy to <provider>.go (e.g., browserstack.go)
+//  2. Replace "example" with your provider name
+//  3. Implement the URL match in the factory
+//  4. Implement ExtractMeta and ReportResult
+//  5. Add tests in <provider>_test.go
+//  6. Add docs in docs/cloud-providers/<provider>.md
+
+package cloud
+
+/*
+import (
+	"fmt"
+	"strings"
+)
+
+func init() {
+	Register(newExampleProvider)
+}
+
+func newExampleProvider(appiumURL string) Provider {
+	// TODO: match your provider's Appium hub URL
+	if !strings.Contains(strings.ToLower(appiumURL), "example") {
+		return nil
+	}
+	return &exampleProvider{}
+}
+
+type exampleProvider struct{}
+
+func (p *exampleProvider) Name() string { return "Example" }
+
+func (p *exampleProvider) ExtractMeta(sessionID string, caps map[string]interface{}, meta map[string]string) {
+	// TODO: extract provider-specific data from session
+	// Most providers just need the session ID as the job ID:
+	meta["jobID"] = sessionID
+}
+
+func (p *exampleProvider) ReportResult(appiumURL string, meta map[string]string, result *TestResult) error {
+	jobID := meta["jobID"]
+	if jobID == "" {
+		return fmt.Errorf("no job ID")
+	}
+
+	// TODO: extract credentials from appiumURL userinfo or env vars
+	// TODO: PUT/PATCH pass/fail to your provider's REST API
+	//
+	// Available data:
+	//   result.Passed      - overall pass/fail
+	//   result.Total       - total flow count
+	//   result.PassedCount - flows passed
+	//   result.FailedCount - flows failed
+	//   result.Duration    - total ms
+	//   result.OutputDir   - path to log, reports, screenshots
+	//   result.Flows       - per-flow name, file, pass/fail, duration, error
+
+	return nil
+}
+*/