feat: Add fileOnLargeOutput outputMode option (#339) by cxx5208 · Pull Request #343 · microsoft/playwright-cli

cxx5208 · 2026-04-04T20:24:45Z

Testing Validation Results for fileOnLargeOutput Feature

Test Summary

Test Case	Configuration	Expected	Result
Default threshold	`outputMode: "fileOnLargeOutput"` (no maxStdOutputSize)	Output to stdout	✓ Pass
Small threshold	`maxStdOutputSize: 100`	Output to file	✓ Pass
Zero threshold	`maxStdOutputSize: 0`	Output to file	✓ Pass
Large threshold	`maxStdOutputSize: 100000`	Output to stdout	✓ Pass
stdout mode	`outputMode: "stdout"`	Output to stdout	✓ Pass
file mode	`outputMode: "file"`	Output to stdout (existing behavior)	✓ Pass
No config	No config file	Default behavior	✓ Pass
snapshot command	`maxStdOutputSize: 100`	Output to file	✓ Pass
--help flag	`outputMode: "fileOnLargeOutput"`	No wrapping	✓ Pass

Manual Tests Performed

Small output (100 byte threshold): Output exceeded threshold, saved to file with message
Default threshold (1MB): Small output printed to stdout normally
Zero threshold: All output saved to file
Empty config (only outputMode set): Uses default 1MB threshold, works correctly

Edge Cases Tested

Config value 0 for maxStdOutputSize → correctly treated as threshold
Config without outputDir → defaults to temp directory
Config without maxStdOutputSize → defaults to 1MB

Verification

Feature implements issue requirements: Feature request: new outputMode - fileOnLargeOutput #339
All existing modes (stdout, file) continue to work unchanged
New fileOnLargeOutput mode correctly switches based on output size

Closes #339

Copilot

Pull request overview

Adds a new outputMode: "fileOnLargeOutput" configuration intended to route small CLI output to stdout and large output to a file, with a configurable maxStdOutputSize threshold.

Changes:

Document fileOnLargeOutput and maxStdOutputSize in the README config schema.
Implement fileOnLargeOutput behavior in playwright-cli.js by buffering output and writing to a file when above a threshold.
Add a TEST_RESULTS.md document describing manual test coverage and outcomes.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 11 comments.

File	Description
TEST_RESULTS.md	Adds a manual test report for the new output mode and threshold behavior.
README.md	Updates the config schema documentation to include `fileOnLargeOutput` and `maxStdOutputSize`.
playwright-cli.js	Adds logic to detect `fileOnLargeOutput`, capture output, and conditionally write to disk.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-04T20:28:51Z

playwright-cli.js

+function getOutputMode() {
+  try {
+    const configPath = path.join(process.cwd(), '.playwright', 'cli.config.json');
+    if (fs.existsSync(configPath)) {
+      const config = JSON.parse(fs.readFileSync(configPath, 'utf8'));
+      return config.outputMode || null;
+    }
+  } catch {}
+  return null;
+}
+
+function getMaxStdOutputSize() {
+  try {
+    const configPath = path.join(process.cwd(), '.playwright', 'cli.config.json');
+    if (fs.existsSync(configPath)) {
+      const config = JSON.parse(fs.readFileSync(configPath, 'utf8'));
+      if (config.maxStdOutputSize !== undefined) {
+        return config.maxStdOutputSize;
+      }
+    }
+  } catch {}
+  return DEFAULT_MAX_OUTPUT_SIZE;
+}
+
+function getOutputDir() {
+  try {
+    const configPath = path.join(process.cwd(), '.playwright', 'cli.config.json');
+    if (fs.existsSync(configPath)) {
+      const config = JSON.parse(fs.readFileSync(configPath, 'utf8'));
+      if (config.outputDir) {
+        return config.outputDir;
+      }
+    }
+  } catch {}
+  return path.join(os.tmpdir(), 'playwright-cli-output');
+}
+
+async function main() {
+  const outputMode = getOutputMode();
+
+  if (outputMode !== 'fileOnLargeOutput') {
+    await program({ embedderVersion: packageJson.version });
+    return;
+  }
+
+  const maxSize = getMaxStdOutputSize();
+  const outputDir = getOutputDir();


getOutputMode() (and related helpers) always reads .playwright/cli.config.json from process.cwd(), which means outputMode: "fileOnLargeOutput" will not work when the user supplies a different config via --config (or any other config resolution the underlying program() supports). Consider reusing the same config resolution logic as program() or parsing process.argv for the selected config path before deciding whether to wrap output.

Suggested change

function getOutputMode() {

try {

const configPath = path.join(process.cwd(), '.playwright', 'cli.config.json');

if (fs.existsSync(configPath)) {

const config = JSON.parse(fs.readFileSync(configPath, 'utf8'));

return config.outputMode || null;

}

} catch {}

return null;

}

function getMaxStdOutputSize() {

try {

const configPath = path.join(process.cwd(), '.playwright', 'cli.config.json');

if (fs.existsSync(configPath)) {

const config = JSON.parse(fs.readFileSync(configPath, 'utf8'));

if (config.maxStdOutputSize !== undefined) {

return config.maxStdOutputSize;

}

}

} catch {}

return DEFAULT_MAX_OUTPUT_SIZE;

}

function getOutputDir() {

try {

const configPath = path.join(process.cwd(), '.playwright', 'cli.config.json');

if (fs.existsSync(configPath)) {

const config = JSON.parse(fs.readFileSync(configPath, 'utf8'));

if (config.outputDir) {

return config.outputDir;

}

}

} catch {}

return path.join(os.tmpdir(), 'playwright-cli-output');

}

async function main() {

const outputMode = getOutputMode();

if (outputMode !== 'fileOnLargeOutput') {

await program({ embedderVersion: packageJson.version });

return;

}

const maxSize = getMaxStdOutputSize();

const outputDir = getOutputDir();

function getCliConfigPath() {

for (let i = 2; i < process.argv.length; ++i) {

const arg = process.argv[i];

if (arg === '--config' && i + 1 < process.argv.length) {

return path.resolve(process.cwd(), process.argv[i + 1]);

}

if (arg.startsWith('--config=')) {

return path.resolve(process.cwd(), arg.substring('--config='.length));

}

}

return path.join(process.cwd(), '.playwright', 'cli.config.json');

}

function readCliConfig() {

try {

const configPath = getCliConfigPath();

if (fs.existsSync(configPath))

return JSON.parse(fs.readFileSync(configPath, 'utf8'));

} catch {}

return null;

}

function getOutputMode(config) {

return config?.outputMode || null;

}

function getMaxStdOutputSize(config) {

if (config?.maxStdOutputSize !== undefined)

return config.maxStdOutputSize;

return DEFAULT_MAX_OUTPUT_SIZE;

}

function getOutputDir(config) {

if (config?.outputDir)

return config.outputDir;

return path.join(os.tmpdir(), 'playwright-cli-output');

}

async function main() {

const cliConfig = readCliConfig();

const outputMode = getOutputMode(cliConfig);

if (outputMode !== 'fileOnLargeOutput') {

await program({ embedderVersion: packageJson.version });

return;

}

const maxSize = getMaxStdOutputSize(cliConfig);

const outputDir = getOutputDir(cliConfig);

Copilot · 2026-04-04T20:28:51Z

playwright-cli.js

+  let outputBuffer = '';
+
+  console.log = (...args) => {
+    outputBuffer += args.map(a => String(a)).join(' ') + '\n';
+  };
+
+  try {
+    await program({ embedderVersion: packageJson.version });
+  } finally {
+    console.log = originalLog;
+  }
+
+  const outputSize = Buffer.byteLength(outputBuffer, 'utf8');
+
+  if (outputSize > maxSize) {
+    const timestamp = new Date().toISOString().replace(/[:.]/g, '-');
+    const fileName = `output-${timestamp}.txt`;
+    const filePath = path.join(outputDir, fileName);
+
+    fs.writeFileSync(filePath, outputBuffer, 'utf8');
+    originalLog(`\nOutput exceeded ${maxSize} bytes (${outputSize} bytes). Saved to: ${filePath}`);
+  } else {
+    originalLog(outputBuffer);


This mode buffers all console output into outputBuffer and only prints/writes it after program() completes. That changes CLI behavior substantially (no streaming output) and can cause large runs to consume a lot of memory (potentially OOM) before the threshold check runs. To avoid this, consider a streaming approach (e.g., tee to a temp file while counting bytes, or redirect to file once the threshold is exceeded).

Suggested change

let outputBuffer = '';

console.log = (...args) => {

outputBuffer += args.map(a => String(a)).join(' ') + '\n';

};

try {

await program({ embedderVersion: packageJson.version });

} finally {

console.log = originalLog;

}

const outputSize = Buffer.byteLength(outputBuffer, 'utf8');

if (outputSize > maxSize) {

const timestamp = new Date().toISOString().replace(/[:.]/g, '-');

const fileName = `output-${timestamp}.txt`;

const filePath = path.join(outputDir, fileName);

fs.writeFileSync(filePath, outputBuffer, 'utf8');

originalLog(`\nOutput exceeded ${maxSize} bytes (${outputSize} bytes). Saved to: ${filePath}`);

} else {

originalLog(outputBuffer);

const tempFilePath = path.join(outputDir, `output-${process.pid}-${Date.now()}.tmp`);

const tempFd = fs.openSync(tempFilePath, 'w');

let outputSize = 0;

console.log = (...args) => {

const line = args.map(a => String(a)).join(' ') + '\n';

outputSize += Buffer.byteLength(line, 'utf8');

fs.writeSync(tempFd, line, null, 'utf8');

};

try {

await program({ embedderVersion: packageJson.version });

} finally {

console.log = originalLog;

fs.closeSync(tempFd);

}

if (outputSize > maxSize) {

const timestamp = new Date().toISOString().replace(/[:.]/g, '-');

const fileName = `output-${timestamp}.txt`;

const filePath = path.join(outputDir, fileName);

fs.renameSync(tempFilePath, filePath);

originalLog(`\nOutput exceeded ${maxSize} bytes (${outputSize} bytes). Saved to: ${filePath}`);

} else {

const output = fs.readFileSync(tempFilePath, 'utf8');

fs.unlinkSync(tempFilePath);

originalLog(output);

Copilot · 2026-04-04T20:28:51Z

playwright-cli.js

+  const originalLog = console.log;
+  let outputBuffer = '';
+
+  console.log = (...args) => {
+    outputBuffer += args.map(a => String(a)).join(' ') + '\n';
+  };


Only console.log is wrapped here, so output written via process.stdout.write, console.error, or other logging APIs will bypass the size accounting and may still flood the terminal even in fileOnLargeOutput mode. If the intent is to gate stdout/stderr output size, consider wrapping process.stdout.write/process.stderr.write (and/or console.error) instead of only console.log.

Copilot · 2026-04-04T20:28:52Z

playwright-cli.js

+      const config = JSON.parse(fs.readFileSync(configPath, 'utf8'));
+      if (config.maxStdOutputSize !== undefined) {
+        return config.maxStdOutputSize;
+      }
+    }
+  } catch {}
+  return DEFAULT_MAX_OUTPUT_SIZE;


maxStdOutputSize is returned without validation. If the config contains null, a string, NaN, or a negative value, the comparison outputSize > maxSize will behave unexpectedly (e.g., null coerces to 0). Consider validating/coercing to a finite non-negative number and falling back to the default when invalid.

Copilot · 2026-04-04T20:28:52Z

playwright-cli.js

+function getOutputMode() {
+  try {
+    const configPath = path.join(process.cwd(), '.playwright', 'cli.config.json');
+    if (fs.existsSync(configPath)) {
+      const config = JSON.parse(fs.readFileSync(configPath, 'utf8'));
+      return config.outputMode || null;
+    }
+  } catch {}
+  return null;
+}
+
+function getMaxStdOutputSize() {
+  try {
+    const configPath = path.join(process.cwd(), '.playwright', 'cli.config.json');
+    if (fs.existsSync(configPath)) {
+      const config = JSON.parse(fs.readFileSync(configPath, 'utf8'));
+      if (config.maxStdOutputSize !== undefined) {
+        return config.maxStdOutputSize;
+      }
+    }
+  } catch {}
+  return DEFAULT_MAX_OUTPUT_SIZE;
+}
+
+function getOutputDir() {
+  try {
+    const configPath = path.join(process.cwd(), '.playwright', 'cli.config.json');
+    if (fs.existsSync(configPath)) {
+      const config = JSON.parse(fs.readFileSync(configPath, 'utf8'));
+      if (config.outputDir) {
+        return config.outputDir;
+      }
+    }
+  } catch {}


The empty catch {} blocks in the config readers will silently ignore JSON parse errors or filesystem errors, which makes misconfiguration hard to diagnose (the CLI will just fall back to defaults). Consider emitting a short warning (or at least handling JSON parse errors explicitly) when the config exists but cannot be parsed.

Suggested change

function getOutputMode() {

try {

const configPath = path.join(process.cwd(), '.playwright', 'cli.config.json');

if (fs.existsSync(configPath)) {

const config = JSON.parse(fs.readFileSync(configPath, 'utf8'));

return config.outputMode || null;

}

} catch {}

return null;

}

function getMaxStdOutputSize() {

try {

const configPath = path.join(process.cwd(), '.playwright', 'cli.config.json');

if (fs.existsSync(configPath)) {

const config = JSON.parse(fs.readFileSync(configPath, 'utf8'));

if (config.maxStdOutputSize !== undefined) {

return config.maxStdOutputSize;

}

}

} catch {}

return DEFAULT_MAX_OUTPUT_SIZE;

}

function getOutputDir() {

try {

const configPath = path.join(process.cwd(), '.playwright', 'cli.config.json');

if (fs.existsSync(configPath)) {

const config = JSON.parse(fs.readFileSync(configPath, 'utf8'));

if (config.outputDir) {

return config.outputDir;

}

}

} catch {}

function readCliConfig() {

const configPath = path.join(process.cwd(), '.playwright', 'cli.config.json');

if (!fs.existsSync(configPath)) {

return null;

}

try {

return JSON.parse(fs.readFileSync(configPath, 'utf8'));

} catch (e) {

console.warn(`Warning: Failed to read Playwright CLI config at ${configPath}: ${e.message}`);

return null;

}

}

function getOutputMode() {

const config = readCliConfig();

return config ? config.outputMode || null : null;

}

function getMaxStdOutputSize() {

const config = readCliConfig();

if (config && config.maxStdOutputSize !== undefined) {

return config.maxStdOutputSize;

}

return DEFAULT_MAX_OUTPUT_SIZE;

}

function getOutputDir() {

const config = readCliConfig();

if (config && config.outputDir) {

return config.outputDir;

}

Copilot · 2026-04-04T20:28:52Z

playwright-cli.js

+}
+
+main().catch(e => {
+  console.error(e.message);


main().catch(e => console.error(e.message)) can hide useful debugging details (no stack trace) and will print undefined if a non-Error value is thrown. Consider logging the full error (or e?.stack ?? e) to keep diagnostics comparable to the previous direct program() execution.

Suggested change

console.error(e.message);

console.error(e?.stack ?? e);

Copilot · 2026-04-04T20:28:52Z

playwright-cli.js

+async function main() {
+  const outputMode = getOutputMode();
+
+  if (outputMode !== 'fileOnLargeOutput') {
+    await program({ embedderVersion: packageJson.version });
+    return;
+  }


There are existing Playwright integration tests for the CLI, but none cover the new outputMode: "fileOnLargeOutput" / maxStdOutputSize behavior (e.g., creating .playwright/cli.config.json, exercising a command that prints output, and asserting stdout vs file output). Adding an integration spec for this new mode would prevent regressions around thresholding and outputDir handling.

Copilot · 2026-04-04T20:28:53Z

TEST_RESULTS.md

+| Zero threshold | `maxStdOutputSize: 0` | Output to file | ✓ Pass |
+| Large threshold | `maxStdOutputSize: 100000` | Output to stdout | ✓ Pass |
+| stdout mode | `outputMode: "stdout"` | Output to stdout | ✓ Pass |
+| file mode | `outputMode: "file"` | Output to stdout (existing behavior) | ✓ Pass |


This report says outputMode: "file" results in "Output to stdout (existing behavior)", but the README schema comment in this PR states that "file" saves all output to files in outputDir. Please align the documentation/test report so users aren't left with conflicting behavior descriptions.

Suggested change

| file mode | `outputMode: "file"` | Output to stdout (existing behavior) | ✓ Pass |

| file mode | `outputMode: "file"` | Output to file | ✓ Pass |

Copilot · 2026-04-04T20:28:53Z

README.md

+   * - "stdout": Output to console immediately
+   * - "file": Save all output to files in outputDir
+   * - "fileOnLargeOutput": Output to stdout if small, otherwise save to file (controlled by maxStdOutputSize)


The docs say fileOnLargeOutput will "Output to stdout if small" / "Output to console immediately", but the current implementation buffers output and only prints it after program() finishes. Please either update the implementation to stream output as intended or clarify in the docs that output may be delayed while the CLI decides whether to write to a file.

Suggested change

* - "stdout": Output to console immediately

* - "file": Save all output to files in outputDir

* - "fileOnLargeOutput": Output to stdout if small, otherwise save to file (controlled by maxStdOutputSize)

* Output may be buffered before it is written, especially when the CLI needs to decide whether to keep output on stdout or save it to a file.

* - "stdout": Write output to stdout. Depending on buffering, output may be emitted after the session completes rather than immediately.

* - "file": Save all output to files in outputDir

* - "fileOnLargeOutput": Buffer output while determining its size; write to stdout if it stays within maxStdOutputSize, otherwise save it to file

Copilot · 2026-04-04T20:28:53Z

playwright-cli.js

+      }
+    }
+  } catch {}
+  return path.join(os.tmpdir(), 'playwright-cli-output');


For fileOnLargeOutput, getOutputDir() falls back to a fixed path under the OS temp dir (<tmp>/playwright-cli-output). This can accumulate files across runs and differs from whatever default outputDir behavior program() may already implement. Consider using a per-run temp directory (or delegating outputDir defaulting to the underlying CLI config logic) to avoid unbounded growth and keep behavior consistent across output modes.

Suggested change

return path.join(os.tmpdir(), 'playwright-cli-output');

return fs.mkdtempSync(path.join(os.tmpdir(), 'playwright-cli-output-'));

cxx5208 added 3 commits April 4, 2026 13:23

Add fileOnLargeOutput outputMode and maxStdOutputSize config options

1fdb21c

Closes #339

Implement fileOnLargeOutput logic for CLI output handling

d856a62

Closes #339

Add testing validation results for fileOnLargeOutput feature

58a53b7

Closes #339

Copilot AI review requested due to automatic review settings April 4, 2026 20:24

Copilot started reviewing on behalf of cxx5208 April 4, 2026 20:25 View session

cxx5208 closed this by deleting the head repository Apr 4, 2026

Copilot AI reviewed Apr 4, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Add fileOnLargeOutput outputMode option (#339)#343

feat: Add fileOnLargeOutput outputMode option (#339)#343
cxx5208 wants to merge 3 commits intomicrosoft:mainfrom
cxx5208:feature/file-on-large-output

cxx5208 commented Apr 4, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 4, 2026

Uh oh!

Copilot AI Apr 4, 2026

Uh oh!

Copilot AI Apr 4, 2026

Uh oh!

Copilot AI Apr 4, 2026

Uh oh!

Copilot AI Apr 4, 2026

Uh oh!

Copilot AI Apr 4, 2026

Uh oh!

Copilot AI Apr 4, 2026

Uh oh!

Copilot AI Apr 4, 2026

Uh oh!

Copilot AI Apr 4, 2026

Uh oh!

Copilot AI Apr 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	\| file mode \| `outputMode: "file"` \| Output to stdout (existing behavior) \| ✓ Pass \|
	\| file mode \| `outputMode: "file"` \| Output to file \| ✓ Pass \|

	return path.join(os.tmpdir(), 'playwright-cli-output');
	return fs.mkdtempSync(path.join(os.tmpdir(), 'playwright-cli-output-'));

Conversation

cxx5208 commented Apr 4, 2026

Testing Validation Results for fileOnLargeOutput Feature

Test Summary

Manual Tests Performed

Edge Cases Tested

Verification

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 4, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants