Skip to content

feat: FM-949 Added e2e playwright automation to circle ci/cd pipeline #1992

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ashwinnair-chimple wants to merge 2 commits into
base: develop
Choose a base branch
from
Open

feat: FM-949 Added e2e playwright automation to circle ci/cd pipeline #1992

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
511e0ed
feat: FM-949 Added e2e playwright automation to circle ci/cd pipeline
ashwinnair-chimple Jul 2, 2026
5a2ece9
fix: handling artifact storage using s3
ashwinnair-chimple Jul 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,30 @@ jobs:
distribution-id: 'E2QD6IZIYBGTJK'
paths: '"/*"'

e2e-tests:
docker:
- image: mcr.microsoft.com/playwright:v1.60.0-jammy
steps:
- checkout
- node/install-packages
- run:
name: Run Playwright E2E tests
command: npm run test:e2e
environment:
CI: 'true'
- store_test_results:
path: test-results
- aws-cli/setup:
aws_access_key_id: $AWS_ACCESS_KEY
region: $AWS_DEFAULT_REGION
aws_secret_access_key: $AWS_SECRET_ACCESS_KEY
- run:
name: Upload Playwright report to S3
when: always
command: |
aws s3 sync playwright-report \
"s3://globallit-aws-s3-static-webapp-test-us-east-2/e2e-reports/${CIRCLE_BRANCH}/${CIRCLE_BUILD_NUM}/"
Comment on lines +109 to +118

@coderabbitai coderabbitai Bot Jul 3, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== .circleci/config.yml: relevant lines =="
nl -ba .circleci/config.yml | sed -n '90,145p'

echo
echo "== orb references =="
rg -n "circleci/aws-cli|aws-cli/setup|aws-context|e2e-tests|when: always|npm run test:e2e" .circleci/config.yml

echo
echo "== top of config for orb version pin =="
nl -ba .circleci/config.yml | sed -n '1,40p'

Repository: curiouslearning/FeedTheMonsterJS

Length of output: 253

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== .circleci/config.yml: relevant lines =="
awk 'NR>=90 && NR<=145 { printf "%5d  %s\n", NR, $0 }' .circleci/config.yml

echo
echo "== orb references =="
rg -n "circleci/aws-cli|aws-cli/setup|aws-context|e2e-tests|when: always|npm run test:e2e" .circleci/config.yml

echo
echo "== top of config for orb version pin =="
awk 'NR>=1 && NR<=40 { printf "%5d  %s\n", NR, $0 }' .circleci/config.yml

Repository: curiouslearning/FeedTheMonsterJS

Length of output: 4124

🌐 Web query:

CircleCI aws-cli orb setup command when parameter circleci/aws-cli@5.2.0 docs

💡 Result:

The aws-cli/setup command in the circleci/aws-cli orb (version 5.2.0 and later) is used to install the AWS CLI and configure authentication, either via environment variables or by assuming an IAM role using OIDC [1][2][3]. Key parameters for the aws-cli/setup command include: - role_arn: The Amazon Resource Name (ARN) of the role to assume [1][2]. - profile_name: The name of the AWS profile to configure (defaults to "default") [1][2]. - role_session_name: An identifier for the assumed role session [1]. - session_duration: The duration of the session in seconds (defaults to 3600) [1]. - region: The AWS region to configure (defaults to ${AWS_DEFAULT_REGION}) [1]. - set_aws_env_vars: A boolean to determine if AWS environment variables should be set (defaults to true) [1]. - version: Specifies the version of the AWS v2 CLI to install (defaults to "latest") [1]. Example Usage: To use the orb in your configuration, include it in the orbs section of your .circleci/config.yml file and invoke the setup command in your job steps [2][3]: version: 2.1 orbs: aws-cli: circleci/aws-cli@5.2.0 jobs: my-job: executor: aws-cli/default steps: - checkout - aws-cli/setup: role_arn: arn:aws:iam::123456789012:role/my-role profile_name: my-profile - run: name: Verify authentication command: aws sts get-caller-identity --profile my-profile For the most up-to-date parameter details and usage examples, refer to the official CircleCI Orb Registry page for the circleci/aws-cli orb [4][5]. Note that in version 5.x, some users have noted changes in how parameters and environment variables are handled compared to previous versions; if you encounter authentication issues, ensuring your profile configuration is explicit can help [6][7].

Citations:

🌐 Web query:

CircleCI config command step when always supported on orb command invocation

💡 Result:

In CircleCI, you cannot directly apply the when: always attribute to the invocation of an orb command in your config.yml file [1][2]. The when attribute is a property of individual steps (such as run), not the command invocation itself [3][2]. To achieve the behavior where an entire orb command runs regardless of the success or failure of previous steps, you must apply the when: always attribute to each individual step defined within the orb command [1]. If you are using a third-party orb, you cannot modify its internal steps. In such cases, if the orb author has not designed the command steps to be configurable or specifically set to when: always, you may be unable to force the desired behavior through standard configuration [1][2][4]. As a workaround when using an orb that lacks this support, users often: 1. Copy the source code of the orb command into their own local config.yml and modify the when attribute on the internal run steps [1]. 2. Create a feature request on the specific orb's repository, as this is a common request for operations like uploading logs or tearing down infrastructure [4].

Citations:

Move AWS setup before the E2E test step
aws-cli/setup is an orb command invocation, so when: always can’t be applied here. If npm run test:e2e fails, this step is skipped and the S3 upload loses the AWS CLI/auth it needs.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.circleci/config.yml around lines 109 - 118, The E2E report upload is
missing AWS auth when the test step fails because aws-cli/setup is currently
placed after the test flow and cannot use when: always. Move the aws-cli/setup
orb invocation earlier in the CircleCI job, before the npm run test:e2e step, so
the AWS CLI credentials are available regardless of test outcome; keep the
Upload Playwright report to S3 run step as the always-executed step using those
credentials.


# TODO: move invalidate-cloudfront-cache parameters to project env variables
# Eventually these will be the new cloudfront name for apps and app name respectively.

Expand All @@ -103,9 +127,17 @@ workflows:
s3-deploy-workflow:
jobs:
- node/test
- e2e-tests:
context:
- aws-context
requires:
- node/test
- build-and-deploy-dev:
context:
- aws-context
requires:
- node/test
- e2e-tests
filters:
branches:
only:
Expand All @@ -115,6 +147,7 @@ workflows:
- aws-context
requires:
- node/test
- e2e-tests
filters:
branches:
only:
Expand All @@ -124,6 +157,7 @@ workflows:
- aws-context
requires:
- node/test
- e2e-tests
filters:
branches:
only:
Expand All @@ -133,6 +167,7 @@ workflows:
- aws-context
requires:
- node/test
- e2e-tests
filters:
branches:
only:
Expand Down
3 changes: 2 additions & 1 deletion playwright.config.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@ export default defineConfig({
['json', { outputFile: 'test-results/results.json' }],
],
use: {
baseURL: 'http://localhost:8080',
headless: process.env.CI ? true : false,
baseURL: process.env.BASE_URL || 'http://localhost:8080',
// Capture screenshot on every failure for debugging
screenshot: 'only-on-failure',
// Retain video for failed tests so failures are reproducible
Expand Down