feat: FM-949 Added e2e playwright automation to circle ci/cd pipeline#1992
feat: FM-949 Added e2e playwright automation to circle ci/cd pipeline#1992ashwinnair-chimple wants to merge 2 commits into
Conversation
📝 WalkthroughWalkthroughCircleCI adds a new Playwright E2E job that runs tests, stores results, and uploads the Playwright report to S3. The deploy workflow then requires that E2E job before dev, test, release, and prod deployment jobs can run. ChangesCircleCI E2E job and deployment gating
Estimated code review effort: 3 (Moderate) | ~20 minutes 🚥 Pre-merge checks | ✅ 5 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.circleci/config.yml:
- Around line 109-118: The E2E report upload is missing AWS auth when the test
step fails because aws-cli/setup is currently placed after the test flow and
cannot use when: always. Move the aws-cli/setup orb invocation earlier in the
CircleCI job, before the npm run test:e2e step, so the AWS CLI credentials are
available regardless of test outcome; keep the Upload Playwright report to S3
run step as the always-executed step using those credentials.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 26858b95-e523-49d5-808e-fa92273c89b4
📒 Files selected for processing (1)
.circleci/config.yml
| - aws-cli/setup: | ||
| aws_access_key_id: $AWS_ACCESS_KEY | ||
| region: $AWS_DEFAULT_REGION | ||
| aws_secret_access_key: $AWS_SECRET_ACCESS_KEY | ||
| - run: | ||
| name: Upload Playwright report to S3 | ||
| when: always | ||
| command: | | ||
| aws s3 sync playwright-report \ | ||
| "s3://globallit-aws-s3-static-webapp-test-us-east-2/e2e-reports/${CIRCLE_BRANCH}/${CIRCLE_BUILD_NUM}/" |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== .circleci/config.yml: relevant lines =="
nl -ba .circleci/config.yml | sed -n '90,145p'
echo
echo "== orb references =="
rg -n "circleci/aws-cli|aws-cli/setup|aws-context|e2e-tests|when: always|npm run test:e2e" .circleci/config.yml
echo
echo "== top of config for orb version pin =="
nl -ba .circleci/config.yml | sed -n '1,40p'Repository: curiouslearning/FeedTheMonsterJS
Length of output: 253
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== .circleci/config.yml: relevant lines =="
awk 'NR>=90 && NR<=145 { printf "%5d %s\n", NR, $0 }' .circleci/config.yml
echo
echo "== orb references =="
rg -n "circleci/aws-cli|aws-cli/setup|aws-context|e2e-tests|when: always|npm run test:e2e" .circleci/config.yml
echo
echo "== top of config for orb version pin =="
awk 'NR>=1 && NR<=40 { printf "%5d %s\n", NR, $0 }' .circleci/config.ymlRepository: curiouslearning/FeedTheMonsterJS
Length of output: 4124
🌐 Web query:
CircleCI aws-cli orb setup command when parameter circleci/aws-cli@5.2.0 docs
💡 Result:
The aws-cli/setup command in the circleci/aws-cli orb (version 5.2.0 and later) is used to install the AWS CLI and configure authentication, either via environment variables or by assuming an IAM role using OIDC [1][2][3]. Key parameters for the aws-cli/setup command include: - role_arn: The Amazon Resource Name (ARN) of the role to assume [1][2]. - profile_name: The name of the AWS profile to configure (defaults to "default") [1][2]. - role_session_name: An identifier for the assumed role session [1]. - session_duration: The duration of the session in seconds (defaults to 3600) [1]. - region: The AWS region to configure (defaults to ${AWS_DEFAULT_REGION}) [1]. - set_aws_env_vars: A boolean to determine if AWS environment variables should be set (defaults to true) [1]. - version: Specifies the version of the AWS v2 CLI to install (defaults to "latest") [1]. Example Usage: To use the orb in your configuration, include it in the orbs section of your .circleci/config.yml file and invoke the setup command in your job steps [2][3]: version: 2.1 orbs: aws-cli: circleci/aws-cli@5.2.0 jobs: my-job: executor: aws-cli/default steps: - checkout - aws-cli/setup: role_arn: arn:aws:iam::123456789012:role/my-role profile_name: my-profile - run: name: Verify authentication command: aws sts get-caller-identity --profile my-profile For the most up-to-date parameter details and usage examples, refer to the official CircleCI Orb Registry page for the circleci/aws-cli orb [4][5]. Note that in version 5.x, some users have noted changes in how parameters and environment variables are handled compared to previous versions; if you encounter authentication issues, ensuring your profile configuration is explicit can help [6][7].
Citations:
- 1: https://github.com/CircleCI-Public/aws-cli-orb/blob/master/.circleci/test-deploy.yml
- 2: https://github.com/CircleCI-Public/aws-cli-orb/releases/tag/v3.1.0
- 3: https://support.circleci.com/hc/en-us/articles/4585261592347-How-to-authenticate-multiple-AWS-profiles-using-OIDC
- 4: https://circleci.com/developer/orbs/orb/circleci/aws-cli
- 5: https://github.com/CircleCI-Public/aws-cli-orb/blob/master/README.md
- 6: [aws-cli/setup] After upgrading from 4.0.0 to 5.1.1, the region is not set correctly anymore CircleCI-Public/aws-cli-orb#213
- 7: [Bug] v5.x.x vs v4.x.x
aws-cli/setupchanges broke Terragrunt setup CircleCI-Public/aws-cli-orb#236
🌐 Web query:
CircleCI config command step when always supported on orb command invocation
💡 Result:
In CircleCI, you cannot directly apply the when: always attribute to the invocation of an orb command in your config.yml file [1][2]. The when attribute is a property of individual steps (such as run), not the command invocation itself [3][2]. To achieve the behavior where an entire orb command runs regardless of the success or failure of previous steps, you must apply the when: always attribute to each individual step defined within the orb command [1]. If you are using a third-party orb, you cannot modify its internal steps. In such cases, if the orb author has not designed the command steps to be configurable or specifically set to when: always, you may be unable to force the desired behavior through standard configuration [1][2][4]. As a workaround when using an orb that lacks this support, users often: 1. Copy the source code of the orb command into their own local config.yml and modify the when attribute on the internal run steps [1]. 2. Create a feature request on the specific orb's repository, as this is a common request for operations like uploading logs or tearing down infrastructure [4].
Citations:
- 1: https://discuss.circleci.com/t/make-custom-command-run-always-with-when-always/38957
- 2: https://discuss.circleci.com/t/run-orb-commands-on-fail/34396
- 3: https://circleci.com/docs/reference/configuration-reference/index.md
- 4: Ability to specify
whenfor setup and install commands CircleCI-Public/aws-cli-orb#54
Move AWS setup before the E2E test step
aws-cli/setup is an orb command invocation, so when: always can’t be applied here. If npm run test:e2e fails, this step is skipped and the S3 upload loses the AWS CLI/auth it needs.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.circleci/config.yml around lines 109 - 118, The E2E report upload is
missing AWS auth when the test step fails because aws-cli/setup is currently
placed after the test flow and cannot use when: always. Move the aws-cli/setup
orb invocation earlier in the CircleCI job, before the npm run test:e2e step, so
the AWS CLI credentials are available regardless of test outcome; keep the
Upload Playwright report to S3 run step as the always-executed step using those
credentials.
2 participants
Changes
How to test
Ref: FM-949
Summary by CodeRabbit
New Features
Bug Fixes