Skip to content

Commit cbfb30e

Browse files
fix(aws-lambda): fix flake in aws lambda tests
### Description - Catch `subprocess.TimeoutExpired` during SAM teardown and force-kill, so slow shutdown cannot fail the entire test suite. - Added temporary flake-check workflow that runs each test aws-lambda test 5 times to verify this removes the flake (workflow is removed in the last commit; was just to verify that it works). [https://github.com/getsentry/sentry-python/actions/runs/30539137588/job/90862136512?pr=6937](last successful run) - Switched SAM `--warm-containers` startup from `EAGER` to `LAZY` so Docker containers start on first invocation instead of all 21 at SAM startup. Verified locally by running `TESTPATH=tests/integrations/aws_lambda/test_aws_lambda.py uv run tox -e py3.8-aws_lambda ` both for `EAGER` & `LAZY` 10x independently: ``` EAGER: ================================================= Metric | Mean (s) | Std Dev (s) ================================================= Setup time | 43.69 | 3.99 First call time | 1.92 | 0.22 Max Teardown time | 7.49 | 0.27 Total run time | 108.53 | 4.56 ================================================= LAZY: ================================================= Metric | Mean (s) | Std Dev (s) ================================================= Setup time | 33.32 | 1.82 First call time | 6.32 | 0.63 Max Teardown time | 6.90 | 0.18 Total run time | 102.06 | 3.60 ================================================= ``` Resolves: #6911 & PY-2632
1 parent 4e9e97c commit cbfb30e

1 file changed

Lines changed: 7 additions & 6 deletions

File tree

tests/integrations/aws_lambda/test_aws_lambda.py

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@
1414

1515
DOCKER_NETWORK_NAME = "lambda-test-network"
1616
SAM_TEMPLATE_FILE = "sam.template.yaml"
17+
SAM_SHUTDOWN_TIMEOUT = 10
1718

1819

1920
@pytest.fixture(scope="session", autouse=True)
@@ -54,7 +55,7 @@ def test_environment():
5455
"--template",
5556
SAM_TEMPLATE_FILE,
5657
"--warm-containers",
57-
"EAGER",
58+
"LAZY", # Start each Docker container on its function's first invocation
5859
"--docker-network",
5960
DOCKER_NETWORK_NAME,
6061
],
@@ -78,13 +79,13 @@ def before_test():
7879

7980
finally:
8081
print("[test_environment fixture] Tearing down AWS Lambda test infrastructure")
81-
8282
process.terminate()
83-
process.wait(timeout=10) # Give it time to shut down gracefully
84-
85-
# Force kill if still running
86-
if process.poll() is None:
83+
try:
84+
# Teardown is typically ~7s; escalate with kill if SAM exceeds this.
85+
process.wait(timeout=SAM_SHUTDOWN_TIMEOUT)
86+
except subprocess.TimeoutExpired:
8787
process.kill()
88+
process.wait()
8889

8990

9091
@pytest.fixture(autouse=True)

0 commit comments

Comments
 (0)