Repository navigation
Reco: fix fetch-incidents skipping alerts during bursts - #3
Open
RecoLabs-YaronBratspiess wants to merge 171 commits into
Open
RecoLabs-YaronBratspiess wants to merge 171 commits into
RecoLabs-YaronBratspiess wants to merge 171 commits into
Conversation
* add logs and rate limit handling * rever not needed logs * rn * update the docker * pack metadata * rn * rn * rn * rn * rn * rn * add unit tests
This PR transitions the non-Core aggregated scripts to use the new PCI built-in commands on the unified Cortex platform, aligned with the corresponding command migration already done in the content repo. On the platform, these scripts now call the server-injected built-in (PCI) commands; on legacy XSIAM they continue to use the Cortex Core - IR integration commands.
…ion (demisto#45082) * XSUP-72930 - improve BeyondTrust Password Safe target account extraction * added more support in regex * refined release notes
* rn * the fix * fix test * format * ai reviewer
* update * bump * Bump pack from version Base to 1.42.9. --------- Co-authored-by: Content Bot <bot@demisto.com>
* Updated Docker Images for Active_Directory_Query. * Updated Release Notes. * Bump pack from version Active_Directory_Query to 1.7.2. * Bump pack from version Active_Directory_Query to 1.7.3. --------- Co-authored-by: content-bot <content-bot@users.noreply.github.com> Co-authored-by: Content Bot <bot@demisto.com> Co-authored-by: Shmuel Kroizer <69422117+shmuel44@users.noreply.github.com>
Map xdm.target.resource.name to "OpenAI API Platform" for login/logout events in openai_admin_audit_raw so auth_target populates in the Authentication Story (event_type 102).
* Updated Docker Images for ServiceNow. * Updated Release Notes. * Bump pack from version ServiceNow to 2.9.13. * bump * bump --------- Co-authored-by: content-bot <content-bot@users.noreply.github.com> Co-authored-by: Content Bot <bot@demisto.com> Co-authored-by: shmuel44 <skroizer@paloaltonetworks.com> Co-authored-by: Shmuel Kroizer <69422117+shmuel44@users.noreply.github.com>
* Add OAuth support (integration code and yml parameters) * Enhance README and description with External OAuth authentication details * Update to version 1.0.13 * ix validation and pre-commit issues * fix after review * Add advanced option for 'Trust any certificate' and 'OAuth Client ID' fields in Snowflake integration * fixes after demo * bump version * fix pre-commit errors --------- Co-authored-by: test-split <test@test.com>
…n FireEye ETP (demisto#45037) * Add Token URL parameter for OAuth 2.0 endpoint override in FireEye ETP integrations * fix after ai-reviewer * update docker image * fix validation erros * bump version * Revert "bump version" This reverts commit d613f33. --------- Co-authored-by: test-split <test@test.com>
…ol (demisto#44829) * possible fix * additional logs * additional logs * 2 tier architecture * prepare for merge * Remove keepalive_timeout from NGINXApiModule * fix pre commit * fix pre commit - mypy * Address marketplace-ai-reviewer comments on PR demisto#44829 - RN wording/styling (approved prefixes, single-asterisk params) - YAML additionalinfo descriptions start with 'The' and end with a period - NGINXApiModule: datetime.now(timezone.utc) instead of deprecated utcnow(); log traceback before re-raising streaming errors - EDL: make _stdout_lock_timeout tweak a true no-op via hasattr guard; debug log when get_request_id has no Flask request context - EDL tests: add coverage for get_request_id and _stdout_lock_timeout restoration/no-op * Bump EDL to 3.4.0 and TAXIIServer to 2.3.0 (deprecations require minor) Per marketplace-ai-reviewer review: deprecating the Cache Lock Timeout/Age parameters is a functional change requiring a minor version bump. * Update EDL README: mark Cache Lock Timeout/Age parameters as deprecated * EDL tests: use a single top-level 'import EDL as edl' (resolve CodeQL mixed-import finding) * fix validations * Bump pack from version ApiModules to 2.4.35. * update docker * empty * fix validation * Bump pack from version ApiModules to 2.4.36. * Update Packs/ApiModules/Scripts/NGINXApiModule/NGINXApiModule.py Co-authored-by: Dean Arbel <darbel@paloaltonetworks.com> --------- Co-authored-by: Content Bot <bot@demisto.com> Co-authored-by: Dean Arbel <darbel@paloaltonetworks.com>
* Fix for snapshot sealing issue * Fix for OOM issues in tenable
…nter display (demisto#45105) * XSUP-72627 * code review * fix * fix * Revert "fix" This reverts commit 7ffc23a. * secret fix
* Initial release notes for az-CRTX-258563 * Bump pack from version Core to 3.5.73. * Sync release notes from GitLab (4a3ecd93) * Sync release notes from GitLab (4c1666ba) * Sync release notes from GitLab (e7734b1e) * Sync release notes from GitLab (74ebc5cd) * Sync release notes from GitLab (bae2119d) * Sync release notes from GitLab (c1ca60d5) * Sync release notes from GitLab (dc098c63) * Sync release notes from GitLab (bff758ac) * Sync release notes from GitLab (05cb0647) * Sync release notes from GitLab (3641acb1) * Sync release notes from GitLab (fa051082) * Sync release notes from GitLab (9354507b) * Sync release notes from GitLab (ae5ca84e) --------- Co-authored-by: CI Bot <ci@demisto.com> Co-authored-by: Content Bot <bot@demisto.com> Co-authored-by: azonenfeld <117573492+aaron1535@users.noreply.github.com> Co-authored-by: Bar Gali <75535203+BarGali@users.noreply.github.com>
* Fix/XSUP-72401/File reputation no HTTP response * add continueonerror * AI review comments
…uccess] (demisto#45156) * Updated Docker Images. * Updated Release Notes. * fix mypy --------- Co-authored-by: content-bot <content-bot@users.noreply.github.com> Co-authored-by: shmuel44 <skroizer@paloaltonetworks.com>
…4172) * potential fix for second in time issue * Fix for missing findings when instance is under pressure. * Changes per findings * Changes per CR * pack metadata * Add tests * ST110 --------- Co-authored-by: Daniel Tal <87daniel.tal@gmail.com>
* PaloAltoNetworks Device Security (SCM) Content Pack (demisto#44866) * feat: scm integration * update: fixes to test and naming * update: fix test with dummy scm url,remove monkey patch, contextPath, outputs_prefix and README.md files update, support scm v2 for get devices, test playbook * add: demisto mock import * update: incident type prevName property value to the same name property value * update: removed is_staging flag and url creation, removed dummy link from description md file * update: removed is_staging flag and url creation, removed dummy link from description md file * update: removed demistomock import in integration test * update: playbook taskid, id - uuid * demisto format * update: addressed review comments * update: PaloAltoNetworks_DeviceSecurity_SCM to PaloAltoNetworks_DeviceSecurity * update: remove staging config from DeviceSecurity README * update: clean integration description link block * update: added recommend changes along with classifier, mapper, layout. * update: added recommend changes along with classifier, mapper, layout. * update: classifier, layout and close incident defaults * update: layout * update: README with generate-docs cmd * update: description file with minimum permissions * update: as per review comments * update: conf.json file * update: documents and integration yaml file * update: applied suggested changes * fix: integration test * update: mapper to use field display names * update: remove is_staging flag * update: remove test config and related yaml value * fix validation errors * fix validation errors * fix validation errors * fix test config bug * add PaloAltoNetworks_DeviceSecurity-Test to skipped tests and update test configuration * moved to tests section * changed the id * skip PaloAltoNetworks_DeviceSecurity-Test - credentials not provisioned --------- Co-authored-by: sarathk-metron <sarath.kumar@metronlabs.com> Co-authored-by: test-split <test@test.com> Co-authored-by: lironcohen272 <lircohen@paloaltonetworks.com> Co-authored-by: Shir <smatathias@paloaltonetworks.com>
* CTM360 Pack Update - More APIs support (demisto#44804) * HackerView DeepScan module, brand field, and pack content. Adds Deep Scan support, incident fields, playbook V2, and a dedicated Deep Scan dashboard while preserving the original HackerView dashboard. * Release notes and pack version 2.3.5 for HackerView DeepScan. * Add CyberBlindspot Social Media Fraud, Money Mules, and Gambling Sites modules. * Always map incident severity to numeric values on fetch. Remove platform-specific text severity so XSOAR and XSIAM incident fetch use convert_to_demisto_severity consistently. * Regenerate integration READMEs and fix pack validation findings. Update CyberBlindspot and HackerView docs from generate-docs, mark numeric incident fields unsearchable, and add pack-ignore entries for validation rules. * Fix CTM360 pack validation and YAML review findings. Address ST110, pack metadata, dashboard export fields, GS URL types, source_uri typo, and description standards in both integrations. * Align HackerView and CyberBlindspot mapping field descriptions. Keep get-mapping-fields descriptions consistent with YAML output standards, and derive CBS mapping tests from the field lists. * Address HackerView Incident Management V2 playbook review feedback. Document inactive-ticket auto-close, expose closeReason as an input, add a default prompt fallback, and remove quiet mode. * Fix validation errors * Update dashboard-HackerView_DeepScan.json * Update dashboard-HackerView_DeepScan.json * Update dashboard-HackerView_DeepScan.json --------- Co-authored-by: S. AlQasim D. <136978057+edx-sayed-salem@users.noreply.github.com> Co-authored-by: Kamal Qarain <45042524+kamalq97@users.noreply.github.com> Co-authored-by: Kamal Qarain <kqarain@paloaltonetworks.com>
* CIAC-16603: add SplunkPy v2 configuration file/stanza commands * fix pre commit * CIAC-16603: address marketplace-ai-reviewer feedback - Make limit None-safe so limit=0 is not overridden to 50 (file-list, stanza-list) - Add return type hint to get_configuration_file - Add explicit argument types and clean up descriptions in SplunkPyV2.yml - Regenerate README from yml * update the HR
* Improvements to collector performance * Performance improvements * Productizing PR * Changes per CR GH * Changes per AI review * SaaS Security: address AI review (minor bump, YAML key order, get-events debug note, keyword/branding fixes) * Changes per CR
* Rubrik Release 1.9.0 * Updated the fetch sensitive objects logic. * Review changes. * remove other content packs' incident fields from Sensitive Data Object layout --------- Co-authored-by: Crest Data <60967033+crestdatasystems@users.noreply.github.com> Co-authored-by: crestdatasystems <crestdatasystems@users.noreply.github.com> Co-authored-by: Kamal Qarain <45042524+kamalq97@users.noreply.github.com>
…5171) * Fix PolySwarmV2 test_get_file VCR cassette A Content-Encoding mismatch broke the tests. Removed a stale `Content-Encoding: gzip` header from the test_get_file.tape cassette. The recorded response body was already decoded plaintext, causing `httpx` to fail when attempting to gunzip it during test replay. - Bumped PolySwarm pack version to 2.0.13 - Added release notes for the fix * Added CONTRIBUTORS.json to PolySwarm pack Co-authored-by: Alan Justino da Silva <alan.justino@yahoo.com.br>
* poetry files * update validation config file * empty commit --------- Co-authored-by: Content Bot <bot@demisto.com> Co-authored-by: yedidyacohenpalo <yecohen@paloaltonetworks.com>
* update version * update version * Bump pack from version Base to 1.42.10. --------- Co-authored-by: Content Bot <bot@demisto.com>
…play name (demisto#45144) * SpecterOpsBloodHoundEnterprise: Deprecate legacy packs and update display name (demisto#45056) * Removed legacy BloodHound packs and fixed SpecterOps display name * Update pack metadata with new contributor credentials * Revert "Removed legacy BloodHound packs and fixed SpecterOps display name" This reverts commit 1861a4c. * Deprecate legacy BloodHound packs and update active pack name * Addressed AI reviewer feedback * Updated the default playbook reference * Update incidenttype-SpecterOpsBHE_Attack_Path.json * Update incidenttype-SpecterOpsBHE_Attack_Path.json * Update .pack-ignore --------- Co-authored-by: tanujs-metron <tanuj.sharma@metronlabs.com> Co-authored-by: Kamal Qarain <kqarain@paloaltonetworks.com>
* Updated Docker Images for Gmail. * Updated Release Notes. --------- Co-authored-by: content-bot <content-bot@users.noreply.github.com>
Co-authored-by: test-split <test@test.com>
…collection refactor (demisto#45385) * Akamai SIEM: refactor decode logic, dedicated events fetch limit, tests + RN * Update Packs/Akamai_SIEM/Integrations/Akamai_SIEM/README.md Co-authored-by: Marketplace AI reviewer <svc-mp-ai-reviewer@paloaltonetworks.com> * AI Reviewer * Update Packs/Akamai_SIEM/Integrations/Akamai_SIEM/Akamai_SIEM.py Co-authored-by: Daniel Tal <87daniel.tal@gmail.com> * Code review * Set advanced flags for long-running params --------- Co-authored-by: Marketplace AI reviewer <svc-mp-ai-reviewer@paloaltonetworks.com> Co-authored-by: Daniel Tal <87daniel.tal@gmail.com>
* CRTX-233866_Update_aws-s3-bucket-objects-list_to_use_v2 * ai reviewr * cr fixes * context update * Trigger CI * readme * cr2 fixes
…isto#45333 (demisto#45478) * fixed output in compute_network_tag_set and unit tests for yml + py verification. * rn * ai review * cr fixes * pre-commit * annotation * revision version
…uccess] (demisto#45396) * Updated Docker Images. * Updated Release Notes. * fix mypy * Bump pack from version ContentManagement to 1.2.39. --------- Co-authored-by: content-bot <content-bot@users.noreply.github.com> Co-authored-by: shmuel44 <skroizer@paloaltonetworks.com> Co-authored-by: Content Bot <bot@demisto.com>
…uccess] (demisto#45487) * Updated Docker Images. * Updated Release Notes. * Bump pack from version CommunityCommonScripts to 1.5.7. * fix PytestConfigWarning * Revert Packs/SOCRadar to origin/master state The SOCRadar docker image bump and release notes were split out to branch AUD-demisto/2026-08-11/SOCRadar so the pack can be handled separately. This is a forward commit; no published history was rewritten. --------- Co-authored-by: content-bot <content-bot@users.noreply.github.com> Co-authored-by: Content Bot <bot@demisto.com> Co-authored-by: shmuel44 <skroizer@paloaltonetworks.com>
…emisto#45166) * add polling argument * implement polling support functionality to system status command * implement polling functionality * add tests * add RN and bump version * resolve ai reviewer comment * fix ruff format * rephrase code comment * Bump pack from version PAN-OS to 2.6.44. * add supportedModules to fetch parameters * skip BlockExternalIP test playbook due to not working instance * bump version --------- Co-authored-by: test-split <test@test.com> Co-authored-by: Content Bot <bot@demisto.com>
…emisto#45357) * CRTX-266332: remove SearchIndicatorInEventsAgentix (capped at 8.13) Delete the script, which carries toversion: 8.13.0 and is dead code on any 8.14+ server. It was a thin polling wrapper that delegated to SearchIndicatorInEvents. Superseded by a 3-generation version band: <=8.13 SearchIndicatorInEventsAgentix (this script, content) 8.14 SearchIndicatorInEventsAgentixV2 (content-private, same id) >=8.15 CortexListEventsByIndicator (content-private, managed pack) The V2 successor carries the identical commonfields.id, so id resolution on 8.14+ is unaffected. The dependency SearchIndicatorInEvents is untouched: it has no toversion and remains marketplacev2-facing. * rn * add cooc_command_examples.txtx to gitignore (demisto#45369) * Trigger GitHub pipeline (user-created PR) * rn * Trigger GitHub pipeline (user-created PR) * Bump pack from version Base to 1.42.17. * Bump pack from version Core to 3.5.82. * Bump pack from version Core to 3.5.83. * rn --------- Co-authored-by: Shir2611 <smatathias@paloaltonetworks.com> Co-authored-by: ilaredo <166304750+ilaredo@users.noreply.github.com> Co-authored-by: CI Bot <ci@demisto.com> Co-authored-by: Content Bot <bot@demisto.com>
…emisto#45315) * Druva: add support for modeling Cybersecurity events
…ters (demisto#45366) * CRTX-271422: Fix Slack invalid_blocks for empty markdown delimiters Empty markdown delimiters (e.g. '__' or '****') produced a rich_text element with an empty text field, which the Slack SDK strips on serialization, causing Slack to reject the message with an invalid_blocks error. As a result, sensitive actions requiring approval in the Cortex Agentic Assistant were sometimes not sent to Slack. Now the original delimiters are preserved as literal, unstyled text. Also added regression tests and non-sensitive debug logging in send_agent_response. * CRTX-271422: Make underscore italics boundary-aware and add attachments fallback - Underscores adjacent to word characters are no longer treated as italic, so intraword tokens (e.g. run__script, first_name) are preserved literally instead of being corrupted/dropped. This also fixes the invalid_attachments error for Assistant plan/step messages. - post_agent_response now falls back to plain text on invalid_attachments (not just invalid_blocks) so the message is not lost. - Added/updated regression tests. * Update Packs/ApiModules/ReleaseNotes/2_4_39.md Co-authored-by: Marketplace AI reviewer <svc-mp-ai-reviewer@paloaltonetworks.com> * Update Packs/Slack/ReleaseNotes/3_8_13.md Co-authored-by: Marketplace AI reviewer <svc-mp-ai-reviewer@paloaltonetworks.com> * Fix review comments --------- Co-authored-by: Marketplace AI reviewer <svc-mp-ai-reviewer@paloaltonetworks.com>
Extracted Packs/SOCRadar changes from branch AUD-demisto/2026-08-11/gitlab-pipeline-11839907/success so the pack can be handled separately.
* Updating the dockerimage-version to demisto/python3:3.12.13.10404775 * Update medatada and releasenotes. Co-authored-by: David <128359288+david-syk@users.noreply.github.com>
) * CRTX-271523: add identity.* to modeling rules (PR 1 of 7) Part of the XDM user.* -> identity.* alignment (epic CRTX-271520). Adds a new _3_19 modeling rule folder (fromversion 8.15.0) per pack that maps xdm.<prefix>.identity.<field> alongside every existing xdm.<prefix>.user.<field> assignment. The previous folder is capped with toversion 8.14.0. BeyondTrustRemoteSupport (44). * CRTX-271523 - fix BC107: align _3_19 modeling rule id with original (BeyondTrustRemoteSupport) * CRTX-271523 - fix RN107: align _3_19 rule name + release-note header with original rule (BeyondTrustRemoteSupport) --------- Co-authored-by: tmizrahi@paloaltonetworks.com <tmizrahi@paloaltonetworks.com>
) * CRTX-271523: add identity.* to modeling rules (PR 3 of 7) Part of the XDM user.* -> identity.* alignment (epic CRTX-271520). Adds a new _3_19 modeling rule folder (fromversion 8.15.0) per pack that maps xdm.<prefix>.identity.<field> alongside every existing xdm.<prefix>.user.<field> assignment. The previous folder is capped with toversion 8.14.0. AzureDevOps (15), FortinetFortiwebVM (3), CybelAngel (3), Barracuda_Cloudgen_Firewall (2), NetmotionVPN (1). * Bump pack from version CybelAngel to 1.1.11. * CRTX-271523 - fix BC107: align _3_19 modeling rule ids with originals * CRTX-271523 - fix RN107: align _3_19 rule names + release-note headers with original rules --------- Co-authored-by: Content Bot <bot@demisto.com> Co-authored-by: tmizrahi@paloaltonetworks.com <tmizrahi@paloaltonetworks.com>
* CRTX-262438_azure_key_vault * ai review fixes * Delete Packs/AzureKeyVault/ReleaseNotes/1_1_62.md * cr fixes * doc fixes * doc fixes2
* Update v2/mitigation endpoint (demisto#45497) * Update release notes --------- Co-authored-by: Ata Berk Gümüş <ata.abg@gmail.com> Co-authored-by: Kamal Qarain <kqarain@paloaltonetworks.com>
…misto#45514) * update sdk, tl3, error handling * remove unneeded file from commit * update to get-version function Co-authored-by: Brad Chiappetta <brad@greynoise.io>
* update version * update version * fixes for cr
…emisto#45290) * fix(okta): handle HTTP 429 with automatic retries in event collector * fix(okta): add missing isFetchEvents and eventFetchInterval params * fix(okta): parse event collector time arguments as UTC * fix(okta): expand event collector debug logging and reach full test coverage * test(okta): cover 429 retry behavior in event collector * Bump pack from version Okta to 3.4.0. * docs(okta): correct incidents to events in collector config labels --------- Co-authored-by: Content Bot <bot@demisto.com>
…demisto#45389) * fix * fix * fix * fix * Bump pack from version ApiModules to 2.4.39. * fix * test * fix * fix * fix * fix * fix * fix * fix * Bump pack from version Core to 3.5.83. * fix release notes * fix * fix * fix release notes * fix * fix * Bump pack from version Core to 3.5.84. * Bump pack from version ApiModules to 2.4.40. * skip playbooks --------- Co-authored-by: Content Bot <bot@demisto.com>
) * CRTX-271526: add identity.* to modeling rules (PR 1 of 7) Part of the XDM user.* -> identity.* alignment (epic CRTX-271520). Adds a new _3_19 modeling rule folder (fromversion 8.15.0) per pack that maps xdm.<prefix>.identity.<field> alongside every existing xdm.<prefix>.user.<field> assignment. The previous folder is capped with toversion 8.14.0. Packs: VMwareESXi (21) - 21 assignments. * CRTX-271526: rename modeling rule folder 3_19 -> 2_11 * Update VMwareESXi_2_11.yml * Fix RN107: match modeling rule name in VMwareESXi release note
) * CRTX-271526: add identity.* to modeling rules (PR 2 of 7) Part of the XDM user.* -> identity.* alignment (epic CRTX-271520). Adds a new _3_19 modeling rule folder (fromversion 8.15.0) per pack that maps xdm.<prefix>.identity.<field> alongside every existing xdm.<prefix>.user.<field> assignment. The previous folder is capped with toversion 8.14.0. Packs: Office365 (27), CiscoFirepower (2), UbiquitiUnifi (1) - 30 assignments. * CRTX-271526: rename modeling rule folder 3_19 -> 2_11 * Fix BC107/BC117/RN107: align new 2_11 modeling rule id/name with existing rules * Fix MR108: ignore legacy modeling rule id suffix for CiscoFirepower_2_11
…isto#45526) * Fix update_incident_status failing on empty API response (demisto#45522) * Update ThreatMon.py --------- Co-authored-by: cengugr <ugr.krcck@gmail.com> Co-authored-by: Kamal Qarain <kqarain@paloaltonetworks.com>
…apture names (demisto#45529) The parsing rule built RE2 named-capture groups from changedValues[].name. Add \. to the replex sanitize alternation so dots become underscores.
* Enhancement/CIAC-16873/ES add mirror * kickoff improvements * fix * Remove redundant classifiers * Added description and fixes * fix mirror * ruff fix * conflicts * pre-commit fixes * fix * f * fix fetch security alerts * revert ut * fix classifier * update RN * Add UT * fix mirror in * ai review fixes * Update Packs/Elasticsearch/Integrations/Elasticsearch_v2/README.md Co-authored-by: cbalnave <cbalnave@paloaltonetworks.com> * Update Packs/Elasticsearch/Integrations/Elasticsearch_v2/README.md Co-authored-by: cbalnave <cbalnave@paloaltonetworks.com> * Update Packs/Elasticsearch/Integrations/Elasticsearch_v2/README.md Co-authored-by: cbalnave <cbalnave@paloaltonetworks.com> * Update Packs/Elasticsearch/Integrations/Elasticsearch_v2/README.md Co-authored-by: cbalnave <cbalnave@paloaltonetworks.com> * pre commit fix * validate and pre-commit fixes * update mapper * t * fix * fix validate * Update Packs/Elasticsearch/Integrations/Elasticsearch_v2/README.md Co-authored-by: cbalnave <cbalnave@paloaltonetworks.com> --------- Co-authored-by: cbalnave <cbalnave@paloaltonetworks.com>
fetch_incidents fetched a single page per cycle from the External API's default (createdAt descending) sort order, then advanced the lastRun cursor to the newest timestamp seen in that page. During a burst larger than Max fetch, the returned page was the newest slice of the window, so the cursor jumped past older alerts that never made it into a page - permanently skipping them (reported: 669/1069 alerts lost in a one-minute burst). get_alerts now requests ascending createdAt order and forwards start_index. fetch_incidents only advances lastRun once a page comes back smaller than max_fetch (the window is fully drained); a full page instead advances a startIndex offset persisted in last_run, so the burst is paginated across fetch cycles instead of being truncated.
fetch_incidents polled the External API's alerts/list with no status filter, so alerts belonging to a Policy in Preview state were ingested into the customer's XSOAR instance. Preview state is explicitly meant to keep alerts out of SOAR/ticketing systems while a Policy is being evaluated, and the webhook and share-service delivery paths already enforce that - this pack was the one path that didn't. get_alerts now always adds `status ne "ALERT_STATUS_PREVIEW"` to the SCIM filter, verified against the live External API via the Reco MCP server.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
During an alert flood,
fetch-incidentscould permanently skip the majority of alerts in a single time window. Separately,fetch-incidentswas also ingesting alerts belonging to Policies in Preview state, which should never leave Reco.Fix 1 - burst pagination:
Root cause:
get_alertsfetched one page per cycle using the External API's default sort (createdAtdescending), thenfetch_incidentsadvanced thelastRuncursor to the newest timestamp seen in that page. When a burst exceededMax fetchin one window, the returned page was the newest slice of the burst, so the cursor jumped past all the older alerts that didn't make it into that page — silently dropping them forever (reported: 669/1069 alerts lost in a one-minute burst, Iron Mountain support ticket).get_alertsnow requests ascendingcreatedAtorder and forwardsstart_index.fetch_incidentsonly advanceslastRunonce a page comes back smaller thanmax_fetch(i.e. the window is fully drained). A full page instead advances astartIndexoffset persisted inlast_run, so a burst larger thanmax_fetchis paginated across multiple fetch cycles instead of being truncated.Fix 2 - exclude Preview-state alerts:
Root cause: a Policy in Preview state is meant to keep its alerts out of the customer's SOAR while it's being evaluated. Reco's webhook and share-service delivery paths already enforce this, but
fetch-incidents(which polls the External API directly) had no equivalent guard, so Preview alerts leaked into XSOAR. Reported in the CAA Slack thread after theTriggerShareguard fix turned out not to cover the content-pack polling path.get_alertsnow always addsstatus ne "ALERT_STATUS_PREVIEW"to its SCIM filter.Version bumped 1.8.0 → 1.8.1.
Test plan
pytest Packs/Reco/Integrations/Reco/Reco_test.py— 72/72 pass (Docker unavailable locally, sodemisto-sdk pre-commit's ruff/xsoar-lint/pylint/mypy steps will run via CI on the upstream PR rather than locally)startIndexwithout skipping;get_alertssends ascending sort + pagination params;get_alertsexcludesALERT_STATUS_PREVIEWalongside other filtersstatus ne "ALERT_STATUS_PREVIEW"verified against the live External API (confirmed a known Preview alert is excluded by this exact filter)Now opened against
demisto/contentas demisto#45577.