Repository navigation
Publish consolidated BackIntel demo proof of concept - #2
Conversation
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f911f8c563
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| if finding['kind'] == 'estimate' and any(r.get('kind') != 'estimate' for r in results if r['evidence_id'] in finding['evidence_ids']): | ||
| raise ValueError('Estimate findings require prediction evidence') |
There was a problem hiding this comment.
When the analyst returns an estimate such as “Group A has high churn risk,” citing a predict result is sufficient here even if Group A has the lowest predicted mean. Because the claim contains neither a number nor one of the comparative words checked below, it bypasses the remaining semantic checks, and structured_fact is applied only to facts. Require estimate prose to use a validated metric/ranking format or otherwise bind qualitative risk levels to the cited prediction table.
Useful? React with 👍 / 👎.
| with db.connect() as c, c.transaction(): | ||
| changed = c.execute('UPDATE backintel.analysis_goals SET body=%s,version=%s,paused=%s,confirmed=%s WHERE id=%s AND version=%s AND body=%s AND paused=%s AND confirmed=%s', |
There was a problem hiding this comment.
If another manager disables this actor, expires the credential, or removes the goal's domain after the authorization at line 80 but before this transaction executes, the update still succeeds because its predicate checks only the prior goal state. The revoked manager can therefore revise or reconfirm/resume a standing goal that later authorized analysts will use; lock and validate the current principal in this transaction before applying the edit.
Useful? React with 👍 / 👎.
| values=estimator.predict_proba(x)[:,list(estimator.classes_).index(1)] if body['kind']=='classification' else estimator.predict(x) | ||
| return {r['id']:float(v) for r,v in zip(rows,values)} |
There was a problem hiding this comment.
For the support and maintenance domains, CatBoost or TabICLv2 can emit negative regression values even though resolution hours and remaining cycles are explicitly nonnegative targets. This path converts those outputs directly to floats, so an approved model can publish impossible negative estimates in analysis tables and findings; apply the same nonnegative output constraint during both held-out evaluation and operational prediction, or reject a model that violates the target domain.
Useful? React with 👍 / 👎.
| 'Keep the normal outer answer schema. Put ONLY a JSON object with these fields in the summary string: ' | ||
| + json.dumps(fields) + '. Use numeric values for value and count. Return exactly one fact finding; ' | ||
| 'its claim must be the same JSON object as the summary and cite the calculation evidence. ' | ||
| 'State limitations in the limitations list. If the calculation is unavailable, state that limitation instead of inventing values.') |
There was a problem hiding this comment.
When --execute-real runs any benchmark scenario, this prompt requires the finding claim to be a bare JSON object. analyze() passes that claim through validate_answer(), whose final structured_fact() check accepts only fixed natural-language metric, ranking, or classification forms; JSON punctuation can never match those forms, and multi-field numeric JSON can also fail the narrative-number checks first. Consequently a model response that follows this prompt is rejected and the run becomes partial before score_answer() can inspect it, so the real analyst campaign cannot pass and may spend provider budget on doomed calls. Request one of the accepted claim forms or add an explicitly validated structured-result path.
AGENTS.md reference: AGENTS.md:L4-L4
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8a3fbe6f5c
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| source = db.source(domain, connection=c) | ||
| if not source['body'].get('terms_acknowledged'): | ||
| raise PermissionError('Source access and terms must be confirmed before import') | ||
| paths = source_files(domain) |
There was a problem hiding this comment.
When the dataset volume is populated directly or an approved file is replaced after setup, this path fingerprints and adapts the current files without calling validate_source_receipt(). The source's existing terms_acknowledged flag is tied only to the configured source specification, so altered or unregistered bytes can become the latest snapshot while appearing approved; the repository-wide caller search shows receipt validation only in benchmark scripts, not application refreshes. Validate the receipt against the current bytes and adapter before reusing or saving a snapshot.
Useful? React with 👍 / 👎.
| if CONFIG['sources'][domain]['kind']=='classification' and r['target'] not in (None,0,1): | ||
| raise ValueError('Invalid binary source outcome') |
There was a problem hiding this comment.
When a support file contains a negative resolution_time_hours or a maintenance label is negative, the numeric parser preserves it and this validation accepts it because it checks only classification targets. The impossible target is then included in snapshots, observed summaries, and regression training/evaluation, corrupting reported means and model evidence. Enforce the nonnegative domain constraint for duration and remaining-life targets at import time.
Useful? React with 👍 / 👎.
| db.write('INSERT INTO backintel.analysis_principals(id,token_hash,role,domains) VALUES(%s,%s,%s,%s) ON CONFLICT(id) DO UPDATE SET token_hash=excluded.token_hash', | ||
| (role,hashlib.sha256(token.encode()).hexdigest(),role,list(CONFIG['sources']))) |
There was a problem hiding this comment.
When an upgrade adds a source to config/analysis.json while reusing the persistent database, catalog() creates the source row but this conflict path updates only credential hashes, leaving every existing principal's domains array unchanged. The grant API cannot recover from that state because it requires the acting manager to already hold every domain being granted, so no principal can obtain access to the new source without manual database intervention. Provide an operator-controlled way for seeding to grant newly configured domains.
Useful? React with 👍 / 👎.
| except (OSError, ValueError, PermissionError, RuntimeError) as error: | ||
| db.write('UPDATE backintel.analysis_sources SET body=body || %s WHERE id=%s', | ||
| (Jsonb({'last_refresh_error':safe_error(error),'last_checked_at':time.time()}),domain), connection=connection) |
There was a problem hiding this comment.
When two managers, or a manual import and the scheduled refresh, process the same source concurrently, _import_source() releases its source advisory lock before this error update is written. A successful refresh can therefore install and clear the source, after which the earlier failed attempt writes last_refresh_error and makes the healthy current snapshot appear failed; the success-side clearing has the same out-of-lock ordering problem. Record success or failure while holding the source lock, or predicate health updates on an import-attempt identity.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4c6b5e2de5
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| raise PermissionError("Only the configured loopback origin is allowed") | ||
|
|
||
| def scoped_artifact(self, store, grant, sha=None): | ||
| artifact = get_artifact(store, grant["audience"], sha) |
There was a problem hiding this comment.
When a task is revised while a token for the prior revision remains unexpired, this unscoped lookup selects the newest artifact for the audience across every task SHA; the following check then rejects that artifact, so the version-bound token can no longer open the default report or its exports even though its pinned artifact still exists. Pass grant["task_sha256"] through the existing task_sha parameter so the latest artifact is selected within the granted revision.
AGENTS.md reference: AGENTS.md:L4-L4
Useful? React with 👍 / 👎.
| raise ValueError("Boolean correction requires true, false or unknown") | ||
| response = {"status": "known", "value": value == "true", "distribution": {"true": int(value == "true"), "false": int(value == "false")}, "reason": form["reason"]} | ||
| else: | ||
| response = {"status": "known", "value": float(value), "distribution": None, "reason": form["reason"]} |
There was a problem hiding this comment.
When an operator corrects the equipment wear observation, any finite number—including negative values or values far above 10—is accepted here because the downstream validator checks only finiteness. Real Jev answers are explicitly constrained to the task's 0–signal_scale ordinal criteria, but an out-of-range correction is persisted, incorporated into semantic features and model refreshes, and exposed as accepted evidence; validate numeric corrections against the configured scale before saving them.
AGENTS.md reference: AGENTS.md:L4-L4
Useful? React with 👍 / 👎.
| with connection.transaction(): | ||
| row = connection.execute("""UPDATE backintel.capability_jobs SET cancel_requested=true, | ||
| state=CASE WHEN state IN ('queued','retry') THEN 'cancelled' ELSE state END,updated_at=now() | ||
| WHERE job_id=%s AND state NOT IN ('completed','failed','cancelled') RETURNING state""",(job_id,)).fetchone() |
There was a problem hiding this comment.
When a job exhausts its attempts, it enters failed, and both claim() and runnable() treat any earlier failed job as a permanent ordering blocker for every later job in that task. This cancellation predicate excludes failed jobs, while resume_failed() is not registered in the capability graph and refuses attempts at the five-attempt ceiling, so the public cancel operation returns unchanged and the task queue cannot ever advance without direct database mutation. Provide an audited abandon/cancel transition for failed jobs, or stop treating explicitly terminal failures as active predecessors.
AGENTS.md reference: AGENTS.md:L4-L4
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5f3864f3f9
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| result = extractor.classify_text(text, {'signal': labels}) | ||
| observation = {**expected, 'output': result} | ||
| cache.parent.mkdir(parents=True,exist_ok=True) | ||
| cache.write_text(json.dumps(observation)) |
There was a problem hiding this comment.
When a commerce or support model worker is killed, times out, or exhausts storage during this write, the cache file can remain truncated. Every retry then takes the cache.exists() branch and fails while parsing the same file before it can recompute the observation, permanently blocking the comparison until an operator deletes the cache. The comparison manifest is atomically replaced, but this per-observation cache is not; write a temporary file and replace it atomically, or recover from an unreadable cache.
AGENTS.md reference: AGENTS.md:L4-L4
Useful? React with 👍 / 👎.
| @app.get('/api/v1/sources') | ||
| def sources(p=Depends(access)): | ||
| p=db.authorize(p) | ||
| return [db.source(row['id']) for row in db.query('SELECT id FROM backintel.analysis_sources WHERE domain=ANY(%s) ORDER BY domain',(p['domains'],))] |
There was a problem hiding this comment.
When an upgrade removes a domain from config/analysis.json, its persistent source row and existing principal grants can remain, so this query still passes that domain to db.source(), which indexes CONFIG['sources'][domain] and raises an uncaught KeyError. The current refresh loop now filters retired domains, but this application route remains unfiltered, causing /api/v1/sources to return 500 for affected users and preventing the workspace's combined initial load. Filter against the current configuration or migrate retired source and grant rows.
AGENTS.md reference: AGENTS.md:L4-L4
Useful? React with 👍 / 👎.
| GROUP BY task_id,kind ORDER BY task_id,kind""",(task_ids,)).fetchall() | ||
| modes = {r[0] for r in connection.execute("""SELECT DISTINCT body->>'implementation_mode' | ||
| FROM backintel.capability_evidence WHERE task_id=ANY(%s) AND kind='model'""",(task_ids,))} | ||
| return {"status":"failed" if errors else "passed" if jobs and not pending and not running else "running", |
There was a problem hiding this comment.
When one or more demonstration jobs are cancelled and no queued, running, retry, or pending-trigger work remains, errors is empty because it includes only failed jobs, so this expression reports passed. Consequently --status exits successfully, and wait_for_completion() can proceed to replay and packaging even though a cancelled stage never produced its result; treat cancellation as an error and require the expected jobs to be completed before returning passed.
AGENTS.md reference: AGENTS.md:L4-L4
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ba71655e95
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| if original["body"]["question"]["type"] != "boolean" and response["status"] == "known": | ||
| values = [response["value"]] + (response.get("distribution") or {}).get("values", []) | ||
| if any(not 0 <= value <= task["policy"]["signal_scale"] for value in values): |
There was a problem hiding this comment.
For a valid task containing a numeric question other than policy.signal_question—for example, a boolean risk signal with signal_scale=1 plus a numeric temperature question—this rejects corrections above 1 even though the task schema and validate_response() impose no such bound on that question. This prevents reviewers from correcting otherwise valid observations; restrict this check to the configured signal question or define and enforce per-question numeric bounds.
Useful? React with 👍 / 👎.
|
The account paying for this security review has reached its Codex usage limits. The payer can check the Codex usage dashboard. For personal accounts, using credits requires enabling “Use credits for security reviews” in Code review settings. If you do not manage the paying account, contact this repository's admins. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: eda9818b54
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| @app.post('/api/v1/runs/{identity}/cancel') | ||
| def cancel_run(identity:str,p=Depends(access)): | ||
| r=db.run(identity) | ||
| db.authorize(p,db.run_domain(r),('manager',)) |
There was a problem hiding this comment.
If another manager disables this actor, expires the credential, or removes the run's domain after this standalone authorization but before cancel() executes, the transaction locks only the run and job, so the revoked actor can still set cancel_requested and terminate queued or running work. Recheck and lock the current principal's enabled, expiry, role, and domain state in the same transaction as the job cancellation.
AGENTS.md reference: AGENTS.md:L4-L4
Useful? React with 👍 / 👎.
| raise ValueError("Measure violates finite numeric/null contract") | ||
| else: | ||
| measures[measure["id"]] = value | ||
| return {**fact, "measures": measures, "source": row, "source_format": source_format} |
There was a problem hiding this comment.
When an admitted JSON, CSV, or text row contains a field that is not referenced by the task's fields or measures mappings, this retains that field in the raw source object. create_artifacts() subsequently copies the complete source record into each eligible audience artifact, and both /evidence/{sha} and packaged evidence JSON expose it, so an unrelated sensitive column can cross the intended contract boundary despite never being selected as a fact or measure. Persist or publish only the explicitly mapped source projection.
AGENTS.md reference: AGENTS.md:L4-L4
Useful? React with 👍 / 👎.
| check_limits(); method_start=time.monotonic() | ||
| artifact=directory/f'{route}-{feature_set}.joblib' | ||
| if prior is not None: | ||
| package=joblib.load(artifact) |
There was a problem hiding this comment.
When a local process can modify the writable BACKINTEL_MODEL_DIR volume between training runs, it can replace a cached .joblib file and update the adjacent manifest.json with the replacement hash. The validation at lines 195–198 trusts hashes from that same editable manifest, so this joblib.load() then executes attacker-controlled pickle code inside the runtime container, which holds database and potentially provider credentials. Anchor cache hashes in immutable database evidence or use a non-executable model format before loading cached artifacts.
AGENTS.md reference: AGENTS.md:L4-L4
Useful? React with 👍 / 👎.
|
The account paying for this security review has reached its Codex usage limits. The payer can check the Codex usage dashboard. For personal accounts, using credits requires enabling “Use credits for security reviews” in Code review settings. If you do not manage the paying account, contact this repository's admins. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5a89b3e21d
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| if domain in ('commerce','support'): | ||
| files.append('observations.json') | ||
| artifacts=prior.get('artifacts',[]) | ||
| expected={name:file_sha(directory/name) for name in files} |
There was a problem hiding this comment.
When PostgreSQL evidence survives but files under BACKINTEL_MODEL_DIR/Analysis are lost or partially removed, this hashes every required cache path and raises before reaching any retraining path. Because the immutable comparison evidence keeps selecting the same prior on every retry, model_job() can never reconstruct a predictor for that snapshot, leaving prediction unavailable until the source or configuration changes. Fresh evidence in the current code is the new trusted_comparisons branch that unconditionally treats database evidence as requiring an intact filesystem cache; treat missing or mismatched files as a cache miss and rebuild them safely.
AGENTS.md reference: AGENTS.md:L4-L4
Useful? React with 👍 / 👎.
| if result['state'] in ('failed', 'cancelled'): | ||
| with c.transaction(): | ||
| _finish_run_failure(c, job_id, result['state'], result.get('error', 'Job stopped before acceptance')) |
There was a problem hiding this comment.
If the dispatcher is killed after execute() commits a job as failed or cancelled but before this separate transaction calls _finish_run_failure(), the associated analysis run remains running. On restart, dispatch() selects only queued, retrying, or running jobs and reconciles only expired jobs still in running, so the already-terminal job is never revisited; _admit_run() also keeps returning the apparently running run instead of allowing resubmission. Reconcile terminal jobs with nonterminal analysis runs during dispatcher startup, or commit both state transitions atomically.
AGENTS.md reference: AGENTS.md:L4-L4
Useful? React with 👍 / 👎.
|
The account paying for this security review has reached its Codex usage limits. The payer can check the Codex usage dashboard. For personal accounts, using credits requires enabling “Use credits for security reviews” in Code review settings. If you do not manage the paying account, contact this repository's admins. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4e06a81b51
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| for method in receipt['methods']] | ||
| detail['dependencies'] = dependencies | ||
| return status, 'Native partial real prediction comparison; missing routes remain excluded', detail | ||
| if receipt.get('schema') != 'backintel-answers/v2' or not receipt.get('scenario_hash'): |
There was a problem hiding this comment.
When a real analyst campaign is produced by scripts/analysis_benchmark.py, it stamps the receipt with SUITE_VERSION, which is currently backintel-answers/v3 in scripts/analysis_benchmark_support.py; this check rejects every such receipt as missing because it accepts only v2. Consequently all current --analyst-receipt inputs are marked blocked before receipt_identity_error() can validate them, so a successful paid grounded-answer campaign can never satisfy campaign acceptance. Compare against the helper's SUITE_VERSION or update this check to v3.
AGENTS.md reference: AGENTS.md:L4-L4
Useful? React with 👍 / 👎.
| return 'blocked', 'Analyst scenario coverage or independent oracle is incomplete', None | ||
| if receipt.get('charge_status') not in ('reconciled', 'settled', 'measured'): | ||
| return 'blocked', 'Analyst charges are not reconciled', None | ||
| status = 'passed' if all(a.get('correct') is True and a.get('status') == 'passed' for a in answers) else 'failed' |
There was a problem hiding this comment.
Once current analyst receipts are admitted, a modified or fabricated receipt can make every domain pass by setting each answer's correct flag and status, supplying any truthy oracle, and providing self-consistent claimed charges. This line derives acceptance solely from those receipt-controlled flags without checking the expected/actual values, cited calculations, evidence IDs, or run identities that score_answer() validates at production time, so the campaign can report that real answers match independent oracles when they do not. Re-run the structured answer/evidence checks or require independently verified evidence rather than trusting these booleans.
AGENTS.md reference: AGENTS.md:L4-L4
Useful? React with 👍 / 👎.
| parser.add_argument('--comparisons',action='store_true',help='Also run full predictor comparisons') | ||
| parser.add_argument('--candidate-manifest',help='Clean host candidate_identity JSON; verify complete runtime file set and bytes inside an image without Git') | ||
| parser.add_argument('--execute-real',action='store_true',help='Explicitly execute hosted analyst calls under existing campaign limits') | ||
| parser.add_argument('--output',default=str(Path(os.getenv('BACKINTEL_MODEL_DIR','/models'))/'Analysis/benchmark-evidence.json')) |
There was a problem hiding this comment.
When /models/Analysis/benchmark-evidence.json already contains a successful paid --execute-real campaign, running the documented no-flag raw-source preflight uses this same default path and then writes a new blocked receipt over it. check_analysis.py reads only this fixed filename, so an otherwise harmless preflight destroys the validator's access to the successful campaign evidence and can force another paid run. Store preflight output separately or use attempt-specific receipt paths while preserving the last accepted real receipt.
AGENTS.md reference: AGENTS.md:L4-L4
Useful? React with 👍 / 👎.
|
The account paying for this security review has reached its Codex usage limits. The payer can check the Codex usage dashboard. For personal accounts, using credits requires enabling “Use credits for security reviews” in Code review settings. If you do not manage the paying account, contact this repository's admins. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 642557bebd
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| cancelled = c.execute('SELECT cancel_requested FROM backintel.capability_jobs WHERE job_id=%s FOR UPDATE', (db.run(run_id)['job_id'],)).fetchone()[0] | ||
| if cancelled: | ||
| raise InterruptedError('Import cancelled') | ||
| db.save_snapshot(domain, identity, body, rows, connection=c) |
There was a problem hiding this comment.
When the registered analysis_refresh graph calls refresh(), it invokes import_source() without a run_id, so the only manager check occurs before the potentially long receipt validation and adaptation. If that selected manager is disabled, expires, or loses the domain during this work, the run_id-guarded recheck is skipped and this line still publishes the snapshot under revoked authority. Lock and revalidate the principal in the publication transaction for scheduled imports as well.
AGENTS.md reference: AGENTS.md:L4-L4
Useful? React with 👍 / 👎.
| c.execute('INSERT INTO backintel.analysis_goals(id,owner,domain,version,body) VALUES(%s,%s,%s,1,%s)', (identity, actor['id'], domain, Jsonb(body))) | ||
| c.execute('INSERT INTO backintel.analysis_goal_versions(goal_id,version,body,actor) VALUES(%s,1,%s,%s)', (identity, Jsonb(body), actor['id'])) |
There was a problem hiding this comment.
When another manager disables this actor, expires the credential, or removes the domain after the preliminary db.authorize() call but before this transaction, the public goal-creation route still inserts a permanently attributed standing goal because the transaction never checks or locks the principal. Perform the current manager/domain check in the same transaction as both goal inserts.
AGENTS.md reference: AGENTS.md:L4-L4
Useful? React with 👍 / 👎.
| tfidf=TfidfVectorizer(max_features=2000,min_df=2) | ||
| xt=tfidf.fit_transform([r['text'] for r in train]) |
There was a problem hiding this comment.
When an otherwise valid commerce or support snapshot has blank training text, or no token appears in at least two training documents, this min_df=2 vectorizer raises an empty-vocabulary error. Because the optional simple-text baseline runs before the structured CatBoost/TabICLv2 loops, the whole model subprocess fails and managers cannot produce or promote a facts-only candidate even though the structured features and labels are sufficient; skip the unavailable text method or record it as unsupported instead of aborting the comparison.
AGENTS.md reference: AGENTS.md:L4-L4
Useful? React with 👍 / 👎.
|
The account paying for this security review has reached its Codex usage limits. The payer can check the Codex usage dashboard. For personal accounts, using credits requires enabling “Use credits for security reviews” in Code review settings. If you do not manage the paying account, contact this repository's admins. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 333a6e7751
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| p=db.authorize(p) | ||
| result=db.query('SELECT * FROM backintel.analysis_goals WHERE domain=ANY(%s) ORDER BY created_at DESC',(p['domains'],)) | ||
| for g in result: | ||
| source=db.source(g['domain']) |
There was a problem hiding this comment.
When an upgrade removes a domain from config/analysis.json while persistent goals and principal grants still reference it, this call passes the retired domain to db.source(), which indexes CONFIG['sources'][domain] and raises KeyError. Although /api/v1/sources now filters retired domains, /api/v1/goals still returns 500 and the frontend's combined initial refresh fails, preventing affected users from loading goals for active domains as well; filter this query against the current configuration or migrate retired goals and grants.
AGENTS.md reference: AGENTS.md:L4-L4
Useful? React with 👍 / 👎.
| db.authorize(p,db.goal(identity)['domain'],('manager','analyst')) | ||
| if body.kind not in ('comment','correction') or (body.run_id and db.run(body.run_id)['goal_id']!=identity): | ||
| raise ValueError('Invalid review context') | ||
| db.write('INSERT INTO backintel.analysis_reviews(goal_id,run_id,actor,body) VALUES(%s,%s,%s,%s)',(identity,body.run_id,p['id'],Jsonb(body.model_dump()))) |
There was a problem hiding this comment.
If another manager disables this actor, expires the credential, or removes the goal's domain after the authorization at line 303 but before this separate write, the revoked manager or analyst can still persist a review attributed to themselves. Perform the current role/domain check and review insertion in one transaction while locking the principal, as the other application write routes do.
AGENTS.md reference: AGENTS.md:L4-L4
Useful? React with 👍 / 👎.
|
The account paying for this security review has reached its Codex usage limits. The payer can check the Codex usage dashboard. For personal accounts, using credits requires enabling “Use credits for security reviews” in Code review settings. If you do not manage the paying account, contact this repository's admins. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 59f582ec0a
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| c.execute('SELECT id FROM backintel.analysis_runs WHERE id=%s FOR UPDATE', (identity,)) | ||
| manager=c.execute("""SELECT id FROM backintel.analysis_principals WHERE id=%s AND enabled | ||
| AND (expires_at IS NULL OR expires_at>clock_timestamp()) AND role='manager' | ||
| AND %s=ANY(domains) FOR UPDATE""", (p['id'],db.run_domain(r))).fetchone() |
There was a problem hiding this comment.
When a manager cancels while a worker is publishing the same run, this transaction locks the run and then the principal, while analysis_service.handle() locks the goal/principal and then updates the run. Each transaction can therefore hold one row while waiting for the other, causing PostgreSQL to abort either the cancellation with a 500 or the completed analysis publication after provider work has already occurred; acquire the principal and run locks in the same order on both paths.
AGENTS.md reference: AGENTS.md:L4-L4
Useful? React with 👍 / 👎.
| kwargs = {"thread_count":2} if body["route"] == "catboost" else {} | ||
| return float(estimator.predict(values,**kwargs)[0]) |
There was a problem hiding this comment.
When an actual CatBoost or TabICLv2 equipment model extrapolates below zero, this returns the negative regression value unchanged. score() persists that value and the audience artifact renders it as mm/s even though the configured target is vibration magnitude, so reports can publish physically impossible forecasts; constrain or reject regression outputs according to the target domain as the separate hosted-analysis predictor path now does.
AGENTS.md reference: AGENTS.md:L4-L4
Useful? React with 👍 / 👎.
| task = load_scenario(name)["task"] | ||
| task["id"] = store.task_id | ||
| task_record = register_task(store,task) |
There was a problem hiding this comment.
When a deployed simulation task changes while its PostgreSQL task store is retained, pending or newly identified work loads the new contract here but always calls register_task() with the default available_at=0. The existing contract was also registered at zero, so the strict revision check rejects the new contract and durable scheduled jobs cannot resume after the upgrade; pass a monotonic revision time or bind each queued job to the task SHA it was created for.
AGENTS.md reference: AGENTS.md:L4-L4
Useful? React with 👍 / 👎.
2 participants
Purpose
Publish the recovered BackIntel demo proof of concept. Users can replay the recorded business demonstration and exercise the analysis workspace with clearly labeled fixtures.
Behavior and controls
Recheck and lock the run owner's current access before publishing analysis, training, or imports. Reject non-finite restored predictions and mismatched benchmark-runner fingerprints.
Align displayed reports with the same event ordering used for prediction features. Launch the workspace through the platform browser, with a local URL fallback. Ship trusted-checkout launch instructions inside the demo ZIP.
Bind numeric counts to their stated metric and missing-data field. Version cached evaluations by scoring code, and validate real-comparison code fingerprints using the same digest as their producer.
Keep mutable demo reviews in user-local storage outside the distributed package; reject linked state and seed only from verified records.
Bind cached trained-model packages and Decide observations to current inputs, model files, configuration, and implementation. Hold provider authorization through dispatch, preserve healthy source status on import cancellation, and keep cleared attention episodes closed during staleness checks.
Preserve source and question metadata when displaying corrected observations. Close SQLite connections after workspace reads and writes, including in the recorded demo package.
Validate source receipts before standalone predictor benchmarks. Require forecast, probability, and likelihood answers to cite prediction evidence in estimate findings.
Reject publication when the source snapshot or approved model changes during a run. Lock and recheck the run owner's current grant before dispatch. Commit job cancellation and run cancellation together. Validate every displayed limitation.
Discard late goal-detail responses, reset the workspace when credentials change, and clear cached records when access is denied.
Preserve the full-product contract in
tabellio.validation.json. Use the owner-approvedtabellio.demo.validation.jsonfor the demo merge.Keep imports, corrections, goal admission, candidate publication, and accepted findings consistent across cancellation, retries, and interruptions. Duplicate requests reuse active work without blocking completion. The independent job supervisor polls cancellation and terminates the complete worker process group, including native model subprocesses.
Bind source consent to the displayed specification. Before a live benchmark acknowledges source terms, validate the receipt's schema, domain, current file hashes, snapshot, adapter output, and source-specification fingerprint. Authorized repeat imports refresh the receipt without replacing matching data. Recheck consent and goal eligibility during admission, provider dispatch, and publication. Reject stale model candidates and invalidate corrected training evidence.
Bind run reuse and answer freshness to the runtime implementation and model configuration, including imported predictor code. Reactivating an earlier model restores its exact still-current saved answer.
Withhold held-out metrics, predictions, and split membership across model, run, finding, and evidence API responses until promotion. Preserve calibration evidence for model selection and viewer access restrictions.
Validate cached comparison identity, domain, snapshot, dependencies, split membership, and the complete expected artifact set. Recompute metrics and predictions from the pinned data and cached models instead of trusting claimed scores in a mutable manifest. Scoring validates only the approved predictor's dependencies, so a missing unused TabICLv2 checkpoint cannot disable CatBoost.
Bind business-demo reviews to the full visible case and analysis. React and Python clients submit that context fingerprint; changed findings or predictions require a fresh review even when source evidence is unchanged. Historical source-only review records remain stored but cannot approve unknown analysis.
Ground facts, estimates, numbers, and rankings in typed, cited calculation evidence. Bind numeric findings to the named table group, require separate numeric findings for different groups, and construct the displayed summary from the validated findings. Source inspection counts empty categorical cells as missing while preserving valid zero and false values. Keep historical results distinct from current evidence.
Revalidate current approval, expiry, and task/source/question/model scope before accepting a cached provider reply; reject replay under a different authorization identity. Enforce scoped access, source and model-use approval, provider spending ceilings, unresolved-charge reconciliation, bounded retries, task ordering, and immutable promoted routes.
Require PostgreSQL authentication and capability API credentials. Package only tracked source and verify the complete immutable package file set. Exclude private audience grants.
Refresh the recorded playback code, both demo clients, frontend, archive hashes, and validation receipt. Retain the historical source bundle and recording, with a distribution note explaining the patch. Original archives remain preserved outside Git.
Show accepted observation corrections in refreshed business-demo snapshots, including results produced after the original staged run.
Restore missing source receipts on identical-data imports without replacing source files.
Select the bounded Home Credit cohort by stable identity hashes across the full CSV stream. The independent raw-data calculation uses the same declared cohort rule and checks row-order invariance.
Verify and serve extracted demos through the trusted repository checkout. Compare the archive with its trusted receipt, match the extracted manifest, and verify all immutable files before serving. The launcher imports no Python code from the extracted package. Verification rejects linked mutable review directories, databases, and SQLite supporting files before serving; both the trusted launcher and bundled playback contain this protection.
Validation
Current candidate:
59f582ec0a4218feae9c263d8909475cc0dae9c8.The latest changes revalidate and lock current manager/analyst authority inside review insertion, and filter retired domains from goal listings. The campaign suite passed, including a revoked-analyst write and retained retired goals alongside an active-domain goal. Dead-code and diff checks passed; changed relationships were refreshed and checked against source. Local evidence:
/Users/hudson/Library/Application Support/BackIntel/Evidence/FinalNativeReview20261007/ReviewAuthorityRetiredGoals/.The preceding publication-authority and text-fallback fixes also passed local and hosted checks: goals and snapshots require current manager authority in their publication transactions, and empty/unusable text methods are marked unsupported while structured models and facts-only cache reuse remain available.
Earlier review fixes cover trusted model-cache hashes and exact-byte loading, recovery after lost caches or interrupted dispatch, declared source-column boundaries, task-specific correction limits, and independent benchmark-answer validation. All 30 offline benchmark tests passed, including fabricated answer/evidence cases and preservation of paid-run receipts during no-call preflight.
All five hosted checks passed on preceding commit
333a6e7. This candidate requires its own passing hosted checks and native code/security reviews before merge. Security review has reported the paying account's Codex usage limit (bot report); an older completed security review does not satisfy the current-head gate.This is a demo PoC. Checks used no paid model calls; live-model and full-product acceptance remain unverified. The historical 341-file demo archive is unchanged. Verified SHA-256:
e75f006136a1b0e9eed83bd5de316c90d166ccbfa265dd54a09de0c5cf437fe7.Demo and migration boundary
No paid provider calls, deployment, or release publication is included. Fixtures do not establish live-model quality, autonomous data preparation, or measured business value. Actual Jev, CatBoost, and TabICLv2 acceptance remains governed by the separate full-product contract. OlistV2 remains removed.
Source specifications require fresh acknowledgment when first fingerprinted or changed. Older source receipts require a fresh authorized import to bind them to the current dataset and terms. Previous answers become stale after implementation/configuration changes. Older predictor packages must be prepared again under the current implementation. Existing services, data volumes, and recovery work are preserved. Before recreating old containers, configure matching database passwords and a capability API token as documented; repository fixes do not secure already-running containers.
To replay the historical recording, extract
docs/demo/Packages/BackIntelDemoBusinessV4.zipinto a new folder. From the trusted repository checkout, runpython3 -m scripts.package_business_demo --package-root /path/to/BackIntelDemo --check, then the same command without--check. Python 3.10+ is sufficient; no model keys or database server are needed. The embeddedRunDemo.py --checkcannot independently establish trust in an altered or redistributed package.