Skip to content

Latest commit

 

History

History
472 lines (284 loc) · 39.8 KB

learning.md

File metadata and controls

472 lines (284 loc) · 39.8 KB
description Understand scoped learning, knowledge-first lookup, failed-attempt evidence and structured agent handoffs, with explicit implementation boundaries.

Learn from evidence and reuse knowledge safely

A useful lesson explains which question it addresses, what was observed and where that observation still applies. forge already retrieves required flycanon context and retains verified repository memory. The initial learning implementation adds deterministic eligibility and exact question-to-evidence associations. A receipt-backed same-run cap also bounds identical repair proposals without replacing verification.

!!! info "Implementation boundary"

Learning uses scoped PostgreSQL records, evidence eligibility, native knowledge-first decisions and explicit flycanon projection. An opt-in lookup can retrieve a citation from the verified projection of an exact eligible local record. Semantic similarity cannot authorize related lessons. Studio includes policy configuration, record and projection inspection, reviewed evidence invalidation, and verification of an existing indexed source with exact-request recovery. Source submission in Studio and triggering ingestion remain open, alongside cross-run classified negative repair circuits and sharing between projects. Generic failed checks record an observed outcome; they do not prove that an approach fails everywhere.

Configure and inspect local learning

Learning policy applies to new runs; admitted runs retain their captured policy. An administrator can inspect it with forge learning policy and save a reviewed JSON policy with forge learning configure policy.json:

{
  "enabled": true,
  "projected_retrieval": false,
  "max_age_hours": 72,
  "max_candidates": 8,
  "max_repeated_attempts": 2,
  "max_lookup_calls": 30
}

In Studio, open Administration > Learning and choose Edit learning settings. Review Knowledge, set the bounds in Limits, then inspect Review before Save learning settings. Only a factory administrator can change this policy. Enable verified learning controls whether new runs can use it; turning it off keeps the projection preference saved but inactive. The review distinguishes policy for new runs from the snapshots held by existing runs, and preserves the repeated-attempt setting. Compatible native runtimes apply that setting to receipt-backed repair proposals; older servers keep their existing repair budgets. If saving has an uncertain outcome, read the current policy before trying another write.

Learning knowledge selection

Knowledge: choose whether new runs may use verified learning and read an eligible record's flycanon projection. This controlled fixture shows the administrator form, not a live source lookup.
Follow limits, review and saved settings

Learning retrieval limits

Limits: set evidence age, fallback source results and lookup calls. These bounds are separate from model, command and repair budgets.

Learning policy review

Review: inspect the effective settings before saving. Disabling learning retains the projection preference for a later admission.

Saved learning settings

Saved settings on a phone: confirm the persisted policy before admitting a new run. The fixture's successful save illustrates the flow; existing runs keep their captured settings.

The values shown are defaults. The native repair-proposal cap uses the frozen repeated-attempt setting as an additional upper bound. Its effective limit is at most two, even when the saved value is three; the existing no-progress stop and repair/clarification budgets stay active. A value of one can hold the next coding proposal after one verified failed round. Disabling the policy affects later admissions. Required framework knowledge remains a separate setting and can still block generation when unavailable.

Set projected_retrieval to true to let newly admitted runs read the separate learning workspace for an exact eligible local record. Older snapshots without this permission retain their prior behavior. Lookup uses existing credentials and never creates a workspace, submits a source, mints a token or expands its workspace allowlist. A token needs both query and source-read permission for that workspace. An unavailable optional projection produces an explicit diagnostic while ordinary required source retrieval and local evidence checks continue.

The selected projection must be verified and match the current project/repository scope, local envelope and original source bytes. The source must still be indexed and return a matching citation. Both its original captured knowledge configuration and the consuming run's configuration must resolve to identical frozen settings, including tenant, endpoint, credentials and CA. Capture revision IDs remain separate provenance references; different random IDs do not establish configuration drift. PostgreSQL record and projection identities are rechecked at fresh admission and cached reuse, with current execution authority and freshness. Remote reads finish before either shared record lock is held.

Action CLI command Factory MCP tool
Read policy forge learning policy get_learning_policy
Save policy forge learning configure policy.json configure_learning_policy
List authorized records forge learning records REPOSITORY_ID list_learning_records
Invalidate evidence forge learning invalidate RECORD_ID --reason REASON --request-id REQUEST_ID invalidate_learning
Read an exact invalidation receipt forge learning invalidation-receipt RECORD_ID --request-id REQUEST_ID get_learning_invalidation_receipt
Read projection state forge learning projection RECORD_ID get_learning_projection
Submit once forge learning submit RECORD_ID --request-id REQUEST_ID submit_learning_projection
Observe an admitted source forge learning observe RECORD_ID --request-id REQUEST_ID --source-id SOURCE_ID observe_learning_projection
Read an exact source observation forge learning observation-receipt RECORD_ID --request-id REQUEST_ID get_learning_projection_observation_receipt

Record, repository and request IDs use 32 lowercase hexadecimal characters; the source ID is the actual flycanon identifier. Each new action needs an explicit request identity. Inspect the record's source, checks and scope before submitting it. Submission requires verified local outcome evidence and current review authority. If its effect is unknown, observe the known source or retain the unresolved state; another submission must not replay it. The API and SDK expose the same actions. These commands do not create a cross-project grant or a new model session.

Inspect an uncertain invalidation

Keep the factory URL, actor, record ID, request ID and reason with a pending invalidation. If the response is lost, use the receipt command above, Python learning_invalidation_receipt(record_id, request_id), or GET /api/v1/learning/records/{id}/invalidations/{request_id}. These reads require current project review authority; unassigned records require a factory administrator.

A receipt identifies the exact committed event, record, repository, origin run, request, reason, sequence, original timestamp and event digest. event_state describes that historical invalidation; current_record_state describes the record when read. New invalidation events retain their original actor_id. Legacy events without attribution return null; reading or repeating an existing request does not replace its actor or rewrite its bytes or digest.

Compare the receipt with the retained request identity and reason. A record invalidated by another request does not establish the pending request's outcome. A 404 means that exact invalidation has not been observed; it does not prove an in-flight writer cannot commit. Retain the uncertainty for operator review. The receipt read never submits, repeats, abandons or promotes a decision. A retained reason that has since become a registered credential is refused by the current read fence while the original private event remains intact.

Decide whether a search is needed

A verified local outcome is captured outside a caller transaction. The factory first reads the verified run and receipts in a short unit, then inspects Git, scans source, observes declarations and admits credentials outside the writer unit. Under the final run lock, it rechecks the actor, execution permission, run, receipts, configuration and ownership. The record, learning event, run event and pending projection commit together; refusal, cancellation or a failed write cannot leave a partial learning record.

Failed-check observations prepare source outside the writer unit and recheck it before retaining an unverified candidate; they neither create a verified projection nor suppress repair.

Bind the question to a task and source; retrieve authorized flycanon candidates; evaluate ledger evidence; either reuse an exact eligible outcome or record a bounded search; verify before retaining a lesson.

Open full diagram · Portrait layout

A missing reference and an unavailable required knowledge service have different outcomes. Required service failure stops the task; it does not silently trigger an internet search.

The intended lookup begins with the current task and inspected source. It consults the authorized repository and framework knowledge scopes, then resolves candidates against the authoritative learning records. A similarity score can help find a record. Eligibility comes from its scope, evidence, source and version assumptions.

A verified exact match can supply its bounded local procedure result without a new provider request. Missing knowledge, stale references or a changed dependency context produce an explicit search reason. A requested refresh must perform a fresh lookup even if an older result exists. Credentials stay in private provider configuration, and external results retain their untrusted-reference status.

No result is silently moved from another repository into the task. If a known framework version differs from the product's version, that difference must remain visible. An absent version is unknown; the system cannot infer compatibility from a framework name alone.

Review saved records in Studio

Open Knowledge > Learning records and choose an authorized repository. The factory returns records visible to your current project permissions. Local evidence remains readable when remote knowledge retrieval is disabled; denied access is shown explicitly.

Select a record to inspect its saved state, question, supported statement, framework version, scope and freshness dates. Verified local outcomes show their recorded verification time. Failure candidates show when the check was observed. Identifiers and verification receipts contains the exact source tree, dependency and question digests, origin run and native receipt references. The filters search only the returned records.

Verified evidence describes a saved observation. Current reuse also requires matching scope, framework and dependency versions, question, source, freshness and intact receipts. A candidate failure cannot authorize a verified solution. Invalidated and superseded evidence remains available for review and is excluded from reuse.

The projection panel reads the selected local outcome's saved flycanon status and provenance. A pending submission, admitted source, verified projection and unknown outcome are separate states. An unknown outcome is never retried by refreshing this view. Verifying a known existing source uses the separate reviewed action below. A denied or unavailable projection does not replace the local evidence with a claim about remote verification.

Refresh records performs an authorized read. It does not submit a source or record a new source observation. Evidence invalidation uses the separate review below. A generic status reload cannot identify the exact receipt of a lost invalidation request.

Learning records for the selected repository

Choose a repository, then a saved record. The list distinguishes a verified local outcome from an observed failure. This guide uses controlled API fixtures; no knowledge service or command runs during capture.
Inspect projection, provenance and a failed-check candidate

Saved flycanon projection

The fixture's projection has a recorded source and citation. The notice keeps saved verification separate from current reuse eligibility.

Native receipt references beside the retained record selector

Open identifiers and verification receipts to inspect the source and journal references. The desktop selector stays reachable while the detail scrolls.

An observed failed-check candidate

A rejected check records an observation in its own scope. This candidate cannot authorize a verified solution or a new projection.
Follow the same inspection on a phone

Learning record selection on a phone

The narrow layout places selection before the detail. Filters remain available above the returned records.

Saved projection on a phone

Selecting a record focuses its detail. Read the saved projection state before opening its identifiers.

Verification references on a phone

Long identifiers wrap in the detail. Scroll to the receipt references within the main content area.

Failure candidate on a phone

The candidate retains its observed purpose and date. Reviewing it performs reads without replaying a submission.

Verify an existing indexed source

Select a verified local outcome whose projection is Source admitted or Outcome unknown, then choose Verify existing source. Current project review permission is required; unassigned records require a factory administrator. A recorded source identifier is filled in for you. If no source was recorded, enter the actual identifier from the original learning workspace. Do not guess an identifier or enter a URL, token or private credential.

Choose Review source and check the source and record. Source and request details holds the factory, actor and exact target references. Acknowledge the operation, then choose Verify source. The factory reads the existing source and its citations using the saved target. It checks the admitted content, metadata and indexed evidence before recording a local observation. This action does not create a source or start ingestion. Unsubmitted, running, abandoned and already verified projections do not offer another observation here.

Before sending a new verification, Studio saves and reads back its request identity and original scope in the current tab. It retains a fingerprint of the selected source instead of saving unconfirmed source text. A storage failure blocks the request. After a successful response, Studio reads the exact authorized attempt receipt. A verified attempt must also match its original observation event, source and target before confirmation. The confirmation describes that historical observation separately from current record and projection states; it does not grant reuse eligibility.

If the response is lost, keep the retained request. After navigation or reload, select the same record and choose Check original verification. This only reads the exact receipt. A generic verified projection, another actor or request, a different source or target, and a 404 cannot confirm the original observation or permit replay. A 404 means the event has not been observed and may still be in flight. Legacy receipts with incomplete original provenance remain unresolved. A denied read clears private evidence while retaining the request marker.

A confirmed local refusal appears as Original request was not dispatched. Choose Review and start a new verification to check current access and the saved projection again. Studio must read the same immutable refusal receipt with current review authority, then preserve the original request and receipt in this tab before offering another review. The next verification receives a new request identity and requires your confirmation. It is never submitted automatically.

A missing receipt, an unknown outcome or a request still being prepared or dispatched keeps the original identity pending. A 404 cannot authorize another request. A retained version 1 request can still be confirmed by its original observation receipt, but it cannot enter this correction flow. If the original target changes or browser storage cannot retain its history safely, correction stays blocked.

Authorized operators can also inspect the event through the receipt command above, Python learning_projection_observation_receipt(record_id, request_id), or GET /api/v1/learning/records/{id}/projection/observations/{request_id}. The receipt preserves the original event identity, actor, timestamp, sequence, digest, source and citation references. No receipt read submits, ingests, promotes or shares knowledge.

These version 0.2.11 illustrations follow one fictional source verification whose response is lost. The first exact receipt read returns 404; the next returns the matching historical observation. The fixture also represents a later invalidation, so the current record is excluded from the active filter while its confirmation remains available. The later invalidation is separate from source verification. No provider or knowledge service is called.

Choose the existing source to verify

Source: review the saved record and its workspace purpose. The recorded source identifier is filled in. This action verifies existing indexed content without creating or ingesting a source.
Review the source and recover its exact observation

Review an existing source verification

Review: read the source and acknowledge the local observation. Factory, actor and target references are available in Source and request details. Verify source sends one request after its identity has been retained.

Unresolved source verification after a missing exact receipt

Unresolved: the response is lost and the first exact receipt read returns 404. Check original verification reads the same request again; it cannot repeat the observation.

Matching historical observation and separate current evidence state

Confirmed: the later receipt matches the original actor, source and target. The historical observation stays visible when a later invalidation excludes the current record from the Verified evidence filter. Reuse still requires eligible evidence.
Verify an existing source and recover its receipt on a phone

Check an existing source identifier on a phone

Source: use the actual identifier from the admitted workspace. Leave URLs and credentials out of this field.

Acknowledge an existing source verification on a phone

Review: the acknowledgment and Verify source action stay reachable. Expand the technical details when you need the exact original references.

Read an unresolved source observation on a phone

Unresolved: keep the retained identity and read its exact receipt. A missing event cannot confirm the observation or permit another write.

Read the matching source observation on a phone

Confirmed: inspect the historical observation separately from current record and projection states. This fictional journey sends one write, reads the same exact attempt twice and then reads its matching observation event.

Invalidate saved evidence

Select a candidate or verified record, then choose Invalidate evidence. Current project review permission is required; unassigned records require a factory administrator. Enter a specific reason without credentials, choose Review invalidation, and acknowledge that the decision excludes the evidence from reuse while keeping its history. Confirm invalidation submits one decision. Keep evidence or closing the review before confirmation submits nothing.

Before sending, Studio saves and reads back the original factory URL, actor, repository, project, origin run, record, request ID and reason in the current tab's storage. Unreadable or corrupt storage blocks submission. The marker survives navigation and reload in that tab; it is not a browser backup. Keep the original request details before closing a tab with an unresolved decision.

Even a successful POST is followed by the exact authorized receipt read. If the response is lost, choose Check original request. That control only reads; it never resubmits, abandons or replaces the pending request. An invalidated record, another actor's event, a different reason, a mismatched request or a 404 cannot confirm the original decision. A 404 means the request has not been observed and may still be in flight. Keep the uncertainty for operator review.

Original invalidation confirmed requires the matching record, repository, run, request, reason and original actor, with a valid event identity, timestamp, sequence and digest. Legacy unknown attribution cannot confirm a newly retained Studio request. The historical invalidation and current record state are shown separately. Reload requires a fresh authorized receipt read. A denied receipt read clears private evidence from the view while retaining the marker; changing actors or factories cannot reconcile another context's request. The CLI, SDK and MCP receipt reads remain available for authorized operator inspection.

If invalidation removes the selected record from the active filter, the filtered list and count update while its decision remains open. Close selected record dismisses that detail and returns focus to the filter control. Selecting it again preserves the pending request identity and requires its exact receipt read.

These captures follow one fictional request whose response is lost. The first receipt read returns 404; a later read returns the matching event. Navigation, forms and retained request storage run in the browser against controlled API fixtures. No provider or knowledge service is called.

Reason for invalidating saved evidence

Reason: explain the changed dependency before reviewing the decision. Keeping the evidence or closing this dialog submits nothing.
Review the decision and read its original receipt

Review the scoped evidence invalidation

Review: check the factory, actor, repository, record and reason, then acknowledge the effect. Confirmation sends one request after its identity has been retained.

Unresolved invalidation after an exact receipt read

Unresolved: the fixture loses the write response, retains its identity across navigation and returns 404 on the first receipt read. Check original request performs another read; it cannot replay the decision.

Exact original invalidation confirmation

Confirmed: the later read matches the original actor, request and reason. The historical event and current state remain separate. Its detail stays available even after the Verified evidence filter excludes the updated record.
Follow invalidation and exact-request recovery on a phone

Enter an invalidation reason on a phone

Reason: the narrow dialog keeps the field and review action reachable. Leave credentials out of the reason.

Review evidence invalidation on a phone

Review: inspect the scoped decision and acknowledgment before confirming. Scroll the form to read its remaining context.

Retained unresolved invalidation on a phone

Unresolved: the original reason and request stay available with the receipt-read control. A missing event leaves uncertainty in place.

Read the matching invalidation receipt on a phone

Confirmed: read the matched decision, then scroll through its retained event provenance. This fictional flow sends one write and two exact receipt reads.

Keep authority and retrieval separate

Component Responsibility Evidence it supplies
PostgreSQL learning ledger Persist status, applicability, decisions and immutable outcome references Exact record version, state transitions and eligibility reasons
flycanon Retain source knowledge, project memory and searchable candidates Source/chunk IDs, content provenance and knowledge versions
Native fireflyframework-agentic worker Propose changes and call the configured bounded tools Typed worker feedback and actual tool results
Factory orchestrator Admit tasks, enforce budgets, verify source and decide stage transitions WAL intents, command outcomes, source custody and delivery receipts
Configured reviewer Review the exact retained source and supported lesson statement A separate review decision tied to the reviewed tree

PostgreSQL is the authority for accepting a learning record. flycanon is its retrieval projection and the source-knowledge service. A semantic result must resolve back to a current authorized record before the factory treats it as an eligible learned outcome. A stale or missing projection never establishes acceptance.

Cached reuse performs the same eligibility checks as a fresh selection. The broker then reads the exact selected record with a native PostgreSQL shared lock, held through its admission transaction. An independent invalidation either commits before that read and refuses reuse, or waits for the admission to finish. The final synchronous guard runs after the execution-authority check and compares the retained policy, outcome, citation and source references. PostgreSQL time plus elapsed monotonic time bounds expiry. This defines the server's admission point; it does not make delivery to a network client atomic with a later invalidation.

If an eligible result has changed, expired or been invalidated, the original succeeded tool receipt remains intact. The caller receives an explicit refresh requirement. The factory does not silently search again, call a provider or replay the original tool. See tool invocation evidence for the request identity and uncertainty rules.

The current runtime binds native tools through fireflyframework_agentic.tools.firefly_tool and executes a FireflyAgent with bounded model usage. Its RuntimeReport, WorkerFeedback, browser plans and QA reports are typed. Official coding CLI workers have their own tool and permission systems; their native calls do not automatically pass through every Python tool contract. See agent tools and architecture for the implemented boundaries.

Record useful outcomes without overstating them

Keep source candidates, verified local outcomes and observed failed checks separate; apply repeat control to repair generation; preserve invalidation history; review any future shared framework lesson.

Open full diagram · Portrait layout

Passing a test supports its observed assertion at a particular source tree. It does not prove every retrieved search result correct or establish behavior on another version.

A positive lesson describes a supported local procedure or source outcome. Its question, task and check receipts identify the behavior that was verified. A model's summary and a successful run status cannot establish that association. Search snippets remain candidates unless the retained evidence and review support the specific statement being admitted.

A negative record says that an observed command or check failed in an exact context. It retains the actual outcome and source identity. Generalizing the result to an entire framework, library or programming technique requires further evidence and review.

Outcome Learning treatment
Actual verification check passes at the retained tree Eligible evidence for a bounded local outcome after the remaining receipt checks
Actual verification check fails Scoped observed failure; the failed command and context remain attached
Deliberately failing regression test Expected development evidence with an explicit purpose; independent checks continue
Rate limit, exhausted credits or authentication failure Operational feedback and recovery state
Timeout, lost response or interrupted external action Uncertain effect that requires reconciliation
A model says an approach is wrong or a search snippet says it works Candidate statement requiring evidence

Bound repairs while preserving independent QA

A native proposal fingerprint identifies the same run and repository/component, exact source tree, observed framework version and dependency declarations, failed command and bounded sanitized output. The additional cap applies only after completed verification has retained ordinary exit-code-one failures at an unchanged source. It counts distinct failed rounds from the intact native journal, not coding-provider errors or repeated proposal requests. Source or dependency drift, an unknown framework version, and older receipts without this provenance cannot establish the cap.

When the frozen learning policy is enabled, the factory records an available or held repair proposal before another coding intent. A held proposal fails generation explicitly and preserves the failed QA receipts; it never turns failure into success. The decision survives an interrupted response or worker resume without counting the same round again. Disabled or missing legacy policies leave the original attempt budgets and two-round no-progress guard intact. This adds no override that can weaken those guards.

These are same-run execution controls, not classified engineering lessons. Existing observed failures remain unclassified candidates and cannot suppress work in another run or repository. Provider rate limits, quota, authentication and transient failures keep their operational pause/retry handling. Unknown journal/tool effects require reconciliation before a repair proposal; they do not become engineering-negative evidence.

Independent final verification still runs. The pure purpose contract keeps diagnostic, intentional RED and regression checks unconditional. Runtime verification commands and test-only runs still execute their actual checks; the cap controls only a subsequent coding proposal. The public command schema remains argv and timeout, with no invented per-command purpose field. Existing repair and clarification budgets continue to limit execution. A known failure is useful input for the next implementation decision; it cannot substitute for current QA.

Transient provider and infrastructure failures must not poison the engineering knowledge base. Their recovery belongs to the execution log, provider controls and reconciliation workflow.

Structure agent and tool communication

A learning exchange needs a schema version, message type and stable correlation identity. Its run/task references identify the originating execution; causation references connect the question, tool request, result and later review. The source and configuration references must be observed by the factory or resolved from retained records.

Contract group Required association
Scope Current actor authority, repository and optional product/component references
Execution Run, task, effective role and stable request identity
Applicability Inspected source identity, observed framework/dependency context and frozen policy revision
Tool result Correlation/request digest, receipt disposition and cited source references
Verification Actual command/checkpoint receipts and configured review evidence
Freshness Database-time observation, validity rule and explicit refresh requirement
Decision Eligibility reason, admitted record version and unresolved effect references

The worker can propose a lesson, ask for missing knowledge or report a blocker. It cannot set a trusted verification flag, grant itself cross-project access or override a pending unknown effect. The native factory checks the message against the declared contract before changing execution state.

The lifecycle view distinguishes deterministic factory roles from model sessions. A product-manager or architect instruction role does not imply a separate autonomous model conversation. The implementation provider and an optional independent reviewer have actual execution bindings. Agent lifecycle explains their participants and retained handoffs.

Preserve updates, expiry and failed writes

Records progress through candidate, verified, invalidated and superseded states with retained evidence. A changed source or version makes an older record ineligible for exact reuse. Expiry requires refresh. Invalidation records the reason and actor; supersession names the replacement while preserving the old observations.

Database time defines freshness and admission decisions across workers. A frozen learning policy identifies the limits used by an admitted run. Legacy runs must keep their prior behavior and authority instead of acquiring new reuse permissions after an upgrade.

A flycanon projection write has a persisted intent before dispatch. A returned source ID supports read-only status inspection. If a response is lost and the remote outcome is unknown, unsupported source or knowledge creation must not be replayed automatically. Reconciliation must establish the actual remote record or retain the unresolved state.

Share between projects deliberately

Repository-private knowledge is the default. A future shared framework lesson requires an explicit reviewed, generalized payload with named public source, version and license provenance. Private code, product details, repository paths and credentials must remain outside that payload. Its review must also define how invalidating the origin or changing version assumptions affects the shared lesson.

L1 retains private origins and does not implement automatic global promotion. Framework manuals already available in the configured shared workspace remain reviewed references with their original scope and provenance.

Qualification required before acceptance

Native tests must establish that an exact eligible result avoids a provider request, an unknown or stale result cannot do so, required flycanon failure prevents internet fallback, and two repositories cannot reuse each other's private lessons. Real owned Git/check evidence must establish positive and negative outcomes. Concurrency, policy freezing and unknown-write recovery need PostgreSQL evidence.

A native FireflyAgent tool call must exercise the learning path; a standalone policy test does not establish runtime integration. Local flycanon projection and actual provider execution remain separate acceptance checks. Configuration alone cannot establish either.

Continue with knowledge configuration, search providers, agent lifecycle and WAL recovery.