Dev/chedli by Ilan-LP · Pull Request #2 · PoCInnovation/MLBlock

Ilan-LP · 2026-07-17T17:07:26Z

Description

Please provide a detailed description of what was done in this PR.
Precise the issue that you are resolving.

Changes include

Bugfix (non-breaking change that solves an issue)
New feature (non-breaking change that adds functionality)
Breaking change (change that is not backwards-compatible and/or changes current functionality)

Breaking changes

Please complete this section if any breaking changes have been made, otherwise delete it.

Checklist

Additional comments

Please post additional comments in this section if you have them, otherwise delete it.

…ode) - ajout children_allowed/description dans BlockSpec - ajout children recursifs dans PipelineNode - ajout build_fn/build_layer/can_build dans BlockMeta - ajout BUILD function dans les 11 blocs neural - auto-decouverte BUILD dans blocks/registry.py - ajout build_model() → nn.Sequential dans Pipeline - CLI --mode {generate,build} dans __main__

…Pipeline, and registry

…el construction

…s and exports

…ntic validation

…unctions

…thout Pydantic

…daptive pools, gelu, leaky_relu, elu)

…norms, activations, attention, identity)

# Conflicts: # mlblock/__main__.py # mlblock/blocks/neural/avgpool2d.py # mlblock/blocks/neural/batchnorm2d.py # mlblock/blocks/neural/conv2d.py # mlblock/blocks/neural/dropout.py # mlblock/blocks/neural/flatten.py # mlblock/blocks/neural/linear.py # mlblock/blocks/neural/maxpool2d.py # mlblock/blocks/neural/relu.py # mlblock/blocks/neural/sigmoid.py # mlblock/blocks/neural/softmax.py # mlblock/blocks/neural/tanh.py # mlblock/blocks/registry.py # mlblock/core/block.py # mlblock/core/pipeline.py # mlblock/models/block_spec.py # mlblock/models/pipeline.py # pyproject.toml

- Pydantic models with nested Category type - Pure Python functions as blocks (no BLOCK dict wrapper) - Auto-discovery via inspect.signature with _ filter - Named params in Pydantic schema for frontend guidance - Two routes: save and execute (no graph, no topological sort) - Code generation with inline block sources - _detect_dependencies() for pip requirements from imports - Vast.ai section moved under Server with per-second billing info - Database schemas with constraints and indexes - Real-time subscription via Supabase Realtime

- Suppression des anciens blocs categorises par dossier (neural, data, models, evaluation, advanced, rl, environment, visualization) - Suppression des anciens tests et documents obsoletes (Reunion.md, doc/diagrammes.md) - Nouveau systeme de blocs : fonctions Python pures avec docstring et type hints - Auto-discovery via inspect.signature avec filtrage des helpers prives (_) - Schema Pydantic avec Category nestee et ParamInfo pour le frontend - Deux routes API : save (genere le .py) et execute (lance sur Vast.ai) - Generation de code par inline des sources de blocs + detect_dependencies() - Execution GPU via Vast.ai avec image uv:python3.13-alpine, facturation a la seconde - Tables BDD avec contraintes, index et Realtime - Documentation complete dans docs/mlblock_v1.md

- Competitor analysis (Teachable Machine, ML for Kids, PictoBlox, etc.) - Cost breakdown for 500 users (2h sessions, 30min GPU) - Pricing: schools 1€/student, individuals 5€/user with 25% min margin - Break-even at 15 individuals or 74 students - 12-month projection and markstrat

…s/month) - Corrected GPU compute: 30min per week = 2h/month per user - Updated margins: schools 51%, individuals 90% - Break-even: 49 individuals or 244 students - 25% min margin constraint still largely satisfied (0.53€ cost vs 3.75€ max)

- Database: UUID primary keys, PostgreSQL/Supabase connection pooling, Job, JobOutput, Profile tables - Auth: JWT validation via python-jose, all user endpoints secured - Blocks: 50+ blocks reformatted as standard Python functions with docstrings and type hints; auto-discovery with dependency extraction - Code generator: cycle detection, multi-output port resolution, full source inlining for GPU execution - GPU orchestration: Vast.ai T4 launch with bookworm image, env injection, callback endpoints, automatic instance destruction - Pydantic schemas: ParamInfo, Block, Page, Pipeline CRUD models - Tests: 51/51 passing with mock UUIDs and auth overrides - Config: .env, .env.example, supabase-agent-skills installed

…ial output naming

- mlblock/ → backend/mlblock/ - pyproject.toml, uv.lock, requirements.txt → backend/ - .env, .env.example → backend/ - Update .gitignore for new paths - Update README.md with new structure and commands - Clean up root: remove duplicate configs/, merged_minds/, tmp.py

Copilot

Pull request overview

This PR bootstraps the MLBlock backend and documentation: it introduces a FastAPI server with pipeline CRUD (graph-based nodes/edges), block auto-discovery, code generation, and Vast.ai GPU execution with callback endpoints, alongside substantial README/spec documentation.

Changes:

Add a FastAPI backend (routes, models, auth, GPU callbacks) plus unit tests.
Add block auto-discovery/registry, a graph + code generator, and a sample MNIST config/demo.
Add/refresh documentation (README + routes/marketing) and OpenSpec spec artifacts; remove template license/contributing files.

Reviewed changes

Copilot reviewed 121 out of 134 changed files in this pull request and generated 24 comments.

Show a summary per file

File	Description
README.md	Replaces template README with MLBlock architecture + API overview + setup.
openspec/config.yaml	Adds OpenSpec configuration scaffold.
openspec/changes/reconcile-v1-spec-with-codebase/tasks.md	Planning tasks for spec/code reconciliation.
openspec/changes/reconcile-v1-spec-with-codebase/specs/pipeline-spec/spec.md	Spec correction: linear → DAG pipeline definition.
openspec/changes/reconcile-v1-spec-with-codebase/specs/database-schema/spec.md	Spec correction: `nodes`/`edges` + remove stored `code`.
openspec/changes/reconcile-v1-spec-with-codebase/proposal.md	Proposal for doc/spec reconciliation.
openspec/changes/reconcile-v1-spec-with-codebase/design.md	Design notes for the reconciliation effort.
openspec/changes/reconcile-v1-spec-with-codebase/.openspec.yaml	OpenSpec metadata for the change.
openspec/changes/migrate-codebase-to-v1-spec/tasks.md	Migration task checklist to v1 spec.
openspec/changes/migrate-codebase-to-v1-spec/specs/user-authentication/spec.md	Auth requirements spec (Supabase JWT).
openspec/changes/migrate-codebase-to-v1-spec/specs/pydantic-models/spec.md	Pydantic schema requirements spec.
openspec/changes/migrate-codebase-to-v1-spec/specs/gpu-execution/spec.md	Vast.ai execution requirements spec.
openspec/changes/migrate-codebase-to-v1-spec/specs/gpu-callbacks/spec.md	GPU callback security + cleanup spec.
openspec/changes/migrate-codebase-to-v1-spec/specs/database-schema/spec.md	DB schema requirements spec (SQLModel tables).
openspec/changes/migrate-codebase-to-v1-spec/specs/database-connection/spec.md	DB connection requirements spec.
openspec/changes/migrate-codebase-to-v1-spec/specs/block-specification/spec.md	Block function format requirements spec.
openspec/changes/migrate-codebase-to-v1-spec/specs/block-discovery/spec.md	Block discovery/deps requirements spec.
openspec/changes/migrate-codebase-to-v1-spec/proposal.md	Proposal for codebase migration to v1.
openspec/changes/migrate-codebase-to-v1-spec/design.md	Design for v1 migration (DB/auth/GPU).
openspec/changes/migrate-codebase-to-v1-spec/.openspec.yaml	OpenSpec metadata for the change.
openspec/changes/archive/2026-07-09-complete-v1-routes/tasks.md	Archived tasks for route completion.
openspec/changes/archive/2026-07-09-complete-v1-routes/specs/pipeline-crud/spec.md	Archived pipeline CRUD spec.
openspec/changes/archive/2026-07-09-complete-v1-routes/specs/job-history/spec.md	Archived job history spec.
openspec/changes/archive/2026-07-09-complete-v1-routes/specs/block-filtering/spec.md	Archived block filtering spec.
openspec/changes/archive/2026-07-09-complete-v1-routes/specs/auth-middleware/spec.md	Archived auth middleware spec.
openspec/changes/archive/2026-07-09-complete-v1-routes/proposal.md	Archived proposal (routes-only).
openspec/changes/archive/2026-07-09-complete-v1-routes/design.md	Archived design (routes-only).
openspec/changes/archive/2026-07-09-complete-v1-routes/.openspec.yaml	Archived OpenSpec metadata.
LICENSE	Removes repository license file.
docs/routes.md	Adds route reference documentation.
docs/marketing.md	Adds marketing/business model doc.
CONTRIBUTING.md	Removes contributing guide template.
backend/test_gen.py	Adds a generated-code sample script.
backend/requirements.txt	Adds pip requirements list for backend.
backend/pyproject.toml	Adds Python project metadata + dependencies.
backend/mlblock/tests/test_server.py	Adds API tests (blocks/pipelines/validate/generate/build).
backend/mlblock/tests/test_pipeline.py	Adds codegen/pipeline tests.
backend/mlblock/tests/test_graph.py	Adds graph construction/validation tests.
backend/mlblock/tests/test_config.py	Adds config loader/validation tests.
backend/mlblock/tests/test_block.py	Adds core block registry tests.
backend/mlblock/tests/init.py	Test package marker.
backend/mlblock/server/schemas.py	Adds Pydantic API schemas.
backend/mlblock/server/routes.py	Adds FastAPI routers (blocks/pipelines/jobs/validate/build/execute).
backend/mlblock/server/models.py	Adds SQLModel tables (Pipeline/Job/JobOutput/Profile).
backend/mlblock/server/main.py	Adds FastAPI app assembly + CORS + lifespan init_db.
backend/mlblock/server/gpu_auth.py	Adds GPU callback auth dependency.
backend/mlblock/server/database.py	Adds engine/session + init_db + dotenv load.
backend/mlblock/server/auth.py	Adds Supabase JWT validation dependency.
backend/mlblock/server/main.py	Adds uvicorn entrypoint.
backend/mlblock/server/init.py	Server package marker.
backend/mlblock/models/registry.py	Adds BlockRegistry type alias.
backend/mlblock/models/pipeline.py	Adds pipeline definition models + validators.
backend/mlblock/models/cnn.py	Adds config-driven codegen/run helpers.
backend/mlblock/models/block_spec.py	Adds BlockSpec / PortSpec / ParamSpec models.
backend/mlblock/models/init.py	Adds lazy exports for model helpers.
backend/mlblock/demo.py	Adds demo script to generate code from config.
backend/mlblock/core/vast.py	Adds Vast.ai wrapper (launch/start/exec/destroy).
backend/mlblock/core/pipeline.py	Adds pipeline generate/run orchestration.
backend/mlblock/core/graph.py	Adds DAG model + topo sort + validation.
backend/mlblock/core/generator.py	Adds code generator for GPU execution + callbacks.
backend/mlblock/core/config.py	Adds JSON config loader + validation.
backend/mlblock/core/block.py	Adds core block registry/execution abstraction.
backend/mlblock/core/init.py	Core package marker.
backend/mlblock/configs/cnn_mnist.json	Adds MNIST example DAG config.
backend/mlblock/blocks/visualization_EC4899/render_rl.py	Adds visualization block stub.
backend/mlblock/blocks/visualization_EC4899/record_video.py	Adds visualization block stub.
backend/mlblock/blocks/visualization_EC4899/plot_predictions.py	Adds plotting block (currently broken).
backend/mlblock/blocks/visualization_EC4899/init.py	Category package marker.
backend/mlblock/blocks/rl_8B5CF6/train_rl.py	Adds RL block stub.
backend/mlblock/blocks/rl_8B5CF6/evaluate_rl.py	Adds RL block stub.
backend/mlblock/blocks/rl_8B5CF6/init.py	Category package marker.
backend/mlblock/blocks/registry.py	Adds auto-discovery + registry + legacy core registration.
backend/mlblock/blocks/neural_6366F1/upsample.py	Adds neural block.
backend/mlblock/blocks/neural_6366F1/tanh.py	Adds neural block.
backend/mlblock/blocks/neural_6366F1/softmax.py	Adds neural block.
backend/mlblock/blocks/neural_6366F1/silu.py	Adds neural block.
backend/mlblock/blocks/neural_6366F1/sigmoid.py	Adds neural block.
backend/mlblock/blocks/neural_6366F1/selu.py	Adds neural block.
backend/mlblock/blocks/neural_6366F1/rnn.py	Adds neural block.
backend/mlblock/blocks/neural_6366F1/relu.py	Adds neural block.
backend/mlblock/blocks/neural_6366F1/prelu.py	Adds neural block.
backend/mlblock/blocks/neural_6366F1/multihead_attention.py	Adds neural block.
backend/mlblock/blocks/neural_6366F1/maxpool2d.py	Adds neural block.
backend/mlblock/blocks/neural_6366F1/lstm.py	Adds neural block.
backend/mlblock/blocks/neural_6366F1/linear.py	Adds neural block.
backend/mlblock/blocks/neural_6366F1/leaky_relu.py	Adds neural block.
backend/mlblock/blocks/neural_6366F1/layernorm.py	Adds neural block.
backend/mlblock/blocks/neural_6366F1/instancenorm2d.py	Adds neural block.
backend/mlblock/blocks/neural_6366F1/input.py	Adds input block (currently broken).
backend/mlblock/blocks/neural_6366F1/identity.py	Adds neural block.
backend/mlblock/blocks/neural_6366F1/gru.py	Adds neural block.
backend/mlblock/blocks/neural_6366F1/gelu.py	Adds neural block.
backend/mlblock/blocks/neural_6366F1/flatten.py	Adds neural block.
backend/mlblock/blocks/neural_6366F1/embedding.py	Adds neural block.
backend/mlblock/blocks/neural_6366F1/elu.py	Adds neural block.
backend/mlblock/blocks/neural_6366F1/dropout.py	Adds neural block.
backend/mlblock/blocks/neural_6366F1/conv3d.py	Adds neural block.
backend/mlblock/blocks/neural_6366F1/conv2d.py	Adds neural block.
backend/mlblock/blocks/neural_6366F1/conv1d.py	Adds neural block.
backend/mlblock/blocks/neural_6366F1/conv_transpose2d.py	Adds neural block.
backend/mlblock/blocks/neural_6366F1/batchnorm2d.py	Adds neural block.
backend/mlblock/blocks/neural_6366F1/batchnorm1d.py	Adds neural block.
backend/mlblock/blocks/neural_6366F1/avgpool2d.py	Adds neural block.
backend/mlblock/blocks/neural_6366F1/adaptive_maxpool2d.py	Adds neural block.
backend/mlblock/blocks/neural_6366F1/adaptive_avgpool2d.py	Adds neural block.
backend/mlblock/blocks/neural_6366F1/init.py	Category package marker.
backend/mlblock/blocks/models_F59E0B/random_forest.py	Adds sklearn model block (currently broken).
backend/mlblock/blocks/models_F59E0B/logistic_regression.py	Adds sklearn model block (currently broken).
backend/mlblock/blocks/models_F59E0B/linear_regression.py	Adds sklearn model block (currently broken).
backend/mlblock/blocks/models_F59E0B/init.py	Category package marker.
backend/mlblock/blocks/evaluation_EF4444/evaluate.py	Adds evaluation block (currently broken).
backend/mlblock/blocks/evaluation_EF4444/init.py	Category package marker.
backend/mlblock/blocks/environment_14B8A6/time_limit.py	Adds environment block stub.
backend/mlblock/blocks/environment_14B8A6/space_def.py	Adds environment block stub.
backend/mlblock/blocks/environment_14B8A6/single_env.py	Adds environment block stub.
backend/mlblock/blocks/environment_14B8A6/normalize_obs.py	Adds environment block stub.
backend/mlblock/blocks/environment_14B8A6/gym_env.py	Adds environment block stub.
backend/mlblock/blocks/environment_14B8A6/frame_stack.py	Adds environment block stub.
backend/mlblock/blocks/environment_14B8A6/custom_env.py	Adds environment block stub.
backend/mlblock/blocks/environment_14B8A6/init.py	Category package marker.
backend/mlblock/blocks/data_22C55E/train_test_split.py	Adds data split block (currently broken).
backend/mlblock/blocks/data_22C55E/load_csv.py	Adds CSV loader block.
backend/mlblock/blocks/data_22C55E/init.py	Category package marker.
backend/mlblock/blocks/advanced_6B7280/code_block.py	Adds custom code block (unsafe as written).
backend/mlblock/blocks/advanced_6B7280/auto_ml.py	Adds advanced block stub.
backend/mlblock/blocks/advanced_6B7280/init.py	Category package marker.
backend/mlblock/blocks/init.py	Exposes BLOCK_REGISTRY.
backend/mlblock/main.py	Adds CLI entrypoint (generate/build).
backend/mlblock/init.py	Triggers auto-discovery on import.
backend/main.py	Adds generated output script (repo artifact).
backend/.env.example	Adds backend env var template.
.gitignore	Adds Python + local env/generated ignores.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+    Args:
+        shape: Parameter.
+    """
+    return nn.Input(shape=shape)(in_1)


+    lines.append("def notify_status(block, status):")
+    lines.append("    try:")
+    lines.append("        requests.post(f'{BACKEND_URL}/jobs/{JOB_ID}/status',")
+    lines.append("            json={'block': block, 'status': status},")
+    lines.append("            headers={'Authorization': f'Bearer {GPU_API_KEY}'}, timeout=5)")


+    lines.append("def notify_output(block, output):")
+    lines.append("    try:")
+    lines.append("        requests.post(f'{BACKEND_URL}/jobs/{JOB_ID}/output',")
+    lines.append("            json={'block': block, 'output': str(output)[:10000]},")
+    lines.append("            headers={'Authorization': f'Bearer {GPU_API_KEY}'}, timeout=5)")


+    lines.append("def notify_error(block, error):")
+    lines.append("    try:")
+    lines.append("        requests.post(f'{BACKEND_URL}/jobs/{JOB_ID}/error',")
+    lines.append("            json={'block': block, 'error': str(error)},")
+    lines.append("            headers={'Authorization': f'Bearer {GPU_API_KEY}'}, timeout=5)")


+def list_blocks(
+    page: int = Query(1, ge=1),
+    category: str | None = None,
+    q: str | None = Query(None),
+    _: str = Depends(get_current_user),
+) -> Page[Block]:


+                from mlblock.core.block import BlockRegistry as CoreBlockRegistry
+                legacy_spec = {
+                    "label": name.replace("_", " ").title(),
+                    "category": cat_name,
+                    "params": {k: {"type": v.type, "default": v.default, "required": v.required} for k, v in block.params.items()},
+                    "inputs": [{"name": "in", "dtype": "Tensor"}],
+                    "outputs": [{"name": "out", "dtype": "Tensor"}],
+                    "template": "{output.out} = nn.SomeLayer({params.in_channels})",
+                }
+                CoreBlockRegistry.register(name, legacy_spec, fn)


+    def execute(self, params: dict[str, Any]) -> Any:
+        if self._build_fn is not None:
+            params.pop("_inputs", None)
+            result = self._build_fn(params)


+            for edge in self.graph.edges:
+                if edge.target == node_id:
+                    inputs[edge.target_port] = outputs.get(edge.source)
+            node.params["_inputs"] = inputs


+
+| | Type |
+|---|---|
+| **Query params** | `page: int = 1`, `size: int = 20`, `category: str \| None`, `search: str \| None` |


+| | Type |
+|---|---|
+| **Entrée** | — |
+| **Sortie** | `{ categories: list[str] }` |


aliisssss and others added 30 commits May 27, 2026 13:49

feat: MLBlocks core engine and CNN blocks

0bdb8ea

merge: ali base + chedli pydantic + auto-discovery + generate_code

03c3616

push all

6868d01

[add] reunion

83ca505

feat: FastAPI server for MLBlock (CRUD, validation, build, generate)

725b658

chore: add *.db, *.sqlite3, .env, mlblock.db to gitignore

3337aed

cleanup: remove merged_minds backup directory

c309831

cleanup: remove chedli directory

010bfb0

feat(core): add build_fn, build_layer(), build_model() to BlockMeta, …

9c4ac3b

…Pipeline, and registry

feat(neural): add BUILD function to all 11 neural blocks for live mod…

a068459

…el construction

feat(cli): add --mode {generate,build} with argparse

446a290

refactor: remove Pydantic models (BlockSpec, PipelineDef), update dep…

21d4a94

…s and exports

refactor(core): dict-based BlockSpec, add Pipeline.run(), remove Pyda…

493fb24

…ntic validation

refactor(blocks): convert all 31 blocks to dict-based BLOCK + BUILD f…

c291bf3

…unctions

refactor(cli,tests): update CLI for dict-based registry, fix tests wi…

dfe1768

…thout Pydantic

[add] configs files

b4db8c9

feat: add 6 test configs (RL, ML, neural) + accuracy/f1 metrics support

0c154b3

feat(neural): add 9 new blocks (conv1d, lstm, embedding, layernorm, a…

46785bd

…daptive pools, gelu, leaky_relu, elu)

feat(neural): add 12 new blocks (gru, rnn, conv3d, conv_transpose2d, …

d0f1bf9

…norms, activations, attention, identity)

docs: add sequence diagrams

6f68c0b

add comprehensive README.md

3265273

fix port-to-param mapping in code generator (in -> x)

88389e7

rename ports to generic convention: in_1/in_2... out_1/out_2...

92f3a3d

Ouaziz-chedli added 3 commits July 15, 2026 17:49

simplify output naming: sequential out_1, out_2, out_3...

28ef7d1

update README with correct API schemas, port conventions, and sequent…

d0c10be

…ial output naming

Copilot AI review requested due to automatic review settings July 17, 2026 17:07

Copilot started reviewing on behalf of Ilan-LP July 17, 2026 17:07 View session

Copilot AI reviewed Jul 17, 2026

View reviewed changes

Merge branch 'main' into dev/chedli

c47d9f5

Ilan-LP merged commit 80c31f2 into main Jul 17, 2026
1 check passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Dev/chedli#2

Dev/chedli#2
Ilan-LP merged 34 commits into
mainfrom
dev/chedli

Ilan-LP commented Jul 17, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

Ilan-LP commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Changes include

Breaking changes

Checklist

Additional comments

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Ilan-LP commented Jul 17, 2026 •

edited

Loading