[feat] Job Runner UI

[dsynkd]

This PR adds a web-based UI that allows running jobs for various purposes. Currently only video inference is supported with the following parameters:

• Inference Steps
• Frames
• Height
• Width
• Guidance Scale
• Seed
• GPUs
• DiT CPU Offload
• Text Encoder CPU Offload
• FSDP Inference

Jobs can be:

• Created
• Started
• Stopped
• Restarted
• Deleted

All models that are registered through FastVideo registry are supported.

Currently the following features are included:

• Console Log
• Download Log
• Inference Timer

To run, navigate to the ui directory and run:

npm i
npm run build
npm start

There are two services that start concurrently: the web server and the API server. They can also be started separately using npm run start:api and npm run start:web. In this case, the .env.local file can be used to configure the API server URL (it defaults to http://localhost:8089).

The web server can also be run in dev mode using npm run dev. This will automatically restart the server upon any changes made to the UI.

Demo video is shown below (sped up to reduce size):

output.mp4

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request delivers a new, interactive web interface designed to streamline the management and execution of FastVideo tasks. It provides a user-friendly way to configure and monitor video generation jobs, abstracting away complex command-line operations. The UI integrates a robust backend for job orchestration and persistence, ensuring a smooth workflow for users interacting with the FastVideo system.

Highlights

• New Web-Based UI: Introduced a comprehensive web-based user interface for managing FastVideo jobs, enabling users to create, start, stop, restart, and delete video generation tasks.
• Video Inference Support: The UI initially supports video inference with configurable parameters such as inference steps, frames, height, width, guidance scale, seed, GPU count, and various CPU offloading options.
• Real-time Logging and Persistence: Implemented real-time console log display and the ability to download full log files for each job, with job and settings persistence via an SQLite database.
• Extensible Job Types: The architecture supports different job types (inference, finetuning, distillation) and workload types (T2V, I2V, T2I, LoRA, DMD), with dynamic model loading and parameter handling.
• Frontend and Backend Integration: Developed a FastAPI backend for API services and a SvelteKit frontend for an interactive user experience, including file uploads and dynamic UI components.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• .python-version
  • Added Python version 3.12 specification.
• collect_env.py
  • Updated uv detection logic to use shutil.which for improved reliability.
  • Imported shutil module.
• fastvideo-kernel/pyproject.toml
  • Modified triton dependency to be Linux-specific, improving cross-platform compatibility.
• fastvideo/entrypoints/video_generator.py
  • Refactored VideoGenerator initialization to accept an optional log_queue for worker log forwarding.
  • Introduced _generate_video_impl for internal video generation logic, allowing generate_video to manage log queue setup and teardown.
  • Updated from_pretrained and from_fastvideo_args class methods to pass log_queue to the executor.
• fastvideo/registry.py
  • Added get_registered_model_paths to list all registered HuggingFace model paths.
  • Implemented _infer_workload_types to determine supported workload types for models based on their path and configuration.
  • Added get_registered_models_with_workloads to return models with associated workload metadata, useful for UI filtering.
• fastvideo/worker/executor.py
  • Added log_queue parameter to Executor initialization.
  • Introduced abstract methods set_log_queue and clear_log_queue for managing worker log forwarding.
• fastvideo/worker/multiproc_executor.py
  • Imported logging and logging.handlers for enhanced logging capabilities.
  • Added _make_queue_log_handler to create a QueueHandler for forwarding logs.
  • Modified MultiprocExecutor to pass log_queue to worker processes during initialization.
  • Implemented set_log_queue and clear_log_queue methods to dynamically manage log forwarding from worker processes.
  • Enhanced worker main loop to add a log handler before model loading and remove it afterwards.
  • Improved error handling during worker initialization by sending detailed error status and tracebacks to the parent process.
  • Added a check in shutdown to ensure workers exist before attempting to shut them down, preventing errors if initialization failed.
  • Updated wait_for_ready to collect and report errors from worker processes during startup.
• pyproject.toml
  • Added fastapi and uvicorn as new dependencies under the Job Runner section.
• ui/.gitignore
  • Added standard .gitignore entries for Node.js projects, including node_modules, .next, build, and various log files.
• ui/.prettierrc
  • Added Prettier configuration for consistent code formatting, specifying useTabs: true and tabWidth: 4.
• ui/README.md
  • Added a comprehensive README for the FastVideo Job Runner UI, detailing quick start instructions, features, API endpoints, and architectural overview.
• ui/init.py
  • Added SPDX license identifier.
• ui/main.py
  • Added __main__.py to allow running the UI server directly via python -m ui.
• ui/database.py
  • Implemented SQLite database for persistence of jobs and settings.
  • Defined DEFAULT_SETTINGS schema and functions for database initialization and schema migration.
  • Provided methods for inserting, updating, deleting, and retrieving jobs, datasets, and settings in a thread-safe manner.
• ui/eslint.config.mjs
  • Added a minimal ESLint configuration for SvelteKit.
• ui/job_runner.py
  • Implemented JobRunner class to manage the lifecycle of video generation jobs, including execution, logging, and generator caching.
  • Introduced JobStatus enum and JobLogBuffer for real-time log streaming and progress tracking.
  • Added LogBufferHandler to integrate job logs with the logging system.
  • Provided methods for creating, starting, stopping, and deleting jobs, handling both inference and training tasks.
  • Integrated multiprocessing queues for efficient log forwarding from worker processes.
• ui/models/init.py
  • Exported Pydantic models for API requests.
• ui/models/create_dataset_request.py
  • Defined CreateDatasetRequest Pydantic model for dataset creation, including name, upload path, file names, and captions.
• ui/models/create_job_request.py
  • Defined CreateJobRequest Pydantic model with extensive parameters for creating inference and training jobs, including model ID, prompt, various settings, and CPU offload options.
• ui/models/settings_update.py
  • Defined SettingsUpdate Pydantic model for updating UI settings, allowing partial updates of various configuration parameters.
• ui/models/update_caption_request.py
  • Defined UpdateCaptionRequest Pydantic model for updating captions of individual files within a dataset.
• ui/package.json
  • Initialized Node.js project for the UI with SvelteKit dependencies and scripts for development, build, and starting both web and API servers concurrently.
• ui/preprocess_runner.py
  • Added build_preprocess_args function to construct CLI arguments for dataset preprocessing.
  • Implemented run_preprocess function to execute preprocessing as a subprocess, handling logging and status updates.
• ui/public/logo.svg
  • Added the FastVideo logo as an SVG asset.
• ui/server.py
  • Implemented the FastAPI server for the UI, providing REST endpoints for managing jobs, models, settings, and file uploads.
  • Configured CORS middleware for frontend access.
  • Added endpoints for uploading images and raw datasets, handling file validation and storage.
  • Integrated JobRunner and Database for backend logic.
  • Set up signal handlers for graceful shutdown and to ignore SIGQUIT from crashed workers.
  • Added create_local_env to generate .env.local for frontend API URL configuration.
• ui/src/app.css
  • Added global CSS variables for consistent styling across the UI.
  • Defined base styles for body, form inputs, and other common elements.
• ui/src/app.html
  • Provided the base HTML structure for the SvelteKit application, including metadata and SvelteKit placeholders.
• ui/src/lib/actions/resizable.ts
  • Implemented a Svelte action resizable to add horizontal resizing behavior to UI elements, with configurable edges, min/max width, and drag callbacks.
• ui/src/lib/api.ts
  • Created a TypeScript API client with functions for interacting with the FastAPI backend, including fetching models, jobs, settings, and handling file uploads.
  • Defined interfaces for CreateJobRequest, Model, Settings, Dataset, and JobLogs.
• ui/src/lib/captionParsing.ts
  • Added utility functions (parseVideos2Caption, parseVideosCaptionsTxt, parseCaptionCsv) for parsing different caption file formats (JSON, TXT, CSV) and validating against uploaded file names.
• ui/src/lib/components/AddDatasetButton.svelte
  • Created a Svelte component for a button that opens the dataset creation modal.
• ui/src/lib/components/CreateDatasetModal.svelte
  • Developed a Svelte component for a modal to create new datasets, featuring file upload, caption format selection (JSON, TXT, CSV), and validation.
• ui/src/lib/components/CreateJobButton.svelte
  • Designed a Svelte component for a button that opens the job creation modal, allowing selection of job and workload types.
• ui/src/lib/components/CreateJobModal.svelte
  • Implemented a Svelte component for the job creation modal, providing extensive input fields for model selection, prompt, inference parameters, and training-specific options, with dynamic UI based on job and workload type.
• ui/src/lib/components/DatasetCard.svelte
  • Created a Svelte component to display individual dataset information, including name, file count, size, and a delete action.
• ui/src/lib/components/DatasetSidebar.svelte
  • Developed a Svelte component for a sidebar to display details of a selected dataset, including media files, captions, and options to download captions.
• ui/src/lib/components/DownloadCaptions.svelte
  • Created a Svelte component for a dropdown button to download dataset captions in JSON, TXT, or CSV formats.
• ui/src/lib/components/Header.svelte
  • Implemented a Svelte component for the application header, displaying the current page title and dynamically rendered action buttons.
• ui/src/lib/components/JobCard.svelte
  • Designed a Svelte component to display individual job details, status, elapsed time, and actions like start, stop, delete, and download video.
• ui/src/lib/components/JobQueue.svelte
  • Created a Svelte component to display a list of jobs, handling periodic polling for status updates and managing the active job selection.
• ui/src/lib/components/PrimarySidebar.svelte
  • Developed a Svelte component for the main navigation sidebar, featuring job type categories, dataset/gallery/settings links, and a resizable/collapsible interface.
• ui/src/lib/components/SecondarySidebar.svelte
  • Implemented a Svelte component for a secondary sidebar to display detailed job information and real-time console output, with log download functionality.
• ui/src/lib/components/Slider.svelte
  • Created a reusable Svelte component for a custom slider input with configurable min/max/step values and value formatting.
• ui/src/lib/components/TabSwitch.svelte
  • Developed a reusable Svelte component for a tab switch UI element.
• ui/src/lib/components/Toggle.svelte
  • Created a reusable Svelte component for a toggle switch input.
• ui/src/lib/components/UploadZone.svelte
  • Implemented a Svelte component for a flexible file/folder upload zone, supporting drag-and-drop, file selection, and text input modes.
• ui/src/lib/defaultOptions.ts
  • Defined DefaultOptions interface and DEFAULT_OPTIONS constant for UI settings.
  • Provided functions loadDefaultOptions and saveDefaultOptions for local storage persistence.
  • Added getDefaultModelForWorkload to retrieve default model IDs based on workload type.
• ui/src/lib/jobConfig.ts
  • Defined WORKLOAD_OPTIONS mapping job types (inference, finetuning, distillation) to available workload options with labels and descriptions.
• ui/src/lib/stores/activeDataset.ts
  • Created Svelte stores (activeDatasetId, activeDataset) and helper functions to manage the currently selected dataset in the UI.
• ui/src/lib/stores/activeJob.ts
  • Created Svelte stores (activeJobId, activeJob) and helper functions to manage the currently selected job in the UI.
• ui/src/lib/stores/createDatasetModalOpen.ts
  • Created a Svelte writable store to manage the open/closed state of the dataset creation modal.
• ui/src/lib/stores/defaultOptions.ts
  • Created a Svelte store defaultOptions for managing global UI settings, integrating with API for persistence and local storage for overrides.
• ui/src/lib/stores/headerActions.ts
  • Created a Svelte writable store headerActions and a helper function setHeaderActions to dynamically control components rendered in the application header.
• ui/src/lib/stores/jobsRefresh.ts
  • Implemented a mechanism (registerRefresh, triggerRefresh) to allow components to register and trigger a refresh of job lists.
• ui/src/lib/types.ts
  • Defined core TypeScript interfaces for Job and JobType, specifying their structure and properties.
• ui/src/routes/+layout.svelte
  • Implemented the main SvelteKit layout, integrating Header, PrimarySidebar, and dynamically rendered SecondarySidebar or DatasetSidebar based on active selections.
• ui/src/routes/+layout.ts
  • Configured SvelteKit layout for client-side rendering (ssr = false).
• ui/src/routes/+page.svelte
  • Added a Svelte page that redirects users from the root URL to the /inference page.
• ui/src/routes/datasets/+page.svelte
  • Created a Svelte page to list and manage datasets, including fetching data, displaying DatasetCard components, and integrating the CreateDatasetModal.
• ui/src/routes/distillation/+page.svelte
  • Developed a Svelte page for displaying and managing distillation jobs, utilizing JobQueue and CreateJobButton components.
• ui/src/routes/finetuning/+page.svelte
  • Created a Svelte page for displaying and managing finetuning jobs, using JobQueue and CreateJobButton components.
• ui/src/routes/gallery/+page.svelte
  • Implemented a Svelte page to showcase a gallery of completed video generation jobs, displaying media and prompts.
• ui/src/routes/inference/+page.svelte
  • Designed a Svelte page for displaying and managing inference jobs, using JobQueue and CreateJobButton components.
• ui/src/routes/lora/+page.svelte
  • Added a Svelte page that redirects users from /lora to the /finetuning page.
• ui/src/routes/settings/+page.svelte
  • Created a Svelte page for configuring application settings, including API server URL, dataset upload path, and default job parameters, with options to reset to defaults.
• ui/static/logo.svg
  • Added the FastVideo logo SVG to the static assets.
• ui/svelte.config.js
  • Configured SvelteKit to use @sveltejs/adapter-node and vitePreprocess.
• ui/tsconfig.json
  • Added TypeScript configuration for the SvelteKit project, extending .svelte-kit/tsconfig.json.

Activity

• The pull request introduces a new UI, so there is no prior human activity to report.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a comprehensive web-based UI for running and managing jobs, complete with a FastAPI backend and a SvelteKit frontend. The changes are extensive and well-structured, particularly the improvements to logging and error propagation in the multiprocessing workers, which significantly enhance the system's robustness. The backend architecture is sound, with a clear separation of concerns. I have a few suggestions focused on improving the maintainability, clarity, and debuggability of the new backend components.