Skip to content

feat: add video analyzer tool with FFmpeg + subtitle support#45

Open
NSIETeam wants to merge 1 commit into
OrionStarAI:opensourcefrom
NSIETeam:feat/video-analyzer-tool
Open

feat: add video analyzer tool with FFmpeg + subtitle support#45
NSIETeam wants to merge 1 commit into
OrionStarAI:opensourcefrom
NSIETeam:feat/video-analyzer-tool

Conversation

@NSIETeam

Copy link
Copy Markdown

What

Adds a new analyze_video tool that downloads, analyzes, and summarizes videos.

Features

  • Download: yt-dlp for YouTube/Zoom/Loom, curl for direct URLs, local file support
  • Key frame extraction: FFmpeg scene detection (0.4 threshold), falls back to uniform sampling
  • Subtitles: YouTube official auto-subs first, Whisper transcription as fallback
  • LLM analysis: Sends frames + subtitles to vision model for structured summary (topics, key moments, action items)
  • Knowledge base: Optionally saves analysis to ~/.easycode/kb/videos/ for future recall

Dependencies

  • yt-dlp (video download)
  • ffmpeg + ffprobe (frame extraction + duration)
  • whisper (optional, for non-YouTube subtitle transcription)

Origin

Ported from Otto. Adapted for EasyCode:

  • Removed SceneType, getResponseText, isCustomModel dependencies
  • Simplified LLM analysis: uses config.getOttoClient() if available, falls back to basic summary
  • Changed temp/kb dirs from ~/.otto/ to ~/.easycode/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant