Repository navigation
feat(desktop): transcript editing - soft delete with ability to restore, deleted words buffers, pause deletion - #1880
ganganimaulik wants to merge 12 commits into
Conversation
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
|
When will this feature be available? |
|
Thanks @ganganimaulik! Greptile is at 4/5 here. We only merge once Greptile reaches 5/5, so please address its findings in the summary above (or reply to them if you think one is wrong), then comment |
f27285b to
570b8db
Compare
|
@ganganimaulik is attempting to deploy a commit to the Cap Software Inc Team on Vercel. A member of the Team first needs to authorize it. |
|
@richiemcilroy I realize the Cap team has plenty on its plate, but this feature could be a major differentiator for Cap compared with competing tools. I currently use Tella, but I’m not entirely happy with their implementation and think it could be better. If Cap releases this feature and it works smoothly, I’d become a subscriber without hesitation! I understand that @ganganimaulik is the one behind this PR, but I wanted to emphasize just how valuable this feature would be! |
parakeet-rs 0.3.4 ends every token where the next one starts, so each word's stored end ran through the pause after it and captions stayed on screen up to MAX_CAPTION_WORD_DURATION into every pause. Add cap_audio::SpeechPauses, which finds pauses in the 16 kHz transcription audio relative to the clip's noise floor and speech level, and use it in the Tauri and GPUI Parakeet paths to end each word 0.1 s after its speech stops. Word starts never move, timings are kept when speech isn't clearly above the noise, and the 2.5 s clamp stays as the backstop. Whisper output is unchanged, and existing transcriptions keep their timings until they are generated again.
|
@greptileai review |
Comments Outside DiffThese findings could not be posted inline.
|
A saved cut piece could only be restored whole, so shrinking a cut (or restoring one of two overlapping cuts) brought the returned footage back plain and then dropped the piece. Overlays and fullscreen text holds in the returned part never came back, and the ones still cut were lost on a later restore. When part of a piece returns, restore the whole piece and cut the rest again, which saves a fresh piece for what stays cut.
A cut saves the overlays it changed as whole before/after snapshots, so they only restore exactly in reverse removal order. Restoring a cut while a later one that edited the same overlay stayed cut skipped the snapshot, and the earlier cut's changes (mask and camera keyframes, a zoom's trimmed edge) were lost. Restoring a piece now also restores any later piece still cut that edited the same overlay, then cuts it again so it saves a fresh snapshot. Trims also leave overlay edges on a cut point through a different float path than the boundary a later cut or restore computes, so an edge could land a hair on the wrong side: a restore stretched the overlay instead of moving it, and a cut left a zero-length sliver. Edges within 1e-9 s of the boundary now snap onto it first.
…ilence Two cut words whose padding grows into the silence between them leave no pause behind: the transcript only offers silence that is still in the timeline, so the padded claims remove it from the pause list.
Restoring a saved piece puts the whole piece back and cuts the rest of it again. On a sped-up clip a sliver of that rest can be too short on the timeline for the cut to remove (under 1 ms there) while still counting as more than 1 ms of footage, so the check that compared it in source time refused the restore with "That cut would remove the whole video." Measure what a cut leaves behind on the timeline, where the cut itself skips it, and stop recording a sliver that stays in the timeline as removed.
A fullscreen text hold that sits right where a cut ends, like a closing title after a cut that reaches the end of the clips, stays with what follows the cut. Restoring the cut put the footage back after the hold instead of ahead of it, so the title played before the restored words, and the cut's saved overlays no longer lined up and stayed trimmed. Cut pieces now record that held time (heldAfter), and restoring puts the footage back ahead of it. Holds are stored in output time but sit at a point in recording time, so one meant to sit at a cut point can land a hair before or after it, and a closing title a hair before the clips' end was swallowed by the cut. Holds within 1e-9 s of a cut point are now snapped onto it before a cut or a restore.
Restore keyboard segments a cut deleted one cut at a time, take the sub-millisecond leftover a cut drops at a clip edge, give restored words their footage and trimmed length back, stop padding at kept words that overlap a cut, end pauses where audio resumes, find pauses after stretched words between waveform blocks, keep "err" and acronyms out of fillers, and reuse the clip transition geometry check.
|
@greptileai review |
| const pauses = | ||
| gap.end - gap.start >= threshold | ||
| ? clipToRange(silence, gap).filter( | ||
| (piece) => piece.end - piece.start >= threshold, | ||
| ) |
There was a problem hiding this comment.
buildTranscriptItems treats the whole gap between words as silence, even when the loaded waveform contains speech there. If transcription misses speech between words ending at 1.4s and starting at 4s, Auto Clean cuts that entire gap and removes the spoken words from the video.
audioSilence only adjusts the neighboring word edges; it never checks the audio inside the gap. When waveforms are available, limit pause cuts to ranges that are actually quiet.
Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/desktop/src/routes/editor/transcript-cuts.ts
Line: 1605-1609
Comment:
**Auto Clean cuts missed speech**
`buildTranscriptItems` treats the whole gap between words as silence, even when the loaded waveform contains speech there. If transcription misses speech between words ending at 1.4s and starting at 4s, Auto Clean cuts that entire gap and removes the spoken words from the video.
`audioSilence` only adjusts the neighboring word edges; it never checks the audio inside the gap. When waveforms are available, limit pause cuts to ranges that are actually quiet.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.Pauses came from the gaps between transcript words, so when transcription missed speech, the whole gap still read as silence even with the waveform loaded. Auto Clean cut it as a long pause, and a filler cut took the gap beside it, removing the speech either way. The recording's waveform analysis now reports where it is audible as well as where it is quiet. Pauses leave out audible audio, and a cut word only takes the quiet on its own side of audio heard in a gap, unless that audio runs into the word itself. Time without a waveform is neither, so pauses there still follow the transcript.
|
@richiemcilroy should integrate CrisperWhisper for this later. dont know if it can run locally like parakeet. this pr is only tested with parakeet. it also doesn't generate captions if video is longer than 7-8 mins. will need to split and generate captions for that. some changes in pr are related to how parakeet works, might not work same with other models. |
|
Thanks so much for this contribution, @ganganimaulik! Cap's desktop app has now fully moved from Tauri to our new native GPUI app ( |
Overview
right click on deleted word:
This pull request implements comprehensive transcript editing tools for the Cap desktop editor, introducing word/pause deletion, adjustable speech buffers, timeline-wide ripple editing, and an automated "Auto Clean" feature with batch undo/redo.
Key Features
Soft Word Deletion & Restoration
deleted: true) rather than splicing them out.Adjustable Word Buffers (Buffer Popover)
bufferStartandbufferEnd(from-0.5sto1.0s) around any deleted word by right-clicking it.Pause Detection & Deletion
⏸ 1.2s).Auto-Clean with Batch Undo
uh,um,ah,er,hmm,mhm) and pauses exceeding a configurable silence threshold.Fix pause detection before merging so Auto Clean does not remove audible speech missed by transcription.
Findings
Fix with agent prompt
Summary
Adds restorable transcript cuts, adjustable cut padding, pause controls, and Auto Clean.
The previous-review ancestry check passed, but the merge bases differed, so this review used the full PR diff. No supplied root threads had Comment numbers.
Reviews (3) · Last reviewed commit: "fix: keep transcript cut restores exact ..." · Reviewed by Greptile