Repository navigation
Generate short video clips from the dialog - #299
Merged
Merged
Conversation
Adds a make_video dialog tool that turns a request into a 5-second silent FastFrames clip on the farm's draw-api and delivers it as a Telegram animation, so clients show it like a GIF. - video/optimizer prompt: rewrites the request into a Wan-style English shot description (style, setting and light, subject, "First... Then..." action, camera move) and classifies it safe/adult/forbidden. - taskman VideoGen job on its own "video" queue with one active clip per user, a delivery obligation and the image-generation timeout. - video worker: optimizes the prompt, blocks forbidden requests, calls the draw-api fastframes_generate endpoint through Discovery, and sends the MP4 with sendAnimation (spoiler for adult clips). - !video, !видео and /video shortcuts enter the agent with make_video as the explicit intent. - VIDEO_GENERATION_ENABLED (default true) turns the tool and worker off.
The delivery obligation watcher treats a completed ticket without a result message as orphaned and sends the failure notice. The video worker now stores the id of the sent animation, or of the safety refusal, on the ticket before it completes it.
Owner
Author
|
Re Danger warnings:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a
make_videodialog tool. The bot turns a request into a 5-second silent FastFrames clip (832×480, 16 fps) on draw-api and replies with it as a Telegram animation. Clients show an MP4 without sound sent throughsendAnimationas a GIF.prompts/video/optimizer.prompt,openplotva-media::video): rewrites the request into one English Wan-style shot description (style → setting and light → subject → "First… Then…" action → camera move → look), 80–120 words, with no negations or readable text. It also classifies the request as safe, adult or forbidden. It runs on the existingmedia_prompt_optimizerroute.VideoGenjobs go to a newvideoqueue. Each user can have one active clip. The usual enqueue rate limit and a delivery obligation apply, so a lost clip still gets a failure notice.video_jobs.rs) runs these steps:send_videopermission;fastframes_generatethrough Discovery;sendAnimation, behind a spoiler when the verdict is adult.make_videofor requests for a video, clip, GIF or animation. The!video,!видеоand/video@botshortcuts enter the agent withmake_videoas the explicit intent.VIDEO_GENERATION_ENABLED(defaulttrue) disables both the tool and the worker.No schema migration: the job reuses
ImageJobData(original_text,prompt= brief,meta.caption).Verification
cargo fmt --allcargo clippy --workspace --all-targets -- -D warnings: cleancargo testforopenplotva-app(1458 passed),-dialog(110),-taskman(68),-media(63),-llm(236),-config(57),-storage(187),-telegram(237),-prompts(10)fastframes_generatealready runs on the farm: about 10 s per warm clip.