valorant-short-maker: the pipeline that generates my Valorant shorts by itself

For a few months now, a YouTube channel has been running without me touching it: @valorant_agents. Valorant agents trash-talking each other between rounds, dubbed, karaoke-subtitled, published as Shorts. Everything is generated by valorant-short-maker, a TypeScript/Bun pipeline that runs on a cron and publishes without anyone having to click anything.

Here is how it works, step by step.

What it looks like

Three frames pulled from the video generated for "Duelist Debate" (Phoenix, Yoru, and Jett):

The result live on this Short: Duelist Debate -- youtube.com/shorts/SX5Kme58aLU. On the channel, Shorts get around 1.2 to 1.5k views. Nothing huge, but it's a channel that has run by itself from day one, so the number that really counts is zero -- zero minutes spent on it once the cron is running.

The pipeline, in order

1. Writing the script -- Groq + Llama 3.3

Each run picks 3 to 4 agents at random from the 26 available, and sends Llama 3.3 70B (via Groq) a system prompt containing, for each chosen agent, a compact summary of their personality and relationships with the other agents in the scene (those personas live in src/lore/, one file per agent). The prompt enforces precise rules: one short punchy line per reply, fair rotation between characters, humor first, and above all pauses.

Concrete example with "Duelist Debate" -- Phoenix, Yoru, and Jett arguing over who gets to play duelist, generated July 6, 2026:

phoenix: I'm telling you, I've got the skills to play duelist this match.
yoru: Skills, you call burning things skills, Phoenix.
jett: I'm the fastest one here, I should play duelist.
phoenix: Fastest, but can you handle the heat, Jett [0.3] I doubt it.
yoru: Heat, ha, you think your flames are hotter than my rifts.
jett: This isn't about heat or flames, it's about speed and agility.
phoenix: Oh, I see, so now you're an expert on duelists, Yoru [0.3] that's rich.
yoru: At least I don't rely on cheap fire tricks.
jett: Cheap fire tricks, that's what you call Phoenix's abilities.
phoenix: Hey, my fire tricks have gotten us out of tight spots before [0.3] can't say the same for your rifts, Yoru.
yoru: Tight spots, you mean like the time I rifted us out of that trap.
jett: Enough, this is getting nowhere, let's just decide already.
phoenix: Fine, but I'm still saying I'm the best duelist here.
yoru: Please, you think you can take on the enemy team alone [0.3] I doubt it.
jett: I can take them on, no problem, I'm the fastest.
phoenix: Fastest, yeah, but can you outmaneuver them [0.3] that's the question.
yoru: Outmaneuver, ha, you think you can outmaneuver anyone, Phoenix.
jett: This is stupid, we're not going to agree on this.
phoenix: Fine, let's just play and see who comes out on top [0.3] I'm game if you are.
yoru: Bring it on, I'll show you what a real duelist looks like.
jett: I'm not backing down, I'm playing duelist.
phoenix: Oh, this should be good [0.3] let's see how you two do.
yoru: We'll see who comes out on top, won't we, Jett.
jett: Yeah, let's end this debate once and for all.
pause: 0.3
phoenix: Alright, let's get started then [0.3] may the best duelist win.
yoru: I'll make sure to burn you, Phoenix, not with fire, but with my rifts.
jett: I'll take you both down, no problem.

Pauses are the detail that makes the pacing feel natural: [0.3] inserted mid-line creates a 0.3s silence in the audio file without hiding the agent's on-screen circle, while a standalone pause: 1.0 line creates a real silence between two speakers, circle hidden. Without that, a TTS chaining lines without breathing sounds robotic.

2. Giving it a voice -- Piper, one model per agent

Each agent has its own Piper model (.onnx) trained specifically, stored in voices/<agent>/. The generated text goes through the matching model, which outputs a WAV. It's the same tech I use for custom voice training in general (see the article on the Piper/Kaggle pipeline) -- here applied directly in prod, on the fly, on every video generation.

3. Karaoke subtitles -- generated ASS, color pulled from the icon

The subtitling is not a plain .srt. It's a .ass (Advanced SubStation Alpha) file generated word by word, with a karaoke effect: each word lights up in a color as it's spoken, while the rest of the text stays in a neutral color. The accent color is not fixed -- it's extracted dynamically from the speaking agent's icon (a Python script runs PIL over the icon PNG, samples non-transparent pixels, and returns the dominant colors). Result: Killjoy's subtitle lights up purple, Jett's in teal-green, without any color ever hardcoded anywhere.

4. The audio-reactive circle -- one FFmpeg expression per frame

This is the most twisted part of the pipeline, and probably the one I'm proudest of. The speaking agent's round icon doesn't sit still: it zooms in and out slightly to the rhythm of its own voice.

The math reads the raw WAV of the line, computes the RMS envelope (root mean square, a measure of signal energy) frame by frame at 60 fps, normalizes by the maximum, then smooths over a 3-frame window to avoid jank. Each envelope value is then converted to a scale factor bounded by MAX_ZOOM_VARIATION (0.2, so ±20% around the base size).

The result of that computation is not applied by pixel-manipulating code -- it's translated into a giant FFmpeg conditional expression (lt(n,K)*val + between(n,K,K')*val + ..., one branch per frame group) that drives the scale video filter parameter directly. FFmpeg evaluates that expression on every rendered frame. For a few-seconds line at 60 fps, that's quickly hundreds of branches in a single expression -- hence the STEP parameter grouping frames to cap the depth.

5. Per-segment rendering, then fisheye on the intro

Each line is rendered individually: video background (random excerpt from one of the gameplay clips in bg-video/, cut to the right length), the agent circle on top with audio-reactive zoom, subtitles burned in via FFmpeg's ass filter, TTS audio mixed with the background gameplay sound.

The very first segment gets special treatment: a fisheye distortion that resolves over the first 20% of frames (lenscorrection filter evaluated frame by frame, plus a tmix=frames=3 that blends adjacent frames to fake motion blur), synced with a "whoosh" sound. It's the intro transition that makes it feel like the camera is "diving" into the scene.

6. Concatenation and final mixing

All segments are concatenated end to end, the background music (Sneaky Snitch, Kevin MacLeod, Creative Commons license) is mixed on top with audio ducking -- sidechain compression that automatically drops the music volume while an agent speaks, and raises it back during silences. Everything runs at 60 fps end to end, no framerate conversion between steps.

7. Automatic publishing

The run-cron.sh script, launched by a plain cron, activates the Python environment, loads the .env, and runs bun src/workflow.ts --upload. The --upload flag additionally triggers metadata generation (title, description, tags) and calls uploaders/upload.py, which publishes the video to YouTube and Instagram via two separate scripts (uploaders/youtube/upload.py and uploaders/instagram/). The whole chain, from LLM prompt to video online, runs with zero human intervention.

Why TypeScript/Bun instead of an all-Python thing

The choice isn't ideological -- Bun gives direct, fast access to Bun.spawn for driving FFmpeg as a subprocess, strong typing over the pipeline's data structures (Phrase, SegmentInfo), and a much faster startup runtime than Node for a script that runs on a cron every X hours. The only two Python bits in the project are where Python is genuinely the best tool: PIL for color extraction, and the upload APIs (google-api-python-client for YouTube, the Instagram Graph API stack for IG).

What it shows

This project is a good example of what you can build today with entirely free or open-source bricks: a fast, free LLM via the Groq API, a local TTS engine that runs without a dedicated GPU, FFmpeg for all video rendering -- and the glue is just a few hundred lines of TypeScript. None of those bricks is new on its own. What makes the pipeline is the arrangement: generating a coherent script with real relationships between characters, turning it into expressive audio with natural pauses, syncing a visual render to that audio's energy frame by frame, and automating the whole chain through to publishing.

---

Resources

• Repo: github.com/fox3000foxy/valorant-short-maker
• Channel: @valorant_agents

3 key points

1. The script is generated by an LLM (Groq/Llama 3.3) with per-agent personas and relationships, not a plain list of pre-written jokes.
2. The agent circle zoom is driven by an FFmpeg expression computed frame by frame from the WAV's RMS envelope -- no classic keyframe animation.
3. The whole chain, from prompt to YouTube/Instagram post, runs through a single cron job with zero human intervention.

valorant-short-maker : le pipeline qui génère mes shorts Valorant tout seul

Depuis quelques mois, une chaîne YouTube tourne sans que j'y touche : @valorant_agents. Des agents Valorant qui se chambrent entre deux rounds, doublés, sous-titrés en karaoké, publiés en Shorts. Tout est généré par valorant-short-maker, un pipeline TypeScript/Bun qui tourne en cron et publie sans que personne n'ait à cliquer sur quoi que ce soit.

Voici comment ça marche, étape par étape.

Ce que ça donne

Trois frames extraites de la vidéo générée pour "Duelist Debate" (Phoenix, Yoru et Jett) :

Le résultat en live sur ce Short : Duelist Debate -- youtube.com/shorts/SX5Kme58aLU. Sur la chaîne, les Shorts tournent autour de 1,2 à 1,5k vues. Rien d'énorme, mais c'est une chaîne qui tourne toute seule depuis le début, donc le nombre qui compte vraiment c'est zéro -- zéro minute passée dessus une fois le cron lancé.

Le pipeline, dans l'ordre

1. Écrire le script -- Groq + Llama 3.3

Chaque run pioche 3 à 4 agents au hasard parmi les 26 disponibles, et envoie à Llama 3.3 70B (via Groq) un prompt système qui contient, pour chaque agent choisi, un résumé compact de sa personnalité et de ses relations avec les autres agents présents dans la scène (ces personas vivent dans src/lore/, un fichier par agent). Le prompt impose des règles précises : une phrase courte et percutante par réplique, rotation équitable entre les personnages, humour en priorité, et surtout des pauses.

Exemple concret avec "Duelist Debate" -- Phoenix, Yoru et Jett se disputent pour savoir qui jouera duelist, généré le 6 juillet 2026 :

phoenix: I'm telling you, I've got the skills to play duelist this match.
yoru: Skills, you call burning things skills, Phoenix.
jett: I'm the fastest one here, I should play duelist.
phoenix: Fastest, but can you handle the heat, Jett [0.3] I doubt it.
yoru: Heat, ha, you think your flames are hotter than my rifts.
jett: This isn't about heat or flames, it's about speed and agility.
phoenix: Oh, I see, so now you're an expert on duelists, Yoru [0.3] that's rich.
yoru: At least I don't rely on cheap fire tricks.
jett: Cheap fire tricks, that's what you call Phoenix's abilities.
phoenix: Hey, my fire tricks have gotten us out of tight spots before [0.3] can't say the same for your rifts, Yoru.
yoru: Tight spots, you mean like the time I rifted us out of that trap.
jett: Enough, this is getting nowhere, let's just decide already.
phoenix: Fine, but I'm still saying I'm the best duelist here.
yoru: Please, you think you can take on the enemy team alone [0.3] I doubt it.
jett: I can take them on, no problem, I'm the fastest.
phoenix: Fastest, yeah, but can you outmaneuver them [0.3] that's the question.
yoru: Outmaneuver, ha, you think you can outmaneuver anyone, Phoenix.
jett: This is stupid, we're not going to agree on this.
phoenix: Fine, let's just play and see who comes out on top [0.3] I'm game if you are.
yoru: Bring it on, I'll show you what a real duelist looks like.
jett: I'm not backing down, I'm playing duelist.
phoenix: Oh, this should be good [0.3] let's see how you two do.
yoru: We'll see who comes out on top, won't we, Jett.
jett: Yeah, let's end this debate once and for all.
pause: 0.3
phoenix: Alright, let's get started then [0.3] may the best duelist win.
yoru: I'll make sure to burn you, Phoenix, not with fire, but with my rifts.
jett: I'll take you both down, no problem.

Les pauses sont le détail qui rend le rythme naturel : [0.3] inséré au milieu d'une réplique crée un silence de 0.3s dans le fichier audio sans couper le cercle de l'agent à l'écran, alors qu'une ligne pause: 1.0 à part entière crée un vrai silence entre deux locuteurs, cercle caché. Sans ça, un TTS qui enchaîne les répliques sans respirer sonne robotique.

2. Donner une voix -- Piper, un modèle par agent

Chaque agent a son propre modèle Piper (.onnx) entraîné spécifiquement, stocké dans voices/<agent>/. Le texte généré passe dans le modèle correspondant, ce qui sort un WAV. C'est la même techno que j'utilise pour le training de voix custom en général (voir l'article sur le pipeline Piper/Kaggle) -- ici appliquée directement en prod, à la volée, à chaque génération de vidéo.

3. Sous-titres karaoké -- ASS généré, couleur extraite de l'icône

Le sous-titrage n'est pas un simple .srt. C'est un fichier .ass (Advanced SubStation Alpha) généré mot par mot, avec un effet karaoké : chaque mot s'illumine dans une couleur au fur et à mesure qu'il est prononcé, pendant que le reste du texte reste dans une couleur neutre. La couleur d'accent n'est pas fixe -- elle est extraite dynamiquement de l'icône de l'agent qui parle (un script Python fait tourner PIL sur le PNG de l'icône, échantillonne les pixels non-transparents, et renvoie les couleurs dominantes). Résultat : le sous-titre de Killjoy s'illumine en violet, celui de Jett en bleu-vert, sans qu'aucune couleur n'ait été codée en dur quelque part.

4. Le cercle audio-réactif -- une expression FFmpeg par frame

C'est la partie la plus tordue du pipeline, et probablement celle dont je suis le plus fier. L'icône ronde de l'agent qui parle ne reste pas statique : elle zoome et dézoome légèrement au rythme de sa propre voix.

Le calcul se fait en lisant le WAV brut de la réplique, en calculant l'enveloppe RMS (root mean square, une mesure de l'énergie du signal) frame par frame à 60 fps, en normalisant par le maximum, puis en lissant sur une fenêtre de 3 frames pour éviter les à-coups. Chaque valeur d'enveloppe est ensuite convertie en un facteur d'échelle borné par MAX_ZOOM_VARIATION (0.2, donc ±20% autour de la taille de base).

Le résultat de ce calcul n'est pas appliqué via du code qui manipule des pixels -- c'est traduit en une immense expression conditionnelle FFmpeg (lt(n,K)*val + between(n,K,K')*val + ..., une branche par groupe de frames) qui pilote directement le paramètre scale du filtre vidéo. FFmpeg évalue cette expression à chaque frame du rendu. Pour une réplique de quelques secondes à 60 fps, ça fait vite des centaines de branches dans une seule expression -- d'où le paramètre STEP qui regroupe les frames pour limiter la profondeur.

5. Rendu par segment, puis fisheye sur l'intro

Chaque réplique est rendue individuellement : fond vidéo (extrait aléatoire d'un des clips de gameplay dans bg-video/, coupé à la bonne durée), cercle de l'agent par-dessus avec le zoom audio-réactif, sous-titres incrustés via le filtre ass d'FFmpeg, audio TTS mixé avec le son du gameplay en fond.

Le tout premier segment reçoit un traitement spécial : une distorsion fisheye qui se résorbe progressivement sur les 20% premiers frames (filtre lenscorrection évalué frame par frame, plus un tmix=frames=3 qui mélange les frames adjacentes pour simuler du motion blur), synchronisée avec un bruit de "whoosh". C'est la transition d'intro qui donne l'impression que la caméra "rentre" dans la scène.

6. Concaténation et mixage final

Tous les segments sont concaténés bout à bout, la musique de fond (Sneaky Snitch, Kevin MacLeod, licence Creative Commons) est mixée par-dessus avec du ducking audio -- une compression sidechain qui baisse automatiquement le volume de la musique pendant qu'un agent parle, et qui remonte pendant les silences. Le tout tourne en 60 fps de bout en bout, aucune conversion de framerate entre les étapes.

7. Publication automatique

Le script run-cron.sh, lancé par un cron classique, active l'environnement Python, charge le .env, et lance bun src/workflow.ts --upload. Le flag --upload déclenche en plus la génération de métadonnées (titre, description, tags) et appelle uploaders/upload.py, qui publie la vidéo sur YouTube et Instagram via deux scripts séparés (uploaders/youtube/upload.py et uploaders/instagram/). Toute la chaîne, du prompt LLM à la vidéo en ligne, tourne sans intervention humaine.

Pourquoi TypeScript/Bun plutôt qu'un truc tout Python

Le choix n'est pas idéologique -- c'est que Bun donne un accès direct et rapide à Bun.spawn pour piloter FFmpeg en sous-processus, un typage fort sur les structures de données du pipeline (Phrase, SegmentInfo), et un runtime largement plus rapide au démarrage que Node pour un script qui tourne en cron toutes les X heures. Les deux seuls bouts de Python dans le projet sont là où Python est réellement le mieux outillé : PIL pour l'extraction de couleurs, et les APIs d'upload (google-api-python-client pour YouTube, la stack Instagram Graph API pour IG).

Ce que ça illustre

Ce projet est un bon exemple de ce qu'on peut construire aujourd'hui avec des briques entièrement gratuites ou open source : un LLM rapide et gratuit via l'API Groq, un moteur TTS local qui tourne sans GPU dédié, FFmpeg pour tout le rendu vidéo -- et le liant, ce n'est que quelques centaines de lignes de TypeScript. Aucune de ces briques n'est nouvelle individuellement. Ce qui fait le pipeline, c'est l'agencement : générer un script cohérent avec de vraies relations entre personnages, le transformer en audio expressif avec des pauses naturelles, synchroniser un rendu visuel sur l'énergie de cet audio frame par frame, et automatiser toute la chaîne jusqu'à la publication.

---

Ressources

• Repo : github.com/fox3000foxy/valorant-short-maker
• Chaîne : @valorant_agents

3 points clés

1. Le script est généré par un LLM (Groq/Llama 3.3) avec des personas et relations par agent, pas une simple liste de blagues pré-écrites.
2. Le zoom du cercle d'agent est piloté par une expression FFmpeg calculée frame par frame à partir de l'enveloppe RMS du WAV -- pas d'animation par keyframes classique.
3. Toute la chaîne, du prompt au post YouTube/Instagram, tourne via un seul cron job sans intervention humaine.

valorant-short-maker：全自动生产 Valorant 短视频的流水线

有几个月，一个 YouTube 频道在无人值守地转：@valorant_agents。Valorant 特工两回合之间互呛，配音，卡拉 OK 字幕，发 Shorts。全由 valorant-short-maker 生成，TypeScript/Bun 流水线跑 cron，没人点一下就发布了。

下面按步骤拆它怎么工作的。

长什么样

"Duelist Debate"那期（Phoenix、Yoru、Jett）生成的三帧：

成片在这条 Short：Duelist Debate -- youtube.com/shorts/SX5Kme58aLU。频道 Shorts 大概 1.2 到 1.5k 播放。不大，但这是从 cron 启动就没人管过的频道，所以真正重要的数字是零，启动后花零分钟。

流水线，按顺序

1. 写脚本，Groq + Llama 3.3

每次运行从 26 个特工里随机抽 3 到 4 个，发给 Llama 3.3 70B（走 Groq），系统提示词里带每个出场特工的精简 persona 和相互关系（persona 在 src/lore/，一特工一文件）。提示词定死规则：每句短而狠、人物轮流讲、幽默优先，最重要的是停顿。

"Duelist Debate"实例，Phoenix、Yoru、Jett 争谁打决斗，2026 年 7 月 6 日生成：

phoenix: I'm telling you, I've got the skills to play duelist this match.
yoru: Skills, you call burning things skills, Phoenix.
jett: I'm the fastest one here, I should play duelist.
phoenix: Fastest, but can you handle the heat, Jett [0.3] I doubt it.
yoru: Heat, ha, you think your flames are hotter than my rifts.
jett: This isn't about heat or flames, it's about speed and agility.
phoenix: Oh, I see, so now you're an expert on duelists, Yoru [0.3] that's rich.
yoru: At least I don't rely on cheap fire tricks.
jett: Cheap fire tricks, that's what you call Phoenix's abilities.
phoenix: Hey, my fire tricks have gotten us out of tight spots before [0.3] can't say the same for your rifts, Yoru.
yoru: Tight spots, you mean like the time I rifted us out of that trap.
jett: Enough, this is getting nowhere, let's just decide already.
phoenix: Fine, but I'm still saying I'm the best duelist here.
yoru: Please, you think you can take on the enemy team alone [0.3] I doubt it.
jett: I can take them on, no problem, I'm the fastest.
phoenix: Fastest, yeah, but can you outmaneuver them [0.3] that's the question.
yoru: Outmaneuver, ha, you think you can outmaneuver anyone, Phoenix.
jett: This is stupid, we're not going to agree on this.
phoenix: Fine, let's just play and see who comes out on top [0.3] I'm game if you are.
yoru: Bring it on, I'll show you what a real duelist looks like.
jett: I'm not backing down, I'm playing duelist.
phoenix: Oh, this should be good [0.3] let's see how you two do.
yoru: We'll see who comes out on top, won't we, Jett.
jett: Yeah, let's end this debate once and for all.
pause: 0.3
phoenix: Alright, let's get started then [0.3] may the best duelist win.
yoru: I'll make sure to burn you, Phoenix, not with fire, but with my rifts.
jett: I'll take you both down, no problem.

停顿是节奏自然的关键细节：句中 [0.3] 在音频里造 0.3 秒静音，屏幕上特工圆圈不消失；单独一行 pause: 1.0 是两个说话人之间的真停顿，圆圈隐藏。没有这个，连珠炮的 TTS 听着就是机器人。

2. 配音，Piper，一特工一模型

每个特工有自己训的 Piper 模型（.onnx），存在 voices/<agent>/。文本丢进对应模型，出 WAV。这就是我训自定义 voice 那套流水线（见 Piper/Kaggle 那篇），这里直接上生产，每次生成视频现跑。

3. 卡拉 OK 字幕，生成的 ASS，颜色从图标取

字幕不是简单 .srt。是逐词生成的 .ass（Advanced SubStation Alpha）文件，卡拉 OK 特效：说到的词按颜色点亮，其余保持中性色。点亮色不是写死的，从说话特工的图标动态取（Python 脚本调 PIL 读 PNG 图标，采样非透明像素，返回主色）。结果：Killjoy 的字幕紫里点亮，Jett 的蓝绿点亮，没有一个颜色是手写死的。

4. 音频反应圆圈，一帧一个 FFmpeg 表达式

流水线最绕、我也最得意的部分。说话特工的圆形头像不是静止的：跟着自己 voice 的能量轻微缩放。

计算是读这句的 WAV 原始数据，按 60 fps 逐帧算 RMS 包络（root mean square，信号能量度量），按最大归一，再 3 帧窗口平滑去抖。每个包络值转成 MAX_ZOOM_VARIATION（0.2，基准尺寸上下 20%）框住的缩放系数。

计算结果不经过像素代码，直接翻译成巨型条件 FFmpeg 表达式（lt(n,K)*val + between(n,K,K')*val + ...，一组帧一个分支），直接驱动视频滤镜的 scale 参数。FFmpeg 渲染时每帧求这个表达式。一句几秒的台词 60 fps，轻松几百个分支，所以有 STEP 参数按组压深度。

5. 分段渲染，开场加鱼眼

每句单独渲染：背景视频（bg-video/ 里随机抽段 gameplay，按时长切）、特工圆圈叠上加音频反应缩放、FFmpeg ass 滤镜烧字幕、TTS 音频和背景 gameplay 声混音。

第一段特殊处理：前 20% 帧渐消的鱼眼畸变（lenscorrection 滤镜逐帧求值，加 tmix=frames=3 混相邻帧模拟动态模糊），配"whoosh"音效同步。这是开场转场，让镜头有"扎进"场景的感觉。

6. 拼接与终混

全部段首尾拼起来，背景音乐（Sneaky Snitch，Kevin MacLeod，CC 许可）用闪避混进去，sidechain 压缩，特工一开口音乐自动压低，停顿回升。全程 60 fps，中间没有任何帧率转换。

7. 自动发布

run-cron.sh 被经典 cron 拉起，激活 Python 环境，加载 .env，跑 bun src/workflow.ts --upload。--upload flag 触发元数据生成（标题、描述、标签），再调 uploaders/upload.py，经两个独立脚本（uploaders/youtube/upload.py 管 YouTube、uploaders/instagram/ 管 IG）把视频发出去。从 LLM 提示词到视频上线，全程无人。

为什么 TypeScript/Bun，而不全 Python

不是意识形态，是 Bun 给 Bun.spawn 起 FFmpeg 子进程又快又直接，流水线数据结构（Phrase、SegmentInfo）有强类型，cron 每 X 小时跑一次的话 Node 启动还更快。项目里仅有的两块 Python 在 Python 真有优势的地方：PIL 取颜色，上传 API（YouTube 的 google-api-python-client、IG 的 Graph API 栈）。

它说明了什么

这个项目很好地说明今天用全免费或开源砖头能搭出什么：Groq 上又快又免费的 LLM，本地跑不用独显的 TTS 引擎，FFmpeg 包办全部视频渲染，粘合只用了几百行 TypeScript。没有一块砖是新的。流水线全在编排：按人物关系生成连贯剧本，带自然停顿转成有表现力的音频，按音频能量逐帧同步渲染画面，整条链自动走到发布。

---

资源

• 仓库：github.com/fox3000foxy/valorant-short-maker
• 频道：@valorant_agents

三句话总结

1. 脚本由 LLM（Groq/Llama 3.3）按特工 persona 和关系生成，不是预写的笑话列表。
2. 特工圆圈的缩放由 FFmpeg 表达式驱动，按 WAV 的 RMS 包络逐帧算出，不是经典关键帧动画。
3. 整条链从提示词到 YouTube/Instagram 发帖，一个 cron 任务无人值守跑完。

valorant-short-maker：Valorantショートを1人で作るパイプライン

数か月前からYouTubeチャンネルが触らずに回ってる。@valorant_agentsだ。ラウンド合間に言い合うValorantエージェント、吹き替え付き、カラオケ字幕付き、Shorts公開だ。全部valorant-short-makerが作ってる。cronで回るTypeScriptやBunのパイプラインで、誰も何も押さずに公開する。

仕組みを段階ごとに見せる。

できるもの

"Duelist Debate"（Phoenix、Yoru、Jett）用に作った動画から抜いた3フレームだ。

live の結果はこのShortだ。Duelist Debate -- youtube.com/shorts/SX5Kme58aLU。チャンネルではShortsが1.2から1.5k再生くらいだ。大したことないけど、最初から1人で回るチャンネルだ。本当に大事な数はゼロだ。一度cron回したら使う時間ゼロ分ってことだ。

パイプライン、順番に

1. 台本を書く。GroqとLlama 3.3

各runは26人から3、4エージェントをランダムに引く。Groq経由のLlama 3.3 70Bにシステムプロンプトを送る。選んだ各エージェントの短い人格要約と場面内他エージェントとの関係付きだ（このpersonaはsrc/lore/にいる。エージェントごとに1ファイル）。プロンプトは正確な規則を課す。台詞ごとに短く刺さる1文、人物間の公平な交代、ユーモア優先、何より pause だ。

"Duelist Debate"の具体例。Phoenix、Yoru、Jettが誰がduelistやるか争う。2026年7月6日生成だ。

phoenix: I'm telling you, I've got the skills to play duelist this match.
yoru: Skills, you call burning things skills, Phoenix.
jett: I'm the fastest one here, I should play duelist.
phoenix: Fastest, but can you handle the heat, Jett [0.3] I doubt it.
yoru: Heat, ha, you think your flames are hotter than my rifts.
jett: This isn't about heat or flames, it's about speed and agility.
phoenix: Oh, I see, so now you're an expert on duelists, Yoru [0.3] that's rich.
yoru: At least I don't rely on cheap fire tricks.
jett: Cheap fire tricks, that's what you call Phoenix's abilities.
phoenix: Hey, my fire tricks have gotten us out of tight spots before [0.3] can't say the same for your rifts, Yoru.
yoru: Tight spots, you mean like the time I rifted us out of that trap.
jett: Enough, this is getting nowhere, let's just decide already.
phoenix: Fine, but I'm still saying I'm the best duelist here.
yoru: Please, you think you can take on the enemy team alone [0.3] I doubt it.
jett: I can take them on, no problem, I'm the fastest.
phoenix: Fastest, yeah, but can you outmaneuver them [0.3] that's the question.
yoru: Outmaneuver, ha, you think you can outmaneuver anyone, Phoenix.
jett: This is stupid, we're not going to agree on this.
phoenix: Fine, let's just play and see who comes out on top [0.3] I'm game if you are.
yoru: Bring it on, I'll show you what a real duelist looks like.
jett: I'm not backing down, I'm playing duelist.
phoenix: Oh, this should be good [0.3] let's see how you two do.
yoru: We'll see who comes out on top, won't we, Jett.
jett: Yeah, let's end this debate once and for all.
pause: 0.3
phoenix: Alright, let's get started then [0.3] may the best duelist win.
yoru: I'll make sure to burn you, Phoenix, not with fire, but with my rifts.
jett: I'll take you both down, no problem.

pause がリズムを自然にする詳細だ。台詞真ん中の[0.3]は画面のエージェント円を切らず音声ファイルに0.3秒無音を作る。対して単独行のpause: 1.0は2話者間に本物無音を作る。円隠しだ。これなしでは、呼吸せず台詞つなげるTTSはロボットっぽく聞こえる。

2. 声を付ける。Piper、エージェントごとに1モデル

各エージェントは独自Piperモデル（.onnx）を持つ。専用訓練したものだ。voices/<agent>/に保存する。作ったテキストは対応モデルを通ってWAVを出す。普段のカスタム声訓練に使うのと同じ技術だ（PiperやKaggleパイプライン記事参照）。ここでは本番に直接、その場で、各動画生成に当ててる。

3. カラオケ字幕。作るASS、アイコンから取る色

字幕は単純.srtじゃない。語ごとに作った.assファイル（Advanced SubStation Alpha）で、カラオケ効果付きだ。各語が言われるにつれて色で光って、残りテキストは中立色のまま残る。強調色は固定じゃない。話すエージェントのアイコンから動的抽出する（PythonスクリプトがPILでアイコンPNGを回して、透明外ピクセルを抽出して、支配色を返す）。結果：Killjoy字幕は紫に光って、Jettは青緑に光る。どこにも色はハードコードされてない。

4. 音反応円。1フレーム1式のFFmpeg

パイプラインでいちばんひねくれた部分で、たぶんいちばん誇らしい。話すエージェントの丸アイコンは静止しない。自分の声のリズムで少しズームインやアウトする。

計算は台詞の素WAVを読んで、60fpsで1フレームごとにRMS envelope（root mean square、信号 energy の尺度）を計算して、最大で正規化して、ぎくしゃく避けに3フレーム窓で滑らかにする。各 envelope 値をMAX_ZOOM_VARIATION（0.2、基準サイズ周りプラスマイナス20パーセント）で区切った倍率に変換する。

この計算結果はピクセルいじるコード経由で当てない。巨大なFFmpeg条件式（lt(n,K)*val + between(n,K,K')*val + ...、フレーム群ごとに1分岐）に翻訳して、動画フィルタのscaleパラメータを直接操る。FFmpegはレンダー各フレームでこの式を評価する。60fpsで数秒の台詞なら1式にすぐ数百分岐だ。だからSTEPパラメータでフレームをまとめて深さを抑える。

5. 区切りごとレンダー、イントロに fisheye

各台詞は個別レンダーする。背景動画（bg-video/内ゲームプレイ切り抜きからランダム抜粋、正しい長さに切り）、上に音反応ズーム付きエージェント円、FFmpeg assフィルタ経由で焼く字幕、背景ゲームプレイ音と混ぜるTTS音声だ。

最初区切りは特別処理を受ける。頭20パーセントフレームで少しずつ解けるfisheye歪みだ（1フレームごとに評価するlenscorrectionフィルタと、動きぼかし再現に隣フレーム混ぜるtmix=frames=3）。"whoosh"ノイズと同期する。カメラが場面に「入る」感じを出すイントロ遷移だ。

6. 連結と最終混合

全区切りを端から端へつなげて、背景音楽（Sneaky Snitch、Kevin MacLeod、Creative Commonsライセンス）をダッキング付きで上に混ぜる。エージェントが話す間音楽音量を自動下げて、無音で上げる sidechain 圧縮だ。全体60fpsで端から端まで、段階間でフレームレート変換なしだ。

7. 自動公開

run-cron.shスクリプトは古典cronで起動して、Python環境を入れて、.envを読んで、bun src/workflow.ts --uploadを走らせる。--uploadフラグはメタデータ生成（タイトル、説明、タグ）を起こしてuploaders/upload.pyを呼ぶ。2つの別スクリプト経由でYouTubeとInstagramに動画公開する（uploaders/youtube/upload.pyとuploaders/instagram/）。LLMプロンプトからオンライン動画まで全連鎖が人手なしで回る。

なぜ全部PythonじゃなくTypeScriptやBunか

選択は思想じゃない。BunがBun.spawnでFFmpegを子プロセス操る直接高速アクセスをくれて、パイプライン構造（Phrase、SegmentInfo）に強い型付けをくれて、X時間ごとにcron回るスクリプト用にNodeより起動が断然速いランタイムだからだ。プロジェクト内Python2つはPythonが本当に最適道具なところにある。色抽出用PILと、アップロードAPI（YouTube用google-api-python-client、IG用Instagram Graph API一式）だ。

示すこと

このプロジェクトは今日無料かオープンソースの部品だけで何作れるかの良い例だ。Groq API経由の速く無料なLLM、専用GPUなしで回るローカルTTSエンジン、全部の動画レンダー用FFmpeg。つなぎはTypeScript数百行だけだ。部品はどれも個別には新しくない。パイプラインを作るのは配置だ。人物間の本物関係付き一貫台本を作って、自然 pause 付き表現豊かな音声に変えて、この音声 energy に1フレームごとに視覚レンダーを同期させて、公開まで全連鎖を自動化する。

---

リソース

• リポジトリ：github.com/fox3000foxy/valorant-short-maker
• チャンネル：@valorant_agents

要点3つ

1. 台本はエージェントごとのpersonaと関係付きLLM（GroqやLlama 3.3）で作る。単なる書き置き冗談一覧じゃない。
2. エージェント円ズームはWAVのRMS envelope から1フレームごとに計算したFFmpeg式で操る。古典 keyframes アニメじゃない。
3. プロンプトからYouTubeやInstagram投稿まで全連鎖が単一cronジョブで人手なしに回る。

valorant-short-maker: 제 Valorant 쇼츠를 혼자 만들어내는 파이프라인

몇 달째 손대지 않아도 돌아가는 YouTube 채널이 있어요. @valorant_agents. 라운드 사이에 서로 놀리는 Valorant 요원들, 더빙되고 가라오케 자막이 입혀져 쇼츠로 올라가요. 전부 TypeScript와 Bun으로 짠 파이프라인 valorant-short-maker가 크론으로 돌면서, 누구도 클릭하지 않아도 게시해요.

어떻게 동작하는지 단계별로 보여 드릴게요.

결과물

"Duelist Debate"(Phoenix, Yoru, Jett) 영상을 위해 만든 세 프레임이에요.

라이브 결과는 이 쇼츠예요. Duelist Debate -- youtube.com/shorts/SX5Kme58aLU. 채널의 쇼츠는 1,2천에서 1,5천 조회수쯤 나와요. 대단한 건 아니지만, 처음부터 혼자 돌아가는 채널이니 진짜 중요한 숫자는 0이에요. 크론을 걸고 나서 들인 시간 0분이요.

순서대로 보는 파이프라인

1. 대본 쓰기: Groq + Llama 3.3

매 실행마다 26명 중 3~4명의 요원을 무작위로 뽑고, Llama 3.3 70B(Groq 경유)에 시스템 프롬프트를 보내요. 선택된 각 요원마다 성격과 장면에 함께 있는 다른 요원들과의 관계를 간결하게 요약한 내용이 들어가요 (이 페르소나는 src/lore/에 요원별 파일로 살아요). 프롬프트는 정확한 규칙을 강제해요. 대사마다 짧고 강렬한 한 문장, 캐릭터 사이 공평한 순환, 유머 우선, 그리고 무엇보다 쉼표.

2026년 7월 6일에 만든 "Duelist Debate"의 실제 예시예요. Phoenix, Yoru, Jett가 누가 듀얼리스트를 할지 다퉈요.

phoenix: I'm telling you, I've got the skills to play duelist this match.
yoru: Skills, you call burning things skills, Phoenix.
jett: I'm the fastest one here, I should play duelist.
phoenix: Fastest, but can you handle the heat, Jett [0.3] I doubt it.
yoru: Heat, ha, you think your flames are hotter than my rifts.
jett: This isn't about heat or flames, it's about speed and agility.
phoenix: Oh, I see, so now you're an expert on duelists, Yoru [0.3] that's rich.
yoru: At least I don't rely on cheap fire tricks.
jett: Cheap fire tricks, that's what you call Phoenix's abilities.
phoenix: Hey, my fire tricks have gotten us out of tight spots before [0.3] can't say the same for your rifts, Yoru.
yoru: Tight spots, you mean like the time I rifted us out of that trap.
jett: Enough, this is getting nowhere, let's just decide already.
phoenix: Fine, but I'm still saying I'm the best duelist here.
yoru: Please, you think you can take on the enemy team alone [0.3] I doubt it.
jett: I can take them on, no problem, I'm the fastest.
phoenix: Fastest, yeah, but can you outmaneuver them [0.3] that's the question.
yoru: Outmaneuver, ha, you think you can outmaneuver anyone, Phoenix.
jett: This is stupid, we're not going to agree on this.
phoenix: Fine, let's just play and see who comes out on top [0.3] I'm game if you are.
yoru: Bring it on, I'll show you what a real duelist looks like.
jett: I'm not backing down, I'm playing duelist.
phoenix: Oh, this should be good [0.3] let's see how you two do.
yoru: We'll see who comes out on top, won't we, Jett.
jett: Yeah, let's end this debate once and for all.
pause: 0.3
phoenix: Alright, let's get started then [0.3] may the best duelist win.
yoru: I'll make sure to burn you, Phoenix, not with fire, but with my rifts.
jett: I'll take you both down, no problem.

쉼표는 리듬을 자연스럽게 만드는 세부 사항이에요. 대사 중간에 넣은 [0.3]은 화면의 요원 원을 자르지 않고 오디오 파일에 0.3초 무음을 만들고, 통째로 한 줄인 pause: 1.0은 화자 사이에 원을 숨긴 진짜 무음을 만들어요. 이게 없으면 숨 쉬지 않고 대사를 잇는 TTS는 로봇처럼 들려요.

2. 목소리 입히기: Piper, 요원별 모델 하나씩

각 요원은 전문적으로 훈련된 자신만의 Piper 모델(.onnx)을 가지고 있고 voices/<agent>/에 저장돼 있어요. 만들어진 텍스트가 해당 모델을 거쳐 WAV가 나와요. 일반적으로 커스텀 음성 훈련에 쓰는 것과 같은 기술이에요 (Piper와 Kaggle 파이프라인 글 참고). 여기서는 매 영상 만들 때마다 즉석에서 바로 프로덕션에 적용해요.

3. 가라오케 자막: 만들어낸 ASS, 아이콘에서 뽑은 색

자막은 단순한 .srt가 아니에요. 단어별로 만들어진 .ass(Advanced SubStation Alpha) 파일이고 가라오케 효과가 있어요. 발음되는 대로 각 단어가 강조 색으로 켜지고, 나머지 텍스트는 중립 색으로 남아요. 강조 색은 고정되어 있지 않아요. 말하는 요원의 아이콘에서 동적으로 뽑아요 (Python 스크립트가 아이콘 PNG에 PIL을 돌려서 투명하지 않은 픽셀을 샘플링하고 지배적인 색들을 돌려줘요). 결과: Killjoy 자막은 보라색으로 켜지고, Jett 자막은 청록색으로 켜져요. 어디에도 색이 하드코딩되어 있지 않아요.

4. 오디오 반응 원: 프레임마다 FFmpeg 표현식 하나

파이프라인에서 가장 꼬인 부분이고, 아마 가장 자랑스러운 부분이에요. 말하는 요원의 둥근 아이콘이 가만히 있지 않아요. 자기 목소리 리듬에 맞춰 살짝 줌인 줌아웃해요.

계산은 대사의 날것 WAV를 읽고, 60fps로 프레임마다 RMS(root mean square, 신호 에너지 측정) 엔벨로프를 구하고, 최댓값으로 정규화한 뒤, 끊김을 피하려고 3프레임 창으로 스무딩해요. 각 엔벨로프 값을 MAX_ZOOM_VARIATION(0.2, 기본 크기 기준 앞뒤 20퍼센트)으로 묶인 스케일 계수로 바꿔요.

이 계산 결과는 픽셀을 만지는 코드로 적용되지 않아요. 거대한 FFmpeg 조건 표현식(lt(n,K)*val + between(n,K,K')*val + ..., 프레임 묶음별 분기 하나)으로 번역돼서 비디오 필터의 scale 매개변수를 직접 조종해요. FFmpeg가 렌더링의 매 프레임마다 이 표현식을 평가해요. 60fps에서 몇 초짜리 대사면 하나의 표현식에 금방 수백 개 분기가 생겨요. 그래서 깊이를 제한하려고 프레임을 묶는 STEP 매개변수가 있어요.

5. 구간별 렌더링, 인트로는 피시아이

각 대사는 따로 렌더링돼요. 배경 영상(bg-video/의 게임플레이 클립 중 무작위 발췌를 적절한 길이로 자름), 위에 오디오 반응 원, FFmpeg ass 필터로 입힌 자막, 배경 게임플레이 소리와 믹스된 TTS 오디오.

맨 처음 구간은 특별 처리를 받아요. 처음 20퍼센트 프레임에 걸쳐 서서히 풀리는 피시아이 왜곡(프레임마다 평가되는 lenscorrection 필터에 모션 블러 흉내용으로 인접 프레임을 섞는 tmix=frames=3 추가)이 "whoosh" 소리와 동기화돼요. 카메라가 장면 속으로 "들어가는" 느낌을 주는 인트로 전환이에요.

6. 이어붙이기와 최종 믹싱

모든 구간을 끝에서 끝으로 이어붙이고, 배경 음악(Sneaky Snitch, Kevin MacLeod, Creative Commons 라이선스)을 오디오 더킹으로 위에 믹스해요. 요원이 말하는 동안 음악 볼륨을 자동으로 낮췄다가 무음 구간에 다시 올리는 사이드체인 압축이에요. 전체가 처음부터 끝까지 60fps로 돌고, 단계 사이에 프레임레이트 변환은 없어요.

7. 자동 게시

고전 크론이 실행하는 run-cron.sh 스크립트가 Python 환경을 켜고 .env를 불러온 뒤 bun src/workflow.ts --upload를 실행해요. --upload 플래그는 메타데이터(제목, 설명, 태그) 만들기에 더해 uploaders/upload.py를 호출해요. 두 개의 분리 스크립트(uploaders/youtube/upload.py와 uploaders/instagram/)로 YouTube와 Instagram에 영상을 올려요. LLM 프롬프트부터 온라인 영상까지 전체 체인이 사람 개입 없이 돌아가요.

왜 완전 Python이 아니라 TypeScript와 Bun인지

선택이 이념적이지는 않아요. Bun이 서브프로세스로 FFmpeg를 조종하는 Bun.spawn에 직접 빠르고 닿기 좋고, 파이프라인 자료구조(Phrase, SegmentInfo)에 강한 타이핑을 주고, 몇 시간마다 크론으로 도는 스크립트에 Node보다 시작이 훨씬 빠른 런타임이라서예요. 프로젝트에서 Python 조각 둘은 Python이 실제로 가장 잘 갖춰진 곳에만 있어요. 색 뽑기용 PIL과 업로드 API(YouTube용 google-api-python-client, IG용 Instagram Graph API 스택)요.

보여주는 것

이 프로젝트는 오늘날 완전히 무료이거나 오픈 소스인 벽돌들로 무엇을 만들 수 있는지 보여주는 좋은 예예요. Groq API로 빠르고 무료인 LLM, 전용 GPU 없이 도는 로컬 TTS 엔진, 모든 비디오 렌더링용 FFmpeg. 그리고 이어주는 것은 TypeScript 몇백 줄뿐이에요. 벽돌 하나하나는 새것이 아니에요. 파이프라인을 만드는 것은 배치예요. 캐릭터 사이 진짜 관계가 담긴 일관된 대본을 만들고, 자연스러운 쉼표가 있는 표현력 있는 오디오로 바꾸고, 그 오디오의 에너지에 프레임마다 시각 렌더링을 동기화하고, 게시까지 전체 체인을 자동화하는 것.

---

자료

• 저장소: github.com/fox3000foxy/valorant-short-maker
• 채널: @valorant_agents

핵심 3가지

1. 대본은 미리 쓴 농담 목록이 아니라 요원별 페르소나와 관계가 담긴 LLM(Groq와 Llama 3.3)이 만들어요.
2. 요원 원의 줌은 고전 키프레임 애니메이션이 아니라 WAV의 RMS 엔벨로프에서 프레임마다 계산된 FFmpeg 표현식이 조종해요.
3. 프롬프트부터 YouTube와 Instagram 게시물까지 전체 체인이 사람 개입 없이 크론 작업 하나로 돌아가요.

valorant-short-maker: Valorant shortlarımı kendi kendine üreten pipeline

Birkaç aydır bir YouTube kanalı ben elimi sürmeden dönüyor: @valorant_agents. İki round arasında birbirine laf sokan Valorant ajanları, seslendirilmiş, karaoke altyazılı, Shorts olarak yayınlanmış. Her şey valorant-short-maker tarafından üretiliyor, cron ile çalışan ve kimsenin hiçbir şeye tıklamasına gerek kalmadan yayınlayan bir TypeScript/Bun pipelineı.

İşte adım adım nasıl çalıştığı.

Ortaya çıkan şey

"Duelist Debate" (Phoenix, Yoru ve Jett) için üretilen videodan çıkarılmış üç kare:

Canlı sonuç bu Shortta: Duelist Debate: youtube.com/shorts/SX5Kme58aLU. Kanalda Shortlar 1,2 ila 1,5k izlenme civarında dönüyor. Muazzam bir şey değil ama kanal başından beri kendi kendine dönüyor, o yüzden gerçekten önemli olan sayı sıfır: cron başlatıldığından beri üzerine harcanan sıfır dakika.

Pipeline, sırasıyla

1. Senaryoyu yazmak: Groq + Llama 3.3

Her run mevcut 26 ajan arasından rastgele 3 ila 4 ajan seçer ve Groq üzerinden Llama 3.3 70B modeline, sahnede bulunan her seçili ajan için kişiliğinin ve diğer ajanlarla ilişkilerinin kompakt bir özetini içeren bir sistem istemi gönderir (bu personalar src/lore/ içinde yaşar, ajan başına bir dosya). İstem kesin kurallar dayatır: replik başına kısa ve çarpıcı tek cümle, karakterler arasında adil dönüş, öncelik mizahta ve hepsinden önemlisi duraklamalarda.

Somut bir örnek "Duelist Debate" ile: hangisinin duelist oynayacağını tartışan Phoenix, Yoru ve Jett, 6 Temmuz 2026da üretildi:

phoenix: I'm telling you, I've got the skills to play duelist this match.
yoru: Skills, you call burning things skills, Phoenix.
jett: I'm the fastest one here, I should play duelist.
phoenix: Fastest, but can you handle the heat, Jett [0.3] I doubt it.
yoru: Heat, ha, you think your flames are hotter than my rifts.
jett: This isn't about heat or flames, it's about speed and agility.
phoenix: Oh, I see, so now you're an expert on duelists, Yoru [0.3] that's rich.
yoru: At least I don't rely on cheap fire tricks.
jett: Cheap fire tricks, that's what you call Phoenix's abilities.
phoenix: Hey, my fire tricks have gotten us out of tight spots before [0.3] can't say the same for your rifts, Yoru.
yoru: Tight spots, you mean like the time I rifted us out of that trap.
jett: Enough, this is getting nowhere, let's just decide already.
phoenix: Fine, but I'm still saying I'm the best duelist here.
yoru: Please, you think you can take on the enemy team alone [0.3] I doubt it.
jett: I can take them on, no problem, I'm the fastest.
phoenix: Fastest, yeah, but can you outmaneuver them [0.3] that's the question.
yoru: Outmaneuver, ha, you think you can outmaneuver anyone, Phoenix.
jett: This is stupid, we're not going to agree on this.
phoenix: Fine, let's just play and see who comes out on top [0.3] I'm game if you are.
yoru: Bring it on, I'll show you what a real duelist looks like.
jett: I'm not backing down, I'm playing duelist.
phoenix: Oh, this should be good [0.3] let's see how you two do.
yoru: We'll see who comes out on top, won't we, Jett.
jett: Yeah, let's end this debate once and for all.
pause: 0.3
phoenix: Alright, let's get started then [0.3] may the best duelist win.
yoru: I'll make sure to burn you, Phoenix, not with fire, but with my rifts.
jett: I'll take you both down, no problem.

Duraklamalar ritmi doğal yapan detay: bir repliğin ortasına yerleştirilen [0.3], ekrandaki ajan çemberini kesmeden ses dosyasında 0.3 saniyelik sessizlik yaratır, oysa tek başına bir pause: 1.0 satırı iki konuşmacı arasında çember gizlenmiş gerçek bir sessizlik yaratır. Bunlar olmadan nefes almadan replikleri zincirleyen bir TTS robotik duyulur.

2. Ses vermek: Piper, ajan başına bir model

Her ajanın özel olarak eğitilmiş kendi Piper modeli (.onnx) vardır, voices/<ajan>/ içinde saklanır. Üretilen metin ilgili modelden geçer ve bir WAV çıkar. Bu, genel olarak özel ses eğitimi için kullandığım teknolojinin aynısı (Piper/Kaggle pipelineı hakkındaki makaleye bak): burada doğrudan prod ortamında, anında, her video üretiminde uygulanmış.

3. Karaoke altyazıları: üretilmiş ASS, ikondan çıkarılan renk

Altyazı basit bir .srt değil. Kelime kelime üretilmiş bir .ass (Advanced SubStation Alpha) dosyası, karaoke efektli: her kelime telaffuz edildikçe bir vurgu renginde aydınlanır, metnin geri kalanı nötr bir renkte kalır. Vurgu rengi sabit değil: konuşan ajanın ikonundan dinamik olarak çıkarılır (bir Python scripti ikonun PNGsi üzerinde PIL çalıştırır, saydam olmayan pikselleri örnekler ve baskın renkleri döndürür). Sonuç: Killjoyun altyazısı mor, Jettinki mavi yeşil aydınlanır ve hiçbir renk bir yerde sabit kodlanmamıştır.

4. Sese tepki veren çember: kare başına bir FFmpeg ifadesi

Bu pipelineın en çarpık kısmı ve muhtemelen en gurur duyduğum kısım. Konuşan ajanın yuvarlak ikonu statik kalmaz: kendi sesinin ritmiyle hafifçe büyüyüp küçülür.

Hesap, repliğin ham WAV dosyasını okuyarak, 60 fps ile kare kare RMS zarfını (root mean square, sinyal enerjisinin bir ölçüsü) hesaplayarak, maksimuma göre normalize ederek, sonra sarsıntıları önlemek için 3 karelik bir pencerede yumuşatarak yapılır. Her zarf değeri sonra MAX_ZOOM_VARIATION (0.2, yani temel boyutun ±%20si) ile sınırlı bir ölçek faktörüne dönüştürülür.

Bu hesabın sonucu pikselleri manipüle eden bir kodla uygulanmaz: doğrudan video filtresinin scale parametresini süren devasa bir koşullu FFmpeg ifadesine (lt(n,K)*val + between(n,K,K')*val + ..., kare grubu başına bir dal) çevrilir. FFmpeg bu ifadeyi renderın her karesinde değerlendirir. 60 fps ile birkaç saniyelik bir replik için bu tek bir ifadede hızla yüzlerce dal yapar, o yüzden derinliği sınırlamak için kareleri gruplayan STEP parametresi vardır.

5. Parça parça render, sonra introda balıkgözü

Her replik tek tek render edilir: video arka planı (bg-video/ içindeki oynanış kliplerinden rastgele alıntı, doğru süreye kesilmiş), üstte sese tepki veren zoom ile ajan çemberi, FFmpegin ass filtresiyle gömülü altyazılar, arka planda oynanış sesiyle karıştırılmış TTS sesi.

En ilk parça özel bir işlem görür: ilk %20 kare üzerinde giderek çözülen bir balıkgözü bozulması (kare kare değerlendirilen lenscorrection filtresi artı bitişik kareleri karıştırıp hareket bulanıklığı simüle eden bir tmix=frames=3), bir "whoosh" sesiyle senkronize. Kameranın sahnenin içine "girdiği" izlenimini veren intro geçişi bu.

6. Birleştirme ve final miksaj

Tüm parçalar uç uca birleştirilir, arka plan müziği (Sneaky Snitch, Kevin MacLeod, Creative Commons lisansı) ses ducking ile üstüne karıştırılır: bir ajan konuşurken müziğin sesini otomatik kısan, sessizliklerde yükselten bir sidechain kompresörü. Her şey baştan sona 60 fps döner, adımlar arasında kare hızı dönüşümü yok.

7. Otomatik yayın

Klasik bir cron tarafından çalıştırılan run-cron.sh scripti Python ortamını etkinleştirir, .env dosyasını yükler ve bun src/workflow.ts --upload komutunu çalıştırır. --upload bayrağı ayrıca meta veri üretimini (başlık, açıklama, etiketler) tetikler ve iki ayrı script üzerinden YouTube ve Instagramda videoyu yayınlayan uploaders/upload.py dosyasını çağırır (uploaders/youtube/upload.py ve uploaders/instagram/). LLM isteminden çevrimiçi videoya tüm zincir insan müdahalesi olmadan döner.

Neden tamamen Python bir şey yerine TypeScript/Bun

Seçim ideolojik değil: Bun, FFmpegi alt süreç olarak sürmek için Bun.spawn üzerinden doğrudan ve hızlı erişim, pipeline veri yapıları üzerinde güçlü tipleme (Phrase, SegmentInfo) ve X saatte bir cronda dönen bir script için Nodetan belirgin şekilde daha hızlı başlatma süresi verir. Projedeki iki Python parçası gerçekten Pythonın en iyi donanımlı olduğu yerlerdedir: renk çıkarma için PIL ve yükleme APIleri (YouTube için google-api-python-client, IG için Instagram Graph API yığını).

Bunun gösterdiği şey

Bu proje bugün tamamen ücretsiz ya da açık kaynak tuğlalarla neler inşa edilebileceğinin güzel bir örneği: Groq API üzerinden hızlı ve ücretsiz bir LLM, adanmış GPU olmadan çalışan yerel bir TTS motoru, tüm video render için FFmpeg, ve tutkal olarak sadece birkaç yüz satır TypeScript. Bu tuğlaların hiçbiri tek başına yeni değil. Pipelineı yapan şey düzenlenişi: gerçek karakter ilişkileriyle tutarlı bir senaryo üretmek, onu doğal duraklamalarla etkileyici sese dönüştürmek, görsel renderı bu sesin enerjisine kare kare senkronize etmek ve tüm zinciri yayına kadar otomatikleştirmek.

---

Kaynaklar

• Repo: github.com/fox3000foxy/valorant-short-maker
• Kanal: @valorant_agents

3 kilit nokta

1. Senaryo önceden yazılmış şaka listesinden değil, ajan başına personalar ve ilişkilerle bir LLM (Groq/Llama 3.3) tarafından üretilir.
2. Ajan çemberinin zoomu klasik keyframe animasyonuyla değil, WAVın RMS zarfından kare kare hesaplanan bir FFmpeg ifadesiyle sürülür.
3. İstemden YouTube/Instagram gönderisine tüm zincir insan müdahalesi olmadan tek bir cron işiyle döner.

valorant-short-maker: la pipeline che genera i miei short Valorant da sola

Da qualche mese, un canale YouTube gira senza che lo tocchi: @valorant_agents. Agenti Valorant che si prendono in giro tra un round e l'altro, doppiati, sottotitolati in karaoke, pubblicati come Shorts. Tutto è generato da valorant-short-maker, una pipeline TypeScript/Bun che gira in cron e pubblica senza che nessuno debba cliccare su niente.

Ecco come funziona, passo passo.

Cosa dà

Tre frame estratti dal video generato per "Duelist Debate" (Phoenix, Yoru e Jett):

Il risultato in live su questo Short: Duelist Debate: youtube.com/shorts/SX5Kme58aLU. Sul canale, gli Short girano intorno a 1,2 - 1,5k viste. Niente di enorme, ma è un canale che gira da solo dall'inizio, quindi il numero che conta davvero è zero: zero minuti passati sopra una volta lanciato il cron.

La pipeline, in ordine

1. Scrivere lo script: Groq + Llama 3.3

Ogni run pesca 3 o 4 agenti a caso tra i 26 disponibili, e invia a Llama 3.3 70B (via Groq) un prompt di sistema che contiene, per ogni agente scelto, un riassunto compatto della sua personalità e delle sue relazioni con gli altri agenti presenti nella scena (queste persona vivono in src/lore/, un file per agente). Il prompt impone regole precise: una frase corta e incisiva per battuta, rotazione equa tra i personaggi, umorismo in priorità, e soprattutto pause.

Esempio concreto con "Duelist Debate": Phoenix, Yoru e Jett litigano per sapere chi giocherà duelist, generato il 6 luglio 2026:

phoenix: I'm telling you, I've got the skills to play duelist this match.
yoru: Skills, you call burning things skills, Phoenix.
jett: I'm the fastest one here, I should play duelist.
phoenix: Fastest, but can you handle the heat, Jett [0.3] I doubt it.
yoru: Heat, ha, you think your flames are hotter than my rifts.
jett: This isn't about heat or flames, it's about speed and agility.
phoenix: Oh, I see, so now you're an expert on duelists, Yoru [0.3] that's rich.
yoru: At least I don't rely on cheap fire tricks.
jett: Cheap fire tricks, that's what you call Phoenix's abilities.
phoenix: Hey, my fire tricks have gotten us out of tight spots before [0.3] can't say the same for your rifts, Yoru.
yoru: Tight spots, you mean like the time I rifted us out of that trap.
jett: Enough, this is getting nowhere, let's just decide already.
phoenix: Fine, but I'm still saying I'm the best duelist here.
yoru: Please, you think you can take on the enemy team alone [0.3] I doubt it.
jett: I can take them on, no problem, I'm the fastest.
phoenix: Fastest, yeah, but can you outmaneuver them [0.3] that's the question.
yoru: Outmaneuver, ha, you think you can outmaneuver anyone, Phoenix.
jett: This is stupid, we're not going to agree on this.
phoenix: Fine, let's just play and see who comes out on top [0.3] I'm game if you are.
yoru: Bring it on, I'll show you what a real duelist looks like.
jett: I'm not backing down, I'm playing duelist.
phoenix: Oh, this should be good [0.3] let's see how you two do.
yoru: We'll see who comes out on top, won't we, Jett.
jett: Yeah, let's end this debate once and for all.
pause: 0.3
phoenix: Alright, let's get started then [0.3] may the best duelist win.
yoru: I'll make sure to burn you, Phoenix, not with fire, but with my rifts.
jett: I'll take you both down, no problem.

Le pause sono il dettaglio che rende il ritmo naturale: [0.3] inserito nel mezzo di una battuta crea un silenzio di 0.3s nel file audio senza tagliare il cerchio dell'agente a schermo, mentre una riga pause: 1.0 a sé stante crea un vero silenzio tra due parlanti, cerchio nascosto. Senza questo, un TTS che concatena battute senza respirare suona robotico.

2. Dare una voce: Piper, un modello per agente

Ogni agente ha il suo modello Piper (.onnx) allenato apposta, salvato in voices/<agent>/. Il testo generato passa nel modello corrispondente, il che tira fuori un WAV. È la stessa tech che uso per il training di voci custom in generale (vedi l'articolo sulla pipeline Piper/Kaggle): qui applicata direttamente in prod, al volo, a ogni generazione di video.

3. Sottotitoli karaoke: ASS generato, colore estratto dall'icona

Il sottotitolo non è un semplice .srt. È un file .ass (Advanced SubStation Alpha) generato parola per parola, con effetto karaoke: ogni parola si illumina in un colore via via che è pronunciata, mentre il resto del testo resta in un colore neutro. Il colore di accento non è fisso: è estratto dinamicamente dall'icona dell'agente che parla (uno script Python fa girare PIL sul PNG dell'icona, campiona i pixel non trasparenti, e restituisce i colori dominanti). Risultato: il sottotitolo di Killjoy si illumina in viola, quello di Jett in blu-verde, senza che alcun colore sia stato hardcodato da nessuna parte.

4. Il cerchio audio-reattivo: un'espressione FFmpeg per frame

È la parte più storta della pipeline, e probabilmente quella di cui sono più fiero. L'icona rotonda dell'agente che parla non resta statica: zooma e dezooma leggermente a ritmo della sua voce.

Il calcolo avviene leggendo il WAV grezzo della battuta, calcolando l'inviluppo RMS (root mean square, una misura dell'energia del segnale) frame per frame a 60 fps, normalizzando per il massimo, poi smussando su una finestra di 3 frame per evitare scatti. Ogni valore di inviluppo è poi convertito in un fattore di scala limitato da MAX_ZOOM_VARIATION (0.2, quindi ±20% intorno alla taglia di base).

Il risultato di questo calcolo non è applicato via codice che manipola pixel: è tradotto in un'immensa espressione condizionale FFmpeg (lt(n,K)*val + between(n,K,K')*val + ..., un ramo per gruppo di frame) che pilota direttamente il parametro scale del filtro video. FFmpeg valuta questa espressione a ogni frame del rendering. Per una battuta di qualche secondo a 60 fps, fa in fretta centinaia di rami in una sola espressione: da cui il parametro STEP che raggruppa i frame per limitare la profondità.

5. Rendering per segmento, poi fisheye sull'intro

Ogni battuta è resa individualmente: sfondo video (estratto casuale di uno dei clip di gameplay in bg-video/, tagliato alla giusta durata), cerchio dell'agente sopra col zoom audio-reattivo, sottotitoli incollati via il filtro ass di FFmpeg, audio TTS mixato col suono del gameplay in sottofondo.

Il primissimo segmento riceve un trattamento speciale: una distorsione fisheye che si riassorbe progressivamente sul primo 20% dei frame (filtro lenscorrection valutato frame per frame, più un tmix=frames=3 che miscela i frame adiacenti per simulare motion blur), sincronizzata con un rumore di "whoosh". È la transizione di intro che dà l'impressione che la camera "entri" nella scena.

6. Concatenazione e mix finale

Tutti i segmenti sono concatenati capo a coda, la musica di sottofondo (Sneaky Snitch, Kevin MacLeod, licenza Creative Commons) è mixata sopra con ducking audio: una compressione sidechain che abbassa automaticamente il volume della musica mentre un agente parla, e rialza durante i silenzi. Il tutto gira in 60 fps da cima a fondo, nessuna conversione di framerate tra i passi.

7. Pubblicazione automatica

Lo script run-cron.sh, lanciato da un cron classico, attiva l'ambiente Python, carica il .env, e lancia bun src/workflow.ts --upload. Il flag --upload innesca in più la generazione di metadati (titolo, descrizione, tag) e chiama uploaders/upload.py, che pubblica il video su YouTube e Instagram via due script separati (uploaders/youtube/upload.py e uploaders/instagram/). Tutta la catena, dal prompt LLM al video online, gira senza intervento umano.

Perché TypeScript/Bun invece di una cosa tutta Python

La scelta non è ideologica: è che Bun dà un accesso diretto e rapido a Bun.spawn per pilotare FFmpeg in sottoprocessi, una tipizzazione forte sulle strutture dati della pipeline (Phrase, SegmentInfo), e un runtime ampiamente più rapido all'avvio di Node per uno script che gira in cron ogni X ore. Gli unici due pezzi di Python nel progetto stanno dove Python è davvero meglio attrezzata: PIL per l'estrazione di colori, e le API di upload (google-api-python-client per YouTube, lo stack Instagram Graph API per IG).

Cosa illustra

Questo progetto è un buon esempio di ciò che possiamo costruire oggi con mattoni interamente gratis o open source: un LLM rapido e gratis via l'API Groq, un motore TTS locale che gira senza GPU dedicata, FFmpeg per tutto il rendering video: e il legante, sono solo qualche centinaio di righe di TypeScript. Nessuno di questi mattoni è nuovo individualmente. Ciò che fa la pipeline, è l'assemblaggio: generare uno script coerente con vere relazioni tra personaggi, trasformarlo in audio espressivo con pause naturali, sincronizzare un rendering visivo sull'energia di quest'audio frame per frame, e automatizzare tutta la catena fino alla pubblicazione.

---

Risorse

• Repo: github.com/fox3000foxy/valorant-short-maker
• Canale: @valorant_agents

3 punti chiave

1. Lo script è generato da un LLM (Groq/Llama 3.3) con persona e relazioni per agente, non una semplice lista di battute pre-scritte.
2. Lo zoom del cerchio di agente è pilotato da un'espressione FFmpeg calcolata frame per frame dall'inviluppo RMS del WAV: niente animazione per keyframe classica.
3. Tutta la catena, dal prompt al post YouTube/Instagram, gira via un solo cron job senza intervento umano.

valorant-short-maker: die Pipeline, die meine Valorant-Shorts allein generiert

Seit ein paar Monaten läuft ein YouTube-Kanal, ohne dass ich ihn anfasse: @valorant_agents. Valorant-Agenten, die sich zwischen zwei Runden necken, synchronisiert, mit Karaoke-Untertiteln, als Shorts veröffentlicht. Alles wird von valorant-short-maker generiert, einer TypeScript/Bun-Pipeline, die per Cron läuft und veröffentlicht, ohne dass irgendwer irgendwas klicken muss.

So funktioniert es, Schritt für Schritt.

Was dabei rauskommt

Drei Frames aus dem für "Duelist Debate" generierten Video (Phoenix, Yoru und Jett):

Das Ergebnis live auf diesem Short: Duelist Debate -- youtube.com/shorts/SX5Kme58aLU. Auf dem Kanal liegen die Shorts bei etwa 1,2 bis 1,5k Views. Nichts Riesiges, aber es ist ein Kanal, der seit Anfang an allein läuft, also ist die Zahl, die wirklich zählt, null, null Minuten darauf verwendet, sobald der Cron läuft.

Die Pipeline, der Reihe nach

1. Das Skript schreiben, Groq + Llama 3.3

Jeder Run zieht 3 bis 4 Agenten zufällig aus den 26 verfügbaren und schickt an Llama 3.3 70B (via Groq) einen Systemprompt, der für jeden gewählten Agenten eine kompakte Zusammenfassung seiner Persönlichkeit und seiner Beziehungen zu den anderen in der Szene anwesenden Agenten enthält (diese Personas leben in src/lore/, eine Datei pro Agent). Der Prompt setzt präzise Regeln: Ein kurzer knackiger Satz pro Replik, faire Rotation zwischen den Charakteren, Humor prioritär, und vor allem Pausen.

Konkretes Beispiel mit "Duelist Debate", Phoenix, Yoru und Jett streiten, wer Duelist spielen darf, generiert am 6. Juli 2026:

phoenix: I'm telling you, I've got the skills to play duelist this match.
yoru: Skills, you call burning things skills, Phoenix.
jett: I'm the fastest one here, I should play duelist.
phoenix: Fastest, but can you handle the heat, Jett [0.3] I doubt it.
yoru: Heat, ha, you think your flames are hotter than my rifts.
jett: This isn't about heat or flames, it's about speed and agility.
phoenix: Oh, I see, so now you're an expert on duelists, Yoru [0.3] that's rich.
yoru: At least I don't rely on cheap fire tricks.
jett: Cheap fire tricks, that's what you call Phoenix's abilities.
phoenix: Hey, my fire tricks have gotten us out of tight spots before [0.3] can't say the same for your rifts, Yoru.
yoru: Tight spots, you mean like the time I rifted us out of that trap.
jett: Enough, this is getting nowhere, let's just decide already.
phoenix: Fine, but I'm still saying I'm the best duelist here.
yoru: Please, you think you can take on the enemy team alone [0.3] I doubt it.
jett: I can take them on, no problem, I'm the fastest.
phoenix: Fastest, yeah, but can you outmaneuver them [0.3] that's the question.
yoru: Outmaneuver, ha, you think you can outmaneuver anyone, Phoenix.
jett: This is stupid, we're not going to agree on this.
phoenix: Fine, let's just play and see who comes out on top [0.3] I'm game if you are.
yoru: Bring it on, I'll show you what a real duelist looks like.
jett: I'm not backing down, I'm playing duelist.
phoenix: Oh, this should be good [0.3] let's see how you two do.
yoru: We'll see who comes out on top, won't we, Jett.
jett: Yeah, let's end this debate once and for all.
pause: 0.3
phoenix: Alright, let's get started then [0.3] may the best duelist win.
yoru: I'll make sure to burn you, Phoenix, not with fire, but with my rifts.
jett: I'll take you both down, no problem.

Die Pausen sind das Detail, das den Rhythmus natürlich macht: [0.3] mitten in einer Replik eingefügt erzeugt eine 0.3s-Stille in der Audiodatei, ohne den Agentenkreis auf dem Bildschirm abzuschneiden, während eine eigenständige pause: 1.0-Zeile eine echte Stille zwischen zwei Sprechern erzeugt, Kreis versteckt. Ohne das klingt ein TTS, das Repliken ohne Atmen aneinanderreiht, roboterhaft.

2. Eine Stimme geben, Piper, ein Modell pro Agent

Jeder Agent hat sein eigenes speziell trainiertes Piper-Modell (.onnx), gespeichert in voices/<agent>/. Der generierte Text läuft durch das entsprechende Modell, was ein WAV ausspuckt. Es ist dieselbe Techno, die ich allgemein für Custom-Stimmtraining nutze (siehe den Artikel über die Piper/Kaggle-Pipeline), hier direkt in Prod angewendet, on the fly, bei jeder Videogenerierung.

3. Karaoke-Untertitel, generiertes ASS, Farbe aus dem Icon extrahiert

Die Untertitelung ist kein simples .srt. Es ist eine Wort für Wort generierte .ass-Datei (Advanced SubStation Alpha) mit Karaoke-Effekt: Jedes Wort leuchtet in einer Farbe auf, während es ausgesprochen wird, während der Rest des Textes in einer neutralen Farbe bleibt. Die Akzentfarbe ist nicht fest, sie wird dynamisch aus dem Icon des sprechenden Agenten extrahiert (ein Python-Skript lässt PIL über das PNG des Icons laufen, sampled die nicht-transparenten Pixel und gibt die dominanten Farben zurück). Ergebnis: Killjoys Untertitel leuchtet lila auf, Jetts in Blaugrün, ohne dass irgendeine Farbe irgendwo hartcodiert wurde.

4. Der audioreaktive Kreis, ein FFmpeg-Ausdruck pro Frame

Das ist der verdrehteste Teil der Pipeline und wahrscheinlich der, auf den ich am stolzesten bin. Das runde Icon des sprechenden Agenten bleibt nicht statisch: Es zoomt leicht im Rhythmus seiner eigenen Stimme rein und raus.

Die Berechnung passiert, indem das rohe WAV der Replik gelesen wird, die RMS-Hüllkurve (root mean square, ein Maß der Signalenergie) Frame für Frame bei 60 fps berechnet wird, durch das Maximum normalisiert, dann über ein 3-Frame-Fenster geglättet, um Ruckler zu vermeiden. Jeder Hüllkurvenwert wird dann in einen Skalierungsfaktor umgewandelt, begrenzt durch MAX_ZOOM_VARIATION (0.2, also ±20 % um die Basisgröße).

Das Ergebnis dieser Berechnung wird nicht über Code angewendet, der Pixel manipuliert, es wird in einen riesigen bedingten FFmpeg-Ausdruck übersetzt (lt(n,K)*val + between(n,K,K')*val + ..., ein Zweig pro Frame-Gruppe), der direkt den scale-Parameter des Videofilters steuert. FFmpeg evaluiert diesen Ausdruck bei jedem Frame des Renderings. Für eine Replik von wenigen Sekunden bei 60 fps sind das schnell Hunderte Zweige in einem einzigen Ausdruck, daher der STEP-Parameter, der Frames gruppiert, um die Tiefe zu begrenzen.

5. Segmentweises Rendern, dann Fisheye aufs Intro

Jede Replik wird einzeln gerendert: Videohintergrund (zufälliger Ausschnitt aus einem der Gameplay-Clips in bg-video/, auf die richtige Dauer geschnitten), Agentenkreis darüber mit audioreaktivem Zoom, Untertitel eingebrannt via FFmpegs ass-Filter, TTS-Audio gemischt mit dem Gameplay-Sound im Hintergrund.

Das allererste Segment bekommt eine Spezialbehandlung: Eine Fisheye-Verzerrung, die sich über die ersten 20 % der Frames progressiv zurückbildet (lenscorrection-Filter Frame für Frame evaluiert, plus ein tmix=frames=3, das benachbarte Frames mischt, um Motion Blur zu simulieren), synchronisiert mit einem "Whoosh"-Geräusch. Das ist die Intro-Transition, die den Eindruck gibt, dass die Kamera in die Szene "eintaucht".

6. Konkatenation und finaler Mix

Alle Segmente werden aneinandergereiht, die Hintergrundmusik (Sneaky Snitch, Kevin MacLeod, Creative-Commons-Lizenz) wird darüber gemischt mit Audio-Ducking, einer Sidechain-Kompression, die die Musiklautstärke automatisch senkt, während ein Agent spricht, und in den Stillen wieder hochfährt. Alles läuft durchgehend in 60 fps, keine Framerate-Konvertierung zwischen den Schritten.

7. Automatische Veröffentlichung

Das Skript run-cron.sh, von einem klassischen Cron gestartet, aktiviert die Python-Umgebung, lädt die .env und startet bun src/workflow.ts --upload. Das --upload-Flag löst zusätzlich die Metadatengenerierung aus (Titel, Beschreibung, Tags) und ruft uploaders/upload.py auf, das das Video via zwei separater Skripte auf YouTube und Instagram veröffentlicht (uploaders/youtube/upload.py und uploaders/instagram/). Die ganze Kette, vom LLM-Prompt bis zum Video online, läuft ohne menschlichen Eingriff.

Warum TypeScript/Bun statt etwas ganz Python

Die Wahl ist nicht ideologisch, sondern Bun gibt direkten und schnellen Zugriff auf Bun.spawn, um FFmpeg in Subprozessen zu steuern, ein starkes Typing auf den Pipeline-Datenstrukturen (Phrase, SegmentInfo) und eine beim Start deutlich schnellere Runtime als Node für ein Skript, das per Cron alle X Stunden läuft. Die einzigen beiden Python-Stücke im Projekt sind dort, wo Python wirklich am besten ausgerüstet ist: PIL für die Farbextraktion und die Upload-APIs (google-api-python-client für YouTube, der Instagram-Graph-API-Stack für IG).

Was das veranschaulicht

Dieses Projekt ist ein gutes Beispiel dafür, was man heute mit vollständig gratis oder Open-Source-Bausteinen bauen kann: Ein schnelles gratis LLM via Groq-API, eine lokale TTS-Engine, die ohne dedizierte GPU läuft, FFmpeg für alles Videorendering, und das Bindemittel sind nur ein paar hundert Zeilen TypeScript. Kein einziger dieser Bausteine ist individuell neu. Was die Pipeline ausmacht, ist das Arrangement: Ein kohärentes Skript mit echten Beziehungen zwischen Charakteren generieren, es mit natürlichen Pausen in expressives Audio verwandeln, ein visuelles Rendering Frame für Frame auf die Energie dieses Audios synchronisieren und die ganze Kette bis zur Veröffentlichung automatisieren.

---

Ressourcen

• Repo: github.com/fox3000foxy/valorant-short-maker
• Kanal: @valorant_agents

3 Kernpunkte

1. Das Skript wird von einem LLM generiert (Groq/Llama 3.3) mit Personas und Beziehungen pro Agent, keine simple Liste vorgeschriebener Witze.
2. Der Zoom des Agentenkreises wird von einem FFmpeg-Ausdruck gesteuert, Frame für Frame aus der RMS-Hüllkurve des WAV berechnet, keine klassische Keyframe-Animation.
3. Die ganze Kette, vom Prompt bis zum YouTube/Instagram-Post, läuft über einen einzigen Cron-Job ohne menschlichen Eingriff.

valorant-short-maker : пайплайн, сам генерирующий мои шортсы Valorant

Уже несколько месяцев YouTube-канал крутится, а я его не трогаю : @valorant_agents. Агенты Valorant, подкалывающие друг друга между раундами, озвученные, с субтитрами в караоке, публикуемые как Shorts. Всё генерируется valorant-short-maker, пайплайном TypeScript/Bun, который крутится в cron и публикует так, что никому ничего нажимать не надо.

Вот как это работает, шаг за шагом.

Что получается

Три кадра, вытащенные из сгенерированного видео для "Duelist Debate" (Phoenix, Yoru и Jett) :

Результат вживую в этом Short : Duelist Debate -- youtube.com/shorts/SX5Kme58aLU. На канале шортсы крутятся в районе 1,2-1,5k просмотров. Ничего огромного, но это канал, крутящийся сам с начала, так что число, которое реально считается, это ноль -- ноль минут, потраченных на него после запуска cron.

Пайплайн по порядку

1. Написать скрипт -- Groq + Llama 3.3

Каждый запуск тянет 3-4 случайных агентов из 26 доступных и отправляет в Llama 3.3 70B (через Groq) системный промпт, содержащий на каждого выбранного агента компактное резюме его личности и отношений с другими агентами в сцене (эти персоны живут в src/lore/, файл на агента). Промпт навязывает точные правила : одна короткая хлёсткая фраза на реплику, честная ротация между персонажами, юмор в приоритете и главное паузы.

Конкретный пример с "Duelist Debate" -- Phoenix, Yoru и Jett спорят, кто будет играть дуэлиста, сгенерировано 6 июля 2026 :

// Реплики скрипта оставлены на английском, как их сгенерировал LLM.
phoenix: I'm telling you, I've got the skills to play duelist this match.
yoru: Skills, you call burning things skills, Phoenix.
jett: I'm the fastest one here, I should play duelist.
phoenix: Fastest, but can you handle the heat, Jett [0.3] I doubt it.
yoru: Heat, ha, you think your flames are hotter than my rifts.
jett: This isn't about heat or flames, it's about speed and agility.
phoenix: Oh, I see, so now you're an expert on duelists, Yoru [0.3] that's rich.
yoru: At least I don't rely on cheap fire tricks.
jett: Cheap fire tricks, that's what you call Phoenix's abilities.
phoenix: Hey, my fire tricks have gotten us out of tight spots before [0.3] can't say the same for your rifts, Yoru.
yoru: Tight spots, you mean like the time I rifted us out of that trap.
jett: Enough, this is getting nowhere, let's just decide already.
phoenix: Fine, but I'm still saying I'm the best duelist here.
yoru: Please, you think you can take on the enemy team alone [0.3] I doubt it.
jett: I can take them on, no problem, I'm the fastest.
phoenix: Fastest, yeah, but can you outmaneuver them [0.3] that's the question.
yoru: Outmaneuver, ha, you think you can outmaneuver anyone, Phoenix.
jett: This is stupid, we're not going to agree on this.
phoenix: Fine, let's just play and see who comes out on top [0.3] I'm game if you are.
yoru: Bring it on, I'll show you what a real duelist looks like.
jett: I'm not backing down, I'm playing duelist.
phoenix: Oh, this should be good [0.3] let's see how you two do.
yoru: We'll see who comes out on top, won't we, Jett.
jett: Yeah, let's end this debate once and for all.
pause: 0.3
phoenix: Alright, let's get started then [0.3] may the best duelist win.
yoru: I'll make sure to burn you, Phoenix, not with fire, but with my rifts.
jett: I'll take you both down, no problem.

Паузы это деталь, делающая ритм естественным : [0.3] внутри реплики создаёт тишину 0.3 с в аудиофайле, не пряча круг агента на экране, а отдельная строка pause: 1.0 создаёт настоящую тишину между двумя говорящими со скрытым кругом. Без этого TTS, шпарящий реплики без дыхания, звучит роботизированно.

2. Дать голос -- Piper, модель на агента

У каждого агента своя модель Piper (.onnx), обученная специально, хранится в voices/<agent>/. Сгенерированный текст проходит в соответствующую модель, на выходе WAV. Та же технология, что я использую для обучения кастомных голосов вообще (см. статью про пайплайн Piper/Kaggle) -- здесь применена прямо в проде, на лету, при каждой генерации видео.

3. Караоке-субтитры -- сгенерированный ASS, цвет из иконки

Субтитры это не простой .srt. Это файл .ass (Advanced SubStation Alpha), сгенерированный пословно, с эффектом караоке : каждое слово подсвечивается цветом по мере произнесения, а остаток текста остаётся нейтральным цветом. Акцентный цвет не фиксирован -- он динамически извлекается из иконки говорящего агента (скрипт Python гоняет PIL по PNG иконки, семплирует непрозрачные пиксели и возвращает доминантные цвета). Итог : субтитр Killjoy подсвечивается фиолетовым, Jett сине-зелёным, и ни один цвет нигде не захардкожен.

4. Аудиореактивный круг -- выражение FFmpeg на кадр

Самая замороченная часть пайплайна и, наверное, та, которой я горжусь сильнее всего. Круговая иконка говорящего агента не стоит статично : она слегка зумится в ритме собственного голоса.

Расчёт идёт чтением сырого WAV реплики, вычислением огибающей RMS (root mean square, мера энергии сигнала) покадрово на 60 fps, нормализацией по максимуму, затем сглаживанием на окне в 3 кадра, чтобы избежать рывков. Каждое значение огибающей затем конвертируется в ограниченный коэффициент масштаба через MAX_ZOOM_VARIATION (0.2, то есть ±20 % вокруг базового размера).

Результат этого расчёта применяется не кодом, манипулирующим пикселями, -- он транслируется в огромное условное выражение FFmpeg (lt(n,K)*val + between(n,K,K')*val + ..., ветка на группу кадров), напрямую управляющее параметром scale видеofильтра. FFmpeg вычисляет это выражение на каждом кадре рендера. Для реплики в несколько секунд на 60 fps это быстро сотни веток в одном выражении -- отсюда параметр STEP, группирующий кадры, чтобы ограничить глубину.

5. Рендер по сегментам, затем fisheye на интро

Каждая реплика рендерится отдельно : фоновое видео (случайный кусок одного из геймплейных клипов в bg-video/, обрезанный под нужную длительность), сверху круг агента с аудиореактивным зумом, вшитые субтитры через фильтр ass в FFmpeg, аудио TTS, смикшированное со звуком геймплея на фоне.

Самый первый сегмент получает особую обработку : искажение fisheye, постепенно рассасывающееся на первых 20 % кадров (фильтр lenscorrection, вычисляемый покадрово, плюс tmix=frames=3, смешивающий соседние кадры для имитации motion blur), синхронизированное со звуком "whoosh". Это интро-переход, дающий впечатление, что камера "входит" в сцену.

6. Конкатенация и финальный микс

Все сегменты конкатенируются встык, фоновая музыка (Sneaky Snitch, Kevin MacLeod, лицензия Creative Commons) миксуется поверх с аудио-дакингом -- сайдчейн-компрессией, автоматически приглушающей музыку, пока говорит агент, и поднимающей её в тишине. Всё крутится в 60 fps от начала до конца, никакой конверсии фреймрейта между шагами.

7. Автопубликация

Скрипт run-cron.sh, запускаемый классическим cron, активирует окружение Python, грузит .env и запускает bun src/workflow.ts --upload. Флаг --upload дополнительно триггерит генерацию метаданных (название, описание, теги) и вызывает uploaders/upload.py, публикующий видео на YouTube и в Instagram через два отдельных скрипта (uploaders/youtube/upload.py и uploaders/instagram/). Вся цепочка, от промпта LLM до видео онлайн, крутится без вмешательства человека.

Почему TypeScript/Bun, а не что-то целиком на Python

Выбор не идеологический -- просто Bun даёт прямой и быстрый доступ к Bun.spawn для управления FFmpeg в субпроцессах, строгую типизацию структур данных пайплайна (Phrase, SegmentInfo) и рантайм заметно быстрее на старте, чем Node, для скрипта, крутящегося в cron каждые X часов. Единственные два куска Python в проекте там, где Python реально лучше оснащён : PIL для извлечения цветов и API загрузки (google-api-python-client для YouTube, стек Instagram Graph API для IG).

Что это иллюстрирует

Этот проект хороший пример того, что сегодня можно собрать на полностью бесплатных или опенсорсных кирпичах : быстрый и бесплатный LLM через API Groq, локальный движок TTS, работающий без выделенного GPU, FFmpeg для всего видеорендера -- а связка это лишь несколько сотен строк TypeScript. Ни один из этих кирпичей по отдельности не нов. Пайплайн делает компоновка : сгенерировать связный скрипт с настоящими отношениями персонажей, превратить его в выразительное аудио с естественными паузами, синхронизировать визуальный рендер с энергией этого аудио покадрово и автоматизировать всю цепочку до публикации.

---

Ресурсы

• Репо : github.com/fox3000foxy/valorant-short-maker
• Канал : @valorant_agents

3 ключевых пункта

1. Скрипт генерируется LLM (Groq/Llama 3.3) с персонами и отношениями на агента, а не простым списком предписанных шуток.
2. Зум круга агента управляется выражением FFmpeg, вычисляемым покадрово из огибающей RMS WAV -- никакой классической анимации по кейфреймам.
3. Вся цепочка, от промпта до поста на YouTube/Instagram, крутится через одну cron-задачу без вмешательства человека.

valorant-short-maker: el pipeline que genera mis shorts de Valorant solo

Desde hace unos meses, un canal de YouTube funciona sin que yo lo toque: @valorant_agents. Agentes de Valorant que se pican entre dos rondas, doblados, subtitulados en karaoke, publicados como Shorts. Todo lo genera valorant-short-maker, un pipeline TypeScript/Bun que corre en cron y publica sin que nadie tenga que hacer clic en nada.

Así funciona, paso a paso.

Cómo queda

Tres fotogramas extraídos del vídeo generado para "Duelist Debate" (Phoenix, Yoru y Jett):

El resultado en directo en este Short: Duelist Debate -- youtube.com/shorts/SX5Kme58aLU. En el canal, los Shorts rondan las 1,2 a 1,5k vistas. Nada enorme, pero es un canal que funciona solo desde el principio, así que el número que de verdad cuenta es cero: cero minutos dedicados una vez lanzado el cron.

El pipeline, en orden

1. Escribir el guion: Groq + Llama 3.3

Cada ejecución elige 3 o 4 agentes al azar entre los 26 disponibles, y envía a Llama 3.3 70B (vía Groq) un prompt de sistema que contiene, para cada agente elegido, un resumen compacto de su personalidad y de sus relaciones con los demás agentes presentes en la escena (esas personas viven en src/lore/, un archivo por agente). El prompt impone reglas precisas: una frase corta y contundente por réplica, rotación equitativa entre los personajes, humor en prioridad, y sobre todo pausas.

Ejemplo concreto con "Duelist Debate": Phoenix, Yoru y Jett discuten sobre quién jugará de duelist, generado el 6 de julio de 2026:

phoenix: I'm telling you, I've got the skills to play duelist this match.
yoru: Skills, you call burning things skills, Phoenix.
jett: I'm the fastest one here, I should play duelist.
phoenix: Fastest, but can you handle the heat, Jett [0.3] I doubt it.
yoru: Heat, ha, you think your flames are hotter than my rifts.
jett: This isn't about heat or flames, it's about speed and agility.
phoenix: Oh, I see, so now you're an expert on duelists, Yoru [0.3] that's rich.
yoru: At least I don't rely on cheap fire tricks.
jett: Cheap fire tricks, that's what you call Phoenix's abilities.
phoenix: Hey, my fire tricks have gotten us out of tight spots before [0.3] can't say the same for your rifts, Yoru.
yoru: Tight spots, you mean like the time I rifted us out of that trap.
jett: Enough, this is getting nowhere, let's just decide already.
phoenix: Fine, but I'm still saying I'm the best duelist here.
yoru: Please, you think you can take on the enemy team alone [0.3] I doubt it.
jett: I can take them on, no problem, I'm the fastest.
phoenix: Fastest, yeah, but can you outmaneuver them [0.3] that's the question.
yoru: Outmaneuver, ha, you think you can outmaneuver anyone, Phoenix.
jett: This is stupid, we're not going to agree on this.
phoenix: Fine, let's just play and see who comes out on top [0.3] I'm game if you are.
yoru: Bring it on, I'll show you what a real duelist looks like.
jett: I'm not backing down, I'm playing duelist.
phoenix: Oh, this should be good [0.3] let's see how you two do.
yoru: We'll see who comes out on top, won't we, Jett.
jett: Yeah, let's end this debate once and for all.
pause: 0.3
phoenix: Alright, let's get started then [0.3] may the best duelist win.
yoru: I'll make sure to burn you, Phoenix, not with fire, but with my rifts.
jett: I'll take you both down, no problem.

Las pausas son el detalle que hace el ritmo natural: [0.3] insertado en medio de una réplica crea un silencio de 0.3s en el archivo de audio sin cortar el círculo del agente en pantalla, mientras que una línea pause: 1.0 entera crea un verdadero silencio entre dos interlocutores, círculo oculto. Sin eso, un TTS que encadena réplicas sin respirar suena robótico.

2. Darle una voz: Piper, un modelo por agente

Cada agente tiene su propio modelo Piper (.onnx) entrenado específicamente, guardado en voices/<agent>/. El texto generado pasa por el modelo correspondiente, que saca un WAV. Es la misma tecnología que uso para el entrenamiento de voces personalizadas en general (ver el artículo sobre el pipeline Piper/Kaggle): aquí aplicada directamente en producción, al vuelo, en cada generación de vídeo.

3. Subtítulos karaoke: ASS generado, color extraído del icono

El subtitulado no es un simple .srt. Es un archivo .ass (Advanced SubStation Alpha) generado palabra por palabra, con efecto karaoke: cada palabra se ilumina en un color a medida que se pronuncia, mientras el resto del texto queda en un color neutro. El color de acento no es fijo: se extrae dinámicamente del icono del agente que habla (un script Python pasa PIL sobre el PNG del icono, muestrea los píxeles no transparentes, y devuelve los colores dominantes). Resultado: el subtítulo de Killjoy se ilumina en violeta, el de Jett en azul verdoso, sin que ningún color se haya codificado a mano en ningún sitio.

4. El círculo audiorreactivo: una expresión FFmpeg por fotograma

Es la parte más retorcida del pipeline, y probablemente de la que estoy más orgulloso. El icono redondo del agente que habla no se queda estático: hace zoom in y zoom out ligeramente al ritmo de su propia voz.

El cálculo se hace leyendo el WAV en bruto de la réplica, calculando la envolvente RMS (root mean square, una medida de la energía de la señal) fotograma a fotograma a 60 fps, normalizando por el máximo, y luego suavizando sobre una ventana de 3 fotogramas para evitar tirones. Cada valor de envolvente se convierte después en un factor de escala acotado por MAX_ZOOM_VARIATION (0.2, o sea ±20% alrededor del tamaño base).

El resultado de ese cálculo no se aplica con código que manipule píxeles: se traduce en una inmensa expresión condicional de FFmpeg (lt(n,K)*val + between(n,K,K')*val + ..., una rama por grupo de fotogramas) que pilota directamente el parámetro scale del filtro de vídeo. FFmpeg evalúa esa expresión en cada fotograma del render. Para una réplica de unos segundos a 60 fps, eso suma rápido cientos de ramas en una sola expresión: de ahí el parámetro STEP que agrupa los fotogramas para limitar la profundidad.

5. Render por segmento, y luego fisheye en la intro

Cada réplica se renderiza por separado: vídeo de fondo (extracto aleatorio de uno de los clips de gameplay en bg-video/, cortado a la duración justa), círculo del agente encima con el zoom audiorreactivo, subtítulos incrustados con el filtro ass de FFmpeg, audio TTS mezclado con el sonido del gameplay de fondo.

El primer segmento recibe un tratamiento especial: una distorsión fisheye que se reabsorbe progresivamente sobre el primer 20% de los fotogramas (filtro lenscorrection evaluado fotograma a fotograma, más un tmix=frames=3 que mezcla los fotogramas adyacentes para simular motion blur), sincronizada con un ruido de "whoosh". Es la transición de intro que da la impresión de que la cámara "entra" en la escena.

6. Concatenación y mezcla final

Todos los segmentos se concatenan uno tras otro, la música de fondo (Sneaky Snitch, Kevin MacLeod, licencia Creative Commons) se mezcla por encima con ducking de audio: una compresión sidechain que baja automáticamente el volumen de la música mientras habla un agente, y que sube durante los silencios. Todo corre en 60 fps de principio a fin, sin conversión de framerate entre etapas.

7. Publicación automática

El script run-cron.sh, lanzado por un cron clásico, activa el entorno Python, carga el .env, y lanza bun src/workflow.ts --upload. El flag --upload dispara además la generación de metadatos (título, descripción, tags) y llama a uploaders/upload.py, que publica el vídeo en YouTube e Instagram mediante dos scripts separados (uploaders/youtube/upload.py y uploaders/instagram/). Toda la cadena, del prompt LLM al vídeo en línea, corre sin intervención humana.

Por qué TypeScript/Bun en vez de algo todo Python

La elección no es ideológica: Bun da acceso directo y rápido a Bun.spawn para pilotar FFmpeg en subprocesos, un tipado fuerte sobre las estructuras de datos del pipeline (Phrase, SegmentInfo), y un runtime mucho más rápido al arrancar que Node para un script que corre en cron cada X horas. Los dos únicos trozos de Python del proyecto están donde Python está realmente mejor equipado: PIL para la extracción de colores, y las APIs de subida (google-api-python-client para YouTube, la stack de Instagram Graph API para IG).

Lo que ilustra

Este proyecto es un buen ejemplo de lo que se puede construir hoy con piezas totalmente gratuitas u open source: un LLM rápido y gratuito vía la API de Groq, un motor TTS local que corre sin GPU dedicada, FFmpeg para todo el render de vídeo, y el pegamento no son más que unos cientos de líneas de TypeScript. Ninguna de esas piezas es nueva por separado. Lo que hace el pipeline es el ensamblaje: generar un guion coherente con verdaderas relaciones entre personajes, transformarlo en audio expresivo con pausas naturales, sincronizar un render visual con la energía de ese audio fotograma a fotograma, y automatizar toda la cadena hasta la publicación.

---

Recursos

• Repo: github.com/fox3000foxy/valorant-short-maker
• Canal: @valorant_agents

3 puntos clave

1. El guion lo genera un LLM (Groq/Llama 3.3) con personas y relaciones por agente, no una simple lista de chistes preescritos.
2. El zoom del círculo del agente lo pilota una expresión FFmpeg calculada fotograma a fotograma a partir de la envolvente RMS del WAV: nada de animación clásica por keyframes.
3. Toda la cadena, del prompt al post de YouTube/Instagram, corre con un solo cron job sin intervención humana.

valorant-short-maker : o pipeline que gera meus shorts de Valorant sozinho

Há alguns meses, um canal do YouTube roda sem eu tocar nele : @valorant_agents. Agentes de Valorant se provocando entre dois rounds, dublados, legendados em karaokê, publicados como Shorts. Tudo é gerado pelo valorant-short-maker, um pipeline TypeScript/Bun que roda em cron e publica sem ninguém precisar clicar em nada.

Veja como funciona, passo a passo.

O resultado

Três frames extraídos do vídeo gerado para "Duelist Debate" (Phoenix, Yoru e Jett) :

O resultado ao vivo neste Short : Duelist Debate -- youtube.com/shorts/SX5Kme58aLU. No canal, os Shorts giram em torno de 1,2 a 1,5k visualizações. Nada enorme, mas é um canal que roda sozinho desde o início, então o número que realmente conta é zero, zero minuto gasto nele depois que o cron foi lançado.

O pipeline, em ordem

1. Escrever o roteiro -- Groq + Llama 3.3

Cada run sorteia 3 a 4 agentes aleatórios entre os 26 disponíveis, e envia ao Llama 3.3 70B (via Groq) um prompt de sistema que contém, para cada agente escolhido, um resumo compacto da personalidade dele e das relações dele com os outros agentes presentes na cena (essas personas vivem em src/lore/, um arquivo por agente). O prompt impõe regras precisas : uma frase curta e impactante por fala, rotação justa entre os personagens, humor em prioridade, e principalmente pausas.

Exemplo concreto com "Duelist Debate", Phoenix, Yoru e Jett discutindo quem vai jogar de duelist, gerado em 6 de julho de 2026 :

phoenix: I'm telling you, I've got the skills to play duelist this match.
yoru: Skills, you call burning things skills, Phoenix.
jett: I'm the fastest one here, I should play duelist.
phoenix: Fastest, but can you handle the heat, Jett [0.3] I doubt it.
yoru: Heat, ha, you think your flames are hotter than my rifts.
jett: This isn't about heat or flames, it's about speed and agility.
phoenix: Oh, I see, so now you're an expert on duelists, Yoru [0.3] that's rich.
yoru: At least I don't rely on cheap fire tricks.
jett: Cheap fire tricks, that's what you call Phoenix's abilities.
phoenix: Hey, my fire tricks have gotten us out of tight spots before [0.3] can't say the same for your rifts, Yoru.
yoru: Tight spots, you mean like the time I rifted us out of that trap.
jett: Enough, this is getting nowhere, let's just decide already.
phoenix: Fine, but I'm still saying I'm the best duelist here.
yoru: Please, you think you can take on the enemy team alone [0.3] I doubt it.
jett: I can take them on, no problem, I'm the fastest.
phoenix: Fastest, yeah, but can you outmaneuver them [0.3] that's the question.
yoru: Outmaneuver, ha, you think you can outmaneuver anyone, Phoenix.
jett: This is stupid, we're not going to agree on this.
phoenix: Fine, let's just play and see who comes out on top [0.3] I'm game if you are.
yoru: Bring it on, I'll show you what a real duelist looks like.
jett: I'm not backing down, I'm playing duelist.
phoenix: Oh, this should be good [0.3] let's see how you two do.
yoru: We'll see who comes out on top, won't we, Jett.
jett: Yeah, let's end this debate once and for all.
pause: 0.3
phoenix: Alright, let's get started then [0.3] may the best duelist win.
yoru: I'll make sure to burn you, Phoenix, not with fire, but with my rifts.
jett: I'll take you both down, no problem.

As pausas são o detalhe que deixa o ritmo natural : [0.3] inserido no meio de uma fala cria um silêncio de 0.3s no arquivo de áudio sem cortar o círculo do agente na tela, enquanto uma linha pause: 1.0 inteira cria um silêncio real entre dois locutores, círculo escondido. Sem isso, um TTS que encadeia falas sem respirar soa robótico.

2. Dar uma voz -- Piper, um modelo por agente

Cada agente tem seu próprio modelo Piper (.onnx) treinado especificamente, guardado em voices/<agent>/. O texto gerado passa pelo modelo correspondente, o que produz um WAV. É a mesma tecnologia que uso para o treinamento de vozes custom em geral (ver o artigo sobre o pipeline Piper/Kaggle), aqui aplicada direto em prod, em tempo real, a cada geração de vídeo.

3. Legendas karaokê -- ASS gerado, cor extraída do ícone

A legendagem não é um simples .srt. É um arquivo .ass (Advanced SubStation Alpha) gerado palavra por palavra, com efeito karaokê : cada palavra se ilumina em uma cor à medida que é pronunciada, enquanto o resto do texto fica em uma cor neutra. A cor de destaque não é fixa, ela é extraída dinamicamente do ícone do agente que fala (um script Python roda PIL sobre o PNG do ícone, amostra os pixels não transparentes, e devolve as cores dominantes). Resultado : a legenda da Killjoy se ilumina em roxo, a da Jett em azul esverdeado, sem nenhuma cor ter sido fixada no código em lugar algum.

4. O círculo audiorreativo -- uma expressão FFmpeg por frame

É a parte mais estranha do pipeline, e provavelmente a de que mais me orgulho. O ícone redondo do agente que fala não fica estático : ele dá zoom in e out suave no ritmo da própria voz dele.

O cálculo é feito lendo o WAV bruto da fala, calculando o envelope RMS (root mean square, uma medida da energia do sinal) frame por frame a 60 fps, normalizando pelo máximo, e depois suavizando em uma janela de 3 frames para evitar solavancos. Cada valor de envelope é então convertido em um fator de escala limitado por MAX_ZOOM_VARIATION (0.2, ou seja ±20% em torno do tamanho base).

O resultado desse cálculo não é aplicado via código que manipula pixels, é traduzido em uma imensa expressão condicional FFmpeg (lt(n,K)*val + between(n,K,K')*val + ..., uma ramificação por grupo de frames) que controla diretamente o parâmetro scale do filtro de vídeo. O FFmpeg avalia essa expressão a cada frame da renderização. Para uma fala de alguns segundos a 60 fps, isso dá rápido centenas de ramificações em uma única expressão, daí o parâmetro STEP que agrupa os frames para limitar a profundidade.

5. Render por segmento, depois fisheye na intro

Cada fala é renderizada individualmente : vídeo de fundo (trecho aleatório de um dos clipes de gameplay em bg-video/, cortado na duração certa), círculo do agente por cima com o zoom audiorreativo, legendas embutidas via filtro ass do FFmpeg, áudio TTS mixado com o som do gameplay de fundo.

O primeiríssimo segmento recebe um tratamento especial : uma distorção fisheye que se dissipa progressivamente nos primeiros 20% dos frames (filtro lenscorrection avaliado frame por frame, mais um tmix=frames=3 que mistura frames adjacentes para simular motion blur), sincronizada com um som de "whoosh". É a transição de intro que dá a impressão de que a câmera "entra" na cena.

6. Concatenação e mixagem final

Todos os segmentos são concatenados um após o outro, a música de fundo (Sneaky Snitch, Kevin MacLeod, licença Creative Commons) é mixada por cima com ducking de áudio, uma compressão sidechain que baixa automaticamente o volume da música enquanto um agente fala, e que sobe de novo nos silêncios. Tudo roda em 60 fps do início ao fim, sem nenhuma conversão de framerate entre as etapas.

7. Publicação automática

O script run-cron.sh, lançado por um cron clássico, ativa o ambiente Python, carrega o .env, e lança bun src/workflow.ts --upload. A flag --upload dispara ainda a geração de metadados (título, descrição, tags) e chama uploaders/upload.py, que publica o vídeo no YouTube e no Instagram via dois scripts separados (uploaders/youtube/upload.py e uploaders/instagram/). Toda a cadeia, do prompt do LLM ao vídeo online, roda sem intervenção humana.

Por que TypeScript/Bun em vez de algo todo em Python

A escolha não é ideológica, é que o Bun dá acesso direto e rápido ao Bun.spawn para controlar o FFmpeg em subprocesso, uma tipagem forte sobre as estruturas de dados do pipeline (Phrase, SegmentInfo), e um runtime bem mais rápido na inicialização do que o Node para um script que roda em cron a cada X horas. Os dois únicos trechos de Python no projeto estão onde Python é realmente a melhor ferramenta : PIL para extração de cores, e as APIs de upload (google-api-python-client para o YouTube, a stack da Instagram Graph API para o IG).

O que isso ilustra

Este projeto é um bom exemplo do que dá para construir hoje com peças inteiramente gratuitas ou open source : um LLM rápido e gratuito via API Groq, um motor TTS local que roda sem GPU dedicada, FFmpeg para toda a renderização de vídeo, e a cola, que são só algumas centenas de linhas de TypeScript. Nenhuma dessas peças é nova individualmente. O que faz o pipeline é o arranjo : gerar um roteiro coerente com relações reais entre personagens, transformá-lo em áudio expressivo com pausas naturais, sincronizar uma renderização visual sobre a energia desse áudio frame por frame, e automatizar toda a cadeia até a publicação.

---

Recursos

• Repo : github.com/fox3000foxy/valorant-short-maker
• Canal : @valorant_agents

3 pontos principais

1. O roteiro é gerado por um LLM (Groq/Llama 3.3) com personas e relações por agente, não uma simples lista de piadas pré-escritas.
2. O zoom do círculo do agente é controlado por uma expressão FFmpeg calculada frame por frame a partir do envelope RMS do WAV, sem animação clássica por keyframes.
3. Toda a cadeia, do prompt ao post no YouTube/Instagram, roda via um único cron job sem intervenção humana.

valorant-short-maker : pipeline yang membangkitkan short Valorant saya sendiri

Sudah beberapa bulan, kanal YouTube berjalan tanpa saya sentuh : @valorant_agents. Agen Valorant yang saling mengejek antara dua round, di dubbing, diberi subtitle karaoke, diterbitkan sebagai Shorts. Semuanya dibangkitkan oleh valorant-short-maker, pipeline TypeScript/Bun yang berjalan dalam cron dan menerbitkan tanpa siapa pun harus mengklik apa pun.

Begini cara kerjanya, tahap demi tahap.

Seperti apa hasilnya

Tiga frame yang diekstrak dari video yang dibangkitkan untuk "Duelist Debate" (Phoenix, Yoru dan Jett) :

Hasilnya live di Short ini : Duelist Debate -- youtube.com/shorts/SX5Kme58aLU. Di kanal, Shorts berkisar sekitar 1,2 hingga 1,5 ribu view. Tidak besar, tapi ini kanal yang berjalan sendiri sejak awal, jadi angka yang benar-benar penting adalah nol -- nol menit dihabiskan di atasnya begitu cron berjalan.

Pipeline, berurutan

1. Menulis skrip -- Groq + Llama 3.3

Setiap run mengambil 3 hingga 4 agen acak dari 26 yang tersedia, dan mengirim ke Llama 3.3 70B (lewat Groq) prompt sistem yang berisi, untuk setiap agen yang dipilih, ringkasan ringkas kepribadiannya dan relasinya dengan agen lain yang ada di adegan (persona ini hidup di src/lore/, satu berkas per agen). Prompt memaksakan aturan tepat : satu kalimat pendek dan menghantam per replika, rotasi yang adil antar karakter, humor sebagai prioritas, dan terutama jeda.

Contoh konkret dengan "Duelist Debate" -- Phoenix, Yoru dan Jett berdebat soal siapa yang bermain duelist, dibangkitkan 6 Juli 2026 :

phoenix: I'm telling you, I've got the skills to play duelist this match.
yoru: Skills, you call burning things skills, Phoenix.
jett: I'm the fastest one here, I should play duelist.
phoenix: Fastest, but can you handle the heat, Jett [0.3] I doubt it.
yoru: Heat, ha, you think your flames are hotter than my rifts.
jett: This isn't about heat or flames, it's about speed and agility.
phoenix: Oh, I see, so now you're an expert on duelists, Yoru [0.3] that's rich.
yoru: At least I don't rely on cheap fire tricks.
jett: Cheap fire tricks, that's what you call Phoenix's abilities.
phoenix: Hey, my fire tricks have gotten us out of tight spots before [0.3] can't say the same for your rifts, Yoru.
yoru: Tight spots, you mean like the time I rifted us out of that trap.
jett: Enough, this is getting nowhere, let's just decide already.
phoenix: Fine, but I'm still saying I'm the best duelist here.
yoru: Please, you think you can take on the enemy team alone [0.3] I doubt it.
jett: I can take them on, no problem, I'm the fastest.
phoenix: Fastest, yeah, but can you outmaneuver them [0.3] that's the question.
yoru: Outmaneuver, ha, you think you can outmaneuver anyone, Phoenix.
jett: This is stupid, we're not going to agree on this.
phoenix: Fine, let's just play and see who comes out on top [0.3] I'm game if you are.
yoru: Bring it on, I'll show you what a real duelist looks like.
jett: I'm not backing down, I'm playing duelist.
phoenix: Oh, this should be good [0.3] let's see how you two do.
yoru: We'll see who comes out on top, won't we, Jett.
jett: Yeah, let's end this debate once and for all.
pause: 0.3
phoenix: Alright, let's get started then [0.3] may the best duelist win.
yoru: I'll make sure to burn you, Phoenix, not with fire, but with my rifts.
jett: I'll take you both down, no problem.

Jeda adalah detail yang membuat ritmenya alami : [0.3] yang disisipkan di tengah replika menciptakan hening 0.3 detik dalam berkas audio tanpa memotong lingkaran agen di layar, sedangkan baris pause: 1.0 tersendiri menciptakan hening sungguhan antara dua pembicara, lingkaran disembunyikan. Tanpa itu, TTS yang merangkai replika tanpa bernapas terdengar robotik.

2. Memberi suara -- Piper, satu model per agen

Setiap agen punya model Piper sendiri (.onnx) yang dilatih khusus, disimpan di voices/<agent>/. Teks yang dibangkitkan melewati model yang cocok, yang mengeluarkan WAV. Ini teknologi yang sama yang saya pakai untuk training suara kustom secara umum (lihat artikel tentang pipeline Piper/Kaggle) -- di sini diterapkan langsung di prod, langsung, di setiap pembangkitan video.

3. Subtitle karaoke -- ASS yang dibangkitkan, warna diekstrak dari ikon

Subtitlenya bukan sekadar .srt. Ini berkas .ass (Advanced SubStation Alpha) yang dibangkitkan kata per kata, dengan efek karaoke : setiap kata menyala dalam satu warna seiring diucapkan, sementara sisa teks tetap dalam warna netral. Warna aksennya tidak tetap -- ia diekstrak dinamis dari ikon agen yang bicara (skrip Python menjalankan PIL pada PNG ikon, mencuplik piksel non-transparan, dan mengembalikan warna dominan). Hasilnya : subtitle Killjoy menyala ungu, milik Jett biru-hijau, tanpa warna apa pun yang di hardcode di mana pun.

4. Lingkaran audio-reaktif -- satu ekspresi FFmpeg per frame

Ini bagian paling bengkok dari pipeline, dan mungkin yang paling saya banggakan. Ikon bundar agen yang bicara tidak tetap statis : ia zoom in dan zoom out sedikit mengikuti ritme suaranya sendiri.

Perhitungannya dilakukan dengan membaca WAV mentah replika, menghitung amplop RMS (root mean square, ukuran energi sinyal) frame per frame pada 60 fps, menormalisasi dengan maksimum, lalu menghaluskan pada jendela 3 frame untuk menghindari sentakan. Setiap nilai amplop lalu diubah menjadi faktor skala yang dibatasi oleh MAX_ZOOM_VARIATION (0.2, jadi ±20% sekitar ukuran dasar).

Hasil perhitungan ini tidak diterapkan lewat kode yang memanipulasi piksel -- ia diterjemahkan menjadi ekspresi kondisional FFmpeg yang raksasa (lt(n,K)*val + between(n,K,K')*val + ..., satu cabang per grup frame) yang langsung mengendalikan parameter scale filter video. FFmpeg mengevaluasi ekspresi ini di setiap frame render. Untuk replika beberapa detik pada 60 fps, itu cepat menjadi ratusan cabang dalam satu ekspresi -- makanya parameter STEP mengelompokkan frame untuk membatasi kedalaman.

5. Render per segmen, lalu fisheye pada intro

Setiap replika di render individual : video latar (potongan acak dari salah satu klip gameplay di bg-video/, dipotong pada durasi yang benar), lingkaran agen di atasnya dengan zoom audio-reaktif, subtitle ditempel lewat filter ass FFmpeg, audio TTS di mix dengan suara gameplay di latar.

Segmen pertama menerima perlakuan khusus : distorsi fisheye yang mereda berangsur pada 20% frame pertama (filter lenscorrection dievaluasi frame per frame, ditambah tmix=frames=3 yang mencampur frame bersebelahan untuk mensimulasikan motion blur), disinkronkan dengan suara "whoosh". Inilah transisi intro yang memberi kesan kamera "masuk" ke adegan.

6. Konkatenasi dan mixing akhir

Semua segmen dikonkatenasi ujung ke ujung, musik latar (Sneaky Snitch, Kevin MacLeod, lisensi Creative Commons) di mix di atasnya dengan ducking audio -- kompresi sidechain yang otomatis menurunkan volume musik sementara agen bicara, dan menaikkannya selama hening. Semuanya berjalan di 60 fps dari ujung ke ujung, tanpa konversi framerate antar tahap.

7. Publikasi otomatis

Skrip run-cron.sh, yang dijalankan cron klasik, mengaktifkan environment Python, memuat .env, dan menjalankan bun src/workflow.ts --upload. Flag --upload memicu tambahan pembangkitan metadata (judul, deskripsi, tag) dan memanggil uploaders/upload.py, yang menerbitkan video ke YouTube dan Instagram lewat dua skrip terpisah (uploaders/youtube/upload.py dan uploaders/instagram/). Seluruh rantai, dari prompt LLM hingga video online, berjalan tanpa intervensi manusia.

Kenapa TypeScript/Bun alih-alih sesuatu yang serba Python

Pilihannya bukan ideologis -- melainkan Bun memberi akses langsung dan cepat ke Bun.spawn untuk mengendalikan FFmpeg dalam subproses, pengetikan kuat pada struktur data pipeline (Phrase, SegmentInfo), dan runtime yang jauh lebih cepat saat startup daripada Node untuk skrip yang berjalan dalam cron setiap X jam. Dua satu-satunya potongan Python dalam proyek ada di mana Python benar-benar paling teralati : PIL untuk ekstraksi warna, dan API unggah (google-api-python-client untuk YouTube, stack Instagram Graph API untuk IG).

Apa yang digambarkannya

Proyek ini contoh bagus dari apa yang bisa dibangun hari ini dengan bata yang sepenuhnya gratis atau open source : LLM cepat dan gratis lewat API Groq, mesin TTS lokal yang berjalan tanpa GPU khusus, FFmpeg untuk seluruh render video -- dan perekatnya hanyalah beberapa ratus baris TypeScript. Tidak satu pun bata ini baru secara individual. Yang membuat pipeline adalah rangkaiannya : membangkitkan skrip koheren dengan relasi nyata antar karakter, mengubahnya menjadi audio ekspresif dengan jeda alami, menyinkronkan render visual pada energi audio ini frame per frame, dan mengotomatisasi seluruh rantai hingga publikasi.

---

Sumber daya

• Repo : github.com/fox3000foxy/valorant-short-maker
• Kanal : @valorant_agents

3 poin kunci

1. Skrip dibangkitkan oleh LLM (Groq/Llama 3.3) dengan persona dan relasi per agen, bukan sekadar daftar lelucon yang sudah ditulis.
2. Zoom lingkaran agen dikendalikan ekspresi FFmpeg yang dihitung frame per frame dari amplop RMS WAV -- bukan animasi keyframe klasik.
3. Seluruh rantai, dari prompt hingga post YouTube/Instagram, berjalan lewat satu cron job tanpa intervensi manusia.

valorant-short-maker : वह पाइपलाइन जो मेरे Valorant शॉर्ट्स खुद बनाती है

कुछ महीनों से, एक YouTube चैनल चल रहा है बिना मैंने उसे छुए : @valorant_agents। Valorant एजेंट जो दो राउंड के बीच एक दूसरे को छेड़ते हैं, डब किए हुए, कराओके उपशीर्षकों के साथ, Shorts में प्रकाशित। सब कुछ valorant-short-maker द्वारा उत्पन्न है, एक TypeScript/Bun पाइपलाइन जो cron में चलती है और बिना किसी को कुछ क्लिक किए प्रकाशित करती है।

यह कैसे काम करता है, चरण दर चरण।

परिणाम कैसा दिखता है

"Duelist Debate" (Phoenix, Yoru और Jett) के लिए उत्पन्न वीडियो से निकाले गए तीन फ्रेम :

इस Short पर लाइव परिणाम : Duelist Debate -- youtube.com/shorts/SX5Kme58aLU। चैनल पर, Shorts लगभग 1,2 से 1,5k व्यूज पर घूमते हैं। कुछ बड़ा नहीं, लेकिन यह एक ऐसा चैनल है जो शुरू से खुद चलता है, इसलिए जो संख्या सच में मायने रखती है वह शून्य है, cron शुरू होने के बाद उस पर बिताए गए शून्य मिनट।

पाइपलाइन, क्रम में

1. स्क्रिप्ट लिखना, Groq + Llama 3.3

हर रन 26 उपलब्ध एजेंटों में से बेतरतीब 3 से 4 एजेंट चुनता है, और Llama 3.3 70B (Groq के माध्यम से) को एक सिस्टम प्रॉम्प्ट भेजता है जिसमें, चुने गए हर एजेंट के लिए, उसके व्यक्तित्व और दृश्य में मौजूद अन्य एजेंटों के साथ उसके संबंधों का संक्षिप्त सार होता है (ये personas src/lore/ में रहते हैं, प्रति एजेंट एक फाइल)। प्रॉम्प्ट सटीक नियम थोपता है : प्रति संवाद एक छोटी और दमदार पंक्ति, पात्रों के बीच निष्पक्ष बारी, हास्य प्राथमिकता में, और सबसे बढ़कर ठहराव।

"Duelist Debate" के साथ ठोस उदाहरण, Phoenix, Yoru और Jett बहस करते हैं कि duelist कौन खेलेगा, 6 जुलाई 2026 को उत्पन्न :

phoenix: I'm telling you, I've got the skills to play duelist this match.
yoru: Skills, you call burning things skills, Phoenix.
jett: I'm the fastest one here, I should play duelist.
phoenix: Fastest, but can you handle the heat, Jett [0.3] I doubt it.
yoru: Heat, ha, you think your flames are hotter than my rifts.
jett: This isn't about heat or flames, it's about speed and agility.
phoenix: Oh, I see, so now you're an expert on duelists, Yoru [0.3] that's rich.
yoru: At least I don't rely on cheap fire tricks.
jett: Cheap fire tricks, that's what you call Phoenix's abilities.
phoenix: Hey, my fire tricks have gotten us out of tight spots before [0.3] can't say the same for your rifts, Yoru.
yoru: Tight spots, you mean like the time I rifted us out of that trap.
jett: Enough, this is getting nowhere, let's just decide already.
phoenix: Fine, but I'm still saying I'm the best duelist here.
yoru: Please, you think you can take on the enemy team alone [0.3] I doubt it.
jett: I can take them on, no problem, I'm the fastest.
phoenix: Fastest, yeah, but can you outmaneuver them [0.3] that's the question.
yoru: Outmaneuver, ha, you think you can outmaneuver anyone, Phoenix.
jett: This is stupid, we're not going to agree on this.
phoenix: Fine, let's just play and see who comes out on top [0.3] I'm game if you are.
yoru: Bring it on, I'll show you what a real duelist looks like.
jett: I'm not backing down, I'm playing duelist.
phoenix: Oh, this should be good [0.3] let's see how you two do.
yoru: We'll see who comes out on top, won't we, Jett.
jett: Yeah, let's end this debate once and for all.
pause: 0.3
phoenix: Alright, let's get started then [0.3] may the best duelist win.
yoru: I'll make sure to burn you, Phoenix, not with fire, but with my rifts.
jett: I'll take you both down, no problem.

ठहराव वह विवरण है जो लय को स्वाभाविक बनाता है : किसी संवाद के बीच में डाला गया [0.3] स्क्रीन पर एजेंट के गोले को काटे बिना ऑडियो फाइल में 0.3s का मौन बनाता है, जबकि अपने आप में एक पूरी pause: 1.0 पंक्ति दो वक्ताओं के बीच एक असली मौन बनाती है, गोला छिपा हुआ। इसके बिना, एक TTS जो बिना साँस लिए संवादों को जोड़ता है, रोबोटिक लगता है।

2. आवाज देना, Piper, प्रति एजेंट एक मॉडल

हर एजेंट का अपना Piper मॉडल (.onnx) है जो विशेष रूप से प्रशिक्षित है, voices/<agent>/ में संग्रहित। उत्पन्न पाठ संबंधित मॉडल से गुजरता है, जिससे एक WAV निकलता है। यह वही तकनीक है जिसका मैं सामान्य रूप से कस्टम आवाज प्रशिक्षण के लिए उपयोग करता हूँ (Piper/Kaggle पाइपलाइन पर लेख देखें), यहाँ सीधे prod में लागू, हर वीडियो पीढ़ी पर तुरंत।

3. कराओके उपशीर्षक, उत्पन्न ASS, आइकन से निकाला गया रंग

उपशीर्षक कोई साधारण .srt नहीं है। यह शब्द दर शब्द उत्पन्न .ass फाइल (Advanced SubStation Alpha) है, कराओके प्रभाव के साथ : हर शब्द एक रंग में जगमगाता है जैसे ही वह बोला जाता है, जबकि बाकी पाठ तटस्थ रंग में रहता है। एक्सेंट रंग स्थिर नहीं है, यह बोलने वाले एजेंट के आइकन से गतिशील रूप से निकाला जाता है (एक Python स्क्रिप्ट आइकन के PNG पर PIL चलाती है, गैर पारदर्शी पिक्सेल का नमूना लेती है, और प्रमुख रंग लौटाती है)। परिणाम : Killjoy का उपशीर्षक बैंगनी में जगमगाता है, Jett का नीले हरे में, बिना कहीं कोई रंग हार्ड कोड किए।

4. ऑडियो प्रतिक्रियाशील गोला, प्रति फ्रेम एक FFmpeg अभिव्यक्ति

यह पाइपलाइन का सबसे टेढ़ा हिस्सा है, और शायद वह जिस पर मुझे सबसे ज्यादा गर्व है। बोलने वाले एजेंट का गोल आइकन स्थिर नहीं रहता : यह अपनी खुद की आवाज की लय पर हल्का जूम इन और आउट करता है।

गणना संवाद के कच्चे WAV को पढ़कर, 60 fps पर फ्रेम दर फ्रेम RMS एन्वेलप (root mean square, सिग्नल ऊर्जा का माप) की गणना करके, अधिकतम से सामान्य करके, फिर झटकों से बचने के लिए 3 फ्रेम की विंडो पर चिकना करके की जाती है। एन्वेलप के हर मान को फिर MAX_ZOOM_VARIATION (0.2, इसलिए आधार आकार के आसपास ±20%) से सीमित स्केल कारक में बदला जाता है।

इस गणना का परिणाम पिक्सेल में हेरफेर करने वाले कोड के माध्यम से लागू नहीं होता, यह एक विशाल सशर्त FFmpeg अभिव्यक्ति (lt(n,K)*val + between(n,K,K')*val + ..., फ्रेम समूह प्रति एक शाखा) में अनुवादित होता है जो सीधे वीडियो फ़िल्टर के scale पैरामीटर को चलाता है। FFmpeg रेंडर के हर फ्रेम पर इस अभिव्यक्ति का मूल्यांकन करता है। 60 fps पर कुछ सेकंड के संवाद के लिए, यह एक ही अभिव्यक्ति में जल्दी सैकड़ों शाखाएँ बना देता है, इसीलिए STEP पैरामीटर फ्रेमों को समूहित करता है ताकि गहराई सीमित रहे।

5. खंड दर खंड रेंडर, फिर इंट्रो पर fisheye

हर संवाद अलग से रेंडर किया जाता है : वीडियो पृष्ठभूमि (bg-video/ में गेमप्ले क्लिप में से एक का बेतरतीब अंश, सही अवधि में काटा गया), ऊपर ऑडियो प्रतिक्रियाशील गोले वाला एजेंट, FFmpeg के ass फ़िल्टर के माध्यम से जड़े गए उपशीर्षक, पृष्ठभूमि में गेमप्ले ध्वनि के साथ मिश्रित TTS ऑडियो।

सबसे पहला खंड विशेष उपचार पाता है : एक fisheye विकृति जो पहले 20% फ्रेमों पर धीरे धीरे ठीक होती है (फ्रेम दर फ्रेम मूल्यांकित lenscorrection फ़िल्टर, साथ में एक tmix=frames=3 जो मोशन ब्लर का अनुकरण करने के लिए आसन्न फ्रेमों को मिलाता है), "whoosh" ध्वनि के साथ समकालिक। यह इंट्रो संक्रमण है जो यह आभास देता है कि कैमरा दृश्य में "प्रवेश" करता है।

6. संयोजन और अंतिम मिश्रण

सभी खंड एक के बाद एक जोड़े जाते हैं, पृष्ठभूमि संगीत (Sneaky Snitch, Kevin MacLeod, Creative Commons लाइसेंस) ऑडियो डकिंग के साथ ऊपर मिश्रित किया जाता है, एक sidechain संपीड़न जो जब कोई एजेंट बोलता है तो संगीत की आवाज अपने आप कम कर देता है, और मौन के दौरान बढ़ा देता है। सब कुछ शुरू से अंत तक 60 fps में चलता है, चरणों के बीच कोई फ्रेमरेट रूपांतरण नहीं।

7. स्वचालित प्रकाशन

run-cron.sh स्क्रिप्ट, एक क्लासिक cron द्वारा शुरू की गई, Python वातावरण सक्रिय करती है, .env लोड करती है, और bun src/workflow.ts --upload चलाती है। --upload ध्वज मेटाडेटा (शीर्षक, विवरण, टैग) की पीढ़ी को भी ट्रिगर करता है और uploaders/upload.py को कॉल करता है, जो दो अलग स्क्रिप्टों (uploaders/youtube/upload.py और uploaders/instagram/) के माध्यम से वीडियो YouTube और Instagram पर प्रकाशित करता है। पूरी शृंखला, LLM प्रॉम्प्ट से ऑनलाइन वीडियो तक, बिना मानवीय हस्तक्षेप चलती है।

सब कुछ Python वाली चीज के बजाय TypeScript/Bun क्यों

चुनाव वैचारिक नहीं है, यह है कि Bun Bun.spawn के माध्यम से उप प्रक्रिया में FFmpeg चलाने के लिए सीधी और तेज पहुँच देता है, पाइपलाइन डेटा संरचनाओं (Phrase, SegmentInfo) पर मजबूत टाइपिंग, और एक ऐसे स्क्रिप्ट के लिए Node से शुरू होने पर काफी तेज रनटाइम जो cron में हर X घंटे चलता है। प्रोजेक्ट में Python के सिर्फ दो टुकड़े वहीं हैं जहाँ Python सच में सबसे अच्छी तरह सुसज्जित है : रंग निकालने के लिए PIL, और अपलोड API (google-api-python-client YouTube के लिए, IG के लिए Instagram Graph API स्टैक)।

यह क्या दर्शाता है

यह प्रोजेक्ट इस बात का अच्छा उदाहरण है कि आज पूरी तरह मुफ्त या ओपन सोर्स ईंटों से क्या बनाया जा सकता है : Groq API के माध्यम से तेज और मुफ्त LLM, एक स्थानीय TTS इंजन जो समर्पित GPU के बिना चलता है, पूरे वीडियो रेंडर के लिए FFmpeg, और जोड़ने वाला बस TypeScript की कुछ सौ पंक्तियाँ। इनमें से कोई भी ईंट अलग से नई नहीं है। जो पाइपलाइन बनाता है वह संयोजन है : पात्रों के बीच असली संबंधों के साथ सुसंगत स्क्रिप्ट उत्पन्न करना, उसे स्वाभाविक ठहराव के साथ अभिव्यंजक ऑडियो में बदलना, इस ऑडियो की ऊर्जा पर फ्रेम दर फ्रेम एक दृश्य रेंडर समकालिक करना, और प्रकाशन तक पूरी शृंखला स्वचालित करना।

---

संसाधन

• Repo : github.com/fox3000foxy/valorant-short-maker
• चैनल : @valorant_agents

3 मुख्य बिंदु

1. स्क्रिप्ट एक LLM (Groq/Llama 3.3) द्वारा प्रति एजेंट personas और संबंधों के साथ उत्पन्न होती है, पहले से लिखे चुटकुलों की साधारण सूची नहीं।
2. एजेंट गोले का जूम WAV के RMS एन्वेलप से फ्रेम दर फ्रेम गणना की गई FFmpeg अभिव्यक्ति द्वारा चलाया जाता है, क्लासिक keyframes एनीमेशन नहीं।
3. पूरी शृंखला, प्रॉम्प्ट से YouTube/Instagram पोस्ट तक, बिना मानवीय हस्तक्षेप एक ही cron job से चलती है।

valorant-short-maker : خط الأنابيب الذي يولّد Shorts Valorant الخاصة بي وحده

منذ أشهر، تعمل قناة YouTube دون أن ألمسها : @valorant_agents. وكلاء Valorant يتشاجرون بين جولتين، مدبلجون، ومترجمون بترجمة كاريوكي، ومنشورون كـShorts. كل شيء مولّد بواسطة valorant-short-maker، وهو خط أنابيب TypeScript/Bun يعمل في cron وينشر دون أن يضغط أحد على أي شيء.

إليكم كيف يعمل، خطوة بخطوة.

ما النتيجة

ثلاث لقطات مستخرجة من الفيديو المولّد لـ"Duelist Debate" (Phoenix وYoru وJett) :

النتيجة مباشرة على هذا Short : Duelist Debate -- youtube.com/shorts/SX5Kme58aLU. على القناة، تدور الـShorts حول 1,2 إلى 1,5k مشاهدة. لا شيء ضخم، لكنها قناة تعمل وحدها منذ البداية، لذا الرقم الذي يهم فعلا هو صفر، صفر دقيقة تُقضى عليها بمجرد إطلاق الـcron.

خط الأنابيب، بالترتيب

1. كتابة السكربت -- Groq + Llama 3.3

كل تشغيل يسحب 3 إلى 4 وكلاء عشوائيا من 26 متاحا، ويرسل إلى Llama 3.3 70B (عبر Groq) prompt نظام يحوي، لكل وكيل مختار، ملخصا مضغوطا لشخصيته وعلاقاته مع الوكلاء الآخرين الحاضرين في المشهد (هذه الـpersonas تعيش في src/lore/، ملف لكل وكيل). يفرض الـprompt قواعد دقيقة : جملة قصيرة ومؤثرة لكل رد، وتناوب عادل بين الشخصيات، والفكاهة أولا، وقبل كل شيء فواصل.

مثال ملموس مع "Duelist Debate"، يتجادل Phoenix وYoru وJett لمعرفة من سيلعب duelist، مُولَّد في 6 يوليو 2026 :

phoenix: I'm telling you, I've got the skills to play duelist this match.
yoru: Skills, you call burning things skills, Phoenix.
jett: I'm the fastest one here, I should play duelist.
phoenix: Fastest, but can you handle the heat, Jett [0.3] I doubt it.
yoru: Heat, ha, you think your flames are hotter than my rifts.
jett: This isn't about heat or flames, it's about speed and agility.
phoenix: Oh, I see, so now you're an expert on duelists, Yoru [0.3] that's rich.
yoru: At least I don't rely on cheap fire tricks.
jett: Cheap fire tricks, that's what you call Phoenix's abilities.
phoenix: Hey, my fire tricks have gotten us out of tight spots before [0.3] can't say the same for your rifts, Yoru.
yoru: Tight spots, you mean like the time I rifted us out of that trap.
jett: Enough, this is getting nowhere, let's just decide already.
phoenix: Fine, but I'm still saying I'm the best duelist here.
yoru: Please, you think you can take on the enemy team alone [0.3] I doubt it.
jett: I can take them on, no problem, I'm the fastest.
phoenix: Fastest, yeah, but can you outmaneuver them [0.3] that's the question.
yoru: Outmaneuver, ha, you think you can outmaneuver anyone, Phoenix.
jett: This is stupid, we're not going to agree on this.
phoenix: Fine, let's just play and see who comes out on top [0.3] I'm game if you are.
yoru: Bring it on, I'll show you what a real duelist looks like.
jett: I'm not backing down, I'm playing duelist.
phoenix: Oh, this should be good [0.3] let's see how you two do.
yoru: We'll see who comes out on top, won't we, Jett.
jett: Yeah, let's end this debate once and for all.
pause: 0.3
phoenix: Alright, let's get started then [0.3] may the best duelist win.
yoru: I'll make sure to burn you, Phoenix, not with fire, but with my rifts.
jett: I'll take you both down, no problem.

الفواصل هي التفصيل الذي يجعل الإيقاع طبيعيا : [0.3] المُدخَل في وسط رد يخلق صمت 0.3s في الملف الصوتي دون قطع دائرة الوكيل على الشاشة، بينما سطر pause: 1.0 كامل يخلق صمتا حقيقيا بين متكلمين، والدائرة مخفية. دون ذلك، TTS الذي يسلسل الردود دون تنفس يبدو آليا.

2. إعطاء صوت -- Piper، نموذج لكل وكيل

لكل وكيل نموذج Piper الخاص به (.onnx) مُدرَّب خصيصا، ومخزّن في voices/<agent>/. النص المولّد يمر في النموذج الموافق، فيخرج WAV. إنها نفس التقنية التي أستخدمها لتدريب الأصوات المخصصة عموما (انظر المقال عن خط أنابيب Piper/Kaggle)، هنا مطبقة مباشرة في الإنتاج، لحظيا، مع كل توليد فيديو.

3. ترجمة كاريوكي -- ASS مولّد، لون مستخرج من الأيقونة

الترجمة ليست مجرد .srt. إنها ملف .ass (Advanced SubStation Alpha) مولّد كلمة بكلمة، مع أثر كاريوكي : كل كلمة تضيء بلون كلما نُطقت، بينما يبقى بقية النص بلون محايد. لون التمييز ليس ثابتا، بل يُستخرج ديناميكيا من أيقونة الوكيل المتكلم (سكربت Python يشغّل PIL على PNG الأيقونة، ويعيّن البكسلات غير الشفافة، ويعيد الألوان السائدة). النتيجة : ترجمة Killjoy تضيء بالبنفسجي، وترجمة Jett بالأزرق المخضر، دون أن يُرمَّز أي لون بشكل صلب في أي مكان.

4. الدائرة المتفاعلة مع الصوت -- تعبير FFmpeg لكل إطار

هذا هو الجزء الأكثر التواء في خط الأنابيب، وربما الذي أفتخر به أكثر. أيقونة الوكيل المتكلم الدائرية لا تبقى ثابتة : تكبّر وتصغّر قليلا على إيقاع صوته.

يُحسب ذلك بقراءة WAV الخام للرد، وحساب غلاف RMS (root mean square، مقياس طاقة الإشارة) إطارا بإطار بسرعة 60 fps، والتطبيع بالأقصى، ثم التنعيم على نافذة 3 إطارات لتجنب التقطعات. كل قيمة غلاف تُحوَّل بعد ذلك إلى عامل مقياس محدود بـMAX_ZOOM_VARIATION (0.2، أي ±20% حول الحجم الأساسي).

نتيجة هذا الحساب لا تُطبَّق عبر كود يتلاعب بالبكسلات، بل تُترجم إلى تعبير شرطي FFmpeg ضخم (lt(n,K)*val + between(n,K,K')*val + ...، فرع لكل مجموعة إطارات) يقود مباشرة معامل scale في مرشح الفيديو. يقيّم FFmpeg هذا التعبير مع كل إطار من العرض. لرد بضع ثوانٍ بسرعة 60 fps، يصنع هذا سريعا مئات الفروع في تعبير واحد، ومن هنا معامل STEP الذي يجمّع الإطارات للحد من العمق.

5. العرض لكل مقطع، ثم fisheye على المقدمة

كل رد يُعرض منفردا : خلفية فيديو (مقتطف عشوائي من أحد مقاطع اللعب في bg-video/، مقصوص للمدة الصحيحة)، ودائرة الوكيل فوقها مع زوم متفاعل صوتيا، وترجمة مطمورة عبر مرشح ass في FFmpeg، وصوت TTS ممزوج بصوت اللعب في الخلفية.

أول مقطع يحظى بمعالجة خاصة : تشويه fisheye يتلاشى تدريجيا على أول 20% من الإطارات (مرشح lenscorrection مُقيَّم إطارا بإطار، مع tmix=frames=3 الذي يمزج الإطارات المتجاورة لمحاكاة motion blur)، متزامن مع صوت "whoosh". إنه انتقال المقدمة الذي يعطي انطباع أن الكاميرا "تدخل" المشهد.

6. التسلسل والمزج النهائي

كل المقاطع تُسلسَل طرفا لطرف، وموسيقى الخلفية (Sneaky Snitch، Kevin MacLeod، رخصة Creative Commons) تُمزج فوقها مع ducking صوتي، ضغط sidechain يخفض تلقائيا صوت الموسيقى بينما يتكلم وكيل، ويرفعه أثناء الصمت. الكل يعمل بسرعة 60 fps من البداية للنهاية، دون تحويل framerate بين الخطوات.

7. النشر التلقائي

سكربت run-cron.sh، الذي يطلقه cron تقليدي، يفعّل بيئة Python، ويحمّل .env، ويشغّل bun src/workflow.ts --upload. علم --upload يطلق إضافة توليد البيانات الوصفية (عنوان ووصف ووسوم) ويستدعي uploaders/upload.py، الذي ينشر الفيديو على YouTube وInstagram عبر سكربتين منفصلين (uploaders/youtube/upload.py وuploaders/instagram/). السلسلة كاملة، من prompt الـLLM إلى الفيديو المنشور، تعمل دون تدخل بشري.

لماذا TypeScript/Bun بدلا من شيء Python بالكامل

الاختيار ليس إيديولوجيا، بل لأن Bun يعطي وصولا مباشرا وسريعا إلى Bun.spawn لقيادة FFmpeg في عملية فرعية، وتنميطا قويا على هياكل بيانات خط الأنابيب (Phrase وSegmentInfo)، وruntime أسرع بكثير عند الإقلاع من Node لسكربت يعمل في cron كل X ساعات. القطعتان الوحيدتان من Python في المشروع هما حيث Python مجهز فعلا الأفضل : PIL لاستخراج الألوان، وواجهات الرفع (google-api-python-client لـYouTube، وحزمة Instagram Graph API لـIG).

ما الذي يوضحه

هذا المشروع مثال جيد على ما يمكن بناؤه اليوم بلبنات مجانية كليا أو مفتوحة المصدر : LLM سريع ومجاني عبر Groq API، ومحرك TTS محلي يعمل دون GPU مخصص، وFFmpeg لكل عرض الفيديو، والرابط ليس إلا بضع مئات من أسطر TypeScript. لا شيء من هذه اللبنات جديد منفردا. ما يصنع خط الأنابيب هو الترتيب : توليد سكربت متسق مع علاقات حقيقية بين الشخصيات، وتحويله إلى صوت معبّر مع فواصل طبيعية، ومزامنة عرض بصري على طاقة هذا الصوت إطارا بإطار، وأتمتة السلسلة كاملة حتى النشر.

---

الموارد

• المستودع : github.com/fox3000foxy/valorant-short-maker
• القناة : @valorant_agents

3 نقاط أساسية

1. السكربت مولّد بواسطة LLM (Groq/Llama 3.3) مع personas وعلاقات لكل وكيل، لا مجرد قائمة نكات مكتوبة مسبقا.
2. زوم دائرة الوكيل يقوده تعبير FFmpeg محسوب إطارا بإطار من غلاف RMS للـWAV، لا تحريك keyframes تقليدي.
3. السلسلة كاملة، من الـprompt إلى منشور YouTube/Instagram، تعمل عبر cron job واحد دون تدخل بشري.

valorant-short-maker : pipeline tự động tạo shorts Valorant của tôi

Vài tháng nay, một kênh YouTube chạy mà tôi không hề đụng vào : @valorant_agents. Các đặc vụ Valorant trêu nhau giữa hai round, được lồng tiếng, phụ đề karaoke, đăng dưới dạng Shorts. Mọi thứ đều do valorant-short-maker tạo ra, một pipeline TypeScript/Bun chạy bằng cron và đăng bài mà không cần ai bấm gì cả.

Đây là cách nó hoạt động, từng bước một.

Thành phẩm trông thế nào

Ba frame trích từ video được tạo cho "Duelist Debate" (Phoenix, Yoru và Jett) :

Kết quả trực tiếp trên Short này : Duelist Debate -- youtube.com/shorts/SX5Kme58aLU. Trên kênh, các Short xoay quanh 1,2 đến 1,5k lượt xem. Không có gì to tát, nhưng đây là một kênh chạy hoàn toàn tự động từ đầu, nên con số thực sự đáng kể là số không -- không phút nào bỏ ra một khi cron đã chạy.

Pipeline, theo thứ tự

1. Viết kịch bản -- Groq + Llama 3.3

Mỗi lần chạy bốc ngẫu nhiên 3 đến 4 đặc vụ trong số 26 đặc vụ có sẵn, và gửi cho Llama 3.3 70B (qua Groq) một prompt hệ thống chứa, cho mỗi đặc vụ được chọn, một bản tóm tắt gọn về tính cách và mối quan hệ của họ với các đặc vụ khác có mặt trong cảnh (các persona này sống trong src/lore/, một tệp cho mỗi đặc vụ). Prompt áp đặt các quy tắc chính xác : một câu ngắn gọn đanh thép cho mỗi lời thoại, luân phiên công bằng giữa các nhân vật, ưu tiên hài hước, và nhất là các khoảng nghỉ.

Ví dụ cụ thể với "Duelist Debate" -- Phoenix, Yoru và Jett tranh nhau xem ai chơi duelist, được tạo ngày 6 tháng 7 năm 2026 :

phoenix: I'm telling you, I've got the skills to play duelist this match.
yoru: Skills, you call burning things skills, Phoenix.
jett: I'm the fastest one here, I should play duelist.
phoenix: Fastest, but can you handle the heat, Jett [0.3] I doubt it.
yoru: Heat, ha, you think your flames are hotter than my rifts.
jett: This isn't about heat or flames, it's about speed and agility.
phoenix: Oh, I see, so now you're an expert on duelists, Yoru [0.3] that's rich.
yoru: At least I don't rely on cheap fire tricks.
jett: Cheap fire tricks, that's what you call Phoenix's abilities.
phoenix: Hey, my fire tricks have gotten us out of tight spots before [0.3] can't say the same for your rifts, Yoru.
yoru: Tight spots, you mean like the time I rifted us out of that trap.
jett: Enough, this is getting nowhere, let's just decide already.
phoenix: Fine, but I'm still saying I'm the best duelist here.
yoru: Please, you think you can take on the enemy team alone [0.3] I doubt it.
jett: I can take them on, no problem, I'm the fastest.
phoenix: Fastest, yeah, but can you outmaneuver them [0.3] that's the question.
yoru: Outmaneuver, ha, you think you can outmaneuver anyone, Phoenix.
jett: This is stupid, we're not going to agree on this.
phoenix: Fine, let's just play and see who comes out on top [0.3] I'm game if you are.
yoru: Bring it on, I'll show you what a real duelist looks like.
jett: I'm not backing down, I'm playing duelist.
phoenix: Oh, this should be good [0.3] let's see how you two do.
yoru: We'll see who comes out on top, won't we, Jett.
jett: Yeah, let's end this debate once and for all.
pause: 0.3
phoenix: Alright, let's get started then [0.3] may the best duelist win.
yoru: I'll make sure to burn you, Phoenix, not with fire, but with my rifts.
jett: I'll take you both down, no problem.

Các khoảng nghỉ là chi tiết khiến nhịp tự nhiên : [0.3] chèn giữa một câu thoại tạo ra khoảng lặng 0.3s trong tệp âm thanh mà không cắt vòng tròn đặc vụ trên màn hình, trong khi một dòng pause: 1.0 riêng biệt tạo ra khoảng lặng thật giữa hai người nói, vòng tròn ẩn đi. Không có nó, một TTS nối các câu thoại không ngừng nghỉ nghe rất robot.

2. Lồng tiếng -- Piper, một mô hình cho mỗi đặc vụ

Mỗi đặc vụ có mô hình Piper riêng (.onnx) được huấn luyện chuyên biệt, lưu trong voices/<agent>/. Văn bản sinh ra đi qua mô hình tương ứng, cho ra một WAV. Đây là cùng công nghệ tôi dùng để huấn luyện giọng tùy chỉnh nói chung (xem bài về pipeline Piper/Kaggle) -- ở đây áp dụng trực tiếp trong prod, tức thì, mỗi lần tạo video.

3. Phụ đề karaoke -- ASS sinh ra, màu trích từ biểu tượng

Phụ đề không phải .srt đơn giản. Đó là tệp .ass (Advanced SubStation Alpha) sinh ra từng từ, với hiệu ứng karaoke : mỗi từ sáng dần bằng một màu khi được phát âm, trong khi phần còn lại của văn bản giữ màu trung tính. Màu nhấn không cố định -- nó được trích động từ biểu tượng của đặc vụ đang nói (một script Python chạy PIL trên PNG biểu tượng, lấy mẫu các pixel không trong suốt, và trả về các màu trội). Kết quả : phụ đề của Killjoy sáng màu tím, của Jett màu xanh ngọc, mà không màu nào bị ghi cứng ở đâu cả.

4. Vòng tròn phản ứng âm thanh -- một biểu thức FFmpeg cho mỗi frame

Đây là phần quái nhất của pipeline, và có lẽ là phần tôi tự hào nhất. Biểu tượng tròn của đặc vụ đang nói không đứng yên : nó phóng to thu nhỏ nhẹ theo nhịp giọng nói của chính mình.

Phép tính thực hiện bằng cách đọc WAV thô của câu thoại, tính đường bao RMS (root mean square, một thước đo năng lượng tín hiệu) từng frame ở 60 fps, chuẩn hóa theo giá trị tối đa, rồi làm mượt trên cửa sổ 3 frame để tránh giật. Mỗi giá trị đường bao sau đó được chuyển thành hệ số thu phóng bị chặn bởi MAX_ZOOM_VARIATION (0.2, tức ±20% quanh kích thước cơ sở).

Kết quả tính toán này không được áp dụng qua mã thao tác pixel -- nó được dịch thành một biểu thức điều kiện FFmpeg khổng lồ (lt(n,K)*val + between(n,K,K')*val + ..., một nhánh cho mỗi nhóm frame) điều khiển trực tiếp tham số scale của bộ lọc video. FFmpeg đánh giá biểu thức này ở mỗi frame render. Với một câu thoại vài giây ở 60 fps, nhanh chóng thành hàng trăm nhánh trong một biểu thức duy nhất -- vì vậy tham số STEP gom các frame để giới hạn độ sâu.

5. Render từng đoạn, rồi fisheye trên intro

Mỗi câu thoại được render riêng : nền video (đoạn trích ngẫu nhiên từ một trong các clip gameplay trong bg-video/, cắt đúng độ dài), vòng tròn đặc vụ phủ lên trên với zoom phản ứng âm thanh, phụ đề nhúng qua bộ lọc ass của FFmpeg, âm thanh TTS trộn với âm gameplay nền.

Đoạn đầu tiên nhận xử lý đặc biệt : méo fisheye tan dần trên 20% frame đầu (bộ lọc lenscorrection đánh giá từng frame, cộng tmix=frames=3 trộn các frame kề để mô phỏng motion blur), đồng bộ với tiếng "whoosh". Đó là chuyển cảnh intro tạo cảm giác camera "lao" vào cảnh.

6. Nối và trộn cuối

Tất cả các đoạn được nối liền nhau, nhạc nền (Sneaky Snitch, Kevin MacLeod, giấy phép Creative Commons) được trộn lên trên với ducking audio -- nén sidechain tự động hạ âm lượng nhạc khi một đặc vụ nói, và nâng lên trong khoảng lặng. Toàn bộ chạy 60 fps từ đầu đến cuối, không chuyển đổi framerate giữa các bước.

7. Đăng tự động

Script run-cron.sh, chạy bởi một cron cổ điển, kích hoạt môi trường Python, tải .env, và chạy bun src/workflow.ts --upload. Cờ --upload kích hoạt thêm việc tạo metadata (tiêu đề, mô tả, tag) và gọi uploaders/upload.py, đăng video lên YouTube và Instagram qua hai script riêng (uploaders/youtube/upload.py và uploaders/instagram/). Toàn bộ chuỗi, từ prompt LLM tới video online, chạy không cần can thiệp con người.

Vì sao TypeScript/Bun thay vì thứ gì đó toàn Python

Lựa chọn không mang tính ý thức hệ -- vì Bun cho truy cập trực tiếp và nhanh tới Bun.spawn để điều khiển FFmpeg trong tiến trình con, gõ kiểu mạnh trên cấu trúc dữ liệu pipeline (Phrase, SegmentInfo), và runtime khởi động nhanh hơn nhiều so với Node cho một script chạy bằng cron mỗi X giờ. Hai mẩu Python duy nhất trong dự án nằm đúng chỗ Python thực sự có công cụ tốt nhất : PIL để trích màu, và các API upload (google-api-python-client cho YouTube, stack Instagram Graph API cho IG).

Điều này minh họa điều gì

Dự án này là một ví dụ hay về những gì có thể xây dựng ngày nay với các viên gạch hoàn toàn miễn phí hoặc open source : một LLM nhanh và miễn phí qua API Groq, một động cơ TTS cục bộ chạy không cần GPU chuyên dụng, FFmpeg cho toàn bộ render video -- và chất kết dính, chỉ là vài trăm dòng TypeScript. Không viên gạch nào mới riêng lẻ. Điều làm nên pipeline chính là cách sắp xếp : tạo một kịch bản mạch lạc với quan hệ thật giữa các nhân vật, biến nó thành âm thanh biểu cảm với khoảng nghỉ tự nhiên, đồng bộ render hình ảnh theo năng lượng âm thanh đó từng frame, và tự động hóa toàn bộ chuỗi tới khi đăng bài.

---

Tài nguyên

• Repo : github.com/fox3000foxy/valorant-short-maker
• Kênh : @valorant_agents

3 điểm chính

1. Kịch bản do LLM tạo (Groq/Llama 3.3) với persona và quan hệ cho mỗi đặc vụ, không phải danh sách câu đùa viết sẵn đơn giản.
2. Zoom vòng tròn đặc vụ được điều khiển bởi biểu thức FFmpeg tính từng frame từ đường bao RMS của WAV -- không phải animation keyframe cổ điển.
3. Toàn bộ chuỗi, từ prompt tới bài đăng YouTube/Instagram, chạy qua một cron job duy nhất không cần can thiệp con người.

valorant-short-maker : กระบวนการสร้างสั้น ๆ ของฉันเอง

Depuis quelques mois, une chaîne YouTube tourne sans que j'y touche : @valorant_agents. Des agents Valorant qui se chambrent entre deux rounds, doublés, sous-titrés en karaoké, publiés en Shorts. Tout est généré par valorant-short-maker, un pipeline TypeScript/Bun qui tourne en cron et publie sans que personne n'ait à cliquer sur quoi que ce soit.

Ce que ça donne

Trois frames extraites de la vidéo générée pour "Duelist Debate" (Phoenix, Yoru et Jett) :

![อีกบรรทัดหนึ่ง, สีของคำบรรยายเปลี่ยนตามตัวแทน](/images

แต่ละรันจะเลือกตัวละคร 3 ถึง 4 ตัวแบบสุ่มจาก 26 ตัวที่พร้อมใช้งาน และส่งพรอมต์ระบบไปยัง Llama 3.3 70B (ผ่าน Groq) ซึ่งประกอบด้วยสำหรับตัวละครแต่ละตัวที่เลือก มีสรุปกระชับเกี่ยวกับบุคลิกภาพและความสัมพันธ์กับตัวละครอื่น ๆ ที่อยู่ในฉาก (เหล่านี้อยู่ใน src/lore/ ไฟล์หนึ่งไฟล์ต่อหนึ่งตัว) พรอมต์บังคับให้มีกฎระเบียบเฉพาะ : ประโยคสั้นแต่กระชับต่อแต่ละการพูด, การวนเวียนที่เป็นธรรมระหว่างตัวละคร, ควรมีความตลกเป็นหลัก, และสำคัญที่สุดคือให้มีพักระหว่างบทพูด

ตัวอย่างจริงกับ “Duelist Debate” -- Phoenix, Yoru และ Jett โต้แย้งกันเพื่อดูว่าใครจะเล่น duelist, สร้างเมื่อวันที่ 6 กรกฎาคม 2026 :

phoenix: ฉันบอกคุณแล้วว่าฉันมีทักษะพอที่จะเล่น duelist ในแมทช์นี้
yoru: ทักษะ คุณเรียกว่าจุดไฟด้านไหนก็ 'ทักษะ' ?, Phoenix
jett: ฉันเร็วที่สุดที่นี่ ฉันควรเล่น duelist
phoenix: เร็วที่สุด, แต่คุณทนไฟไหม, Jett [0.3] ฉันสงสัยว่ามันจะเป็นไง
yoru: ไฟ ล่ะ พูดว่าไฟของคุณร้อนกว่าแผ่นแตกของฉัน
jett: นี่ไม่ใช่เรื่องไฟหรือแสง, มันคือความเร็วและความคล่องตัว
phoenix: โอเค เข้าใจแล้ว, ดังนั้นคุณอาจจะเป็นผู้เชี่ยวชาญคนนั้นเดเฉลย, Yoru [0.3] นั่นคือเรื่องที่เสมือน
yoru: อย่างน้อยฉันไม่ใช้วิธีไฟแบบราคาถูก
jett: ไฟแบบราคาถูก, นั่นคือสิ่งที่คุณเรียกว่าเทคนิคของ Phoenix
phoenix: เฮ้, เทคนิคไฟของฉันช่วยให้เราหัวเข้าแก่นอ้างก่อน [0.3] ไม่รู้สึกเหมือนสำหรับแผ่นแตกของคุณเอง, Yoru
yoru: แก่นอ้าง, ความหมายว่าเมื่อฉันข้ามผ่านเกาะของเราออกจากกล่องกลม
jett: พอ, เรื่องนี้มันถือในเวลา, ให้เราตัดสินใจพูดกันแล้วหน่อย
phoenix: ตายตัว, แต่ฉันยังคงบอกว่าฉันเป็น duelist ที่ดีที่สุดที่นี่
yoru: เชิญ, คุณคิดจะจับพวกข้างตรงคนเดียว [0.3] ฉันสงสัยว่ามันจะเป็นไง
jett: ฉันจะเอามันไป, ไม่มีปัญหา, ฉันเร็วที่สุด
phoenix: เร็วที่สุด, ใช่, แต่คุณจะควบคุมปริชัดกว่าเขา [0.3] นี่คือข้อสังเกต
yoru: ควบคุม, ล่ะ, คุณคิดว่าคุณจะควบคุมใคร? Phoenix
jett: นี่มันกลิ้ง, เราจะกันคำนี้ได้เลย
phoenix: ตายตัว, เรามาเล่นทีละสมาชิกรับว่าดีกว่า [0.3] ถ้าคุณก็ทำคลิกเดียวกัน
yoru: ไปเลย ฉันจะแสดงให้คุณเห็นว่าผู้ duelist จริงจังเป็นอย่างไร
jett: ฉันไม่ถอย, ฉันเล่น duelist
phoenix: โอ, นี่ดี [0.3] มาดูกันว่าท่านที่มากระเป๋ากันได้จริงไหม
yoru: เราจะมา, จะได้มุมมองใหม่ลงไป Jett
jett: ใช่เลย ให้สิ้นสุดการโต้แย้งนี้ครั้งนี้
pause: 0.3
phoenix: โอเค, ก็เริ่มเลย [0.3] ขอให้ duelist ที่ดีที่สุดชนะ
yoru: ฉันแน่ใจว่าจะทำให้คุณไหม, Phoenix, ไม่ใช่ไฟ แต่ด้วยแผ่นแตกของฉัน
jett: ฉันจะทำพวกคุณโดด, ไม่มีปัญหา

Les pauses sont le détail qui rend le rythme naturel : [0.3] inséré au milieu d'une réplique crée un silence de 0.3s dans le fichier audio sans couper le cercle de l'agent à l'écran, alors qu'une ligne pause: 1.0 à part entière crée un vrai silence entre deux locuteurs, cercle caché. Sans ça, un TTS qui enchaîne les répliques sans respirer sonne robotique.

2. Donner une voix -- Piper, un modèle par agent

Chaque agent a son propre modèle Piper (.onnx) entraîné spécifiquement, stocké dans voices/<agent>/. Le texte généré passe dans le modèle correspondant, ce qui sort un WAV. C'est la même techno que j'utilise pour le training de voix custom en général (voir l'article sur le pipeline Piper/Kaggle) -- ici appliquée directement en prod, à la volée, à chaque génération de vidéo.

3. Sous-titres karaoké -- ASS généré, couleur extraite de l'icône

Le sous-titrage n'est pas un simple .srt. C'est un fichier .ass (Advanced SubStation Alpha) généré mot par mot, avec un effet karaoké : chaque mot s'illumine dans une couleur au fur et à mesure qu'il est prononcé, pendant que le reste du texte reste dans une couleur neutre. La couleur d'accent n'est pas fixe -- elle est extraite dynamiquement de l'icône de l'agent qui parle (un script Python fait tourner PIL sur le PNG de l'icône, échantillonne les pixels non-transparents, et renvoie les couleurs dominantes). Résultat : le sous-titre de Killjoy s'illumine en violet, celui de Jett en bleu-vert, sans qu'aucune couleur n'ait été codée en dur quelque part.

4. วงกลมตอบสนองเสียง -- การแสดงผล FFmpeg frame โดย frame

C'est la partie la plus tordue du pipeline, et probablement celle dont je suis le plus fier. L'icône ronde de l'agent qui parle ne reste pas statique : elle zoome et dézoome légèrement au rythme de sa propre voix.

Le calcul se fait en lisant le WAV brut de la réplique, en calculant l'enveloppe RMS (root mean square, une mesure de l'énergie du signal) frame par frame à 60 fps, en normalisant par le maximum, puis en lissant sur une fenêtre de 3 frames pour éviter les à-coups. Chaque valeur d'enveloppe est ensuite convertie en un facteur d'échelle borné par MAX_ZOOM_VARIATION (0.2, donc ±20% autour de la taille de base).

ผลลัพธ์ของการคำนวณนี้ไม่ได้รับการประยุกต์โดยโค้ดที่จัดการกับพิกเซล -- มันถูกแปลเป็นนิพจน์เงื่อนไขยักษ์ใหญ่ของ FFmpeg (lt(n,K)*val + between(n,K,K')*val + ..., สาขาหนึ่งต่อกลุ่มเฟรม) ที่ควบคุมพารามิเตอร์ scale ของตัวกรองวิดีโอโดยตรง FFmpeg จะประเมินนิพจน์นี้ในทุกเฟรมของการเรนเดอร์ สำหรับการซิงค์อันสั้นประมาณสองสามวินาทีที่ความเร็ว 60 fps ก็ทำให้มีหลายร้อยสาขาในนิพจน์เดียว -- นั่นจึงเป็นพารามิเตอร์ STEP ที่รวมหรือรวมเฟรมเพื่อจำกัดความลึก

5. การเรนเดอร์โดยส่วน, จากนั้น fisheye ในตอนเปิดตัว

ทุกซิงค์ถูกเรนเดอร์แยกตัวมันเอง: พื้นหลังวิดีโอ (สุ่มคลิปจากกราฟิกใน bg-video/, ตัดให้เหมาะสม), วงกลมของตัวแทนบนสุดพร้อมซูมเสียง-ตอบสนอง, แซทท์อัพดีเนี้่นผ่านตัวกรอง ass ของ FFmpeg, เสียง TTS ผสมกับเสียงเล่นเกมอยู่พื้นหลัง

ส่วนแรกทั้งหมดรับการดูแลพิเศษ: การบิดเบือน fisheye ที่ค่อยๆ ค่อย ๆ เชื่อมกันใน 20% ตัวเริ่มแรก (ตัวกรอง lenscorrection ประเมินเฟรมต่อเฟรม, เพิ่มอีก tmix=frames=3 ที่ผสมเฟรมที่ถัดมาเพื่อจำลองมอชท์บลูร์) โดนซิงค์กับเสียงหัวเราะ "whoosh" เป็นจุดเปลี่ยนต่อที่ทำให้ผู้ชมรู้สึกว่าพลิกกล้อง "กลับไปกลับมาจาก" ออกมาจากฝั่ง เหมือนกับว่าเป็นการเปิดตัว

6. การเชื่อมต่อและการผสานน้ำเสียงสุดท้าย

(หมายเหตุ: พยายามครั้งก่อนพัง: backticks. ระวังเลย)

Tous les segments sont concaténés bout à bout, la musique de fond (Sneaky Snitch, Kevin MacLeod, licence Creative Commons) est mixée par-dessus avec du ducking audio -- une compression sidechain qui baisse automatiquement le volume de la musique pendant qu'un agent parle, et qui remonte pendant les silences. Le tout tourne en 60 fps de bout en bout, aucune conversion de framerate entre les étapes.

7. การเผยแพร่อัตโนมัติ

Le script run-cron.sh, lancé par un cron classique, active l'environnement Python, charge le .env, et lance bun src/workflow.ts --upload. Le flag --upload déclenche en plus la génération de métadonnées (titre, description, tags) et appelle uploaders/upload.py, qui publie la vidéo sur YouTube et Instagram via deux scripts séparés (uploaders/youtube/upload.py et uploaders/instagram/). Toute la chaîne, du prompt LLM à la vidéo en ligne, tourne sans intervention humaine.

ทำไมเลือกใช้ TypeScript/Bun แทนวิธีการเป็นเพียง Python

Le choix n'est pas idéologique -- c'est que Bun donne un accès direct et rapide à Bun.spawn pour piloter FFmpeg en sous-processus, un typage fort sur les structures de données du pipeline (Phrase, SegmentInfo), et un runtime largement plus rapide au démarrage que Node pour un script qui tourne en cron toutes les X heures. Les deux seuls bouts de Python dans le projet sont là où Python est réellement le mieux outillé : PIL pour l'extraction de couleurs, et les APIs d'upload (google-api-python-client pour YouTube, la stack Instagram Graph API pour IG).

สิ่งที่แสดงออก

Ce projet est un bon exemple de ce qu'on peut construire aujourd'hui avec des briques entièrement gratuites ou open source : un LLM rapide et gratuit via l'API Groq, un moteur TTS local qui tourne sans GPU dédié, FFmpeg pour tout le rendu vidéo -- et le liant, ce n'est que quelques centaines de lignes de TypeScript. Aucune de ces briques n'est nouvelle individuellement. Ce qui fait le pipeline, c'est l'agencement : générer un script cohérent avec de vraies relations entre personnages, le transformer en audio expressif avec des pauses naturelles, synchroniser un rendu visuel sur l'énergie de cet audio frame par frame, et automatiser toute la chaîne jusqu'à la publication.

---

ทรัพยากร

• คลัง : github.com/fox3000foxy/valorant-short-maker
• ช่อง : @valorant_agents

3 points clés

1. สคริปต์ถูกสร้างโดย LLM (Groq/Llama 3.3) พร้อมบุคลิกภาพและความสัมพันธ์ตามตัวแทน ไม่ใช่เพียงแค่รายชื่อมุกตลกที่เขียนล่วงหน้า
2. การซูมของวงกลมตัวแทนถูกควบคุมโดยนิพจน์ FFmpeg ที่คำนวณเฟรมต่อเฟรมจาก envelope RMS ของ WAV -- ไม่ใช้การเคลื่อนไหวแบบ keyframes แบบคลาสสิก
3. ทั้งชุดทั้งหมด ตั้งแต่ prompt ไปจนถึงโพสต์บน YouTube/Instagram ทำงานผ่าน cron job เดียว ไม่มีการแทรกแซงจากมนุษย์