VIDEO.md
Last Updated: September 18, 2025 Commit: Refactored video recording code to fix bugs and improve reliability
A video recording feature has been implemented for the FastLED web application in src/platforms/wasm/compiler/. This feature allows users to capture both the canvas animations and audio output to create high-quality video recordings.
Ctrl+R / Cmd+Rfastled-recording-YYYY-MM-DD-HH-MM-SS.{mp4|webm} (extension matches codec)The video recording system implements comprehensive dropped frame handling:
// Dropped frame detection
let expectedFrameCount = 0;
let actualFrameCount = 0;
const frameDropThreshold = 2; // Allow 2 consecutive drops before intervention
mediaRecorder.addEventListener('dataavailable', (event) => {
actualFrameCount++;
const timestamp = performance.now();
const expectedTimestamp = startTime + (expectedFrameCount * (1000 / fps));
if (timestamp - expectedTimestamp > frameDropThreshold * (1000 / fps)) {
console.warn(`Dropped frames detected: expected ${expectedFrameCount}, got ${actualFrameCount}`);
// Trigger recovery actions
handleDroppedFrames(expectedFrameCount - actualFrameCount);
}
console.log(`frame ${actualFrameCount}`);
expectedFrameCount++;
});
function handleDroppedFrames(droppedCount) {
// Option 1: Duplicate last frame to maintain timing
// Option 2: Reduce capture frame rate temporarily
// Option 3: Skip frame interpolation for smooth playback
}
The H.264 encoder in MP4 format provides several optimizations for static or repetitive content:
// Configure encoder for optimal duplicate frame handling
const encoderConfig = {
mimeType: 'video/mp4;codecs=h264,aac',
videoBitsPerSecond: 10000000,
audioBitsPerSecond: 128000,
// H.264 specific optimizations
keyFrameInterval: 30, // I-frame every 30 frames (1 second at 30fps)
frameSkipping: true, // Allow encoder to skip identical frames
contentHint: 'motion' // Optimize for animation content
};
// Monitor encoding efficiency
mediaRecorder.addEventListener('start', () => {
console.log('Encoder supports duplicate frame optimization: ',
mediaRecorder.mimeType.includes('h264'));
});
When MP4/H.264 is not available, the system falls back to WebM/VP9:
// Track encoding performance
const encodingStats = {
framesEncoded: 0,
framesDuplicated: 0,
bytesGenerated: 0,
compressionRatio: 0
};
mediaRecorder.addEventListener('dataavailable', (event) => {
encodingStats.bytesGenerated += event.data.size;
encodingStats.framesEncoded++;
// Estimate duplicate frame optimization
const expectedSize = (canvas.width * canvas.height * 3) / 8; // RGB to bytes
const actualSize = event.data.size;
const efficiency = 1 - (actualSize / expectedSize);
if (efficiency > 0.8) {
encodingStats.framesDuplicated++;
console.log(`High compression frame ${encodingStats.framesEncoded} (${efficiency.toFixed(2)} efficiency)`);
}
});
modules/video_recorder.js (NEW)
index.html (MODIFIED)
index.css (MODIFIED)
index.js (MODIFIED)
// Canvas capture
canvas.captureStream(fps) // Creates MediaStream from canvas
// Audio capture with MP3 mixing
audioContext.createMediaStreamDestination() // Creates audio destination node
// MP3 player audio routing
mp3Player.connect(audioDestination) // Route MP3 audio to recording destination
synthesizer.connect(audioDestination) // Route synthesized audio to destination
// Combined stream with mixed audio
new MediaStream([
...videoStream.getVideoTracks(),
...audioDestination.stream.getAudioTracks() // Contains mixed audio from all sources
])
// Recording with mixed audio
mediaRecorder = new MediaRecorder(stream, {
mimeType: 'video/mp4;codecs=h264,aac', // Default: MP4 with H.264 video and AAC audio
videoBitsPerSecond: 10000000,
audioBitsPerSecond: 128000 // Captures all mixed audio sources
})
// Frame logging during recording
mediaRecorder.addEventListener('dataavailable', (event) => {
console.log(`frame ${++frameCounter}`); // Logs each frame insertion
});
File Save Timing: Files are saved AFTER recording stops (not when it starts)
High Bitrate Choice: 10 Mbps video bitrate
MP4/H.264 Format: Primary codec choice
// Get recorder instance
window.getVideoRecorder()
// Start recording programmatically
window.startVideoRecording()
// Stop recording programmatically
window.stopVideoRecording()
Ctrl+R / Cmd+R: Toggle recording on/offMediaRecorder.isTypeSupported()