Roadmap

This is what we are planning to implement:

1. Automatic Face Detection:

   • Run face detection jobs across your entire library or selected videos
   • Automatically cluster similar faces together as potential matches
   • Process both photos and videos for comprehensive face detection
   • Support for batch processing of multiple files

2. Manual Face Management:

   • Add manual face annotations when automatic detection misses faces
   • Correct misidentified faces through the dedicated face management panel
   • Remove false positives or unwanted face detections
   • Merge multiple face clusters into a single entity
   • Name and tag individuals for easy organization

3. Face Visibility Controls:

   • Toggle face detection overlays on/off in the video player
   • Customize how faces are displayed (boxes, names, confidence scores)
   • Filter videos based on detected faces
   • Search your library by recognized individuals

4. Metadata Integration:

   • Import face metadata from Digikam format
   • Export face recognition data for use in other applications
   • Maintain face recognition data across library updates
   • Sync face metadata with your existing tagging system

5. Use Cases:

   • Legal and Law Enforcement:
     • Process discovery footage to find specific individuals (e.g., clients, officers, suspects)
     • Configure high-priority thorough searches for critical faces
     • Create collections of known individuals for quick filtering
     • Document and track appearances across multiple videos
   • Content Creation and Rights Management:
     • Verify model release forms by tracking all faces in your content
     • Create collections of approved talent for quick filtering
     • Identify unapproved appearances that need releases
     • Track recurring characters or talent across your content library
   • Family and Personal Archives:
     • Create collections of family members to easily find all appearances
     • Filter videos to show only clips with specific family members
     • Track appearances of loved ones across years of footage
     • Share collections of specific people with family members
   • Educational and Training Content:
     • Track instructor appearances across lecture series
     • Create collections of subject matter experts
     • Filter content by presenter for curriculum planning
     • Document student participation in recorded sessions
   • Event and Conference Management:
     • Track speaker appearances across multiple sessions
     • Create collections of keynotes and panelists
     • Filter content by presenter for post-event distribution
     • Document audience participation in Q&A sessions

6. Technical Implementation:

   • Multi-Model Stack:
     • YOLO for fast person detection
     • InsightFace for high-quality face embeddings
   • Tiered Processing Pipeline:
     • Fast initial pass to detect people in video
     • Face alignment and extraction
     • High-accuracy face embedding generation
   • Smart Processing Controls:
     • Configurable effort levels per video (rough to thorough)
     • Adaptive frame sampling based on face quality
     • Prioritize frames with well-aligned faces
     • Incremental processing with resume capability
   • Performance Optimizations:
     • Process 1 hour of footage in ~1 minute on a laptop
     • Configurable frame sampling rates (e.g., 10% sampling for 10x speedup)
     • Smart heuristics to select optimal frames for analysis
   • Adaptive Search:
     • Configurable search effort per face profile
     • Dynamic effort adjustment for high-priority matches
     • Reuse results from previous scans to accelerate higher-effort processing
     • Adjustable confidence thresholds

This feature is particularly valuable for:

• Legal professionals and law enforcement
• Content creators and rights managers
• Family archivists
• Educational institutions
• Event organizers
• Anyone managing large collections of people-focused content

The face recognition system is designed to be both powerful and user-friendly, allowing you to leverage AI assistance while maintaining full control over the results through manual annotation and correction tools.

Here's how it works:

1. Original Video Library: You have a gallery of media files (videos) that you've imported into Video Clip Library. These could be webinars, interviews, event recordings, or any other video content.

2. Clip Creation: Instead of creating actual new video files when you want to mark a specific segment, you can create a "virtual clip" that:

   • References the original video file
   • Specifies a start and end time within that video
   • Stores metadata (titles, notes, tags) specific to that segment
   • Doesn't create any new video files until you actually need to export

3. Key Benefits:

   • Storage Efficiency: You can mark hundreds of moments across your content without creating a single new file
   • Flexible Organization: You can add tags, titles, and notes to specific segments
   • Offline Access: The virtual clips work even when your drives are disconnected
   • Export Control: When you're ready to use a clip, you can export it in the format you need (low-resolution proxy for editing or full-quality for publishing)

4. Use Cases:

   • Creating highlight reels from long interviews
   • Marking key moments in webinars for social media
   • Organizing specific segments of customer testimonials
   • Preparing rough cuts before opening your editor
   • Creating collections of related moments across different videos

5. Technical Implementation:

   • The system stores metadata about the clip (start/end times, tags, etc.)
   • The original video files remain untouched
   • When you need to use a clip, the system can generate the actual video segment on demand
   • Supports export in .FCPXML format for seamless integration with editing software

This approach is particularly valuable for content creators, marketers, and anyone working with large video libraries because it allows for efficient organization and discovery of specific moments without the storage overhead of creating multiple copies of the same content.

The feature was added in response to user requests for the ability to specify ranges within video files and attach metadata to those specific segments, which led to a reorganization of the data model to support this functionality.

Platforms: Windows, Mac, and Linux

Key Features

• Supports 100,000s of videos
• Manages 100s of TB
• One year of free updates included
• Perfect for data hoarders and privacy enthusiasts