Introducing Intelligent Sampling: Advanced Video Analytics with Multimodal AI

Our mission at Coactive AI is to unlock visual content. With Coactive, AI can understand video content like a human would, “seeing” attributes such as moods, emotions, people, and places–no metadata or tagging required. Today, that ability to “see” just got a whole lot better. We’re proud to announce the release of Intelligent Sampling.

With Intelligent Sampling, Coactive customers can efficiently process large video datasets by identifying key moments, like changes in background scenery, gestures such as a hand wave, or actions like scoring a goal. This innovative approach to processing results in significant efficiency gains, particularly for media & entertainment companies with massive media libraries. At the same time, it improves the accuracy of results for the person searching by cutting and isolating the most valuable content. Intelligent Sampling streamlines the content workflow, saving time, reducing spend on data processing, and ensuring you discover the most unique and valuable content.

Intelligent Sampling provides benefits across the business. For users, Intelligent Sampling is a content discovery superpower when searching large libraries of video content. For business leaders, Intelligent Sampling provides a far more cost-effective approach when processing large video datasets. What used to be prohibitively expensive is now available at your fingertips. 

Putting Intelligent Sampling to work

Traditional approaches to preprocessing uniformly sample keyframes at a specific framerate (e.g. two seconds per frame), resulting in a ton of unnecessary frames. This requires extra work for users who need to sort through redundant content and increases the cost of compute and storage.

Intelligent Sampling is a smarter AI-based approach using a proprietary algorithm that selects fewer and more targeted keyframes. Coactive detects changes in the video that indicate something of interest is happening and delivers these key moments to users. The result? Better search results for users and fewer frames to process and store for the company.

While building and testing Intelligent Sampling, we explored several use cases where this approach significantly improves the video analytics experience:

• News Segments: If you’re searching for a news segment about Hurricane Milton, you may only want frames that show live footage vs. frames showing the news anchor speaking. With Intelligent Sampling, results will provide clean cuts between anchor and background footage, making it easy to discover the results you need.
  
  Combine this technique with Coactive Dynamic Tags, and you can tag the on-site footage in seconds, completely removing results that include the anchor.
  
  
• TV Talk Shows: Intelligent Sampling will capture even subtle movements, such as head nods or hand gestures, making it easy to spot key transition moments in a long video segment.
  
  
• Sports Broadcast: Sports segments are typically 1-2 hours long, with scene changes due to the movements of the ball and players. With Intelligent Sampling, you can easily pick out the key transition moments and avoid sorting through the more repetitive parts of the game.
  
  
• User-Generated Content: Community platforms and consumer brands manage large amounts of user-generated content. With Intelligent Sampling, moderators can quickly spot valuable brand moments for marketing amplification, as well as surface and remove brand-damaging content. 

‍

The benefits of Intelligent Sampling

Intelligent Sampling is powered by the Coactive Multimodal Application Platform (MAP) and is part of Coactive’s analytics capabilities. For users of the Coactive platform, Intelligent Sampling provides significant benefits:

• Scale: Unlock massive media libraries with performant and cost-effective search that delivers the results you need. 
  ‍
• Accuracy: Reduce the noise in your search results and focus only on the most unique and high-value content.
  ‍
• Speed: Reduce redundancy in results so you can find relevant video clips faster. 
  ‍
• Customization: Combine the benefits of uniform and Intelligent Sampling with adjustable parameters. For example, in sports content where keyframes need to capture critical moments like a goal, Coactive has an adjustable parameter that can increase the frequency of keyframe capture, ensuring no key moments are missed. 

How to use Intelligent Sampling

Intelligent Sampling is now available for all Coactive customers. Whether you use the Coactive UI or power an application with Coactive, you can turn on Intelligent Sampling with a simple configuration, or send us a note, and we’re happy to help you set it up!

Interested in learning more about how Coactive can help you unlock your visual content? Get in touch! 

‍