On Demand Intelligent Video Analytics


1. Introduction

Our initial conversations with potential customers focus on presenting:

1.1. The Need for Video Analytics

  • Rapid expansion of video cameras deployed in public & private spaces

    • Various industry studies estimate the total number of video cameras shipped to be over 200 million

  • Industry study found that a human will miss up to 95% of activity after viewing video for 22 minutes

  • 1.2. The IVA Value proposition

    IVA analyzes video and generates a rich set of descriptive metadata about every object observed. Size, speed, color, area/track of motion, duration, object type/features are indexed in a database, enabling customers to:

    • Reduce the time investigators need to forensically search for events of interest on recorded video

    • Increase the number of cameras that can be effectively monitored in real-time by a human operator

    • Perform Statistical Analysis and Correlation of the metadata to discover trends and patterns

    2. IVA Capabilities

    2.1. IVA Metadata

    There are two categories of IVA Metadata:

    • Events are attribute-based descriptions of every object

      • Events support both forensic searching and statistical analysis.

    • Alerts are configured to be generated when objects meeting specified criteria match configured behavior patterns

      • Alerts support real-time monitoring and can also be used for statistical analysis

    2.2. IVA Differentiators

    Video Analytics are relatively immature when compared to many information technologies. Marketing hype, outlandish claims about capabilities, and perceptions created by television and movies have created unrealistic expectations, leading to dissatisfied, disillusioned customers. IVA is designed and developed to deliver true value to customers, addressing current technological limitations with a number of innovations:

    • Attribute Rich Metadata

      IVA Events are exceptionally detailed in the set of attributes they produce

      • The richness of the attributes improves the precision of both search queries and alerts

      • In addition to generic attributes such as size, speed, and color, IVA detects facial features of people in suitable camera views giving its people-search capabilities a level of precision unmatched in the industry.

    • Confidence Scoring and Ranking

      Rather than producing exact results many IVA analytics algorithms, such as color classification and object/feature detection, generate confidence scores.

      • Combined with confidence ranking, this allows search results to be presented to investigators sorted by confidence so the “best” matches to their queries are seen first.

      • This practice is similar to the way IBM Watson presents results. IVA makes “suggestions” that greatly improve productivity while allowing humans to make final decisions.

      • The rich attribute set of IVA event metadata allows search criteria to be highly granular, improving the effectiveness of confidence ranking. When multiple attributes are selected to be searched on, the results are quite spectacular.

    • Scenario-based Profiles

      IVA includes many advanced algorithms, such as motion detection, object tracking, color classification, and object type/feature detection, each focused on a single purpose. Profiles combine different algorithms in different combinations targeted at specific scenarios.

      • Since every camera view is different, all algorithms are not appropriate to all cameras. Profiles allow each scene and set of use cases to be analyzed by the algorithms best-suited for the scenario.

      • In addition to determining the algorithms to be used, profiles contain configuration parameters for the algorithms, allowing them to be optimized for each scenario.

      • IVA includes a set of Profiles that are pre-configured for the most common scenarios. Custom Profiles can be created in the field to meet a customer’s particular requirements

    • Advanced Analytic Algorithms created by IBM Research

      IBM researchers are continuously refining existing algorithms, developing new algorithms, and applying new technological developments to IVA algorithms

      • IBM holds a large number of computer vision patents and adds to this patent library on an ongoing basis.

      • IBM Research has developed a tool set and framework to statistically measure the accuracy of its algorithms. Until an algorithm has met the accuracy requirements, it is not released.

      • IBM Researchers are part of the world-wide computer vision research community. They regularly publish articles in the relevant academic journals and stay abreast of developments in the field, allowing them to apply significant advances in the field to IVA capabilities.

      • IVA analytics include innovations that allow them to perform well in environments where high activity tends to decrease the effectiveness of typical analytics.

    • Massive Scalability

      Built on enterprise-class IBM software products, IVA can:

      • Scale to support a theoretically unlimited number of cameras

      • Handle extremely high activity levels common to large urban scenes, transportation portals, and other similar environments

    • Open and Extensible

      The IVA product is built on an extensible framework, allowing the product and solutions including it to be extended to include:

      • Analytics inputs from 3rd parties

      • External data source inputs

      • Output of IVA metadata to external applications

      • Extensions to the IVA application user interface

    • Sophisticated Real-time Alerting

      Real-time alerting is the premise that allows operators to monitor live video more effectively, but high numbers of false alerts will diminish the value of the alerts. IVA reduces the number of false alerts by:

      • Using the rich metadata to allow alerts to include combinations of conditions. More exacting conditions specified in an alert make it less likely that a false alert will be generated.

      • Combining alerts into “Compound Alerts” that will trigger only when multiple basic alerts are triggered within a specified time frame.

    3. Demonstrations and Proofs of Concept

    The fundamental difference between a demonstration and a proof-of-concept is that a demonstration is delivered using pre-produced materials and is intended to introduce a customer to IVA’s capabilities, while a proof-of-concept allows the customer to select the video content that will be used and is intended to prove the viability of IVA for their use cases.

    A demonstration is expected to provide a high level of predictability while a proof-of-concept is subject to different factors which are prone to affect the results.

    3.1. Demonstrations

    Demonstrations come in two types and should be used to demonstrate some of the basic IVA capabilities to emphasize how they can deliver the forensic searching and real-time alerting value points. They should also introduce the customer to the high level flow of video and metadata within an IVA solution to help the customer understand how the metadata is used to find past events of interest and alert operators to ongoing activity of interest.

    • “Canned” demos are pre-recorded videos that walk the viewer through a scripted demonstration

      • They should show the customer the basic operation of IVA including searching and alerting, and demonstrate a limited set of use cases.

    • “Live” Demos use pre-selected recorded videos to simulate live cameras which serve as the video inputs to a functional IVA system.

      • Videos are selected to contain interesting activity and support pre-configuration of the system for selected use cases such as alerts and saved searches.

      • A live demo provides for interaction with the customer, in contrast to the scripted nature of a canned demo.

      • While customer interaction is one advantage to a live demo, this can also affect the predictability of the demo. While interactively changing search criteria is encouraged, keep in mind that the simulated cameras may not contain activity that matches the new search criteria. Where possible, customer questions that can be anticipated based on early conversations with the customer should be rehearsed before being demonstrated to the customer.

    3.2. Proofs of concept

    Customers who want more confidence that they will be making a sound decision to purchase IVA frequently request a proof-of-concept (POC). While POC’s can be a major gating factor in closing a deal they should be approached with caution, as a failed POC will kill a deal more surely than a successful POC will seal one.

    POC’s are costly, time consuming, and can slow the sales cycle down. If possible they should be avoided, but many customers will require one before buying.

    Before a POC is started, a preferable option that may accomplish the desired result is to collect customer videos and analyze them with the IVA Analytic Tuning Tool. Videos showing the results can be produced and shown to the customer without the cost of delivering a POC.

    • Free POCs are not recommended. A customer who is not willing to make a small investment in a POC is, in all likelihood, not a good prospect.

    • POCs should be limited in scope, demonstrating a handful of use cases that will provide the highest value to the customer.

    • POCs must have clearly specified criteria for success and measuring success. This is the most important requirement for starting a POC. Specific time frames are also required but some flexibility is desirable in completing the POC. The best criteria for success is to show that IVA will deliver on its value points as opposed to executing any specific use cases.

    • Video for a POC should be carefully selected to meet IVA video requirements and must contain activity that will demonstrate the selected use cases.

      • In many cases, activity needed to demonstrate the use cases will not happen frequently so appropriate activity will have to be staged.

      • Staging activity will likely need to be done more than once_ first to support tuning of the analytics, and again to demonstrate performance.

      • Because of the unpredictable nature of many activities, it is strongly recommended that recorded video containing relevant activity be exported to files which can then be used both to tune the analytics and to demonstrate their performance. If recorded video is an option, the best way to validate that it is viable is to use the IVA Analytic Tuning Tool to visualize analytic performance.

    • The fastest and least costly way to deliver a POC is to use exported video recordings from the customer’s cameras and use them to simulate cameras in a private cloud environment.

      • This approach is very similar to a “live” demo with the customer’s video being used instead of pre-selected videos.

      • Private cloud environments are accessible only by VPN making them more acceptable to customers who are concerned about controlling access to their video.

    4. Setting Expectations

    It is very important to set customer expectations realistically. The customer should be focused on the value and benefit that IVA can deliver and not on unrealistic expectations they may have from marketing hype, competitor claims, or mass media.

    4.1. The Imperfect Nature of Video Analytics

    • Always make it clear that Video Analytics are imperfect. Expectations that results will be perfect or accurate to some arbitrary percentage are unrealistic and cannot be delivered.

      • Use this as an opportunity to emphasize that IVA is designed to deliver value despite the imperfections inherent with the technology

      • Use competitor claims to the contrary to call into question the trustworthiness of those making such claims.

    • Redirect the conversation to IVA’s value propositions

      • IVA has been reported to reduce investigator search times by up to 30 times. EG a search that would require 30 hours can be performed in IVA in 1 hour.

        • How many hours do investigators spend searching for video?

        • How much does an hour of an investigator’s time cost?

        • Simple multiplication will yield a viable Return on Investment figure

      • IVA has been reported to increase the number of cameras an operator can monitor by 10X. EG an operator can monitor 40 cameras instead of 4.

        • How many cameras does the customer need monitored and how many hours a day are they monitored?

        • How much does an hour of an operator’s time cost?

        • Simple multiplication will yield a viable Return on Investment figure

        • In addition to the hard ROI figure, operators tend to have a longer attention span when monitoring alerts than when watching video on cameras but this has yet to be quantified.

    4.2. Viability of Use Cases and Video Inputs

    IVA only includes algorithms that have met IBM Research-defined criteria designed to ensure that they have a reasonable probability of delivering sufficient ROI to customers. Different factors, such as the value of success and the cost of failure, are so different from customer to customer that meaningful calculations are impossible. However, the better the system performs, the more likely that it will deliver value to the customer. With this in mind, it is important that viable use cases are selected and that the video inputs to the system are of sufficient quality to allow the system to perform reliably.

    While available video assets and expectations of performance are important, use case viability is the single most important factor in qualifying an IVA prospect.

    • IVA includes capabilities, such as Abandoned Object and Parked Car alerts, that are targeted at very specific use cases. Other use cases may be supported due to the high flexibility that IVA’s rich metadata provides.

    • Known use cases are typically categorized as Likely, Possible, Unlikely, and Impossible, in reference to the likelihood that they will perform well enough to deliver the required ROI.

    • Many unknown use cases can be supported using existing IVA capabilities.

    • Many use cases will be impossible to support even given IVA’s flexibility. In general, if the use case cannot be supported with a defined set of criteria and combination of IVA metadata attributes, it should not be attempted. Customers who are insistent on non-viable use cases are not good IVA prospects.

    • Even with Likely use cases, the quality of video and, most importantly, the position of cameras are make or break factors determining whether IVA will perform acceptably.

    4.3. Typical IVA Deployment Cycle

    IVA is a complex solution that will not be fully operational immediately after installation. A typical IVA deployment will include several cycles, each of which will add to the value that the system delivers.

    • Installation and Deployment

      • Starts with installation of cameras and other video assets if required

      • Includes deployment of server, networks, and storage

      • After installation, video assets are configured in IVA and analytics engines are configured for the cameras. Once this configuration is complete, the system will support forensic search functions.

      • Validation of video stability and quality are performed and corrections made if necessary.

      • Initial validation of forensic search results is then performed, and basic tuning or calibration of analytic engines may be suggested. Basic tuning and calibration focuses on improving the quality of event metadata based on any inaccurate results discovered during initial validation, and normally completes the first stage.

      • Alerts may optionally be configured in the first stage but in many cases insight gained from performing and/or validating forensic searches will be invaluable in initial alert configuration so it is recommended that alert configuration be performed in the second stage

      • At the conclusion of the first stage, video quality and stability should have been validated and the quality of event metadata optimized to an acceptable level.

    • Alert Configuration and Tuning

      • Once video and event metadata are stable and acceptable, alerts can be configured. Without video stability/quality, alerts cannot be expected to perform well.

      • Initial validation of alerts is performed and alerts are tuned where results are not found to be acceptable. Alert tuning is an iterative process.

      • Alert validation may require staging of activity to test alert performance, especially when alerts are meant to detect activity that happens infrequently.

    • Validation

      • The IVA Operational Model is designed to identify a customer’s desired “sweet spot” operating point which is a balance between false negatives and false positives

        • False negatives occur when the system fails to detect an actual activity.

        • False positives occur when the system incorrectly detects an activity that has not actually occurred.

      • The operating point is selected based on the cost of a false positive or negative, a direct extrapolation of the value of an event. EG if an alert is intended to detect an abandoned bag that could contain a bomb, the cost of a false positive could be extremely high so the operating point should be selected to tolerate false negatives in order to minimize false positives.

      • The Operating Model’s X-axis counts false positives while its Y-axis counts false negatives

        • The Origin, where both false negatives and positives are zero, would be a perfect system

        • Typical Operating points are distributed in a logarithmic curve

        • The goal of optimizing and tuning is to move an operating point closer to the origin.

      • Statistically accurate validation is a costly process that requires large data sets and human input to provide accurate data (aka ground truths) for comparison with analytics output. It is most common to validate that the system is operating at the desired operating point using forensic searching and manually evaluating the quality of the results.

    4.4. Staff Training and Experience

    An IVA solution typically includes three basic staff roles, each with different training requirements and experience profiles

    • The Operator role performs real-time monitoring functions enabled by IVA alerts.

      • This role requires minimal training. Operators can typically be effective after several hours of training.

      • Experience is beneficial to this role mainly in the ability to quickly recognize false positives. The extent to which experience affects operators is directly related to the selected operating point (the more false positives are tolerated, the more experience is needed). It is typical for Operators to achieve sufficient experience after a week or two.

    • The Investigator role performs forensic searching of recorded video.

      • This role requires more training than the Operator role. Investigator training can normally be completed in a full day.

      • Experience is beneficial to this role both in the ability to recognize false positives and in defining optimal criteria for search queries. Because forensic searching activity is determined by the need to investigate incidents, there is insufficient data to determine a typical time required to achieve competence.

    • The Administrator role configures the system, performs operational tasks, and optionally tunes analytics.

      • This role requires the most training. A typical training period for an administrator is several full days

      • Required experience to achieve competence varies depending on the tasks being performed. Configuration typically requires several days of focused activity while operations, troubleshooting, and basic analytics optimization usually require several weeks of experience.

      • An administrator’s experience in tuning and optimizing alerts will directly affect the experience and skill required for the Operator role.

    5. Handling Objections and Customer Requests

    Recommended responses to the most frequently encountered objections and requests are:

    • Video analytics does not work_ the market is full of unhappy customers

      • If expectations have been set correctly this should already be handled. To reiterate, emphasize that IVA is designed to deliver value despite the imperfections inherent with the technology and call attention to some of IVA’s capabilities such as confidence ranking and compound alerts.

    • The price is too high

      • Focus on the value delivered by IVA and offer some ROI figures to justify expense.

      • Note that volume discounts apply to IVA licenses.

      • IVA pricing is not completely intractable so negotiations are an option.

    • Competitor XXX provides statistics that say their analytics are 99% accurate. Can you provide similar accuracy statistics?

      • IVA tests their algorithms for accuracy in the lab but we do not provide these, as setting expectations based on this data would be misleading. Every camera and scene is different so lab results, even for similar use cases, cannot be expected to predict real-world performance.

    • Can you provide customer references or arrange a visit to an existing customer?

      • IVA References are provided by request. Several of our best customers are engaged in sensitive areas and do not wish to be referenced publicly. Customer visits may also be arranged by request.

    • Competitor XXX claims their solution can automatically learn normal behavior and alert only when anomalous behavior is detected

      • Automatic learning of behavior has questionable value as many events of interest follow normal behaviors patterns while many events that follow abnormal behavior patterns are of little significance. IVA’s rich attribute indexing is more flexible and makes searching for and alerting on behavior possible regardless of whether it fits into normal behavior patterns

    6. Defining and Scoping Solutions

    Solution scope and definition are normally the responsibility of the technical seller.

    6.1. Requirements and Compatibility

    • IVA MUST be integrated with a VMS system. Supported VMS systems are listed on the IBM Support Portal, while unsupported VMS systems can usually be integrated via services.

    • IVA will only run on the supported Operating systems as listed on the IBM Support Portal.

    • IVA requires IBM DB2, WebSphere, and WebSphere MQ, but these are included in the IVA product bundle.

    6.2. Product Architecture

    • IVA includes three major components:

      • Analytics Engine (SSE)

        Requests live video streams, analyzes, the streams, and generates metadata

      • Metadata Engine (MILS)

        Accepts incoming metadata from the SSE, indexes and stores it in the database, and makes the metadata available to clients via push or pull interfaces

      • User Interface

        The application used by clients to perform searches and monitor real-time alerts

    • IVA includes several interfaces for access to metadata

      • REST Web Services API

        Supports both metadata queries and subscription to alerts

      • WebSphere MQ

        Supports subscription to alerts over reliable transport

      • Other interfaces (unpublished at time this document was created)

        • SSE Analytics Framework

          Supports integration of 3rd party analytics

        • MILS Task interface

          Supports creation of custom classes that can perform custom logic triggered by incoming metadata

        • UI Extension

          IVA’s dojo-based web interface can be extended to suit customer requirements

    6.3. Solution Architecture

    • Every IVA Solution MUST include a Video Management System (VMS) to provide access to cameras and to record video. The most likely prospects will use one of IVA’s supported VMS systems but integration with others is possible.

    • The SSE receives live video streams from the VMS via the Microsoft DirectShow interface

    • The IVA Web UI receives live and recorded video streams from the VMS via the VMS SDK.

    • In many IVA Solutions a Command Center application is deployed as the top level operational console.

      • IBM Intelligent Operations Center is commonly used as the Command Center application

      • Any Command Center application can be integrated with IVA to receive IVA alerts and incorporate them into the broader operational picture

      • When integrated with a Command Center application, the IVA UI is typically still used as the forensic search user interface

    • IVA can be deployed in three topologies based mainly on the number of cameras to be supported by the system. These topologies determine how the MILS will be deployed.

      • The Single Node topology supports approximately 100 cameras (actual number depends on secondary factors such as activity level and analytic profiles deployed)

      • The Clustered topology supports approximately 500 cameras and is normally selected when camera counts are higher than 100 or where more fault tolerance is required

      • The Federated topology supports a theoretically unlimited number of cameras and offers additional flexibility for distribution of computing sites

        • Federations can include both Single Node and Clustered IVA instances.

        • Each IVA instance in a Federation is administered as an autonomous deployment but IVA objects that are exposed to users are aggregated to present a single view of the overall solution. Solutions integrated via the REST API are also presented with a single aggregated view of the solution.

    • When sizing an IVA system, the relevant inputs include number of cameras, analytic profiles to be deployed on each camera, and activity level for each camera. These inputs determine:

      • The deployment topology that should be selected

      • The number of SSE servers required

      • The amount of storage that will be required to store the IVA metadata. Note that retention period for the metadata is also a factor when calculating storage requirements.

    • A sizing and pricing calculator and a storage calculator are included in the IVA Architecture and Pricing Guide which is available upon request.

      • The sizing and pricing calculator will generate server and storage requirements for the provided inputs and a representative Bill of Materials and pricing on these requirements

      • This calculator will also provide representative estimates of service hours required for deployment, configuration, performing basic tuning and calibration, and configuring simple or complex alerts.

        • IBM SWG Lab Service can be included in quotes to deploy, configure, and tune the system, as well as integrate IVA with external systems. Several IBM qualified Business Partners can also provide these services.

        • Services to deploy video assets (IE cameras and VMS) are available from IBM Global Technology Services (GTS) and several qualified IBM Business Partners.

    • IVA licenses are priced by Resource Value Unit (RVU) based on number of cameras

      • Each camera requires a single license.

      • RVU’s are priced in tiers according to license volume.

      • Analytic profiles, retention period, and camera activity levels do not affect pricing.

    7. Sizing and Pricing

     Back to top