Available with Image Analyst license.
Full Motion Video (FMV) in the ArcGIS Image Analyst extension provides capabilities for playing and geospatial analysis of video data that is FMV-compliant. FMV-compliant refers to the combination of a video stream and associated metadata into one video file, which makes the video geospatially aware. The sensor systems collect camera pointing information, platform position and attitude, and other data, and encode it into the video stream so that each video frame is associated with geopositional information.
This geospatially enabled video data, along with the computational functionality of ArcGIS Pro, provides you with the ability to view and manipulate the video while being fully aware of the sensor dynamics and field of view (FOV), and display this information in the map view. It also allows you to analyze and edit feature data in either the video view or the map view, providing telestration capability.
FMV uses the metadata to seamlessly convert coordinates between the video image space and map space, similar to the way that the image coordinate system in Image Space Analysis transforms still imagery. This conversion provides the foundation for interpreting video data in the full context of all other geospatial data and information within your GIS. For example, you can view the video frame footprint, frame center, and position of the imaging platform on the map view as the video plays, together with your GIS layers such as buildings with IDs, geofences, and other pertinent information.
FMV data can be analyzed in real time or forensically, immediately after a collection. It is well suited for situation awareness since it is often the most current imagery available for a location. For example, if you are engaged in damage assessment after a natural disaster, you can use FMV to analyze the latest video data collected from a drone, together with existing GIS data layers. Since the video footprint is visible on the map, you know exactly what buildings and infrastructure are visible in the video, and you can assess their condition, mark ground features in the video and map, bookmark their locations, and describe them in notes. FMV is designed to quickly assess, analyze, and disseminate actionable information for timely decision support.
FMV features and benefits
FMV takes advantage of crucial metadata, provides visual and analytical processing tools, and provides ArcGIS Pro platform capabilities to support project and mission-critical workflows.
- FMV is fully integrated into ArcGIS Pro, which takes advantage of system architecture, data models, tools and capabilities, and sharing across the ArcGIS platform.
- View and analyze live-stream videos and archived videos.
- The video player can be moved anywhere on your computer display, resized, minimized, and closed.
- The video player is linked to the map display, enabling the following:
- Display of the video footprint, sensor location, and field of view on the map.
- Any information collected in the video player is projected and displayed on the map, together with your existing GIS data.
- Update the map to zoom to the video frame and follow videos across the map.
- Several videos can be opened and played at the same time. Each video, and its associated information, is identified by a unique color when displayed on the map.
- Use intuitive digital video recorder (DVR) controls, image and video clip capture, and analysis tools.
- Display metadata in real time.
- Create and manage bookmarks.
- Mark locations and phenomena of interest.
- Display is accelerated by GPU.
These functional capabilities form the building blocks for workflows that integrate rich spatial context into the video analysis process.
Overview of Full Motion Video
A primary use of FMV is for decision support in operational environments. Understanding who uses FMV and how it is collected and used provides important context for evaluating the functionality of FMV.
Industries that use FMV
FMV is useful for monitoring remote, inaccessible, or dangerous locations. The types of organizations that utilize FMV include the following:
- Public safety and emergency management
- Defense
- Oil companies
- Local and federal governments
- Border patrol
- Utilities
- Natural resources professionals
There are many types of applications for FMV:
- Emergency management and public safety
- Situation awareness
- Damage assessment
- Response
- Mitigation
- Security monitoring
- Asset management
- Corridor mapping, monitoring, and management
- Utility transmission, such as electric, gas, water, and other conveyance
- Transportation, such as roads, bridges, rail, highways, and air-land-sea ports
- Communication, such as cell towers and infrastructure
- Hydrology, such as natural and human-made infrastructure
- Riparian, such as stream buffers per reach, habitat, diversity mapping, and links
- Defense
- Intelligence, surveillance, and reconnaissance
- Mission support
- Counterintelligence
- Monitoring infrastructure
- Inventory
- New development
- Verifying as-built to planned
- Compliance and safety
How FMV data is collected
The FMV application in ArcGIS Pro utilizes data collected from a variety of remote sensing platforms. The video data and metadata are collected concurrently, and the metadata is encoded in the video file either in real time onboard the sensor platform (MISB compliant), or later in a processing step called multiplexing.
FMV data is collected from a variety of sensor platforms:
- Unmanned aerial vehicles (UAVs, UASs, RPVs, drones)
- Fixed-wing and helicopter aerial platforms
- Orbital spaceborne sensors
- Vehicle-mounted cameras
- Handheld mobile devices and cameras
- Stationary devices for persistent surveillance
FMV supports 12 video formats:
| Description | Extension |
|---|---|
MPEG-2 Transport Stream | .ts |
MPEG-2 Program Stream | .ps |
MPEG File | .mpg |
MPEG-2 File | .mpg2 |
MPEG-2 File | .mp2 |
MPEG File | .mpeg |
VLC (mpeg2) | .mpeg2 |
MPEG-4 Movie | .mp4 |
MPEG-4 File | .mpg4 |
H264 Video File | .h264 |
VLC Media File (mpeg4) | .mpeg4 |
VLC Media File (vob) | .vob |
FMV capabilities
FMV is designed to support a variety of diverse operational environments, from time-critical situation awareness and emergency response utilizing live-streaming video, to analysis of hourly or daily archived videos for monitoring, to forensic analysis of historical archived data. Each type of application requires intuitive visualization and analysis tools to quickly and easily identify and record features or situations of interest, and to disseminate extracted information to decision makers and stakeholders. FMV capabilities for decision support are categorized into essential functionality below.
Video player
The video player is called a digital video recorder (DVR) and has a familiar look and feel to a common DVR. The video player includes standard video controls such as play, fast forward, rewind, step forward, step backward, jump to the beginning, or jump to the end of the video. You can zoom in to and roam a video while it is in play mode or in pause mode. The size and position of the zoomed view within the video frame are indicated in the overview window.
When a video is loaded in the display, the video file is listed in the Contents pane, and the View tab appears on the main ribbon. The View tab contains tools for navigating the video, adding and managing graphics, and viewing video metadata. These tools are also available on the DVR for ease of operation.
Tool operations include zooming the associated map view to display the full video frame on the ground, displaying the sensor ground track and field of view of the video frame on the ground, and zooming and panning the map to follow the video on the ground. These tools allow you to have geographical context when working with your video data.
Add graphics
You can collect ground features of interest that are visible in the video player by marking them with a feature, which is also displayed on the map. Conversely, you can mark ground features in the map view, and the features are also displayed in the video player. These points can be saved as a feature class and fully utilized later for further analysis.
Bookmarks
Creating video bookmarks is an important function for recording phenomena and features of interest when analyzing a video. Video bookmarks can be collected in the different modes of playing a video, such as play, pause, fast-forward, and rewind. Bookmarks can be described in the Bookmark pane that opens when you collect a video bookmark. Bookmarks are collected and managed in the Bookmarks pane available on the ArcGIS Pro Map tab, in the Navigate group.
Add metadata to videos
Only videos that contain essential metadata can be used in FMV. Professional-grade airborne video data collection systems generally collect the required metadata and encode it into the video file in real time. This data is readily input directly into the FMV application, either in live-streaming mode or from an archived file.
Consumer-grade video collection systems often produce separate video data and metadata files that must be combined into a single FMV-compliant video file. This process is performed in the software and is referred to as multiplexing. FMV provides a tool named Video Multiplexer that encodes proper metadata at the proper location of the video file to produce a single FMV-compliant video file. Each video and metadata file uses time stamps to synchronize the encoding of the proper metadata in the proper location in the video file.
The metadata is generated from appropriate sensors, such as GPS for x,y,z position, altimeter, and inertial measurement unit (IMU), or other data sources for camera orientation. The metadata file must be in the comma-separated values (CSV) format.
The FMV metadata is used to compute the flight path of the video sensor, the video image frame center, and the footprint on the ground of the video image frames. FMV also supports the Motion Imagery Standards Board (MISB) metadata specifications, composed of 87 parameters. All the MISB parameters that are provided will be encoded into the final FMV-compliant video, including all 87 parameters, or a subset.
One set of FMV parameters includes the map coordinates of the four corners of the video image frame projected to the ground. If these four corner map coordinates are provided, they will be used. Otherwise, the tool will compute the video footprint from a subset of required parameters.
To compute and display the relative corner points of the video frame footprint as a frame outline on the map, FMV needs the 12 essential metadata fields listed below:
- Unix Time Stamp
- Sensor Latitude
- Sensor Longitude
- Sensor Altitude
- Platform Heading
- Platform Pitch
- Platform Roll
- Sensor Relative Roll
- Sensor Relative Elevation
- Sensor Relative Azimuth
- Horizontal Field of View
- Vertical Field of View
When the metadata is complete and accurate, the tool will calculate the video frame corners, and thus the size, shape, and position of the video frame outline, which may then be displayed on a map. These 12 parameters comprise the minimum metadata required to compute the transform between video and map, to display the video footprint on the map, and to enable other functionality such as digitizing and marking on the video and the map.
The performance of the resulting multiplexed video file depends on the type and quality of the data contained in the metadata file, and how accurately the video data and metadata files are synchronized. If the metadata is not accurate or contains anomalies, these discrepancies will be encoded in the video and displayed on the map when the video is played. If your metadata.csv file only contains UNIX Time Stamp, Sensor Latitude, and Sensor Longitude fields, the location of the sensor will be displayed on the map, but the footprint of the video frames cannot be displayed, and some functionality such as digitizing features and measuring distance within the video will not be supported.
Your metadata can be input into the FMV metadata template, FMV_Multiplexer_Field_Mapping_Template.csv, obtained from C:\Program Files\ArcGIS\Pro\Resources\FullMotionVideo. This template contains the 12 essential metadata fields needed to compute the video frame footprint, and additional fields included in the MISB specification. If your metadata field names don’t match those needed for FMV, they can be matched to the FMV field names in the FMV_Multiplexer_Field_Mapping_Template.csv template.
Ideally, the video data and metadata are time synchronous. If the time stamp linking the video and metadata are not accurately synchronized, the video footprint and sensor location on the map will be offset from the view in the video player. If the time shift is observable and consistent, the multiplexer can adjust the timing of the metadata to match the video. The time shifts are applied by creating and inputting an optional CSV file into the Video Multiplexer tool that identifies the offset location in the video and its associated time discrepancy. Use the FMV_Multiplexer_TimeShift_Template.csv template obtained from C:\Program Files\ArcGIS\Pro\Resources\FullMotionVideo to adjust the offsets. The template contains columns labeled elapsed time, which is the location in the video where the time shift occurs, and time shift, which is the amount of time offset. If the time shift between your video and metadata is inconsistent, you can list multiple positions in the video with the associated time shift in the template. This instructs the multiplexer where to place the appropriate metadata relative to the video timing.
Summary
The geospatially enabled characteristics of FMV are well suited for real-time or forensic analysis and situation awareness applications. The capability to project and display the video frame footprint and flight path on the map gives the video important context, and allows bidirectional collection of features in the video player and the map.
These capabilities, together with annotated video bookmarks, allow an analyst to identify specific video frames or segments for further analysis and easily share this information with other stakeholders. By integrating FMV-compliant video with full GIS functionality, you have the ability to synthesize important contextual information to support informed decision making in operational environments.