CB3M: Content-Based 3D Mosaics with Dynamic Pushbroom Stereo
We propose a
content-based three-dimensional
(3D) mosaic representation for long
video sequences of 3D and dynamic scenes captured by a camera on a
mobile
platform. The motion of the camera has a dominant direction of motion
(as on an
airplane or ground vehicle), but 6 degrees-of-freedom (DOF) motion is
allowed.
In the first step, a pair of generalized parallel-perspective
(pushbroom)
stereo mosaics is generated that captured both the 3D and dynamic
aspects of
the scene under the camera coverage. In the second step, a
segmentation-based
stereo matching algorithm is applied to extract parametric
representation of
the color, structure and motion of the dynamic and/or 3D objects in
urban
scenes where a lot of planar surfaces exist. Based on these results,
the
content-based 3D mosaic (CB3M) representation is created, which is a
highly
compressed visual representation for very long video sequences of
dynamic 3D
scenes. Our goal is
to acquire geo-referenced 3D mosaic
maps with motion tracking information for 3D (moving) targets using a
light
aerial vehicle flying over or a ground vehicle move across an unknown
3D environment.
Related Publications
- Z.
Zhu, H. Tang, G.
Wolberg and J. R. Layne, Content-Based
3D Mosaic Representation for Video of
Dynamic 3D Scenes. IEEE/AIPR
Workshop 2005: Multi-Modal
Imaging,
Washington DC, October 19-21, 2005
- Z.
Zhu, H. Tang, B. Shen, G. Wolberg, 3D and
Moving Target Extraction from Dynamic Pushbroom Stereo Mosaics,
IEEE
Workshop on Advanced 3D Imaging for Safety and Security, June 25, 2005,
San
Diego, CA, USA
- Z.
Zhu, A. R. Hanson, Mosaic-Based
3D
Scene Representation and Rendering, Special Session on
Interactive Representation of Still and Dynamic Scenes, the Eleventh
International Conference on Image Processing,
Genova, Italy, September 11-14, 2005,.
- Z.
Zhu, L. Zhao, J.
Lei, 3D
Measurements in Cargo Inspection with a Gamma-Ray Linear Pushbroom
Stereo System, IEEE Workshop on Advanced 3D Imaging for Safety
and
Security, June 25, 2005, San Diego, CA, USA
- Z.
Zhu, E. M.
Riseman, A. R. Hanson and H. Schultz, An Efficient
Method for Geo-Referenced
Video Mosaicing for Environmental Monitoring.
Machine Vision Applications Journal,
16(4), 2005, 203-126
- Z.
Zhu, E. M. Riseman, A. R. Hanson, Generalized
Parallel-Perspective Stereo Mosaics from
Airborne Videos, IEEE Transactions on Pattern Analysis and
Machine Intelligence, vol. 26, no. 2, Feb 2004, pp 226-237.
Collaborators:
- Professor George Wolberg, Department
of Computer Science, The City College of
New York
- Professor
Robert Haralick, Department of Computer Science, The CUNY
Graduate Center
- Dr. Jeffery R. Layne, The Air Force Research Laboratory (AFRL)
Students:
- Hao Tang, Department
of Computer Science and Department of Mathematics, The City College of
New York
- Bing
Shen,
Department
of Computer Science, The City College of
New York
- Li
Zhao,
Department
of Computer Science, The City College of
New York
Related Grants
- AFRL
RASER Program (Award
No. FA8650-05-1-1853), Dynamic Pushbroom Stereo Mosaics for 3D and
Moving
Target Extraction - A New Geometric Approach, Feb, 15, 2005 - Feb 14,
2009, PI - Z. Zhu,
Co-PIs, G. Wolberg, R. T. Haralick
- New
York Software Industrial Association (NYSIA)
Research Project, Tracking
Moving Objects via a PTZ or Moving
Camera, Aug 2004- July 2005, PI- Z. Zhu
- PSC/CUNY
36 Research Award, Dynamic
Pushbroom Stereo Mosaics for Video Surveillance over Large-Scale
Scenes, 07/01/2005-06/30/2006,
PI- Z. Zhu