Light-field content is required to provide full-parallax 3D view with dense angular resolution. However, it is very hard to directly capture such dense full-parallax view images using a camera system because it requires specialised micro-lens arrays or a heavy camera-array system. Therefore, we present an algorithm to synthesise full-parallax virtual view images using image-based rendering appropriate for light-field content generation. The proposed algorithm consists of four-directional image warping, view image blending using the nearest view image priority selection and the sum of the weighted inverse Euclidean distance, and hole filling. Experimental results show that dense full-parallax virtual view images can be generated from sparse full-parallax view images with fewer image artefacts. Finally, it is confirmed that the proposed full-parallax view synthesis algorithm can be used for light-field content generation without a dense camera array system.
In this paper, we introduce a high efficient and practical disparity estimation using hierarchical bilateral filtering for real-time view synthesis. The proposed method is based on hierarchical stereo matching with hardware-efficient bilateral filtering. Hardware-efficient bilateral filtering is different from the exact bilateral filter. The purpose of the method is to design an edge-preserving filter that can be efficiently parallelized on hardware. The proposed hierarchical bilateral filtering based disparity estimation is essentially a coarse-to-fine use of stereo matching with bilateral filtering. It works as follows: firstly, the hierarchical image pyramid are constructed; the multi-scale algorithm then starts by applying a local stereo matching to the downsampled images at the coarsest level of the hierarchy. After the local stereo matching, the estimated disparity map is refined with the bilateral filtering. And then the refined disparity map will be adaptively upsampled to the next finer level. The upsampled disparity map used as a prior of the corresponding local stereo matching at the next level, and filtered and so on. The method we propose is essentially a combination of hierarchical stereo matching and hardware-efficient bilateral filtering. As a result, visual comparison using real-world stereoscopic video clips shows that the method gives better results than one of state-of-art methods in terms of robustness and computation time.
A method of quantifying the amount of moirés in contact-type 3-D displays is described. The color moirés in the displays are induced by the periodic blocking of a part of each pixel on the panel by the boundary lines or the barrier lines consisting of the viewing zone forming optics. The method starts calculating the intensity of an image laden with moirés and that of the image with no moirés. The moirés contrast is defined as the intensity difference of the two images. The contrast values match well with those from the simulated moirés for the crossing angle range of 0° to 20°.
Two experiments were conducted to examine the visual comfort of stereoscopic images. The test video sequences
consisted of moving meteorite-like objects against a blue sky background. In the first experiment, a panel of viewers
rated stereoscopic sequences in which the objects moved back and forth in depth. The velocity of movement, disparity
(depth) range, and disparity type (i.e., depth position with respect to the screen plane: front, behind, or front/behind) of
the objects varied across sequences. In the second experiment, the same viewers rated stereoscopic test sequences in
which the target objects moved horizontally across the screen. Also in this case, the velocity, disparity magnitude, and
disparity type of the objects varied across sequences. For motion in the depth direction, the results indicate that visual
comfort is significantly influenced by the velocity, disparity range, and disparity type of the moving objects. We also
found significant interactions between velocity and disparity type and between disparity type and disparity range. For
motion across the screen in the horizontal plane, ratings of visual comfort depended on velocity and disparity
magnitude. The results also indicate a significant interaction between velocity and disparity. In general, the overall
results confirm that changes in disparity of stereoscopic images over time are a significant contributor to visual
discomfort. Interestingly, the detrimental effect of object velocity on visual comfort are manifested even when the
changes are confined within the generally accepted visual comfort zone of less than 60 arc minutes of horizontal
disparity.
In this paper a new method for the autostereoscopic display, named the Dual Layer Parallax Barrier (DLPB) method, is
introduced to overcome the limitation of the fixed viewing zone. Compared with the conventional parallax barrier
methods, the proposed DLPB method uses moving parallax barriers to make the stereoscopic view changed according to
the movement of viewer. In addition it provides seamless stereoscopic views without abrupt change of 3D depth feeling
at any eye position. We implement a prototype of the DLPB system which consists of a switchable dual-layered Twisted
Nematic Liquid Crystal Display (TN-LCD) and a head-tracker. The head tracker employs a video camera for capturing
images, and is used to calculate the angle between the eye gazing direction and the projected direction onto the display
plane. According to the head-tracker's control signal, the dual-layered TN-LCD is able to alternate the direction of
viewing zone adaptively by a solid-state analog switch. The experimental results demonstrate that the proposed
autostereoscopic display maintains seamless 3D views even when a viewer's head is moving. Moreover, its extended use
towards mobile devices such as portable multimedia player (PMP), smartphone, and cellular phone is discussed as well.
KEYWORDS: Video, Video coding, Computer programming, Receivers, 3D video streaming, 3D displays, 3D video compression, Quantization, Televisions, 3D acquisition
This paper presents a study on the stereoscopic 3D codec for non-real-time 3DTV services. Delivering stereoscopic
3DTV contents via broadcast networks can be based on either real-time or non-real-time scenarios. For a digital
television system where its bandwidth is somewhat limited to accommodate the full 3D HD quality video, a
complementary enabler is the non-real-time delivery in which 3D video component is downloaded in advance. In the
case of ATSC system, MPEG-2, H.264/AVC and any other codec can be used for stereoscopic video based on Non-
Real-Time services. In order to put in the best performance, the analysis on the stereoscopic codec, scenarios and
preliminary considerations are required. In this paper, the NRT 3D codec scenarios and preliminary considerations are
addressed to develop the prerequisites required for the analysis on NRT 3D codec. The experimentation of both
independent and inter-view coding are covered to investigate the codec combination for NRT 3D video services based on
considerations and scenarios.
In this paper, we suggested a new way to overcome a shortcoming as stereoscopic depth distortion in common
stereoscopy based on computer graphics (CG). In terms of the way, let the objective space transform as the
distorted space to make a correct perceived depth sense as if we are seeing the scaled object volume which is
well adjusted to user's stereoscopic circumstance. All parameters which related the distortion such as a focal
length, an inter-camera distance, an inner angle between camera's axes, a size of display, a viewing distance and
an eye distance can be altered to the amount of inversed distortion in the transformed objective space by the
linear relationship between the reconstructed image space and the objective space. Actually, the depth distortion
is removed after image reconstruction process with a distorted objective space. We prepared a stereo image
having a right scaled depth from -200mm to +200mm with an interval as 100mm by the display plane in an
official stereoscopic circumstance and showed it to 5 subjects. All subjects recognized and indicated the
designed depths.
KEYWORDS: Video, 3D displays, Receivers, 3D video compression, 3D image processing, 3D metrology, Video coding, 3D vision, 3D video streaming, Multimedia
In this paper, we present a development of 3D-T DMB (three-dimensional digital multimedia broadcasting) receiver for
providing 3D video and data service. First, for a 3D video service, the developed receiver is capable of decoding and
playing 3D AV contents that is encoded by simulcast encoding method and that is transmitted via T-DMB network.
Second, the developed receiver can render stereoscopic multimedia objects delivered using MPEG-4 BIFS technology
that is also employed in T-DMB. Specially, this paper introduces hardware and software architecture and its
implementation of 3D T-DMB receiver. The developed 3D T-DMB receiver has capabilities of generating stereoscopic
viewing on the glasses-free 3D mobile display, therefore we propose parameters for designing the 3D display, together
with evaluating the viewing angle and distance through both computer simulation and actual measurement. Finally, the
availability of 3D video and data service is verified using the experimental system including the implemented receiver
and a variety of service examples.
KEYWORDS: Video, 3D image processing, Receivers, Computer programming, Imaging systems, Multimedia, 3D displays, 3D video compression, Cameras, Image quality
We present a 3-D mobile broadcasting system based on a depth-image-based rendering (DIBR) technique in terrestrial digital multimedia broadcasting (T-DMB). It is well known that a 3-D mobile broadcasting service based on the DIBR technique can be one of the solutions to meet service requirements, because the required bit rates of depth images in DIBR schemes are less than additional video bit rates of other 3-D formats, while keeping good 3-D quality and guaranteeing backward compatibility with conventional broadcasting systems. We propose an implementation of a DIBR-based 3-D T-DMB system that supports real-time rendering with good image quality and realistic depth effect at the receiver, verifying that it could be perfectly applicable in mobile broadcasting. Specifically, the proposed 3-D T-DMB receiver adopts a look-up table (LUT)-based simultaneous method to accomplish the real-time implementation of DIBR algorithms, including warping, hole filling, and interleaving. Moreover, we establish the parameter values that are needed for generating the LUT based on theoretical analysis. The verification is accomplished through objective and subjective evaluations, based on simulation and real-time implementation of the system under actual service conditions.
In this paper we address the estimation of scene range for multiview 3DTV applications. Based on their similar setup
requirements, we propose a depth camera system combining the strengths of Structured Light and Active Stereo
techniques to obtain a fast depth map recovery for a larger variety of scenes and from a wider view with less occlusion.
The proposed system is composed by a stereo rig and a projector. A stripe-based one shot pattern is projected into the
target scene. Using an efficient decoding technique, reliable correspondences can be found between each camera and the
projected pattern. Decoded areas common to both cameras are triangulated and used to calibrate the projector on the fly
in order to estimate depth in decoded areas only visible in one camera. Correspondences in undecoded areas are
estimated via a stereo matching procedure. The different estimated data is then combined in a single depth map. We
introduce a 3 projection method that improves the structured light identification in strong illumination conditions and
more robust to color objects surfaces of the scene. We demonstrate the efficacy of the integration method experimental
results with special emphasis on its performance in the context of the development and contents creation for depth
image-based representation (DIBR) 3DTV.
KEYWORDS: 3D displays, Video, 3D video streaming, Receivers, 3D video compression, LCDs, Video compression, Multimedia, 3D acquisition, 3D image processing
The mobile broadcasting services getting deployed around the world are being evolved to new services. One of the
notable services is mobile stereoscopic service, called 3D, which can provide users with a stereoscopic view of TV
contents while on the move. This paper presents the design and the implementation of 3D DMB receiver enabling the
reception of mobile 3DTV and BIFS based interactive data services. Main concepts of 3D DMB receiver are the nonglasses
viewing experience, backward and forward compatibility and high transmission efficiency. We demonstrate the
main concepts and features of our 3D DMB receiver and outline the implementation result along with the future works.
KEYWORDS: Video, 3D displays, 3D image processing, 3D video streaming, 3D acquisition, Multimedia, Stereoscopic cameras, Video compression, Visualization, 3D video compression
This paper presents a stereoscopic contents authoring system that covers the creation and editing of stereoscopic
multimedia contents for the 3D DMB (Digital Multimedia Broadcasting) data services. The main concept of 3D DMB
data service is that, instead of full 3D video, partial stereoscopic objects (stereoscopic JPEG, PNG and MNG) are
stereoscopically displayed on the 2D background video plane. In order to provide stereoscopic objects, we design and
implement a 3D DMB content authoring system which provides the convenient and straightforward contents creation
and editing functionalities. For the creation of stereoscopic contents, we mainly focused on two methods: CG (Computer
Graphics) based creation and real image based creation. In the CG based creation scenario where the generated CG data
from the conventional MAYA or 3DS MAX tool is rendered to generate the stereoscopic images by applying the suitable
disparity and camera parameters, we use X-file for the direct conversion to stereoscopic objects, so called 3D DMB
objects. In the case of real image based creation, the chroma-key method is applied to real video sequences to acquire the
alpha-mapped images which are in turn directly converted to stereoscopic objects. The stereoscopic content editing
module includes the timeline editor for both the stereoscopic video and stereoscopic objects. For the verification of
created stereoscopic contents, we implemented the content verification module to verify and modify the contents by
adjusting the disparity. The proposed system will leverage the power of stereoscopic contents creation for mobile 3D
data service especially targeted for T-DMB with the capabilities of CG and real image based contents creation, timeline
editing and content verification.
Display adaptation is one of the key factors for the success of visual services. For two-dimensional (2D) imaging, the
display adaptation is generally accomplished by 2D image re-sampling (i.e., up-/down-sampling). However, when it
comes to stereoscopic three-dimensional (S3D) images, 2D re-sampling methods are inadequate because additional
consideration on the third dimension of depth is not incorporated. In this paper, we analyze S3D image resizing from two
aspects: geometrical deformation and frequency-domain aliasing. A number of S3D displays are available in the market
and they have various screen dimensions. As we have more varieties of the displays, efficient S3D image resizing is
becoming more emphasized. We present the conditions for equi-proportional S3D image resizing using the model of the
binocular vision; and propose an anti-aliasing filter for stereoscopic 3D image up/down-sampling.
In this paper, we present the depth value accuracy requirements for novel view synthesis in 3DTV and our approach for
depth camera based on hybrid method of structured light and active stereo matching. We first review the background on
the use of multiview video plus depth(MVD) representation for future 3DTV system, depth acquisition methods and
tools for real-time and non-real-time dense depth map acquisition in terms of performance and limitations. Then we use a
simple analysis model for novel view synthesis based on DIBR to draw depth value accuracy requirements. Finally we
present our approach on depth camera development along with future directions and huddles in developing real-time
depth camera usable for 3DTV content production.
KEYWORDS: Video, Receivers, 3D image processing, Computer programming, Image quality, 3D displays, Video coding, 3D video compression, Visualization, 3D video streaming
This paper presents a 3D (three dimensional) mobile broadcasting service based on depth-image-based rendering (DIBR)
technique in terrestrial digital multimedia broadcasting (T-DMB). In designing and developing a 3D visual service based
on mobile broadcasting system, we must consider system requirements such as the minimization of additional bit-rates
for 3D depth information due to the limitation of transmission channel bandwidth, the assurance of backward
compatibility with existing T-DMB, and the maximization of 3D effect while reducing eye fatigue. Therefore, the 3D
mobile broadcasting service based on DIBR technique can be one of the solutions to meet such requirements because the
allocated bit-rates of depth image with DIBR scheme is less than additional video bit-rates of another 3D format, while
keeping 3D quality and guaranteeing backward-compatibility with T-DMB. In this paper, we introduce an
implementation of DIBR-based 3D T-DMB system that supports the real-time rendering with good image quality and
depth effect at the receiver, verifying that it could be available in the mobile broadcasting. The verification is achieved
through objective and subjective evaluation, based on the simulation and implementation of the system. Finally, we will
confirm that DIBR-based 3D mobile broadcasting service would be commercialized in near future.
In this paper, we present motivation, system concept, and implementation details of stereoscopic 3D visual services on
T-DMB. We have developed two types of 3D visual service : one is '3D video service', which provides 3D depth feeling
for a video program by sending left and right view video streams, and the other is '3D data service', which provides
presentation of 3D objects overlaid on top of 2D video program. We have developed several highly efficient and
sophisticated transmission schemes for the delivery of 3D visual data in order to meet the system requirements such as
(1) minimization of bitrate overhead to comply with the strict constraint of T-DMB channel bandwidth; (2) backward
and forward compatibility with existing T-DMB; (3) maximize the eye-catching effect of 3D visual representation while
reducing eye fatigue. We found that, in contrast to conventional way of providing a stereo version of a program as a
whole, the proposed scheme can lead to variety of efficient and effective 3D visual services which can be adapted to
many business models.
This paper presents a practical way of adjusting global disparity with binocular energy model and image partition
for given stereoscopic images. Previous method estimated a single global disparity and then used it directly to
control the convergence angle between cameras. But, the previous method might cause local disparities to be
excessive in some regions since a single global disparity has been considered. Hence in this paper, we consider how
to mitigate the excessive disparities in some regions. To begin with, we partition the stereoscopic images into 4
sub-images, respectively, and then calculate multiple local disparities for a pair of partitioned images. Secondly,
we define a new disparity by the average value of the local disparities. Lastly, the newly defined disparity is used
to adjust the global disparity of the given stereoscopic images. Through experimental results, we show that the
proposed method can prevent the local disparities from being excessive in some regions.
We present a depth map-based disparity estimation algorithm using multi-view and depth camera system. When many objects are arranged in the 3D space with a long depth range, the disparity search range should be large enough in order to find all correspondences. In this case, traditional disparity estimation algorithms that use the fixed disparity search range often produce mismatches if there are pixels that have similar color distribution and similar textures along the epipolar line. In order to reduce the probability of mismatch and save computation time for the disparity estimation, we propose a novel depth map-based disparity estimation algorithm that uses a depth map captured by the depth camera for setting the disparity search range adaptively as well as for setting the mid-point of the disparity search range. The proposed algorithm first converts the depth map into disparities for the stereo image pair to be matched using calibrated camera parameters. Next, we set the disparity search range for each pixel based the converted disparity. Finally, we estimate a disparity for each pixel between stereo images. Simulation results with various test data sets demonstrated that the proposed algorithm has better performance in terms of the smoothness, global quality and computation time compared to the other algorithms.
A method of producing depth maps for depth-image-based rendering (DIBR) of stereoscopic views is proposed and tested. The method is based on depth-from-defocus techniques, utilizing two original images, one with the camera focused at a near point and the other with it focused at a far point in the captured scene to produce depth maps from blur at edge locations. It is assumed that the level of blur at an edge reflects the distance it is from the focused distance. For each image, estimates of the level of blur edges at local regions are obtained by determining the optimal scale for edge
detection based on a luminance gradient. An Edge-Depth map is then obtained by evaluating differences in blur for corresponding regions in the two images. This is followed by an additional process in which regions in the Edge-Depth map that have no depth values are filled to produce a Filled-Depth map. A group of viewers assessed the depth quality of a representative set of stereoscopic images that were produced by DIBR using the two types of depth maps. It was
found that the stereoscopic images generated with the Filled-Depth and the Edge-Depth maps produced depth quality ratings that were higher than those produced by their monoscopic, two-dimensional counterparts. Images rendered using the Filled-Depth maps, but not the Edge-Depth maps, produced ratings of depth quality that were equal to those produced with factual, full depth maps. A hypothesis as to how the proposed method might be improved is discussed.
It is well known that some viewers experience visual discomfort when looking at stereoscopic displays. One of the factors that can give rise to visual discomfort is the presence of large horizontal disparities. The relationship between excessive horizontal disparity and visual comfort has been well documented for the case in which disparity magnitude does not change across space and time, e.g. for objects in still images. Much less is known about the case in which
disparity magnitude varies over time, e.g., objects moving in depth at some velocity. In this study, we investigated the relationship between binocular disparity, object motion and visual comfort using computer-generated stereoscopic video sequences. Specifically, viewers were asked to rate the visual comfort of stereoscopic sequences that had objects moving periodically back and forth in depth. These sequences varied with respect to the number, size, position in depth, and velocity of movement of the objects in the scene. The results indicate that change in disparity magnitude over time might be more important in determining visual comfort than the absolute magnitude of the disparity per se. The results also suggest that rapid switches between crossed and uncrossed disparities might negatively affect visual comfort.
KEYWORDS: Computer programming, 3D modeling, Solid modeling, Visualization, Computer simulations, Iterated function systems, Surgery, Visual compression, Digital electronics, Data communications
In MPEG-4, 3D mesh coding (3DMC) achieves 40:1 to 50:1 compression ratio over 3-D meshes (in VRML IndexedFaceSet representation) without noticeable visual degradation. This substantial gain comes not for free: it changes the vertex and face permutation order of the original 3-D mesh model. This vertex and face permutation order change may cause a serious problem for animation, editing operation, and special effects, where the original permutation order information is critical not only to the mesh representation, but also to the related tools. To fix this problem, we need to transmit the vertex and face permutation order information additionally. This additional transmission causes the unexpected increase of the bitstream size. In this paper, we proposed a novel vertex and face permutation order compression algorithm to address the vertex and face permutation order change by the 3DMC encoding with the minimal increase of side information. Our proposed vertex and face permutation order coding method is based on the adaptive probability model, which makes allocating one fewer bits codeword to each vertex and face permutation order in every distinguishable unit as encoding proceeds. Additionally to the adaptive probability model, we further increased the coding efficiency of the proposed method by representing and encoding each vertex and face permutation order per connected component (CC). Simulation results demonstrated that the proposed algorithm can encode the vertex and face permutation order losslessly while making up to 12% bit-saving compared with the logarithmic representation based on the fixed probability model.
We propose a software-based minimum-time vergence control scheme
(MTVCS) for a parallel-axis stereoscopic camera (PASC). First, a
global horizontal disparity is estimated by using modified
binocular energy models and transformed stereoscopic images via
Radon Transform with a specified angle parameter. Second, with the
estimated global disparity, the actual disparity command is
derived through a nonlinear function such that the resulting
horizontal disparity is equal to the command exactly with the
control in a fastest time interval. Through experimental results,
we will show that the proposed MTVCS achieves better tracking and
regulating performances than those of the previous scheme.
Among the 3D stereoscopic cameras to acquire the stereo views, the parallel-axis stereo camera is considered as the simplest one of binocular stereo cameras. However, it is not able to control vergence since its left and right imaging sensors are fixed. In order to overcome such limitations of the stereoscopic cameras, we propose a parallel-axis stereoscopic camera that has functions for the vergence control and video multiplexing simultaneously, which can be implemented by simple and real-time processing without image deterioration. In this paper, we simulate the effects of the vergence control according to the proposed methods, which is accomplished by the over-sampling at ADC and extracted disparity with help of multiplexing function. It is confirm that the processed stereoscopic images by the proposed PASC are very comfortable for viewing on the 3D display within a limited disparity range.
This paper introduces 3D HDTV relay broadcasting experiments of 2002 FIFA World Cup Korea/Japan using a terrestrial and satellite network. We have developed 3D HDTV cameras, 3D HDTV video multiplexer/demultiplexer, a 3D HDTV receiver, and a 3D HDTV OB van for field productions. By using a terrestrial and satellite network, we distributed a compressed 3D HDTV signal to predetermined demonstration venues which are approved by host broadcast services (HBS), KirchMedia, and FIFA. In this case, we transmitted a 40Mbps MPEG-2 transport stream (DVB-ASI) over a DS-3 network specified in ITU-T Rec. G.703. The video/audio compression formats are MPEG-2 main-profile, high-level and Dolby Digital AC-3 respectively. Then at venues, the recovered left and right images by the 3D HDTV receiver are displayed on a screen with polarized beam projectors.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.