Exact wave theories of specular reflectance from rough surfaces are computationally intractable thus
motivating the practical need for geometric reflectance models which treat only the geometric ray
nature of light reflection. The cornerstone of geometric reflectance modeling from rough surfaces in
computer vision and computer graphics over the past two decades has been the Torrance-Sparrow
model. This model has worked well as an intuitive description of rough surfaces as a collection of planar
Fresnel reflectors called microfacets together with the concept of geometric attenuation for light which is
obscured during reflection under an assumed rough surface geometry. Experimental data and analysis
show that the current conceptualization of how specularly reflected light rays geometrically interact with
rough surfaces needs to be seriously revised. The Torrance-Sparrow model while in qualitative agreement
with specular reflection from rough surfaces is seen to be quantitatively inaccurate. Furthermore there
are conceptual inconsistencies upon which derivation of this reflectance model is based. We show how
significant quantitative improvement can be achieved for a geometric reflectance model by making some
fundamental revisions to notions of microfacet probability distributions and geometric attenuation.
Work is currently undergoing, to relate physical surface reconstructions from Atomic Force Microscope
data to reflectance data from these same surfaces.
Image fusion of complementary broadband spectral modalities has been extensively studied for providing performance enhancements to various military applications. With the growing availability of COTS and customized video cameras that image in VIS-NIR, SWIR, MWIR and LWIR, there is a corresponding increase in the practical exploitation of different combinations of fusion between any of these respective spectrums. Equinox Corporation has been developing a unique line of products around the concept of a single unified video image fusion device that can centrally interface with a variety of input cameras and output displays, together with a suite of algorithms that support image fusion across the diversity of possible combinations of these imaging modalities. These devices are small in size, lightweight and have power consumption in the vicinity of 1.5 Watts making them easy to integrate into portable systems.
This paper presents a study of face recognition performance as a function of light level using intensified near infrared imagery in conjunction with thermal infrared imagery. Intensification technology is the most prevalent in both civilian and
military night vision equipment, and provides enough enhancement for human operators to perform standard tasks under extremely low-light conditions. We describe a comprehensive data collection effort undertaken by the authors to image subjects under carefully controlled illumination and quantify the performance of standard face recognition algorithms on visible, intensified and thermal imagery as a function of light level. Performance comparisons for automatic face recognition are reported using the standardized implementations from the CSU Face Identification Evaluation System, as well as Equinox own algorithms. The results contained in this paper should constitute the initial step for analysis and
deployment of face recognition systems designed to work in low-light level conditions.
KEYWORDS: Facial recognition systems, Near infrared, Cameras, Visible radiation, Detection and tracking algorithms, Light sources and illumination, Microchannel plates, Sensors, System identification, Video
This paper presents a systematic study of face recognition performance as a function of light level using intensified near infrared imagery. This technology is the most prevalent in both civilian and military night vision equipment, and provides enough intensification for human operators to perform standard tasks under extremely low-light conditions. We describe a comprehensive data collection effort undertaken by the authors to image subjects under carefully controlled illumination and quantify the performance of standard face recognition algorithms on visible and intensified imagery as a function of light level. Performance comparisons for automatic face recognition are reported using the standardized implementations from the CSU Face Identification Evaluation System. The results contained in this paper should constitute the initial step for analysis and deployment of face recognition systems designed to work in low-light level conditions.
Equinox Corporation has developed two new video board products for real-time image fusion of visible (or intensified visible/near-infrared) and thermal (emissive) infrared video. These products can provide unique capabilities to the dismounted soldier, maritime/naval operations and Unmanned Aerial Vehicles (UAVs) with low-power, lightweight, compact and inexpensive FPGA video fusion hardware. For several years Equinox Corporation has been studying and developing image fusion methodologies using the complementary modalities of the visible and thermal infrared wavebands including applications to face recognition, tracking, sensor development and fused image visualization. The video board products incorporate Equinox's proprietary image fusion algorithms into an FPGA architecture with embedded programmable capability. Currently included are (1) user interactive image fusion algorithms that go significantly beyond standard "A+B" fusion providing an intuitive color visualization invariant to distracting illumination changes, (2) generalized image co-registration to compensate for parallax, scale and rotation differences between visible/intensified and thermal IR, as well as non-linear optical and display distortion, and (3) automatic gain control (AGC) for dynamic range adaptation.
Recent research has demonstrated distinct advantages using thermal infrared imaging for improving face recognition performance. While conventional video cameras sense reflected light, thermal infrared cameras primarily measure emitted radiation from objects at just above room temperature (e.g., faces). Visible and thermal infrared image data collections of frontal views of faces have been on-going at NIST for over two years producing the most comprehensive database known to involve thermal infrared imagery of human faces. Rigorous experimentation with this database has revealed consistently superior recognition performance of algorithms when applied to thermal infrared particularly under variable illumination conditions. An end-to-end face recognition system incorporating simultaneous coregistered thermal infrared and visible has been developed and tested both indoors and outdoors with good performance.
Over the last decade there has been study of separating ground objects from background using multispectral imagery in the reflective spectrum from 400-2500nm. In this paper we explore using two broadband spectral modalities; visible and ShortWave InfraRed (SWIR),
for detection of minelike objects, obstacles and camouflage. Whereas multispectral imagery is sensed over multiple narrowband wavelengths, sensing over two broadband spectrums has the advantage of increased signal rsulting from integrated energy over larger spectrums. Preliminary results presented here show that very basic image fusion processing applied to visible and SWIR imagery produces reasonable illumination invariant segmentation of objects against background. This suggests the use of a simplified compact camera architecture using visible and SWIR sensing focal plane arrays for performing detection of mines and other important objects of interest.
Research has shown that naturally occurring light outdoors and underwater is partially linearly polarized. The polarized components can be combined to form an image that describes the polarization of the light in the scene. This image is known as the degree of linear polarization (DOLP) image or partial polarization image. These naturally occurring polarization signatures can provide a diver or an unmanned underwater vehicle (UUV) with more information to detect, classify, and identify threats such as obstacles and/or mines in the shallow water environment. The SHallow water Real-time IMaging Polarimeter (SHRIMP), recently developed under sponsorship of Dr. Tom Swean at the Office of Naval Research (Code 321OE), can measure underwater partial polarization imagery. This sensor is a passive, three-channel device that simultaneously measures the three components of the Stokes vector needed to determine the partial linear polarization of the scene. The testing of this sensor has been completed and the data has been analyzed. This paper presents performance results from the field-testing and quantifies the gain provided by the partial polarization signature of targets in the Very Shallow Water (VSW) and Surf Zone (SZ) regions.
A key issue for face recognition has been accurate identification under variable illumination conditions. Conventional video cameras sense reflected light so that image gray values are a product of both intrinsic skin reflectivity and external incident illumination, obfuscating intrinsic reflectivity of skin. It has been qualitatively observed that thermal imagery of human faces is invariant to changes in indoor and outdoor illumination, although there never has been any rigorous quantitative analysis to confirm this assertion published in the open literature. Given the significant potential improvement to the performance of face recognition algorithms using thermal IR imagery, it is important ot quantify observed illumination invariance and to establish a solid physical basis for this phenomenon. Image measurements are presented from two of the primarily used spectral regions for thermal IR; 3-5 micron MidWave IR and the 8-14 micron LWIR. All image measurements are made with respect to precise blackbody ground-truth. Radiometric calibration procedures for two different kinds of thermal IR sensors are presented and are emphasized as being an integral part to data collection protocols and face recognition algorithms.
Existing polarization-based image understanding techniques use information only from reflected light. Apart form incandescent bodies thermally emitted light radiation from elements of a scene in the visible spectrum is insignificant. However, at longer wavelengths such as in the IR thermal emission is typically quite prevalent form a number of scene elements of interest. FLIR imagery of both indoor and outdoor scenes reveals that many objects thermally emit a significant amount of radiation. Polarization from thermally emitting objects has been observed as long as 170 years ago from incandescent objects but since then there have only ben a limited number of empirical investigations into this phenomenon. This paper present a comprehensive model for explaining polarization of thermal emission from both rough and smooth surfaces, in agreement with empirical data, that can significantly enhance the image understanding of FLIR imagery. In particular it is possible to discern metal from dielectric materials under certain conditions, and from an accurate model for thermally emitted polarization it is possible to predictively model polarization signatures form CAD models of importance to automatic target recognition.
Through the use of novel imaging devices called Polarization Cameras polarization vision can be attained in underwater environments. Whereas human vision is oblivious to components of light polarization, polarization parameters of light provide an important visual extension to intensity and color. A physical state of polarization can be visualized directly in human terms as a particular hue and saturation, and this paper utilizes such a scheme presenting image of ordinary scenes as never seen before by humans in the domain of Polarization Vision. Metaphorically, humans are 'color blind' with respect to the perception of polarization and even though this does not appear to inhibit human visual performance, we show how polarization vision is a sensory augmentation that can potentially enhance underwater vision for a diver.
We present a new formalism for the treatment and understanding of multispectral imags and multisensor fusion based on first order contrast information. Although little attention has been paid to the utility of multispectral contrast, we develop a theory for multispectral contrast that enables us to produce an optimal grayscale visualization of the first order contrast of an image with an arbitrary number of bands. In particular, we consider multiple registered visualization of multi-modal medical imaging. We demonstrate how our methodology can reveal significantly more interpretive information to a radiologist or image analyst, who can use it in a number of image understanding algorithms. Existing grayscale visualization strategies are reviewed and a discussion is given as to why our algorithm performs better. A variety of experimental results from medical imagin and remotely sensed data are presented.
A new method is introduced for the registration of MRI and CT scans of the head, based on the first order geometry of the images. Registration is accomplished by optimal alignment of gradient vector fields between respective MRI and CT images. We show that the summation of the squared inner products of gradient vectors between images is well-behaved, having a strongly peaked maximum when images are exactly registered. This supports our premise that both magnitude and orientation of edge information are important features for image registration. A number of experimental results are presented demonstrating the accuracy of our performance.
An invariant related to Gaussian curvature at an object point is developed based upon the covariance matrix of photometric values within a local neighborhood about the point. We employ three illumination conditions, two of which are completely unknown. We never need to explicitly know the surface normal at a point. The determinant of the covariance matrix of the intensity three-tuples in the local neighborhood of an object point is shown to be invariant with respect to rotation and translation. A way of combing these determinant to form a signature distribution is formulated that is rotation, translation, and scale invariant. This signature is shown to be invariant over large ranges of poses of the same objects, while being significantly different between distinctly shaped objects. A new object recognition methodology is proposed by compiling signatures for only a few viewpoints of a given object.
A robust and accurate polarization phase-based technique for material classification is presented. The novelty of this technique is three-fold in (1) it theoretical development, (2) its application, and (3) its experimental implementation. The concept of phase of polarization of a light wave is introduced to computer vision for discrimination between materials according to their intrinsic electrical conductivity, such as distinguishing conducting metals, and poorly- conducting dielectrics. Previous work has used intensity, color and polarization component ratios. This new method is based on the physical principle that metals retard orthogonal components of light upon reflection while dielectrics do not. This method has significant complementary advantages with respect to existing techniques, is computationally efficient, and can be easily implemented with existing imaging technology. Experiments for real circuit board inspection, non-conductive and conductive glass, and outdoor object recognition have been performed to demonstrate its accuracy and potential capabilities.
Many animals, both marine and terrestrial, are sensitive to the orientation of the e-vector of partially linearly polarized light (PLPL). This sensitivity is used for navigation, spatial orientation, and detection of large bodies of water. However, it is not clear what other information animals may receive from polarized light. Natural light fields, both in the sky and underwater, are known to be partially polarized. Additionally, natural objects reflect light that is polarized at specific orientations. Sensors capable of measuring the characteristics of PLPL, namely partial polarization and orientation, throughout an image are not yet available. By placing 2 twisted nematic liquid crystals (TNLCs) and a fixed polarizing filter in series in front of a video camera, and by controlling the angles of rotation of the orientation of polarization produced by the TNLCs, we are able to fully analyze PLPL throughout a full image on a single pixel basis. As a recording device we use a small camcorder. The sensor can be operated autonomously, with the images analyzed at a later stage, or it can be connected (in a future phase) via a frame grabber to a personal computer which analyzes the information online. The analyzed image can be presented as a false color image, where hue represents orientation of polarization and saturation represents partial polarization. Field measurements confirm that PLPL is a characteristic distributed both under water and on land. Marine background light is strongly horizontally polarized. Light reflected from leaves is polarized mainly according to their spatial orientation. Differences between PLPL reflected from objects or animals and their background can be used to enhance contrast and break color camouflage. Our sensor presents a new approach for answering questions related to the ecology of vision and is a new tool for remote sensing.
KEYWORDS: 3D modeling, Data modeling, Lung, 3D metrology, Nickel, 3D image processing, Computer simulations, Image segmentation, Optimization (mathematics), Algorithm development
Accurate physiological measurements of the parameters like branching angles, branch lengths, and diameters of bronchial tree structures help in addressing the mechanistic and diagnostic questions related to obstructive lung disease. In order to facilitate these measurements, bronchial trees are reduced to a central axis tree. The approach we take employs first setting up a theoretical computerized tree structure, and then applying a 3D analysis to obtain the required anatomical data. A stick model was set up in 3D, with segment endpoints and diameters as input parameters to the model generator. By fixing the direction in which the slices are taken, a stack of 2D images of the generated 3D tree model is obtained, thereby simulating bronchial data sets. We design a two pass algorithm to compute the central axis tree and apply it on our models. In the first pass, the topological tree T is obtained by implementing a top-down seeded region growing algorithm of the 3D tree model. In the second pass, T is used to region growth along the axes of the branches. As the 3D tree model is traversed bottom-up, the centroid values of the cross sections of the branches are stored in the corresponding branch of T. At each bifurcation, the branch point and the three direction vectors along the branches are computed, by formulating it as a nonlinear optimization problem that minimizes the sum of least squares error of the centroid points of the corresponding branches. By connecting the branch points with straight lines, we obtain a reconstructed central axis tree which closely corresponds to the input stick model. We also studied the effect of adding external noise to out tree models and evaluating the physiological parameters. We conclude with the results of our algorithm on real airway trees.
We present a novel robust methodology for corresponding a dense set of points on an object surface from photometric values, for 3-D stereo computation of depth. The methodology utilizes multiple stereo pairs of images, each stereo pair taken of exactly the same scene but under different illumination. With just 2 stereo pairs of images taken respectively for 2 different illumination conditions, a stereo pair of ratio images can be produced; one for the ratio of left images, and one for the ratio of right images. We demonstrate how the photometric ratios composing these images can be used for accurate correspondence of object points. Object points having the same photometric ratio with respect to 2 different illumination conditions comprise a well-defined equivalence class of physical constraints defined by local surface orientation relative to illumination conditions. We formally show that for diffuse reflection the photometric ratio is invariant to varying camera characteristics, surface albedo, and viewpoint and that therefore the same photometric ratio in both images of a stereo pair implies the same equivalence class of physical constraints.
Polarization vision has recently been shown to simplify some important image understanding tasks that can be very difficult to perform with intensity vision. This, together with the more general capabilities of polarization vision for image understanding, motivates the building of camera sensors that automatically sense and process polarization information. Described in this paper is a design for a liquid crystal polarization camera sensor that has been built to automatically sense partially linearly polarized light and computationally process this sensed polarization information at pixel resolution to produce a visualization of reflected polarization from a scene and/or a visualization of physical information in a scene directly related to sensed polarization. As the sensory input to polarization camera sensors subsumes that of standard intensity cameras, they can significantly expand the application potential of computer vision for object detection. A number of images taken with polarization cameras are presented showing potential applications to image understanding, object recognition, circuit board inspection, and marine biology.
Gaussian curvature is an intrinsic local shape characteristic of a smooth object surface that is invariant to orientation of the object in 3D space and viewpoint. Accurate determination of the sign of Gaussian curvature at each point on a smooth object surface (i.e., the identification of hyperbolic, elliptical and parabolic points) can provide very important information for both recognition of objects in automated vision tasks and manipulation of objects by a robot. We present a multiple illumination technique that directly identifies elliptical, hyperbolic, and parabolic points from diffuse reflection from a smooth object surface. This technique is based upon a photometric invariant involving the behavior of the image intensity gradient under varying illumination under the assumption of the image irradiance equation. The nature of this photometric invariant allowed direct segmentation of a smooth object surface according to the sign of Gaussian curvature independent of knowledge of local surface orientation, independent of diffuse surface albedo, and with only approximate knowledge of the geometry of multiple incident illumination. In comparison with photometric stereo, this new technique determines the sign of Gaussian curvature directly from image features without having to derive local surface orientation, and, does not require calibration of the reflectance map from an object of known shape of similar material or precise knowledge of all incident illuminations. We demonstrate how this segmentation technique works under conditions of simulated image noise, and actual experimental imaging results.
Inhomogeneous dielectric surfaces exhibit both diffuse and specular reflection components. Although various reflection models have been proposed for both of these components, the prediction of the relative strengths of these components in computer vision and computer graphics has so far not had a strong physical motivation. We propose a reflectance model for combined diffuse and specular reflection from dielectric materials that involves purely physical parameters (i.e., no ad hoc weighting of specular and diffuse components). This reflectance model is used to predict the relative strength of diffuse and specular reflection components ih terms of imaging geometry, dielectric surface parameters, and solid angular extent of incident light. We derive lower bounds on the contrast ratio between a specularity and surrounding diffuse reflecting regions. These can be used effectively to rule out highly contrasting diffuse reflecting regions being misidentified as pecularities under a number of conditions that can significantly aid intensity-based specularity detection methods, and in turn image understanding. The presented theoretical developments can be used to predict the photometric dynamic range of illuminated objects, which can be essential to inspection methods in machine vision. These developments can also be used in computer graphics for the physically precise rendering of the relative strengths of specular and diffuse reflection from inhomogeneous dielectrics.
One of the most common assumptions for recovering object features in computer vision and rendering objects in computer graphics is that the radiance distribution of diffuse reflection from materials is Lambertian. We propose a reflectance model for diffuse reflection from smooth inhomogeneous dielectric surfaces that is empirically shown to be significantly more accurate than the Lambertian model. The resulting reflected diffuse radiance distribution has a simple mathematical form. The proposed model for diffuse reflection utilizes results of radiative transfer theory for subsurface multiple scattering. For an optically smooth surface boundary this subsurface intensity distribution becomes altered by Fresnel attenuation and Snell refraction making it become significantly non-Lambertian. The reflectance model derived in this paper accurately predicts the dependence of diffuse reflection from smooth dielectric surfaces on viewing angle, always falling off to zero as viewing approaches grazing. This model also accurately shows that diffuse reflection falls off faster than predicted by Lambert's law as a function of angle of incidence, particularly as angle of incidence approaches close to 90 degree(s). We present diffuse reflection effects near occluding contours of dielectric objects that are strikingly deviant from Lambertian behavior, and yet are precisely explained by our diffuse reflection model. An additional feature of our diffuse reflection model is that is predicts the diffuse albedo purely in terms of the physical parameters of a smooth dielectric surface, allowing rigorous derivation of the relative brightness of specular and diffuse reflection.
One of the most common assumptions for recovering object features in computer vision and rendering objects in computer graphics is that the radiance distribution of diffuse reflection from materials in Lambertian. We propose a reflectance model for diffuse reflection from smooth inhomogeneous dielectric surfaces that is empirically shown to be significantly more accurate than the Lambertian model. The resulting reflected diffuse radiance distribution has a simple mathematical form. The proposed model for diffuse reflection utilizes results of radiative transfer theory for subsurface multiple scattering. For an optically smooth surface boundary this subsurface intensity distribution becomes altered by Fresnel attenuation and Snell refraction making it become significantly non-Lambertian. We present a striking diffuse reflection effect at occluding contours of dielectric objects that is strongly deviant from Lambertian behavior, and yet is explained by our diffuse reflection model. The proposed diffuse reflection model for optically smooth surfaces can be used to describe diffuse reflection from rough dielectric surfaces by serving as the diffuse reflection law for optically smooth microfacets.
We present a fully automated system which unites CCD camera technology with liquid crystal technology to create a polarization camera capable of sensing the polarization of reflected light from objects at pixel resolution. As polarization affords a more general physical description of light than does intensity, it can therefore provide a richer set of descriptive physical constraints for the understanding of images. Recently, it has been shown that polarization cues can be used to perform dielectric/metal material identification, specular and diffuse reflection component analysis, as well as complex image segmentations that would be immensely more complicated or even infeasible using intensity and color alone. Such analysis has so far been done with a linear polarizer mechanically rotated in front of a CCD camera. The full automation of resolving polarization components using liquid crystals not only affords an elegant application, but reduces the amount of optical distortion present in the wobbling of a mechanically rotating polarizer. In our system two twisted nematic liquid crystals are placed in front of a fixed polarizer placed in front of a CCD camera. The application of a series of electrical pulses to the liquid crystals in synchronization with the CCD camera video frame rate produces a controlled sequence of polarization component images that are stored and processed on Datacube boards. We present a scheme for mapping polarization states into hue, saturation, and intensity which is a very convenient representation for a polarization image. Our polarization camera outputs such a color image which can then be used in polarization- based vision methods. The unique vision understanding capabilities of our polarization camera system are demonstrated with experimental results showing polarization-based dielectric/metal material classification, specular reflection, and occluding contour segmentations in a fairly complex scene, and surface orientation constraints for object recognition.
We introduce a novel methodology for accurate determination of surface normals and light source location from depth and reflectance data. Estimation of local surface orientation using depth data alone from range finders with standard depth errors can produce significant error. On the other hand, shape-from-shading using reflectance data alone produces approximate surface orientation results that are highly dependent upon just the right initial surface orientation estimates as well as regularization parameters. Combining these two sources of information gives vastly more accurate surface orientation estimates under general conditions than either one alone, even when the light source location is not initially known. Apart from increased knowledge of local orientation, this also can provide better knowledge of local curvature. We propose novel iterative methods which enforce satisfaction of the image irradiance equation and surface integrability without using regularization. These iterative methods work in the case where the light source is any finite distance from the object producing variable incident light orientation over the object. These are realistic machine vision conditions in a laboratory setting.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.