Multispectral object detection technology has important application prospects in the fields of autonomous driving and so on. Conventional multispectral object detection algorithm rely solely on deep neural networks to learn multispectral image sample information, lacking the guidance of prior knowledge, and not fully utilizing infrared, visible, and other spectral information, resulting in decreased accuracy of object detection in complex scenes. To address this problem, this paper proposes an object detection algorithm based on infrared visible sample augmentation and illumination guidance. The algorithm adopts the MMDetection framework and extracts multispectral object features based on a designed sample augmentation method based on the fusion of positive and negative samples in multispectral images. Based on a designed adaptive weight allocation method guided by illumination, it enhances the algorithm's adaptability to the lighting environment. Finally, through the design of a multi-task loss function, it achieves high-precision and robust object detection in complex scenes. Experimental results on datasets such as FLIR and M3FD show that the proposed algorithm has significant advantages over comparative algorithms such as CFR_3 and GAFF in terms of average detection precision.
The evaluation of port wine stain based on three-dimensional information can overcome the inaccuracy of twodimensional image evaluation methods commonly used in clinic. In this paper, an end-to-end multitasking method is designed for the application of 3D information acquisition of port wine stain. Based on deep learning and position map regression network, the reconstruction from 2D pictures to face 3D point cloud is realized. the facial information of patients with port wine stain is represented by UV position map recording 3D point information of the face, and the dense relationship between 3D points and points with semantic meaning in UV space is characterized with this method. The deep learning network framework based on Encoder-Decoder structure is used to complete unconstrained end-to-end face alignment and 3D face reconstruction, whose parameters are obtained by training the data set with lightweight CNN structure. In the process of neural network training and end-to-end unconstrained image facial reconstruction, each point on the UV position map can be assigned different weights, which can not only be used to improve the network performance in neural network training, but also be used to assign corresponding weights to the focus areas with different disease course in the three-dimensional information reconstruction of the focus area therefore the accuracy of the reconstruction results can be increased. With the help of this method, the three-dimensional reconstruction results can be quickly obtained from a single patient's face image, which can be used for subsequent accurate lesion information analysis and treatment.
Objective and accurate lesion assessment is one of the key factors for the optimal photodynamic therapy (PDT) of Port Wine Stains (PWS). The evaluation method based on 3D point cloud can effectively solve the problems. Comparing to the traditional point cloud registration algorithms such as ICP which tend to global registration, the feature description method is more suitable for facial point clouds fusion since each clouds is obtained from different angle and just overlap in certain area. In this paper, the method of fusion of multiple point clouds is applied, in which point cloud fusion is achieved by characterizing randomly selected sampling points. The point cloud fusion result is obtained by matching the public area between the two point clouds. Several fusion experiments are designed and conducted using the facial point cloud acquired by Artec Eva three-dimensional scanner. The experimental results show that it can accomplish the point cloud fusion of patient facial point cloud (including all the information on the skin lesions of PWS), which could be an efficient support for effective 3D lesion assessment.
Optical Manufacturing Technology is a compulsory course for postgraduate and advanced undergraduate majors in optical engineering and similar majors. Modern fabrication and testing equipment are extremely expensive, which cost millions or even tens of millions, and having high requirements for the use environment. Most schools that do not focus on optical fabrication rarely purchase these devices for educational purposes only. Relevant professional teachers are also not familiar with the working principle and use process of the new fabrication and testing equipment such as Computer Controlled Optical Surfacing (CCOS), Magnetorheological Finishing (MRF), Ion Beam Figuring (IBF), laser tracker and LuphoScan interferometer. This has caused the teaching content of the course to stay at the stage of singleaxis machining, knife-edge tester, so that students' knowledge of modern equipment is almost blank. While teaching with micro-course videos, the relevant equipment use process can be visually presented to students, and relevant technical experts can also explain the working principle and scope of application to students more professionally. According to the survey, 82% of students like this micro-course video teaching, acknowledging that it helps to better understanding modern optical fabrication and testing equipment, expand students' knowledge, and increase students' advantages in employment competition.
Advanced professional courses (APCs) in the senior year will lay the foundation for further graduate study. Meanwhile, they are summaries and applications of the learnt fundamental professional courses (FPCs). Thus APCs form a connecting link between the preceding and the following studies. For example, Principles and Design of Optoelectronic Instruments (PDOI) is a lecture-based APC aiming at familiarizing students with the operating principles and basic design methods of commonly used optoelectronic instruments. Students will be able to describe the operating procedure of the instruments, distinguish the structure and function of each part, and present preliminary results of both overall design and parameter design. Problem-based approach with the following implementation is a good choice for such APCs. An assignment of system design is announced as the problem at the beginning of the semester. Students are asked to (1) describe the basic working principle, (2) do the overall design and draw the schematic diagram of the system, (3) do the module devision as well as the budget, and (4) finally analyze a critical parameter of the system. Then during the explanation of corresponding chapters, four times of in-class practices are arranged to help the students finish the assignment question-by-question with the help of textbook, internet and the teacher. Compared with straightforward explanation of the chapters and leaving the assignment as a homework at last, the proposed problem-based approach helps improving the motivation and achievement of the students.
Traditional lectures do hardly satisfy or benefit all of graduate students because of their diversity background, which might lead to inactive in the class, especially when the courses are professionally concentrated. Jigsaw cooperative learning (JCL), a student-centered learning method which engages all students and is beneficial in facilitating relationship-building, might be an alternative solution. A trial was made in an optical course “Optical Interferometric Measurement (OIM)”. However, the first trial we made is hardly counted as success. After thorough analysis on all factors that might cause the frustration, a new course plan is formed detailed. In this paper, we would like share our trials made in the course, both the unsuccessful one and the one work in progress.
Optoelectronic Instruments is a comprehensive professional course for senior students majored in optical engineering and similar specialties. Due to the low lecturer/ student ratio, typically less than 1:100, most of the students gave up the chance of one-to-one communication with the lecturers even when they were confused about the principle or applications of the instruments. A smart phone App Rain Classroom associated with messaging App Wechat is introduced. It enables the lecturers to receive instant feedback from students through bullet screen, push preview and review materials and post in-class quiz. Investigation also shows that 76% of the students enjoyed the new interactive tool, acknowledging its help in understanding the topic better, improving in-class interaction, and after class communications.
When talking about higher education, it’s hard not to run into a discussion on what’s really better for student learning: online learning or traditional learning? Of course, the key is to offer both, and potentially emphasize blended learning as the less polarizing option. Online courses are much more flexible and less expensive, but powerless while hands-on practical capacity is involved. Traditional experimental course can maintain a fluid and solid learning process but is less productive due to its scheduled time and simplex access. In this paper, a buffet-style knowledge service mode applied in a 12-week-long project-based experimental course Optoelectronic Instrument Experiments (OIE) is discussed. Our purpose is to find a blended learning mode in experimental education.
Planning of irradiance distribution (PID) is one of the foremost factors for on-demand treatment of port wine stains (PWS) with photodynamic therapy (PDT). A weighted optimization method for PID was proposed according to the grading of PWS with a three dimensional digital illumination instrument. Firstly, the point clouds of lesions were filtered to remove the error or redundant points, the triangulation was carried out and the lesion was divided into small triangular patches. Secondly, the parameters such as area, normal vector and orthocenter for optimization of each triangular patch were calculated, and the weighted coefficients were determined by the erythema indexes and areas of patches. Then, the optimization initial point was calculated based on the normal vectors and orthocenters to optimize the light direction. In the end, the irradiation can be optimized according to cosine values of irradiance angles and weighted coefficients. Comparing the irradiance distribution before and after optimization, the proposed weighted optimization method can make the irradiance distribution match better with the characteristics of lesions, and has the potential to improve the therapeutic efficacy.
Principle and Design of Optoelectronic Instruments (PDOI) is a comprehensive engineering course set in the fourth year in School of Optoelectronics, Beijing Institute of Technology, China. After three years of study, the students have acquired basic knowledge in optics, mechanics, electronics, and computer science. They are ready to make a comprehensive application of the knowledge they have learnt in something really important. Like most engineering courses, PDOI needs practical section to help students understand how theories work in engineering. Thanks to the China International Exhibition of Lasers, Optoelectronics and Photonics which is held annually in Beijing, it offers a good opportunity for undergraduates to see advanced instruments with their own eyes. It is a wonderful extracurricular practice for PDOI. In this paper, we will introduce the exhibition-involved curriculum design and give the initial results.
The precision of manufacturing and installing together with the flexibility is a serious challenge for laser induced fluorescent detector (LIFD) of microfluidic chip. In this paper, a focus tunable liquid lens based on liquid zoom system for LIFD with automatic adjustment is proposed. With the help of liquid zoom lens whose surface curvature can be varied continuously by current, the system can achieve a continuous zoom. Instead of using the traditional mechanical axial displacement scanning motion mechanism, the proposed zoom system can implement axial displacement scan by means of the well-designed autofocus feedback current control function. The simulation results show that the focal length variation range of the designed optical system is 4.87mm~ 8.40mm, which is also the axial scanning displacement range. The size of scanning spot is around 15μm when a 473nm wavelength laser is used, which can meet the demand of microfluidic chip detection. With this design, the required precision of the LIFD could be reduced significantly as well as costs. Moreover, it also makes the detection of microfluidic chip qualified to adapt to different size of detecting channel.
KEYWORDS: Photodynamic therapy, Projection systems, Light sources, Clouds, Optimization (mathematics), Light, 3D modeling, Control systems, 3D image reconstruction, Head
Controllable and effective irradiation of lesions is among the key factors that affect the potency of photodynamic therapy (PDT). An optimization method for the irradiance distribution of treatment was proposed which can be used to improve the efficacy of PDT and allow more lesions to receive the desired irradiance level in a single therapy session. With the proposed digital illumination binocular treatment system, the preferred surface normal vectors, irradiation angles, as well as area and weight coefficients of lesions can be achieved and used as characteristic parameters to optimize the irradiation direction. Two port-wine stain phantom experiments were performed. The comparison of the illumination area between preoptimization and postoptimization showed that the proposed method can effectively guide the light source control, improve the distribution of light dose, and increase the effective treatment area.
Photodynamic Therapy is regarded as the best treatment for port wine stains, which has the main adverse effect of various degrees of pain (mild to moderate) during the illumination. Though the cooling and cold water have been used to reduce such pain, there is still no scientific evidence for these relief. In this paper, a realistic skin model is built to simulate the distribution of light under treatment, which helps control the light dose and temperature, and improve the clinical results. Comparing with the general parallel skin model, a curving stratum basale layer is used in this paper, and various blood vessel configurations such as single and multiple vessels with horizontally and vertically oriented, curve vessels, various vessel diameter and various radius of curvature of stratum basale layer are simulated. The results shows a more realistic modeling for the thermal damage and help to relief the pain in the treatment.
Venipuncture is the most common way of all invasive medical procedures. A vein display system can make vein access
easier by capturing the vein information and projecting a visible vein image onto the skin, which is correctly aligned with
the subject’s vein. The existing systems achieve correct alignment by the design of coaxial structure. Such a structure
causes complex optical and mechanical design and big physical dimensions inevitably. In this paper, we design a stereovision-
based vein display system, which consists of a pair of cameras, a DLP projector and a near-infrared light source.
We recover the three-dimensional venous structure from image pair acquired from two near-infrared cameras. Then the
vein image from the viewpoint of projector is generated from the three-dimensional venous structure and projected
exactly onto skin by the DLP projector. Since the stereo cameras get the depth information of vessels, the system can
make sure the alignment of projected veins and the real veins without a coaxial structure. The experiment results prove
that we propose a feasible solution for a portable and low-cost vein display device.
KEYWORDS: Photodynamic therapy, Image segmentation, Projection systems, 3D modeling, 3D acquisition, 3D image processing, Dermatology, Light sources, 3D scanning, Cameras
Light dosimetry is an important parameter that affects the efficacy of photodynamic therapy (PDT). However, the irregular morphologies of lesions complicate lesion segmentation and light irradiance adjustment. Therefore, this study developed an illumination demo system comprising a camera, a digital projector, and a computing unit to solve these problems. A three-dimensional model of a lesion was reconstructed using the developed system. Hierarchical segmentation was achieved with the superpixel algorithm. The expected light dosimetry on the targeted lesion was achieved with the proposed illumination procedure. Accurate control and optimization of light delivery can improve the efficacy of PDT.
Subcutaneous vein images are often obtained by using the absorbency difference of near-infrared (NIR) light between vein and its surrounding tissue under NIR light illumination. Vein images with high quality are critical to biometric identification, which requires segmenting the vein skeleton from the original images accurately. To address this issue, we proposed a vein image segmentation method which based on simple linear iterative clustering (SLIC) method and Niblack threshold method. The SLIC method was used to pre-segment the original images into superpixels and all the information in superpixels were transferred into a matrix (Block Matrix). Subsequently, Niblack thresholding method is adopted to binarize Block Matrix. Finally, we obtained segmented vein images from binarized Block Matrix. According to several experiments, most part of vein skeleton is revealed compared to traditional Niblack segmentation algorithm.
Imaging of subcutaneous veins is important in many applications, such as gaining venous access, vascular surgery and
venipuncture. Traditional vein imaging system can only obtain the two dimensional information of the vein which loses
the depth information of the vein. It may cause the errors of judgment and increase the venipuncture failure.
On the basis of previous research, a new system was proposed to acquire the three dimensional of the vein. In this paper,
the infrared absorption characteristics of the vein and the principle of binocular vision were combined to obtain infrared
images of subcutaneous veins and recovery the three dimensional information. The binocular vision system was consists
of several 850 nm near-infrared LEDs to illuminate the back of the hand and two near-infrared CCD devices to obtain
the transmission of IR image.
The couple of CCDs will get IR images of the hand which contain the disparity information. The principle of stereo
vision was used to recover the three dimensional structure. The algorithm processes includes camera calibration, image
preprocessing, epipolar rectification, stereo correspondence and three dimensional reconstructions. Experimental result
shows that it can acquire a good three dimensional structure. Since the new system can recover the depth of the vein, it
can be applied as the venipuncture auxiliary equipment to improve the success rate of venipuncture.
3D reconstruction, an important point in computer vision, has a promising application potential in endoscopic-assisted
minimally invasive surgery. In this paper, a 3D reconstruction method based on image sequences is proposed, in which
the improved SIFT feature is applied for point extraction and matching, as well as an optical tracker is used to get the
orientation of the camera in real time. The proposed approach is evaluated on sequence digital images gotten from an
1394 camera and the experimental results show that the proposed approach is effective.
A new computation technique is presented for calculating pixel colors in anaglyph images. The method depends
upon knowing the RGB spectral distributions of the display device and the transmission functions of the filters in the
viewing glasses. It requires the solution of a nonlinear least-squares program for each pixel in a stereo pair and is based
on minimizing color distances in the CIEL*a*b* uniform color space. The method is compared with several techniques
for computing anaglyphs including approximation in CIE space using the Euclidean and Uniform metrics, the Photoshop
method and its variants, and a method proposed by Peter Wimmer. We also discuss the methods of desaturation and
gamma correction for reducing retinal rivalry.
Mutual occlusion is an attribute of an augmented reality system. It makes the user confirm that the
virtual objects truly exist in the real world. Traditional optical see-through displays are not capable of
correctly presenting the mutual occlusion of real and virtual environments, since the synthetic objects
always appear as translucent ghosts floating in front of the real scene. This paper presents a novel
optical see-through HMD. Aim at this new type HMD some feasible method is presented to realize
the mutual occlusion. A LCD panel is introduced in our display for the occlusion. Experimental
results show that the methods based on the novel display can integrate a virtual object in a real scene
seamlessly.
The traditional tabletop AR systems general use head mounted display (HMD) that has some shortcomings, such as
the imprecise precision and low flexibility. To solve these problems, a new design of video see-through tabletop system
is presented. In this paper, we describe an outline of the system and the registration algorithms for the system. A system
origin calibration algorithm is proposed. In the calibration experiment, a sign cube is introduced for the first shooting of
the camera. The position of the sign cube becomes the origin of the world reference frame, in which the translation and
rotation of the virtual objects relative to the origin can be calculated easily. The experimental results show that the video
see-through tabletop system meets the precision and flexibility requirements very well.
In order to integrate seamlessly a virtual object in a real scene in augmented reality (AR) system, we need to simulate the
interactions of the virtual object with the illumination of the scene. Acquiring the knowledge of illuminant direction is
crucial in this work. We present a novel approach for estimating the direction from a single image of a scene that is
illuminated by a light source regardless it is point light source or directional one. We propose to employ a marker cube,
which is used to register to determine the rigid transformation relating 2D images to known 3D geometry, and a
Lambertian probe sphere, which is used to estimate the light source direction by image processing. The key process is
finding and extracting the intensity occluding curve on the sphere. Experimental results show that our approach is
computationally efficient and the light source direction can be accurately obtained by it.
Estimation of camera pose is an integral part and classical problem of augmented reality (AR) system and computer
vision. Accurate pose estimation is crucial in determining the rigid transformation relating 2D images to known 3D
geometry. Therefore, the algorithm should be not only fast and accuracy, but also robust in AR system. Orthogonal
iterative (OI) algorithm is a good method, but it requires a proper initialization and cannot deal with problems of pose
ambiguity. A new method based on OI we presented before, provides a good initialization and solves a problem of pose
ambiguity introduced by coplanar markers. However, two more potential problems usually make the algorithm calculate
some wrong results, and lead to the algorithm unsteady and not robust. In this paper, we develop the method by resolving
pose ambiguities, which originate from potential problems in algorithm. Two more constraints are employed in our
method. One is camera must be located in front of the marker, while the other is camera must be oriented to the marker.
It's proved that the improved method is steady in experiments, and can calculate the pose of camera fast and correctly.
Moreover, since the method can deal with pose ambiguity, it is rather robust in AR system.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.