2.1.

Architecture of RediNet

The structure of RediNet is shown in Fig. 2. The typical neural networks employed in phase modulation regard pixel-wise light distribution and the corresponding pixel-wise CGH as the input and output. Although effective and straightforward, this direct approach often encourages an oversized and overspecialized network.^24,25 To circumvent this issue, we propose an indirect three-step procedure to isolate the variation of structured light species and resolution. The neural network serves as the computation kernel [Fig. 2(b)], while the parameter space acquisition [Fig. 2(a)] and primitive functions mapping [Fig. 2(c)] are externalized as pre- and postprocessing.

Fig. 2

The architecture of RediNet and CGH generating workflow. (a) Pre-processing. The table contains examples of CPFs $\alpha (x,y)$ of four different structured light species. Two rows in the table correspond to two parameters of structured light properties, and also correspond to the coordinates in parameter space. The CPF species can be further extended, and the values of the parameter can be expanded. (b) The neural network in RediNet. A parameter space $P$ in the left box is the input, carrying the configuration of the target structured light array. The neural network in RediNet possesses an architecture of encoder, decoder, and skip connection. The trained network’s output is primitive function $S$, displayed in the right box. (c) Post-processing by mapping the 3D primitive function to a 2D phase CGH. For determining the phase value of one pixel at $(x_0,y_0)$, the first step is positioning, putting $(x_0,y_0)$ into every CPF and obtain ${\alpha }_i(x_0,y_0)$. The second step is evaluation, finding the value of the primitive function at the coordinates of ${\alpha }_i(x_0,y_0)$. All the pixels on a CGH share the same mapping procedure.

In first step, the preprocessing is about determining the CPFs by the properties of the structured light and constructing a uniform parameter space. CPF is essentially an exact phase distribution for producing a structured light beam with a single optical property, as listed in Fig. 2(a). Most structured lights have their corresponding analytic CPFs, which are linear with the modulation parameters. In the examples shown in Fig. 2(a), the CPF for the $x$ direction shifting $\mathrm{\Delta }x$ is the blazed grating phase $kx\mathrm{\Delta }x/\mathrm{z}$, for the vortex beam with topological charge (TC) $l$ is the spiral phase $l\theta$, and for the Airy beam with modulation parameter $b$ is the cubic phase $(x^3+y^3)b$, where $x$, $y$, $z$, and $\theta$ are the Cartesian or polar coordinates. Beyond these cases, some important CPFs defy this linear property, but their first-order approximations can serve as the acceptable CPFs. For instance, when axial defocusing amount $\mathrm{\Delta }z$ is negligible relative to focus length $f$, a proper defocusing CPF approximation is $-k{r}^2\mathrm{\Delta }z/(2f^2)$. More examples of CPFs and corresponding parameters are listed in the Supplementary Material.

The linear modulation parameters, including $\mathrm{\Delta }x$, $l$, $b$, and $\mathrm{\Delta }z$ mentioned above, are the key elements of controlling the structured light properties (e.g., altering the TC $l$ to change the carried OAM in a vortex beam). Based on them, the description of the structured light array using CPFs and modulation parameters is similar to describing several vectors in a multidimensional linear space with a set of basis vectors and coordinates. In this concept, we can build the multidimensional parameter space, where the “coordinates” about the structured light array are stored, and each dimension represents an independent modulation parameter and is intrinsically linked to the corresponding CPF. In addition, the dimensions in parameter space are with equal standing and are redefinable. Typically, this parameter space is more compact than the complex amplitude profile on a pixel-wise basis. For visualization clarity, our exploration is confined to the 3D parameter space, depicted as the left cube in Fig. 2(b). Nonetheless, this framework is scalable and naturally adaptable for parameter spaces of higher or lower dimensions.

The second step focuses on harnessing the neural network to calculate an effective primitive function. As in the above description, the CPFs and the parameter space are sufficient to determine a certain structured light array, but the exact solution for the CGH is still confusing. Here, we present a mathematical framework to link the target parameter space and the CGH (see Supplementary Material for a more detailed description). Like the analysis of Dammann grating,²⁸ the discrete and periodic nature of the array enables us to represent the CGH as a weighted summation of the complex amplitudes of the CPF’s multiples,

Equation (1) is noted to be very similar to a Fourier series. Assume there is a multidimensional primitive function $\exp [jS(t_1,\dots ,t_N)]$, whose moduli all equal to 1 and whose angle is $S(t_1,\dots ,t_N)$. The value of $S(t_1,\dots ,t_N)$ is subtly arranged to meet the criterion that its Fourier series coefficients share the same moduli with $|a_{l_1,\dots ,l_N}|$. Then, the Fourier series of primitive functions will become exactly the same on the right side of Eq. (1) when we consider ${\alpha }_i(x,y)$ as $t_i$ and it indicates

In this way, $\phi (x,y)$ is converted into a composite function form. The inner function is the CPFs ${\alpha }_i(x,y)$ and the outer one is the primitive function $\exp [jS(t_1,\dots ,t_N)]$. In the following, our claims about primitive function only refer to the angle part $S$ for simplicity.

To determine the value in $S$, we introduce a neural network, illustrated in Fig. 2(b), where the input and output are shown with particular emphasis. The input of the network is the aforementioned 3D parameter space, with the same eight-pixel resolution in every dimension. The output is the primitive function $S$ with 64-pixel resolution. The network adopted the well-established U-net structure²⁹ and integrated the resblock.³⁰ U-net framework features an encoder-decoder configuration and contains skip connections that simultaneously capture both overall trends and local details. The loss function in training is first the classical L2 loss and then shifts to $0.5[1-\cos (x-\hat{x})]$, where $x$ is the predicted value of the network and $\widehat{x}$ is the ground truth. The latter loss function can perfectly evaluate the periodic phase values. Even though the common 3D convolutional networks are parameter-dense, RediNet is exceptionally lightweight, consisting of mere 22.4 million network parameters. This conciseness translates to swift prediction time, typically under 3 ms on a consumer-grade graphics processor. Details of the neural network structure and training can be found in the Supplementary Material.

In the third step, as shown in Fig. 2(c), the postprocessing maps a 2D CGH from the 3D primitive function $S$, which is the output of the network. For each pixel, the mapping procedure is the same, similar to the composite function evaluation. For pixel $(x_0,y_0)$, we can compute three ${\alpha }_i(x_0,y_0)$ values. Taking them as 3D coordinates, we can evaluate $\phi (x_0,y_0)$ by computing $S[{\alpha }_1(x_0,y_0),{\alpha }_2(x_0,y_0),{\alpha }_3(x_0,y_0)]$ as the instruction in Eq. (2). Then, repeating this procedure for every pixel yields the whole CGH $\phi (x,y)$.

2.2.

Generation of 3D Primitive Function Data Set

Network training demands a data set containing 3D Fourier series of coefficients and corresponding primitive functions, but there is no suitable data set, to our knowledge. Therefore, we employed an iterative algorithm to generate the data set. The data set is the randomly distributed parameter space and the corresponding primitive function as ground truth. Normally, nonlinear optimization is needed for this kind of ill-posed problem, so we give a parallel iterative Fourier transform procedure. The constraints in the iteration contain two crucial criteria: for the primitive function, it needs to be real-valued; for the parameter space, its modulus needs to be identical to the given parameter space. The details of this iteration algorithm can be found in the Supplementary Material.

The generated data set is saved as a 3D matrix in a MAT file (with commercial software MATLAB) and can be loaded and converted to tensors. Code for generating the data set and several pairs of data samples are available at https://github.com/LiHengyang1/RediNet.

2.3.

Computational Environment

The calculation, including data set generation, training, prediction, CGH generation, and evaluation, is performed on a personal desktop with an Intel processor I7 13700, 32 GB of memory, and a graphic processor Nvidia RTX3080Ti. Network training and prediction are based on Pytorch 1.12, and the pre- and postprocessing stages are performed on MATLAB R2023a. All the comparison studies are performed in the same computational environment.

2.4.

Phase Modulation System

The laser holographic system consists of an Nd:YAG laser with a central wavelength of 1064 nm. The beam is expanded to 7.9 mm diameter and collimated. A phase-only reflective liquid crystal SLM (X13138, Hamamatsu) is utilized for the phase modulation, which features a $1272\times 1024$ resolution and a $12.5\mu \mathrm{m}$ pixel pitch. In the experiment, only the central circular portion of the SLM is used to modulate the wavefront, while the other pixels are set to blazed grating. A phase CGH with 8-bit depth is loaded onto the SLM, corresponding to the phase modulation depth from 0 to $2\pi$. A 12-bit depth camera is used to capture the intensity distribution. The pictures of the experimental setup are in the Supplementary Material.

To determine the phase pattern on the focal plane, a reference beam is used for interference. By deliberately introducing an angle between the modulated beam and reference beam, the phase distribution can be solved with a single fringe intensity picture. The detailed approach is given in the Supplementary Material.

3.1.

Customizing Structured Light Array with RediNet

RediNet is applied to the generation of myriad structured light arrays with diverse distributions. We should underline that the network parameters of RediNet remain constant, and no iterative procedures are performed throughout the experiments. As mentioned above, the experimental setup is easily accessible, where a phase SLM is used as the only modulation device.

First, we have generated focus arrays on the focal plane and in 3D space. As shown in Fig. 3(a), by utilizing only two dimensions of the parameter space as $x$- and $y$-direction shifting, a symmetric equal-energy four-foci array can be generated with an intensity root-mean-square error (RMSE) of 0.051. By additionally including the modulation over the axial defocusing in the third dimension of the parameter space, we can sculpt a 3D focus array with four layers, a total of 19 foci randomly distributed, and an RMSE intensity of 0.193. At this point, the dimensions in parameter space exactly correspond to the physical space dimensions, so the intuitive graphical match between them is apparent in Fig. 3(a). As mentioned before, we used an approximate CPF for axial defocusing, so the positions of the four planes show a slight deviation on the $z$ axis. Associated with the 400-mm-focus-length lens used in the experiments, the actual four planes are located at $z=25.5$, 12.4, 0, and $-11.7$, corresponding to the target 24, 12, 0, and $-12\mathrm{mm}$, respectively. In practical applications, this discrepancy is negligible when the defocusing magnitude is small and it is virtually nonexistent in tight focusing situations, confirmed by its theoretical axial defocusing CPF.³¹

Fig. 3

Customizing 2D and 3D focus arrays, LG beam arrays, and Bessel and Airy beam arrays with RediNet. (a) Four foci in a square pattern are shown, with the phase CGH generated by RediNet. In addition, a four-layer focus array is generated and captured, respectively. The target parameter space is shown, where a similar distribution with intensity images can be found. The defocusing distances of the pictures are labeled. (b) Two LG beam arrays are generated. The top one includes LG00, LG01, LG10, and LG11 modes. Detailed intensity distributions on the left are individually enlarged and normalized. Separated phase CGHs for each mode and the final CGH are illustrated. Likewise, the bottom one shows the results and CGHs about LG22, LG23, LG32, and LG33 modes. (c) Two kinds of nondiffracting beam arrays are generated, including Bessel beam arrays and Airy beam arrays. The intensity distributions of a transverse plane and the 3D volume are given, respectively.

RediNet can produce parallel arrays of LG mode beams by imitating their phase distributions. The corresponding modulation phase patterns for different orders of LG mode are given in Fig. 3(b), encoded as a combination of spiral phase and radial step phase. Since the radial step phase is difficult to express as a linear function of a parameter, all its inner phase rings are considered as invariants, and the outermost ring is independently manipulated with one dimension in the parameter space with values only “switching” between binary numbers 0 and 1. Using the generated CGHs, a fundamental mode Gaussian beam is sculpted into beam arrays with LG00, LG01, LG10, and LG11 modes and LG22, LG23, LG32, and LG33 modes at different positions.

Researchers have experimentally produced a variety of nondiffracting beams whose transverse light field patterns remain almost unchanged after long-distance propagation.³² This unique feature has propelled a range of practical applications, from optical micromanipulation,³³ laser drilling,³⁴ to light bullets.³⁵ Using RediNet, we multiplexed three Bessel beams and three Airy beams, respectively. Three identical Bessel beams are generated with different propagation directions, as shown in Fig. 3(c). Three Airy beams are distributed at different positions, and the modulation parameter $b$ in the CPF takes the values of $+1$, $+2$, and $-3$, respectively, which leads to the observable different bending trajectories and directions. At the same time, the larger absolute value of the parameter $b$ contributes to more pronounced sidelobes of the Airy beam that appeared on the focal plane.

The light field expression of a vortex beam has an $\exp (jl\theta )$ term, so it possesses a spiral phase pattern. Not limited in the optical traps and microstructure fabrication, extensive applications could benefit from RediNet owing to customized control on various vortex beam arrays. There are similar circular intensity patterns in Figs. 4(a) and 4(b), yet they emerge from different modulation strategies. In Fig. 3(b), we utilize a radially linear CPF $kr\mathrm{\Delta }R/z$, which mimics a conical lens, making the intensities on the focal plane transform into rings with the radius proportional to the parameter $\mathrm{\Delta }R$, but without the spiral phase property. In contrast, in Fig. 4(c), we used the angularly linear CPF $l\theta$ to obtain a vortex beam array. The smaller donut distributions on the diagonal all carry $+8$ TCs, and the larger ones on either side carry $+12$ TCs.

Fig. 4

Customizing ring-focus arrays, vortex and perfect vortex beam arrays, helico-conical beam arrays, and snowflake arrays with RediNet. (a)–(e) The distributions of intensity and the distributions of the product of intensity and phase. The TCs $l$ and normalized ring radiuses $r$ (minimum as 1) are labeled in the figures. (f) The generation and the result of the snowflake intensity pattern. Arbitrarily built CPFs and the final CGH are illustrated and are used to generate four snowflakes in different positions on the focal plane.

The ring radius of a conventional vortex beam depends on the TC $l$, as can be observed in Fig. 4(b). To circumvent this relevance, a perfect vortex beam is introduced whose ring radius is independent.³⁶ It can be realized by overlaying spiral and conical phases, similar to combining the effects of Figs. 4(a) and 4(b). This implementation is realized in Fig. 4(c), where perfect vortex beams at different positions carry different TCs but with nearly identical ring radii and widths. Moreover, RediNet allows for independent modulation of the angular and radial CPFs at the same time. In Fig. 4(d), four perfect vortex beams are shown on the left side with the ring radii and TCs varying from each other. On the right side, there are four concentric rings with TCs progressively changing from $-2$ to $-8$ as the ring radius increases.

Dissimilar from the vortex beam, the helico-conical beam produces an unclosed helical intensity distribution on the focal plane.³⁷ It is obtained by multiplying radial and angular CPFs, leading to a nonseparable term. To deal with this special multiplication relation, we take the cross-product term as a whole. In Fig. 4(e), a four helico-conical beams array is realized, forming dual sets of unclosed helical trajectories surrounded by each other.

The arrays with a large number of structured light species have been enumerated in Figs. 3 and 4, and they basically all correspond to a family of wave equation solutions so that the CPFs and the formed structure light field have exact physical meanings and properties. Beyond them, structured light could be a broader concept, not limited by physically grounded solutions. In the architectural essence, RediNet does not distinguish the physical meaning of structured lights. Any CPFs are equally input and multiplexed with the same principles, and this arbitrariness is the quintessence of network redefinability. To verify this interesting capability, the snowflake beam array is generated. First, we arbitrarily build two CPFs: $6\{\mathrm{sgn}[\sin (8r)-0.1]\}+1.5r$ and $0.8r\sin (6\theta )$ and add $y$ shifting. Then, the phase CGH is generated just in the same way as other commonly seen structured light arrays. The obtained intensity distribution is shown in Fig. 4(f), where different hexagonal periodic structures appear at different locations, similar to snowflakes with different structures. In this implementation, complicated patterns are controlled by only adjusting two parameters, indicating a property like the vector graph that a small number of parameters are enough to control a huge number of pixels.

3.2.

Modulating a Multichannel Compound Vortex Beam Array with RediNet

Recently, many studies have explored the possibilities of OAM in free-space optical communication.^38,39 Because of the orthodox property, the vortex beams with different OAMs can be used as separate signal channels or coding formats, or even both.^40,41 Going even further, the compound vortex beam array is generated with RediNet, offering a novel example of concurrent control on spatial position, energy proportion, and multiple encoded OAM states.

From the implementation level, there were OAM multiplexing experiments about splitting multiple vortex beams at discrete spatial positions⁴² or gathering them into a singular compound beam.⁴³ The compound vortex beam array here is a much more integrated solution thanks to the versatility of RediNet. With only a phase SLM and a fundamental mode Gaussian incidence, RediNet can produce a multichannel vortex beam array, with each carrying the simultaneously dual OAM states shown in Fig. 5(a).

Fig. 5

Conceptional graph and results of multichannel compound vortex beam array generating with RediNet. (a) The conceptional graph about sculpting a fundamental mode beam to multichannel compound vortex beams with respective multiple OAMs. (b) The captured intensity distribution of multichannel compound vortex arrays on the focal plane, which is similar to interference results, but is generated from only one modulated beam instead of real existing signal and reference beams. (c) An example of a compound vortex beam with $+3$ and $-4$ TCs. The detailed distributions of intensity and the distributions of the product of intensity and phase in simulation and experiment are shown for comparison.

To realize the array in Fig. 5(b), we designate $x$- and $y$-shifting CPFs in two dimensions, and the angular CPF in the third dimension in the parameter space. Only 0 or 2 nonzero values are associated with an exact $x$ and $y$ coordinate, which implies only 0 or 2 vortex beams are generated in an exact position on the focal plane. In Fig. 5(b), randomly distributed eight-channel compound vortex beams are shown, with each channel simultaneously carrying a positive and a negative OAM state. This phenomenon finds its physical counterpart in multiposition vortex beam interference experiments. For instance, in Fig. 5(c), compound vortex beams with OAM states 3 and –4 are interfered, resulting in seven petals corresponding to their gap. The experimental intensity and phase distribution are compared with the simulation results, which are basically consistent. We added an overall conical phase in the CGH to achieve the compound perfect vortex beam array. As a result, uniform-sized ring radii are realized, probably simplifying the engineering effort of coupling beams into the same model of special-designed waveguides.⁴⁴

It is also worth mentioning that RediNet takes roughly 12 ms to generate this phase CGH with ${1000}^2$ resolution, which is faster than the mainstream SLM frame response time (16.7 ms or 60 Hz). This efficiency is crucial for optical communication and other real-time required scenarios, outperforming the iteration algorithms encumbered by their compute-storage-use procedure.

3.3.

Flexibility, Speed, and Accuracy of RediNet

A uniform and redefinable parameter space underpins the versatility of RediNet. Within the conventional neural networks related to spatial 3D light fields, the $z$ axis (wave vector direction) often plays a specialized role due to its implication with wave propagation characteristics. In contrast, RediNet’s architecture endows each dimension with equal standing. This symmetry permits users to arbitrarily redefine the designation of each dimension, thereby designing the parameter space complying with their own rules. Here, to verify this feature, we provide an illustrative example of exchanging the designation of dimensions [Fig. 6(a)]: first, we define three dimensions as $x$ shifting, $y$ shifting, and the radius of ring focus, respectively, fill in some arbitrary values into the target parameter space and save it as a 3D matrix. Accordingly, an intensity distribution on the image plane can be obtained. In the second step, the parameter designations on the three dimensions are kept unchanged, but the values of the parameter space $P$, i.e., the 3D matrix is permuted. Now, the correspondence of the value and parameter is disordered so that the obtained distribution of light intensity is divergent from that obtained in the first step. In the third step, the parameter designations and the values are permuted together from the initial state. Although the values are now transferred to new positions in the parameter space, the value-parameter correspondence is restored to its initial alignment. Consequently, the result intensity distribution is identical to that of the initial state.

Fig. 6

Numerical evaluation of RediNet performance. (a) Flexibility of dimension designation in parameter space. Compared with the first column, values in parameter space are permuted, and the simulation result is different in the second column. The values in the parameter space and the dimension designations are both permuted in the third column, but the simulation result is identical to that in the first column. (b) Flexibility of CGH resolution in mapping procedure. The CPFs of $x$, $y$, and $\theta$ with three different resolutions are involved in the mapping, and then three CGHs with resolutions of ${480}^2$, ${960}^2$, and ${3840}^2$ are produced. Time consumption in each step is labeled. The partially enlarged patterns are shown. (c) Computation time comparison of five algorithms, including CGH by non-convex optimization (NOVOCGH), 3DIFTA, SACAD, DeepCGH, and RediNet. Tasks of generating CGHs of resolution of ${512}^2$ and ${1024}^2$ are included. (d) Diffraction efficiency and RMSE of RediNet with different numbers of beams in an array. In the blue region, numbers of beams are not within the training data set. (e) Correlation analysis of RediNet based on target parameter space and the Fourier series coefficients expanded from the RediNet output primitive function. Data points under the situations of 10, 20, and 30 beams in an array are shown in different colors.

In the realm of neural networks, the resolution of the output is typically immutable, firmly anchored by the network’s architecture, and infeasible to adjust after training is completed. If a high-resolution CGH is required, one needs to reconstruct and retrain a larger-scaled network, bringing about an increase in training cost. RediNet, however, has an unusual architecture liberated from the confinement of output CGH resolution. The mapping procedure allows the resolution of the primitive function to remain static, while the resolution of the CGH can be arbitrarily adjusted through different CPF resolutions. In this way, the power of resolution control on the CGH is ceded to the explicit CPFs, which are some simple and exact formulations. This exotic property of providing arbitrary resolution CGHs with a fixed network is verified in Fig. 6(b). RediNet accomplishes a prediction task and then the single output primitive function generates three different CGHs with different resolutions. Since the mapping from the CPFs to the CGHs is a serial procedure, time consumption increases proportionally with the increase in resolution. From the zoomed-in figures, it is evident that the three CGHs share the same pattern, whereas the highest-resolution one keeps richly detailed phase variations, allowing more accurate manipulation of the spatial frequency. Unlike the upsampling on a bitmap, the approach and performance from ${480}^2$ to ${3840}^2$ resolution are also similar to vector graphic scaling, based on a set of control parameters and fundamental elements rather than relying on the low-resolution image itself.

Figures 6(a) and 6(b) show the flexibility coming from the input preprocessing and output postprocessing of the network, respectively. Furthermore, RediNet is fast due to its small-scale input as well as its simple network structure. We restrict the application scenario to 3D multifocal arrays to compare our method with some established algorithms. Among them, NOVOCGH utilizes a nonconvex optimization method, mainly aiming at the phase retrieval of arbitrary light-field distributions on multiple transverse planes;⁴⁵ the 3DIFTA method adopts Gerchberg-Saxton-like iterations and the 3D Fourier transform relationship between the Ewald’s cap and the 3D image space;¹⁶ the SACAD algorithm uses automatic differentiation to obtain high-quality 3D focal array inside the material;¹⁷ and DeepCGH also relies on neural network to realize multidepth light-field reconstruction, which can output complex CGH on both amplitude and phase.⁴⁶ RediNet consumes significantly less time than the other algorithms in generating a CGH for the seven-layer focus arrays, both in the case of ${512}^2$ and ${1024}^2$ resolutions. This swift performance underscores RediNet’s computational efficiency, but admittedly it compromises the complexity of the target light field due to the small resolution of the parameter space.

Finally, a brief assessment of the accuracy of RediNet is illustrated. The trends of diffraction efficiency and RMSE with the number of subbeams in an array, $N$, are shown in Fig. 6(d). As $N$ becomes larger, the diffraction efficiency decreases and the RMSE increases. When $N$ is 50, the diffraction efficiency is about 70%, and the RMSE does not exceed 0.2. In the training data set we used, data with $N$ more than 32 is nonexistent. The blue area on the right side in Fig. 6(d) shows the performance exceeding the biggest $N$ in training data, and the trends of diffraction efficiency and $\mathrm{RMSE}$ remain basically unchanged, which reflects the generalization of the network.

As a regression task, the correspondence between the input $A_{l_1,l_2,l_2}$ and the $|{\alpha }_{l_1,l_2,l_2}|$ expanded from output is illustrated in Fig. 6(e). The regression plots for the three different $N$ cases are given, revealing a satisfactory input–output correlation. When the input value is negligible (a beam with energy below 10% of the maximum), the network may handle it as 0. The overall correlation coefficient is 0.9711.