Recent Progress in the Correlative Structured Illumination Microscopy

Abstract

The super-resolution imaging technique of structured illumination microscopy (SIM) enables the mixing of high-frequency information into the optical transmission domain via light-source modulation, thus breaking the optical diffraction limit. Correlative SIM, which combines other techniques with SIM, offers more versatility or higher imaging resolution than traditional SIM. In this review, we first briefly introduce the imaging mechanism and development trends of conventional SIM. Then, the principles and recent developments of correlative SIM techniques are reviewed. Finally, the future development directions of SIM and its correlative microscopies are presented.

1. Introduction

Since their first use in the 17th century to study living cells, optical microscopes have attracted wide attention for their simple and flexible implementation and an ability to facilitate the non-destructive observation of samples at high magnifications. Thus, the development of microscopy raised the exciting prospect of being able to study the nanoscale morphologies of cells. However, because of the diffraction limit of microscopic imaging systems, their spatial resolution cannot be increased indefinitely by increasing magnification and eliminating aberrations (about 200–300 nm) [1].

In recent years, with the development of ultrafast optical techniques and highly sensitive probes in biosensors, various methods of super-resolution microscopy (SRM) have been developed to solve the diffraction limit problem [2,3,4,5,6,7,8,9]. The representative techniques are structured illumination microscopy (SIM), stimulated emission depletion microscopy (STED), reversible saturable optical-linear-fluorescence transitions (RESOLFT) [10,11,12,13,14], stochastic optical-reconstruction microscopy (STORM), and photoactivation localization microscopy (PALM) [15,16,17,18,19,20,21,22,23,24,25,26]. STED and RESOLFT confine the region of fluorescence using a high-powered red-shifted doughnut-shaped depletion beam [27]. STORM and PALM improve resolution by randomly turning on and off the luminescence of a single molecule in a diffraction-limited volume at different time points [26,28,29,30]. During the imaging process, STORM and PALM switch the excitation light to different wavelengths, repeatedly activating, exciting, and bleaching different fluorescent molecules, only recording a random subset of the fluorescent molecules in the field of view. Only one molecule emits light in the diffraction limit region, and subsequently, different molecules’ sets are recorded. Finally, a super-resolution image is reconstructed. In some cases, these techniques can provide spatial resolutions below 10 nm. However, these imaging techniques also have associated limitations, such as low time resolution and specific requirements for fluorescent dyes.

It is worth mentioning that except in the case of photobleaching resistance, the super-resolution imaging technique of SIM involves no special requirements in terms of the fluorescence dyes for the labeled sample. In SIM, the sinusoidal excitation light is superimposed on the sample, which then passes through the objective, and the observed fluorescence emission takes the form of a Moiré fringe. The Moiré fringe contains the mixed frequency of the excitation light and the sample, together with spatial frequency information. Using a known spatial frequency and phase shift in the structured illumination pattern, the spatial frequency information in the image is separated and reconstructed, and super-resolution is achieved through the SIM reconstruction algorithm. Only nine images are normally required to obtain super-resolution images. Thus, SIM presents unique advantages over other SRMs: (1) SIM has no special requirements for dyes, and it can be guaranteed to excite fluorescence to meet imaging standards. (2) SIM requires only a small number of raw images. Linear SIM requires just nine images for reconstruction, and its imaging speed is faster than those of PALM and STORM, which require hundreds of raw images.

However, SIM is far from perfect, and, within the confines of SIM itself, one can only improve certain aspects of the performance. To obtain more detailed sample structural information and depth, achieve multi-functional imaging, and render SIM more adaptable to wider applications, researchers have developed correlative microscopy technologies that can compensate for the shortcomings of existing SIM techniques. It is worth noticing that the development of correlative SIM can also promote the development of the biosensors; in particular, some protein tags and components of biosensors can also be utilized for multicolor microscopy and deep tissue imaging [31].

In this review, we focus on the research progress related to the correlative SIM. First, a brief review of the principles of SIM and correlative microscopy are presented. Next, we discuss the current state-of-the-art techniques in this field, including SIM with other SRM techniques, SIM with other microscopy techniques, correlative SIM for deep learning, correlative SIM for surface plasmon interference, and correlative SIM for quantitative phase imaging (Figure 1). Finally, we analyze the challenges to be overcome for the future development of this field. We believe that this review will provide an overview of representative developments of this field, together with a useful perspective and guidance for the development of correlative structured illumination microscopy.

2. Principle of Structured Illumination Microscopy

A well-known tenet of the theory of optical imaging is that diffraction prevents the precise convergence of light: sharp points on an object will be blurred into finite-size spots (Airy disks) in the image [32]. The three-dimensional intensity distribution of the Airy disk is defined as the point spread function (PSF) of the microscope [33]. The resolution of optical microscopic imaging techniques is limited by the system cutoff frequency of the optical transfer function (OTF) [34,35], where the OTF and PSF are a Fourier transform pair. In general, under wide-field illumination conditions, the frequency components that are higher than the cutoff frequency are filtered out, while components that are lower than the cutoff frequency can be transmitted.

SIM can employ periodically structured illumination to excite a sample to mix high spatial frequency details with low-frequency signals to overcome limitations imposed by optical diffraction [36,37]. When a sample is excited by the excitation light in an optical microscope (Figure 2A), the low-frequency information within the OTF passband can be detected, where $\left|k_0\right|$is the cutoff frequency of the system (Figure 2D), and information higher than the cutoff frequency will be lost. In SIM, the sinusoidal excitation light is superimposed on the sample, and when this is convolved through the objective lens, the observed fluorescence emission takes the form of a Moiré fringe pattern. This pattern represents a mix of frequencies from both the excitation light and the object, with frequency information higher than the cutoff frequency [38,39].

We can express the intensity of the sinusoidal illumination pattern $I(\overrightarrow{r})$ as (Figure 2B):

$$I(\overrightarrow{r})=I_0[1+\cos (2\pi {\overrightarrow{k}}_a\overrightarrow{r}+\phi )]$$

(1)

where $I_0$ is the average intensity of the structured illumination, $\overrightarrow{r}$ represents the polar position, $\phi$ is the phase of the structured illumination, and ${\overrightarrow{k}}_a$ is the spatial frequency of the structured illumination.

The sinusoidal illumination pattern excitation produces a fluorescence response in the sample$B(\overrightarrow{r})$. The raw image,$S(\overrightarrow{r})$, which is projected onto the detector, is then blurred by the PSF (convolution with PSF, convolution symbol is $\otimes$) (Figure 2C):

$$S(\overrightarrow{r})=[I(\overrightarrow{r})\otimes PSF\times B(\overrightarrow{r})]\otimes PSF(\overrightarrow{r}).$$

(2)

SIM improves the resolution that is obtained using Fourier transformation. Thus, the Fourier transform of sinusoidal illumination pattern $I(\stackrel{\rightharpoonup }{k})$ purely consists of delta pulses [40] and is expressed as (Figure 2E):

$$I(\stackrel{\rightharpoonup }{k})=I_0[\delta (\stackrel{\rightharpoonup }{k})+\frac{1}{2}\delta (\stackrel{\rightharpoonup }{k}+{\overrightarrow{k}}_a)\exp (-i\phi )+\frac{1}{2}\delta (\stackrel{\rightharpoonup }{k}-{\overrightarrow{k}}_a)\exp (i\phi )].$$

(3)

The polar position $\overrightarrow{r}$ in the spatial domain becomes the frequency variable $k$ in the Fourier domain.

Assuming that the frequency space of the system S(k) is expressed as (Figure 2F):

$$\begin{gathered} S(\stackrel{\rightharpoonup }{k})=\left[(I(\stackrel{\rightharpoonup }{k})\times OTF(\stackrel{\rightharpoonup }{k}))\otimes B(\stackrel{\rightharpoonup }{k})\right]\times OTF(\stackrel{\rightharpoonup }{k})= \\ {I}_0\left[B(\stackrel{\rightharpoonup }{k})+\frac{1}{2}B(\stackrel{\rightharpoonup }{k}+{\overrightarrow{k}}_a)+\frac{1}{2}B(\stackrel{\rightharpoonup }{k}-{\overrightarrow{ k}}_a)\exp (i\phi )\right]\times OTF(\stackrel{\rightharpoonup }{k}). \end{gathered}$$

(4)

Note that PSF and OTF are Fourier transform pairs of each other. The PSF that is convolved in real space is present in Equation (2), and the multiplication operation with the OTF in Fourier domain is contained in Equation (4). Equation (4) reveals the key to SIM:$I(\stackrel{\rightharpoonup }{k})\otimes B(\stackrel{\rightharpoonup }{k})$ results in a shifting of $B(\stackrel{\rightharpoonup }{k})$ to $\pm {\overrightarrow{k}}_a$ (Figure 2F). Although the frequency $k_a$ is also limited by OTF, it does contain higher spatial frequencies.

The spectral information obtained by the SIM system is a mixture of high-frequency and low-frequency information. Each spectral component must be separated by the algorithm. Then, the high-frequency information is moved back to its original position. Since Equation (4) contains three unknown components $B(\stackrel{\rightharpoonup }{k}),B(\stackrel{\rightharpoonup }{k}+{\overrightarrow{k}}_a),B(\stackrel{\rightharpoonup }{k}-{\overrightarrow{k}}_a)$, at least three independent equations must be solved. The phase $\phi$ of the structured illumination pattern is the most important modulation parameter. Three patterns with periodic equidistant initial phases $\left\{{\phi }_1,{\phi }_2,{\phi }_3\right\}$ are selected to excite the sample (Figure 3A), and the corresponding frequency-domain outputs of $S(\stackrel{\rightharpoonup }{k}),S_1(\stackrel{\rightharpoonup }{k}),S_2(\stackrel{\rightharpoonup }{k})$are recorded.

The separation matrix is constructed to separate the three unknown components, and the corresponding frequency-domain output is obtained from the linear Equation (5):

$$\left[\begin{array}{c}S(\stackrel{\rightharpoonup }{k})\\ S_1(\stackrel{\rightharpoonup }{k})\\ S_2(\stackrel{\rightharpoonup }{k})\end{array}\right]=I_0\left[\begin{array}{ccc}1& 0.5e^{-i{\phi }_1}& 0.5e^{i{\phi }_1}\\ 1& 0.5e^{-i{\phi }_2}& 0.5e^{i{\phi }_2}\\ 1& 0.5e^{-i{\phi }_3}& 0.5e^{i{\phi }_3}\end{array}\right]\left[\begin{array}{c}B(\stackrel{\rightharpoonup }{k})\\ B(\stackrel{\rightharpoonup }{k}+{\overrightarrow{k}}_a)\\ B(\stackrel{\rightharpoonup }{k}-{\overrightarrow{k}}_a)\end{array}\right]·OTF(\stackrel{\rightharpoonup }{k}).$$

(5)

The separated high-frequency information is first multiplied by the shift factor to move it back to its original position. It is reset to obtain the frequency information of the sample$B’(\stackrel{\rightharpoonup }{k}),B’(\stackrel{\rightharpoonup }{k}+{\overrightarrow{k}}_0),B’(\stackrel{\rightharpoonup }{k}-{\overrightarrow{k}}_0)$as follows:

$$B’(\stackrel{\rightharpoonup }{k})=B(\stackrel{\rightharpoonup }{k})$$

(6)

$$B’(\stackrel{\rightharpoonup }{k}+{\overrightarrow{k}}_0)=ℱ\{[ℱB(\stackrel{\rightharpoonup }{k}+{\overrightarrow{k}}_a)]exp(-i2\pi {\overrightarrow{k}}_a\overrightarrow{r})\}$$

(7)

$$B’(\stackrel{\rightharpoonup }{k}-{\overrightarrow{k}}_0)=ℱ\{[ℱB(\stackrel{\rightharpoonup }{k}-{\overrightarrow{k}}_a)]exp(+i2\pi {\overrightarrow{k}}_a\overrightarrow{r})\}.$$

(8)

Then, the high-frequency information obtained after separation is superimposed on B(k). Thus, the frequency-domain information of the sample in a single direction has been expanded to improve the resolution (Figure 3C). Finally, the range of the OTF is extended to $[(-{\overrightarrow{k}}_a-{\overrightarrow{k}}_0),({\overrightarrow{k}}_a+{\overrightarrow{k}}_0)]$.

As mentioned above, the frequency-domain expansion theory of structured illumination only represents expansion in a single direction. To improve the resolution isotropy and realize frequency expansion in two dimensions, it is necessary to record samples of structured illumination modulation in different directions to achieve spectrum expansion in multiple directions; see Figure 3D.

The final reconstruction structure of linear SIM extends the cutoff frequencies of the spectrum from $({\overrightarrow{k}}_0,-{\overrightarrow{k}}_0)$to $({\overrightarrow{k}}_0+{\overrightarrow{k}}_a,-{\overrightarrow{k}}_0-{\overrightarrow{k}}_a)$; see Figure 3B. Since linear SIM is limited by the diffraction limit (${\overrightarrow{k}}_0\ge {\overrightarrow{k}}_a$), the spatial resolution cannot be more than doubled using this technique.

In addition to traditional wide-field SIM, SIM has also been implemented in the form of “spot-scanning” [41,42,43,44,45,46,47]. In spot-scanning SIM, the structured illumination pattern is often generated by galvanometer scanning a focused light; then, it is projected onto the detector with a 2D CCD/sCMOS camera or PMT to record the structured illumination images. The fringes are formed with time-focused light that has a much greater penetration depth than that in wide-field illumination. Spot-scanning SIM used in combination with multiphoton excitation can achieve greater imaging depths and signal-to-noise ratios (SNR) (the ratio of power of the useful signal to noise in the optical system) [48,49,50,51].

3. Research Progress in Correlative Structural Illumination Super-Resolution Microscopic Techniques

3.1. Correlating SIM with Other SRM Techniques

STORM can achieve higher spatial resolution in the range of 10 nm utilizing single molecule localization [17]. However, STORM data are incomplete without clear structural information to place it into the cellular context. In this regard, correlating SIM with STORM is an excellent way to analyze single molecule localization data (SIM-STORM). Virginie Hamel et al. [52] used this strategy to realize multicolor imaging of cells from the U2OS cells. A dual-channel two-color imaging system was constructed with the channels using different laser excitation wavelengths to image the same region using SIM and STORM (Figure 4A). After acquiring and reconstructing the SIM and STORM images, it is necessary to ensure that the precise molecular localization provided by STORM is placed in the structural background of SIM (Figure 4B). However, the reconstructed image has a different field of view and pixel size, which creates difficulty for reconstructing a SIM-STORM image. To further reconstruct the SIM-STORM image, they used ImageJ plugin TurboReg to perform sub-pixel registration based on intensity [53]. In the plugin, they set the STORM image as a reference while manually scaling the SIM image until the two could overlap. The authors utilized this system to image the triply stained U2OS cells to study their structure. The F-actin network (Figure 4C), microtubule network (Figure 4D), and myosin (Figure 4E) of the U2OS cells were imaged using different SIM wavelengths, and the related structural information was obtained and reconstructed. Switching to STORM mode, the precise molecular location of myosin in the same region was obtained (Figure 4F). Finally, the four images were merged to generate a multicolor image (Figure 4G). Thus, correlative SIM-STORM provides a valuable solution for accurately mapping specific structures in their cellular context.

In addition to multicolor labeling and accurate positioning information for biological structural features, 3D nanoscale imaging of the organization and functionality of live cells is essential for biological research. In some cases, it is also helpful to correlate other SRM techniques with SIM for 3D imaging. The main operational principle of 3D structured-illumination microscopy (3D-SIM) is the use of a transmission phase grating within the system to diffract the excitation light into three beams and directly irradiate the sample [54]. The interference among the three beams generates 3D spatial–structural patterns [55]. Although 3D-SIM is based on a wide-field technique, it has the advantage of high imaging speed and does not require any special fluorescent dyes. However, the optical resolution of 3D-SIM is still limited to approximately half the fluorescence wavelength, and 3D-SIM can generate structured illumination only in one radial direction as well as the axial direction rather than all three dimensions. This leads to a lack of information in 3D-SIM reconstruction, which is referred to as the “missing cone” [56].

To avoid optical resolution limitations, scientists have proposed the concept of nonlinear structured illumination microscopy (NL-SIM), which is essential for further extending the cutoff frequency of SIM. The nonlinear response in each fluorophore, which is excited by the structured illumination and superimposed on the sample, generates high-spatial-frequency components. Theoretically, the number of higher harmonics and information components resulting from the high-spatial-frequency components is unlimited.

In general, NL-SIM methods can be classified into two types. The first type involves producing higher harmonics using saturated excitation light [57,58]. In this type of method, by harnessing the saturated excited state of the molecule for imaging, a spatial resolution of 50 nm can be achieved. Although the resolution can theoretically be increased to infinity, the experimental results indicate that the SNR and photobleaching limit the actual resolution of this type of NL-SIM. Most fluorescent molecules exhibit poor photostability under light intensities corresponding to saturated excitation; hence, biological tissues cannot be imaged. The second type of NL-SIM method achieves nonlinear effects through photoswitching [59,60,61]. This method uses photoswitchable fluorescent molecules to improve the optical resolution under low excitation laser power conditions. However, this approach has disadvantages, such as high photostability requirements for the dyes and slow switching times [59,62].

The principle of STED is similar to that of photoswitching, and the fluorescence is selectively depleted by the STED effect (via a stimulated emission depletion process). Moreover, STED depletes the fluorescence under a lower power than that in saturated NL-SIM, and the switching time is faster than that achievable with photoswitchable fluorescent proteins [63]. Therefore, the correlation between SIM and STED measurements may be more meaningful in improving the optical resolution in 3D-SIM imaging.

Fumihiro Dake [64] theoretically predicted the feasibility of 3D SIM-STED. The difference between this system and traditional 3D-SIM is that structured excitation and a structured STED pattern are generated by three-beam interference. In 3D SIM-STED, interference (caused by a diffraction grating) between the structured excitation (${\lambda }_{ex}$) and the two-structured illumination of the structured STED beams (${\lambda }_{STED}$) creates a 3D interference pattern in the sample in which the pitch and orientation are equal. Since the structured patterns of excitation and STED have phase differences, the structured STED pattern “turns off” the fluorescence around the structured excitation pattern via stimulated depletion. In contrast, owing to the STED effect, the structured patterns disappear, compensating for an imperfect overlap. Consequently, background fluorescence can be reduced. Thus, the phase difference is the key to realizing 3D SIM-STED; it not only determines the amount of quenched fluorescence at each lateral position, which plays a vital role in the nonlinear effect, but also effectively suppresses background fluorescence. The nonlinear effect of this STED technique efficiently enhances the strength of the nonlinear harmonics. Figure 5 illustrates the 3D SIM-STED illumination pattern distribution over the spatial frequencies. The two structured patterns are produced by the incident beams on similar gratings, in which the lateral pitches are designed to be the same; however, the axial pitches are not completely identical owing to the difference in wavelength. The spatial frequencies of the STED beams are distributed closer to the circumference than those of the excitation beams. 3D SIM-STED introduces more harmonics from the spatial frequency images than the traditional 3D-SIM, resulting in an enhanced optical resolution. In contrast, the optical power required by 3D SIM-STED is lower than that required by the traditional STED. This is because 3D SIM-STED does not need to deplete the fluorescence completely, but just enough to produce nonlinear effects, which can effectively inhibit photobleaching.

The nonlinearity of the STED effect correlates with the 3D-SIM, where a structured STED pattern and a structured excitation pattern efficiently induce nonlinearity and further improve the optical resolution. However, this does not solve the problem of the “missing cone.” Yi Xue et al. [65] proposed 3D SIM-STED based on the interference among five beams, resulting in a 3D grid depletion pattern; this approach is called 3D 5-SIM-STED. This pattern can generate symmetric SIM patterns radially in comparison with those in a single radial direction of 3D SIM-STED, thus serving as an excellent solution to the “missing cone” problem of 3D SIM-STED.

The system setup is illustrated in Figure 6A. The STED depletion pattern is diffracted into five beams using a diffractive optical element (DOE), and a spatial light modulator (SLM) is used to rapidly generate the illumination patterns. The excitation patterns generate wide-field illumination on the image plane. 3D 5-SIM-STED is inspired by the SIM wide-field technique and uses the 3D grid depletion pattern to generate a donut-shaped spot matrix, which has a higher imaging speed than those of the STED techniques using the point-scanning method (Figure 6B).

For filling the “missing cone” of widefield illumination, the interference pattern can compensate for the “missing cone” through convolution of the modulation transfer functions (MTFs) of wide-field microscopy, and five beams produces 17 spatial frequency components [55,66,67]. MTF represents the magnitude of OTF and measures the spatial frequency response of the system [68] (Figure 6C). In comparison with 3D SIM-STED, the multiple beam interference of 3D 5-SIM-STED can generate structured illumination in multiple dimensions.

In addition to the techniques described above, correlative imaging involving more than one type of SRM modality has also been proposed: for example, the combined application of single-molecule localization microscopy (SMLM). The first implementation of imaging the same region of H3K293 cells with a high structural resolution was achieved using SMLM by Sabrina Rossberger et al. [69]. In addition, Han Zhang et al. [70] analyzed and predicted the feasibility of an SIM-STED technique with surface plasmon resonance enhancement to achieve high-speed imaging with a 30 nm resolution and sub-second acquisition time.

3.2. Correlating SIM with Other Microscopy Techniques

3.2.1. Correlating SIM with Total Internal Reflection Fluorescence Microscopy Techniques

Total internal reflection fluorescence microscopy (TIRF) is a technique with good optical sectioning capability [71,72,73]. TIRF uses an evanescent field localized at the boundary between refractive-index media to excite fluorophores within a thickness of one wavelength from the coverslip surface. Its superb background rejection and sensitivity can significantly improve the SNR, and it also has the advantages of high temporal resolution and low phototoxicity fluorescence imaging. However, conventional TIRF enhances temporal resolution at the cost of spatial resolution, reducing its effectiveness in studying dynamic phenomena. SIM requires the collection of a small amount of raw data to achieve super resolution. Therefore, it is an excellent compromise between a moderate increase in spatial resolution and a tolerable temporal resolution loss. The correlative of SIM with TIRF (SIM-TIRF) measurements is an effective approach to realize living cell imaging.

The imaging speed of SIM-TIRF is essential to record the process of living cells. Two main factors affect the speed of SIM-TIRF. First, there is the switching time between different phases and illumination angles of the SIM. Second, there is the acquisition time for each individual raw image. Compared with increasing the fluorescence lifetime of the sample, researchers are generally more enthusiastic about the possibility of improving the system performance to achieve fast illumination switching [74,75,76,77,78].

Chung et al. [79] first proposed correlating TIRF with SIM technology. They achieved a resolution of approximately 100 nm, which is twice the resolution of conventional TIRF microscopy. Kner et al. [74]. further developed this technology. They used an SLM instead of traditional SIM mechanical modulation, enabling switching of the excitation light mode in microseconds. SIM-TIRF using an SLM allowed the recording of the trajectory of FGFP-α-tubulin in S2 cells at a frame rate of 11 Hz, over hundreds of time points, with a resolution of 100 nm. Then, Laurence J. Young et al. [77] proposed an SIM-TIRF system that realizes automatic control of the polarization state and spatial structure of the illumination patterns, with imaging up to 10 Hz in three colors. An SLM grating with a circular aperture is used to align the excitation path. Only a perfectly aligned beam produces a regular circle of illumination in the image; otherwise, artifacts are produced. The setup also incorporates a liquid crystal variable retarder (LCVR) to control the linear polarization state of the excitation illumination for polarization control and synchronization [80,81,82]. While further improving the imaging speed of SIM-TIRF, it also improves the SNR. Various studies have verified the excellent contrast of the SIM-TIRF method for live cell imaging and the visualization of subcellular structures. For example, emGFP-labeled microtubules and cytosolic GFP have been imaged.

To achieve super-resolution imaging of low-fluorescence biological samples with a several hertz acquisition rate, Roth et al. [75] introduced an SIM-TIRF system based on a scanning mirror and a Michelson interferometer (Figure 7), which can generate images with a spatial resolution of 110 nm and a temporal resolution of up to 8 Hz. By optimizing the illumination interference contrast, high resolution can be achieved, even for moving samples with weak fluorescence. This system is suitable for weakly fluorescent samples and is inherently dynamic. Therefore, this super-resolution technology can be used to analyze biological structures. For example, the imaging of dynamic proteins such as MreB in bacteria and actin in eukaryotic cells has been demonstrated [83,84].

The combination of SIM and TIRF is an effective approach to improve optical-sectioning capability and temporal resolution simultaneously. Thus, it is possible to gain new comprehensive insights into intracellular dynamics.

In addition to the continuous optimization of improving the system performance to achieve fast illumination switching. SIM-TIRF also has an interesting correlation method, which is the correlative of SIM with a photonic chip TIRF (SIM-cTIRF) [85]. SIM-cTIRF used a planar photonic chip that replaced the glass slide. The integrated waveguide chip is used to guide the excitation illumination so that it reaches the sample directly without passing through the objective lens; the fluorescence information is collected by the objective lens. Therefore, the spatial frequency of the illumination pattern is no longer limited by the numerical aperture (NA) of the objective lens, allowing the use of a low-NA objective lens to increase the field of view without sacrificing SIM resolution. Helle et al. [85] proposed utilization of the high refractive index silicon nitride as a planar photonic chip to image the marker-labeled primary liver sinusoidal endothelial cells. This proves that compared with the traditional SIM, the SIM-cTIRF resolution increases by 2.3 fold, while the field of view is also widened. Moreover, wide-field imaging has no mechanism to remove out-of-focus image blur from scattered light. However, the SIM-TIRF can effectively suppress the out-of-focus blur; it can be utilized to conduct biological imaging of PI 3-kinase biosensor localization [86]. Similar protein biosensors such as Grx1-roGFP2 green fluorescent protein biosensor [87] and C-Py biosensor [88] can also be detected by other correlative SIM to increase their sensitivities.

3.2.2. Correlating SIM with Two-Photon/Second Harmonic Generation Microscopy Techniques

Although super-resolution microscopy has made significant progress in improving spatial resolution and imaging speed, it is still plagued by background scattering and defocusing problems in thick samples. Two-photon microscopy (2P) is an ideal choice for thick samples because it can achieve deep penetration into samples [89,90,91]. However, with increasing penetration depth, increased light scattering will reduce the image resolution and contrast [92]. In an attempt to enhance the imaging depth in SIM by reducing the scattering of the excitation light, a 2P excitation scheme has been used to excite fluorescent samples (2P-SIM).

Ben E. Urban et al. developed a 2P-SIM [93] method for nanoscopic imaging of ganglion cell dendrites in ground squirrel retina at a depth of 100 µm from the vitreal surface (Figure 8B). The excitation scheme improved the lateral resolution, by a factor of 1.9, as well as the SNR. The illumination pattern is generated by using the temporal modulation of the excitation intensity and spatial scanning instead of wide-field illumination, which avoids the power problem of wide-field illumination [94,95]. The optical setup for 2P-SPIM is shown in Figure 8A.

Second harmonic generation (SHG) enables the direct imaging of biological tissues that are non-centrosymmetric [96,97]. In addition, the SHG imaging modality is easy to implement within a 2P imaging modality setup. The difference is only in the optical filter selection and detector placement. Hence, taking advantage of the similarity, Chia Hua Yeh et al. [98] proposed correlating SHG with SIM (SHG-SIM), based on laser scanning, and demonstrated resolution improvement using chicken tendon and mouse skin samples (Figure 9). Compared with traditional SHG microscopy, imaging resolution in highly scattering biological tissues is improved by a factor of approximately 1.4.

While most of the reported applications of other microscopy techniques correlated with SIM have involved some form of NL-SIM, in recent years, the breadth as well as the number of applications has increased. For example, SIM has been correlated with lattice light-sheet fluorescence microscopy (SIM-LLSM). LLSM has orthogonal independent excitation and detection devices. It uses objective excitation to project a thin sheet of light through a specimen. The advantage of LLSM is that it confines the light to a plane to eliminate the defocused background. However, previous reports detailed the optimization of the light sheets by patterned activation or focusing a Bessel beam [99,100]. So far, a significant challenge with LLSM is to obtain multiple parallel sheets of light [101,102]. Hence, Panchen Gu et al. [103] reported a novel method of using the isosceles triangular array (ITA) to produce multiplane parallel light sheets. The ITA was projected onto the SLM as a phase mask, and the thickness and distance of the light sheet can be adjusted by changing the ITA period. To explore the relationship between molecular function and biological nanostructure, Frederik Görlitz et al. [104] presented correlated SIM and wide-field optically sectioned fluorescence lifetime imaging microscopy (SIM-FLIM). They demonstrated the capability of SIM-FLIM to provide super-resolution images of cell morphology with colocalized fluorescence lifetime readouts by imaging of discoidin domain receptor 1 in Cos 7 cells following ligand stimulation [105,106,107] and the compaction of DNA during the cell cycle [108]. SIM-FLIM adds another powerful orthogonal dimension to SIM. In contrast to SIM-FLIM, correlating SIM with fluorescence nuclear track detector confocal microscopy (SIM-FNTD) provides particle energy distribution information based on alpha radiation spectroscopy [109].

3.3. Correlating SIM with Other Techniques

3.3.1. Correlating SIM with Deep Learning (DL-SIM)

With the rapid development of correlated SIM microscopy techniques, fruitful results have been recently achieved with regard to improving speed and resolution. However, there have been few reports regarding another limitation of SIM: obtaining a series of structured illumination pattern images requires multiple acquisitions to generate each high-resolution image, which accelerates photobleaching and increases the scanning time. Recently, there has been an explosion of deep learning (DL) applications in the fields of biological morphology analysis, SNR improvement, and super-resolution imaging [110,111]. DL has also been applied on raw images to increase the image quality and resolution of images [112]. Since DL has a notable advantage in increasing the speed of SIM by reducing the number of raw images, the adoption of DL for reconstructing SIM images has shown great promise in improving the time resolution.

In order to minimize photobleaching and increase imaging speed, Luhong Jin et al. [113] applied deep learning methods to reconstruct images using deep neural networks trained on raw images in the spatial domain. DL-SIM requires only a few raw images to achieve multicolor live-cell SRM under low-light illumination conditions.

The process of DL-SIM is as follows: First, the ability of convolutional neural network architectures (U-Net) in DL to reconstruct super-resolution images is tested [114,115]. Linear SIM requires nine raw images for 2D-SIM with two-beam interference and 15 raw images for 3D-SIM with three-beam interference. The U-Net requires 15 SIM raw images as the input and the corresponding SIM reconstruction results as the ground truth, and it is named U-Net-15 [113]. Second, the U-Net-15 is trained continuously to reduce the number of input raw images to three, while the ground truth is still the result of SIM reconstruction of 15 raw images, and it is named U-Net-3 [113]. This method uses three raw images to reconstruct super-resolution images, which is five-fold less than that in conventional SIM. The schematic of U-Net is shown in Figure 10A. Finally, in order to recover the signal in low light, another U-Net is trained, named U-Net-SNR. Thus, a new DL network architecture named scU-Net was established. The scU-Net links U-Net and U-Net-SNR through skip-layer connections to realize SRM under low-light illumination (Figure 10B) [116].

The reconstruction results of the microtubules indicate that the lateral resolution of the SIM image reconstructed using U-Net-15 and U-Net-3 is comparable to SIM reconstruction (Figure 11A). As shown at Figure 11B, both U-Net-SIM15 and scU-Net have better reconstructed images than the conventional SIM in low-light illumination, and scU-Net can retrieve details missed by U-Net-SIM15. The ability of DL-SIM reconstruction results for microtubules in living cells imaging was verified (Figure 11C,D).

Although scU-Net can be used to detect structural differences in the spatial domain to reconstruct SIM images, this DL-SIM method cannot be utilized under every imaging condition. In particular, when better and quantifiable spatial distributions are required for nanoscale structures, this method is not appropriate. Therefore, Chang Qiao et al. [117] devised a deep Fourier channel attention DL network (DFCAN) and its derivative trained with generative adversarial network (GAN) strategy, named DFGAN, which uses the inherent difference in the frequency content of distinct features in the Fourier domain to adaptively rescale their weightings to learn about the precise structural representations of various biological structures. This is another approach for obtaining high-frequency information of the sample [118,119,120,121].

For DFCAN, SIM raw images as used as input in the training phase. Subsequently, these raw images are fed into the Fourier channel attention (FCA) mechanism in the DFCAN network architecture [122]. The role of FCA is to enable the network to compute the rescaling factors according to the comprehensive contribution of all frequency components contained in its power spectrum [123]. Then, the image is made the same size as the ground truth image to analyze the high-frequency information. Finally, the network outputs a super-resolution image. For DFGAN, this method is inspired by the conditional generative adversarial network (cGAN) [124] framework. cGAN consists of two parts: the generative model G, which is used to learn the data distribution and perform image transformation, and the discriminative model D, which is used to distinguish whether the image comes from training data or is generated by the generator G. In the DFGAN framework, with DFCAN as G, D takes G or the ground truth image as input and provides a score that reflects the probability of the input image being the ground truth. Thus, the purpose of G is to make D provide the ground truth as its output, and in contrast, the purpose of D is to determine whether the input is from G or the ground truth. G and D are cross trained to make them compete with each other and finally reach a state of equilibrium. Furthermore, SIM reconstruction images of F-actin by DFCAN, DFGAN, and scU-net are used to compare the three methods of SIM reconstruction ability. The result shows that DFCAN and DFGAN can clearly resolve the adjacent F-actin filaments. However, scU-net may sometimes fail to resolve them. Thus, DFCAN and DFGAN are significantly better than scU-Net in representing precise biological structures.

Then, the capabilities of DFCAN and DFGAN were verified via multicolor live cell imaging experiments. To investigate the interaction dynamics over an extended time course, raw SIM images with about ten-fold less fluorescence than that of ground truth were acquired. Despite the relatively low fluorescence, DFCAN- and DFGAN-SIM successfully resolve the ring structures of the CCPs and the fine branches of actin filaments for more than 400 time points, and there is no reconstruction artifact. It has been demonstrated that DFCAN and DFGAN can achieve an image quality equivalent to that achieved using SIM, which has an image acquisition time that is ten-fold longer. These experiments revealed the detailed structure of mitochondria cristae and nucleosides, the interaction between organelles, and cytoskeleton dynamics.

3.3.2. Correlating SIM with Surface Plasmons Interference (PSIM)

Surface plasmons (SPs) have the ability to confine optical waves into a subwavelength scale [125]. They can also produce interference patterns with higher spatial frequency when multiple SP waves overlap [126]. The wave vectors of the SPs can be expressed as |k_sp | = |k_photon|[(ε_m ε_d/(ε_m + ε_d)]^(1/2) [127], where k_photon is the wave vector of the excitation light, and ε_m and ε_d are the dielectric constants of the metal and dielectric, respectively. Therefore, k_sp is a function of the variables k_photon, ε_m, and ε_d. It is known from the function expression that the SP wave vector |k_sp| can be much larger than the corresponding |k_photon| by adjusting ε_m and ε_d. Figure 12A shows the dispersion curves of SPs (a semi-infinite-metal/dielectric interface (green) and a thin-metal-film/dielectric interface (blue)). All these SP modes have a higher wave vector than the corresponding excitation light. In addition to adjusting k_photon, ε_m, and ε_d, the SP mode can also be modulated by changing the thickness of the metal film (Figure 12A, blue line) [128]. Therefore, correlating SIM with surface plasmon interference is another novel way to further improve resolution.

Feifei Wei et al. [129] proposed two SP designs for PSIM: a thick-metal-film/dielectric structure (Figure 12B) and a thin-metal-film/dielectric structure (Figure 12C), which increases the resolution by factors of four and five, respectively, compared with conventional epi-fluorescence microscopy. Specifically, the first option involves the use of excitation light with wavelengths of 390 and 653 nm and different incident angles to irradiate a 100-nm-thick Ag film to excite the PSIM patterns (Figure 12B(a)). Figure 12B(b,c) illustrate the time-averaged electric-field energy density distributions of different excitation light wavelengths. Figure 12B(d,e) show the Fourier transforms of the SIM patterns excited by wavelengths of 390 and 653 nm, respectively. It can be seen that there are high-frequency components other than the zero frequency. The second SP design involves using 442 nm excitation light with a 508 nm emission wavelength. The time-averaged power density distribution measured under the Ag film spans 20 nm (Figure 12C(a)). The corresponding Fourier transform (Figure 12C(b)) has multiple high-frequency components. PSIM provides an opportunity to create periodical illumination to replace the traditional excitation light pattern and is suitable for monitoring near-field effects [129,130,131].

Inspired by PSIM, Anna Bezryadina et al. [125] proposed localized plasmonic structured illumination microscopy (LPSIM). LPSIM utilizes nanoscale plasmon antenna arrays to generate adjustable illumination patterns, which is used with an ultra-high numerical aperture objective. LPSIM can reduce the resolution to 50 nm while retaining high speed and low phototoxicity.

They fabricated several uniform hexagonal lattices using 60 nm diameter silver discs embedded in polymethyl methacrylate (PMMA) (Figure 13A(a−c)). By simulating the near-field excitation patterns of different incident illumination angles, the optimal size and spacing of the nano-disks were estimated with the high NA objective (Figure 13B). To characterize the resolving power, LPSIM and conventional fluorescent microscopy were compared for imaging green microtubes (488/550 nm) (Figure 14A) and the spatial frequency spectra (Figure 14B,C), and then, the full width at half maximum (FWHM) was calculated (Figure 14D,E). With the latest design of the LPSIM system, the nanoantenna pitch size has been reduced, which has pushed the resolution down to a few tens of nanometers.

The LPSIM method has further developed the correlating SIM for surface plasmon interference. However, in previous works, tuning the pattern of structured SP by varying the excitation angle or wavefront of incident light usually required a complex optical setup or expensive optical components such as Galvo scanners, digital micromirror devices, and high NA objectives [125]. Qilong Tan et al. [132] reported polarization tunable structured plasma field (SPF) structured illumination microscopy (SPF-SIM) to eliminate the need for mechanical components. This technique forms SPs using fishbone-shaped gold membranes. SPF is continuously moved by changing the linear polarization state of the incident beam. The fishbone grating array is shown in Figure 13C. Each fishbone is composed of two orthogonal nanoslits, and the direction forms an angle $\phi$ of $-\mathsf{\pi }/4$ and $\mathsf{\pi }/4$with respect to the y-axis [133,134]. Therefore, each nano-aperture can be regarded as a local subwavelength dipole antenna.

The specific method of forming sinusoidal fringes by SP is as follows: For a linearly polarized incident beam, two polarization states, namely left circular polarization (LCP) and right circular polarization (RCP), can exist. Their phase difference is $e^{\mathrm{i}2\theta }$. Therefore, the SP excited by each hole can be obtained as a particular circularly polarized component. Hence, there is a phase difference of 2$\theta$ between the left and the right going SPs. Under illumination by linearly polarized incident light, the SP on the left is excited by the LCP component, the SP on the right is excited by the RCP component, and the two components form an interference fringe (Figure 13D). Then, the linear polarization angle $\theta$ of the incident beam is continuously adjusted to change the relative phase of the two components to 2θ. Finally, a sinusoidal fringe is formed.

The imaging ability of polarization-controlled SPF-SIM was discussed by the same authors. In the simulations, the imaging sample was a fluorescent bead with a diameter of 50 nm. In addition, it was proposed to use the SPF-SIM system shown in Figure 13E, in which the liquid crystal (LC) polarizer was able to control the incident linear polarization angle quickly and accurately in three different directions. The simulation results showed that the resolution is 2.3 folds higher than that of conventional FM. SPF-SIM can tune the phase shift range of periodic SP from 0 to 2π by rotating the incident linear polarization direction without requiring any mechanical control.

3.3.3. Correlative SIM with Quantitative Phase Imaging Techniques

Quantitative phase imaging (QP) techniques are used to capture endogenous contrasts on unstained cellular samples, which can noninvasively probe the structural and biological characteristics of cells. These techniques have been widely used in the analysis of whole-cell morphology, mass, dispersion spectrum, and absorption/scattering, among others [134,135,136,137]. QP uses reference light to extract the phase information associated with the object from the intensity information. The methods used for achieving this can be classified into two types: phase-shifting and off-axis methods. The phase-shifting method involves changing the phase shift of the reference and recording images at different phase shift positions, after which all the recorded images are combined [138,139,140,141]. The off-axis method uses spatial modulation to generate phase information from a single intensity record. First, the reference light is tilted to create spatial modulation. Then, the obtained interference image is Fourier transformed, preserving a single piece of high-frequency information in the baseband, and filtering out the rest. Finally, the inverse Fourier transform of the processed spectrogram is performed to obtain the phase image [142,143,144].

Since QP operates on unlabeled specimens, it exhibits low phototoxicity and photobleaching; thus, it is complementary to fluorescence microscopy [134,135]. However, QP imaging has no molecular specificity and does not have the ability to locate the special structure of biological samples. In general, the two techniques are fundamentally different, and it is difficult to find an imaging method compatible with both. Chowdhury et al. [145] demonstrated the correlative SIM with quantitative phase imaging technique (SIM-QP) and achieved 3D SIM and QP multi-modal imaging. They used SIM in the frequency domain to maintain similarity with the QP off-axis method and realized compatibility between QP and fluorescence imaging. The schematic of the system device is illustrated in Figure 15A. Here, 488 nm single-mode broadband light serves the dual purpose of QP illumination and fluorescence excitation. The sinusoidal pattern written onto the SLM reaches the sample, and the sample information (including the diffraction information and fluorescence information) is collected via the objective lens. The fluorescence signal and diffraction signal of the sample are separated by a DM. The fluorescence signal of the sample is imaged onto the camera (CMOS-F), whereas the diffraction signal is passed through a diffraction-phase setup (DG) to divide the signal into different diffraction orders. The function of the mask (M) on the Fourier plane of the DG is to block the 1st-order diffraction while retaining only one piece of high-frequency information and spatially filtering the 0th-order diffraction to generate a uniform wavefront reference. Figure 15B details the achievement of the off-axis QP imaging using the mask. In the case of SIM, each diffraction order from the DG contains the diffraction orders produced by the SLM. Thus, M is asymmetrical with a pinhole filter at the center. The asymmetric structure is used to block the diffraction orders; it only allows the first-order diffraction, and the pinhole filter is used to filter out the zero-order diffraction from the SLM contained in the DG and generate the uniform wavefront reference required for the off-axis imaging. Finally, the phase shift of the SIM allows solving for the spatial frequency components. Then, these components are moved back to the appropriate region in the frequency space and then combined to acquire the QP image.

To demonstrate the multi-modal imaging performance of the system in a biological sample, Chowdhury et al. [145] labeled A549 cells with AlexFluor-488 phalloidin for F-actin visualization imaged QP and fluorescence (Figure 15C). As depicted in Figure 15C(a), the vesicles surround the apical portion of the cell with QP. Figure 15C(b) shows the visualization result of F-actin imaging with fluorescence. The biological structure information of QP imaging is different from the 3D visualization information of the molecular labeling components provided by the fluorescent signals.

Therefore, SIM-QP can achieve multi-modal imaging, which can simultaneously evaluate the morphology and other descriptions of specific cytological components, providing a multi-angle research direction for biological applications.

In addition, SIM-QP enables multi-modal imaging of complementary information on biological structures and multi-modal imaging of biological samples with 3D-SIM and refractive index tomography (IR) [146,147,148]. Cells alter the path of light, which can lead to aberrations in cell or tissue imaging. QP technology can measure the biophysical properties of these aberrations known as RI and improve our potential for understanding cells and tissues. Seungwoo Shin et al. [149] proposed and demonstrated a multi-modal approach for measuring both the RI and fluorescence distributions of live cells with correlative SIM-QP. To minimize the photobleaching and phototoxicity that occur in fluorescence imaging of live cells [150], the dispersion property of a DMD can be utilized to generate different modulation modes for QP illumination and SIM fluorescence excitation [151]. The two imaging methods use the same optical system but different modulation modes and excitation wavelengths. The correlation between 3D fluorescence and 3D RI data can be easily analyzed using QP technology. This method is expected to open up new avenues for research in cell biology for medical and diagnosis purposes.

In addition to the typical techniques described above, the correlating SIM techniques can also be implemented in many other ways [152]. Recently, Ana I.Gómez-Varela et al. [153] proposed the correlative SIM with a tip-scanning QI nanomechanical mapping atomic force microscopy techniques (AFM-SIM). AFM-SIM provides a powerful nanoscale observation tool that enables simultaneous colocalization imaging, allowing the simultaneous recording of nanomechanical data and visualization of cell dynamics. The correlating SIM techniques with spectroscopy also have co-location capability. Isotta Cainero et al. [154] proposed the correlative SIM with cross-correlation spectroscopy techniques (SIM-ICCS). They used cross-correlation spectroscopy to study and quantify colocalization analysis between multicolor images to have a more intuitive understanding of cellular processes and their interactions through the nano-scale spatial distribution of fluorescent probes. Moreover, in order to obtain ultrastructure information and 3D super-resolution imaging in cryopreserved samples, researchers have proposed a series of correlating SIM techniques. Ilias Kounatidis et al. [155] presented the correlative SIM with soft X-ray microscopy (SIM-X-ray). They used SIM-X-ray to perform 3D imaging of cells at low temperatures. Through the process of reovirus release from intracellular vesicles during the early stages of infection, they identified the intracellular virus-induced structures. In addition, Michael A. Phillips et al. [156] developed the correlative SIM with cryo-electron microscopy (SIM-cryo). They have proved that SIM-X-ray and SIM-cryo can offer good imaging quality even under low-temperature conditions.

4. Summary and Prospects

The super-resolution microscopy technique uses various ingenious methods to “bypass” the diffraction limit. It is a powerful tool for biomedical research. However, as an emerging technology, SIM still encounters multiple challenges when applied to biology, especially when the research object is dynamic and has a finite thickness, complex composition, and internal structures.

The correlative SIM techniques (as we summarize in

Table 1. Representative the correlative SIM techniques and their applications.

Type	Technique	Characteristic	Resolution	Application
SRM	STORM-SIM [52]	The accurate position information for cells provided by STORM is placed in the structural light background of SIM.	Lateral: 40 nm (STORM); 150 nm (SIM)	Multicolor imaging
	STED-SIM [64,65]	Achieving nonlinear effects through STED beams.	Lateral: 59.9 nm Theoretically; Axial: 163.2 nm Theoretically	3D imaging
Other microscopy techniques	SIM-TIRF [75]	Faster imaging speed for live-cell high resolution imaging.	Lateral: 110 nm	Multicolor, live-cell imaging
	2P-SIM [93]	Illumination pattern is generated by temporal modulation and spatial scanning of 2P excitation with larger sample penetration depth.	Lateral: 145 nm (imaging depth greater than 100 µm)	Thick sample imaging
	SHG-SIM [98]	Enables direct imaging of biological tissues that are non-centrosymmetric.	Lateral: 231 nm; Axial: 693 nm	SHG imaging
Other techniques	DL-SIM [113,117]	Fewer raw images are required to achieve multicolor live-cell SRM under low-light illumination conditions.	Lateral: 160 nm	Multicolor, live-cell imaging
	PSIM [125,129, 132]	Higher spatial frequency interference patterns is created by near field surface plasmon waves.	Lateral: 75 nm	Near-field 2D imaging
	QP-SIM [145,157,158]	Compatible with QP and fluorescence imaging.	Lateral: QP: 230 nm; SIM: 180 nm; Axial: QP: 210 nm	3D multi-modal imaging

) have overcome the above shortcomings. However, correlative SIM is not just a simple extension of traditional correlative microscopy. It can overcome multiple technical problems and solves the problem of the difference in resolution between optical microscopy and high-resolution methods, while adding a new dimension to the powerful SIM method.

Among the correlations between SIM and other SRM techniques, SIM-STORM uses dual-channel image registration to achieve multicolor imaging of biological structures. However, reconstructed SIM-STORM images require a complicated process because the SIM and STORM images must be generated in the same region of interest, but they all differ in terms of field of view and pixel size. To preserve high SIM-STORM reconstructed images, the SIM image size must be manually adjusted to ensure that the two images completely overlap. Thus, the SIM-STORM system can be further optimized on the reconstruction algorithm that can automatically generate an interest region and overlap the two images.

SIM-STORM is expected to help solve critical biological problems by accurately mapping the location of proteins within complex structures. Electron microscopes, which easily attain nanometer and sub-nanometer resolutions, can image more delicate biological structures. Recent developments of SRM have correlated it with other SRM techniques and electron microscopy methods, achieving good results [159,160,161]. Hence, future significant advancements could be achieved by the correlation of SIM-STORM and electron microscopy for multicolor imaging. This could potentially offer further improvements in the localization accuracy of a given protein within its biological context.

We introduced SIM-STED, which can effectively enhance the harmonics and expand the frequency cutoff at low optical power in theory. The improvement in resolution and the reduction in background fluorescence are theoretically compatible. However, SIM-STED cannot solve the aberration problem. In future research, the correlation of adaptive optics techniques and SIM-STED should further minimize aberration limitations caused by background fluorescence [162,163,164], can be truly applied to biological sample imaging, and can advance from the theoretical research stage. Another novel development direction is to generate an SIM pattern on STED excitation light to increase the resolution two-fold of STED.

In correlating SIM with other microscopy techniques, SIM-TIRF stands out. The superior optical slicing ability of TIRF microscopy can increase the frame rate achievable at each time point, providing a critical window on living specimens for physiological research. In future, SIM-TIRF technology will be extended to the imaging of biological tissues with more complex structures and morphologies, such as macromolecular yeast nuclear–pore complexes [165,166]. Although theoretical research on the yeast nuclear–pore complex has reached an advanced stage, experimental verification remains challenging [167,168,169]. Another meaningful research area in the future is the SIM-cTIRF. It has been demonstrated that SIM-cTIRF enables the use of a low-NA objective lens to increase the field of view without sacrificing SIM resolution; however, challenges remain with this method. For example, the power loss of the waveguide chip results in reducing the visibility of fringes. There are many reasons for power loss, such as hot spots, process defects, etc. In future work, it is possible to reduce power loss by designing different waveguide chip geometries and discovering more suitable materials for waveguide chips. More importantly, waveguide chips can manipulate light, providing new ideas for beam shaping. Photonic chips may be used to replace mechanical modulation, which opens up an unprecedented new path for the high-speed imaging of living cells.

The spatial scanning technique provided a solution to realize 2P excitation. Correlative 2P-SIM and SHG-SIM can effectively suppress background fluorescence and is suitable for imaging thick samples. Several 2P-SIM and SHG-SIM methods have been reported in recent years. However, most involve filtering out second harmonic frequency components, thus providing limited lateral resolution enhancement. Therefore, adding high-frequency information components to the sample to further improve the resolution is an important frontier for future development. Multifocal excitation and line scanning modulate the fringe grating pattern rapidly to improve the time resolution.

DL-SIM correlates SIM with a deep neural network trained on raw images to visualize specific complex cell compartments, such as microtubules, mitochondria, etc. This has been applied to realize multicolor, live-cell super-resolution imaging of these cell compartments. Despite the impressive results achieved using current DL-SIM methods, it is noteworthy that image transformation is essentially unstable. Although DL-SIM uses a significant amount of raw data for training to achieve good statistical transformation, it is impossible to obtain SRM images of every detail. Regardless of which DL algorithm is adopted, asymptotic stability relative to the details of all aspects of the acquired image is relatively straightforward to achieve, even though the ideal value cannot be attained.

Correlating SIM with SPs realizes SRM imaging with low phototoxicity and high-speed capability by exciting light with appropriate frequencies on suitable custom-designed metal structures. As a result of the SPF-SIM precise phase modulation of SPs without mechanical control, this method is expected to achieve a wide range of applications, including chemical analysis and maskless lithography. In spite of its unique advantages, the technique also has its intrinsic limitations. It can only form high-resolution images of objects close to the metal surface; thus, it is only suitable for 2D imaging and cannot be extended to the 3D field.

Multi-mode compatible techniques correlating SIM with QP imaging realized the combination of the coherent and fluorescence imaging system in the SIM framework. The synergy for the two techniques enables comprehensive study of biological components with different molecular and biophysical/biochemical functions. SIM-QP can offer a unique ability to the application of whole cells imaging, such as cell growth, differentiation, locomotion, cytokinesis, and apoptosis [170,171,172]. In future work, SIM may have multi-mode compatibility with imaging modes apart from QP. For example, photo-acoustic tomography (PAT) [173,174] is an ultrasonic imaging technique. In the past decade, ultrasonic imaging has been one of the most attractive techniques in the biomedical field, and it is widely applied to clinical imaging, such as magnetic resonance imaging [175], position emission tomography [176], computed tomography, etc. [177,178,179,180]. This technique, excited by thermoelastic expansion, is used to reconstruct optical absorption distribution. Compared with QP imaging, the scattering of PAT imaging is two to three orders of magnitude weaker in samples. Consequently, the correlative SIM with PAT can provide good resolution with non-invasive imaging of internal structure and function, which is a good future development prospect.

In addition, the wide popularity of correlative SIM in biological and chemical sensor also benefits greatly from its three distinct advantages, i.e., target specificity, compatibility with live samples, and wide field of view. Yet, the resolution quantitative imaging of fluorescent biosensors [181,182,183] is still challenging. Therefore, the development of a variety of protein fluorescent probes is also vitally important. In conclusion, alongside the progress in sample preparation, image processing, and other related technologies, correlative SIM will provide more increasingly meaningful data for biomedical research and create new insights for nanoscale research in the future.