Dynamic Imaging Using Ptychography

Jesse N. Clark, Xiaojing Huang, Ross J. Harder, and Ian K. Robinson London Centre for Nanotechnology, University College London, London WC1E 6BT, United Kingdom National Synchrotron Light Source II, Brookhaven National Laboratory, Upton, New York 11973, USA Advanced Photon Source, Argonne National Laboratory, Argonne, Illinois 60439, USA Research Complex at Harwell, Didcot, Oxfordshire OX11 0DE, United Kingdom (Received 26 November 2013; published 18 March 2014)

Dynamic systems exist everywhere, from the motions of atoms and molecules to weather systems.Our ability to understand these systems is often improved through taking "snapshots" through observations such as images.Dynamic systems on the nano-and mesoscale are found widely in biology and condensed matter.Imaging has been hugely successful for investigating these, but many challenges exist when going from a static to a dynamic regime.For example, imaging requires temporal resolutions much smaller than the time scales under investigation.One imaging technique that offers a solution to this temporal resolution problem is ptychography [1][2][3][4][5][6][7].As has been shown recently [8], ptychography can be used to image dynamical systems to high spatial resolution.
Ptychography [1][2][3][4][5][6][7] is a microscopy technique that forms (complex) images by using a "virtual" lens rather than a physical image-forming optic.From a set of j intensity measurements, I j ¼ ψj ψÃ j , ptychography determines the missing phase information associated with each diffracted wave field ψj .The diffracted wave field ψj is related to an object through free-space propagation of the exit surface wave (ESW) ψ j , which under the assumptions of the projection, paraxiality and single scattering (Born) approximations are given as the product of an "object" O and "probe" P function via ψ j ¼ OP j .The j ESWs are constructed by rastering a partially coherent probe across an object with adjacent probe positions overlapping.Provided this measurement scheme is followed, the missing phase of the diffracted wave fields can be recovered through single-step [1][2][3] or iterative [4][5][6][7][8] methods.
Ptychography can be used in transmission in two [5][6][7] or three dimensions [9] as well as in Bragg geometries which permit the imaging of displacement fields [10][11][12] and defects [13].Advances have permitted simultaneous recovery of the object and the probe [6,7,14] which has allowed the technique to find widespread use as a wave field and optics characterization tool [15,16].More recent [8] advances have allowed the recovery of not a single probe or object but multiple probes or objects.This permits ab initio characterization of the coherence properties of wave fields (probe modes) or imaging sample dynamics [8,17,18] (object modes) without stringent requirements on temporal resolution.The immediate impact of this is that any system that can be considered to be made of a number of different "states" (in the object, probe, or both) can now be imaged.The implications of this are broad and should enable investigation of fundamental science, for example, the validity of the Born-Oppenheimer approximation, which assumes matter interacts with a time-averaged electronic configuration.Here we demonstrate experimentally how ptychography can be used to image a sample with dynamics considerably shorter than the acquisition time of the experiment.We also show that with the addition of a vibrating element, the mode shapes and coherence properties of the incident wave field can be manipulated and customized.
To test the ability of ptychography to recover multiple modes, experiments were carried out at beam line 34-ID-C at the Advanced Photon Source in Chicago.A Kirkpatrick-Baez (KB) mirror system was used to focus 9 keV x rays onto a sample which was placed at the approximate focus 100 mm (200 mm) from the vertical (horizontal) mirror.Horizontal and vertical slits prior to the KB mirrors were used to adjust the spatial coherence entering the optics.The sample was a lithographed 1.5 μm thick tungsten Siemens star.Scanning of the sample stage perpendicular to the x-ray direction was achieved by a NPXY100Z25A piezo stage, with larger movements obtained by stepper motors.Diffraction was recorded 1.5 m downstream using a TimePix pixel detector [19], which consisted of 256 × 256 pixels of side length 55 μm.To improve the dynamic range of the data, a partially attenuating beamstop (200 μm thick silicon square) was placed over the central region (30 × 30 pixels) of the diffraction pattern [see Fig. 1(a)].To reduce absorption and scattering from the air between the sample and detector, a 1 m evacuated flight tube was installed.To examine the effect of sample dynamics on a ptychography experiment, an analog signal generator (WaveTek) was used to input an additional periodic signal to the nominal sample positions.This resulted in the piezo motors now having a time-dependent motion which caused periodic intraexposure sample vibration.Two different types of vibration were introduced, one from a sine wave and the other a square wave, with each having a frequency of 1 Hz, which was considerably less than an exposure time of 6 s per position.A periodically vibrating object is an important example of sample dynamics since mechanical coupling between the components or the environment may introduce some periodic vibration of the sample, a particularly relevant consideration for highresolution imaging with x rays or electrons.For a sample illuminated by a quasimonochromatic wave field with wavelength λ, the diffraction pattern IðqÞ on a detector (with reciprocal space coordinate q) some distance z can be described using free-space propagation as where r is a two-dimensional real-space position coordinate.The illuminating wave field is described by its mutual optical intensity (MOI) Jðr 1 ; r 2 Þ, and the sample is described by its density function ρðr 1 ; r 2 Þ.The general form of the MOI is given as [20] Jðr which describes the partial spatial coherence of the illumination via a sum of orthonormal probe modes P n with weight η n , which are themselves fully coherent but mutually incoherent.The sample dynamics is described by its density function, which is given in an analogous form as the MOI as where, for example, the modes O m ðrÞ (with weights μ m ) can describe different states or a discretization of dynamics [21] which could arise from charge density waves [17] or phonons [18].For the case of full spatial coherence and a static sample (or single state), , and Eq. ( 1) becomes the Fourier transform of the autocorrelation of the ESW multiplied by a propagator term.Equation ( 1) results in the recorded intensity comprising a sum of intensities from each of the different combinations of the N probe and M object modes, respectively.Without explicit consideration of this, iterative algorithms used in ptychography will fail to converge, as the underlying assumption of full coherence (from the probe and/or object) is no longer valid [8,[21][22][23][24].As has been shown recently [8], the probe and object modes (or, alternatively, J and ρ) can be reconstructed ab intio through an iterative scheme using data that have been collected following the ptychographic prescription, that is, adjacent positions share some common features or overlap.In order to reconstruct the object and probe modes, an initial guess is made for both before propagating the current ESW iterates (for each mode combination) for each position and enforcing consistency with the measured data.After enforcing consistency with the measured data, the sample plane overlap constraint is enforced [7,8] to yield new estimates for the probe and object modes.A new iterate of the ESW for each position is then constructed using previous and updated estimates [6,7,14,25].
For the case of a dynamic object, at each scan position we can discretize the object "vibration" and write the recorded diffraction pattern as the incoherent sum of intensities for all of the "vibrating" positions during the exposure [21].If the general description for an object mode m at position j is O m ðr þ r j Þ, then for the case of a vibrating sample, we can write the object modes as Oðr þ r j þ r m Þ where r m is the translation of the vibrating sample from its scan position.However, we note that ptychography does not rely on global positions of the sample and probe but relative translations.As such, the product of a translated sample and probe PðrÞOðr þ r j þ r m Þ can be rewritten as Pðr − r m ÞOðr þ r j Þ, implying that the object vibration can be absorbed into probe modes.The diffracted intensity can be written as where P z is the free-space propagator operator for a distance z.The consequence of Eq. ( 4) is that to ameliorate the deleterious effects of periodic sample vibration, a number of modes will need to be included in the forward propagation.Shown in Fig. 2(a) is the reconstructed sample phase for the static case reconstructed using the difference map with simultaneous reconstruction of the object and probe [7].To expedite the reconstruction process, a multiresolution approach [26] was employed (for all reconstructions) where initial low-resolution (by cropping the data) reconstructions were obtained which were then used to seed the next higher-resolution reconstruction.Three levels of resolution were used (64 × 64, 96 × 96, and 128 × 128 pixels), each lasting 500 iterations.A round scan pattern (495 positions) was employed (400 nm radius and 5n points in the nth shell) [27] with positions refined at the final resolution every five iterations.This was achieved by performing a nearest-neighbor search and selecting the position which minimized the difference between the calculated and measured intensity [6].For all reconstructions, the initial object consisted of a real array of random numbers while the probe for the static case was an approximately calculated probe for a KB mirror system (assumed to be a rectangular lens).This retrieved probe was then used as a start for all subsequent reconstructions.Because of the symmetry of the update schemes, i.e., identical modes are equivalent to a reweighting of a single mode, the initial modes need to differ.This was achieved by multiplying the probes by a real random array of values, altering the original probe amplitude by AE10%.Performance was found to increase if random translations were also introduced into the initial probes.The translations used here were only ≈ 10% of the actual vibration amplitudes, indicating that no knowledge of the coherence was required.
Shown in Fig. 2 is the phase of the reconstructions obtained once the time-dependent motion was introduced using the square wave [Fig.2(b)] and the sine wave [Fig.2(c)] with amplitude of 1 μm.It can be clearly seen that the reconstructions from the dynamic data sets are considerably worse than the static case with the star becoming significantly distorted.This failure is attributed to the fact that the forward model for the recorded intensity is incorrect since it does not consider any vibrations.Shown in Figs.2(d)-2(f) are the reconstructions for the static, square wave, and sine wave cases, but now simultaneously reconstructing five probe modes along with the sample, which equates to modifying our intensity model to take the form of Eq. ( 4).The reconstructions from the square and sine wave now closely resemble the static case with the sample vibrations being absorbed into the additional probe modes.The static case with additional probe modes shows only a marginal improvement, indicating that the initial spatial coherence of the illumination was sufficient.It should be noted that no knowledge of the mode weights or translations is necessary.More importantly, the ability to resolve sample dynamics is independent of acquisition time and pulse duration (provided the dynamics are shorter than the exposure time or pulse duration) since using longer exposures or longer pulses simply results in a linear scaling of the total recorded intensity.
The overall image quality is high, with small defects in the lithographic process of the sample well reproduced in the reconstructions and are visible in Figs.2(a) and 2(d)-2(f).Based on the spatial frequency where the phase retrieval transfer function [28] calculated from the average of 20 reconstructions drops below 0.5, we estimate the resolution to be 49 AE 1 nm.
The first three recovered probe modes (n ¼ 1, 2, 3) are shown in Fig. 3 for the static (a), square wave (b), and sine wave (d) with amplitude 1 μm, and square wave (c) and sine wave (e) with amplitude 1.5 μm.The modes shown have been orthogonalized, in accordance with their definition from the MOI in Eq. ( 2).This is necessary since the linearity of the intensity [Eq.( 4)] permits some linear FIG. 2 (color online).Reconstructed phase of a test sample using a single illumination mode for a static sample (a), vibrating sample using a square wave of amplitude 1 μm (b), and vibrating sample using a sine wave of amplitude 1 μm (c).This is compared to the reconstructed phase using five illumination modes for the static sample (d), vibrating sample using a square wave of amplitude 1 μm (e), and vibrating sample using a sine wave of amplitude 1 μm (f).The black scale bar represents 1 μm.combinations of the orthogonal probe modes to be reconstructed.This has a relatively simple solution for the probes, as they can easily be made orthogonal (for example, by Gram-Schmidt orthonormalization or singular value decomposition).When the case extends to object modes, it is possible there will be no governing equations such as Eq. ( 2) and orthogonalizing the object modes may result in it being difficult to interpret their physical significance (e.g., in terms of their projected refractive index for x rays), as the easily interpretable object modes will be some unknown linear combination of the orthogonal object modes.However, in the simulation example given in Ref. [8], where the spins of an Ising model are imaged, orthogonalization of adjacent couplings is able to reveal the statistics of the various configurations.Consequently, the ultimate interpretation of reconstructed object modes will depend on a priori information regarding the system under investigation, for example, permitting combinations that yield transmission functions with physically allowed phase and amplitude ranges, e.g., jOj < 1 or arg½O < 0.More sophisticated methods may involve placing constraints on the real and imaginary components of the refractive index [29] or by adapting techniques from spectromicroscopy used to obtain quantitative information from energy-resolved images [30].
We see that with the addition of different forms of vibration, the form of the modes changes.The net effect can be seen in the bottom row (rms) of Fig. 3, which shows the root mean square of the probe intensities, demonstrating that the square wave [Figs.The modes for the different vibration regimes shown in Fig. 3 demonstrate how the mode shapes and coherence properties can be effectively manipulated.This custom mode manipulation may prove very useful in designing custom shapes for use in high-resolution x-ray or electron imaging where illuminations with very specific properties are required, such as in structured illumination.
The change in mode shape implies that a degradation in spatial coherence may, in fact, lead to probes that have more overlap between adjacent positions.It is well established that sufficient overlap in ptychography is crucial to high quality reconstructions [31].This may lead to certain situations where small degradations in spatial coherence may paradoxically yield some benefit due to the increase in overlap.To quantify such an increase, we can calculate the average overlap via where Δr j is a relative translation, and J is the total of translations to average over.Using Δr j ¼ r jþ1 − r j and the round scan pattern, the overlap for the static case is 0.57, which is less than the case for the sine wave of amplitude 1.5 μm which has an overlap parameter of 0.60.However, the increase in overlap is relatively small compared to the degradation in overall coherence, so any presumed benefit from the increased overlap is most likely lost.The change in probe mode power can be seen in Fig. 4, which shows in the presence of vibration, the higher-order modes contain more power, a scenario that is consistent with a decrease in spatial coherence [32].
We have demonstrated experimentally that an object with dynamics considerably faster than the measurement times can be imaged using a "mixed state" reconstruction algorithm [8].This was shown for a vibrating sample using an x-ray wave field.We have shown how an equivalence FIG. 4 (color online).The relative mode power demonstrating the increase in higher-order (n > 1) mode power as the coherence is degraded with the dynamic cases.
between the object and probe modes can be used to accommodate decoherence when it originates from a vibrating element.This equivalence demonstrates how coherent wave fields can be manipulated through incoherent methods to create custom mode shapes with custom intensity and coherence properties.

FIG. 1 (
FIG. 1 (color online).(a) Unprocessed logarithm of the data for a single position for the static case.(b)-(d) Comparison of the data for the (b) static, (c) square, and (d) sine wave cases with amplitude 1 μm.A loss of fringe visibility is evident with the vibrating sample [(c) and (d)].The white scale bar represents 0.027 nm −1 .
3(b) and 3(c)] essentially splits into two distinct regions, and the sine wave [Figs.3(d) and 3(e)] has a more continuous distribution, reflecting quite accurately the differences in the wave shapes.The increased amplitude of oscillation is also reflected in the increased width of the rms amplitude for the square [Figs.3(b) and 3(c)] and sine [Figs.3(d) and 3(e)] cases.

FIG. 3 (
FIG. 3 (color online).The first three (n ¼ 1, 2, 3) recovered probe modes for the cases of (a) a static sample, (b) a square wave with amplitudes 1 μm and (c) 1.5 μm, and (d) a sine wave with amplitudes 1 μm and (e) 1.5 μm.Also shown is the square root of the summed intensity (rms, right column, linear scale) from five probe modes for the cases listed previously.The rms shows clearly how the probe forms can be manipulated using a vibrating sample.The scale bar is 1 μm.