Bond Ordering and Molecular Spin-Orbital Fluctuations in the Cluster Mott Insulator GaTa$_4$Se$_8$

For materials where spin-orbit coupling is competitive with electronic correlations, the spatially anisotropic spin-orbital wavefunctions can stabilize degenerate states that lead to many and diverse quantum phases of matter. Here, we find evidence for a dynamical spin-orbital state preceding a T$^*$=50 K order-disorder spin-orbital ordering transition in the $j\!=\!3/2$ lacunar spinel GaTa$_4$Se$_8$. Above T$^*$, GaTa$_4$Se$_8$ has an average cubic crystal structure, but total scattering measurements indicate local non-cubic distortions of Ta$_4$ tetrahedral clusters for all measured temperatures $2<T<300$ K. Inelastic neutron scattering measurements reveal the dynamic nature of these local distortions through symmetry forbidden optical phonon modes that modulate $j\!=\!3/2$ molecular orbital occupation as well as intercluster Ta-Se bonds. Spin-orbital ordering at T$^*$ cannot be attributed to a classic Jahn-Teller mechanism and based on our findings, we propose that intercluster interactions acting on the scale of T$^*$ act to break global symmetry. The resulting staggered intercluster dimerization pattern doubles the unit cell, reflecting a spin-orbital valence bond ground state.


I. INTRODUCTION
It is a maxim of condensed matter physics that the many and varied properties of strongly correlated materials arise from an intricate interplay of spin, orbital, and lattice degrees of freedom.Often, the spin and orbital degrees of freedom have a mutual influence, but can be treated as distinct energy scales.Static orbital configurations impart a spatial anisotropy to affect many and diverse magnetic phenomena.For instance, orbital ordering can influence magnetic exchange interactions to generate magnetic frustration in nominally unfrustrated lattices, [1] or relieve magnetic frustration and drive valence bond solid transitions in frustrated magnets [2][3][4].Orbital overlap in metals can create one-dimensional bands in nominally three dimensional materials.The resulting Peierls instability on those bands generates intricate superstructures of spin singlets on structural dimers through an orbitally induced Peierls mechanism [5][6][7].
Both relativistic spin-orbit coupling and covalency can act to reduce the separation of spin and orbital energy scales, enabling quantum fluctuations between nearly degenerate states.In the extreme limit of large spin-orbit coupling orbital degeneracy is either partially or totally removed depending on the electron filling.For instance, for a single electron occupying a t 2 orbital triplet, spin orbit coupling stabilizes a total j = 3/2 quartet where orbital degrees of freedom naturally support spatially anisotropic and higher order, multi-polar exchange interactions [9][10][11].Covalency plays a similar, but lesser explored, role to reduce the tendency towards classical Jahn-Teller distortions in solid state materials [12][13][14].Typically, the effective strength of spin-orbit coupling is reduced in covalent materials, but, in materials where small clusters of atoms hybridize to form molecular units, the effects can be competitive.In such cluster Mott insulators, the interplay of orbitals, magnetism, and spinorbit coupling on the molecular clusters leads to diverse possibilities [15].
In this work, we use x-ray diffraction, total scattering, and neutron spectroscopy to reveal dynamic spin-orbital fluctuations preceding a spin-orbital ordering transition in the j = 3/2 cluster Mott insulator GaTa 4 Se 8 .At high temperatures, we find an average cubic crystal structure, but local cubic symmetry is dynamically broken on Ta 4 clusters for all measured temperatures, T≤ 300 K.The Ta 4 clusters distort along local 110 directions and fluctuate incoherently in space and time [Fig.1].Global symmetry is broken at T * =50 K as clusters coherently align towards a single neighboring cluster through a collective spin-orbital ordering transition.The distorted clusters exhibit an anti-ferro configuration doubling the cubic unit cell in a bond ordered structure as illustrated in Fig. 1 (c).Inelastic neutron scattering measurements directly resolve the fluctuations associated with the disordered state above T * through the appearance of symmetry forbidden optical phonon intensity.These lattice fluctuations are a manifestation of the dynamic spin-orbital state arising from a highly degenerate potential.We propose that intercluster spin-orbital exchange interactions act to break the degeneracy at T * and stabilize a longrange spin-orbital order in GaTa 4 Se 8 .
GaTa 4 Se 8 belongs to a family of cluster Mott insulating lacunar spinels with chemical formula GaM 4 X 8 , M = (V, Mo, Nb, Ta), X =(S, Se) [16][17][18][19][20][21][22][23].The essen-tial structural units are cubane (M 4 X 4 ) 5+ clusters that occupy an FCC lattice.For M =(V, Nb, Ta), a single unpaired electron is localized on the cubane tetrahedral transition metal clusters.The transition metal atomic orbitals hybridize across M 4 clusters to form molecular orbital wavefunctions with the highest occupied molecular orbitals on a tetrahedron comprised of triply degenerate t states as shown in Fig. 1 (a) [17].As a consequence of the Jahn-Teller active molecular orbital, many of the lacunar spinels exhibit a typical separation of spin and orbital energy scales whereby a high temperature cooperative Jahn-Teller, orbital ordering, structural transition precedes a lower temperature magnetic ordering [16,18,19].However, in those compounds with heavy transition metals (Nb, Ta), such a separation of spin and orbital energy scales is either drastically diminished or does not exist [17,24,25].For both GaTa 4 Se 8 and GaNb 4 S 8 , the spatial extent of transition metal wave functions and strong atomic spin-orbit coupling act to control the ground states and a single temperature scale T * defines magnetic and structural transitions [17,26,27].
The importance of spin-orbit coupling in GaTa 4 Se 8 is borne out through density functional theory (DFT) and resonant inelastic x-ray scattering that suggest the molecular orbitals form a spin-orbit entangled j = 3/2 quartet [Fig. 1 (a)], analogous to the case of octahedrally coordinated d 1 Mott insulators [28][29][30].Such j = 3/2 spinorbit entangled molecular orbitals are consistent with the reported small paramagnetic moment and entropy released at T * [25,26].In this case, the magnetic ground state below T * is formed from spin-orbital singlets as predicted for j = 3/2 on an FCC lattice [10,11] with profound implications for the electronic properties of GaTa 4 Se 8 .Above T * and at ambient pressure, GaTa 4 Se 8 is a strongly correlated insulator, but it exhibits an insulator to metal and superconducting transition under the application of hydrostatic pressure [17,31,32].Molecular j = 3/2 degrees of freedom would support a topo- logical superconducting state at high pressure [30,33].However, the j = 3/2 quadruplet on t 2 orbitals is also Jahn-Teller active.For atomic j = 3/2 insulators, spinorbit coupling acts to suppress Jahn-Teller distortions and, to first order, octahedral compression and elongation provide equal energy gain.This additional degeneracy results in a non-adiabatic "Mexican Hat" potential energy surface that promotes quantum fluctuations among different distortions [8].Weak lattice anharmonicity or additional electronic interactions, may then act to ultimately select a distortion that is different from what a conventional Jahn-Teller effect would predict.A similar situation may apply to the molecular orbitals of GaTa 4 Se 8 ; however, currently the low temperature crystal structure is unknown and there is little information on the spin-orbital configuration below T * .
There is indirect evidence for a magnetic singlet, valence bond solid like, ground state from neutron diffraction, µSR, and NMR [25,26].However, there is no information about the actual valence bond covering on the highly frustrated on the FCC lattice of GaTa 4 Se 8 and limited understanding of the intercluster interactions that might generate such spin-orbital singlets across molecular orbitals.Furthermore, expectations of strong Ta-Se covalency and configurational mixing not captured by DFT bring the j = 3/2 picture into question [34][35][36].Given the strong connection between the spin, orbital, and lattice degrees of freedom and the lattice, a detailed account of the evolution of the lattice through T * is essential to understand the low temperature electronic state and it's connection to high pressure superconductivity in GaTa 4 Se 8 .

II. X-RAY CRYSTALLOGRAPHY AND UNIT CELL DOUBLING
GaTa 4 Se 8 has a single magnetic and structural transition at T * =50 K where the magnetic susceptibility abruptly drops and the unit cell doubles, as indicated by the sudden appearance of structural superlattice Bragg reflections at [1/2, 0, 0] positions [Fig. 2 (a)].As shown in Fig. 2 (b) and (c), a bifurcation of cubic [h, 0, 0] reflections and abrupt volume decrease at T * indicate a first-order cubic to tetragonal transition, but with no detectable hysteresis or phase coexistence.Above T * , single crystal XRD consistently refines to the cubic space group F 43m. Below T * , the [1/2, 0, 0] reflections indicate a unit cell doubling along a unique cubic axis and we observed all [1/2, 0, 0] reflections consistent with six domains of the tetragonal cell.We did not find any additional peak splitting or evidence for an orthorhombic distortion.We have evaluated our T=10 K single crystal XRD data against all subgroups of the cubic cell that are consistent with a k = [1/2, 0, 0] distortion; body centered space groups are ruled out by the observation of reflections with h+k+l = 2n and based on the absence of any [2n+1, 0, 0] reflections we ruled out all possible primitive groups except for P 42 1 m [37].
We obtain high fidelity refinements for P 42 The structural transition in GaTa 4 Se 8 is distinct from reports in other lacunar spinels.For Ga(V,Nb) 4 (S,Se) 8 , the Jahn-Teller active M 4 clusters undergo a rhombahedral distortion, displacing along the cubic body diagonals, in a ferrodistortive pattern for vanadium based compounds [16,[38][39][40][41] and antiferrodistortive pattern for the niobium compounds [19,27].A primary mode analysis GaNb 4 S 8 found a single Jahn-Teller active mode within irreducible representation X 5 and dominated by displacements of the Nb 4 tetrahedra [27].In contrast, we find that the distortion in GaTa 4 Se 8 is described by a superposition of modes within many irreducible representations.The most dominant distortions belong to X 5 (subgroup P 42 1 m, amplitude= 0.12 Å) and X 3 (subgroup P 4m2, amplitude= 0.07 Å) representations, both modes involving nearly equivalent displacements of Ta and Se atoms [37,42,43].Unit cell doubling is affected through a combination of W 4 and ∆ 1 modes with amplitudes of 0.02 Å and nearly equivalent Ta/Se displacements [37].Thus, we cannot attribute the transition in GaTa 4 Se 8 to a classic Jahn-Teller mechanism acting through a single normal mode of the cell.The intercluster dimer motif, unit cell doubling, mode mixing, and equivalent contributions from displacements of metal and ligand ions implicates the complex spin-orbital wavefunction and a mechanism likely involving intersite interactions, are at play in GaTa 4 Se 8 .

III. LOCAL DISTORTIONS AND DYNAMIC ORBITAL DEGENERACY BREAKING
Having established the low temperature space group in GaTa 4 Se 8 , we now turn to atomic pair distribution analysis (PDF) to provide detailed insight into the evolution of atomic correlations across different length-scales through T * .Temperature dependent x-ray (xPDF) and neutron (nPDF) PDF measurements shown in Fig. 3 provide a histogram of atomic pair correlations from which atomic pair separations can be read off independent of a specific structural model.The PDF data for GaTa 4 Se 8 have two striking features.First, the double-peak structure around 3 Å indicates at least two intercluster Ta-Ta distances between 2.9 and 3.1 Å for temperatures up to 300 K. Thus, the local crystal structure of GaTa 4 Se 8 is not cubic well into the temperature regime where the average structure is cubic.Second, nPDF and xPDF data do not show any discernible qualitative changes across all measured temperatures beyond thermal broadening.The absence of any change in the local structure is highlighted in Fig. 3 (c) that shows a negligible difference in nPDF across T * and in Fig. 3 (d) that shows the normalized inter-Ta 4 cluster PDF peak intensities for inter and intra-cluster pair separations.x acts as an effective local order parameter for displacive transitions [44].The smooth evolution of x with temperature for all pair separations is consistent with thermal disorder (Debye-Waller) and directly implicates an orderdisorder transition at T * .Local distortions in GaTa 4 Se 8 are present but uncorrelated across length scales beyond one unit cell at least up to 300 K.At T * , global cubic symmetry is broken as the distorted units select a unique 110 axis and stack in a c − c − c − c, forming the cell doubled tetragonal structure of Fig. 1

(b).
In Fig. 4, we show a model dependent analysis of the PDF data, covering broad length and temperature scales.A cubic model fails to account for the observed local structure (r < 5 Å) for all measured temperatures T< 300 K.In particular, the cubic structure cannot capture the two Ta-Ta distance around 3 Å and overestimates the PDF weight for Se-Se pairs at 3.5 Å.This local structure is reproduced at all measured temperatures by the tetragonal P 42 1 m lattice that includes four intercluster Ta-Ta bond lengths > > s > s.Within the accuracy of our measurements, the degree of local distortion in GaTa 4 Se 8 as measured by the bond length difference − s does not depend on temperature, indicating local distortions of Ta clusters are fully set in up to at least T=300 K [37].On the other hand, the PDF data for r > 5 Å are equally well described by the cubic and tetragonal models at all temperatures because the tetragonal cell is metrically cubic (c/2 ∼ a).We find that within experimentally achievable resolution, the average structures of cubic and tetragonal cells are only distinguished in a PDF measurement through a small anomalous upturn in thermal parameters upon cooling through T * [37].
Ta 4 tetrahedral clusters in GaTa 4 Se 8 are distorted at all measured temperatures, indicating that local, and potentially dynamic, Jahn-Teller distortions are already present at 300 K. Local distortions preceding an orbital ordering structural transition are common in transition metal compounds, but global symmetry breaking is generally also apparent in total scattering measurements through the appearance of a new length-scale or an abrupt change in bond lengths as atoms displace through

IV. PHONON DENSITY OF STATES AND DYNAMIC DISTORTIONS
The energy integrated diffraction measurements presented above find an average cubic structure for GaTa 4 Se 8 at T>T * , implying that local distortions are incoherently fluctuating in space, time, or both.In Fig. 5 we show the measured phonon density of states (PDOS) as a function of temperature that provide direct evidence for the dynamic nature of local distortions above T * through the appearance of a symmetry forbidden optical phonon mode (13 meV) well into the cubic phase.
Measured PDOS at select temperatures are shown in Fig. 5 (a).The most striking feature of this data is a temperature dependent 13 meV phonon intensity appearing above background below T=200 K.This mode is visible in the raw neutron scattering data as a non-dispersive feature with an intensity that increases quadratically with momentum transfer, ruling out a magnetic origin [37].In Fig. 5 (c) we show the temperature dependent in-tegrated intensity of the 13 meV mode, corrected for the Bose factor, and normalized to it's intensity at T=10 K. Above 200 K it was not possible to reliably resolve the 13 meV intensity from the adjacent modes at 10 and 15 meV.We find that the integrated intensity increases linearly with decreasing temperature from 200 K and saturates at T * while the energy position and width are constant and set by the energy resolution of the spectrometer [37].
We have carried out first principles calculations of the neutron weighted PDOS for GaTa 4 Se 8 in order to obtain insight into the origin of the 13 meV intensity.The results of these calculations for both F 43m and P 42 1 m space groups are shown in Fig. 5 (b) and are in excellent agreement with the measured spectra.There is a notable absence of phonon modes around 13 meV for the cubic cell; but, significant 13 meV PDOS weight arising from optical phonons in the tetragonal cell.Indeed, the 13 meV phonon intensity is the primary qualitative distinction between the cubic and tetragonal PDOS.The appearance of this optical phonon intensity for T>T * , where the average structure is cubic, directly reflects the dynamic structural fluctuations associated with the local symmetry breaking for T>T * in GaTa 4 Se 8 as schematically illustrated in Fig. 1 (b).We have evaluated the partial contributions of Ga, Ta, and Se to the PDOS of the tetragonal cell and find the dominant contribution is from Se with additional secondary contributions from Ta and Ga [Fig.5 (b)].Optical A 1 , A 2 , B 1 , B 2 , and E phonon modes all contribute the PDOS between 12 to 14 meV and our data are not consistent with the assignment of a unique Jahn-Teller phonon mode on Ta  clusters to this feature of the PDOS.
It is instructive to compare the temperature dependent phonon spectra in GaTa 4 Se 8 measured by INS with that measured by infrared spectroscopy (IR).The PDOS as measured by INS contains the full spectrum of predicted vibrational modes, while IR measurements only find a subset of the symmetry allowed modes all attributed to vibrations on the ligand sites [50].Such a discrepancy must arise as a result of a substantial screening of vibrational modes from electron delocalization over Ta 4 tetrahedral units in insulating GaTa 4 Se 8 .Neutrons interact directly with nuclear cores and are not subject to electronic screening.Such an electronic delocalization is most apparent in GaTa 4 Se 8 because of the spatially extended nature of Ta d orbitals.The apparently metallic Ta 4 clusters are at odds with local Jahn-Teller mechanisms generally accepted as the origin of structural transitions in Lacunar spinels and evidence that a pic-ture from mean field arguments for M 4 molecular clusters should be revisited, especially in the case of Ta and Nb compounds [34][35][36].For GaTa 4 Se 8 , a minimal description should involve active and hybridized Ta and Se orbitals on cubane clusters, spin-orbit coupling on the molecular orbitals, as well as interactions between clusters.The mixed character of dynamic distortions and decomposition of the structural transition into a superposition of irreducible representations implicate a degeneracy among possible Jahn-Teller distortions in GaTa 4 Se 8 .This is likely a consequence of spin-orbit coupling and covalency on the Ta 4 clusters.Ultimately, additional interactions between clusters acting on the scale of T * select a unique valence bond state and quench the structural fluctuations.

V. DISCUSSION
We find that local and dynamic distortions are present up to at least 300 K in GaTa 4 Se 8 .These reduce tetrahedral symmetry of the Ta 4 clusters and dynamically break orbital degeneracy.Heat capacity measurements have found an entropy change associated with T * approaching ln(4) [25,26] indicating that the orbital and spin degrees of freedom simultaneously freeze out at T * .In other words, T * corresponds to a collective, order-disorder, spin-orbital ordering between two insulating phases.A discrete six-fold rotational symmetry on the FCC lattice is broken through T * and we propose this order is selected through intercluster spin-orbital interactions that act on the scale of T * .
Interestingly, diffraction measurements indirectly implicate dynamic structural fluctuations above global structural transitions in other lacunar spinels [24,27,40].However, the nature of structural and magnetic transitions, local distortions of transition metal clusters, low temperature crystal structure, and magnetic state are unique in GaTa 4 Se 8 among the family of d 1 Lacunar spinels.GaV 4 (S,Se) 8 and GaNb 4 Se 8 exhibit a typical separation of spin and orbital energy scales with rhombahedral structural distortions that quench orbital moments occurring at high temperatures preceding lower temperature spin only magnetic transitions [16,18,23,25,40,51].GaNb 4 S 8 exhibits a unit cell doubling spinorbital transition that is similar to GaTa 4 Se 8 , but the symmetry of the low temperature structure is further reduced from P 42 1 m to P2 1 2 1 2 1 with Nb 4 clusters distorting along cubic body diagonal directions in an antiferro pattern [19,27,52].The material trend implicates both spatially extended electron density and spin orbit coupling as these both increase from V→Nb→Ta and can respectively act to drastically alter Jahn-Teller physics [8,53].In particular, for j = 3/2 degrees of freedom, spin orbit coupling can lead to a degeneracy among different types of possible Jahn-Teller distortions [8].The symmetry forbidden phonon intensity we observe above T * in GaTa 4 Se 8 is consistent with such a degeneracy that is ultimately broken by additional interactions.
Above T * , GaTa 4 Se 8 is a correlated Mott insulator [31,32], containing an odd number of electrons in the cubic cell.While the Tetragonal unit cell below T * contains four formula units and an even number of valence electrons, our band structure calculations find a metallic state for this structure, supporting the Mott insulating picture.The absence of magnetic ordering must then indicate a valence bond solid type ground state.The structural dimerization suggests a staggered valence bond covering shown in Fig. 1 (b).Such a sharp reduction in magnetic susceptibility and dimerized superstructure below T * is remarkably similar to the orbitally induced Peierls metal-insulator transitions observed in spinels [6,7,54] and other orbitally degenerate compounds exhibiting temperature dependent metal insulator transitions [55,56].However, in GaTa 4 Se 8 the transition is between two insulating states.
Resonant inelastic x-ray scattering found that at the level of DFT the molecular orbital state in GaTa 4 Se 8 is molecular analog of the atomic j = 3/2 [29].Although, covalency, configurational interactions within cubane clusters, and local distortions alter the pure j = 3/2 description [35,36], it may still stand as an approximate description of the magnetism.In this case GaTa 4 Se 8 may realize a spin-orbital dimer ground state proposed for Mott insulators on an FCC lattice with d 1 filling and large spin-orbit coupling [11].Our measurement show how the lattice degrees of freedom can act to select an ordered valence bond solid from the glassy spin-orbit dimer covering proposed in [11].
Our measurements implicate an essential role played by cluster orientational ordering to affect a Peierls like intercluster dimerization at T * .This ordering is likely driven by intercluster spin-orbit interactions that break the degeneracy of orbital configurations.The importance of such interactions is further underscored by the primary involvement of both Ta and Se ions across intercluster bonds through the structural transition.Despite the large intercluster separation, closest Ta-Ta distance 4.274 Å closest Ta-Se distance 2.744 Å, there is strong evidence that intercluster exchange is a relevant energy scale in lacunar spinels.For example, recent inelastic neutron scattering measurements of spin waves in magnetically ordered GaV 4 S 8 suggest exchange interactions between the clusters on the order of 20 K [57].A more spatially extended electron density on Ta 5d orbitals should give rise to enhanced interactions compared with the V compounds so that exchange interactions on the scale of T * = 50 K are not unreasonable.In GaTa 4 Se 8 the broken symmetry at T * involves both spin and orbital degrees of freedom and such spin-orbital exchange may be mediated through lattice vibrations [58,59].The 13 meV mode we observe distorts Ta 4 clusters, simultaneously modulating orbital occupancy, intercluster distances, and Ta-Se bond angles to influence intercluster interactions.In this respect, the optical phonon response we observe in GaTa 4 Se 8 is reminiscent of spin Peierls compound CuGeO 3 [60,61], but in the case of GaTa 4 Se 8 , lattice vibrations affect a complex spin-orbital exchange.Further theoretical effort investigating the role of optical phonons in generating effective spin-orbital interactions in cluster Mott insulators is required [62,63].Our results provide empirical insight on the microscopic mechanisms that may generate such orbital interactions.
In summary, we have used a combination of single crystal diffraction, total scattering, and inelastic neutron scattering measurements to show that Ta 4 tetrahedral clusters in GaTa 4 Se 8 are locally and dynamically distorted up to temperatures of at least 300 K. We resolve the optical phonon modes associated with lattice fluctuations between degenerate orbital configurations on the FCC lattice.We propose that the degeneracy of these configurations is broken through inter-cluster spin-orbit exchange interactions.A collective order-disorder spinorbital transition in GaTa 4 Se 8 is characterized by the long range orientational ordering of distorted Ta 4 clusters to form weak structural dimers aligned along 110 directions.This suggests that a spin-orbital valence bond solid ground state is realized in GaTa 4 Se 8 .
Appendix A: Methods

Sample Synthesis
Single crystal samples of GaTa 4 Se 8 were synthesized by solid-state reaction in box furnaces.About 1.5 g of stoichiometric quantities of Ga, Ta, and Se were loaded into a I.D. =10 mm, 170 mm long quartz ampules under an argon glovebox, evacuated, and sealed.The ampules were heated to 1050 • C with 80 • C/hour ramping rate and stayed at 1050 • C for 24 hours.Then cool down to 850 • C with 1 • C/hour cooling rate before air quench.We were able to obtain thousands of high-quality single crystals with a dimension above 200 microns in a single batch.The quality of single crystals was confirmed by x-ray diffraction (Ag Kα radiation).The composition was confirmed by powder x-ray diffraction with ground crystals (Bruker Cu Kα radiation).

Synchrotron x-ray crystallography
Temperature-dependent synchrotron x-ray diffraction measurements were carried out on the QM2 beamline at Cornell High Energy Synchrotron source using an x-ray energy of 20.67 keV (0.5998 Å), and the Pilatus 300K area detector positioned 56.2 cm from the sample.The single crystal sample was mounted on a Cu post on the coldfinger of a closed cycle cryostat for measurements in reflection geometry between 3 and 300 K. Synchrotron x-ray crystallography measurements were carried out on 15-ID at Advanced Photon Source, Argonne National Laboratory using an x-ray energy of 30.00 keV (0.4133 Å) and Pilatus CdTe detector positioned at 7 cm from the sample.The 20 µm single crystal was mounted on a quartz capillary, and measurements were carried out between 10 K and 300 K using a He cryostream.Diffraction peaks were indexed, unit cell, and UB matrix of each tetragonal domain were determined in reciprocal lattice and CELLNOW software for six domains.All frames were integrated using Bruker SAINT software, and multidomain absorption corrections were performed through TWINABS.The crystal structure was solved by direct method and processed leastsquare method refinements on F 2 by SHELXTL [64] and Olex2 [65] packages.The converged solution using full single domain reflection data and all Ga, Ta and Se atoms were modeled by isotropic atomic displacement parameters (ADPs).We checked 1×1×2, 2×2×1 and 2×2×2 superlattices in −4 and −42m tetragonal lattice classes.Based on the extinction rules of x-ray reflection, we find the P 42 1 m space group with 1×1×2 superlattice is the best structure for GaTa 4 Se 8 below the transition temperature.The details of the crystallographic analysis are shown in the supplementary material [37].

Pair distribution function measurements
Synchrotron total scattering measurements (xPDF) were carried out on 28-ID-1 (PDF) beamline at the National Synchrotron Light Source-II at Brookhaven National Laboratory.The sample was loaded in a 1 mm I.D. kapton capillary, mounted in a He cryostat, and data was collected between 5 K and 330 K on warming in 5 K steps.Measurements were carried out using an x-ray energy of 74.46 keV (0.1665 Å).We used an amorphous silicon PerkinElmer area detector with a sample to detector distance of 1008 mm for diffraction and 204 mm for PDF.Detector position was determined by calibration using a Ni standard.The two dimensional diffraction data were integrated using PyFAI [66] and then corrected, normalized, and Fourier transformed to obtain the atomic pair distribution function (PDF), G(r), using PDFgetX3 [67] and a Q max = 27 Å−1 .
Neutron total scattering measurements (nPDF) were carried out on the NOMAD beamline at Spallation Neutron Source at Oak Ridge National Laboratory.A 4.2 g sample was loaded into a vandadium sample can and mounted on the coldfinger of a He cryostat.Data was collected for 160 minutes at 5 K, 55 K and 300 K, and 80 minutes for intermediate temperatures.The structure factor S(Q) was checked to determine the Q max for each temperature.Background subtracted time-of-flight data were reduced and transformed to PDF using a Q max = 34.5 Å−1 and ADDIE available at NOMAD [68].xPDF and nPDF refinements were carried out using PDFgui and DIFFPY-CMI modeling platforms [69].

Inelastic neutron scattering
Inelastic neutron scattering measurements were carried out on the SEQUIOA spectrometer at the Spallation Neutron Source at Oak Ridge National Laboratory.A 5 g polycrystalline sample was loaded into an Al sample can and mounted on the coldfinger of a closed cycle cryostat to achieve a base temperature of 5 K. Measurements were conducted using a fixed incident energy E i = 60 meV and the coarse Fermi chopper rotating at 300 Hz, providing an energy resolution at the elastic line of 3.1 meV.All data was normalized and reduced using algorithms in the MANTID analysis software [70].The normalized neutron intensities were converted into neutron weighted phonon density of states by integrating over Q (0.2 to 5 Å−1 ) and correcting for multiphonon and multiscattering processes within the incoherent approximation [71].without (PAW-PBE) and with spin-orbit coupling (PBE+SOC).All atomic positions and cell parameters were restricted by P 42 1 m space group and the force threshold was set to 4×10 −5 a.u..We used a 680 eV (PAW-PBE) and 620 eV (PBE+SOC) cutoffs for plane-wave energy with 6×6×6 (cubic) and 5×5×3 (tetragonal) Monkhorst-Pack k-point mesh for selfconsistent-field calculations.The projector-augmentedwave (PAW) method [72] and Perdew-Burke-Ernzerhof (PBE) exchange-correlation functional [73] were used for all calculations.All calculations were performed using Quantum ESPRESSO [74,75].
The optimized structures are consistent with Ta 4 cluster distortions and intercluster dimerizations along 110 as we observed in x-ray diffraction.Details of the local Ta 4 cluster structures are presented in Fig. 6 and the inter and intracluster bond lengths for PAW-PBE (PBE), PBE+SOC (SOC), and experimentally determined values are presented in table I.The optimized structures display a slightly elongated c-axis (c>2a), which also agrees with our experimental result.In addition, we find that PAW-PBE overestimated the unit cell size (a=10.4677Å, c=21.0086Å) as PBE+SOC (a=10.3988Å, c=20.8129Å) has smaller cell and slightly closer to experimentally determined values (a=10.3437Å, c=20.6878Å).Also, Ta 4 clusters form dimers along 110 directions in both calculated structures.
The calculated band structures and electronic density of states for both PAW-PBE and PBE+SOC calculations are shown in Fig. 7 and Fig. 8.In both calculations, the density of electronic states at the E F is drastically reduced from that of the cubic cell.Without spin-orbit coupling, the unit cell doubled structure opens a small gap since the total electron number is even in a primitive cell.However, with spin-orbit coupling, the splitting of the electronic bands retains a metallic state in the unit cell doubled structure.A large Coulomb interaction, U, is required to open an insulating gap and our band structure calculations indicate that GaTa 4 Se 8 is retains its Mott insulating nature at low tempertures.

Phonon calculations
We used Phonopy package [76] for ab-initio phonon calculations and Quantum ESPRESSO [74,75] as the force calculator.For phonon calculation, we used 680 eV cutoff for plane-wave energy with a 2×2×1 Monkhorst-Pack k-point mesh for self-consistent-field calculations.We checked the total energy convergence with the cutoff energy and k-point mesh for calculations.The minimal kpoint were used for the cell with 104 atoms.To compare with the results measured by inelastic neutron scattering, the neutron weighted phonon DOS was calculated by neutron cross-section for each atom and the projected phonon density of states as follows where σ i and g i (E) are neutron cross-section and projected phonon density of states for each species of atom.To simulate the measured PDOS, we convolved the calculated PDOS with an empirical instrumental resolution function for SEQUOIA [77] f (E) = 1.435 × 10 −6 × E 3 + 3.885 × 10 −4 × E 2 − 7.882 × 10 −2 × E + 4.295.
The irreducible representation of phonon modes can be calculated with the information of occupied Wyckoff positions (WPs).In P 42 1 m tetragonal structure, Ga atoms occupy 4e WPs.Ta and Se atoms are on 4e and 8f WPs.Each atom on 4e site has 2A 1 + A 2 + B 1 + 2B 2 + 3E optical phonon modes, and 3A 1 + 3A 2 + 3B 1 + 3B 2 + 6E for atoms on 8f WPs.Therefore, the entire unit cell with 20 asymmetric atoms has 46A 1 + 32A 2 + 32B 1 + 46B 2 + 78E optical modes.As Fig. 5 in main text shows, the most distinguishable difference between cubic and tetragonal structure is the modes around 13 meV.From our DFT calculation, we assigned phonon modes between 12 and 14 meV to irreducible representation 2A 1 + 2A 2 + 2B 1 + 3B 2 + 4E, which is the mixed modes of all Ga, Ta and Se atoms.

FIG. 1 .FIG. 2 .
FIG. 1. Unit cell doubling and intercluster dimers in GaTa4Se8.(a) Spin-orbit coupling driven j = 3/2 states on Ta4 cluster molecular orbitals.j z spin-orbital states are Jahn-Teller degenerate.(b) Monoclinically distorted Ta4 clusters with four inequivalent Ta-Ta distances > > s > s point along local 110 axis.(c) Local distortions of Ta4 clusters on FCC lattice fluctuate among 110 directions for T>T * , shown as colored arrows.Intercluster interactions select a global rotational symmetry breaking below T * .Unit cell doubling occurs through a formation of two inequivalent Ta4 stacked in a c− c −c −c pattern.

FIG. 3 .
FIG. 3. Temperature dependence of atomic pair separation in GaTa4Se8.(a) Temperature dependent xPDF and (b) nPDF measurements show local distortions of Ta4 clusters exist well above the transition through two distinct intercluster Ta-Ta distances, s =2.95 Å and =3.06 Å observed at all measured temperatures.Real space resolutions of PDF data are δr = 0.233 Å for xPDF and δr = 0.182 Å for nPDF (c) Detailed comparison of small r nPDF above and below T * .45 -80 K difference data show no measurable change in atomic pair displacements through the transition.(d) Temperature dependence of normalized inter-Ta4 cluster PDF peak intensities x showing the continuous and smooth evolution of atomic distances with temperature.Filled symbols are from nPDF and open from xPDF.
1 m with a = b = 10.3437 and c = 20.6878(R 1 = 0.0879 for 13104 reflections).Within the tetragonal cell there are two distinct Ta 4 clusters labeled c and c in Fig. 1 (c), each having the same symmetry, but different Ta-Ta bond lengths.The symmetry of tetrahedral clusters is reduced to monoclinic and retains a single mirror plane.Each cluster contains four different bond lengths shown in Fig. 1 (b) and listed in table I, with three long-bonds, labeled and ( > ), and three short-bonds labeled s and s (s < s ) resulting in a characteristic elongation of tetrahedra along cubic face diagonals.Each Ta 4 cluster points along 110 towards a single neighboring cluster, forming an intercluster dimer motif as illustrated [Fig.1 (b) and (c)].The cubic unit cell doubling occurs through a c − c − c − c stacking arrangement of Ta clusters along the c-axis, giving rise to [1/2, 0, 0] reflections from six domains of the tetragonal cell.

FIG. 4 .
FIG. 4.Local and average structures of GaTa4Se8.Model fits to neutron pair distribution function measurements at T = 5 K and T = 300 K for (a) nPDF and (b) xPDF.At all temperatures the tetragonal cell (P 421m) with distorted tetrahedra provides a significantly improved description of the local structure for r < 5 Å, while the average structure is equally well described using the cubic or tetragonal cell.

FIG. 5 .
FIG. 5. Dynamic symmetry breaking in GaTa4Se8.(a) Measured neutron weighted phonon density of states in GaTa4Se8.A representative Lorentzian fit to the cubic symmetry forbidden optical phonon intensity at 13.8 meV is shown as the grey shaded area.(b) DFT calculated neutron weighted phonon DOS for the cubic (F 43m) and tetragonal (P 421m) cells.Partial contributions to the tetragonal cell DOS are also shown.(c) Temperature dependence of the 13.8 meV intensity extracted from Lorentzian fits to the dynamic susceptibility.

FIG. 6 .
FIG. 6.(a) Local distortion of Ta4 clusters, s < s < l < l.(b) The inter-cluster distances.The bond lengths for the two inequivalent small clusters c and c are shown in TableI.

FIG. 7 .
FIG. 7. Electronic structure of GaTa4Se8 without SOC.(a) Cubic structure (F 43m, a=10.3849Å).(b) Tetragonal structure (P 421m, a=10.4677Å, c=21.0086Å).In tetragonal structure the electronic density of states is reduced at the Fermi level and opens a small gap below T * .

TABLE I .
Intra and intercluster distances in Å.The definition of bonds are shown in Fig. 6. c and c denote the inequivalent clusters in P 421m tetragonal structure.PBE (PAW-PBE), SOC (PBE+SOC) are optimized structures from DFT.