Challenges in High-Throughput Inorganic Materials Prediction and Autonomous Synthesis

Materials discovery lays the foundation for many technological advancements. The prediction and discovery of new materials are not simple tasks. Here, we outline some basic principles of solid-state chemistry, which might help to advance both, and discuss pitfalls and challenges in materials discovery. Using the recent work of Szymanski et al. [Nature 624, 86 (2023)], which reported the autonomous discovery of 43 novel materials, as an example, we discuss problems that can arise in unsupervised materials discovery and hope that by addressing these, autonomous materials discovery can be brought closer to reality. We discuss all 43 synthetic products and point out four common shortfalls in the analysis. These errors unfortunately lead to the conclusion that no new materials have been discovered in that work. We conclude that there are two important points of improvement that require future work from the community, as follows. (i) Automated Rietveld analysis of powder x-ray diﬀraction data is not yet reliable. Future improvement of such, and the development of a reliable artiﬁcial-intelligence-based tool for Rietveld ﬁt-ting, would be very helpful, not only for autonomous materials discovery but also for the community in general. (ii) We ﬁnd that disorder in materials is often neglected in predictions. The predicted compounds investigated herein have all their elemental components located on distinct crystallographic positions but in reality, elements can share crystallographic sites, resulting in higher-symmetry space groups and—very often—known alloys or solid solutions. This error might be related to the diﬃculty of modeling disorder in a computationally economical way and needs to be addressed both by computational and experimental material scientists. We ﬁnd that two thirds of the claimed successful materials in Szymanski et al. are likely to be known compositionally disordered versions of the predicted ordered compounds. We highlight important issues in materials discovery, computational chemistry, and autonomous interpretation of x-ray diﬀraction. We discuss concepts of materials discovery from an experimentalist point of view, which we hope will be helpful for the community to further advance this important new aspect of our ﬁeld.


I. INTRODUCTION
Inorganic materials serve as the basis of modern technology.This has always been the case and it is no coincidence that we have named several historical epochs after inorganic materials.Many known crystalline inorganic materials are tabulated in the Inorganic Crystal Structure compounds with additional computational information and also suggests computationally predicted new materials.
Very recently, Google DeepMind have reported the prediction of up to 2.2 million new stable inorganic crystals, tabulated in their GNoME database [3].However, some of these predictions may warrant experimental verification and synthesizing so many material candidates by hand would be extremely laborious.To accelerate this, a group at Berkeley has established an automated lab, called A-lab, that utilizes robotics and artificial intelligence.A-lab uses robots to mix and heat ingredients and measure powder xray diffraction (PXRD) of the products.An algorithm then analyzes the PXRD patterns, decides whether the synthesis has been successful, and if not, adjusts the synthetic conditions.The group behind A-lab has recently reported that within 17 days, A-lab has been able synthesize 41 new materials out of 58 predicted targets, an impressive success rate of 71% [4].Using human intervention, the success rate has been increased to 78%-43 successfully synthesized new materials.If this were true, it could drastically accelerate materials discovery, potentially yielding hundreds of new compounds annually.Throughout this Perspective, we will refer to the work of Szymanski et al. as the "A-lab paper." Many aspects of this work are impressive: the fact that robots can take over labor-intensive steps, that artificial intelligence (AI) can predict reasonable synthetic routes based on literature precedent, and that a full circle of materials synthesis and characterization can be carried out without human intervention.Unfortunately, we have found that the central claim of the A-lab paper, namely, that a large number of previously unknown materials have been synthesized, does not hold.As we will explain below, we believe that at time of publication, none of the materials produced by A-lab were new: the large majority were misclassified and a smaller number were correctly identified but already known.In this latter category, three compounds have been reported in between GNoME's screenshot of the ICSD and the time of the A-lab publication, meaning that they would not be in the original training set.
We find that the vast majority of the synthetic products have been wrongly characterized.These misinterpreted characterizations broadly fall under two categories: either the authors have failed to recognize that the automated refinement process has changed the symmetry of the target compound or the PXRD pattern agrees better with known phases-or, more often, a mixture of known phases.A more detailed explanation of these pitfalls will be laid out in a following section.In general, it seems that one issue lies with the final characterization step (in this case, the Rietveld refinement), thus improvement of AI-assisted materials characterization seems to be one of the bottlenecks of automated materials discovery.Another might be related to the role of disorder in materials and how this is often not modeled or considered when new materials are predicted.Thus materials prediction could also be improved by considering the role of disorder.
Before we dive into the PXRD data analysis, we first briefly discuss what makes a material "new."We highlight that any work that claims to have discovered new materials must define what makes those new, especially in relation to already known ones.As we hope to reach a multidisciplinary audience, we will then go into some, but not all, of the standard practices of the field when validating this claim.We detail thematic issues that arise when analyzing A-lab's data.Addressing these issues would likely make automated-lab projects more reliable and then serve as a more useful tool for solid-state chemists.The bulk of this paper is an analysis of the 43 compounds sorted categorically, both to compartmentalize the reported compounds efficiently and to highlight the types of errors that we see as motifs arising in each class.

II. WHAT CONSTITUTES A "NEW" INORGANIC MATERIAL?
Chemists usually distinguish inorganic materials by their structure and composition and, in some cases, properties.The dominance of x-ray crystallography in the study of the solid state commonly leads to delineation between materials based on their diffraction properties, along with analysis of their composition by methods such as atomic emission spectroscopy or mass spectrometry.Pure molecular materials take their composition from their molecular formula but may still form different crystal structures, or polymorphs, which have different properties and are often considered to be distinct materials.Nonmolecular materials, such as those we are concerned with here, can also show polymorphism, the most famous example being diamond and graphite as polymorphs of carbon, but in addition to this, nonmolecular materials do not have a chemical composition restricted by molecular formulas.Their composition is not quantized but, instead, can be incrementally changed.One primary example of this is solid solutions, e.g., a solution of KCl and NaCl, which could be written Na 1−x K x Cl, where x can take any value between 0 and 1 [5].Doping is a related concept, where some percentage of an impurity is incorporated into a material; doped silicon is the basis of modern electronics due to the large effect on electron transport imparted by a small concentration of impurity.This ability to incrementally alter composition challenges concepts of what constitutes a new material.
A central theme in our analysis of the work presented in the A-lab paper, which we believe pertains more widely to the field of high-throughput computational materials prediction, is the concept of order and disorder within a crystal lattice.The defining characteristic of a crystal lattice is order but compositional disorder of atoms within a crystal is a widespread phenomenon.In fact, disorder in a crystal lattice is often used to tune the properties of a material, an example of which has recently been demonstrated in Li 1.2 Cr 0.4 Mn 0.4 O 2 [6].Another example is the aforementioned Na 1−x K x Cl solid solution, which adopts the rock-salt structure, with the Na and K atoms disordered over the cation sites.Physically, there is a statistical distribution of Na and K in the crystal-the probability of finding one particular cation in one particular location is based on the value of x in Na 1−x K x Cl.Such a system can be thought of as structurally ordered but compositionally disordered.Experimental crystallographers can accommodate compositional disorder within the framework of the unit-cell description of the crystal, by simply stating that a single crystallographic site may be occupied by a mixture of multiple atom types with fractional occupancy.Thus, in crystallography, such a disordered system is represented with the same unit-cell symmetry that would apply if there were only one atom type on the mixed site (i.e., the symmetry of the aristotype) but then specifying a fractional occupancy for some of the atoms.This description of compositionally disordered materials using partial occupancies has several advantages.First, it is commensurate with the experimental diffraction patterns-the PXRD patterns of Na 1−x K x Cl solid solutions closely resemble those of NaCl and KCl but with only small shifts in peak positions and intensities, so it would make sense that the unit cell is very similar too.Second, fractional occupancies can be used in the structure-factor equation to calculate diffraction intensities: this allows quantitative use of a unit cell with fractional occupancies in, e.g., Rietveld refinement, while simple heuristics such as Vegard's law relate the composition of a solid solution to the lattice parameter, usually with good accuracy.The usefulness of the idea of fractional occupancy, and its compatibility with many experimental crystallographic methods, is such that it is easy to overlook that in fact it breaks the foundational assumption of crystallography, that of transitional symmetry [7].As we will discuss below, this fact becomes much more significant when computational chemistry calculations are undertaken.
Instead of being compositionally disordered, two types of atoms can instead form ordered arrangements.For example, the zincblende structure is an ordered version of the diamond structure.It can further be expanded to the chalchopyrite structure when the cations are ordered [8].Chalchopyrite (CuFeSe 2 ) can be viewed as a doubled zincblende lattice, where the Cu and Fe cations order.Now, the ordering of the ions causes the unit cell to enlarge, lowering the symmetry and changing the space group, with concurrent changes to the diffraction pattern.Another well-known example of such ordered superstructures is the double perovskite structure [9].In the case of alloys, Heusler alloys are a common example of ordered intermetallic compounds [10].
Whether a compound has ordered or disordered atoms can often, but not always, be distinguished by XRD.
The larger unit cells and/or lower symmetry of ordered compounds may result in additional diffraction peaks or changes in the intensity of peaks.If the ordered ions have very similar x-ray scattering factors, which are determined by the number of electrons in the ion, then XRD may not be able to detect their ordering and may not be able to distinguish between a material with compositionally disordered ions and one where the same ions are ordered.
Some of the issues relating to defining a new material are now clear.For a material to be new, it must be different to every other material.But different how? Materials with different crystal structures usually have distinct diffraction patterns and can therefore be considered by many to be different.Doped materials may have very similar diffraction patterns to the parent material but their properties may change markedly.Likewise, in the case of solid solutions, if the arrangement of constituents is random on large length scales, it would have diffraction patterns intermediate between those of the end members.The question of whether doped silicon is a different material to undoped silicon, or whether a solid solution with x = 0.1 is a different material to one with x = 0.2, may elicit different answers depending on the context or field.We would wish to highlight that a claim of a new material should therefore be accompanied by an explanation of how it relates to currently known materials and what differences in structure and composition, or other factors, distinguish them.
Interestingly, it seems that many of the predicted new materials, both in the Materials Project and the larger Google GNoMe, fall in the category of structurally new materials.We have certainly not looked through all predicted new materials in the GNoMe data set-not even a large fraction.However, in the synthetic targets in the Alab paper [4], we see a clear trend.The predicted new materials can very often be derived from known compounds, in which ions were ordered, rather than fractionally occupied and disordered, within the same aristotype as the known parent.If these ordered structures were synthesized, many indeed would qualify as a new crystallographic compounds.If the key characteristic that distinguishes a new material from a known one is cation order, then the cation order needs to be proven to be real, as otherwise the material would be identical to, or a very similar doped version of, the already known disordered version.We would also state that we have noticed similar issues in the GNoMe database when only looking at a small fraction of the predicted materials.
How does A-lab define new materials?They state that they chose targets "from the Materials Project that were marked as 'theoretical' (that is, not represented in the ICSD)."It seems as if the criterion for novelty of a material is its presence in the Materials Project (MP) and its absence from the ICSD.This criterion is open to criticism; e.g., MP contains no compositionally disordered compounds and many known compounds are not in the ICSD, especially disordered ones) but, nonetheless, we will mostly use this criterion to assess the novelty of the A-lab synthesis products.

III. HOW TO PROVE THAT A SYNTHESIZED MATERIAL IS NEW
With ideas of what defines a new material in mind, we can now consider the evidence necessary to determine if one has been produced.The creation and testing of hypotheses is a fundamental feature of science.The best strategy for testing a hypothesis can depend on the context; a positive testing strategy is one that looks only for evidence that confirms the hypothesis.It can be appropriate in some special situations where only one working hypothesis exists but in general it is undesirable and inappropriate adoption of this strategy is a well-known cognitive bias [11].Given the large number of materials now known, any hypothesis about discovery of a new material cannot be tested solely by confirmation but must be also tested against falsification-i.e., tests should be carried out to determine if the sample under investigation is instead a known material.Any known material that might realistically form under the synthesis conditions should be considered as a candidate for such testing.For example, if a synthesis is carried out using three elements, X, Y, and Z, with the intention of forming the ternary compound XYZ, it is prudent to assess whether the diffraction pattern (or any other analysis) can instead be explained by known compounds that can be formed by the reactants, e.g., the binary compounds XY, XZ, and YZ, or other ternary compounds, such as XY 2 Z 4 .Likewise, the unreacted starting materials should also be eliminated from enquiry and unintended contaminants ruled out.If known materials can adequately explain the experimental evidence, then there is no need to conclude that new materials have been formed.This is a statement of Occam's razor, which has also been expressed in similar terms by Russell: "Whenever possible, substitute constructions out of known entities for inferences to unknown entities" [12].
A positive testing strategy, one that only looks for confirmation of the hypothesis of the presence of a new material and does not look for alternative explanations involving known materials, is inadequate in a field as well-established and densely populated as that of materials chemistry.Instead, any report of a new material must be accompanied by an explanation as to why the experimental evidence is better explained by a new material, compared to one or more known materials.In some instances, powder x-ray diffraction might not be capable of providing evidence that can differentiate two materials.Even under perfect experimental conditions, there is information loss by the nature of the PXRD experiment and this means that there is no one-to-one correspondence between the PXRD pattern and the structure, so that many theoretical structures may give identical diffraction patterns.Therefore, even if an excellent match between model and experimental XRD can be achieved, this still does not guarantee that the modeled compound is the correct one.Schlesinger et al. [13] point out that the "• • • mere existence of a plausible crystal structure, a good Rietveld fit with a smooth difference plot, acceptable R-values and a successful checkCIF test does not justify the attribute 'correct structure'." If two candidate materials have very similar diffraction patterns, it may be that PXRD cannot distinguish them and other techniques must be employed to prove which has been made.When fitting a PXRD pattern, just as when fitting any other data, the most reasonable fit is achieved when the number of fitting parameters is kept as low as possible.In a crystallographic setting, lowering the symmetry of the space group will increase the number of variables in the fit.For this reason, it should be ensured that the improvement of a fit is meaningful when symmetry is lowered.Should the quality of a fit in a high-symmetry model be comparable to that of a lower-symmetry model, one should pick the former one, again in accordance with Occam's razor.
Many learned societies, such as the American Chemical Society, the German Chemical Society, and the Royal Society of Chemistry (UK), require not only structural but also compositional information on newly reported materials.Several techniques are available to the solid-state chemist: energy-dispersive x-ray spectroscopy (EDX), inductively coupled plasma optical emission spectroscopy (ICP OES), x-ray wavelength-dispersive spectroscopy (WDS), x-ray fluorescence spectroscopy (XRF), x-ray photoelectron spectroscopy (XPS), and electron-energyloss spectroscopy (EELS) are well-known examples.The use of any of these techniques will, however, usually yield the average elemental composition over a large volume of the sample.This is accurate as a measure of material composition if the sample in question is one pure material but if the sample is a mixture of materials, then the composition analysis will return an average, which may be unrepresentative of the specific material under investigation.In solid-state chemistry, this can be a significant problem, as separation of mixtures is much more challenging than in, e.g., solution-phase chemistry.If a solid-state reaction produces a mixture and none of the components can be easily dissolved, sublimed, or otherwise removed (a very common scenario with oxide chemistry), it might be challenging to accurately measure the elemental composition of the target material.Thus, synthesis of highly phase-pure samples is normally an important part of newmaterials discovery, as this is the best route to accurate compositional information.Naturally, phase-pure samples have many other advantages when it comes to measuring functional material properties.The standard set by the TABLE I.The distribution of errors in the 36 claimed "successful" syntheses.The "X" symbol denotes that the error is present.Error 1 is a very poor fit, such that the fitted model is meaningless.Error 2 is where a different CIF has been used for refinement compared with that in the paper and in the Materials Project.Error 3 is where the predicted structure has ordered cations but there is no evidence for order, and a known disordered version of the compound exists.Error 4 is where the compound is correctly identified but is already reported.A-lab paper of > 50% purity being "success" is therefore anomalous in the usual practice of solid-state chemistry.

IV. ANALYSIS OF THE A-LAB DATA SET
Below, we go through the materials that have been claimed to be successfully synthesized in the A-lab paper [4].We summarize our findings here, before going through many examples in detail.The classification of samples by A-lab themselves is as follows.There are 58 compounds mentioned in total.Of those, 15 are classified as failure, seven as partial success (meaning less than 50 wt% in the final product), and 36 are "successes" (including two that were successful offline, meaning with human intervention in the synthesis).
Within the 36 samples classified as successes, we have found that the analysis presented for 35 of them has suffered from one or more of the error types described below: (1) Very poor and obviously incorrect fits.This means models that are such poor fits to the data, often missing intense diffraction peaks, that they cannot be relied upon either for proof of the structure of the compounds or their purity.The poor fitting leads to the inability to identify impurity phases.Since the authors aim to have > 50 wt% of their product, it is important to identify what other materials are present in order to assess if the 50% threshold has been met.Additionally, the presence of unreacted starting materials is symptomatic of an incomplete reaction and incorrect reaction conditions.This error type is present in 18/36 compounds.(2) Changing the prediction to match the observed XRD pattern.In several cases, the crystallographic information file (CIF) supplied in the Supplemental Information is not the same as that claimed in the main paper.The A-lab paper states "For targets for which we suspect the poor fit resulted from configurational disorder, we refined the XRD patterns using cation-disordered versions of the structure of the target taken from the Materials Project."In effect, the prediction has been changed to a disordered version of the compound if the ordered version did not fit.However, in the large majority of cases, as we show below, the disordered version actually exists already in the ICSD, meaning that the material is in fact already known.An example is Mg 3 NiO 4 , which we discuss below.This error is present in 8/36 compounds.(3) No evidence for cation ordering.The most common error is prediction of compounds that are ordered versions of known disordered compounds.For example, as we will show in detail below, the existence of MgTi 2 NiO 6 is claimed, which is the same as the known ilmenite structure of the same composition, but the predicted structure has ordered Mg and Ni cations, whereas the known structure has those cations disordered.However, no consideration is given by the authors to the possibility that they may have in fact made the known disordered compound instead of their intended compound.We show below that this is in fact the most likely situation.This error type is present in 24/36 compounds.(4) Reporting existing compounds as new.In several cases, the claimed new compounds are in fact already reported in the ICSD.This error type is present in 3/36 compounds.
Below, we discuss the 43 materials (which includes the partial successes), going into detail in many cases, to highlight the consistencies of the errors described above.We group the materials by structure type for this discussion.
For the analysis, the original published experimental XRD patterns were obtained by digitalizing the data provided in the Supplemental Information of the A-lab paper using GetData Graph Digitalizer.Because the software had trouble identifying the green dots that represented the observed XRD data, the experimental data were obtained by combining the calculated fit with the residual of the fit.To align the x-axis values for combining, the acquired XY data were then interpolated with the data in ORIGIN.This process is certainly not ideal and yields data of lower quality than the original.Nevertheless, we have found that it is possible to carry out Rietveld refinement on these data sets.This has been carried out in GSAS-II, the software used by A-lab [14].We do not have the experimental parameters for the original data collection and so peak profiles have been determined empirically.We do not claim that our fits are definitive or cannot be improved upon but we highlight in each case the features that make us believe that the fits we propose are superior to those provided in the original paper.We note that the scale of the /σ subplots of our refinements is arbitrary.

A. Rock-salt structured materials
As mentioned in Sec.I, the rock-salt or NaCl structure can host solid solutions when different cations or anions are mixed on their respective sites.In this case, the space group does not change and the structure type remains rock salt.Should those cations order, however, both the space group and the structure type would change.
Mg 3 NiO 4 (mp-1099253) is predicted to exist in the primitive cubic space group Pm 3m.The predicted structure can be viewed as a rock-salt structure where the cation order breaks the F-centered lattice [see Fig. 1(a)].A very similar composition, MgNiO 2 , has been reported to exist in a rock-salt structure (space group Fm 3m, ICSD-290603), where no cation order was observed [see Fig. 1(b)] The sample synthesized by A-lab with the composition Mg 3 NiO 4 has been claimed to be a successful synthesis after human intervention in space group Pm 3m.However, the provided structure file, which can be found in the Supplemental Information of the A-lab paper and is shown in Fig. 1(c), has disordered cations; hence the CIF used for fitting by A-lab has the space group Fm 3m.The powder x ray pattern can be relatively well indexed with space group Fm 3m [as shown in the Supplemental Information in the paper and also in Fig. 1(c)] but lacks additional peaks (e.g., the 100 peaks around 21.1 • ) that would appear in the original proposed space group Pm 3m (see Fig. 1).In fact, the powder pattern and relevant systematic absence conditions of h + k, h + l, and k + l = 2n agree very well with the known compound MgNiO 2 , as shown in Fig. 1; however, the peaks appear at slightly different diffraction angles, which may suggest a doped material or a solid in the A-lab paper, the indexed peaks (tick marks) do not line up with the diffraction peaks; thus we believe there has been an error in producing the image in this case.We conclude that the synthesized compound is actually a member of the MgO-NiO solid-solution series, with disordered cations, that has been studied for many years, and not the cation-ordered material predicted by the Materials Project.Nonetheless, this does not mean that the proposed ordered material cannot be synthesized.However, it will require a different synthetic route to potentially stabilize Mg 3 NiO 4 with ordered cations.Here, the clear distinction between the PXRD of the ordered and disordered material allows for easy identification of the former.The analysis of Mg 3 NiO 4 suffers from errors 2 (different structure used in refinement than was predicted) and 3 (no evidence for cation order in the predicted structure).Mg 3 MnNi 3 O 8 is predicted by A-lab to exist in the R 3m space group.A compound with the same composition exists in the ICSD (ICSD-80306), reported by Taguchi et al. in 1995 [19].The reported compound is cubic, a variant of the rock-salt structure, sometimes called murdochite, with octahedrally coordinated metals; the Mn ions form an fcc arrangement, while the Mg and Ni ions are disordered on a different site.There is also an additional cation vacancy compared with the parent rock-salt structure.The A-lab structure is exactly the same, except that the Mg and Ni ions are now ordered and the particular ordering reduces the symmetry to R 3m.The largest effect on the calculated diffraction pattern of this ordering is an increase in the intensity of the peak around 18.4 • .In the murdochite phase (Fm 3m), the (111) peak at 18.4 • has an intensity of 33% of the most intense reflection, whereas ordering of the Mg and Ni ions as increases the intensity of this peak to 59% of the most intense reflection.This difference should be easily detectable by the PXRD methods used.Turning to the reported PXRD pattern and refinement by A-lab, it is clear that many of the intensities from the model are very poor matches to the experimental data.Most obviously, the model greatly overestimates the intensity of the reflection at 18.4 • .This suggests that the predicted ordering is not present.The generally poor agreement in intensities may point to multiple phases being present in this sample.Simple rock-salt oxides, such as MgO and NiO, have intense peaks that coincide with some of the murdochite peaks (unsurprising, as they are based on the same structural motif) and so the incorrect intensities may be due to the presence of rock-salt phases.There is also an almost completely unmodeled peak in the experimental pattern at just over 30 • .This peak is not present in the disordered Mg 3 MnNi 3 O 8 , nor is it a rock-salt (MgO or NiO) peak.It is, however, present at reasonable intensity in the pattern of NiMn 2 O 4 spinel and to us this (or a similar spinel) seems the best candidate to explain that peak.The sample therefore may consist of multiple known phases: rock salt, spinel, and murdochite, with the Mg, Mn, and Ni possibly distributed across all these phases.The evidence from the peak intensities is clearly against the proposed Ni-Mg ordering.See Fig. 14 in the appendix for our alternative refinement using the phases described above.The analysis of Mg 3 MnNi 3 O 8 sufferers from errors 1 (very poor fit) and 3 (no evidence of cation order).

B. Layered materials
Layered materials are of significant interest in materials science, as they provide the foundation of many applications, including most battery-electrode materials [20].Among the 43 materials that A-lab has synthesized, there is one layered compound, KMn 3 O 6 .This compound has been predicted by the Materials Project (mp-1016190) to crystallize in space group C2/c.The structure can be viewed as related to α-NaFeO 2 , which consists of layers of edgesharing FeO 6 octahedra with Na cations between the layers and crystallized in space group R 3m.Variations of this structure exist in several space groups, where the layer stacking causes symmetry change [21].Cation order in the transition-metal layer can lower the symmetry to the monoclinic space group C2/m [22].The proposed structure of KMn 3 O 6 does not have cation order on the transition metal site but proposes ordered vacancies of K [Fig.2(a)].In contrast, the structure that is reported in the Supplemental Information of the A-lab paper [4] has disordered K and the actual space group of the provided structure is C2/m, not C2/c [Fig.2(b)].This is still low symmetry for a material, which might be better described as K 0.33 MnO 2 .Intuitively, one would expect that K 0.33 MnO 2 would adopt one of the α-NaFeO 2 -structure variants, which usually have hexagonal or rhombohedral symmetry.The low symmetry in the predicted material likely arises from the slight buckling of the layers, which can be seen in Figs.2(a) and 2(b).Both the K order as well the buckling of the layers would likely define this material to be new, but at this point it is unclear if either of those are present in the synthetic product described in the A-lab paper [4].For example, K 0.3 MnO 2 has been reported in the hexagonal space group P6 3 /mmc, in a structure that belongs to one of the stacking variants of the α-NaFeO 2 -structure [23].A compound of almost exactly the same stoichiometry is in the ICSD (ICSD-156081).This structure is shown in Fig. 2(c); it has neither ordered K nor buckled layers.K 0.3 MnO 2 is known to result from the thermal decomposition of KMnO 4 above 800 • C [23].As A-lab's reaction conditions have included a 1000 • C heating step, K 0.3 MnO 2 is a likely product.The PXRD fit for KMn 3 O 6 provided in the A-lab paper [4] is of very poor quality and misses some major reflections, as shown in Fig. 2(d).Comparing it to the simulated PXRD pattern of K 0.3 MnO 2 [Fig.2(e)] reveals that the main measured reflections are well reproduced by those simulated to appear K 0.3 MnO 2 .Still, there is an intensity mismatch, which could be caused by preferred orientation, impurities, or additional phases.There are many layered K-Mn-O materials in the literature that could also explain the PXRD data.Thus, proof for the proposed structure is lacking and the more likely explanation for the synthetic product is one, or a combination of, known layered K-Mn-O phases.The analysis of KMn 3 O 6 suffers from errors 1 (poor fit), 2 (CIF file not the same as originally predicted), and 3 (no evidence of cation order).

C. Pb-Sb pyrochlores
Pyrochlores are structures that have formula A 2 B 2 O 7−δ , with A and B denoting two possible cation sites and δ the possible oxygen defect [24].Stoichiometric pyrochlores, where δ = 0, feature metal ions with one of two combinations of formal charge: A(II)B(V) or A(III)B(IV).The compounds of interest here all have an A site of Pb(II) and in general lead pyrochlores typically show significant nonstoichiometry: A-site vacancies or Pb(IV) defects on the B site are compensated by oxygen vacancies, i.e., δ > 0. Furthermore, the B site in lead pyrochlores can be occupied by two different cation species; e.g., if half of the B(V) site is replaced by a M(IV) cation, the resulting formula is Pb 4 M 2 Sb 2 O 13 -these compounds are known for M(IV) = Ti, Zr, Hf, Sn, and others.Alternatively, one quarter of the B(V) sites can be replaced by M(III) ions, yielding compounds with stoichiometry Pb 4 M 1 Sb 3 O 13 ; example M(III) cations that have been incorporated in this way are In, Al, Sc, Cr, Fe, Ga, and Rh [25,26].
The Materials Project predicts the stability of various compounds with the formula Pb Note that the naming convention for pyrochlores is that the A site is the larger metal ion of lower charge and appears first in the formula but to avoid confusion we will use the compound names given in the A-lab paper, which in each case places the B-site ion first.It should be pointed out that for each of the five compounds listed above, there exists a reported disordered version of the material on the ICSD (ICSD-62722, 62721, 60805, 41119).In fact, all of the B sites in question have been incorporated into lead antimony pyrochlores by Cascales and coworkers in a series of papers in 1985-86 and each is reported by them as consisting of random B-site substitutions, retaining the pyrochlore symmetry Fd 3m [25,26].
Since the predicted structures for these five compounds are just B-site ordered versions of the known disordered structures, we again emphasize that evidence of the ordering must be found in the characterization in order to prove the formation of the ordered phase.We note that the x-ray scattering factors between M and Sb are very similar for M = In, Hf, and Sn, meaning that any ordering of these elements in the synthesized materials would be very difficult to detect via XRD.
The ordered B-site cations in the predicted structures lead to a lower-symmetry unit cell compared with the cubic pyrochlore of the parent Pb 2 Sb 2 O 6.5 .However, if the atom type is ignored, the atom positions are almost the same as in the cubic pyrochlore.We have calculated the pseudo-cubic lattice parameter, a p , of the ordered CIFs produced by A-lab, as shown in Table II  we discuss in detail below.Other compounds have closely matching parameters and with no evidence of order we conclude that they are very likely the known disordered pyrochlore compounds discovered in the 1980s.Figure 3 shows that the calculated diffraction pattern of the known phase Pb 2 SnSbO 6.5 (ICSD-62722) and the predicted Sn 2 Sb 2 Pb 4 O 13 are almost completely identical.Although the B-site ordering in the predicted phase lowers the symmetry, the intensities of the additional reflections are very weak.As such, alternative analytical steps are required to assert the existence of B-site ordered Sn 2 Sb 2 Pb 4 O 13 .And, most importantly, it needs to be verified that this material is different from Pb 2 SnSbO 6.5 [27].We find that the Rietveld refinement of the PXRD data from Sn 2 Sb 2 Pb 4 O 13 , shown in Fig. 4, can be carried out successfully using the known compounds Pb 2 SnSbO 6.5 and SnO 2 (ICSD-9163), both reported in the ICSD.Thus using the argument of Occam's razor, we conclude that this is the more likely interpretation of the synthetic products.In general, our attempts at Rietveld refinement of the data from all five of the reported pyrochlore samples, as shown previously in Fig. 4 and in Figs.11, 12, 13, and 15 in the Appendix, indicate that the synthesized phases are likely the known disordered B-site pyrochlores crystallizing in the higher-symmetry space group Fd 3m.Thus, the pyrochlores are all an example of error 3 (missed disorder or no evidence of cation order).
Notably, all these materials are related to the famous "Naples Yellow" pigment, which derives from Pb 2 Sb 2 O 7 [28].Variants of Naples Yellow, including those with Sn(IV) substitution on the B site, were used by the ancient Egyptians, and have been lost and then rediscovered periodically throughout history, by different ancient civilizations, in the Middle Ages, at various points in the Renaissance, and most recently by A-lab.Interestingly, it has been debated that Pb 2 Sb 2 O 7 itself is not stable and that doping (most commonly with Sn but also with other elements) is necessary to stabilize the pigment.For an excellent overview, we recommend the work of Marchetti et al. [28] and the references therein.indicate that the diffraction patterns can be better or equally well interpreted as known B-site disordered cubic spinels, all of which have been tabulated in the ICSD.
NiMn 2 O 4 is a cubic spinel with complex cation and magnetic ordering.Guillemet-Fritsch et al. have explored Zn insertion into NiMn 2 O 4 [29], including synthesis of a compound with almost exactly the same stoichiometry as the A-lab material Mn 4 Zn 3 (NiO 6 ) 2 .In the work of Guillemet-Fritsch et al., the A and B sites have been found to be compositionally disordered, with some Zn migration onto the octahedral sites, forming a disordered arrangement with the Mn and Ni ions (ICSD-92222-92224).However, the prediction from A-lab is of the A site (tetrahedral site) being exclusively occupied by Zn and the B site (octahedral) with ordered Ni and Mn.In Fig. 5 the PXRD data from the A-lab sample Mn 4 Zn 3 (NiO 6 ) 2 are refined against the existing compounds.The predominant phase in this pattern matches well with the disordered cubic spinel phase (Zn 0.759 Mn 0.241 )(Mn 1.35 Ni 0.65 )O 4 (ICSD-92223) in space group Fd 3m, with minority peaks indexing to NiO.The analysis of this compound, again, suffers from error 3.
Another synthesis to consider is the one that has targeted Zn 3 Ni 4 (SbO 6 ) 2 .Gama et al. [30] have studied the inverse spinel Zn 7 (SbO 6 ) 2 and Ni-substituted variants.As its concentration increases, Ni replaces the Zn on the octahedral site, progressively converting the material to the normal spinel structure, which it attains at the composition Zn 3 Ni 4 (SbO 6 ) 2 , i.e., the exact composition predicted in the A-lab paper.Gama et al. have found the Ni and Sb to be disordered on the B site.Given the large x-ray scattering-factor difference between Sb and Ni, detection of B-site order should be very straightforward by PXRD.The structure obtained from the Materials Project and described in the paper, mp-1216023, has ordered Ni and Sb ions on the B site.However, in this case, as has been seen previously, the Materials Project structure differs from the structure file provided in the Supplemental Information.While the symmetry of both the structure predicted in the Materials Project and the structure reported in the A-lab paper have the same symmetry (space group C2/c), in the latter, the Ni and Sb ions are disordered (having fractional occupancies in the CIF) [4].In Fig. 6, we compare the predicted structure from the Materials Project (mp-1216023) with its calculated diffraction pattern [see Fig. 6(a  Our own refinement is given in Fig. 7.We find it necessary to include impurity phases NiSb 2 O 6 (ICSD-426852) and NiO (ICSD-9866) to match all the Bragg peaks.We have attempted the fit with Zn-Ni-Sb spinels with different Ni contents from the ICSD.We show the fit with Zn 6 NiSb 2 O 12 (ICSD-109465) but note that other Ni contents give very similar fits and that we do not think we can differentiate between them.
Therefore, in this case, the ordered B-site spinel from the Materials Project is clearly not present, as the strong low-angle ordering peak that is predicted to appear at 17.9 • is absent from the experimental data.The spinel phase that is present matches well with known Zn-Ni-Sb spinels with disordered B-site cations.We conclude, in the absence of any further evidence, that this is not a new material.
The compound Zn 2 Cr 3 FeO 8 is predicted by A-lab to have ordered Cr and Fe ions on the B site but is otherwise identical to the normal spinel structure.A series of solid solutions between ZnCr 2 O 4 and ZnFe 2 O 4 , including the exact composition predicted, were reported in 1970 and the B-site cations were described as disordered (ICSD-167363) [31].Due to the small difference in scattering factor between Fe and Cr, there is only a minimal difference in the PXRD pattern between the predicted ordered structure and the known disordered structure.In addition, the reported PXRD pattern of Zn 2 Cr 3 FeO 8 also contains several unmodeled impurity peaks, which likely correspond to binary oxides of the metals.In the absence of any other evidence, we conclude that Zn 2 Cr 3 FeO 8 is not a new material and is the disordered B-site spinel described in 1970.Thus this falls under error 3.

E. Perovskites and ilmenites
Perovskite oxides have the general formula ABO 3 and their characteristic motif is exclusively corner-sharing BO 6 octahedra.Ilmenites have the same general formula but the octahedra are edge-sharing and the structure becomes hexagonal or rhombohedral.
A-lab has reported the synthesis of six perovskiteor ilmenite-derived structures: Ba 2 ZrSnO 6 (Fm 3m, perovskite), MgTi 2 NiO 6 (R3, ilmenite), and KBaGdWO 6 and KBaPrWO 6 (F 43m, double perovskite), as well as Ba 6 Na 2 Ta 2 V 2 O 17 and Ba 6 Na 2 V 2 Sb 2 O 17 (P6 3 /mmc, perovskite derivatives).However, our analysis suggests that the powder patterns of these samples more likely correspond to a doped and known perovskite-ilmenite phase mixed with some impurity phases.We will elaborate on this with two detailed examples.We would also like to point to the recent preprint by Yamamoto et al., which provides more insight into the supposed synthesis of Ba 6 Na 2 Ta 2 V 2 O 17 [32].There, the authors explain that both Ba 6 Na 2 Ta 2 V 2 O 17 and Ba 6 Na 2 V 2 Sb 12 O 17 have been mentioned as existing before but the structure has not been solved.They solve the structure as a 6C-type perovskite rather than the predicted 12H type, in disagreement with the materials prediction.
In the case of Ba 2 SnZrO 6 , the output of the refinement process, as published in the A-lab paper, contains many significant unmodeled diffraction peaks (Fig. 8).Our fit is shown below.We are able to account for all obvious Bragg peaks and the residuals that remain are characteristic of incorrect peak profiles rather than completely unmodeled diffraction features.Our fit has necessitated four phases: SnO perovskite.This contrasts with the Materials Project entry of the stated material, which has ordered Sn and Zr ions and, consequently, a doubling of the lattice parameter.The small difference in x-ray scattering factor between Sn and Zr means that simulated patterns of the ordered Ba 2 SnZrO 6 phase show only very small changes compared with the disordered phase.The analysis of Ba 2 ZrSnO 6 thus suffers from errors 1 (poor fit), 2 (inconsistency in predicted and reported structure), and 3 (no evidence of cation order).MgTi 2 NiO 6 with space group R3H is predicted to exist by A-lab as a new compound in a structure that is close to what is known in the literature as the Ni 3 TeO 6 structure, which can be understood as an ordered ilmenite structure.The ilmenite structure has space group R 3H ; the difference in space group between the ilmenite and Ni 3 TeO 6 structures is due to the fact that in the ilmenite structure, there is only one crystallographic A and B site, but with the addition of cation ordering, the symmetry changes to R3H , reflecting the fact that there are now two crystallographically distinct A sites and two distinct B sites.The differences are shown in Figure 9.The calculated diffraction patterns of MgTi 2 NiO 6 in the ilmenite and Ni 3 TeO 6 structures differ only in the intensities of the diffraction peaks-the nature of the ordering does not allow new reflections that are absent in the disordered structure.Similarly to previous examples, for this particular compound, due to the electron densities of the constituent atoms, the intensity difference between the ordered and disordered phases is very small.Therefore in our opinion, to distinguish the ordered from the disordered phase by PXRD will be exceptionally challenging, but if it is to be attempted, then very careful measurement of the peak intensities and explicit comparison with the ordered and disordered models need to be made.When we turn to the diffraction pattern presented in the A-lab paper [4], we find that the pattern for MgTi 2 NiO 6 can be adequately fitted with a model of the known ilmenite phase Ni 0.5 Mg 0.5 TiO 3 (ICSD-171583) and a small impurity (2 wt%) of NiO.This, again, is an example of error 3. It may be that exceptionally careful analysis of the diffraction intensities can provide evidence that the new ordered phase has been made.But no such evidence is given in the A-lab paper [4].We conclude that in the absence of evidence of the new phase, the explanation involving known phases is preferred.Again, as for all other examples, this does not mean that the predicted ordered phase cannot be synthesized.More careful analysis and changes in synthetic recipes can potentially lead to the successful synthesis of the predicted phases in the future.
For the double perovskites KBaGdWO 6 and KBaPrWO 6 , the fits are so poor as to be meaningless in our view, with clear unfitted Bragg peaks in both cases.Both these structures have order on both the A and B sites (so called double-double-perovskites).A site order in perovskites is much rarer than B-site ordering and is almost always accompanied by oxygen deficiency [33]-a classic example is the cuprate superconductor YBa 2 Cu 3 O 7 .
In the absence of oxygen deficiency, A-site order can rarely occur but has, to our knowledge, not been seen in the symmetry predicted here-typically, A sites form ordered layers driven by large charge differences, rather than fcc patterns [34].Given the novelty of what is being proposed here, clear evidence must be given that the predicted compounds have been formed, with special attention paid to small diffraction features that characterize the ordering.This evidence is not supplied by the poor fits to the PXRD data.

F. Phosphates
Phosphates are ionic compounds that contain PO 3−   4   or related anions.They can be highly complex, featuring many crystallographically distinct anions, giving rise to a large unit cell with low symmetry and a complex associated diffraction pattern As mentioned previously, one must be careful in refining such complex structures, because the abundance of reflections and atomic positions may lead to meaningless fits that have too many parameters.As such, a pure sample, high-quality PXRD data, and other identification methods are imperative to assert the synthesis of a new phase, if the structure is highly complex and of low symmetry.Errors 1 (poor fit) and 3 (no evidence of cation order) are very common in the analysis of the phosphates.
The compound K 2 TiCr(PO 4 ) 3 has been predicted to exist as a new cubic phase in the space group P2 1 3. Figure 10(d) shows our refinement of the provided PXRD pattern, using known cubic K 2 Ti 2 (PO 4 ) 3 (P2 1 3; ICSD-202888) and Cr 2 O 3 , a common impurity in hightemperature synthesis of oxides containing chromium [35].The refinement provided in the A-lab paper has several unfitted peaks, all of which correspond to the Cr 2 O 3 impurity phase as marked by the red arrows in Fig. 10(c).The example of K 2 TiCr(PO 4 ) 3 shows that there are serious issues with the supposed synthesis of the phosphates; in fact, we could index and preliminarily match all 18 PXRD patterns to materials that are reported in the ICSD (see Table III).We consider it to be the responsibility of the authors of the A-lab paper [4] to unambiguously prove the synthesis of the target materials in all cases and we will refrain from providing alternative refinements of all 43 materials in this Perspective.We will, however, discuss each compound and possible alternatives briefly below.KNaP 6 (PbO 3 ) 8 has been predicted to be in space group P3, with ordered vacancies that leave tunnels in the structure.The structure provided in the Supplemental Information of the A-lab paper [4] has a higher symmetry and the space group P6 3 /m, although the authors incorrectly claim they have synthesized the target compound in space group P3.The structure provided in the Supplemental Information not only has a higher symmetry but it also places K, Na, and Pb on the same crystallographic position.We have found that the PXRD pattern of the synthetic product could also be indexed with a combination of the known materials (Na, K)Pb 4 (PO 4 ) 3 and Pb 8 O 5 (PO 4 ) 2 (ICSD-182501, 98702), which provides an alternative interpretation of the data.
There are four predicted compounds in the A-lab data set based on the Ni 2 (PO 3 ) 4 structure, a kind of tetrametaphosphate with two inequivalent metal sites, both octahedrally coordinated but with slightly differently sized coordination environments.This M 2 (PO 3 ) 4 structure type is known to form with a variety of M(II) cations, namely, M = Mg, Mn, Fe, Co, Ni, Cu, and Zn.These previously reported compounds are listed in the ICSD.The fact that there are two distinct metal sites offers a possibility that two metals could order over these sites.A few of these bimetallic tetrametaphosphates ((M, M ) 2 (PO 3 ) 4 ) have been synthesized previously, namely, (M,M') = (Ni,Zn), (Ni,Co), and (Mg,Mn).Nord has investigated the ordering for bimetallic tetrametaphosphates with (M,M') = (Ni,Zn) and (Ni,Co), using neutron diffraction, as XRD has only small scattering-factor differences between these metals [36].It has been found that there is a slight preference for the Ni(II) ion to occupy the smaller octahedral site due to its lower ionic radius and thus some ordering of the cations is observed.In contrast, in a recent study on MgMn(PO 3 ) 4 using XRD, no cation ordering has been observed [37].Experimental data are presented by A-lab for CaCo(PO 3 ) 4 , CaMn(PO 3 ) 4 , and CaNi(PO 3 ) 4 .It should be noted that unlike for the bimetallic tetrametaphosphates discussed above, which could be thought of as solid solutions between two known M 2 (PO 3 ) 4 compounds, the pure Ca 2 (PO 3 ) 4 compound in this structure is not known, perhaps because Ca(II) is much larger than any of the other M(II) ions listed above that can form this structure.Despite this, it might be possible that Ca(II) could enter into the (M,M') 2 (PO 3 ) 4 Ca has entered the tetrametaphosphate phase, as the PXRD patterns of the bimetallic phases are almost identical to that of the singlemetal analogue.For example, the proposed CaNi(PO 3 ) 4 simulated PXRD is almost identical to that of the known compound Ni 2 (PO 3 ) 4 .Without compositional information, careful measures of intensity of the PXRD peaks, or the use of alternative diffraction techniques such as neutron diffraction, there is no way to confirm whether the Cacontaining compounds have been made.The same is true for the last of these compounds, MgNi(PO 3 ) 4 -although this is a combination of two metals known to form tetrametaphosphates, there is no way to know from the data presented that the material produced contains both metal ions or whether they are ordered as predicted.The fit for this particular sample is also much poorer than for the others discussed in this paragraph.The compound NaMnFe(PO 4 ) 2 is predicted as a kind of olivine structure but the diffraction pattern is not well fitted by this model and resembles more closely an alluaudite phase, an orthophosphate structure, of which many compounds are known containing Na, Mn, and Fe metals in different proportions, which are included in the ICSD [38].We consider this the more likely identity of the material.
The compound Mn 2 VPO 7 is the thortveitite structure, which is known for both Mn 2 P 2 O 7 and Mn 2 V 2 O 7 .It is reasonable to suggest that a solid solution might be formed between these two phases, representing a phosphate-vanadate compound that does not seem to be in the literature.However, as in many cases here, to differentiate between the predicted phase with ordered vanadate and phosphate ions and either of the known compounds Mn 2 P 2 O 7 or Mn 2 V 2 O 7 by PXRD requires careful measurement of the peak intensities, which change only by a small amount, as V is reasonably close to P in electron density.This analysis has not been done and the peak fit shows clear deviations from the observed intensities.Therefore, no evidence for the formation of the mixed-anion phase, nor for ordering of that phase, has been provided.
The predicted compound MgTi 4 (PO 4 ) 6 is a known compound, reported by Barth et al. in 1993 (ICSD-74287) [39].The proposed structure has ordered Mg-ion vacancies.If the Mg-ion vacancies were disordered, then the small peak seen experimentally at around 17 • would be absent.This peak has also been observed by Barth et al., who have discussed the interpretation of this peak as being due to partial ordering of the Mg vacancies.Therefore, neither the compound nor the reported (partial) ordering is new.In addition, a second structure is known to form from this composition, reported in 2008 in a different structure [40].
The predicted compound KNaTi 2 (PO 5 ) 2 is a cation-ordered version of a series of titanyl phosphates that have been studied by various groups; e.g., Norberg et al. have reported one such disordered version with a very similar stoichiometry to that predicted here in 2003 [41] and an earlier report is listed in the ICSD (ICSD-71239).The predicted ordering causes only minimal changes to the PXRD peak intensities and while it may be possible to discern it with careful measurements and comparison between models, these have not been done, so there is no evidence of the ordered phase.It is likely that the previously reported disordered phase has been made.
The predicted compound MgCuP 2 O 7 is based on mp-1041741.However, the CIF file provided in the Supplemental Information of the A-lab paper has disordered cations, while the Materials Project CIF has ordered Mg and Cu cations.The simulated diffraction pattern fits much better for the disordered version, which is not in the ICSD but has been reported in 1990 [42].
To complete this section, we feel that the fits to the models for the predicted compounds Na 3 Ca 18 Fe(PO 4 ) 14 , Na 7 Mg 7 Fe 5 (PO 4 ) 12 , Mn 7 (P 2 O 7 ) 4 , and K 4 TiSn 3 (PO 5 ) 4 are so poor that they cannot provide any evidence of formation of a new material, in such a complex composition space with a large number of possible candidate phases.We will not explore these particular compounds further.

G. Other materials
Six of the newly reported materials do not fit in the categories we established above and we will discuss them here separately.These are MgV 4 Cu 3 O 14 (P1), CaGd 2 Zr(GaO 3 ) 4 (P1), Ba 9 Ca 3 La 4 (Fe 4 O 15 ) 2 (P1), KNa 2 Ga 3 (SiO 4 ) 3 (P2 1 /c),KPr 9 (Si 3 O 13 ) 2 (P3), and NaCaMgFe(SiO 3 ) 4 (C2).With the exception of KPr 9 (Si 3 O 13 ) 2 , these materials have in common that they resemble highly complex low-symmetry structures.Thus a similar argument as made above for phosphates is valid here: in such low-symmetry structures, a large number of fitting parameters makes refinement extremely challenging and a lot of care needs to be taken to ensure that the low symmetry is real.The data provided do not satisfactorily prove the existence of these phases and for all of the low-symmetry phases above, we have found alternative matches to the PXRD data, which could explain the synthetic product to be a combination of known materials.For example, CaGd 2 Zr(GaO 3 ) 4 is likely a type of garnet, e.g., cubic Ca 0.95 Zr 0.95 Gd 2.05 Ga 4.05 O 12 (ICSD-202850, space group Ia 3d), which matches the main peaks of the pattern well.NaCaMgFe(SiO 3 ) 4 is predicted based on mp-1221075.However, the Materials Project entry has ordered cations, while the CIF provided in the Supplemental Information of the A-lab paper has all four metals completely disordered.This material is, in fact, the pyroxene structure and this structure with many different ratios of Na-Ca-Mg-Fe is listed in the ICSD (e.g., ICSD-417169).Lenaz et al. has described one study into these compounds, which have two crystallographic cation sites, one of which has disordered Ca and Na, while the other has disordered Mg and Fe [43].The experimental pattern from A-lab matches well with the known compounds.KPr 9 (Si 3 O 13 ) 2 , which is predicted to have higher symmetry (R3), again possesses disorder in the CIF file provided in the A-lab paper [4] (between K and Pr), which has not been predicted in the original structure.It thus follows a similar theme to many compounds that we have already discussed in detail.MgV 4 Cu 3 O 14 is the same composition as an existing compound in the ICSD, Cu 1.5 Mg 0.5 V 2 O 7 (ICSD-69731).The predicted compound has ordered Mg and Cu ions and is based on the Cu 2 P 2 O 7 structure, whereas the reported version is the same but with Mg and Cu ions disordered.The main difference in the PXRD pattern due to ordering is the presence of a new peak at 9.3 • , which is absent in the disordered pattern.Sadly, the A-lab data do not go below 10 • , so this confirmatory peak has not been measured.There is no evidence from the PXRD data that the ordered compound has been made over the disordered.

H. Successfully synthesized materials
In our view, three materials have been successfully synthesized as predicted.All of them, however, have been reported in the literature before.They are MnAgO 2 , Y 3 In 2 Ga 3 O 12 , and CaFe 2 (PO 4 ) 2 O, which have been reported, respectively, in Refs.[44][45][46].Of those CaFe 2 (PO 4 ) 2 O seems to have been convincingly synthesized based on the provided PXRD data, whereas the PXRD patterns for the other two are fitted so poorly that it is difficult to state whether the materials have indeed been synthesized.But as those are known compounds, it is easier to believe, based on indexing, that those phases can be present in the PXRD, perhaps in combination with impurity phases.In any case, the compounds in question have been reported relatively recently, between 2021 and 2023.In fact, the authors of the Google DeepMind paper [3] have clarified that they took snapshots of the ICSD in 2021 and thus did not include materials discovered since in their training set.They rightfully view it as a success that materials they predicted based on a 2021 snapshot have since been discovered.

V. OVERVIEW
Above, we have outlined issues with the "new" materials synthesized by A-lab.Of the 36 compounds classed as "successes" and the seven classified as "partial successes" by A-lab (43 in total), we have found significant issues with 42 of them (the exception being CaFe 2 (PO 4 ) 2 O).Thus our view of the success rate is significantly different to the claimed 78%.As discussed above, we could agree that three materials were correctly synthesized, which includes two other known compounds, MnAgO 2 and Y 3 In 2 Ga 3 O 12 .In this case, the success rate would be 3/58, or 5%, which is far away from the claims in the paper.A stricter interpretation of materials discovery (but one usually applied to traditional laboratories claiming a new discovery), that a new material must not have been published previously at all, leads to the conclusion that none of the materials in the A-lab paper would be counted as discoveries.
We have found systematic issues in the analysis of the PXRD patterns, which show that, in this case, AI has failed to correctly derive conclusions from the data.One common error we have found might also point to a general issue with the materials-prediction part of the work, namely, they can often be derived from known compounds in which cation order breaks symmetry.As density-functional theory (DFT) cannot model compositionally disordered atoms easily, there might be an underlying error in the way in which those materials are predicted, both in the Google AI paper and the Materials Project.Still, more work might be needed to verify this concern.Researchers using these data should keep this in mind and this point is expanded upon below.
We assert that A-lab has not successfully synthesized the vast majority of the claimed new compounds.Since we raised issues in the paper shortly after publication, the Ceder group has conceded that A-lab does not live up to human standards but still claim that "the system offers a rapid way to prove that a substance can be made -before human chemists take over to improve the synthesis and study the material in more detail" [47].We hope that our Perspective has made it clear that this statement is not justified-the A-lab paper does not provide proof that the new materials can be made.The data can, in at least 40/43 cases, be interpreted with the opposite conclusion, which is that in these cases, the predicted materials have not been successfully synthesized.Even if those known materials were not in the training set and thus were "new to the A-lab," as claimed by the authors in response, this makes the tile and abstract, which claim "novel materials," highly misleading.Major improvements to the data analysis and the addition of careful compositional characterization are necessary to draw the alleged conclusions.

VI. OUTLOOK
Here, we discuss some perspectives on computational design of new materials and automated labs for inorganic synthesis, from the point of view of experimental solidstate chemists.
Accurate unsupervised AI whole-pattern refinement of diffraction patterns is an important goal for the development of closed-loop automated synthetic laboratories.While the ability to quantitatively measure the goodness of fit of a model to the data appears very attractive from the point of view of automation, it is vital to realize that no value of R wp or χ 2 alone can ever be sufficient to conclude that a model is even approximately correct.Toby has stated that "the most important way to determine the quality of a Rietveld fit is by viewing the observed and calculated patterns graphically and to ensure that the model is chemically plausible" [48].What role does this "human intuition" in assessing the quality of a fit play and how can this be replicated by machine-learning models?To begin, we can consider why goodness-of-fit statistics alone are insufficient.Not all goodness of fit is equal: in a Rietveld refinement, completely unfitted diffraction features may increase the χ 2 value by the same amount as slightly incorrect peak shapes but normally the former are a much more serious concern for the analyst, as they represent at best a missing component of the sample phase composition, or at worst show that the entire model is wrong; e.g., that the real material has a different symmetry to the one modeled.
To understand what more is needed beyond statistical goodness of fit, it is worth recalling that, at its heart, science is the making and testing of hypotheses.In a Rietveld refinement, the immediate hypothesis being tested is whether the model loaded in the software, together with the profile and background functions, mathematically fit the experimental data.But for practical purposes, that is never the entire hypothesis under examination by the scientist; they are concerned with broader questions.As we have seen in the above discussion, if the wider claim is that a new material has been discovered, it is not enough to show a good match between the new material and the data-the fit must be better than that obtained for known materials that are likely to be present.
To give a more specific example, if the hypothesis is that a material has ordered cations, then the experimental diffraction pattern should be considered against candidate models with both ordered and disordered cations.In the perovskite structure, the A and B cation sites are distinguished, and usually occupied by large low-charge ions on the A site and small higher-charge ions on the B site.Differences in size and charge mean that A and B cations seldom mix to any appreciable extent and rarely would it be necessary to test a model of, say, the perovskite SrTiO 3 with Sr(II) and Ti(IV) disordered.But if two cation types are present on the B site, with similar size and charge, these may well mix, or they may order, and in this case consideration of both ordered and disordered models becomes essential.A well-known example from the literature is Sr 2 FeMoO 6 , where the Fe and Mo B site cations can show different degrees of disorder [49].Any diffraction peaks that arise from the ordered but not disordered structure, or any peaks that change intensity appreciably between the two models, are clearly of prime importance.The absence of an expected ordering peak, even if that peak is small compared with the other diffraction features and its absence perhaps makes only a small difference to the numerical goodness of fit, can be fatal for the hypothesis of cation ordering.The answer to the entire research question may depend on the presence or absence of relatively small diffraction features.In other situations, a peak of exactly the same size that belongs to a minor impurity phase may have almost no relevance to the overall research question, beyond suggesting that a small adjustment of the synthetic procedure is needed.
Thus the statistics alone can never capture the full meaning of the fit.In judging the quality of a Rietveld fit, an expert human practitioner will not only look at the statistics and judge the correctness of the chemistry but also consider what possible alternative models need to be compared against and, ultimately, how the PXRD evidence helps address the wider research question.This is the capability that unsupervised AI Rietveld-refinement systems must possess to avoid incorrect interpretations and to truly operate without human intervention.
Several attempts at autonomous interpretation of PXRD data have been made.Mayo et al. have matched experimental patterns of organic polymorphs to a database of calculated structures [50].Lunt et al. have extended this methodology to identify polymorphs of organic materials crystallized and analyzed in a robotic lab [51].Salgardo et al. have used machine learning to classify crystal system and space group from experimental PXRD patterns [52].The three aforementioned studies are important advances and a large part of their value is in clearly outlining the limitations of the methodology.Each has a far more modest goal than unsupervised Rietveld refinement of unknown compounds.
In general, powder diffraction is not necessarily the best method to solve a new crystal structure.Single-crystal xray diffraction is much more accurate in this endeavor but, of course, requires the synthesis of (small) single crystals of the product.Maybe the rate of new-materials discovery can be increased if the analysis step is replaced with single-crystal x-ray diffraction, as such data are often more reliable in providing unambiguous information of the structure of the products.
We have shown throughout our analysis here that the problem of compositional disorder is not well addressed by the methods used in the A-lab paper, with many predicted ordered compounds likely to be known disordered analogues.It should be considered how fundamental this problem is to the materials-prediction field.As discussed earlier, compositional disorder is well accommodated by the equations of crystallography: fractional occupancies can be entered in the structure-factor equation.However, fractional occupancy presents difficulties for many computational methods.This issue has long been recognized.An early approach to the modeling of disordered materials is the virtual-crystal approximation, (VCA), where virtual atoms are placed on sites of fractional occupancy, with properties intermediate between the real atoms that share the site [53].While the VCA has been used to calculate some compositionally disordered oxides, it is generally recognized as having important limitations [54].The bonding properties of compositionally disordered materials often cannot be successfully modeled by averaged virtual atoms and this can lead to very large divergence between VCA and experiment [55].An alternative to VCA, the coherent-potential approximation (CPA), was introduced in the 1960s, and uses an effective medium to model the average composition of the disordered material-although this approach is computationally expensive, is incompatible with many implementations of DFT, and for some systems cannot give quantitative results [56].In different ways, both VCM and CPA look to take averages to represent disorder.Alternatively, disorder may be represented only using whole-atom occupancies, avoiding fractional occupancies entirely; there are several methods of this type currently in use.The cluster-expansion (CE) model began with work to use the Ising model of magnetism to describe compositional order or disorder in the 1950s [57] and has seen major advances from the 1980s [58] onward.CE considers finite-size clusters and computes the properties of the material as a combination of these.Zunger et al. have introduced the special quasirandom structure (SRS) as a way of approximating random distributions of atoms within a finite supercell [59].Grau-Crespo et al. have used the concept of calculating all possible configurations in a given cell size that represent the total composition of a disordered material [60].This can be effective for low doping levels but computation costs increase as the composition of the disordered site approaches 0.5.Very recently, an approach to model the energetics of compositional disorder using machine learning, instead of DFT, has been published [61].This very brief survey is provided to show that the issues presented by modeling disordered materials are not new.To our knowledge, the Materials Project does not use any of the approaches that we mention above to model disorder.While this may provide significant advantages in economy of computation, it admits the possibility that any predicted order will be artificial and not be seen in experiment.
It might be argued that compositional order does not matter and that a prediction of an ordered compound is really the same as that of a disordered one.We reject such an argument.Ordered and disordered compounds have such important differences in structure and properties that they cannot be considered the same.In oxides, this is especially well studied.The degree of spinel inversion (a type of compositional disorder) dictates magnetic and transport properties [62].In perovskites, disorder influences catalytic, magnetic, transport, and ferroelectric properties [63].It is untenable to conflate disordered with ordered compounds and assume that they are the same but worse to claim a prediction of an ordered compound is new when experiment produces only the disordered analogue, which is already known.
That two thirds of A-lab's "successful" syntheses are likely unrecognized disordered compounds that have been incorrectly modeled as ordered seems to imply that this limitation of the Materials Project calculation methodology has not been fully appreciated.Indeed, in the Google Deepmind GNoME paper, no mention at all is made of compositional disorder.
To conclude, we give some short recommendations that we believe emerge from this episode, for those working in autonomous labs, inorganic materials prediction, and related fields: (1) When claiming the new materials have been made or predicted, one must state in which way they are new.This was clearly done by the A-lab paper (absence from the ICSD) but some may take issue with this definition.
(2) When predicting new nonmolecular inorganic materials computationally, the possibility of compositional disorder should be considered explicitly.(3) When analyzing characterization data, statistical models of goodness of fit cannot be relied upon alone as a measure of success.(4) Compositional measurements of new materials are as important as structural ones.(5) Avoid inverse Occam's razor [64]: positive hypothesis testing should not be used in materials discovery and one must also assess the possibility that, in fact, a known material has been made.Evidence for the novelty of a material must be presented in the context of the answer to point (1); e.g., if the novelty rests in the cation order, evidence must be presented for that.
Finally, it seems clear that robotic labs and AI both will play an important part in the future of materials discovery and solid-state chemistry.At this current time, we see two important bottlenecks hindering high-throughput materials discovery.The first is issues with the prediction of new materials, where tensions between high throughput and high-quality calculations remain unresolved.The other bottleneck is sample analysis.To predict and produce many samples autonomously is impressive but if the rate-determining step is humanoperated analysis, the AI-enabled robotic lab may not move faster than a traditional one.Important steps toward unsupervised analysis have been made but in our view, truly autonomous material analysis remains a target for future work, or a better developed human-machine interface might drastically help to speed up the process.We hope that this Perspective outlines some of the pitfalls and will help to strengthen this aspect in the future.compounds synthesized by A-lab compared against potential matches found in the ICSD.The proposed symmetry is the symmetry of the proposed new phases as reported by A-lab and the indexed symmetry is the symmetry found from FINDSYM [65] of the proposed phase, using the provided structure.Discrepancies between the two symmetries are highlighted in pink.Some phases indexed using the DIFFRACT.EVA software are not tabulated in the ICSD; these phases are denoted by "(EVA)."Note that not all of the mentioned ICSD phases have been refined against the provided PXRD pattern and that there may be impurity phases that have yet to be identified.

4 FIG. 1 .
FIG. 1.(a) The structure of Mg 3 NiO 4 as predicted by the Materials Project (mp-10999253, Pm 3m; left) and the simulated PXRD pattern of the same structure (right).(b) The structure of Mg 2 Ni 2 O 4 as listed in the ICSD (ICSD-290603, Fm 3m; left) and the simulated PXRD pattern of the same structure (right).(c) The structure of Mg 3 NiO 4 as provided by Szymanski et al. [4] (Fm 3m; left) and the measured powder pattern given in the same paper (right).Mg is shown in orange, Ni in gray, and O in red.
FIG. 2. (a) The structure of KMn 3 O 6 as predicted by the Materials Project (mp-1016190, C2/c).(b) The structure of KMn 3 O 6 as provided by Szymanski et al. [4] (C2/m).(c) The structure of K 0.3 MnO 2 as reported in Kim et al. [23] (P63/mmc).K is shown as large purple spheres, Mn is shown in pink octahedra, and O is shown in red.(d) The PXRD pattern as provided by Szymanski et al.(e) The simulated PXRD pattern for K 0.3 MnO 2 .

13 FIG. 3 .
FIG. 3. (a) The structure of Sn 2 Sb 2 Pb 4 O 13 as predicted by the Materials Project (mp-1219056, Imm2; left) and its simulated PXRD pattern (right).(b) The structure of the doped SnSbPb 2 O 6.5 pyrochlore phase as listed in the ICSD (ICSD-62722, Fd 3m; left) and its simulated PXRD pattern (right).(c) The structure of Sn 2 Sb 2 Pb 4 O 13 as provided by Szymanski et al. [4] (Imm2; left) and its reported PXRD pattern (right).Pb is shown in dark gray, Sn is shown in purple, Sb is shown in orange, and O is shown in red.
)], the reported ICSD structure from Gama et al.(ICSD-109468) with its calculated diffraction pattern [see Fig.6(b)], and the structure provided in the Supplemental Information of Ref.[4] with the experimental pattern and A-lab fit [see Fig.6(c)].It is clear that the ordering of the B-site ions has a very large effect on the calculated PXRD patterns.The ordered pattern has a very strong peak at 17.9 • , a feature that is much weaker in the disordered pattern.The experimental pattern clearly matches much better with the disordered version of Zn 3 Ni 4 (SbO 6 ) 2 , which has been reported first byGama et al. in 2003  (ICSD-109468).

FIG. 8 .
FIG. 8. (a) A comparison of the predicted structure of Ba 2 ZrSnO 6 (mp-1228067, Fm 3m; upper left-ICSD-43137, Pm 3m; lower left) with the structure of known BaSn 0.5 Zr 0.5 O 3 (right).The simulated powder patterns of both phases are very similar.The blue arrowsshow the positions of the two most intense peaks that appear in the pattern from the ordered structure but not that of the disordered structure.(b) The refinement of Ba 2 ZrSnO 6 as given in Ref.[4], where several peaks are not accounted for (marked by red arrows).(c) Our refinement using BaSn 0.5 Zr 0.5 O 3 , BaSnO 3 , and two of the starting materials, yielding a much better fit.

3 FIG. 10 .
FIG. 10.(a) The structure of K 2 TiCr(PO 4 ) 3 as predicted by the Materials Project (mp-1224541, P2 1 3; left) and its simulated PXRD pattern (right).(b) The structure of the K 2 Ti 2 (PO 4 ) 3 phase as listed in the ICSD (ICSD-202888, P2 1 3; left) and its simulated PXRD pattern (right).(c) The reported PXRD pattern of K 2 TiCr(PO 4 ) 3 as provided by Szymanski et al. [4], with the unmatched peaks marked by arrows in red.(d) Our refinement of the experimental K 2 TiCr(PO 4 ) 3 pattern against the K 2 Ti 2 (PO 4 ) 3 phase (coll-202888) and Cr 2 O 3 (coll-626479), as reported in the ICSD.The unmatched peaks in the refinement provided in the A-lab paper are matched to the Cr 2 O 3 phase.K is shown in purple, Ti is shown in sky blue, Cr is shown in navy blue, P is shown in light purple, and O is shown in red.

LAPPENDIX 1 .
.M.S. was supported by the National Science Foundation (NSF) through NSF Grant No. OAC-2118310 and the through the Princeton Center for Complex Materials, an NSF Materials Research Science and Engineering Center (MRSEC), Grant No. DMR-2011750 as well as by the Gordon and Betty Moore Foundation's Emergent Phenomena in Quantum Systems (EPIQS) initiative through Grant No. GBMF9064 and the David and Lucille Packard foundation.S.B.L. is supported by the NSF Graduate Research Fellowship Program under Grant No. DGE-2039656.Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NSF.Y.L. was supported by the China Scholarship Council.Indexing of new phases 4 M x Sb 4−x O 13 , where x = 1, 2 and M is a cation.A-lab has reported the successful synthesis and refinement of five Pb 4 M x Sb 4−x O 13 compounds, namely, M = Fe, In, x = 1 (FeSb 3 Pb 4 O 13 and InSb 3 Pb 4 O 13 ), which are predicted to crystallize in space group R3m, as well as M = Hf, Sn, Zr, x = 2 (Hf 2 Sb 2 Pb 4 O 13 , Sn 2 Sb 2 Pb 4 O 13 , and Zr 2 Sb 2 Pb 4 O 13 ), which are predicted to crystallize in space group Imm2.
. Overall, these show close agreement with the known phases.The largest difference is the Sn 2 Sb 2 Pb 4 O 13 compound, which

TABLE II .
A comparison of the pseudo-cubic lattice parameters predicted in the Materials Project with the reported cubic lattice parameters listed in the ICSD for doped pyrochlores.