How not to be deceived by doing physics

Particle physics and astrophysicists use a variety of tools to avoid erroneous results.

In the 1990s, an experiment conducted in Los Alamos, about 55 km northwest of the capital of New Mexico, seemed to reveal something strange.

Scientists have developed a detector of neutrinos with a liquid scintillator [ the Liquid Scintillator Neutrino RTOS Detector in ] at Los Alamos National Laboratory, the US Department of Energy to calculate the neutrinos - elusive particles come in three types, and rarely interact with other matter. LSND looked for evidence of neutrino oscillations — the transfer of neutrinos from one type to another.

In several previous experiments, signs of such oscillations were found, which meant that neutrinos have small masses that are not part of the Standard Model., the main theory of particle physics. Scientists at LSND wanted to re-check those early measurements.

Studying an almost pure source of similar neutrinos — muon neutrinos — at LSND, we found evidence of oscillations into another type of neutrino, electronic. However, much more neutrinos were detected in the detector than was predicted, which gave rise to another riddle.

This excess could be a sign that neutrinos oscillate not between three, but between four different types, which would mean the existence of a new type of neutrino, sterile - theorists suggested to include tiny neutrino masses in the Standard Model.

Or there could be another explanation. The question is what? And how can scientists protect themselves from mistakes in physics?

Something completely new

Many physicists are looking for results that go beyond the Standard Model. They invent experiments to test predictions; if they detect any inconsistencies, it could potentially mean the discovery of something completely new.

“Do we get the result of what the predicted calculations are if we use one Standard Model?” Says Paris Sfikas, a researcher at CERN. “If yes, then we have nothing new. If not, then the following question: “Does the result fall within the error limits of our estimates? Could the result be due to a rating error? "And so on and so forth."

A large list of possible factors may lead scientists to believe that they have made a discovery. An important part of scientific research is their identification and the invention of ways to verify what is actually happening.

“The level for discovery in the community is very high, and rightly so,” says Bonnie Fleming, a physicist at Yale University neutrino. "It takes time to convince ourselves that we really discovered something."

In the case of an anomaly on LSND, the scientist wonders if this was due to unaccounted background events, or if some mechanical problem led to an error in the measurements.

Scientists have developed follow-up experiments to see if they can reproduce the result. In the MiniBooNE experiment conducted in Fermilabrecently reported signs of similar excess. In other experiments, for example, in MINOS , carried out in the same Fermilab, such an excess was not found, which only complicates the search.

"[LSND and MiniBooNE] clearly measure the excess of events versus the expected number," said MINOS spokeswoman Jenny Thomas, a physicist at University College London. “Are these signals important, or is it just the wrong estimated background?” That's what they are working on. ”

Management expectations

Most of the work on understanding the signal occurs before it is received. When developing an experiment, researchers need to understand which physical processes can produce or mimic the desired signal, and these events are often called “background”.

Physicists can predict background using simulations or experiments. Some types of background detectors can be determined through “null tests”, for example, the direction of the telescope on an empty wall. Other types of background can be determined using tests with data, “folding knife tests” when data is divided into subgroups — say, data from Monday and data from Tuesday — which by definition should produce the same results. Any discrepancies will warn scientists about the signal that appears only in one subgroup.

Researchers in the search for a particular signal are trying to better understand what other physical processes may produce the same signal in the detector. For example, the MiniBooNE studies a beam, consisting mainly of muon neutrinos, to measure how often they oscillate into other types. But sometimes he catches random electron neutrinos, and it looks as if muon neutrinos have become them. In addition, other physical processes can mimic the signal from the electron neutrino.

"We know that we will be deceived because of them, so we must do everything possible to understand how many of them there can be," says Fleming. “And the excess we found should be added to these events.”

Human beings are impermanent even stronger than the particle beam. Science tries to objectively measure facts, but this process is performed by a group of people whose actions may suffer from bias, personal problems and emotions. A biased opinion about the result of the experiment may imperceptibly affect the work of the researcher.

“I think there is such a stereotype that scientists are such impassive, cold and calculating observers of reality,” says Brian Keating, an astrophysicist at the University of California at San Diego, author of Losing the Nobel Prize, which describes how the desire to make a discovery leading to reward can distract a scientist from the right behavior. “In fact, we participate in these processes, there are sociological issues that affect people. Scientists, despite the stereotypes, are exactly the same people. "

Being aware of this and using methods that eliminate bias is especially important if a statement reverses long-existing knowledge — for example, our understanding of the neutrino. In such cases, scientists adhere to a well-known aphorism: emergency statements require extraordinary evidence.

“If you go past your house and see a car, you might think,“ This is a car, ”says John Canner, a researcher from Caltech. “But if you see a dragon, you might think,“ Is it really a dragon? Am I sure this is a dragon? You will need another level of evidence. "

Dragon or discovery?

Physicists have suffered from dragons before. For example, in 1969, scientist Joe Weber announced the discovery of gravitational waves: ripples on the fabric of space-time, predicted by Albert Einstein in 1916. Such a discovery, which many considered impossible, would prove the key dogma of the theory of relativity. Weber knew instant glory, but only until other physicists discovered that they could not reproduce his results.

The false discovery shocked the community of researchers of gravity waves, which for the next several decades began to be wary of similar announcements.

Therefore, in 2009, when the laser-interferometric gravitational-wave observatory LIGOearned with the aim of conducting the following experiment, the collaboration of scientists came up with a new way to make sure that its members would be skeptical about their results. They developed a method for adding a false, simulated signal to the detector data stream, without warning most of the 800 researchers. They called it "blind injection". All other members of the community knew that an infusion is possible, but not guaranteed.

“We haven’t detected any signals for 30 years,” said Kanner, a member of the LIGO collaboration. - How clear or obvious should a sign be for everyone to believe in it? It made us lean more heavily on algorithms, statistics and procedures, as well as check sociology and see if we can convince a group of people of this. "

At the end of 2010, the team received a warning, which they were waiting for: computers recognized the signal. Six months, hundreds of scientists were engaged in the analysis, and eventually concluded that the signal was similar to gravitational waves. They wrote a paper with a detailed description of the evidence, and more than 400 people voted for its approval. And then one of the project leaders told them that it was all rigged.

Spending so much time on sampling and studying such an artificial signal may seem empty, but the test worked as it should. This exercise forced scientists to work out all the methods necessary for a close study of the real result even before it appears. This forced the collaboration to develop new tests and approaches to demonstrate the reliability of detecting a possible signal before the onset of a real event.

“In a sense, this system was designed to be fair to us,” says Kanner. - Anyone has some guesses or expectations about the results of this experiment. Part of the idea of ​​blind injection was to touch on this bias so that our opinion about what nature should give out would not play such an important role. ”

And all this hard work paid off: in September 2015, when the real signal reached the LIGO detectors, the scientists knew what to do. In 2016, the collaboration announced the first confirmed direct detection of gravitational waves. A year later, this event won the Nobel Prize.

No easy answers

And while blind injections worked for the gravitational wave community, each field of physics has its own unique challenges.

Physicists studying neutrinos have an extremely small sample of data with which they can work, since their particles interact so rarely. Therefore, the NOvA experiments and the deep underground neutrino experiment use such gigantic detectors.

Astronomers have even fewer samples: they have only one universe to study, and there is no way to conduct control experiments. Therefore, they conduct observations that last for decades, collecting as much data as possible.

Researchers at the Large Hadron Collider have enough interactions to study - about 600 million events occur there every second. But because of the enormous size, cost, and complexity of the technology, scientists have built only one LHC. Therefore, inside the collider there are several different detectors that can check each other's work, measuring the same things in different ways with the help of detectors of different structures.

And although there are many principles for verifying results - to understand the experiment and its context well, run simulations and check that they match the data, check alternative explanations for the result - there is no comprehensive list of checks that each physicist would perform. Different experiments use different strategies, varying from region to region and from time to time.

Scientists are obliged to do everything possible to verify the result, since in the end he will have to be tested by independent reviewers. Colleagues will challenge the new result, subject it to their own analysis, try to provide alternative interpretations, and repeat the measurements in a different way. Especially when it comes to dragons.

More articles on the popular science topic can be found on the Golovanov.net website . See also: why our perception of time is heterogeneous ; when dark matter and dark energy appeared ; how to build a tower up to the height of space ; why Hubble does not see the very first galaxies ; a series of articles on cosmology " Ask Ethan ".

I remind you that the project exists only thanks to the support of readers (bank cards, Yandex.Money, WebMoney, Bitcoins, and at least as). Thanks to everyone who has already provided support!

Now you can support the project through the Patreon automatic subscription service !