The Integration of Neural Networks as Tools in Scientific Research

Article

When Data Outpaced Understanding

The history of science is frequently characterized as a succession of significant ideas. Equally important, however, is the history of scientific instruments.

The telescope made Jupiter’s moons observable. The microscope revealed cells and microorganisms. X-ray diffraction enabled the investigation of the internal structure of crystals. For centuries, scientific progress depended on expanding the scales accessible to observation. New instruments revealed phenomena concealed by distance, size, or the limitations of the human senses.

In the twenty-first century, science has encountered a new limitation. Modern instruments and computational models generate extensive data on molecular motion, cellular activity, genome sequences, atmospheric conditions, and material properties. Acquiring information is now often less challenging than discerning which aspects of the data are most significant.

This problem is particularly apparent in the study of complex systems. A molecular simulation may contain the coordinates, velocities, and energies of an enormous number of particles. A climate model describes numerous interconnected processes in the atmosphere, the ocean, and on Earth’s surface. Biological databases integrate the sequences, structures, and functions of millions of molecules.

In each instance, the data are already available. However, scientific explanations do not arise solely from the volume of data collected. Researchers must identify variables that render a system’s behavior comprehensible. Appropriately selected variables can reveal consistent relationships, whereas poorly chosen variables may obscure them, even within precise datasets. Consequently, scientific discovery often begins not with new measurements, but with creative approaches to describing phenomena.

Thermodynamics became possible when researchers learned to relate macroscopic quantities, such as temperature, pressure, and volume, to the state of matter. Molecular biology was transformed when heredity could be described in terms of the structure and sequence of DNA.

Within high-dimensional systems, the relevant variables are frequently unknown a priori. Researchers must identify them among thousands or even millions of potential features. In this context, machine learning assumes a novel and critical role.

For decades, computers primarily accelerated calculations that had already been formulated by humans. Researchers defined the model, the equations, and the quantities to be determined. The machine performed the calculations.

Contemporary algorithms can contribute at earlier stages of research. They can identify structure in data, reduce dimensionality by selecting essential variables, and isolate feature combinations that are challenging to detect through manual analysis.

Understanding how computational systems acquired this role requires consideration of two foundational concepts that predate modern neural networks: predictions can be formulated probabilistically, and rules can be both programmed and learned from experience.

From Rules to Learning

In one experiment, participants were presented with a segment of text and asked to predict the subsequent letter. As the amount of contextual information increased, their predictions became more accurate. The preceding characters constrained the range of likely continuations. Thus, language can be conceptualized as a probabilistic system. Context reduces uncertainty, and knowledge of the prior text enables the estimation of the probabilities of subsequent elements.

Shannon’s research addressed information, redundancy, and uncertainty, and demonstrated that predicting continuations can be formulated mathematically [1]. This principle was incorporated into everyday technology prior to the emergence of artificial intelligence. For example, in the 1990s, Tegic Communications developed T9 (Text on 9 Keys), which expedited text entry on mobile phones. Rather than pressing a key multiple times, users pressed each number once per letter, and the software matched the sequence to an electronic dictionary, ranking words by frequency of use. Although T9 did not interpret messages or construct language representations comparable to modern models, it exemplified the use of context and statistical methods to select probable continuations [2].

T9 did not constitute artificial intelligence in the contemporary sense, nor was it a direct precursor to large language models. However, both systems share a conceptual similarity: they evaluate possible continuations based on the provided context. For T9, the objective was to select a word corresponding to the entered numeric sequence. In contrast, modern language models estimate the probabilities of the next token, which may be a word, a word fragment, punctuation, or another textual element.

The difference in scale is enormous. T9 operated with a compact dictionary and relatively simple statistical rules. A modern AI model is trained on vast amounts of text and uses billions of parameters to account for complex relationships among words, concepts, and contexts.

Yet the underlying question still derives from Shannon’s information theory: What is most likely to come next?

Prediction represented only one aspect of the challenge. T9 also exemplifies the limitations inherent in systems that operate solely according to parameters predefined by developers. Estimating the probability of a word required a human-constructed dictionary and statistical data, with all significant relationships specified in advance. This raises a critical question: how can systems address problems where the underlying rules or features cannot be fully articulated beforehand?

In conventional programming, a developer analyzes a problem, formulates a procedure, and translates it into code. The computer executes the instructions. This approach is effective when the rules are known. Accounting systems, engineering calculations, navigation software, and spacecraft software were all created in this way. But many problems are difficult to describe exhaustively.

For example, a child rapidly learns to distinguish cats from dogs without memorizing formal descriptions of every breed. Through exposure to examples, making mistakes, and adjusting their understanding, children are able to recognize animals they have not previously encountered.

Efforts to replicate this ability in conventional programs encounter major challenges. For instance, while dogs are often assumed to have longer snouts and cats more rounded faces, exceptions such as pugs complicate this distinction. Ear shape, fur length, paw size, tail shape, and body proportions introduce additional variability, leading to numerous exceptions. As additional rules are introduced, the list of exceptions can outpace the rules themselves. The core issue is not simply computational power; much of the knowledge humans apply confidently is difficult to formalize as a finite set of instructions.

As early as 1950, Alan Turing proposed a different approach. Instead of attempting to program the behavior of an adult mind from the outset, he considered the possibility of creating a “child machine” whose abilities would develop through learning [3].

This development marked a significant shift in the formulation of the problem. Rather than focusing on which rules should be explicitly programmed, researchers began to consider how experience could be structured to enable a system to derive rules autonomously.

One of the first practical answers was Arthur Samuel’s checkers program. It used game results and generalization procedures to improve its evaluation of game positions. In his 1959 paper, “Some Studies in Machine Learning Using the Game of Checkers,” Samuel described two machine-learning procedures and demonstrated that a program could improve its play through experience [4].

The role of the developer evolved rather than disappeared. Researchers continue to select the data, model architecture, objective function, and evaluation criteria. However, the specific relationships between inputs and outputs no longer need to be explicitly programmed; instead, they are encoded in the model’s parameters during training.

A neural network begins training with parameters that do not yet reflect the structure of the problem. It receives an example, makes a prediction, compares it with the known result, and calculates the magnitude of the error. The parameters are then adjusted slightly to reduce the likelihood of a similar error in the future. This process is repeated many times, sometimes millions or billions of times. After numerous iterations, the model develops internal representations that are useful for solving the problem.

These internal representations are not equivalent to human concepts. A model does not possess an understanding of a cat, a word, or a protein analogous to human cognition. Instead, it encodes statistical relationships that facilitate distinction, prediction, or data generation.

For a long time, the capabilities of this approach were limited by small datasets, insufficient computing power, and the difficulty of training deep networks. By the beginning of the twenty-first century, these limitations had begun to diminish.

A landmark event was the 2012 ImageNet competition. Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton introduced a deep convolutional neural network that later became known as AlexNet [5]. It was trained on more than 1 million labeled images and tasked with distinguishing among 1,000 object categories.

At the time, many of the best computer vision systems relied on features designed in advance by researchers. Specialists determined which edges, textures, and geometric characteristics should be extracted from an image and then passed them to a classifier.

AlexNet simultaneously learned to identify relevant features and perform the primary classification task, substantially outperforming previous object classification systems. Its performance advantage was significant enough to alter the trajectory of the entire field. Notably, the network’s method of organizing internal image processing was particularly influential.

The first layers responded to relatively simple elements, such as edges, line orientations, and color contrasts [5]. Deeper layers combined them into textures, object parts, and complex configurations. This produced a hierarchy of representations, progressing from local visual features to categories of entire objects [6].

This hierarchical structure was not manually defined by researchers; rather, it emerged during the training process. This outcome demonstrated that a neural network could autonomously discover effective representations of complex data for specific tasks. The success of this approach with images prompted further inquiry into its applicability to other types of information, leading to the expansion of scientific applications for machine learning.

From Prediction to Scientific Representation

In numerous scientific disciplines, researchers address highly complex systems. However, complexity does not equate to randomness. Experimental data and numerical modeling frequently reveal consistent patterns. With sufficient observations, algorithms can learn to recognize intricate relationships between input data and outcomes of interest. Many contemporary scientific machine-learning systems are founded on this principle.

One of the best-known examples is AlphaFold2.

A protein begins as a linear chain of amino acids, but its biological function is largely determined by the three-dimensional structure that the chain assumes after folding. Experimentally determining this structure often requires considerable time and resources. AlphaFold2 used amino acid sequences, evolutionary information about related proteins, and patterns extracted from known protein structures to predict the most probable three-dimensional structure of a protein. In 2021, its developers reported a dramatic improvement in the accuracy of computational protein structure prediction. In many cases, the results approached the level of accuracy required for practical structural analysis [7].

The model did not eliminate the need for experimental validation or resolve all challenges in structural biology. However, it substantially altered the initial conditions for research. In many instances, scientists gained access to computational models sufficiently accurate to support further analysis.

The same general principle also proved effective in meteorology. GraphCast was trained on global reanalysis data, reconstructions of atmospheric conditions that combine real-world observations with the results of physical models. Given two consecutive snapshots of the atmosphere, the model forecasts the atmosphere's subsequent evolution for up to 10 days. In a 2023 study, GraphCast outperformed the operational deterministic system of the European Centre for Medium-Range Weather Forecasts on most of the selected evaluation metrics [8].

This development does not imply that the neural network supplanted physical models of the atmosphere. Its training data were derived from decades of observations and outputs from physical models. GraphCast instead demonstrated that the behavior of complex dynamic systems can be effectively modeled by leveraging patterns present in extensive datasets.

A similar approach was implemented in materials science. GNoME used graph neural networks to evaluate the stability of the proposed crystal structures. As a result, researchers obtained approximately 381,000 new structures computationally predicted to be stable. This does not mean that every one of these materials can be synthesized or that all of them will prove useful in practice. However, the system drastically reduced the number of candidates that must be tested using more expensive methods, significantly accelerating the search for new materials. The model enables ranking candidates and directing resources toward those with a higher probability of producing a significant result [9].

Although AlphaFold2, GraphCast, and GNoME appear to address distinct scientific challenges, they are unified by a common computational principle. In each case, the task is predefined: researchers specify the input data and the desired type of model output.

But there is another class of scientific problems. Sometimes, not only is the answer unknown, but it is also unclear which variables should be used to describe the phenomenon. This type of problem is illustrated by a study of liquid water published in Nature Physics on June 4, 2026 [10].

Water appears to be one of the simplest substances: each molecule consists of two hydrogen atoms and one oxygen atom. Yet the collective behavior of vast numbers of these molecules is exceptionally complex.

Unlike most liquids, water at atmospheric pressure reaches its maximum density at approximately 4 °C and begins to expand as it cools further. Its heat capacity, compressibility, and other properties also exhibit unusual behavior, particularly in the supercooled state.

One hypothesis links this behavior to two types of local molecular organization. In one, denser structure, the molecules are packed more closely together. In the other, they form a more open, nearly tetrahedral arrangement. These are statistically distinguishable local structures. The main difficulty lies in detecting them. A molecular dynamics simulation generates trajectories for vast numbers of molecules. Each configuration is described by coordinates, distances, angles, and hydrogen-bond characteristics. But this dataset does not indicate which combinations of parameters best distinguish the system’s states. Researchers typically select an order parameter in advance. This may be density, the number of neighboring molecules, or a measure of tetrahedrality. Such a choice already incorporates an assumption about the nature of the process.

In a 2026 study published in Nature Physics, researchers applied unsupervised learning methods to trajectories from molecular dynamics simulations of water. Within the resulting representation, they identified two families of local structures and different pathways of transition between them. Far from the transition region between the high- and low-density states, a relatively simple pathway was observed. Near this region, the transition structure became more complex. The authors interpreted this result as evidence supporting a two-state description of deeply supercooled water [10].

This conclusion warrants careful interpretation. The algorithm analyzed data from molecular dynamics simulations based on a specific water model, and the results alone do not establish that ordinary water comprises two coexisting macroscopic liquid phases. Comparison with alternative models and experimental data is necessary. Nonetheless, the methodological significance of this study extends beyond water physics. In this context, the algorithm facilitated the construction of a data representation in which local structures and their transition pathways could be distinguished. Thus, when the appropriate descriptive variables are not known a priori, machine-learning methods can assist in constructing representations that reveal previously hidden patterns.

Whereas early generations of scientific machine learning primarily accelerated the retrieval of known answers, the subsequent stage may prove substantially more consequential. Algorithms are now beginning to assist not only with calculations but also with identifying research questions. As a result, artificial intelligence is gradually becoming an integral component of the scientific method.

It is likely that the current period represents only the initial phase of machine learning’s adoption in scientific research. The first telescopes did not immediately revolutionize astronomy, nor did the earliest microscopes promptly lead to cell theory. Historically, it has taken decades for scientists to differentiate genuine structures from instrument-induced artifacts and to translate new observations into established knowledge.

Machine learning is likely at a comparable developmental stage. Currently, it enables researchers to navigate data spaces that are inaccessible to direct analysis. In the future, it may become as integral to the scientific method as the microscope, spectroscopy, or computer modeling.

If this transition occurs, the principal contribution of machine learning will not be its capacity to answer questions on behalf of humans, but rather its ability to render observable patterns that previously remained concealed within complex datasets.