In “Deep Learning is Hitting A Wall,” Gary Marcus argues that deep learning is reaching its limits as a paradigm for artificial intelligence. Recent difficulties with self-driving cars and unreliable language model outputs illustrate the limitations of systems that cannot understand the world, or the meaning of the words they are parroting. Instead, Marcus advocates that field of AI turn its attention to symbolic approaches. Such approaches would focus on encoding information about the world and deriving information using simple operations:

What does “manipulating symbols” really mean? Ultimately, it means two things: having sets of symbols (essentially just patterns that stand for things) to represent information, and processing (manipulating) those symbols in a specific way, using something like algebra (or logic, or computer programs) to operate over those symbols.

This article is as interesting for its exposition of symbolic approaches (relatively unknown to me) as for its sociology of science. Embraced by early computer scientists such as von Nuemann, symbolic approaches became dominant in the 1970s after intradisciplinary fighting between the symbolic camp and the neural network camp. Neural nets regained prominance in the 1980s, advanced by researchers who avoided the symbolic approach. The schism remains until today: Marcus’s polemic is itself is a testament to the rift. These are competing paradigms, though there are glimmers of a synthesis on the horizon.