December 2025

The Hidden Forest: The Biography of an Ecosystem, John R. Luoma. 1999/2006

About the Book

I’m reading this with RB. Originally published in 1999, it proved unexpectedly popular and was re-published in 2006 by Oregon State University Press. I had mis-remember this book as being about a periodic study, over the course of a year, of about 20 square meters of forest. Instead, this is an account of several decades of study of the H. J. Andrews experimental forest — a Douglas Fir ecosystem in the Pacific Northwest set aside for scientific investigation. The book appears (I’m writing this after reading four chapters) to combine forest ecology science with an account of how scientific thinking about forests (and their management) have changed over the last 5+ decades. It is very well-written.

The Book

Chapter 1 <chapters have no titles>

• An experiential introduction to the Andrews Experimental Forest.
• In the forests of the Pacific Northwest, every genus of conifer is the largest of its kind on the planet: the Sitka spruce, the western red cedar, the western hemock, the sugar pine, the noble fir, and the Douglas fir. It is believed that this immensity is an adaptation to the climate – 90% of the region’s abundant rain falls in the winter, and the immense boles of the trees enable them to store water in quantity.
• These forests hold more biomass per acre than tropical rain forests: 500 tons/acre
• Most studies of forests are (a) very short term and (b) cover a very small error. The Andrews Experimental Forest is a notable exception, covering 16,000 acres, and permitting studies that will last many decades.

Chapter 2 – ((some history))

• The aim of this experimental forest is to understand how an entire ecosystem works.
• Carolus Linneaus, the inventor of the taxonomic naming system, also wrote a book called The Oeconomy of Nature (1749), which was, essentially, the first book on ecology. He introduced concepts such as the ecological niche, the food chain, and the idea of succession.
• In 1789, Gilbert White, an amateur naturalist, published a book describing a four decade long study of a ecosystem: The Natural History of Selborne. White’s book introduced the idea that waste (e.g., cow droppings) could become food for other organisms (e.g., insects and worms), which in turn became food for yet other organisms (e.g., fish).
• A century later Henry David Thoreau documented the succession of forest trees in New England presented at a lecture in 1860).
• In 1866 Ernest Haeckle coined the word Okologie as the name for a science that would study the complex interconnections of things noted by Darwin in Origin (1858)
• In 1895 Eugenius Warming, a Dutch biologist, began to lay out, in detail, how organisms functioned as parts of biological communities. He introduced the notion of comensalism, in which species are so composed as to minimally compete with one another for resources
• In 1894 Henry Cowles Chandler, on a train trip, happened to view a surprising succession of ecosystems in the dunes of Northern Indiana. Investigating, he realized that what he was seeing was an example of how ecosystems adapted to environmental change.

Cowles concluded that the marram grass colonizing a bare dune| could survive high wind and searing wind, even burial by sand and direct sunlight. But once it was firmly established, the marram not only stabilized the landform-the dune—it provided for other life-forms a bit of shelter and cooling shade, and as it lived, then died, its decay made just a trace of nutrient-rich soil. In time, less hardy plants eventually could prosper. Eventually those plants would crowd out the marram, building more soil, more shade, making way for a new guild of species, and on and on until the ecosystem reached a “chi-max” stage, a mature, dense forest that would operate-or so Cowles supposed—in a steady, stable state. In time, the progressive shade and shelter of plants and the steady decay of accumulating life would change the very character of the soil, and then of the ecosystem itself.

—ibid., 29

• In the 1970’s, Fred Swanson, initially trained as a geologist, became interested in the landforms in the pacific northwest, and morphed into a geomorphologist. He found that the landscape was basically participating in a very slow landslide, and that trees, roots and all, moved millimeters to feet every year.

Chapter 3 – ((the paradigm shift in forestry))

• The idea of forests as managed tree plantations, designed to maximize productivity revenue. Some of the problems resulting from the implementation of this paradigm, and the harmful believes (e.g., old growth forests are decadent; downed trees and logs should be removed).
• Transition to recognizing the value of diversity — of structure, species, niches, etc. — in forests.
• In 1969 Jerry Franklin succeeded in getting an NSF grant to support multidisciplinary study of forests. I believe that this is when the Andrews Experimental Forest was set aside, but I’m not sure.

Chapter 4–((discoveries in the high canopy))

• Description of using a crane to study the canopy of the forest. Claim that the canopy of an old growth forest is its own ecosystem — similar in kind to that of a coral reef, in that its functioning is driven by air and light.
• Lobaria Oregana, a lichen that grows in the canopy, and only in trees over a hundred years old, is responsible for fixing Nitrogen at the rate of 22 pounds/acre, making it one of the dominant N2 fixers of the forest.
• A different researcher discover a fungus (Rhabdocline parkeria) that lives inside Douglas fir needles — it is an endophytic symbiont that synthesizes alkaloids for defense against pests in exchange for sugar and other nutrients. This makes sense because fungi adapt and evolve far more quickly than trees.
• Another researcher discovered that large trees can develop above ground roots to take advantage of the detritis and soil that accumulates in the upper canopy over the centuries.

Chapter 5—((decay of litter, branches, et al))

The forest clonks and bangs and sings with the hissing and the booming and the knocking and the thwacking of bits of litter, from pieces of limb or lichen, moss or needle, of seed cone or sloughing bark. For hour by hour, day by day, the standing forest ecosystem fairly rains down bits of its own life, and death, to the forest floor.

—ibid., p 68

• Over a year, five tons of litter will fall onto a single acre of the floor of an old growth forest. In addition, rain, snow and fog-drip dissolve nutrients that live as ‘lawns’ on Lobaria lichen in the canopy, and carry nutrients down to the forest floor. This links the ecosystem of the canopy to that of the forest floor.
• Although rich in carbon, a log has only the fraction of the nutrients required by a tree: those are found in leaves, buds, and the thin layer of inner bark containing the phloem and the xylem.
• An early survey of the Andrews forest found 219 tons/acre of downed wood, and another 49 tons in the form of standing snags.
• Description of the structure of leaves, and the process of photosynthesis. Photosynthesis is described at the atomic level, with electrons being knocked out of orbits in a cascade, culminating in the electrolysis of water into oxygen and hydrogen, the latter of which is enzymatically (in a process called “the dark phase”) joined with carbon dioxide to make sugars. Over the course of a season, a tree will produce about 2 tons of sugar.
• A tree trunk consists of: bark, inner bark (the tissue-thin cambium that produces wood), the sapwood (that transports water and nutrients [N, P, K, Ca, S, Mg, & Fe (and trace amounts of I, Mn, Cl, & Co)), and the heart wood (that provides structural support). The water is transported upward because it is in a continuous column, and due to hydrogen bonds has a significant tensile strength approach that of metal wires).
• A tree that falls in a forest will open a large ‘gash’ — in part because it will pull down other trees as well – within which saplings, forbes and other organisms may grow. Likewise, snags offer habitat for a large variety of organisms.
• It may take a large log 2 centuries to completely decay. During the process, the log will come to contain more biomass than the living tree. 20% of its weight may be biomass, in contrast to 5% of the weight of a living tree.
• Log decay:
  • Decomposers start fermentation
  • Ambrosia beetles, attracted by the scent of alcohol, create tunnels under the bark
  • As they burrow, the Ambrosia beetles transport fungi on their backs
  • The fungi grow in the gallaries of tunnels, providing food for the beetles and their larva
  • At the same time female bark beetles will burrow into the bark
  • After the inner bark is consumed, new organisms appear. The Ponderous borer will bore into the heart wood, laying its own eggs. Carpenter ants will do the same.
  • Predators and parasites will follow the borers — for example, a small wasp that feeds on borer larva
  • After about ten years the Pacific dampwood termite will invade the log, and begin digesting the the cellulose; the microbes in their gut also fix nitrogen…

I asked AI to clean up and expand the above list:

Early Decomposers (Year 0-1)

Bacteria and yeasts - Begin fermentation of simple sugars and starches in the sapwood, producing ethanol and carbon dioxide as byproducts; lower pH through production of organic acids, creating anoxic conditions that inhibit some competing microorganisms

Early-Middle Stage (Years 1-3)
2. Ambrosia beetles - Attracted by ethanol and volatile organic compounds from fermentation; bore through bark into sapwood creating extensive gallery systems; physically fragment wood and increase surface area for decomposition

Ambrosia beetle-associated fungi (Raffaelea, yeasts) - Transported in beetle mycangia; colonize gallery walls, breaking down cellulose and hemicellulose; provide nutrition for beetle larvae; further acidify wood environment
Bark beetles (females) - Burrow into inner bark (phloem/cambium); vector ophiostomatoid fungi into sapwood; create entry points for other organisms; deplete inner bark nutrients

Middle Stage (Years 3-10)
5. Wood-boring beetles (Ponderous borers, cerambycid beetles) - Attack after bark nutrients depleted; bore into heartwood creating larger tunnels; fragment wood structure; larvae develop over 2-3 years

Carpenter ants - Excavate galleries in softened wood (don't consume wood but remove it); prefer wood with 20-40% moisture content; create extensive cavity systems that increase wood exposure to air and moisture

Predatory and parasitoid wasps - Follow wood borers; parasitize beetle and borer larvae; their activity creates additional openings in wood structure

White-rot fungi (basidiomycetes) - Begin colonizing wood through beetle galleries and cracks; produce lignin-degrading enzymes; break down complex lignin compounds into simpler molecules; increase nitrogen availability. [[TE: For lignin breakdown to occur the basidiomycetes require molecular O2]]

Late-Middle Stage (Years 10-20)
9. Pacific dampwood termites (Zootermopsis) - Colonize wood with elevated moisture content (>20-30%); digest cellulose with help of gut protozoa and bacteria; their gut microbiome includes nitrogen-fixing bacteria that convert atmospheric nitrogen into ammonia; create fine galleries throughout remaining wood

Nitrogen-fixing bacteria (Rhizobiales, others) - Populations increase as wood C:N ratio decreases; fix atmospheric nitrogen, enriching wood and surrounding soil with bioavailable nitrogen; support fungal decomposition

Late Stage (Years 20-50+) - fungi & bacteria => soil invertbrates
11. Soft-rot fungi (ascomycetes) - Dominate in wet, well-decayed wood; attack cellulose in cell walls; work under high moisture, low oxygen conditions where white-rot fungi less active

Brown-rot fungi (basidiomycetes) - Break down cellulose and hemicellulose while leaving lignin relatively intact; create simpler organic compounds; and create cubical cracking pattern in wood; make remaining lignin accessible to other decomposers

Diverse bacterial communities (Acidobacteria, Burkholderia, Actinobacteria) - Populations increase dramatically as fungi create simpler organic compounds; metabolize fungal exudates, wood sugars, organic acids; continue nitrogen transformations; prepare substrate for soil incorporation

Soil invertebrates (mites, springtails, millipedes, salamanders) - Colonize highly decayed, moist wood; physically fragment remaining wood; mix decomposed material with mineral soil; transport fungi and bacteria; accelerate final stages of incorporation into forest floor

• Benefits of logs:
  • Decomposing logs act as sponges, retaining enormous amounts of water, even after fires.
  • Nurse logs (especially western red cedars and Sitka spruce) enable saplings to grow on them (which would otherwise have difficulty taking root on the mossed-over forest floor).
  • Downed logs slow erosion, and the soil built up on their uphill side provides habitat for insects and small mammals
  • Rotting logs provide habitat for salamanders, voles, chipmunks
  • Logs in rivers slow streams, reduce erosion, aid sedimentation, and provide fish and wildlife habitat
• Functions of snags:

Chapter 6—((bugs and living soil))

• In terms of number of species, arthropods [== “jointed feet”] vastly outnumber non-microbial species: 41,000 vertebrates; 500,000 plants; 30 million insects. The combined weight of insects on earth may exceed that of all humans by a factor of 12.
• Diversity and evolution: The diversity of arthropods is in large part due to the combination of their short lifespans with high mobility due to flight.
• The diversity of arthropods in the forest floor of Andrews appears to rival the diversity found in tropical forests. They are referred to as a “precision barometer,” a rather odd metaphor, but I take the meaning.
• The community of arthropods in the forest floor is a very accurate mirror of the ecosystem above.
• Soil, rather than being primarily an assemblage of inorganic material, is mainly made of biogenic substances… countless microbes and the bodies and feces of invertebrates.

On one such slide, Moldenke showed me the image of what was clearly a needle from a Douglas fir—or so it seemed. … But a closer look showed that it was not intact at all. It was a collection of unconnected fragments, thousands of infinitesimal bits arranged in almost precisely the pattern of the needle. Without moving the needle at all, countless tiny arthropods had swallowed parts of it. In fact, every bit of the needle had been chewed up and swallowed. Bacteria living in the insects’ digestive systems had worked furiously for a few hours on the outside of the bit of food, extracting nutrients both for themselves and for the arthropod. And then the remaining cell tissue-constituting most of what had been chewed off in the first place-was repackaged into a tiny pellet of ground-up plant matter, and then excreted almost precisely in place. The needle had been, in short, thoroughly reprocessed through the first stages of decay. And it still looked much like a needle, or at least a needle rendered by a pointalist.

—ibid., p 100

• There is a lot of detail on how arthropods (and their microbes) rapidly transform detritus into soil. (Note that many arthropods – mites, springtails, microspiders, et al. — are extremely small sub-millimeter scale).
• Most of the nutrients used by the forest are very near the surface in the biogenic portion of the soil. Roots go deep primarily for water and to provide support.

Moldenke: “If you went to Andrews Forest and brought me back a handful of dirt, I could tell you what time of year you dug the sample up—all critters have life cycles-the altitude it was taken, the slope face-whether north or south. I could tell you the vegetative cover—whether it was Oregon grape or wood sorrel or something like that. I could tell you the successional stage of the forest—whether it is early, middle, or late— whether it was old growth or not. In some areas, I could tell you what kind of tree was nearby and how far away,” he says. … “And it’s easy! Anybody could do it with a little bit of training.”

—ibid., p 106

Chapter 7—((roots and fungi))

• A. B. Hatch discovered the relationship between fungi and tree roots and coined the term micorrhizae. He discovered that saplings associated with fungus grow faster than those with no such associations.
• Fungal hyphae interpenetrate tree roots, growing between individual cells and sometimes even growing into cells. The hyphae get sugar and some vitamins from the tree roots and in response grow into mats that cover hundreds of square feet that bring water to the tree; fungal hyphae are also capable of taking in certain nutrients (like phosphorous, which is not water soluble). Dendrologists now believe that hyphae connect trees to a thousand times more soil area than the roots themselves. These fungal mats also seem to act as reservoirs for nutrients that would otherwise get washed away.
• Some organisms, such as voles, shrews, and pika, feed on fungal fruiting bodies (e.g. digging for truffles), and spread the spores in their feces.
• Tree roots grow — at a micro-scale — with extreme rapidity. They extend root hairs which, in the absence of water melt away (well, melt means eaten by microbes and small arthropods).
• A single rye plant has approximately 600 thousand miles of root hairs, and has an estimated 14 billion individual hairs.
• In some cases there appears to be a one to one relationship between a root and a leaf — that former provides water to the leaf, and the later sugar and nutrients to the roo.
• Scientists now believe that about 40% of the photosynthate made by the leaves of a tree go to support its micorrhizal fungi and the ecosystem that exists just beyond the roots.
• Soil Structure. The structure of soil — particular the diversity of pores or spaces in it — seems to be correlated with its health/fertility. This structure is a consequence of fungal hyphae exuding polysaccharides that ‘glue’ grains of soils into clumps.
• Some trees that grow in intense shade — maples, hemlocks — appear to be able to obtain nutrients from the very overstory trees that are making the shade; they do this thru hyphal bridges.

… discussion break—next two chapters for March 9th …

Chapter 8—((disturbances))

• The original conception of ecosystems did not get everything right.
  (1) One mistaken idea was that ecosystems evolve until they reach a permanent, self-sustaining stable climax community
  (2) Ecologists also did not pay attention to the role of disturbances in ecosystems — to the fact that they may rely on disturbance. “…the weaving and wedging of evolution…” Without fire some forests cannot reproduce; without drought, some wetlands can not thrive or even survive.
• Jack Pine forests. Jack pines thrive on poor soil and harsh climates; in the absence of these they will be out-competed by other trees. Jack pines build up thick mats of highly flammable needles, and, in the area of Michigan being discussed, burn every few decades. The burns are needed for their cones to open, so that they can reproduce; the burns are also essential for maintaining the landscape needed by the Kirkland Warblers.
• Jack Pine forests provide habitat for the Kirtlands Warbler. This bird nests on the ground under the low hanging branches of Jack Pine (and only Jack Pine). It also relies on insects that associate with Jack Pine for nutrition, and also requires forests with open areas where blueberries and other easily picked edibles are available.
• Foresters are exploring ways of creating forest-landscapes that will support the Kirlands Warbler. So far these are not as effective as the regular cycle of burns, but they succeed to a degree.
• The exemplifies a more general approach sometimes referred to as “new forestry,” that tries to preserve properties of the natural ecosystem (e.g., snags, downed logs etc).
• Douglas Firs are not the climax ecosystem — without disturbances like wind, they would eventually be replaced by Western Hemlock.
• Om 1977 the Franklin teams writes a paper that will eventually cause major changs in how people think about forests: The Ecological Characteristics of Old Growth Douglas Fir Forests.
• In 1988 Chris Maser and Jim Trappe published a small booklet: The Seen and Unseen World of the Fallen Tree.
• Mt St Helens story. After the eruption the Franklin team started studying recovery of the ecosystem. To their surprise, only a few weeks post-eruption, there was life amid the ash: gophers, deer mice, fungi, plants (like blackberry) re-sprouting from rhizomes; there were also amphibians and aquatic invertebrates that had survived in the mud beneath streams and lakes. And of course many kinds of seeds which, as the ash eroded, were able to sprout. The Franklin team referred to this as the biological legacy, and went on to propose that preserving this legacy was one of the keys to the new forestry.

Chapter 9—((The New Forestry))

• 1967: E. O. Wilson and Robert Macarthur: The Theory of Island Biogeography. This is a classic book that was the beginning of Conservation Biology, and argued that the issues involving species extinction were more complex than simply habitat loss.
• Issues that are related to species extinction include inbreeding; population biology (with a small overall population, natural population fluctuations may hit zero, in which case the cycle stops); edge vulnerability.
• Endangered Species act was passed in 1973, but it took time for species to be studied to determine whether they could be listed as threatened or endangered.
• The Spotted Owl story.
• Job loss in Pacific Northwest lumber industry was due primarily to automation, not the Endangered Species Act. From 1979 to 1987 about 15% of the industries jobs were lost to automation. And even with job loss related to that — e.g., due to the preservation of old growth forest for the spotted owl — was only moved ahead in time: there was not much old growth left, and the mills and personnel that serviced that aspect of the industry were doomed.
• New Forestry: Recovery biological legacies. Emphasis on preserving larger areas, and creating corridors between preserved areas for species migration and re-colonization.

… reading break—likely will discuss 3/16…

C10—((Flows in forests ))

• The 1996 flood in the PNW. More sediment entered streams during the flood than in the preceding three decades.
• A river flows downstream not in two dimensions but in three — over time it changes the land around it. Floods alter the contours of fluvial landscapes and also reshape the ecological communities that inhabit them.
• There is a remarkably consistent correlation between the width of a river channel and the wavelength of its meanders: 1::10-14.
• There are dramatic differences between recently logged forests and unlogged ones: Within 5 years flows of a recently logged area increase 50%; and they remain 20-40% higher for as much as 25 years. Even logging as little as 5% of a forest can increase peak flow rates by 10% to 55%.
• Fungal mats occupy large areas of the forest floor (up to a quarter of the area) and constitute lots of biomass (as much as 50%) One previously unsuspected function is that fungi can extract nutrients directly from humic molecules left over after bacteria break down sugar, starches and lignin.
• Some mycorrhizal species can break down rocks and extract their minerals — what are they, and what kind of rocks do they break down?
• Various pests and diseases (spruce budworm and other defoliators may not be so much a problem as a solution: they take out the weakest branches and trees, leaving the healthy.
• The idea that old growth forests consume less CO2 fails when one includes the the co2 released by wood debris, etc., in managed forests.

C11–((Into the Future))

• This chapter was of less interest to me. It advocated for more long term research, and for the value of simply describing what happens in ecosystems. And it gives examples of the value that such long term ecosystem scale research can provide.
• “Destruction is even more likely to occur at a ponderous pace in the secrecy of the invisible presence.” —John Magnuson

Views: 22

Fall 2025

I Contain Multitudes: The Microbes within Us and a Grander View of Life, Ed Yong, 2016.

I am late to this book. I’ve had it for years, and, lately, have kept moving it deeper into the to-be-read stack in the belief that something newer — in what is obviously a rapidly evolving field — will appear. But nothing had as yet, my curiosity is unabated, and my friend Rachel wanted to read it as well, so here we go.

In Summary

I very much enjoyed this book. Fascinating material, and although it is about 10 years old now (a long time in the world of science), I felt like I was getting a very up-to-date picture. Certainly, I’m not aware of any more recent book for the general science-loving reader on this topic.

I will note that I liked it significantly more the Immense World, though I enjoyed and recommend that as well. I think the difficulty with writing this kind of book is how to organize it coherently. Sometimes one can take a temporal path through an area, if there is a strong theme or through-line (as in The Tangled Tree, Quamann or The Master Builder, Arias), but I don’t see that as possible here, where the topic is incredibly broad: microbes, individually and in community, and their interaction with other organisms. I thought Yong did as well as possible (and it is where I feel Immense World fell a little short).

Detailed chapter-by-chapter notes follow, but at a high level this is the arc of the book.  
The first chapter establishes core concepts: microbiomes as ecosystems, and their variation across body sites and individuals. The second chapter provides a capsule microbiology. 
       Next the book turns to ways in which microbes shape the development of their hosts, and the ways hosts, in turn manage their microbiome. In typical Yongian fashion, this is done through a series of example organisms that range from the Hawaiian Bobtail squid (with its light organs manned by microbes that it encases in its body), to the important role of mucus in more complex organisms) in keeping microbes in their proper places. 
       The book then pivots to a discussion of dysbiosis, a situation in which the entire microbial ecology changes in a way that is problematic for its hosts – the leading example here are the microbiomes of coral reefs. It’s a fascinating example. This segues into a discussion of the function of microbiomes (rather than individual microbes) and their diversity (or poverty) over time, and their distribution through populations. 
      After this, I feel like I lose the thread of the book. The subsequent chapters take up interesting topics, but is seems to me more like, having laid the foundations and established the basics of the phenomenon, it turns to special topics. This is still very much worth reading, but except at ending up at possible real-world applications, I feel like that narrative arc falters. 

The Book

Prologue: A Trip to the Zoo

A very ‘soft’ beginning, describing a researcher sampling a pangolin for its skin microbiome. We’ve got a cute animal, comments from a scientist, and an introduction to the basic idea that all living things host ecosystems of microbes that play a variety of surprising roles.

C1: Living Islands

Chapter 1 begins laying the groundwork, sketching out the evolutionary history of life — emphasizing the microbes have been around far longer than any life form — and laying out the basic forms of life: archae, prokaryotes, and eukaryotes.

• Human vs. Microbiome Cell/Genes. There are roughly as many microbial cells in the human body as human cells, although this is still somewhat speculative. The human genome consists of 20 – 25 thousand genes; the human microbiome has 500x as many.
• There’s a recap of Alfred Russel Wallace’s voyage, and his claim that: “Every species has come into existence coincident in space and time with a pre-existing closely allied species.“
• No core human microbiome. Scientists initially hoped to identify a core microbiome that was the same from human to human, but that has not held up. At most, there may be said to be a core of functionality that the human microbiome consists of.
• Variation in the Human Microbiome. The human microbiome varies more between body parts than between humans. The human microbiome also varies in time, from birth to death. The books suggests it follows stages of succession, but all the text says here is that it takes a baby’s microbiome about three years to become an adult one.
• Given that microbiomes provide essential functionality to animals, etc., what does it mean to be an individual.

Looking ahead to some themes that will be pursued in later chapters:

• Many conditions from disease (diabetes, colon cancer) to other maladies (autism, obesity) appear to be correlated with the makeup of the microbiome, though of course causality is not clear.
• Organisms that exhibit convergent evolution in their behavior (e.g., ant-eating animals) also exhibit converence in their microbiomes.
• Perhaps health problems may be re-envisioned as ecological problems at the microbial level.
• In some cases microbial genes can permanently inflitrate the genome of their host organism.

…reading break…

C2: The People Who Thought to Look

A brief history of microbiology. Microscopy, microbes, et al.

C3: Body Builders
[microbial modulation of hosts’ development]

• Hawaiian Bobtail squid have two chambers on their undersides that produce luminescence that protects them by eliminating their silhouette at night when seen from below. The luminescence is produced by bacteria — V. Fischeri — that colonize the chambers shortly after birth.
• Development of the squid’s luminescence organ is induced by bacteria. The Bobtail squid chambers are covered with mucus and cilia. When a V. Fischeri first makes contact, nothing happens, but when five or more make contact that triggers the expression of genes that produce a cocktail of anti-microbial substances that kill of everything but V. Fischeri . Other enzymes break down the mucus and produced substances that attract even more V. Fischeri . Eventually the V. Fischeri migrate down pores to spaces lined with pillar like cells that envelope the V. Fischeri , and the luminescent ‘organ’ reaches its mature form. What is interesting here is that the development of the squid occurs in a dialog of genetic expression with V. Fischeri
• MAMPs – Microbial-associated Molecular Patterns. Not sure why the term “patterns” is used. But in general it applies to substances released by microbes that impact, for good or for ill, a host organism. It is now clear that many organisms develop under the influence of microbial partners, often using the same molecules that the squid’s V. Fischeri produces.
• Germ-free organisms. Organisms that are isolated and raised in a completely sterile environment are often only marginally viable and require artificial substitutes for what microbes would produce.
• Microbial triggered gene expression. We can see that microbes often trigger gene expression (e.g., in the gut) that leads to the creation of blood vessels, and structures that aid the intake of nutrition and maturation of cells.
• Choanoflagellates (choans) — S. rosetta. These are water-dwelling eukaryotes that prey on bacteria; Choans are the closest living relatives of all animals. Under certain conditions Choans can aggregate into colonies of about 20 organisms, growing a connecting sheath the binds the separate organisms into a sphere; it turns out that the colonial form is more effective at catching food. The formation of colonies turns out to be triggered by a bacterium, which causes the choan to release a molecule that triggers the formation of the colony.
• Squggly worms — H. elegans and P-luteo. H. elegans begin as larva; they only attach to a surface and mature when they encounter a biofilm, and this in turn is induced by a particular bacterium referred to as P-luteo. The ocean is swarming with larval animals that only mature when they encounter bacteria, often P-luteo.
• The ubiquity of bacteria. A repeating theme here is that it’s not surprising that more complex organisms rely on bacteria — bacteria were ubiquitous when the complex organisms evolved, and it makes as much sense to make use of them as any other feature of the environment.
• Bacteria and the immune system. Bacteria play a crucial role in tuning the immune system. Microbes both influence the production of inflammation producing cells as well as anti-inflammatory cells.
• Spotted hyenas and bacteria-mediated scents.
• Microbes and behavior. Changing a mouse’s microbiome can change its behavior. It can make them more or less anxious, and more or less depressed. There is speculation that this may be true for humans as well, and interest in developing “psychobiotics.”

…reading break…

C4: Terms and Conditions Apply
[Mucus, Mile, and the Immune System: Managing the microbiome]

• Wolbachia reproduces by inserting itself in its host’s female eggs. Over time it has developed many methods of increasing the female/male ratio. This is probably the most successful bacterium outside of the ocean.
• Prochlorococcus — so numerous that 1 ml of seawater contains 10⁵ bacteria. Produces about 20% of O₂
• “Certain bacteria can even turn their owners into magnets for malarial mosquitoes, whilst others put off the little bloodsuckers. Ever wonder why two people can walk through a midge-filled forest and one emerge with dozens of welts while the other just has a smile? Your microbes are part of the answer.”
• Symbiosis doesn’t mean “mutually beneficial.” Symbiosis means organisms that live together; but this doesn’t mean that they are necessarily mutually beneficial. It can be largely good, largely bad, a tradeoff, and, most importantly, the cost/benefit ratio and degree of symmetry can very over time.
  • Interesting example: Acacia trees prevent their ants from using other types of sugar. 
• SIRS — Systemic Inflammatory Immune Rsponse. Sepsis occurs when our ordinarily beneficial bacteria get into the wrong places. 
• Bacteriocytes Insects have special ‘containers’ for housing and controlling bacterial symbiotes.
• Mucus. In vertebrates most bacteria are kept out of cells (e.g. within the gut)) and mucus (made of giant entangled mucin molecules) is used as a protective barrier.
  • Mucus provides an environment for Bacteriophages — viruses that infect bacteria — love mucus. It’s hypothesized that animals can alter the composition of their mucus to recruit particular phages. 
  • AMPs. The inner layer of mucus also contains AMPs(antimicrobial peptides) which kill bacteria. Particular AMPs are released in response to the presence of bacteria. 
  • Immune cells. Finally, on the other side of the mucus barrier there are lots of immune cells which ‘reach through’ the mucus barrier to sample the bacteria on the outside. 
• Immune system as management rather than protection.
  Claim: Our immune system has evolved to manage our microbial community — warding off disease is just a useful side effect. 
• Establishment of microbiome. Babies are vulnerable to infection for their first six months not because their immune systems are immature but because they’ve been suppressed to allow establishment of the microbiome. 
  • Mammalian milk is an important way of controlling the microbiome— human milk contains over 200 HMOs (Human Milk Oligiosaccarides). But humans can’t digest the HMOs—rather they are food for a particular gut bacterium: b infants. B infant is in turn produces short chain fatty acids that nourish infant gut cells and stimulate them to produce adhesive proteins and anti inflammatory molecules. 

…reading break…

C5: In Sickness and in Health
[Dysbiosis–Diseases as ecosystem turnover]

Reefs

• Reef microbiomes. Reefs are covered with microbes — 10x more than an equivalent area of human skin: 100 million/sq cm.
• Colonization resistance: Most microbes occupy space, so if a reef has been colonized by ‘good’ microbes, there is little room for the ‘bad’ microbes to move in. If you disrupt the microbiome, bad microbes can move in.
• Fleshy algae vs. coral. Reef microbiology has to do with a balance between coral organisms and fleshy algae. Fleshy algae are kept in check by ‘grazers’ like parrot fish and surgeonfish. If humans eliminate sharks, it causes a population explosion in mid-sized fish, who then decimate the grazers. Similarly, humans can kill the grazers directly by hunting/fishing them. Either way, that removes limits on fleshy algae, which proceed to take over the reef by consuming all the oxygen and smothering the coral organisms.
• Sharks as energy stores. A single shark contains the stored energy equivalent to that in several tons of algae. If sharks are eliminated, that energy — in the form of DOCs (dissolved organic carbon — carbohydrates and sugars) – is available to the microbes, which bloom and extract all the oxygen in the water.
• Coral death. Corals are rarely killed by exotic organisms, but rather by parts of their own microbiome which have experienced explosive growth due to DOCs. As coral organisms die it creates more space for algae and other micro-organisms, leading to a positive feedback loop that kills the reefs.
• Reef death. A coral reef can die incredibly quickly, within a year.
• Black reefs. A wrecked boat containing iron can stimulate the growth of fleshy algae (for whom iron is a limited resource) to the extent that even grazers can not keep it under control. Even a single iron bolt can form a miniature black reef around it.
  

Dysbiosis

• Dysbiosis. In cases like these, the cause of a reef’s demise is not a single organism, but rather a turnover of the ecosystem where it shifts into a pathogenic state. This is a different paradigm for disease that contrasts with the invasion of a single foreign pathogen: it is disease as an ecological problem.
• Germ-free Mice. Germ free mice can eat as much as they like and not gain weight. If they are given a microbiome, they eat no more but become better at extracting energy and put on weight.
• Lean vs. Fat. The microbiomes of fat organisms differ from those of non-fat organisms. Transferring a ‘fat’ microbiome to a germ-free mouse will make it fat; transferring a ‘lean’ microbiome to a germ free mouse (or fat mouse) will make it lean.
• Gastric bypass surgery reconfigures the microbiome.
• Microbiomes + Nutrients. It is not just the microbiomes, but the nutrients that an organism has access to. Particular types of nutrients will favor particular types of microbiomes.
• Dysbiosis ➔ Inflammatory diseases. It appears that a lot of diseases which are associated with inflammation — IBD (inflammatory bowel disease); Type I diabetes; Multiple Sclerosis; allergies; asthma; rhumetoid arthritis – may be due to dysbiosis.
• Causes of dysbiosis. One hypothesis is that overly hygienic environments produce organisms with immune systems that are too ‘jumpy.’
• Countering dysbiosis. (1) Dogs and to a lesser degree cats, introduce a wider variety of microbes into modern homes, which may strengthen the immune system. (2) Likewise, mother’s milk (as seen in the last chapter) introduces a varied microbiome. (3) Fiber in the diet is broken down into SCFAs (short chain fatty acids) and triggers the production of anti-inflammatory cells.
  

Microbiome Diversity

• Generational microbiomatic poverty. An impaired microbiome can be passed along to the next generation. Continued encounters with diversity-decreasing effects (e.g. high fat diet’; antibiotics; etc.) can continue such trends.
• Antibiotics. Antibiotics have long lasting effects on the microbiome.
• Microbiome diversity. Inhabitants of third-world countries, and members of hunter-gatherer tribes, have far more diverse microbiomes (which could also be due to other factors like fiber, low-fat diets, breast feeding, hygiene). Other primates such as chimpanzees, bonobos and gorillas have more diversity than any human.
• Microbiome dynamics. The microbiome varies over time, both longer term — e.g. during pregnancy — and shorter term (e.g., the diurnal cycle)

C6: The Long Waltz
[The evolution of symbionts]

• Sodalis. Sodalis is a bacterium that is a symbiont — it is only found growing in the blood of a tsetse fly.
• Human Sodalis. HS is similar to Sodalis, but resembles what Sodalis might have looked like before it became a symbiont. The researcher suspects that Sodalis started out as a bacterium that infected trees, and that used insects to move from tree to tree to reproduce. But, over time, it figured out how to reproduced and just move from one insect to another.
• Symbiots. Many microorganisms adapt so that they can reproduce by entering an egg cell and be passed from one organism to another. The mitochondrion is likely an ancient example of this. Some argue that ‘social’ organisms may exist because it is easy for them to share symbionts.
• Holobionts and holobiomes. Controversial.

…reading break…

C7: Mutually Assured Success
[Win-win: Microbes/microbiome and nutrition]

• Microbial assists to nutrition. Hemipterans (Leafhoppers and other sap-sucking insects) use microbes to produce nutrients they need. What do microbes get? Perhaps protection and transportation to the right niches?] About 10-20% of insects rely on microbes…
• Microbes as the sole source of nutrition [chemosynthesis]. Riftia (Tubeworms) do not take in any nutrition themselves: they have no mouth, gut or anus. Instead about half of their body is devoted to a trophosome containing bacteria that convert sulphides to energy, producing pure sulpher as a byproduct. It turns out chemosynthesis (based on sulphides or methane) is a very common strategy for organisms that live in the deep ocean.
• Chemosynthesis is also found in surface organisms. Olavious, a worm found near the island of Elba, uses five symbionts to produce energy from sulphates and sulphides.
• Microbiome diversity. As far as using microbes to assist in the capture and creation of nutrients, plant eating organisms have the most diverse microbiomes, then omniovores, and then carnivores. This seems mainly related to the variety and complexity of substances consumed.
• Rift || Gut, and adaptive radiation. Yong suggests that there is a sort of parallel between microbiomes found in the deep sea and in the dark acidic anoxic environment of the gut — not in microbes per se, but in that it looks as though those microbiomes adaptively radiated from a few species of microbe.
• Microbiome adaptation is very rapid. The human (and other) gut microbiomes can adapt to accomodate dietary changes in a few days.
• Microbes can confer immunity to toxins.

C8: Allegro in E Major
[Horizontal Gene Transfer and adaptive speed]

• Horizontal gene transfer (HGT), which is commonplace among bacteria, enables very rapid adaptation to changing conditions (e.g., antibiotic resistance).
• Rapid adaptation. HGT can support very rapid adaptation by complex organisms by altering the abilities of microbiome microbes.
• Integrated bacterial genes. Various agricultural ‘pests’ such as root knot nematodes and coffee bean borers, as well as beneficial organisms such as brachnid wasps, owe their specialized abilities to genes that originated in bacteria. These genes have become integrated into their hosts DNA. Genes that lend themselves to this kind of uptake must be highly useful and must be self-sufficient (i.e. don’t require a lot of other genes to support their functionality).

The citrus mealybug is a mash-up of at least six different species, five of which are bacterial and three of which aren’t even there. It uses genes borrowed from former symbionts to control, cement, and complement the relationship between its two current ones, one of which lives inside the other.

—ibid., 203

…reading break…

C9: Microbes à la Carte

• Filariasis (Elephantiasis, River Blindness). Why so severe? It is bodies immune response to both the nematodes and their bacterial symbionts, and the fact that when you kill the nematodes they release all the wolbachia that is the problem. A good treatment is to kill just one of them, and then let the other die more slowly due to absence of their symbiont. 
• Frogs and Bd. The Bd fungus is spreading rapidly and driving many species of frog into extinction. But some frog species are immune — it turns out to be because they are covered with a microbiome that kills the fungus. This can be transferred to some (but not all) species of frog.

…discussion break…

• Probiotics. Probiotics seem of limited value: First, the amount that one can consume — perhaps a 100 billion organisms in a very concentrated probiotic, is at most 1% of the # of organisms present at the very most. Second, the bacteria found in probiotics are unlikely to survive in the gut — the ecosystem they are coming from is very different from the one they’re going to. That said, there are a couple of things that probiotics can do: shorten infectious diarrhea and that caused by antibiotics, and save the lives of those who have necrotizing enterocolitis. But that is the complete list.
• Goats. Probiotics have been successful in transferring immunity to a plant with a toxic substance — limousine – between goats; it is applied to their coats as a ‘drench.’
• Prebiotics. Prebiotics are substances that nourish gut bacteria — the HMOs in human breast milk are one example.
• Networks of bacteria / FMT. No bacterium exists in a vacuum — one may need a supporting cast of others to thrive. The most practical way to achieve this is a faecal microbiota transplant. This works astonishingly well for C-diff infections, but C-diff may be a special case because people get it after taking antibiotics which has pretty much cleared out their normal microbiota ecology.
• Synthetic bacteria. Synthetic biologists are working on engineering bacteria that can detect a substance, and in response switch on genes to produce enzymes that attack the organisms causing the problems. Others are working on kill-switches and other ways to stop engineered bacteria from exchanging genes with wild bacteria.

C10: Tomorrow the World

• Each person aereosols about 37 million bacteria/hour. We walk around with microbiome halos.
• Likewise every home has a distinctive microbiome — and that is created incredibly rapidly — within about 24 hours. 
• Dolphin research. The water chemistry and health of the dolphins are better if the water is filtered less frequently. This raises questions about what a healthy level of hygiene is.
• 5 – 10% of hospitalized people develop infections. In the US that means 1.7 million infections and 90,000 deaths per year. Sampling the air shows that inside air is far less healthy than that outdoors — many pathogens that are rare or absent. Best thing to do is open windows!
• Can we seed buildings with beneficial microbiomes via miniature plastic spheres that provide microbe-friendly niches?
• Earth Microbiome Project. Predict/characterize the sort of microbiome that can be found in different sorts of ecosystems.

# # #

Views: 85

Co-Intelligence: Living and Working with AI, by Ethan Mollick. 2024

About the Book

I read this for my book club. It’s a well-written book, relatively hype-free, and very much worth reading for those who don’t know much about AI. I felt it was a bit verbose, but that might be appropriate for the more general audience he is trying to reach. I did not learn a lot from the book, but then I’ve been following the discourse on LLMs for a long time, and have also been getting Mollick’s newsletter in my in-box, so this is neither surprising nor a mark against the book.

But now I’ve gone through my various marginalia and underlining, and feel that I got more out of the book than I had realized when I wrote the previous paragraph. A few of the high-water marks for me:

• The fourth ‘rule’ – assume this is the worst AI you will ever use – is a good reminder. We are used to software improving at a glacial pace, but that may not be true of LLMs. [C3]
• I appreciated the confirmation of my belief that an AI cannot track the reasons for its responses, and that any explanation of a response is a hallucination. [C5]
• It made me think more (in [C9]) about the consequences of the erasure of digital groundtruth by generative AI, and how that will undermine public confidence in ‘facts’ as presented online. Perhaps everyone will retreat into their own filter bubbles; or perhaps there will be a turn towards traditional curated media (though the phenomenon of Fox news makes this seem unlikely).
• An interesting argument is that the speed of innovation has been dropping 50% every 13 years), presumably because one must know more and more to make progress. Perhaps AI can provide a remedy here.

That said, I didn’t think Mollick did a great job of delving into the potential of AIs to enable people to educate themselves. There is much to be said about the pros and cons of using AI in this way (e.g., AIs creating study guides, problem sets, etc.) Little of this is discussed.

Chapter by Chapter Notes

C!: Creating Alien Minds

• A brief history of AI, particularly the way it is used in business; a slightly more in-depth history of the rise of LLM’s and generative AI.
• An introduction to the Transformer architectures with its ‘attention mechanism’ and the resulting LLM’s
• An interesting note on whether LLM’s violate copyright or not, since the LLM does not contain text, just weight-vectors.
• A mention of RLHF — Reinforcement Learning from Human Feedback. This is how LLM’s are ‘taught’ to avoid certain topics.
• Some good examples of how slightly changing prompts can significantly change the response of the LLM.

C2: Aligning the Alien

• Discussion of various ways in which AI can have detrimental impacts
• More discussion of RLHF and guardrails, and how guardrails have been circumvented.

C3: Four Rules for Co-Intelligence

Didn’t learn anything here, but mainly because I’ve been experimenting myself and following Mollick’s newsletter. He also had a not unreasonable piece of advice: “Become the world expert in how to use AI to do a task you know well.“

For the record, the four rules are:

• Always invite AI to the table
• Be the human in the loop
• Treat AI Like a Person (but tell it what kind of person it is)
• Assume this is the worst AI you will ever use

This is not bad advice. I particularly like the last rule.

C4: AI as a Person

A discussion of how AIs can appear to be sentient. Possibly a useful for those who have never interacted with an LLM, but otherwise I don’t think the chapter did much in the way of making useful points. Not sure that there are useful points to make here.

C5: AI as a Creative

• Returns to the point that LLMs don’t store text, they only store weights. So in a sense they don’t know anything.
• Nor can they actually give a real account of why they gave a particular answer, though of course they can generate a plausible explanation. This seems like an important thing for people to understand.
• LLMs are trained on text. The training does not take quality into account; it does not even distinguish between fiction or non-fiction. All it is doing is learning weights.
• We are back on the topic of hallucination, a term I very much dislike. But it does make a good point, which hadn’t sunk in for me: “Anything that requires exact recall is likely to result in a hallucination.”
• And this is a nice quote:

It [an AI] is not conscious of its own processes. So if you ask it to explain itself, the Al will appear to give you the right answer, but it will have nothing to do with the process that generated the original result. The system has no way of explaining its decisions, or even knowing what those decisions were. Instead, it is (you guessed it) merely generating text that it thinks will make you happy in response to your query. LLMs are not generally optimized to say “I don’t know” when they don’t have enough information. Instead, they will give you ananswer, expressing confidence.

—Ethan Mollick, Co-Intelligence, p 96

• A lot of talk about creativity, which did not interest me.
• Cites studies (his own? and colleagues?) that show that using AI greatly decreases the time to perform creative tasks. …I agree this is likely.
• Makes the interesting point that ‘ceremonial tasks,’ such as writing recommendation letters, are likely to be rendered meaningless, or at least greatly reduced in value, as the letter no longer necessarily represents a significant time investment.

C6: AI as a Coworker

• Distinguished between jobs and tasks — AI may radically change the way job-related tasks are carry out, but may not necessarily eliminate the job.
• Argues that the systems within which a job takes place play a crucial role in shaping the job — but I don’t think the argument is taken very far re implications….
• Discusses different types of tasks: delegated tasks, and automated tasks. And different types of workers: Centaurs (with a clear strategic separation of tasks) and cyborgs (a more blended approach).

C7: AI as a Tutor

• Talks about using AI in education. Uses example of introduction of calculators in classrooms to think about this. I suspect some readers will find this very useful.
• Talks about flipped lectures and how AI might be used in the classroom.
• A rather disappointing chapter: I think there is huge potential in individuals using AI to educate themselves, and a lot to be said about the pros and cons of using AI in this way. There is also the prospect of AIs creating study guides, problem sets, etc. None of this is discussed.

C8: AI as a Coach

• Begins with a favorite point of mine about the danger of AI eliminating the on-ramps to expertise — it can do the tasks that were formerly assigned to interns, and eliminate the possibility of apprenticeship.
• Offers the prospect of AI’s as coaches that will help novices and journeymen do the difficult reflective practice that builds expertise. Interesting, but he is just making all this up, as far as I can tell.
• Cites a study of his own that claims that the quality of the middle manager explains 20% of the revenue that a video game eventually produces. This would be a very difficult study to operationalize, and I’m a bit skeptical. But hard to say since I don’t understand how video game companies work, or the role middle managers play in them.

C9: AI as our Future

This chapter offers four scenarios on the future of AI (==LLMs)

• 1. As Good as it Gets.
  AI will not improve significantly from here on out, either because of technical limitations (running out of text to train on), or because of regulatory intervention. He argues that this isn’t a very likely future, but it is what most people and organizations are planning for. I agree that regulation is unlikely; less sure about the certainty of significant improvement.
  • Makes the point (not sure why it is relevant to this scenario) that the erasure of digital groundtruth will undermine public confidence in ‘facts’ as presented online. Perhaps everyone will retreat into their own filter bubbles; or perhaps there will be a turn towards traditional curated media (though Fox news makes this seem unlikely).
• 2. Slow Growth.
  The exponential growth in AI capability will slow to 10%-20% a year. He cites various reasons for this, from cost of training, to technical limits for large LLMs (apparently LeCun, chief AI person at Meta has argued this), and my own favorite, not-enough-high-quality text. In this scenario, tasks will change more than jobs, and more jobs will be created than destroyed.
  • In my view it may be that LLM’s have ‘used up’ the supply of high quality information, and trying to train LLMs on broader swaths of material will introduce ‘semantic pollution.’
  • Mollick also talks about the decline in the ‘speed of innovation’ (dropping by 50% every 13 years), and the fact that most major scientific contributions are made by scientists over 40 (whereas the opposite used to be the case). You must know more to make progress, and that slows progress. He suggest that perhaps AI can help here.
• 3. Exponential Growth. In this scenario the speed of AI growth continues. AI-assisted hacking, targeted marketing, AI-assisted law enforcement (and military, which, interestingly, he does not address) proliferate, and government policies/regulations can not keep up. But maybe AI and robotics eliminate the need for a lot of human work, and things like basic income, shortened workweek, and so forth usher in a post-scarcity economy.
  • He also comments on AIs becoming better and more interesting companions than other humans, and the possibilities of a decrease in loneliness, but a rise in new forms of social isolation.
• 4. The Machine God. AI becomes sentient. There is not a lot to say here. Could be horrible, could be wonderful.

Afterword: AI as Us

Brief essay on how AI has grown out of our knowledge, includes our own biases, etc, etc. Not very interesting.

# # #

Views: 108

*The Catalyst: RNA and the Quest to Unlock Life’s Deepest Secrets, Thomas R. Cech, 2024. Cech won the Nobel Prize in Chemistry in 1989 (with Sidney Altman) for the discovery of the catalytic properties of RNA, which means that in addition to carrying genetic information, RNA could also act on other molecules.

Reading this with the inaccurately-named “26-minute book club” in the Spring of 2025.

Continue reading The Catalyst: RNA…*, Thomas Cech

Views: 38

The Disordered Mind: What Unusual Brains Tell us about Ourselves, Eric R. Kandel, 2018.

Kandel is an eminent neuroscientist, known for his work on the low-level mechanisms of learning and memory as demonstrated in Aplysia. He’s won a host of prizes, including the Nobel for this work. Interestingly, as an undergraduate he majored in humanities, and afterwards became a psychiatrist, before migrating into neuroscience. Now in his 90’s, he is writing about larger themes, and addressing himself to more general audiences.

Writing after having read four chapters: The book is interesting, but I wish it went into more detail. Though it is also possible that the detail is not available — i.e. we still not shockingly little about mental disorders.

Continue reading The Disordered Mind…, Eric R. Kandel

Views: 24

November 2024 – April 2025

* Why Machines Learn: The Elegant Math Behind Modern AI, Anil Ananthaswamy, 2024. (Ananthaswamy is a science journalist, not an AI person, as I initially assumed. That said, he’s quite good.)

My Take on the Book

As discussed below, this is not the sort of book I’d usually read — my interest stops at the level or algorithms, and understanding the underlying math just doesn’t grab me. This really is a book for people interested in the math. But I learned some interesting, mostly-meta things from it.

• Early learning systems. Early research focused on using simple networks (e.g. Perceptrons) to recognize fuzzy inputs like hand-written letters. This involved presenting a system with an input, having it try to map that input to a category, and then providing feedback to the system based on the mismatch between the input and the right (label for the input) result. Much of the math to do with measuring and tracking the degree of error.
  Even though they sound different things, this is also how adaptive filters: they try to learn the characteristics of noise so as to minimize it.
• Matrices model networks! More generally, matrices and linear algebra are exceptionally useful for modeling networks.
• A Lot of AI learning is just gradient descent. A lot of the math involves figuring out multi-dimensional spaces that characterize a domain, and using some sort of algorithm, usually involving gradient descent, to find the minima. The minima, of course, being where error is minimized, meaning that the system is performing as well as possible at its learning task.
• People made jaw-droppingly crude simplifications, and yet the math still worked! Something I found very interesting was that, once you have an algorithm that appears to do what you want, you can make shockingly crude simplifications to it that (1) make it possible to run it in very high dimensional spaces, and (2) it will still work well.
  For example, if you are doing gradient descent in a multi-dimensional space, you can get away with finding the gradient for a randomly chosen single dimension (stochastic gradient descent), rather than in all dimensions.
  I still don’t get why that works, but it does. Not only does god play dice with the universe, but he’s OK with kludges tool.
• Adding ‘noise’ can improve learning. Adding noise to data sets on which machines are trained can make the learning more robust. To me this seems sensible, in that, especially if you add different amounts of noise to the same features, you can multiply the training set and counter the tendency to overfit.
  Of course the noise ought to be ‘natural,’ which is to say that it ought to be native to the distribution from which you are sampling. I presume it is possible to figure that out for particular domains, but don’t actually know.
• Monty Hall as Enabling Inside Trading. Hurrah. I finally understand the Monty Hall problem. In my defense, I will note that the ‘story problems’ I had in my primary education worked against my ability to solve it. In story problems, the stories didn’t actually matter, they were just frameworks for presenting a math problem. But in the Monty Hall problem, the key bit is Monty — he has inside information and will not choose to spoil the game by revealing where the prize is. Thus his action is not random, and provides additional information. If the Monty Hall problem were instead re-presented as an earthquake or windstorm accidentally revealing a goat behind one of the doors, the answer would be different.
• WTF? — Why does deep learning work so well? We don’t, currently, really understand why deep learning works as well as it does. Spooky.,
• Why do deep neural nets keep learning new things after they’ve overfit their training data? In particular, there is the paradox of benign overfitting / harmless interpolation: a deep neural network has so many parameters that it should perfectly overfit the training data, but — after enough training — it is able to generalize correctly to new data, something that it would not do if it had overfit. So what is going on?
  You’d think that once it was able to model the training data perfectly, it would stop learning. But perhaps the continued ‘training’ is introducing some kind of ‘noise’ or fluctuation which keeps perturbing the neural net?
• Is something happening in the hidden layers??? I feel like there must be interesting stuff going on in the hidden layers of deep neural networks as their training continues beyond the point of benign overfitting/harmless interpolation. I don’t really understand what continued training of a deep neural net does — either directly or via back propagation. I have a vague idea idea that structure of the hidden layers undergoes some kind of phase change, in an analog to fractional crystallization. (In a magma chamber, where when the magma cools to a certain point a particular mineral crystalizes and precipitates, changing the nature of the magma and enabling further crystallization (and/or re-solution of prior crystalites) as the mix changes.)
• In the epilogue the author raises the questions of whether LLMs have really learned, or are just doing statistics. I’m in the “stochastic parrot” camp.

Continue reading Why Machines Learn*

Views: 16

Four Billion Years and Counting: Canada’s Geological Heritage. Produced by the Canadian Federation of Earth Sciences, by seven editors and dozens of authors. 2014.

November-December, 2024.

I am reading this with CJS. It is a nice overview of regional geology, and it is nice that all the examples come from Canada, and at least some of the discussion is relevant to Minnesota Geology. The book is notable for its beautifully done pictures and diagrams.

Continue reading Four Billion Years and Counting…

Views: 22