the jsomers.net blog.

About two years ago I was introduced to j-archive.com by a couple of friends who were “playing” one of the thousands of Jeopardy! games hosted on the site. What they were really doing, though, was rolling a mouse over each of the clues—laboriously typed up and submitted by nerdy volunteers, who have so far archived more than 3,300 actual episodes—and sort of quizzing themselves. There was no frantic buzzing in, no score-keeping, no wagering. It was a fraction of the fun that it could be.

One of these friends of mine, thinking that I was a more competent programmer than I actually was, suggested that I build a playable version of this wonderful archive. I considered it for a moment, convinced myself that it would be too complicated and difficult, and declined.

Fast forward six months, and, with the same friends, at the same house, under essentially the same circumstances, we had the exact same conversation. But by now I had one or two modest Rails projects under by belt, and the idea seemed quite plausible. Simple, even.

I made no promises, but that night I spent the next two or three hours before bed thinking about my approach, and the next day I set to work in a frenzy fueled mostly by the prospect of surprising my buddies with a minimal version. Six or seven hours later I had one, and though it was only stocked with about a hundred episodes, had no final jeopardy, didn’t include wagering or daily doubles, and was littered with all sorts of little bugs, it worked, and my friends and I got a huge kick out of huddling around the laptop and competing.

It’s often said that software is best developed incrementally, with lots of feedback from users. That way you can slowly ratchet your way up to a product that people actually want, rather than trying to blindly design a perfect system ab initio.

This is exactly what happened with what I ended up calling “Jimbo Jeopardy!” We’d be a few games in and someone would suggest that I add indicators for whose turn it was, or they’d remark that it’d be neat if I added the “think” music to final jeopardy. So I’d step aside and hack away for half an hour or so, add the feature or fix the bug, and we’d get back to playing. If the request called for something more involved—like when I was asked to add “correctors” that would (a) display the outcome of each question and (b) allow players to tweak scores (if someone mistyped an answer, for instance), or to change the algorithm that decides when to allow players to buzz in—I would take a late night or two before coming back with a new-and-improved version of the game. So I was continuously debuting new ideas and encouraging different people to try it out.

I knew I was onto something when people would call me asking if I could bring the laptop by to play. The game was fun. More fun, I think, than the official versions of the game that Sony puts out. Why?

1. For whatever reason, they write new clues for the computer games, and because it takes a long time to write new clues, there aren’t that many—they boast about having “over 3,000.” Jimbo Jeopardy! has 187,217.
2. As a corollary to the first point, the clues in these official games aren’t culled from actual episodes, so they don’t necessarily cohere in the right way. What people want is to play games that are just like the shows.
3. Finally, all of the officially licensed versions of the game use a multiple choice guessing system. Needless to say this was a terrible decision, since it makes every question about an order of magnitude easier than it ought to be.

Even though I had built the game in Rails, and even though my friends would ask me from time to time for versions they could play on their own computers, I hesitated for a while to put the project online. Instead, I actually installed my full development stack on several friends’ computers, Mac and Windows, so that they could play without me; I created clickable icons for the commands that started the server, and even tied the project folder to a Dropbox so that I could synch code across systems. This, in retrospect, was a laughably ridiculous hack.

So when I got some spare time after graduation, I finally pushed the code to jimbojeopardy.com.

About a month or so after it “launched,” a friend of mine who was slated to appear on the real live version of the show asked if I had something he could use to study. I thought the version I had up at that point was adequate, but that it would be nice if there were something geared a little more toward deliberate practice.

So once again, I spent the next night trying to surprise a friend with something cool—what ended up being the “Blast!” version of the game. There you can carve out a list of questions from the database based on whatever dollar amounts, seasons, categories, or search terms you’re most interested in. The system then creates a rapid-fire “flash card”-style single-player timed game, at the end of which you’re quizzed on the questions you skipped or answered incorrectly.

I’m not sure how much he used it, or whether it even helped at all, but my friend did go on to win $24,600. You can play his game here.

Anyway, I’m constantly tweaking the site, trying to fix bugs, keep load times down, etc. But there is still lots of work to do. In particular, I’m planning to build tools for aggregating analytics about which questions and categories are hardest, what people guess for each clue, how much they decide to wager in various situations, etc.—with versions that let users track their own play, too—and if I get the time, I’d like to make a version that will let people play over the web, rather than having to all sit down at the same computer (even though it’s arguably more fun with friends in the same room).

I encourage you to try it, and, of course, to let me know what you think.

In college I was a relatively infrequent contributor to class discussions, even though I almost always had more fun when I did talk. I tried to explain why I held back in a half-finished essay, some years back, about a more general kind of “mental block”:

Last semester I had a class where I didn’t speak but should have—not only because the grade was 20% “participation,” but because I felt like I could add to what were mostly enjoyable conversations. Yet I never did.

The problem was that I didn’t say anything in the first few days, before other students had time to establish themselves. Because once the “talkers” were well known, the usual pressure to find something clever to say was made worse by concern that my debut would be inadequate. I had by that time become more of a silent face than an active participant, and I thought I would appear especially foolish if, when I finally did speak up, it was to say something trite or inconsequential.

This is transparently dumb reasoning, but I was taken by it. What’s more, I’m sure that at least a few of my classmates were too—this kind of “mental block”, or convenient self-deception, is as common as it is powerful.

Rather than dwell on the psychology of the “silent faces”—which, if interesting, is a bit well-worn—I thought I’d ask a different question: as a teacher, how do you get kids to participate? How do you foster a rich vibrant discussion, especially among students who are used to being quiet?

I think the trick is to cold call—to require that everyone speak, not just the volunteers.

As long as you are also highly encouraging, even for shallow or poorly articulated responses, the cold-calling process (a) directly forces some insular kids out of their shells, (b) indirectly coaxes out others who think they can do better, and (c) proofs participation as a socially acceptable activity.

Cold calling can come in many different flavors. The most salient might be the kind you’d find in law school classrooms that use the Socratic method, as in this classic opening scene from the 1973 film, The Paper Chase:

Of course Professor Kingsfield here isn’t very encouraging, which is perhaps why the experience is so hard on our protagonist, the young James T. Hart, who actually ends up vomiting after class. But it is no doubt possible to use the Socratic method without being a jerk about it.

As an example, I once attended a talk on corporate responsibility at the University of Michigan’s Ross School of Business. The professor (whose name I unfortunately now forget) spoke energetically about a handful of big-time corporate disasters—like the Chicago Tylenol murders, which forced Johnson & Johnson to recall 31 million bottles of Tylenol and which led to the development of the now-ubiquitous tamper-proof containers—and the different ways that executives handled the fiascos. But before revealing what did happen, the professor took advantage of the fact that most of the crowd was hearing these stories for the first time, and asked us what should happen. In effect he forced us to relive all those tough decisions, a process which ended up being far more engaging than a straight lecture. When we found out what the various CEOs actually did, we could compare it to our own responses; we paid more attention because we had a stake in the outcome. The hour and a half went by too fast.

But it wouldn’t have worked without the following trick: anytime the professor would ask the audience a question or solicit an opinion, a few hands would shoot up, and instead of picking on any of these folks, he would point to a random person seated elsewhere and say, “There, you, I see a hand—I’m not sure if it’s up, but I see a hand”—and ask them to speak. He did this enough that everyone in the smallish audience had a reasonable expectation of being called on—which meant that we did think about every question he asked, if for no other reason than the fear of being publicly unprepared.

Cold-calling doesn’t have to be unexpected to be effective. Take creative writing classes—these are like any seminar or “discussion section,” with the caveat that, for whatever reason, the people who run them insist that everybody speak. Whereas it’s quite normal for a handful of students to stay quiet in other seminars, I’ve never been to nor heard of a creative writing class where someone could opt to be idle. The result being that creative writing classes turn out to be a lot tighter—not everyone becomes best friends, obviously, but (initially) forced participation engenders a kind of social closeness which, in turn, leads to freer and franker discussion, and on and on in a kind of virtuous circle.

It could be that fiction writers are self-selected to be outgoing, or that the process of workshopping poems and stories itself demands complete participation (maybe because it would be unfair to have your stuff critiqued if you weren’t also willing to critique other people’s stuff). Which would suggest that not all classes are created equally, and that something like, say, a math seminar can work just fine—optimally, even—without lots of participation.

But that doesn’t sound right, nor, again, does it jive with my experience. One of the best math classes I took was a small upper-level seminar on logic (we used Enderton). In every section, professor Mummert would ask one of the nine or ten of us to present a proof to the rest; we were “respectfully grilled” and were encouraged to do the same when someone else was up. Naturally the process forced us to have a solid grip on the material, and helped us learn how to present mathematical arguments—which, for most of us, didn’t come naturally.

The professor also—and this is rare for math classes—made a big deal out of having us interrupt his lectures. So if there was a point you were unclear on, even if it was something rather silly, you would stop him and ask—rather than feverishly trying to clear up your confusion while simultaneously trying to listen to everything that followed.

I later learned, while reading Paul Halmos’s I Want to be a Mathematician, that Mummert was using a kind of modified “Moore method”:

[Moore] turned out a record-breaking number of Ph.D.’s in mathematics; they loved him and imitated him as far as they could. He did it by what has come to be called the Moore method. It is also called the Texas or Socratic or discovery or do-it-yourself method.

At the first meeting of the class Moore would define the basic terms and either challenge the class to discover the relations among them, or, depending on the subject, the level, and the students, explicitly state a theorem, or two, or three. Class dismissed. Next meeting: “Mr Smith, please prove Theorem 1. Oh, you can’t? Very well, Mr Jones, you? No? Mr Robinson? No? Well, let’s skip Theorem 1 and come back to it later. How about Theorem 2, Mr Smith?” Someone almost always could do something. If not, class dismissed. It didn’t take the class long to discover that Moore really meant it, and presently the students would be proving theorems and watching the proofs of others with the eyes of eagles. One of the rules was that you mustn’t let anything wrong get past you – If the one who is presenting a proof makes a mistake, it’s your duty (and pleasant privilege?) to call attention to it, to supply a correction if you can, or, at the very least, to demand one.

The procedure quickly led to an ordering of the students by quality. Once that was established, Moore would call on the weakest student first. That had two effects: it stopped the course from turning into an uninterrupted series of lectures by the best student, and it made for a fierce competitive attitude in the class – nobody wanted to stay at the bottom. Moore encouraged competition. Do not read, do not collaborate – think, work by yourself, beat the other guy. Often a student who hadn’t yet found the proof of Theorem 11 would leave the room while someone else was presenting the proof of it – each student wanted to be able to give Moore his private solution, found without any help. Once, the story goes, a student was passing an empty classroom, and, through the open door, happened to catch sight of a figure drawn on a blackboard. The figure gave him the idea for a proof that had eluded him till then. Instead of being happy, the student became upset and angry, and disqualified himself from presenting the proof. That would have been cheating – he had outside help!

Moore famously said “That student is taught best who is told the least.” And the way to do that—whether you use some version of his eponymous method, or the “invisible hand” style cold-call, or nothing more than a round table and high expectations—is to demand that students speak, that they trade their jeans of silence for partici-pants.

Since April 2008, I have slowly been building a large repository of notes. It’s a combination of (a) original writing:

• It is words that jostle in our brains when we read or ask questions. To know them fluently lets one “think without even thinking about it.” You become a better translator: between languages, for sure, but also between their thoughts and yours, that structure and this one. –Fri, 21 Aug 2009 13:27:00

• An essay can only be slowed—not stopped—by ignorance. Compare mathematics. –Wed, 19 Aug 2009 19:30:00

(b) excerpts from stuff I’ve read:

• William James once argued that every philosophic system sets out to conceal, first of all, the philosopher’s own temperament: that pre-rational bundle of preferences that urges him to hop on whatever logic-train seems to be already heading in his general direction. This creates, as James put it, “a certain insincerity in our philosophic discussions: the potentest of all our premises is never mentioned … What the system pretends to be is a picture of the great universe of God. What it is—and oh so flagrantly!—is the revelation of how intensely odd the personal flavor of some fellow creature is.” (The One Argument Ayn Rand Couldn’t Win — New York Magazine) –Fri, 30 Oct 2009 18:18:49

• “Like every other creature on the face of the earth, Godfrey was, by birthright, a stupendous badass, albeit in the somewhat narrow technical sense that he could trace his ancestry back up a long line of slightly less highly evolved stupendous badasses to that first self-replicating gizmo—which, given the number and variety of its descendants, might justifiably be described as the most stupendous badass of all time. Everyone and everything that wasn’t a stupendous badass was dead.” — Neal Stephenson “Cryptonomicon” page 1. –Tue, 20 Oct 2009 00:51:07

(c) jokes (some mine, some not):

• asking for the cute librarian’s call number –Tue, 24 Jun 2008 19:13:07

• Waiter to a table of Jewish Women: Is anything all right? (3quarksdaily) –Fri, 11 Dec 2009 10:17:26

and (d) handy words or phrases:

• “licenses” as verb –Tue, 11 Nov 2008 18:39:14

• “square these facts” is a good phrase –Tue, 22 Dec 2009 00:01:52

It’s an incredibly useful resource for me. Most of the blog posts I’ve written, for instance, began as small wisps on my notes page. And I’ve gotten used to having my favorite quotes, jokes, and ideas instantly searchable from anywhere in the world.

It’s also quite simple—it’s not like I have to go through some big process each time I’d like to jot something down or capture what I’m currently reading. In fact, using the Tumblr API, a few PHP and Ruby scripts, a cron task, and a retrofitted Firefox extension, I have stitched together a system that lets me take notes in any of the following ways:

• Highlighting some text in Firefox and pressing ⌥⌘N. This also grabs the title and URL of the current page.

• Typing “n ” into my Firefox address bar or at any Terminal prompt; I can even format the text using Markdown.

• Going to a public (but hopefully secret) URL, either for when I’m away from my home computer or when I need to compose longer notes.

• Opening up the Tumblr iPhone app.

I realized after many failures with more ambitious systems that if there is any friction at all in the process, I won’t stick with it. So the above was designed to be as quick as possible, which means that I considered even seemingly trivial actions like opening a new browser tab too onerous, and I avoided any kind of organization or categorization.

I am, in any case, happy with the result. I now have 2,686 notes approaching 200,000 words, amounting to several hundred pages of stuff-I’d-like-to-remember. More importantly, I have a relatively concrete sense of what I’ve been reading, doing, or thinking about. I can quickly see that from October to December 2009, for instance, I went on a “hard problem of consciousness” kick, or that I read Raise High the Roofbeams, Carpenter and Seymour: An Introduction—

“If there is an amateur reader still left in the world—or anybody who just reads and runs—I ask him or her, with untellable affection and gratitude, to split the dedication of this book four ways with my wife and children” –Sat, 26 Jul 2008 21:19:28

—right after I put down a collection of Ezra Pound’s letters:

Love to you and mother. Salutations to the entourage. Cheer up, ye ain’t dead yet. And as Tourgeneff says, most everything else is curable. –Mon, 21 Jul 2008 23:29:32

And as a strange sort of bonus, by aggregating the metadata attached to each note, I have been able to compile a histogram of submission times. Which means I can now say with confidence that I am least likely to be awake or intellectually active from 7-10am, and that my single most prolific hour is midnight:

A commenter on Why The Law wondered whether we can “have a society whose output has equivalent K-complexity as ours, but is generated from simpler rules”—in effect seeking the simplest legal system that can sustain something like the U.S. economy.

The idea reminded me of two papers—”Economic Production as Chemistry,” by Padgett, Lee, and Collier (2003), and “Evolution of Cooperative Problem-Solving in an Artificial Economy,” by Baum and Durdanovic (2000)—because in each, the authors are trying to develop a minimal model of fruitful markets: what are the necessary conditions for productive collaboration? Which treatments of property and compensation generate stability, and which degrade? Must agents be intelligent? Altruistic?

Hypercycles

Padgett et al. imagine a grid-world with three basic components: firms, one in each cell, each of which has a random initial endowment of rules, which are simple procedures for transforming products. So for example the cell at (2, 3) may have a rule C -> F that takes the product C as input from one neighboring firm and spits out an F to another. It looks something like this:

A natural question to ask of such a world is whether it can sustain chain reactions, or coincidences where firm after firm spits out the very product that the next guy needs as input, as when a firm with rule A -> B happens to send off its B to a firm with rule B -> C.

In fact the model is deliberately set up with these chains in mind. Here are the rules of the game:

1. Initially, technologies are distributed randomly among the firms, which are arrayed on a large grid with wraparound—so that every firm, even if it’s on an “edge,” has eight potential trading partners.
2. In each “round” of the model (which proceeds in asynchronous discrete steps):
   • A random rule is chosen or “activated.”
   • The firm that owns this rule grabs a product from its “input environment,” modeled as an urn containing a bunch of products.
   • If the product it chooses is an input for the activated rule (e.g., if the product is a C and the rule is C -> Z), the rule does its thing and the output of its transformation is passed randomly onto one of the firm’s neighboring trading partners.
   • Otherwise, the input product is sent to an output environment (also modeled as an urn).
3. The round’s action continues as firms receive their neighbors’ output. This step is important: if you’ve just successfully transformed A into Z, and pass the Z onto me, and I have a Z -> V rule, then we’ve just engaged in a “successful transaction.”
4. Every time there is a successful transaction, one of the rules involved (in our example above, that would be one of A -> Z or Z -> V) is reproduced. It turns out that the behavior of the overall model depends a great deal on which one of the source or destination rules is copied. (See below.)
5. To encourage competition, every time one rule is reproduced, another one, chosen randomly, is killed off. So the total number of rules in play is held constant. This selection pressure, combined with the ability for rules to reproduce, is what keeps the world from degrading into an unproductive mess. (Again, more on this below.)
6. Firms that run out of rules die or “go bankrupt.” Products sent their way are immediately directed to the output environment.
7. Firms continue to pass products around until one of these lands on a firm that doesn’t have any compatible rules (i.e., if a D shows up at a firm that only has Z -> F, E -> F, and C -> D). At that point, the product is ejected into the output urn and a new rule is randomly chosen, as in step 1. (Meanwhile, and here’s the “asynchronous” part, other firms are passing around other products. So the whole model doesn’t reset when a single chain comes to an end.)

So how does anything interesting happen in this world? How do firms even survive? The short answer is “loops”:

In this model, the minimal requirement for long-term survival, both of firms and of clusters, is to participate in at least one spatially distributed production chain that closes in on itself, to form a loop. Not all production chains within a trading cluster need be closed into loops. And more than one loop within a cluster is possible, in which case we may have a dense hypercyclic network of spatially distributed production rules. But loops within distributed chains of skill are crucial, not for production itself, but for the competitive reproduction of skills. Loops set up positive feedbacks of growth in skills that give firms that participate in them the reproductive ability to out-produce firms that do not. Put another way, clusters of firms can produce products with or without hypercycles, but firms whose skill sets participate in production chains that include loops have the further capacity to keep renewing each other through time. This is the chemical definition of life.

So the key is to promote these stable hypercycles. And it turns out that for loop maintenance, the model’s most important ingredient is one touched on in step #4 above—the choice between “source” and “target” reproduction:

In the spatial topology setting, there are two variants of “learning by doing” that can and will be explored:

• “Source reproduction” is where the originating rule in a successful transaction is reproduced.
• “Target reproduction” is where the receiving rule in a successful transaction is reproduced.

For example, if A -> B receives an A from the input environment, transforms it into a B, and then successfully passes that B onto a neighboring B -> C, who transforms it again, then “source reproduction” is where the initiating A -> B reproduces, and “target reproduction” is where the recipient B -> C reproduces. Variation in mode of reproduction thus defines who benefits from the transaction.

We think of source reproduction as “selfish learning,” because the initiator of the successful transaction reaps the reward (like a teacher). And we think of target reproduction as “altruistic learning,” because the recipient of the successful transaction reaps the reward (like a student).

So which works better?

…in comparison with source reproduction, target reproduction dramatically increases the likelihood of growing stable hypercycles. And it also increases the spatial extensiveness and complexity of the firm cluster that hypercycles produce.

And why?

As explained in Padgett (1997), the basic reason for this superiority of target reproduction, or altruistic learning, is repair. Target reproduction combats dynamic instability in a way that source reproduction does not. The basic process of dynamic instability, causing hypercycles to crash, is that as one skill reproduces rapidly, under competition other skills are driven to zero, thereby breaking the reproductive loop of skills. Spatial topology distributes this dynamic into an overlapping series of neighborhoods, thereby inducing local heterogeneity. This opens the door for localized co-adaptive feedbacks to operate. But source reproduction, or selfish learning, does not really attack the basic dynamic instability itself. Source reproduction is this: an initial activated rule passes on its transformed product to a neighboring compatible rule, causing the original activated rule to reproduce. Under source reproduction, frequently activated rules reproduce more frequently, often eventually driving out of business even compatible neighbors on whom they depend for their own survival. As we shall see in the next section, other factors like endogenous environment can sometimes moderate this destructive dynamic, but source reproduction in and of itself does not eliminate the non-spatial instability problem.

In contrast, target reproduction is this: an initial activated rule passes on its transformed product to a neighboring compatible rule, causing the recipient rule to reproduce. Here the more frequently the initial rule is activated the more frequently the second recipient rule reproduces. In this way, a hypercycle can repair itself: as the volume of one skill in a loop gets low, high volumes of compatible skills in neighboring firms reach in to that low-volume skill to build it back up. Peaks and valleys along loops are smoothed.

To help get a grip on this distinction, it’s worth asking whether there are any real-world examples of “altruistic reproduction.” These would be domains in which the output of “upstream” firms promotes the growth of other firms down the line, thereby creating a rich self-sustaining production chain.

This seems true of many industries but especially true of information technology, where small software companies create huge markets (for themselves and others) by developing cheap, portable code for large enterprises. Even open source components—like Linux or MySQL—accelerate industry growth and feed opportunities back to software providers in the form of support contracts and custom coding projects. The result is symbiotic.

(In contrast, one could imagine how—in the kind of closed world explored in this paper—the success of something like a casino would have the perverse effect of draining its own lifeblood, by siphoning the bankrolls of active gamblers. In that way it could be construed as an example of “selfish (source) reproduction.”)

The Hayek Machine

The immediate goal of Baum et al. is to find a way to solve computational problems with large state spaces, like a Rubik’s cube. Their approach is to kickstart an economy of modules—little bits of computer code—that dynamically combine into complicated chains. If these chains were somehow arranged to exploit the structure of the problem at hand, the system would be able to limit its search to a tiny fraction of the space of possible moves.

Intelligent computation of that sort is easier said than done, and so most of the paper is devoted to exploring precisely which features of the module “economy” improve performance, and why.

Before getting into that, though, it might help to explore the simplest of the three problems under attack, called “Blocks World.” With that under our belt it should be easier to understand the model more generally.

An instance of Blocks World has four stacks (0, 1, 2, and 3) of colored blocks. The leftmost stack (0) is your template—you never move any of its blocks as you play. Instead, your goal is to rearrange the blocks in the other three stacks—with the important caveats that (a) you can only grab the topmost block off a given stack and (b) you can only pick one up at a time—until stack 1 matches the template.

Here’s a picture:

The “Hayek” referred to in the caption is the name of the economics-inspired computer program developed in the paper. It is of course an homage to the great Austrian economist Friedrich von Hayek, and it works as follows:

1. The program is a collection of modules, or mini computer programs, each with an associated “wealth.” The authors think of these modules as agents because they can interact with the world (the game).
2. “The system acts in a series of auctions. In each auction, each agent simulates the execution of its program on the current world and returns a nonnegative number. This number can be thought of as the agent’s estimate of the value of the state its execution would reach. The agent bids an amount equal to the minimum of its wealth and the returned number. The solver with the highest bid wins the auction. It pays this bid to the winner of the previous auction, executes its actions on the current world, and collects any reward paid by the world, as well as the winning bid in the subsequent auction. Evolutionary pressure pushes agents to reach highly valued states and to bid accurately, lest they be outbid.” (You might want to re-read the preceding paragraph a few times, until you get a handle on exactly how the auctions work. It’s not complicated—it’s just that there’s a lot of procedure packed into a few tight sentences.)

3. “In each auction, each agent that has wealth more than a fixed sum 10W_init creates a new agent that is a mutation of itself. Like an investor, the creator endows its child with initial wealth W_init and takes a share of the child’s profit. Typically we run with each agent paying one-tenth of its profit plus a small constant sum to its creator, but performance is little affected if this share is anywhere between 0 and .25. Each agent also pays resource tax proportional to the number of instructions it executes. This forces agent evolution to be sensitive to computational cost. Agents are removed if, and only if, their wealth falls below their initial capital, with any remaining wealth returned to their creator. Thus, the number of agents in the system varies, with agents remaining as long as they have been profitable.

   “This structure of payments and capital allocations is based on simple principles (Baum, 1998). The system is set up so that everything is owned by some agent, interagent transactions are voluntary, and money is conserved in interagent transactions (i.e., what one pays, another receives) (Miller & Drexler, 1988). Under those conditions, if the agents are rational in that they choose to make only profitable transactions, a new agent can earn money only by increasing total payment to the system from the world. But irrational agents are exploited and go broke. In the limit, the only agents that survive are those that collaborate with others to extract money from the world.

   “When not everything is owned, or money is not conserved, or property rights are not enforced, agents can earn money while harming the system, even if other agents maximize their profits. The overall problem, then, cannot be factored because a local optimum of the system will not be a local optimum of the individual agents” [emphasis in original].

What happens when this system is let loose on Blocks World?

The population contains 1000 or more agents, each of which bids according to a complex S-expression that can be understood, using Maple, to be effectively equal to A * NumCorrect + B, where A and B are complex S-expressions that vary across agents but evaluate, approximately, to constants. The agents come in three recognizable types. A few, which we call “cleaners,” unstack several blocks from stack 1, stacking them elsewhere, and have a positive constant B. The vast majority (about 1000), which we call “stackers,” have similar positive A values to each other, small or negative B, and shuffle blocks around on stacks 2 and 3, and stack several blocks on stack 1. “Closers” bid similarly to stackers but with a slightly more positive B, and say Done.

At the beginning of each instance, blocks are stacked randomly. Thus, stack 1 contains about n/3 blocks, and one of its lower blocks is incorrect. All agents bid low since NumCorrect is small, and a cleaner whose B is positive thus wins the auction and clears some blocks. This repeats for several auctions until the incorrect blocks are cleared. Then a stacker typically wins the next auction. Since there are hundreds of stackers, each exploring a different stacking, usually at least one succeeds in adding correct blocks. Since bids are proportional to NumCorrect, the stacker that most increases NumCorrect wins the auction. This repeats until all blocks are correctly stacked on stack 1. Then a closer wins, either because of its higher B or because all other agents act to decrease the number of blocks on stack 1 and thereby reduce NumCorrect. The instance ends successfully when this closer says Done. A schematic of this procedure is shown in Figure 3 [below].

This NumCorrect that they keep referring to is a hand-coded module that counts the number of blocks in stack 1 (contiguous from the bottom up) that match the template. It’s a good proxy for performance, and the measure is critical in helping the agents to know whether they’re contributing value to the world (i.e., whether the program is on its way toward a solution).

If NumCorrect is left out of the system, then, performance degrades. Although it’s possible for Hayek to evolve its own quick-and-dirty version, with this mere approximation it’s only able to solve instances about 1/4 of the size as when NumCorrect comes hand-coded.

It’s worth asking, then: how does code, be it NumCorrect or something simpler like Grab(i,j), “evolve” in the first place?

Agents are composed of S-expressions, or recursively-built trees of procedures. You can think of these as sort of like recipes:

The difference here, of course, is that whereas in the Recipe World you might have “sautée” or “chop,” in Blocks World you have things like “grab” or “drop” or “look”:

Our S-expressions are built out of constants, arithmetic functions, conditional tests, loop controls, and four interface functions: Look(i,j), which returns the color of the block at location i, j., Grab(i) and Drop(i), which act on stack i; and Done, which ends the instance. Some experiments also contain the function NumCorrect, and some contain a random node R(i, j), which simply executes branch i with probability 1/2 and j with probability 1/2. [This last randomizer function has the effect of “smoothing” the fitness landscape.]

All our expressions are typed, taking either integer, void, color, or boolean values. All operations respect types so that colors are never compared to integers, for example. The use of typing semantically constrains mutations and thus improves the likelihood of randomly generating meaningful and useful expressions.

It’s hard to overstate how cool this process is. Out of a soup of random S-expressions—primitive codelets like adders and subtracters, if…then statements, loops, and “look” / “grab” / “drop” modules—it’s possible for coherent strategical “agents” to evolve. On its own this idea is incredibly powerful: one can imagine all sorts of programs being developed automatically in this way, growing increasingly powerful by mutating and recombining useful snippets of code. Indeed, there’s a whole field devoted to this approach to programming.

But what’s even more spectacular is that these evolved computational lifeforms (of a sort) cooperate—via the auction process—in long computational chains, even when this requires that the initiating agents defer their rewards.

This last point is critical. It’s not always obvious what the game’s next best move may be, and often moves that are in the interest of long-term progress are (or look) counterproductive in the short term—as when you throw a bunch of incorrect blocks onto stack 1 to fish out an important block on stack 2. So it becomes basically impossible to solve any but the easiest Blocks World instances without the ability to “delay gratification,” that is, to link up lots of low-or-no-reward-actions into chains that collectively work toward some bigger reward at their end.

The only way for this to work is to enforce the rules specified earlier:

A program cannot hope to form long chains of agents unless conservation of money is imposed and property rights are enforced. To evolve a long chain, where the final agent achieves something in the world and is paid by the world, reward has to be pushed back up the long chain so that the first agents are compensated as well. But pushing money up a long chain will not succeed if money is being created or is leaking, or if agents can steal money or somehow act to exploit other agents. If money is leaking, the early agents will go broke. If money is being created or being stolen, the system will evolve in directions to exploit this rather than creating deep chains to solve the externally posed problems. Solving the world’s problems is very hard, and an evolutionary program will always discover any loophole to earn money more simply. [Eric Baum, What is Thought, p. 246.]

That’s also why it’s so important for auction winners in this round to pay their bids to winners from the last round. That way, those agents who most help folks downstream are able to collect larger bid payments—which helps them reproduce and stay alive—and, if their work consistently leads to large rewards down the line, their downstream partners will also reproduce (having collected money from the world) and set up yet more collaboration. It’s exactly the same principle that we saw in the hypercycles world.

Conclusion

To return to the comment that kicked off this whole discussion, it now seems clearer what sorts of laws one minimally needs to sustain a rich economy. The trick is to encourage long, dynamically stable collaborative chains, and to do so requires mechanisms for transferring the rewards of productive activity to everyone who contributed; otherwise, agents who make the kind of short-term sacrifices that fuel deep cooperation will die out, and only the shallowest computation (i.e., production) will be possible.

In the hypercycles model, this was achieved by connecting reproductive success to successful transactions. Target reproduction in particular ensures that successful firms don’t “burn out” their own partners.

And in the Hayek machine, property rights and conservation of money ensure that (a) the only way to earn money oneself is to create wealth for the world and that (b) money trickles back along productive chains to every contributor. [1]

These, then, are the deep compuchemical reasons that such a substantial portion of the U.S. legal system is devoted to enforcing contracts and protecting property rights. Such laws are the bedrock of our—or any—economy.

Notes

[1] Although it may seem that in these models payments go in opposite directions, they actually don’t—even under a regime of “target” reproduction, hypercycles loop back on themselves, which means that A’s altruistic contribution to B’s reproductive success ends up helping A in the end, precisely because the two are wound up in a circle. Target reproduction works better because it keeps such circles alive, and thus allows payments to continue flowing appropriately.

Yesterday a paper I was reading made reference to a “Galtonian composite photograph.” From the context I had a vague idea of what the phrase referred to, but I wanted to learn more. So I:

1. Googled “Galton” and clicked on the first result, the Wikipedia page for Sir Francis Galton.
2. Searched the text there for “composite,” which turned up the following:

   Galton also devised a technique called composite photography, described in detail in Inquiries in human faculty and its development…

3. Googled once again for “Inquiries in human faculty and its development” (no quotes), which turned up a complete Google Books entry.
4. Searched inside that text for “composite,” and found, finally, “Appendix A.—Composite Portraiture,” which contained everything I might want to know on the subject.

The whole process took about two minutes.

I couldn’t help but wonder, though, what my search would have looked like forty years ago, long before the Internet and the proliferation of personal computers. How would I have traced a casual allusion to its source?

Step 1: Go to the library

Short of phoning a friend, there would have been no way to look something up without going to some sort of library. If I lived in the sticks, my best hope would have been a wealthy resident’s set of encyclopedias or some sort of bookmobile—essentially a cart that hauled boxes of books around to small towns. For any remotely obscure topic, I would have been out of luck.

Step 2: Card catalogs and the metasearch

Once I got to my local library, probably funded by Carnegie, my first task would have been to figure out where to start looking. It’s a process that has been mostly obviated by search engines, whose crawling and indexing and relevance rankings allow one to scour the whole world at once, without much regard to where the info comes from.

Back in the day, though, I would have had to start a sort of metasearch to find the right books, periodicals, journals, or microfilms in which to begin my actual search.

Luckily, by the 1960s these materials were indexed exhaustively in large card catalogs, which mapped author names, titles, subject headings, or keywords to the precise coordinates of actual items.

Not so luckily, I would have had to know some author name, title, subject heading, or keyword to look for. In the case of my query above, “Galtonian composite image,” I would have had several plausible angles of attack: the name “Galton, Sir Francis,” which if I didn’t know, I could discover in an encyclopedia; the key words “composite image”; or the subject heading “photography.”

But what if I had no context for my query? For example, what if I ran into a sentence like, “He resembled Little Chandler, if not in size then in stature”? Clearly this is a reference to some moderately famous person or character, but without more to go on, how could I figure out what to look for in the card catalog?

It depended. If I suspected that this was a real person, I could have searched the Encyclopedia Brittanica, something like the Dictionary of National Biography (for dead people), or a Who’s Who (for living people). Once I found my man, I could pick up a biography or two.

Finding a fictional character would have been more difficult, but still possible. There was actually something called the “Cyclopedia of Fictional Characters,” or I could have consulted the once-apparently-indispensable “Benet’s Reader’s Encyclopedia,” which in addition to characters also catalogs author names, novels, stories, literary terms, etc. If I had gotten a handle on where my character first appeared, it would then have been a matter of picking up that particular book. Problem solved.

Unfortunately, it turns out that “Little Chandler” doesn’t appear in any of these. Which means I would have had to either ask a librarian—who might have some tricks up his sleeve (Dewey, of the Decimal System, once said, “…the librarian must be the index of indexes”)—or a very well-read friend, or maybe a professor of literature.

Step 3: The deep crazy world of indexes

Supposing I did have some success with the card catalog—as I would if I were searching for “Galtonian composite photography” instead of “Little Chandler”—I would now be sitting at a table with a large stack of reference works, regular books, a newspaper microfilm, and maybe a few journal indexes, which are like regular indexes, but span the full output of hundreds or thousands of journal issues.

Indexes, by the way, are curious things. They don’t work the same way as Google’s “index,” which is, in technical parlance, really a concordance, because it maps every word to either a snippet or the full text of the page on which it originally appears. Concordances, because they are such a pain to compile by hand, were really only created for stuff like the Bible or the works of Shakespeare. (Example.)

Real indexes (don’t say “indices”) only contain a small, carefully chosen subset of all possible terms. They’re put together by professional indexers who work for publishers on a contractual basis. (Some authors do write their own indexes, but this is apparently frowned upon by the American Society for Indexing and one of Kurt Vonnegut’s characters).

Theirs is difficult work. Indexers must take care to use a controlled vocabulary, i.e., a set of canonical headings that “solve the problems of homographs, synonyms and polysemes by a bijection between concepts and authorized terms.” The idea is to avoid multiple entries like “Cats, 50-62″ and “Felines, 175-183″, and to make sure not to confuse the two meanings of words like “pool” (think swimming and billiards).

They must compose a semantically sensible and reliable syndetic structure, which is the complex network of inter-index references (“See also…”) that link related topics. It’s important to avoid circularity (A -> B -> A) and to consistently connect semantically “close” entries (i.e., if “General Patton” points to “WWII”, you probably also want “General MacArthur” to point there). To do so requires a fairly detailed understanding of the book and its subject matter: Which topics are related? Are these four ideas subunits of some general concept? Is X important enough to index?

Finally, good indexers have to put themselves in the reader’s shoes. What sorts of things will someone new to the subject be looking for? If they want to know more about X, what word will pop into their head? Will they care to see the reference on p. 124, or is it too cursory to be included?

There are whole journals devoted to the subject. In fact, if you want to see a great index, presumably you couldn’t do much better than the journal index for the journal Indexer. (Among other things, there is a section in each issue devoted to “praising” and “censuring” indexes found and submitted by readers. Down the rabbit hole…)

Step 4: Read, read, read

In any case, to return to my hypothetical 1960s search, I would now be (reverently) perusing the indexes of each item in my stack. I’d be looking for words like “Galton” and “composite,” and I’d quickly skim the text of every reference until I found something promising. If yet more books were mentioned—or if I found some encouraging titles in a bibliography—I’d dig them up as well. My stack would slowly grow, and recede, and grow again, as I discarded dead ends and pursued new leads.

In contrast to how things work today, I wouldn’t just be reading the stuff most directly relevant to my query. I’d be tempted by all sorts of tangents along the way: unusual or obscure topics, or hilariously terrible writing, or some new fantastic author I’d never yet encountered. In the search for truth I’d be taking the scenic route—longer, sure, but maybe more rewarding.