the jsomers.net blog.

The first two weeks of college are exceptional for a lot of reasons. But there's one phenomenon that stands out. It's as peculiar and as powerful and as rare as magic. [1]

I'll call it "social annealing" because it so resembles the process by which metals are heated---their particles freed into a loose homogenous jumble---and then recrystallized.

It's the phenomenon whereby college freshmen, having just been ejected from the world they know and loosed upon a campus they don't, mutually decide, in the heat of their eager anxiety, to willingly engage every stranger as a friend.

It happens in the dorms, the cafeteria, coffee shops, classrooms, libraries, and just around town: students with no prior connection approach one another and strike up conversations. Everyone tries everyone out.

Groups do form, but they're not the kind of groups we're used to---they congeal and dissolve with remarkable ease. So a lone student can approach five others without feeling like he's intruding; two sets of roommates can combine; a pack can split with no friction.

Think of how bizarre that is. Think of what it would take, for instance, to introduce yourself to a group of four friends, none of whom you'd ever met. It's practically preposterous.

That's because in normal circumstances, confronting strangers without an overt excuse---the elevator breaks down, say, or a plane is canceled---is an act of aggression. If not outright threatening, it's intrusive, or at the very least distracting.

Even in settings that seem to encourage mixing, like parties or bars, it's not kosher for a person to engage just anybody---one must abide all kinds of cues and conventions and rules; contact must be made with a measure of finesse.

Which is to say that nothing you can find elsewhere in the workaday world even resembles the two-week college free-for-all, the strange fever in which everyone is basically pleased as hell to meet everyone else.

It almost sounds like a fantasy. But I assure you it happened. I'm not a spectacularly outgoing guy, but for the first two weeks of my freshman year at the University of Michigan, I introduced myself to just about everyone I saw. When I'd go down to the cafeteria, I could sit anywhere. At parties, on the way to class, in the dorms, etc., I---like everyone else---would flit from group to group in a crazy kind of convivial Brownian motion. Our social graph was effectively amorphous---fully connected. We were open to each other in a way that I imagine swingers must be open to sex, or hippies to psychedelics.

* * *

It's probably worth asking how this happens, or why. I don't think it's all that complicated:

1. Bizarre things are bound to happen when you throw a large number of eighteen year-olds into close quarters, especially if you don't give them any work to do.

2. For the most part, nobody knows anybody when they first arrive at college. And even if you did know some people, say, a few other kids from your high school, it's good sense to avoid them for a little while, if only to participate in the social madness I've been describing.

   Which means everyone is effectively looking to make a fresh start, to find replacements for their now-disbanded troupe of close dependable friends.

   The trick is that everyone knows this. They know that everyone is in the same boat. And that pervasive common knowledge---where for any two people, A knows that B knows that A knows... that they're both looking to make friends---is enough to fuel cold approaches.

3. Freshmen are called "freshmen" for a reason: they're fresh; they don't yet have a reputation. Everyone understands that, and they understand that first impressions can stick. So it makes sense that they're unusually warm and friendly. It's the best way to keep their social options open.

4. There is what I'll call the "New Year's Resolution Effect," where kids just entering college decide, in light of their radical dislocation and the discontinuity from their life at home, to change themselves in fairly significant ways. In particular, they tend to commit to being more extroverted than they were in high school. It's a common enough ambition to accelerate the pandemonium.

* * *

I mostly wanted to articulate this phenomenon because of something that happened last weekend. After seventeen months away I was back in Ann Arbor, that great college town and the site of my alma mater, ostensibly for a big football game, but really to reunite with lots of old friends, many of whom I hadn't seen since the day we walked the Big House in our robes.

It was quite nostalgic. I really loved that place---still love it---and a lot of what I did that weekend was to reminisce, to reconnect with a mass of pleasant memories and in some cases to relive them.

But I also thought of all the things I didn't do, of all the people I never met. I thought of how little I took advantage of Ann Arbor's unbelievable density of young, curious kids with lots of free time and energy, all part of that same proud collective: students of the University of Michigan.

Walking around campus and the town, then, I had this remarkable urge, much like the one I had as an incoming freshman---but here I was older and more confident and more capable---to engage with everyone I saw.

But not much came of it. I wasn't quite as bold as I could have been, for sure, but nor was the place as ripe as I'd imagined. I didn't understand why until a friend explained it: the kids I'd seen that day had done the same thing we'd done, what we would later come to vaguely regret---they had annealed, and settled, and made themselves a home among a certain set of friendly faces. In the bargain they'd retired from the frenzy of their freshman year, the thrill of radical openness. And I had become a stranger once again.

Notes

[1] Why only two weeks? There are a few reasons: things in general have a way of lasting two weeks; school starts to get serious after about two weeks; and two weeks is just enough time for solid social bonds to form, for kids to get comfortable in their surroundings, and for everyone to pretty much sample everyone else in their little pocket of the campus. After two weeks, the magic's over and the metal cools.

What do you do when you want the last piece of bread? Do you just take it? Or do you offer it to the table?

You offer it. Everybody knows the score: they know that you want the last piece, and that you're only offering it to everyone else to be polite. So they're going to decline, and you're going to end up with the bread.

Why not be more direct? Because then you'd be skipping a ritual that gives everyone a chance to demonstrate how cooperative they are. Rituals like that are important.

Of course, making the offer opens you up to someone accepting it. That's the price you pay for coming off as polite and cooperative; what you've done, effectively, is to wager the bread to earn a bit of social credit.

Nine times out of ten you'll win---you'll look good and keep your baguette---but occasionally you'll lose the bread. You should be prepared for that.

The odds of pulling off a successful perfunctory offer are usually worse than in this bread situation. The reason is that the bread situation happens so often, and is so well-understood, that people rarely deviate from the script: offer, refuse, eat. Whereas in other cases---like when you offer a friend a ride but don't want him to accept---the game isn't so clear; your friend might suspect that your offer is just for show, but he can't be sure. And free rides are attractive. So he's more likely to accept.

The good news is that you get paid for taking on this extra risk. Offering your friend a ride is a bigger deal than offering your last piece of bread to the table, which means you stand to earn more social credit.

The stakes are higher for the other guy, too. He also earns points by turning down your empty offers; we saw that in the bread situation. It's because turning down an empty offer (a) lets you off the hook, which is a nice and cooperative move for him to make, and (b) demonstrates his ability to detect empty offers in the first place, which feeds his reputation as a skilled coordinator.

That, then, is the arithmetic of perfunctory offers: you balance the cost of giving up some X against the points you'd earn for seeming generous, while he weighs the value of receiving X against the points he'd earn for skillfully detecting---and then abiding---your true intentions.

When I was planning my first long drive from school in Ann Arbor back home to New Jersey, I remember looking up directions on Google Maps and noticing, as in the results here, that it really doesn't take a lot of steps -- or driving maneuvers -- to get what seems to be a pretty long way across the country. In fact it only requires seventeen turns using the route Google gives, and even that number is inflated (those "keep left" and "keep right" steps are helpful, maybe, but not necessary).

Just for fun, I tried comparing that 500+ mile trip to one a fraction of its size, something closer to ten miles. You can see such a route here.

Remarkably, at only 2% of the distance, this short hop from one small town in New Jersey to another requires just fifteen steps, or only two fewer than the hefty road trip to Michigan.

I began to wonder: what's the relationship between the length of a road trip and the complexity of the route? Do most trips, long or short, require roughly the same number of steps? How many steps are there in the most complex route in the country? What's the distribution of step counts for every possible route in the contiguous United States?

These questions turn out to be tractable, thanks in large part to Google's Directions API, which gives lowly developers like me access to their full suite of mapping, geolocation, and pathfinding algorithms, huge stores of data, and fast servers that can deliver tens of thousands of query results to a single client computer in a matter of minutes.

Before we dive into methods and results, though, let's lay out exactly what we're looking for:

1. We want a histogram of step counts for some representative sample of routes within the US. This will give us a really good sense of how complex a typical road trip might be.

2. We'd like to find the most complex route in the country, i.e., a pair of points such that the driving directions between them, given by Google, include a larger number of steps than for any other pair in the contiguous US. It's extremely unlikely that we'll find the monster route, but at least we'd like a ballpark estimate of its step count -- is it 35, 500, 90, 180?

3. We want a plot of route distance against route complexity. What will the plot look like? Is complexity a linear function of distance? Is there a direct or inverse relationship? Will there be any pattern at all?

4. It would be pretty cool to find a "coefficient of friction" for regions in America, that is, a numerical estimate of how hard it is to drive through a particular region based on how many steps there are, on average, in a route passing through it. We could use this information to create a "heat map" of the entire US, with individual counties or zip codes shaded by friction. Such a map would help us figure out which states have the thorniest roads, or where the most straightforward routes are, or which cities are the hardest to get out of.

To get started, then, I went looking for a random sample of points. One way to do that would be to draw a box inscribed within the continental US and simply generate random lat-longs within that box. The trouble with that method, I thought, was that it could easily drop you in barren or ridiculous places like deserts or lakes; I wanted to focus on plausible real-life trips from one population center to another.

So I went looking for a data set, and before long, found one: the "MaxMind World Cities with Population" file, a 33 MB free download with more than enough data to get things rolling: after eliminating non-U.S. cities (a simple grep did the trick, since the text is nicely structured), I was left with 141,989 points covering nearly every corner of the country.

I hacked together a tiny Rails project (RoR being my hammer-that-makes-everything-look-like-a-nail at the moment) to (a) load the cities and lat-longs into some structured form, (b) drop that data into an HTML page hooked up to the JavaScript Google Directions API, and (c) write the results back to a database. All of the relevant code, along with a SQLite3 database with the structured cities data and results, is available at this github project page.

Perhaps the most important snippet, which I'll excerpt below, is the code that actually samples points and talks to Google:

var map;
var directionDisplay;
var directionsService;
var stepDisplay;
var markerArray = [];
var step_counts = [];
var step_summaries = [];

function initialize() {
  // Instantiate a directions service.
  directionsService = new google.maps.DirectionsService();
}

function calcRoute(start, end) {
  // Retrieve the start and end locations and create
  // a DirectionsRequest using DRIVING directions.
  var city_start = [start[2], start[3]].join(", ");
  var city_end = [end[2], end[3]].join(", ");
  var latlong_start = [start[0], start[1]].join(",");
  var latlong_end = [end[0], end[1]].join(",");
  var request = {
      origin: latlong_start,
      destination: latlong_end,
      travelMode: google.maps.DirectionsTravelMode.DRIVING
  };

  // Route the directions and pass the response to a
  // function to count the number of returned steps.
  directionsService.route(request, function(response, status) {
    if (status == google.maps.DirectionsStatus.OK) {
      var step_ct = countSteps(response);
      step_counts.push(step_ct);
      step_summaries.push([step_ct, city_start, city_end])
    } else {
      console.warn("Couldn't count steps for this route.");
    }
  });
}

function countSteps(directionResult) {
    var myRoute = directionResult.routes[0].legs[0];
    return myRoute.steps.length;
}

function getAndExecutePairs(n) {
    $.get("/directions/get_pairs", {n: n},
        function(ret) {
            pairs = ret;
            console.log(n + " lat/long pairs downloaded successfully.");
            execute(pairs);
        }
    )
}

function execute(pairs) {
    for (i = 0; i < pairs.length; i++) {
        pair = pairs[i];
        start = pair[0];
        end = pair[1];
        calcRoute(start, end);
    }
}

It's all pretty straightforward. The process is kicked off by the getAndExecutePairs() function, which just hits the Rails server for n pairs of cities. This is the code it calls:

def get_pairs
  n = params[:n].to_i
  cities = City.find(:all, :limit => n * 2, :order => "random()");
  lat_longs = cities.collect {|c| [c.latitude, c.longitude, c.city, c.state]}
  pairs = Hash[*lat_longs].to_a
  render :json => pairs
end

And that's it. With just ~100 or so lines of code, I was able to get a decent grip on the first three of the four questions posed above. In particular:

1. What's the distribution of step counts for every possible route in the contiguous United States?

The answer, based on a random sample of some 2,000 points (and confirmed later by 8,000 more trials), is that you have a sort of right-skewed distribution centered at 20-30 steps and tailing out near 60.

2. How many steps are there in the most complex route in the country?

This is a lot less definitive, but the answer I got was 69 steps, in a route from Ponderose Pine, NM to Wildwood, MN. A friend suggested something like the following approach for finding more complicated routes:

Suppose you're getting "good" (i.e., stepful) routes between points A and B. Draw a box around each of A and B and "wiggle" your start points within that box. If wiggling in one direction removes steps, try wiggling in another direction; or if it's not direction that matters, but rather something tricky like "being within a development or behind a river," maybe you could just select points within the box randomly and assign scores to different areas (sort of like a game of Battleship). That way you slowly optimize promising routes until you end up with truly high numbers.

One potential pitfall of this approach is that there could be discontinuities -- A to B could take an unremarkable 35 steps, but (A + ε) to B could take 70 steps -- in which case you might not choose the right starting points to begin with. But this would probably only happen if there was something like a maze next to a normal neighborhood.

3. What's the relationship between trip distance and route complexity?

The graph above plots route distance (measured as the surface distance between the two lat-long pairs) against step counts. Aside from a few outliers, you'll notice that a really wide range of step counts is covered by a relatively narrow range of distances: that is, most of the variation in step counts is accounted for in trips less than a few hundred miles; and at the margin, an extra mile buys you very little in terms of route complexity.

Perhaps the most interesting points are those short routes with a large number of steps. See, for example, the 69-step route called out above (just over 1,000 miles), or the 67-step route from South Lyndeborough, NH to Hartley, GA (875 miles).

(You'll notice a few routes with more than 70 steps -- these can be safely ignored, since they either originate from or end up in Alaska (oops!)).

4. Can you generate a "heat map" that shades regions by their "coefficient of friction," i.e., a numerical estimate of how hard it is to drive through a particular region based on how many steps there are, on average, in a route passing through it?

This is left as an exercise for the reader, as is the task of finding even longer routes (here's a 75-stepper) and the more general underlying problem of understanding which features -- of cities and roads and geography -- are implicated in route complexity.

In the January 1983 volume of the journal Analysis, Gregory Kavka presented the following philosophical thought experiment:

An eccentric billionaire places before you a vial of toxin that, if you drink it, will make you painfully ill for a day, but will not threaten your life or have any lasting effects. The billionaire will pay you one million dollars tomorrow morning if, at midnight tonight, you intend to drink the toxin tomorrow afternoon. He emphasizes that you need not drink the toxin to receive the money; in fact, the money will already be in your bank account hours before the time for drinking it arrives, if you succeed. All you have to do is... intend at midnight tonight to drink the stuff tomorrow afternoon. You are perfectly free to change your mind after receiving the money and not drink the toxin.

It's probably worth asking why this is considered an interesting problem in the first place, since on face there might not seem to be anything difficult going on.

But Kavka seems to think that it would actually be very hard, if not impossible, for someone to intend at midnight tonight to drink a painful poison tomorrow if they knew that they could eventually back out without losing any money. In asserting this he is really taking two steps: first, to argue that a person would never actually drink the poison, because by the time they reach the moment of truth, the prize will have already been won or lost---and no one would choose pain vs. not-pain if all else is equal; and second, to argue that if you knew that you weren't going to drink the poison tomorrow, there is no way you could intend to tonight.

The first step is fairly sound, although one can imagine challenging it in the following way: suppose that you had somehow managed, last night, to intend to drink the poison today---isn't there then a sense in which you're committed to go through with it (such commitment being what initially enabled your intention)? If so, then it may be that to win the money you do have to drink the poison after all. (This was actually the tack taken by philosopher David Gauthier in a paper published shortly after Kavka's. See the Wikipedia page for more.)

It's an interesting possibility that cuts to the core of the problem: can someone intend to drink the poison, and if so, what would such a person look like (epistemologically, psychologically)? Would they be somehow bound, Gauthier-style, to follow through, or is there a way to win the money without having to endure a day of pain?

I certainly don't think it would be the strangest thing to ever happen, and actually I could imagine a few concrete ways you might pull it off:

1. You could force yourself to literally forget about the option to back out tomorrow. I doubt I could do this, but there must be some people whose minds are sufficiently powerful (or broken) to selectively "delete" memories. (In which case they would in fact follow through and drink the poison, unless the billionaire reminded them that they had the option not to.)
2. More easily, you could construct for yourself a sort of conspiracy theory in which the billionaire is tricking you, and in which you'll have no choice but to drink the poison tomorrow. You could get yourself all riled up this way, momentarily terrified by the prospect, but gradually talk yourself down to the point of acquiescence---"there are no lasting effects; it'll only be painful for a day"---mentally preparing yourself so well to drink the poison that you essentially "forget" about backing out, since that option no longer seems real. This train of thought may sound bizarre, but I'd wager that every day all sorts of paranoid people immerse themselves in far more elaborate fantasies. (And the benefit of pulling this one off is that you would wake up the next day a million dollars richer and amazingly relieved the moment the billionaire let you go.)
3. You could be one of those thrill-seekers who thinks it would add to your stock of "life experience" to endure a temporary bout of intense pain. (In this case you probably would end up drinking the poison.)
4. Or you could, quite like a regular person, decide that you really want the million dollars and that you'd be willing to endure a day of pain for it. You might then set out to find some way to intend tonight to drink the poison tomorrow, even though you know in the back of your mind that you are very likely to back out. So for the next several hours you might fight yourself, trying hard as you can to eliminate these nagging thoughts of backing out, flitting back and forth between "totally committed" and "knowing full well I won't go through with it." You might even start to consider options like #1-3, but you'd probably dismiss them for being too weird or somehow infeasible. Then finally you'd hit on a brilliant idea: a contract! You have long been telling yourself that "my word is my bond," that you are trustworthy, that other people might renege or cave but that you always come through. You are certain that if you signed a contract with yourself you couldn't break it. And so you'd draw one up, sign it, and sleep comfortably, slightly nervous about the day of painful illness ahead, but pleased to know that you've got a million dollars in the bag. And it would probably be enough to net you the cash. (Of course odds are that the next day you'd go back on your word, but I imagine we all know a few people who really are so steadfast that they'd actually drink the poison.)

There are no doubt many more methods available. The key point is that human psychology is warpable enough to satisfy the billionaire's demands---this is, after all, the same psychology that can willingly kamikaze itself in the name of God or country, that is so often ruled by fear and insecurity, that regularly violates expected utility theory, and that acts, for the most part, as though it's not headed toward an infinite doom.

So the real puzzle here, in my mind, is why Kavka thought this was a "puzzle" at all.

"It turns out" became a favorite phrase of mine sometime in mid 2006, which, it turns out, was just about the time that I first started tearing through Paul Graham essays. Coincidence?

I think not. It's not that pg is a particularly heavy user of the phrase---I counted just 46 unique instances in a simple search of his site---but that he knows how to use it. He works it, gets mileage out of it, in a way that other writers don't.

That probably sounds like a compliment. But it turns out that "it turns out" does the sort of work, for a writer, that a writer should be doing himself. So to say that someone uses the phrase particularly well is really just an underhanded way of saying that they're particularly good at being lazy.

Let me explain what I mean.

Suppose that I walk into a new deli expecting to get a sandwich with roast beef, but that when I place my order, the person working the counter says that they don't have roast beef. If I were to relay this little disappointment to my friends, I might say, "You know that new deli on Fifth St.? It turns out they don't even have roast beef!"

Or suppose instead that I'm trying to describe a movie to a friend, and that this particular movie includes a striking plot twist. If I wanted to be dramatic about it, I might say "...and so they let him go, thinking nothing of it. But it turns out that he, this very guy that they just let go, was the killer all along."

So far so good. Now suppose, finally, that I'm a writer trying to make an argument, and that my argument critically depends on a bit of a tall claim, on the sort of claim that a lot of people might dismiss the first time they heard it. Suppose, for example, that I'm trying to convince my readers that Cambridge, Massachusetts is the intellectual capital of the world. As part of my argument I'd have to rule out every other city, including very plausible contenders like New York. To do so, I might try something like this:

When I moved to New York, I was very excited at first. It's an exciting place. So it took me quite a while to realize I just wasn't like the people there. I kept searching for the Cambridge of New York. It turned out it was way, way uptown: an hour uptown by air.

Wait a second: that's not an argument at all! It's a blind assertion based only on my own experience. The only reason that it might sort of work is that it's couched in the same tone of surprised discovery used in those two innocuous examples above---as though after lots of rigorous searching, and trying, and fighting to find in New York the stuff that makes Cambridge the intellectual capital, it simply turned out---in the way that a pie crust might turn out to be too crispy, or a chemical solution might turn out to be acidic---not to be there.

That's what I mean when I say that pg (who, by the way, actually wrote that passage about Cambridge and New York) "gets mileage" out of the phrase: he takes advantage of the fact that it so often accompanies real, simple, occasionally hard-won neutral observations.

In other words, because "it turns out" is the sort of phrase you would use to convey, for example, something unexpected about a phenomenon you've studied extensively---as in the scientist saying "...but the E. coli turned out to be totally resistant"---or some buried fact that you have recently discovered on behalf of your readers---as when the Malcolm Gladwells of the world say "...and it turns out all these experts have something in common: 10,000 hours of deliberate practice"---readers are trained, slowly but surely, to be disarmed by it. They learn to trust the writers who use the phrase, in large part because they come to associate it with that feeling of the author's own dispassionate surprise: "I, too, once believed X," the author says, "but whaddya know, X turns out to be false."

Readers are simply more willing to tolerate a lightspeed jump from belief X to belief Y if the writer himself (a) seems taken aback by it and (b) acts as if they had no say in the matter---as though the situation simply unfolded that way. Which is precisely what the phrase "it turns out" accomplishes, and why it's so useful in circumstances where you don't have any substantive path from X to Y. In that sense it's a kind of handy writerly shortcut or, as pg would probably put it, a hack.