the jsomers.net blog.

Wandering the web stacks to improve your information diet

Alarm bells should go off any time you open your web browser, poke around for a few minutes, and close it feeling roughly like you had turned on the television and found that “there’s nothing on.” Because if that happens, it means either that the Internet has somehow become less dynamic and informationally rich than is commonly thought or, more likely, that you’re now using it like the library patron who, having fallen in love with the periodicals section, has forgotten all about the books.

It’s surprisingly easy to let this happen. All you have to do is curate a collection of channels — blogs, magazines, newspapers, social news sites, twitter users, etc. — sufficiently abundant and interesting that you can afford to stop looking. At that point, information-satisfaction is merely a matter of consuming your feed. The wider web might as well not exist.

Think again of the library. Imagine if all you did there was peruse the latest editions of your favorite periodicals. Certainly you could keep yourself busy this way, but there would be times, no doubt, when there’s just nothing new. And if it doesn’t occur to you to wander the stacks, you’ll probably just leave.

So it goes with the web. Which leads us to the obvious question: what’s the Internet equivalent of a book?

If I had my way the answer might well be “books,” but until those become openly available we’ll have to settle for something else. Here are a few candidates:

  • Professors’ home pages. Find the X department at a good university, where X is some subject you’re interested in, and odds are there will be at least one professor there with a gigantic, ugly web page full of links to top-notch papers, blogs, essays, books, and other, even bigger pages of links. Examples: Cosma Shalizi, John Lawler, John Baez, Timothy Gowers, Tyler Cowen, etc.
  • Along the same lines, I have recently discovered that grad students at many universities, especially if they’re in small-ish departments, will collectively run a group blog. See for example Go Grue, run by philosophy grad students at the University of Michigan.
  • If you chase around the blogrolls of good blogs, you’ll not only get a feel for the online territory staked out by some discipline, but you’ll also start to notice that a few really strong blogs show up in nearly everyone’s list.
  • Surf Wikipedia, of course, but follow the footnotes. Most Wikipedia articles draw heavily on one or two much more comprehensive sources. For example, many philosophy articles either point to or are in some sense abridged versions of pages at the Stanford Encyclopedia of Philosophy. I’m not sure but I would imagine other fields would have a few of their own go-to resources.
  • Classic short fiction and nonfiction is very easy to find on the web. Google “[famous author name] essays” and you’ll likely find a page like this (for Montaigne) or this (for Orwell).
  • Pay attention to people who are themselves voracious readers, and ask them what they read.
  • Many law reviews post the full text of each issue online. See Harvard’s, for instance. The “articles” and “essays” are mostly by professors, but the “notes” are written by students and are generally more accessible.
  • Find “classic” academic papers, especially in fields with which you’re unfamiliar. Most classics are so called because they are especially readable, on top of being major contributions to the state of the art. One way to find them is through profs’ pages, as above, or by googling “classic papers in X“. In some fields, like cognitive science, people have made lists for just this purpose. Or you can look at the syllabi of introductory grad school courses, citation counts at Google Scholar, etc.

So if ever you catch yourself closing your browser after an unsatisfying short session — and you still have time to consume, rather than create — realize that you’re probably turning your back on the stacks. The wonderful, wanderable stacks.

Some excellent talk on crime

I transcribed the following from the bloggingheads.tv dialog between Mark Kleiman and Reihan Salam:

SALAM: I wanted to ask you: we’ve had this sharp decrease in the crime rate over the past couple of decades, and there are a lot of people who believe that this is a function of the “tough on crime” policies, three-strike rules, and also more incarceration. And yet you take exception to this view.

KLEIMAN: Well the three-strike rules are, from a crime-control view, certainly a mistake. “Three strikes” means turning your prisons into retirement homes for former burglars. Right now the average age of our prisoners is about fifteen years older than the average age of the offender, which means we’re not targeting those prison cells very well on people who would otherwise be committing crimes if they were out. The other thing is that three strikes, because it’s a cumulative idea over a lifetime, doesn’t do much to measure the rate at which somebody commits offenses. Someone who has two felony convictions by the time he’s twenty, he’s a much more dangerous character than someone who has three by the time he’s forty. So there’s no way three-strikes contributed anything… the contribution to crime-control was negative.

Building more prisons, up to some point, prevents crime by putting people away who don’t commit crimes on the outside. If we included prisoner-on-prisoner crime in the crime rate, then the crime rate wouldn’t have gone down nearly as much.

[. . .]

SALAM: So you propose a different approach … and that of course is the core idea of the book [When Brute Force Fails] … Can you tell us what introduced you to some of those ideas, and also just the broad sweep of the book itself, of the book’s thesis.

KLEIMAN: I should say there are I think two goals here, not one. One is the one you mentioned, which is the one everybody thinks about, which is reducing crime. The other goal is reducing the amount of damage we do through the enforcement system. I count the 2.3 million people behind bars not merely as an expense, but as an intolerable infliction of human suffering… unless there were no other way to control crime. So the subtitle of the book is “how to have less crime and less punishment,” and I weight those about equally.

And the basic idea is that the criminal justice system has to learn what every parent and what every animal trainer knows, which is that in order to change behavior, it doesn’t do any good to issue vague threats of serious punishments and randomly inflict them sometimes. You need to be clear about what the rules are, you need to be completely consistent about making sure that every violation of the rules draws a sanction, you have to make sure the sanction happens very close in time to the offense, and if you do all of that, you don’t have to make those sanctions severe.

And one of the subsidiary points of the book is that severity isn’t merely worthless — it’s worse than worthless, because severity is the enemy of swiftness and certainty. The more severe a punishment is, the more due process you’re gonna want to spend before you inflict it, that means the further it gets from the offense, and, the less of it you can do because the punishment itself is expensive. So every 25-year sentence that California hands out to someone who committed two burglaries when he was twenty, then steals a slice of pizza when he’s forty, that 25-year sentence is 50 six-month sentences that can’t be imposed. [. . .] So we have to start thinking about punishments as costs and not as benefits.

SALAM: I know that in the book you discuss a number of more optimistic stories, in which people have kind of broken with that pattern [of heavy reliance on costly incarceration]. And the central story you tell involves a judge in Hawaii, Steven Alm.

KLEIMAN: The alternative to prison, the original alternative to incarceration, is probation and parole. [. . .] And the problem is that neither of those systems works very well, because they both reproduce the central failing of the larger system, which is that they’re not consistent in punishment. It’s occasional, random, and severe punishment, which is exactly the wrong pattern.

So, what Judge Alm discovered when he got to be a judge in Hawaii, was that probation officers were bringing him revocation cases. Okay, here’s a guy with seven positive tests for methamphetamine and three missed probation appointments over the years — “your Honor, I can’t control this guy, so send him off to prison.” And the judge started saying to the probation officers, “Okay, I see that, this guy’s clearly not getting the message. But why am I only seeing this when it’s too late to do anything about it? If this is his tenth violation, what happened the first nine times?”

And they explained, “we’ve got caseloads of 150-180, we see them once a month, 30% either don’t show up at all or show up and test dirty. So that’s 50 people for each of us every month. We can’t possibly write up that many revocation motions, and if we did, you couldn’t possibly hear that many revocation motions. So, we jawbone on the ones we can — ‘look, if you keep doing this, you’re going to finally have to see the judge’ — the ones that will comply we manage, and the ones that won’t comply we bring to you.”

And the judge said, “Okay, I still want to know what happened the first nine times. You wouldn’t train a puppy that way, you wouldn’t raise your kid that way. This can’t be the right system. I want to see the first violation.”

And they said, “Your Honor, is there some part of ‘im-possible’ that you’d like us to explain more slowly? We can’t do that — there aren’t that many hours in the day.”

So the judge, being a reasonable judge, said, “Okay, let’s compromise. Make a list of the people who have violated so often that the next time you’d bring them to me for a revocation.” So they made a list, 35 people on the list, and he brought them all in for he called a “warning hearing” [. . .] and he read them the Riot Act. He basically said, “Look, you’re not doing your part. This is a deal, you were supposed to go to prison, and you’re not obeying the rules. So from now on, every violation you’ll be in jail that night. I promise.”

And he put them on random drug testing [. . .], and if your number’s called, you’ve got till 2:30 to show up for a drug test. If you’re dirty or if you don’t show up, you’re put in jail right away. [. . .] So now you’ve got the punishment happening right up against the crime, happening every time.

It was strange — he addressed them as responsible adults: “Look, I’m not interested in your excuses. You’re a grown-up, you’re supposed to be able to weigh the rules. Show up when you’re supposed to show up and show up clean.”

[. . .]

KLEIMAN: About 20% of the new arrests in a typical city for felonies are people who are already on probation. And another 5% is parole, and another 10% is bail. So a big chunk of new arrestees are already in the system. So I think [this program will] make a big difference. It could reduce crime, it could reduce drug use, because after all, the heavy drug users mostly wind up on probation. Because you can’t maintain a heavy methamphetamine, or heroin, or crack habit without doing something you can get arrested for, unless you have a trust fund or a rock band. So about 3/4 of the heavy users turn out to be criminally active — not in the Bernie Madoff way, but in the “getting busted once a year” way.

[. . .]

KLEIMAN: [The judge] started out with 35 people, surprised by the fact that even though he was testing them much more frequently — under the old system, not only were the drug tests once a month, they were pre-announced. So it was very easy to pass your drug test: all you had to do was not use for three days. Nonetheless the violation rate was 30% per test. That’s not atypical. [. . .] Probation is such a weak system that people learn to ignore it. Now that’s not wise — eventually they’re likely to get revoked — but not this time. And “not this time” is not a good way to control behavior.

But the big surprise was that the warning alone — the first week, with 35 people he would have expected to have ten hearings, and he only had three, next week he had two. Of those 35 people, fewer than half ended up having to get sanctions. The warning alone, backed up with organization that made it clear that the warning was going to be carried out, was sufficient. That’s been true in the larger group — we did a randomized control trial [. . .] and fewer than half ever violate, fewer than half of those who violate once violate twice, fewer than half of those who violate twice violate a third time… now you’re down to people where you say, “Um, buddy, are you having trouble controlling your meth use? Because it looks like you are, since you’ve now been to jail three times and you’re not stopping. How about a nice residential treatment program? Or if you don’t want our residential treatment program there’s a prison cell available for you.” But now we’re talking about 8% of the population, and now you can concentrate. This is what Hawken calls “behavioral triage.”

[. . .]

KLEIMAN: [on concentration:] Across time, something like 6% of the young men account for somewhere between half and two-thirds of the violence. That’s consistent with what we know about other kinds of human behavior. Most behaviors cluster in that way. Alcohol, for example: 50% of the alcohol is used by the top 10% of drinkers, those who have more than four drinks per day.

[. . .]

KLEIMAN: It’s strange that people who call themselves fiscal conservatives think nothing about a 25-year prison sentence, which is about (discounting to present value) about a million dollars, for stealing a slice of pizza. Now why isn’t that wasteful government spending? The answer is that wasteful government spending only counts when you’re trying to help people.

[. . .]

KLEIMAN: [After talking about how seeing a friend on parole having a curfew might dissuade you from crime:] My teacher Mark Morry used to say that what people really want to inflict as punishment on criminals is a middle-class lifestyle. You’ll get disputes about this — the people on the sort of Foucauldian end of the spectrum are going to disagree with this — but I think that imposing middle-class lifestyles on people who have committed felonies is a perfectly reasonable thing to do.

[. . .]

SALAM: You’ve described this idea as a more effective means of dealing with juvenile offenders. And that strikes me as an unusually important issue…

KLEIMAN: The problem with juveniles, is that it’s very hard to punish a juvenile in a way that he minds that we’re willing to do. Because you don’t want to damage the kid; you’re hoping that he’s going to turn around and be a productive citizen — and most of them will. Two-thirds of the people who are an adjudicated delinquent, for doing something that for a grown-up would be called a felony, never commit an adult felony. [. . .] So one goal of the juvenile justice system is not to interfere with natural aging out… and so you’re always a little bit worried — sending a kid to juvenile hall, his probability of being an adult criminal may just have gone up instead of going down.

So you’re looking for something you can do that the kid will not want to have happen again, that other kids hearing about it will not want to have happen to them, but which isn’t so damaging as to knock the kid off the chance of re-establishing a more responsible pattern.

My nominee for that is a “48-hour timeout”: find an abandoned motel room, take out the television set and the telephone, put in a Bible, a [something else], a Koran, three days’ worth of meals ready to eat, put ’em in the room at six o’clock on Friday and say “we’ll see you on Monday morning in time for school.” And that will be a pretty unpleasant weekend, not cruel and unusual, it seems to me, not damaging, but pretty aversive, and not interesting to talk about. He’s not going to want to go home and brag to his friends about his 48-hour weekend, and he won’t want to do it again.

[. . .]

SALAM: [Talking about how prison can be criminogenic:] How is it that prison fails to be aversive? Because I mean it sounds pretty darn unpleasant, given the sky-high crime rates in prison, you’re isolated from your family and people you care about, how could it not lead you to think “I never want to go back to that horrible place.”

KLEIMAN: You’re right, it absolutely terrifies you and me, and we’re not there. Now let’s think about the people who go there. Where are they coming from? They’re coming from neighborhoods characterized by noise, violence, and idleness, and we send them to a place characterized by noise, violence, and idleness. I mean it’s the Briar patch. It’s a great deterrent for the people who are designing the system, not so much for the people the system applies to. If a prison looked a lot more like a Benedictine monastery, which you or I might regard as not a bad way to spend a couple of weeks and de-tox our heads, it would be Clockwork Orange-ish to a lot of the people there.

So I think we need to create prisons that don’t reproduce the social order of underclass neighborhoods.

I haven’t done the careful work on this that it needs, but I have a strong suspicion that the modern prison, which is designed for 1,500 prisoners, is about 15 times too large, and that much smaller units (you could have several of them on the same campus), much smaller social units, even if that meant fewer services, could be a good deal. And you don’t have to worry about the gangs… at least that’s the hope.

[. . .]

KLEIMAN: If you’re black and male and drop out of high school, you’re more likely than not to end up in prison at some point.

[. . .]

KLEIMAN: One of the speeches I make to people who are to my left on this stuff is that the current system, with all of its disproportionate incarceration by race, is still failing to deliver equal protection of the laws. It’s a lot safer to commit a burglary in Harlem or South L.A. than on the Upper East Side or in Georgetown or in Brentwood.

SALAM: It reminds me of Ed Glazer’s findings: even if you look at vehicular manslaughter, it seems that there’s a really dramatic difference between the punishments meted out to those who kill African Americans to those who kill non-black individuals.

KLEIMAN: Absolutely right. And I think that’s part of what generates high crime rates in black neighborhoods. If you grow up in that neighborhood, you are surrounded by people that it’s relatively safe to victimize. So the high-net incarceration rate is the result of a social trap. This is my tipping story: they’re caught in the high-crime, high-violation, high-punishment-but-low-punishment-per-offense equilibrium, and the question is how to force that over into the low-violation, low-total-violation-but-high-punishment-per-offense equilibrium. That’s the better place to be, and that’s what all this concentration stuff is about.

[. . .]

KLEIMAN: The number of cocaine dealers in prison now is fifteen times what it was in 1980, and the price of cocaine has actually gone down. [. . .] We know that the mechanism has been a drop in the wages of retail cocaine dealers. There’s a wonderful paper by Venkatesh called, “Why do Crack Dealers Live with Their Mothers?” They’re making less than the minimum wage, when they used to be making $30 an hour. My belief is that the primary reason is that we sent so many crack dealers to prison who now can’t get a legitimate job, that we’ve created an industrial reserve army of potential crack dealers. [. . .] So the enforcement we did in 1980 is driving the price of cocaine down, just showing that the world is full of perverse consequences.

[. . .]

KLEIMAN: Alcohol is responsible for more crime than all the illicit drugs combined. And that’s not on the dealing side — we don’t have alcohol dealers now as we did during prohibition, but we have alcohol users doing all kinds of horrible stuff to themselves and other people. And there’s an easy fix for that one: raise taxes. If we double the tax on alcohol we’d reduce homicide by about 6%. That’s Phil Clips’s number. And of course we’d substantially improve public health. It turns out that when you double alcohol taxes it raises the price of drink by about 10% — you’re not impacting the alcohol use of light users. If you have a couple of drinks per week, paying an extra dime for a drink is just not a big deal. It’s the folks at the top of the distribution, who are probably made better off by drinking less even though they may be spending more money on it.

It’s an absolute win all around — you could get about $9B a year out of it at the federal level. That’s 10% of what we need to pay for the health bill.

[. . .]

SALAM: The kinds of folks who are exercised by [paternalistic mechanisms for fighting crime] don’t often experience what it’s like to live in a high-crime neighborhood. I think that skews their perspective to some degree.

KLEIMAN: But also they don’t understand what they’re paying for crime. They think the fact that they live in a suburban neighborhood that means they spend 45 minutes each way commuting to work is a fact of life. It doesn’t occur to them that that’s one of the consequences of crime. [. . .] A lot of middle-class folks don’t appreciate how much the liberty of poor folks is limited by crime. It’s odd that liberals, who are usually aware that liberty can be limited by private as well as public action, aren’t as alive to that. But I think they’re also unaware as to how much their own liberty is limited.

You know I’m old enough now to talk about the good old days — in the good old days, I grew up in Baltimore, which is a fairly rough town, and I was a minor baseball fan, and a couple times a year I’d go to a twilight night double header at Memorial Stadium. Which meant that at midnight I would take two buses home, changing at Park Circle, which was a tough place even back then. And it never occurred to my parents to worry about it. No thirteen year-old has that liberty today.

SALAM: Yeah, it’s extraordinary to me just how, even over the course of a twenty-year period, the number of kids you see under the age of 15 walking around by themselves unsupervised. I saw a kid in downtown Washington, about 12 years old, riding a skateboard by himself — and I was shocked he was by himself. But of course he should be riding! It’s perfectly safe. Yet, the psychological burden of that, that’s a really good point.

KLEIMAN: I’m against capital punishment in general now, but I’ll make an exception for whoever invented the pictures of the kids on milk cartons.

Tic takeover, a.k.a the mindvirus

The instant I notice that a speaker paces, or a writer abuses the em dash, or a partner in conversation says “uh” too often–that is the instant my attention crumbles. From that point forward my brain can process nothing but the tic.

Another example: I remember a friend pointing out that one of the guys in our informal study group would often whisper-chuckle at whatever he was reading, usually as a way of baiting us to ask him what was so funny, and when we bit, he’d read aloud the relevant passage.

This didn’t annoy me in the least, nor did it register that it was some kind of a regular thing. But my friend, well, he thought that this guy’s “habit” was obnoxious, distracting, and happening at a frequency that had “gotten out of hand.”

And of course, once he said this I couldn’t help but look for examples myself. When one cropped up, all I saw were the ways in which it was annoying. In effect my friend had infected me with his frustration.

This worked so well, as a matter of fact, that I started to unconsciously reason that the whisper-chuckler was actually an annoying person–for who else would do such an obnoxious thing so often?–and my negative attention spread to his other “tendencies.” Soon I saw the worst in his every move.

Not to mention that this had all become a topic of regular conversation among the rest of the group, which meant we were constantly sharing and confirming our evil observations, feeding off each other’s captious energy.

We all actually had to “take a break” from our victim, i.e., keep him out of the study group loop for a few weeks, just to save the friendship. He had become temporarily unbearable, for no other reason than because someone had asked if I’d “ever noticed that he always…?”

We later came up with the word “mindvirus” to describe what had happened. We realized that even offhand remarks, if they draw attention to a tic or tendency, have enormous power: they seize a person’s mind, seeding peeves that can multiply into the worst kind of judgment and disgust.

So beware of what you notice, and when in doubt keep it to yourself.

Rule 110, or, how a little orange dot becomes a universal computer

See jsomers.net/110. Won’t take all of five minutes but I think it’s neat.

Open books

I would happily trade the world’s libraries, all its cozy coffee shops, and the feel of a hardback in my hands for on-demand access to the complete plain text of every book.

Without difficulty, I can find nearly every published academic paper written in the last fifty years—you cite it and, within minutes, I’m reading it. That should work for books, too.

I’d want the plain text in particular because it’s so easy to work with. It’s already small but can be compressed very efficiently; it’s fully searchable; everyone knows what to do with it; it can be copied, pasted, annotated, etc. trivially; and it works everywhere.

Voracious book readers will tell you that, in its present state, “there is very little on the web but elementary expositions, provisional and forbidding technicalities, and rubbish.” Part of the reason for that is that there’s a lot you can do with hundreds of carefully wrought, tightly integrated pages that won’t work in the kinds of short forms encouraged on the Web. We will be cheating ourselves intellectually if reading books is even relatively inconvenient.

Would people stop writing if the text of their books were distributed this way? I interact with a dozen small-time authors every day, and yes every one of them is just goddamn thrilled that their book is “in print”—which makes you think that they’re excited by the physical object. But really what excites them is having finished their work, which is hard, and earning the approval of people whose approval is hard to earn.

As an analogy, publishing on a blog is less satisfying than publishing in Vanity Fair because Vanity Fair is very selective and your blog is hardly selective at all. Getting tens of thousands of people to read your post, though, would probably do the same trick as the VF piece (or similar). Once books are broadcast instantaneously over the Web, and shared in kind among their readers, “hard” and “satisfying” tasks for authors won’t go away—they’ll just take on new forms.

What’s nice is that there are no technical roadblocks in the way. In fact, someone at Google could probably run a single MySQL query to finish the first 90% of the project. (UPDATE books SET full_text = 1;)

(One wonders why there has to be so much scanning. Haven’t authors been using word processors for like thirty years? How about TeX?)