Why are track records unpopular?

Robin points out that people dislike track records across a range of contexts, and poses a puzzle: why?

Naively, track records seem pretty helpful for deciding who to trust and hire. Yet Robin points to a lack of track records or interest in track records for doctors, lawyers, pundits, teachers and academics.

He has also answered his puzzle in a later post, but I’m not going to read his answer until I’ve tried to guess myself (though I did glance at it enough to learn it is about elites and status).

Here are some theories I can think of:

  1. The tracked don’t like it. Most of the effort to avoid track records is from the people who would be tracked. They don’t like it because they are risk averse, and influence is not particularly well correlated with skill. When a large class of people spread throughout society dislikes something and most other people don’t automatically have strong views, most people’s views will end up being negative.
  2. Track records show mixed allegiance. People habitually try to publicize information which supports their own allies. Publishing track records in general is agreeing to publicize a giant pile of information, randomly supporting friends and foes alike. For instance if all records of doctor success become public, this would somewhat endanger any doctors you care about: doctors in your political party and friendship circles and minority doctors and nice doctors so on. That is, unless you had been careful to only side with competent doctors, which you probably haven’t, since you haven’t seen their track records. Similarly, if you read about the track records of pundits, half of them will be pundits you like looking bad or pundits you don’t like looking good, so you will have to surmise that the thing is bunk.
  3. People aren’t quantitative. Customers are not interested in track records because they are not interested in anything quantitative.
  4. Track records conflict with perceived quality. People perceive traits like charisma directly as goodness, and so when explicit indicators don’t align with it, they doubt the explicit indicators.
  5. Track records are hard to integrate. Customers have an overall sense of the worth of particular people, and the explicit measure only tells them one thing. They don’t want to trust the explicit measure on its own because they know it misses important things. But they don’t know what makes up their overall sense, so it’s hard to integrate the explicit track record into it. Similarly, suppose I have a general sense that I want to live in Berkeley instead of Boston, knowing lots of facts. Then I find that people in Boston have 10% better romantic success. I know that I don’t want to move on that basis, and I’m not really sure how to integrate that info into my overall view, except by crossing my fingers and hoping my subconscious took care of it. In this case, I might be disinterested in the romantic success figures, and generally wish people would stop talking about them.
  6. Track records undermine hypocrisy. Customers are looking for characteristics other than those they say they are looking for, and sellers are selling the characteristics that customers want. Both sides have to be vague about the measure of success, because if they explicitly and carefully measured the thing they say they want, it would make it harder to get the thing they really want. For instance, if people really want teachers to be good at indoctrinating their children into some particular culture, they might prefer to use their judgment rather than test scores, since they can’t ask for a cultural indoctrination test, directly.
  7. Publishing track records makes enemies. People who might publish track records know that some substantial fraction of the people being tracked will hate them for it, and on net this is bad for them. When they say that people aren’t interested, they mean ‘not interested enough to justify the backlash’.
  8. Track records bring unwanted responsibility. If customers have to decide which doctor to use, they enter a world where there are good and bad doctors and it is their own choice that determines whether their treatment is good. They prefer to live in the world where they can hand their problems to doctors and not be responsible, because that’s the whole point. So they just stubbornly live in that world. The same goes for other professionals.
  9. Track records for professionals make decisions harder. Perhaps people would like track records if the track record told them exactly what to do, but they just some more information to take into account, and the people already have trouble making decisions about things like who to hire, and so more information just makes them feel worse about not using it.
  10. Track records are socially awkward. Track records lead to embarrassing situations where you have to say ‘I really want to see Dr Phillips, not any of your other doctors’, which is pretty painful compared to some tiny chance of death, so it seems better not to know about them.
  11. People don’t like careful analysis that much. People basically don’t collect information and make informed decisions about almost anything, except sometimes when it is required professionally.

Ok, Robin’s answer is that the elites currently have opinions on who is good, or on heuristics that should be used to judge people, and they see other sources of information on this question as a threat to the influence of their opinions. They respond by trying to undermine the alternative sources, and punish the people responsible for them.

This doesn’t ring particularly true to me, but perhaps I’m misunderstanding. For instance, when I choose a doctor, I feel fairly ignorant about what elite opinion is on which doctor I should choose. I know that a few schools are considered good, though if I choose a doctor who didn’t go to the good schools over one who did, I find it hard to imagine any threat of elite retribution that I might fear. Nobody except my doctor knows which doctor I go to, and even if they did I assume it would usually mean nothing to them—it’s not as if doctors are public figures. And indeed, I don’t feel very worried.

I’ll leave the criticisms of my own theories to the reader. Do you have a preferred explanation?

Dance dance evaluation

One of the best things I came across in 2014 was Dance Dance Revolution (DDR), or specifically Stepmania, a free version you can play on your computer. It is a dance game that looks basically like this (skip the first minute) or this if you are very good and play in arcades.

I’m never sure whether to draw attention to seemingly large improvements to my life: mileages vary, and improvements are often imaginary (especially if excessive imagination makes it onto the list of things to be improved). However DDR solves what I think is a very basic and common problem: the problem of exercise being less addictive than computer games.

For instance, I sometimes exercise by running. A good fraction of my thoughts while running are about whether I can stop running yet, and the other ones are mostly appeals to not do that. Incidentally, ‘can I stop yet?’ is not a thought I have ever had while playing Civ IV, though I have played much more Civ IV than I have run. Conversely, ‘just one more turn?‘ is a rare thought while running.

Some other sources of exercise are more fun for me, but still mostly require an active effort to do instead of not doing. Based on gym membership’s most famous feature being its liability to be forgotten about, and the connection between exercise and willpower in the popular mind, I think I am in the majority when I say exercise does not usually remind me of playing Civ IV. Except in the sense that exercise reminds me to play Civ IV instead.

For me at least, DDR is enough like Civ IV for the analogy to be salient. I frequently get to the end of a song, and just want to play one more. Often even when I am in a state of exhaustion beyond what I could dream of achieving by running. The addictive quality is not nearly as extreme as for some actual computer games: I’m mostly capable of deciding to stop, and usually actively want to at some point. Sometimes I don’t even want to play at all. But often, the strongest temptation is to keep going.

I might just guess that some people find some exercises randomly fun, and this one happens to be fun for me. But DDR is much more like a computer game than most forms of exercise (it even involves a computer!) And computer games are known to be unusually addictive among pastimes. So I doubt that it is by chance that DDR seems so compelling to me. So, if you are looking for an exercise routine that is very easy to want to do, I recommend trying DDR.

DDR also avoids a further problem, which is less universal, but perhaps disproportionately important for people who would most benefit from exercise: that good exercise often requires you to leave your house. Exercise purportedly helps a lot with mental health problems like anxiety and depression. But anxious and depressed people are often particularly unenthusiastic about going to distant new places to do physically difficult things with a bunch of semi-strangers (I claim), which rules out many forms of exercise. And mentally healthy people too often prefer to spend less time and bother on their exercise. Being able to pull a dance mat out from under your bed and start dancing, with the option of stopping and putting it back under your bed at any moment, is a particularly easy and low commitment way to begin exercising.

Even if DDR is not that great an activity for you, an important point is that exercise can be easy to start and hard to stop, even for people to whom almost all exercises seem arduous. So if you think that exercise is fundamentally an exertion of willpower, it may be that you just haven’t tried enough kinds of exercise. I realize this is obvious to many people, but it wasn’t to me, and I would pay a lot to have been informed of it in high school, or to have just been given a DDR pad.


Here is some concrete evidence about how DDR affects my exercise routine:


My exercise Beeminder

The red dots are basically times when I exercised just enough to not fail; then it goes orange, blue, green. As you can see, I used to almost always exercise just enough not to fail, and often less than that. I also often didn’t start the graph up again for a while after failing, though this is influenced by other factors e.g. Beeminder’s recent policy of restarting automatically. I started playing DDR on around the 13th of April 2014, which is shortly before where the graph goes from mostly red to mostly blue and green.


Here is roughly how to play DDR:

Option A:

  1. Buy a dance pad, probably a USB one unless you have some fancier device to play on than your computer. I use some that look like this and they seem decent.
  2. Download Stepmania.
  3. Get some songs. I did this by getting the files from a friend. You can also get them from here (or here) it seems. You probably want the ones labeled ‘Pad (4-panel) — Community Compilations’. I have mostly ‘In the Groove’ ones. Put the folder you download into the ‘Songs’ folder in the ‘Stepmania’ folder you hopefully got by downloading Stepmania.
  4. Plug in the pad. Open Stepmania. Press ‘enter’ a bunch. If anything bad happens, Google it. Step on the arrows. Expect to be very bad at it at first.

Option B (recommended):

  1. Find someone with the things listed in option A set up, and ask them if you can try it out.

Why do all the considerations point the same way?

Suppose you have a lot of reasons to believe a thing, and no good reasons to not believe it. Should this make you more or less likely to believe it, relative to a case where the considerations are a bit more mixed?

At first glance, it appears you should believe it more, since things with lots of good reasons for them and no reasons against them tend to be true.

However often when people have many reasons for a thing and no reasons against it, it is because they have been collecting them, probably unintentionally. Humans seem to do this when they have a belief they care about.

For instance, when I was younger I could have told you fifteen reasons that logging old growth forests in Tasmania was harmful. They were more economically valuable as a tourism attraction, and the logging was perpetuating corruption, and the forests harbored endangered species, and so on. Somehow, coincidentally, at least almost all the considerations aligned.

For another instance, vegetarians often think that vegetarianism is very easy, and healthy, and moral, and good for the environment, and more enjoyable. Meat eaters often think that vegetarianism is inconvenient, and unhealthy, and not morally important, and worse for the environment in certain ways, and unpleasant. It’s even less likely that all the considerations align, but in random different directions depending on who does the set of unbiased analyses. I think here both groups are aware of some people on the other side doing this.

This seems common enough that if you find yourself with a collection of considerations all pointing in one direction, you should be somewhat worried.

On the other hand, often you have lots of aligned reasons because there is some fundamental reason behind them all. For instance, if you can’t prove a statement in math, because every different way you can think of to try to prove it fails, this may be because it is false. Here you expect to get entirely evidence pointing in one direction.

A less clear case is whether exercise is good. It seems there are lots of different good reasons to exercise. But all of them go through you being healthy, so this is not so surprising — you will look better, you will feel better, you will live for longer, you will be more sane and happy. 

Some situations are more conducive to evidence all pointing one way, or coming out in different directions. If the question is whether something is on net good, and it has a variety of effects, probably some should point in different directions. If the ‘considerations’ are a number of somewhat noisy measurements of a hidden quantity, then if the first measurement is high, probably the others will be too.

In whatever situation, you should also expect some things to come out with all of the evidence pointing in one direction, by chance. You might also expect all of the considerations to come out one way due to a selection effect in combination with chance. For instance, if you thought of a particular example because it was the case which you think is most overwhelmingly lopsided, then it was selected for being lopsided, and so this is less surprising than if another random belief had this character.

I think if you find yourself with lots of aligned beliefs like this, you should consider asking yourself:

  1. Is there some root consideration pushing all the other consideration one way?
  2. Am I motivated to believe this thing?
  3. Is this a kind of situation where I should expect all of the evidence to point in one direction?

Socially optimal weirdness

I wrote recently about considerations in choosing how weird to be. Today let us consider the question from an impersonal perspective: what is the socially optimal allocation of weirdness? Society and weirdness are complicated, so again let us just discuss some considerations.

Social costs of people being judged badly

When individuals avoid being weird, it is often because they want to be judged well in some way. From an impersonal perspective, does it matter if you judge me badly? This seems to depend on the extent to which people judge one another absolutely, versus relatively, and whether people care about the judgement absolutely or relatively.

If you judge me as a relatively bad friend, and then you replace me with a different friend, this seems bad for me, but good for your other friend, so socially neutral. If you judge me as an absolutely bad friend, this might hurt me without providing a compensating benefit to someone else. It will hurt me more if  I care about my absolute quality as a friend than if I’m mostly worried about being at least as good as most people. It seems to me that a combination of these things happens in practice. So the private costs of being judged for weirdness partly translate to social costs.

It also matters how much you care about making judgments in a particular way (e.g. correctly). If you actually don’t want to interact with people with the wrong political beliefs for instance, then if I hide my political beliefs and we become friends this will be bad for you. If you merely don’t want to have awkward political discussions, then it is fine if I hide my beliefs.

Signaling race

In some cases, ‘not weird’ is continually and narrowly redefined, to make locating it a reasonable sign of social savvy. For instance, if you are a girl in high school, you might learn that it is weird to not own any barbie dolls. However once you manage to get a barbie doll, you may find that it is the wrong barbie doll, or that barbie dolls are no longer the normal thing any more and are now the preserve of weird kids like you. This race presumably takes some amount of effort from the weird and the non-weird people alike, which would be averted if people didn’t try to avoid weirdness.

Neutral views

Suppose everyone chooses one topic on which to spend their weirdness budget, and there they think deeply and advocate hard for what they think is right. On all the other topics, they take the most common position. Then virtually every view on every topic will be directed by conformity, and it won’t matter that each person put thought and effort into their own cause. The status quo will reign forever, on almost every issue. If everyone has many more implicit votes than they have weirdness to fund them with, then public opinion is almost completely uninformative. Thus in such a case it seems probably better overall for people to be at least weird enough that public opinion is informed by thought. This can happen for instance if people express a lot more minority views, or if there are multiple non-weird views on every issue.

Economies of scale and congestion

It is good for efficient consumption if people aren’t weird with respect to tastes in information goods like music and TV. For instance, people who don’t enjoy Game of Thrones are just going to miss out on what could have been basically free pleasure. For goods where one person using them means another person cannot, there is more of a trade-off. There are still often economies of scale, so others sharing your tastes makes it easier for you to get what you want (e.g. it is very easy to get Coca-Cola relative to rice milk, which is not because rice is hard to grow). However other people can also get in your way and buy up the things you want, so it can be better for people to be more weird for some tastes. For instance, it’s better if people have different favorite mountains to climb, if everyone likes to climb in peace.


There are often costs from people using different standards. For instance, when I took the GRE I suffered a cost from having learned to type using the Dvorak keyboard layout, because the GRE computers can only use Qwerty. I and a bunch of French people also suffered costs when I went to France and sat in their train seats and we couldn’t talk about it, and when they closed their restaurants for meal times, unexpectedly.


Weirdnesses offer variety, which has various benefits. Some people like it for its own sake. It also naturally allows experimentation, which enriches the lives of the non-weird later. For instance, it seems good that some people want to entirely live on synthetic nutrient slurries, because eventually they might find some that are delicious and well tried enough that it becomes a common lifestyle choice.

Variety also produces robustness. That some people like to live in the countryside means everyone can’t be killed by an epidemic so easily. That some people keep a thousand cans of beans in their basement makes society even safer.


Honesty about weirdness is useful people who contribute to policy to learn information about people’s values. For instance, if almost everyone who was homosexual decided that it wasn’t an optimal place to seem unusual, and avoided mentioning it ever, then nobody would ever have known how important improving the treatment of homosexuals was.


In sum, from society’s perspective, it seems pretty unclear how weird it is best for people to be. Several considerations point in different directions. Incidentally, it also seems very unlikely to align with how weird individuals want to be.

How to pay for lives to be worth living

Slate Star Codex writes about a patient (or patient amalgam) who was suicidal, apparently for want of a few thousand dollars:

…So what bothered me is that psychiatric hospitalization costs about $1,000 a day. Average length of stay for a guy like him might be three to five days. So we were spending $5,000 on his psychiatric hospitalization, which was USELESS, so that we could send him out and he could attempt suicide again…

…Problem is, you don’t have to be an economics PhD to realize that “give $5,000 to anyone who attempts suicide and says they need it” might create some bad incentives.

I have no good solution to this…

I’m curious about solutions to this. However I’m going to talk about a slightly different situation, where the person in question is driven by desperation to be in a drug experiment which will make the rest of their life of neutral value. The drug, Neutrazine, has no social value, and is being trialed for entirely morally neutral reasons.

So we want to be able to give people a few thousand dollars at times when their not taking Neutrazine is worth more than a few thousand dollars to us, and where a few thousand dollars would be enough to keep them away from Neutrazine, without causing them to get into such situations more readily, or to lie to you about whether they are really so badly off that they would take Neutrazine.

This sounds kind of hopeless: if you are willing to rescue people in a bad situations, and they know this ahead of time, surely some people will get into bad situations more and/or lie about it.

Lets start with just the problem of people taking more risks, knowing that you will save them. That is, suppose they will honestly report their values.

This actually seems like a case where moral hazard should be avoidable. The person in question has the option to make their life worth nothing using Neutrazine, from any initial level of value. This is worth a positive amount to them, in the cases where you are hoping to help them. But if you give them just the same positive amount in money, this also makes their life neutral and takes the Neutrazine option off the table (because it would do nothing). So it doesn’t change the expected value from their perspective at all, and thus doesn’t influence their decisions ahead of time. Yet it is much better from your perspective, because you valued their life a lot more than a few thousand dollars.

This might seem unsatisfactory, in that you got all the gains. However you could give them some gains, without influencing their behavior much. Also, there may  be gains to their future self that were discounted more than you would like. And it might be that a person joining a Neutrazine trial will tend to be underestimating their future opportunities (due to the selection effect), so it is better for their life to be neutral according to their expectations than guaranteed to be neutral, on average.

This isn’t a solution, because it requires you to know how valuable things are to the other person. As mentioned earlier, they can just tell you their life is worse than it is. People whose lives are not bad at all can claim they are going on Neutrazine. Partial solutions to this could come from mechanism design, neuroimaging or lie detection. I’ll talk about the mechanism design option.

We have a collection of people whose lives have varying degrees of value to them. We would like to distinguish them, but they all look the same. One obvious difference is their willingness to join a Neutrazine trial. Once we have an action like this, that people with worse lives are more willing to take, we can use it to construct a choice that people will make differently, and which will also differentially help those who need it.

Here is an imperfect one: offer a bundle of $1,000 and a %10 chance of joining a Neutrazine trial. This is of negative value for people whose lives have more than $9,000 of value to them, and positive for those whose lives are worse than that. This isn’t great, in that you help some people who are less desperate, and you can only help people a small amount, but it seems better than the apparent status quo.

Can you design something better?