Misunderstandings of not understanding

There are probably lots of attitudes to a thing we might call ‘understanding’ it. Here are examples of two:

“I understand combustion”

“I understand her feelings”

In the first statement, the person has a good explicit model of combustion, and the range of ways it can happen, and what causes it, and that sort of thing. In the second, her feelings feel natural and comprehensible to the speaker. T might have an intuitive sense for how her feelings will evolve, but they probably don’t have an explicit model of the psychology, biology or game theoretic reasons behind her feelings.

These different instances of ‘understanding’ probably vary on a few different dimensions that could be separated, but this is going to be a short blog post.

One thing that separates these kinds of understanding is their objects. People rarely say “I understand combustion” to mean that it feels intuitively natural to them, in the absence of any explicit model. And they rarely say “I understand her feelings” to mean that can answer questions about her alien neural chemistry.

There are some things where both kinds of understanding sometimes apply though. For instance “I don’t understand art” can mean “when I look at art, it doesn’t feel especially natural or good to me” or it can mean “I do not know what caused humans to like art so much”. The seconds speaker may love art.

I think failure to distinguish these kinds of understanding causes misunderstanding. For instance, suppose you say you don’t understand why people go out to bars. This could mean you don’t enjoy doing so personally, or that you don’t see why bars in particular evolved to be a big thing while public swimming pools full of tiny balls did not. If the two aren’t really distinguished in people’s minds, then they will suppose you are saying both, even once your elaboration strongly implies the latter.

Perhaps this is a reasonable shorthand, because in practice the only people who wonder why humans gather at bars in particular are those who aren’t enjoying bars enough. But I think it confuses things (for instance, impressions of how fun I am).

A particularly costly kind of misunderstanding is when people themselves assume they are saying both, because they think there is just one thing. Then for instance they notice that they don’t know why art exists, and don’t think to independently check whether they like art. Or they don’t understand why sports is in the curriculum, so they assume that they don’t enjoy it. Or ‘tradition’ seems an incomprehensible justification for a behavior, so they don’t pay attention to whether they personally get anything out of traditions.

I don’t really know how much any of this is a thing—I feel like I have seen quite a few examples of it, but I don’t remember many. What do you think?

(Related: Meta-error: I like therefore I am)

Mistakes I’ve made part 3: Poor sacrificial accounting

There was a time when I routinely refused car travel, in favor of my more sustainable bicycle. Not always, but I had a high bar—if it was bucketing down with rain and I had no plastic pants, this was not sufficient excuse for instance. I would sometimes decline a ride even when others were purportedly driving somewhere anyway, to avoid encouraging them to be ‘driving anyway’ more often. I did enjoy cycling, on a good day, but many days weren’t good. People at school laughed at me on my bike, so sometimes I walked instead because I couldn’t stand them looking at me. Refusing cars was often a sacrifice.

My concern was the climate. My basic model of ‘sustainability’ was this: humanity has a bunch of resources, and when they run out we all die. If we use them slow enough, they can mostly last forever, but if we use them faster, we will die soon. Much like a bank account that earns lots of interest, or can be spent down in an afternoon. Space in the atmosphere for greenhouse gasses was a key resource. Driving was known to be unsustainable: it added carbon dioxide to the atmosphere faster than could be supported. This model has a few problems I think, but it is debatable, and this blog post is not about them. So lets suppose for now that I was right, and every car trip took humanity a notch closer to annihilation.

Given that driving was unsustainable, I wanted people to stop driving, so we wouldn’t all die. Naturally, it followed that I should not drive (I thought). If the cost of a billion people driving was too great for the benefits of a billion people driving, then the cost of one of those people driving was probably too great for the benefit of that person driving. Furthermore, I seemed a particularly easy person to convince, from my perspective. Furthermore, the benefit of me getting to school sooner and with dry pants was obviously unimaginably minuscule compared to any decrement in humanity’s ability to survive in the long term.

This is all very wrong. Why? Let’s say cycling cost me ten minutes per day (it took more like half an hour per day, but driving also takes time, and cycling had some benefits, such as exercise). If I had spent this 5h per month on writing about the problem, I think I could have convinced more than one person ever to habitually ride a bike. If I had just earned money and given it to an organization specializing in this, they probably could have done better. This is an empirical question, so maybe not, but the problem was that I didn’t even look for such things.

One way to think about this is to say that I was wrong in thinking myself an especially cheap person to change. I might be easy to communicate with and readily convinced (from my perspective). I might be able to get on a bike right now, whereas I had not much idea how to convince anyone else. But I am an extremely expensive person to use the time of (from my perspective), because I am one of the few people who is on board with my schemes and willing to do what I tell myself, and so my own time is one of the few resources that can be spent on the plan of forwarding my values. And given this, there are ways to leverage ten minutes of my time into more than an extra bike ride’s worth of value.

Relatedly, I didn’t compare between options further afield. Even if unsustainable transport was set to destroy the world, and personal action was the cheapest way to get change, it does not follow that I should cycle. For instance, maybe it is even more important to change people’s unsustainable purchasing behavior, and I should have spent that time purchasing things better. Or earning money, and spending it on forwarding sustainability in other ways. Or thinking about how robots might destroy the world more imminently.

More fundamentally, my error was comparing the costs of my sacrifice to what was gained by it, instead of comparing my gains to what I might have had instead. This was both the wrong comparison, and naturally blinded me to the better purchases I might have made. If the question you ask is whether cycling everywhere is worth a tiny fraction of the future, then there is no reason to check how valuable reading about geoengineering is.

So this is what I would tell my younger self. Whether your personal sacrifices are money, time or frustration, you should usually be comparing the benefits of making one personal sacrifice to the benefits of making another. You should not be comparing the personal cost of making a personal sacrifice to the benefits of making that sacrifice. Because what matters is ‘opportunity cost’—the value of the next best option that you are foregoing. And rarely is the next best option to ramp up your leisure budget. Usually the next best option is making a sacrifice for something else overwhelmingly valuable; something measured in units of humanity’s future, or in senseless torment averted. This is true whether you are willing to sacrifice everything for the greater good, or only a tiny bit. Either way, you should spend the sacrifices you make on the best things sacrifices can buy.

Mistakes I’ve made, part 2: Father Christmas

I believed in Father Christmas for longer than I believed in God. This is mostly surprising because it is unusual. There was both more evidence for Father Christmas, and more motivation to propel cognition.

I stopped ‘believing’ in Father Christmas in grade seven, when my drama teacher cavalierly mentioned his unreality. It wasn’t shocking—more a feeling of something I knew somewhere in my heart being made official. It was a bit embarrassing.

How could I neglect to explicitly infer the lack of a Father Christmas until then?

I don’t think my epistemic failures here were embarrassing for a human, given the proliferation of supernatural objects of belief, and the fact that Father Christmas alone among them actually leaves concrete stuff that you wish for.

My guess is that most people do better, not by noticing their confusion, but by interacting more with other children. Noticing confusion would also alert people to the implausibility of other supernatural things (unless perhaps it is the very evidence for Father Christmas which makes him confusing). And contact with subject matter experts is known to be a good prophylactic against being wrong.

But what if I wanted to instruct my past self on how to independently notice that Father Christmas was implausible? What should have been clues, to someone fairly ignorant about the world?

Father Christmas purportedly travels very fast, and delivers more presents than might fit in a sleigh. Also his sleigh can fly. From a child’s perspective though, many things travel very fast. Especially ones that fly. Not usually ones that are propelled by reindeer, but it wouldn’t be shocking if the reindeer were not the main engine, or if the illustrations portrayed the vehicle somewhat whimsically. Having a lot of presents could also be explained by being able to move things fast. Father Christmas himself would also have to move very fast, compared to a human. Though it also shouldn’t be surprising if much of his effort were outsourced; we also say that Steve Jobs made the iPhone. Nonetheless, one could notice that these speeds were suspiciously high.

This would require some understanding of contemporary technology, since such speeds are not obviously inconceivable for any technology. This is complicated by the world being big and varied. For instance, these days I sometimes hear that people elsewhere in the world are doing things well beyond the technological capability of people and companies I usually interact with. For instance, creating exotic physical particles or going to Mars. Yet I don’t usually suspect it at all. Should I? For better or worse, I believe other groups have technology that is much better than that in commercial use, and am not very surprised by it.

Perhaps I should have noticed that there was a more plausible explanation? The real explanation, as it turns out, is that there is a global conspiracy by one demographic group against another. The ‘very large scale conspiracy’ hypothesis is perhaps best known for its role as a reductio against theories that requires its truth. So I’m not sure that this alternative should have jumped out at me. Maybe I should give such theories more credence, given Father Christmas. My guess is mostly not, but that global conspiracies can grow from smaller conspiracies which become tradition and then spread.

Father Christmas has other surprising purported characteristics, such as an abundant appetite and access to all buildings. But presumably he doesn’t eat all the cookies at the time, and we used to leave the door open for him.

I’m not actually convinced that I should have done better, as an ignorant child independently assessing Father Christmas. Had I looked into the technology purportedly being used, then I should have done better. However I’m not sure Father Christmas should have stood out as the most worthy target of deeper investigation, even if I had been sensible enough to investigate the most worthy targets of deeper investigation. So, if I made mistakes, probably I’m still making them! Do you have better hints for cheaply being appropriately confused?

Mistakes I’ve made, part 1: greedy altruism

I have made many mistakes. Unfortunately, for some of my older, larger mistakes, it becomes hard to remember or imagine why I would have made them. The alternative position comes to seem literally inconceivable. At the same time, I forget that I ever conceived of it. This is tragic, because for anything realized some way into my life, there are surely many people who don’t (yet?) share my view. Also because any insight into why I might have believed inconceivable things in the past might prevent me from believing such in the future.

So it seems a valuable exercise to recall and dissect some of my errors before they evaporate from my memory. Here’s one:

For almost all of my teenage years, I believed that small amounts of money could be used to save lives in the developing world (an error, but one for another time), but consequently collected small amounts of money where possible, rather than optimizing for long-run ability to earn money (which would also not necessarily be the best thing, but is obviously superior to greedily earning small sums). e.g. I would do chores for money instead of practicing useful skills.


One obvious possibility is that it didn’t occur to me. That would seem surprising, and I don’t think it’s true, but it might be. Also something intermediate seems plausible—like, I was not fully and abstractly aware of a trade-off between greedily accruing small amounts of money and investing in better opportunities, though it did occur to me that I could practice math instead of doing a chore and that this might help.

If we suppose I was at least somewhat aware of this option, I think I probably didn’t see how any alternative activities would genuinely improve my prospects. Especially at the five minute level. If I didn’t do this chore, it didn’t seem like I would really find something else to do, right here and now, that would cause me to earn twenty cents in the long run.

I suspect one mistake was failing to add these five minutes up, and say that if I always didn’t do chores, I would have a fair bit more time, and even if the first twenty increments weren’t that great, given the options in my house, it should be possible to look further afield and find ways to make a difference.

In my past self’s defense, I can’t remember what I actually did with my free time, and I doubt it was that sensible, so I may have been right that the isolated intervention of buying more time for myself was not great. In my past self’s further defense, I think my ‘free’ time was often poorly spent because I had to take care of my younger brothers most of the time, which was pretty distracting. However I don’t think any of this is sufficient defense at all.

Another error might have been failing to have a model of myself as something that could accrue any kind of improvements. I think I saw most of my value as residing in my intelligence, which I supposed was fixed. I was fairly oblivious of the differences between me and e.g. a competent middle-aged person. However it’s hard to see how this could be so, in conjunction with having high hopes for the future, and achieving little at the time. I saw living at home as a huge impediment to anything, which would naturally end at some point. I think I may also have implicitly supposed that becoming an adult would actually bring a host of improvements, such as knowing how to usefully improve the world. I doubt I checked whether these beliefs were quantitatively plausible and consistent.

I suspect I also focussed too much on the comparison of someone’s life and me making effort now—I can save a bit of a life by doing this annoying thing; what could be better than that?!—instead of focusing on the comparison between this way of saving a bit of a life and other ways. This is an issue of psychological focus rather than a wrong belief. It seems like an especially easy mistake to make if other people fail to notice the comparison between your effort and other people’s lives, and you are kind of indignant about it.

I think relatedly, I believed that earning some money would really help, whereas everything else felt nebulous. Could you really let a person die, so you could do something more fun that might help at some point on some messy, half-thought out model? It perhaps felt easy to cheat.

How could I avoid such problems, if I were talking to my teenage self, or hoping to not make analogous errors now? Some ideas, more welcome:

  1. Explicitly state my models of how my actions are reasonable, which would hopefully reveal gaps and incoherencies
  2. Describe my explicit models to other people, who are hopefully more in touch with common sense (e.g. you can often learn things, which will affect your earnings)
  3. Ask what will happen if I follow my current behavior forever, and/or when I intend to actually stop it.
  4. If I’m unreasonable in one way, try to fix that more before giving up and directing resources away from myself.
  5. Note that I am a physical system, not a disembodied spirit of pure intellect.
  6. Try to be clear on counterfactuals, rather than what is being directly traded. e.g. if you give money to charity A, the counterfactual is often giving to charity B, rather than making less effort. So the comparison between your effort and charity A is irrelevant.
  7. Adopt growth mindset

Why are track records unpopular?

Robin points out that people dislike track records across a range of contexts, and poses a puzzle: why?

Naively, track records seem pretty helpful for deciding who to trust and hire. Yet Robin points to a lack of track records or interest in track records for doctors, lawyers, pundits, teachers and academics.

He has also answered his puzzle in a later post, but I’m not going to read his answer until I’ve tried to guess myself (though I did glance at it enough to learn it is about elites and status).

Here are some theories I can think of:

  1. The tracked don’t like it. Most of the effort to avoid track records is from the people who would be tracked. They don’t like it because they are risk averse, and influence is not particularly well correlated with skill. When a large class of people spread throughout society dislikes something and most other people don’t automatically have strong views, most people’s views will end up being negative.
  2. Track records show mixed allegiance. People habitually try to publicize information which supports their own allies. Publishing track records in general is agreeing to publicize a giant pile of information, randomly supporting friends and foes alike. For instance if all records of doctor success become public, this would somewhat endanger any doctors you care about: doctors in your political party and friendship circles and minority doctors and nice doctors so on. That is, unless you had been careful to only side with competent doctors, which you probably haven’t, since you haven’t seen their track records. Similarly, if you read about the track records of pundits, half of them will be pundits you like looking bad or pundits you don’t like looking good, so you will have to surmise that the thing is bunk.
  3. People aren’t quantitative. Customers are not interested in track records because they are not interested in anything quantitative.
  4. Track records conflict with perceived quality. People perceive traits like charisma directly as goodness, and so when explicit indicators don’t align with it, they doubt the explicit indicators.
  5. Track records are hard to integrate. Customers have an overall sense of the worth of particular people, and the explicit measure only tells them one thing. They don’t want to trust the explicit measure on its own because they know it misses important things. But they don’t know what makes up their overall sense, so it’s hard to integrate the explicit track record into it. Similarly, suppose I have a general sense that I want to live in Berkeley instead of Boston, knowing lots of facts. Then I find that people in Boston have 10% better romantic success. I know that I don’t want to move on that basis, and I’m not really sure how to integrate that info into my overall view, except by crossing my fingers and hoping my subconscious took care of it. In this case, I might be disinterested in the romantic success figures, and generally wish people would stop talking about them.
  6. Track records undermine hypocrisy. Customers are looking for characteristics other than those they say they are looking for, and sellers are selling the characteristics that customers want. Both sides have to be vague about the measure of success, because if they explicitly and carefully measured the thing they say they want, it would make it harder to get the thing they really want. For instance, if people really want teachers to be good at indoctrinating their children into some particular culture, they might prefer to use their judgment rather than test scores, since they can’t ask for a cultural indoctrination test, directly.
  7. Publishing track records makes enemies. People who might publish track records know that some substantial fraction of the people being tracked will hate them for it, and on net this is bad for them. When they say that people aren’t interested, they mean ‘not interested enough to justify the backlash’.
  8. Track records bring unwanted responsibility. If customers have to decide which doctor to use, they enter a world where there are good and bad doctors and it is their own choice that determines whether their treatment is good. They prefer to live in the world where they can hand their problems to doctors and not be responsible, because that’s the whole point. So they just stubbornly live in that world. The same goes for other professionals.
  9. Track records for professionals make decisions harder. Perhaps people would like track records if the track record told them exactly what to do, but they just some more information to take into account, and the people already have trouble making decisions about things like who to hire, and so more information just makes them feel worse about not using it.
  10. Track records are socially awkward. Track records lead to embarrassing situations where you have to say ‘I really want to see Dr Phillips, not any of your other doctors’, which is pretty painful compared to some tiny chance of death, so it seems better not to know about them.
  11. People don’t like careful analysis that much. People basically don’t collect information and make informed decisions about almost anything, except sometimes when it is required professionally.

Ok, Robin’s answer is that the elites currently have opinions on who is good, or on heuristics that should be used to judge people, and they see other sources of information on this question as a threat to the influence of their opinions. They respond by trying to undermine the alternative sources, and punish the people responsible for them.

This doesn’t ring particularly true to me, but perhaps I’m misunderstanding. For instance, when I choose a doctor, I feel fairly ignorant about what elite opinion is on which doctor I should choose. I know that a few schools are considered good, though if I choose a doctor who didn’t go to the good schools over one who did, I find it hard to imagine any threat of elite retribution that I might fear. Nobody except my doctor knows which doctor I go to, and even if they did I assume it would usually mean nothing to them—it’s not as if doctors are public figures. And indeed, I don’t feel very worried.

I’ll leave the criticisms of my own theories to the reader. Do you have a preferred explanation?