Reverse lotteries with friends

reverse lottery pays out a little bit each time you pay but sometimes lead to horrific disaster. For instance, going without a seatbelt. You enjoy a momentary convenience every time you drive, but occasionally you die. Relatedly, wearing your seatbelt can be seen as a lottery: you pay a tiny inconvenience each time for an occasional huge win.

Like all lotteries, whether it is good to play depends on the payoffs, and one might reasonably decide to play some lotteries and reverse lotteries and not others. However, as Scott points out, it can be tempting to play reverse lotteries too much. I think this happens in particular from learning what is good by experience. If you play a reverse lottery once, probably you get a reward, and want to do it again. So you do, and get another reward, and it starts to seem like a pretty good idea. You get a lot of visceral feedback about the good aspect, and none about the bad. At least for a while. This seems like a real problem, and a neat way of thinking about it.

So presumably normal lotteries should be the opposite. You play them a few times, and it is a bit bad each time. So you quickly give up and never see the glorious reward. This doesn’t seem true of the literal lotteries in which people gamble for fun. At least plenty of people are not put off for a very long time, in spite of never winning. But maybe those are a weird instance of the abstract lottery class—for instance, because the prospect of winning a lot of money is made very salient. You might imagine that the negative lotteries would be very off putting in the analogous case: if every time you don’t wear your seatbelt, you hear about another person dying from that very choice, you wouldn’t be so tempted by the no-seatbelt reverse lottery.

I’m still confused about how individuals feel about lotteries, because I’m failing at thinking of clear examples where I know how people behave and they don’t have a really salient message about how the thing might go well. Possibly this is just because nobody does things that go badly almost all the time. Which is as good a segue as any into the thing I actually wanted to think about.

How do groups feel about lotteries and reverse lotteries?

Groups often learn whether a thing is good by one person trying it, and telling their friends, who then try it if the message was favorable, and so on. On this model, we might expect groups to do the worst kind of failing to think about the low chance outcomes.

For instance, suppose that a bakery sells reverse lottery cakes. They taste nice, but occasionally they make you sick for a day. Alice tries them, and likes them, and tells her friends. They like them. Soon lots of people are loving the reverse lottery cakes, and saying nice things about them. Eventually Zoe gets a badly upset stomach.

This might go ok for group epistemology—maybe Zoe tells her friends, and they radically lower their opinions of the cakes upon hearing how much Zoe hates them (probably too much, since Zoe is one of their few friends), and not only cease eating them themselves, but also warn their other friends, and maybe eventually everyone becomes accurately aware of the costs (I haven’t checked under what circumstances if any you end up at a good equilibrium).

There are many ways this might not happen though. For instance, if Zoe is less of an enthusiastic proponent of not eating reverse-lottery cakes than her friends are of eating them. Or if people just count up how many people around them like a thing—e.g. “of all my friends, only one doesn’t like reverse-lottery cakes, so probably I will like them”. Or if people disproportionately trust a large number of agreeing friends more than an outlier. These seem more plausible to me than the opposite alternatives, where Zoe becomes a disproportionately fierce critic of the reverse lottery bakery, or where other people hear that there is someone out there—a friend of a friend of a friend—who really didn’t like the reverse lottery cakes and update too much on this.

There are some important ways these opposite things happen though. For instance, large negatives are more newsworthy than small positives, so for things where the downsides are worthy of a story, I’d expect there to be some correction, though I’m not sure whether it should over or undershoot in general. For instance, people are famously over-concerned about sharks. If they just got their information from swimming and hearing about swimming trips from their friends, I might expect them to be under-concerned. However shark attacks make for fairly compelling reading.

So the kinds of situations that I expect crowds to over-invest in reverse lotteries are those where the costs aren’t really huge or fascinating. Or where it is hard to trace the effect to the (partial) cause. An example off the top of my head is going to theme parks. Most people I know like going to theme parks, as far as I can tell. Two people I know strongly dislike it, because they were injured at theme parks in the past. Without looking at the statistics, it is tempting (for me at least, intuitively) to say, ‘Well, basically everyone thinks it’s good. Maybe those two people were doing something weird. It’s probably fine’.


I was thinking about this because I was writing about a principle of being a good person that I feel emotionally compelled by. I wondered about the causal history of my liking it. I figured I should know about that, so that I could have a more consequentialist view on whether it was actually good. And I realized that maybe I got excited about it after a handful of times where it seemed very useful for making good decisions or being in a good mental state. But I can see how it might go wrong one time in a thousand, and when it does, how it might perhaps sometimes causes some sort of humanitarian crisis or something. If I go around praising this principle based on my own feelings after buying a handful of reverse lottery tickets, then other people might buy more of them, even if it was never a good idea in expectation, and even if I could have predicted that. So I figured I should rethink my principle (I haven’t yet), and not go around praising things that I admire just because I feel like it.


Two kinds of responses

Suppose that you are listening to music, and you reach a song that makes you sad. How do you respond? Here are two ideas:

A) Be sad. Perhaps think about bittersweet memories. Stare into space. Get completely sidetracked and cry a bit. Downgrade your assessment of how good your life is overall. Until some happy music comes on, at least.

B) Decide if you want to be sad, and adjust the playlist accordingly. Maybe you determine that your goals would be better served by listening to patriotic music on this occasion. So you change it. Even though the sad music is now making you feel like listening to sad music.

It seems to me that these correspond to two natural classes of things people are doing when they ‘respond’ to stuff.

The first kind of response is a relatively automatic reaction to a stimulus. Feeling sad when you hear sad music. Finding a clever retort if someone is rude to you.

The second kind of response is a continued pursuit of your goals, adjusted for any information contained in the stimulus. Turning off sad music if it doesn’t seem helpful. Walking away from a rude person if talking to them is not creating a lot of value, while perhaps considering whether their criticism is relevant to you.

I’m going to call these reflexive responses and agentic responses.

More examples of stimuli and reflexive and agentic responses to them:

  • Being cold: curl up under your meager blanket and shiver vigorously // get out of bed, cross the very cold room, turn on the heater.
  • Hearing a joke: Laugh proportional to the humor of the joke, adjusted for offensiveness and attractiveness of the joke teller // laugh if you think it would be good to raise the status of the person telling you the joke, or to make them happy, or to make the situation less awkward, etc.
  • Seeing misinformation on a blog: write a comment correcting the error // add ‘online misinformation’ to your mental list of problems in the world, and then if you decide that it is the most important one at some point, seriously scheme about what to do there.
  • Someone starts a conversation with you: say the natural next thing at every juncture // decide if there is something you want to achieve by talking to the person, and then steer the conversation appropriately (perhaps toward winding up)
  • Physically suffering: curl up in a ball, close your eyes and whimper // search for painkillers, make a doctor’s appointment then leave your house and go to it
  • Emotionally suffering: avoid thinking about the topic, cry, go over the source of distress in your head, tell other people that you are suffering // try to figure out why you are suffering, and then stop it, even if that involves some amount of thinking about unpleasant things and having uncomfortable conversations.
  • Being employed: do the things you are meant to do at jobs, perhaps hinted at by the instructions // choose the bits of the job that are relevant to your goals and emphasize those to the extent that makes sense within the bounds of not jeopardizing your job or failing at your promises.
  • Being called on to give a speech: say the things that are meant to go in speeches // say the things you want the audience to hear
  • Your partner being rude to you in front of your friends: disrespect them aggressively right back // infer that your partner may not respect you enough or doesn’t understand social norms or made an error, and make a mental note to figure out which and address the problem later. Decide whether it is valuable to save face in front of present company, and aggressively disrespect them right back, or be nice, or whatever, as appropriate.

For a more real example, at the time of writing most of this, I was in much pain, and was lying in bed thinking something like ‘Pain! Why pain? Why me? Ow. Pain! Pain pain pain’. Then I thought that perhaps all the thinking about pain was worse than the actual pain, and that even though noticing that I’m in pain every two seconds comes pretty naturally to me when I’m in pain, there are probably actions that better achieve my goals, if I can do them. So I decided to write a blog post instead, which to my surprised actually worked, at least for a bit. [Added upon coming back to this draft: my ability to focus on things other than being in pain didn’t last terribly long if I recall, but it was good for a bit.]

I claim that it is helpful to distinguish reflexive and agentic responses.

It seems that a key pattern in how humans interact with the world is that they notice events, and then feel the need to respond to them. Even to a person who is otherwise pretty consequentialist, it somehow seems very natural to say a thing right now for no apparent reason, except that some random person said a different thing in your vicinity, and the words you are saying are a semantically and socially natural response to the words that they said. You put your actual projects on hold, because you have to respond.

Furthermore I think when we respond, we usually do it in the reflexive style. Which is natural: there is arguably a lot of responding to be done, and we can’t think about all of it. But it is nice to remember sometimes that there is also the option of responding agentically. Responding agentically takes mental and perhaps other effort, but that aside will tend to be better (by definition).

I’ll say more about these classes of behavior as I see them, partly in the hope that this helps with drawing them so that other people know what I’m talking about.

You might think of reflexive responses as sort of based on feelings and agentic responses as sort of based on explicit thought. I think that isn’t right—in particular, there are a lot of reflexive responses that also mostly involve explicit thought. It’s just that the explicit thought isn’t about how to achieve your goals. For instance, if a person argues with you, you might have a reflexive response of constructing a counterargument and then sending it to the person. Or if you are watching the news and there is a surprising event, you might have a reflexive response of thinking about its implications, and remarking upon them verbally to your companion.

Some reflexive responses were designed by an agentic response previously. For instance, if you noticed before that you should just never listen to a particular song because it will ruin your day, then if it comes on you might skip past it near automatically.

It is often hard to not do a reflexive response. For instance, if you are angry, it can be hard (and arguably destructive) not to express it, perhaps without careful regard for the social consequences. This seems like a fine reason to react reflexively often. I still think that observing the existence of a decent alternative reaction often makes the reflexive response less naturally appealing.

Agentic responses very often involve not doing anything. Because it’s not that common that an event in your vicinity substantially alters what is the best thing for you to do next. More common than by chance, because things in your vicinity are much more relevant to you than other things. But still not that common.

I’m rolling a lot of different kinds of behavior into reflexive responses. Intuitive completion of patterns you are part of, fulfillment of roles, expression of the feelings that the stimulus makes you feel, following immediate incentive gradients, doing what feels right, fulfilling instincts.

Reflexive responses are often good because they are cheap and predictable. It is cheaper to finish a pattern or to fulfill the role than to rethink your whole plan in light of new evidence. And if people usually respond to X with Y, this perhaps makes them easier to interact with. Arguably, you can just live a whole life of one reflexive response after another, and then you don’t have to have goals at all, which potentially represents a real saving (at the cost of everything you might have wanted, if you had wanted anything).

Reflexive responses are associated with getting stuck in local optima. For instance, shivering in bed, which is less cold than getting out of bed, but more cold than turning on the heater and waiting five minutes. My guess is that they are also more associated with pathological large scale social interaction traps, such as the toxoplasma of rage. (While one might reasonably decide that reacting with outrage to something on the internet is the best way at hand of forwarding one’s goals, this has got to be a lot rarer than reacting out of anger is).

I get some value out of having these concepts. My friend has argued that they are part of a fundamentally wrong worldview, and I think he’s partly right (a discussion for another time), but I still think they are a good enough approximation of an important thing, and his better worldview doesn’t seem to naturally support a similar distinction.

Related: systems and stories, deontology and utilitarianism, reflex agents and goal based agents, thinking outside the box vs. thinking inside the box.

On using computers as brains and brains as computers

I visited the DC area recently and recorded a couple more podcasts with Robin Hanson while there:

Talking to Robin is pretty entertaining and intellectually invigorating, and I recommend it.

The rest of our very occasional podcast series:

Iterating hurt

A truish story with all names and many circumstances altered:

Alice and Bob were friends, but their friendship had seen some rough patches recently due to an ill-advised business experiment and some ensuing uncomfortable feelings. Today they were co-working, with the intent of mending whatever was broken through dedicated inattention. The two of them sat down in Alice’s kitchen. Bob took out paper, pens, and a loud bell. He picked up the bell and proceeded to pace back and forth across the room jangling it with the vigor of two arms and staring into space.

It turned out that Alice loved the sound of loudly jangling bells, which reminded her of the beautiful church she used to live next to. And she was working on her own general solution to coordination problems—a puzzle so fascinating that she could not possibly be distracted, even by the delightful bell song. So the noise caused Alice no suffering.

However, Alice reasoned that Bob had no way of knowing that she would not be harmed, and that he really should have expected that his bell-ringing would annoy her a lot in expectation. So she had little choice but to infer that Bob didn’t care about hurting her. And that did hurt.

Furthermore, she reasoned that Bob must be aware that she would be hurt by observing that he didn’t care about hurting her. So he should have anticipated not only some suffering in expectation from the bell, but also this second level of more certain suffering from observing Bob’s indifference. Knowing that he was happy to deal her even that much suffering was even worse. And furthermore he didn’t even care about this extra suffering!

On top of that, she reasoned, once there is enough common knowledge of enough certainty of enough suffering willingly inflicted, Bob can’t be doing this by accident. It becomes an intentional affront. A message of hatred, rather than an inadvertent sign of indifference.

She became extremely angry and marched out of the front door.

(Bob made a mental note that Alice really didn’t like bell sounds.)

Knowing that someone knowingly hurt you is hurtful. And knowing that someone knowingly hurt you by indirectly causing you to know that they knowingly hurt you is hurtful. And so on.

I suggest that social injury often has this character of being magnified iteratively by approaching common knowledge.

Perfectly legitimate offense doesn’t even need to stand on the ground at all. Suppose that I like being slapped in the face. Also, I know that you know this. But I also know that you don’t know that I know that you know that I like being slapped in the face. Then you slap me in the face. I’ve got to figure you are willingly harming me with your seeming desire to harm me, even if you don’t think I will actually mind the slap per se. Alice was reasonable to be upset, even though she liked the bell sound.

I expect something similar can happen at a group level. There is an action that hurts a small fraction of some group of people. Then doing it indicates that you are fine with a chance of hurting people from that group, which hurts the feelings of the whole group, and causes enmity with whatever groups you are saliently a member of. Then if people continue to do the action, the victim group takes it as an even larger sign of disrespect and at some point an intentional slight. Then even if the action ceases to hurt anybody on the object level, or is replaced altogether by things that are thematically similar but not object-level harmful, it has become a slight, and continues to hurt, because connotations are hard to erase. I don’t really know if this happens—I don’t keep up with current offense. More informed opinions welcome.

This theory predicts that actions would often be offensive in spite of probably not directly harming anyone on the object level. I think this does happen. I also guess that it leads to some confusion around whether other people are just pretending to be offended. (I also expect people sometimes are pretending to be offended, because often there are incentives to, and at least some people respond to incentives sometimes).

I also wonder if something like this explains why people jump on random insensitive statements that weakly suggest offensive views, even when there is no chance that the person holds the offensive view in question. If I really believed that someone thought I was good at drawing, but I also heard them accidentally momentarily imply to someone else that my drawings were rubbish, I would figure that they weren’t very interested in whether this might hurt me. Also that they might be trying to intentionally anger me. And people intentionally trying to anger me can be angering.

I posit that offense should almost always be out of proportion to the action that caused it. Hurtful actions automatically snowball into being more hurtful, and the offense of the victim is a response to the hurt accrued by the time the snowball lands.

On liking things about crushes

Sometimes I have had crushes on people, and then all kinds of miscellaneous characteristics they had seemed good. Not just their face or their sense of style or the exact way they pronounce my name. But also things that would usually be considered unattractive. For instance, if they are balding, I might suddenly find myself excited by sparse head stubble, when I had previously liked luxuriant hair. And then subsequently I would be more attracted to every other balding guy I met.

I think this is not just directly because the person having those characteristics makes the characteristics by association the most excellent characteristics a person could have. Though that is maybe part of it (your face reminds me of…you!)

I think it is also because I implicitly infer that the person in question likes those characteristics, and I expect people to like me more if I like the things they like. For instance, if they are grumpy and have crumpled clothes, I think I implicitly infer that they like people being grumpy and wearing crumpled clothes, and that if I favor those things too, it will help us be friends. And I can appreciate a pretty wide range of things, so I implicitly give attention to the ones that are helpful.

So I suppose that I must implicitly believe everyone likes almost all of their characteristics. Explicitly, I think this is unlikely to be true. Though I do expect people relate more to people who share their characteristics, whether or not they like the characteristics. So maybe that is what I’m implicitly going for.

All this leads me to think that that my brain is probably doing a milder version of the thing it does with crushes with respect to other people who I like in less extreme ways all the time. “Ooh—I guess you like being mildly irritated! I can do that too! Grr. Do you like me?” It is just only so strong as to be introspectively perceptible in the case of crushes. Which I guess matches the observation that people copy each other a lot.

I have long had the abstract impression that I should choose who I spend much time with carefully because company makes an alarmingly large difference to one’s own behavior. But the way that my brain updates on crushes makes that concern feel more viscerally real to me. Happily (not coincidentally) current company seems pretty good. Though unusual, so probably I don’t give things like religiosity and being athletic proper thought. These concerns are is not news, but a new angle from which to feel like it is actually a real problem and not just one of those problems that it would be virtuous to be troubled by.