Incentives are important, but discussing them can feel amorphous.

There’s a lot of scattered terminology. We have terms like incentives, incentive gradients, signaling, motivated reasoning, cognitive dissonance, bias, and even flexing.

I find this terminology to be frustratingly vague in practice. I like modeling decisions in terms of estimatable costs and benefits, but I haven’t had success putting these concepts into satisfying cost-benefit models.

After playing around with different ideas, I’ve so far settled on the concept of convenience as a helpful construct.

Simply put, a convenience is a specific local/private benefit. They’re typically intangible.

To many, this definition might closely match their existing understanding of the word convenience, enough that much of this essay might seem trivial.

I think the concept is a decent mix of straightforward and valuable. Actually estimating the full costs and benefits of conveniences would be a lot of work, but at least it can be clear what the path forward would be.

I’m particularly interested in conveniences in communication and thought and present examples accordingly. Our work at QURI is about building effective evaluations and estimates, so it’s important to understand where these might be hindered or assisted based on different pressures.

Since formalizing the concept, I’ve noticed many of these incentives in the wild. I’ve built further terminology on top of it and intend to expand on this in future posts.

As always, I’m eager to get feedback.

Examples of convenient actions for myself

Let’s start with examples of conveniences in my life. I think these examples should generally match some of people’s existing intuitions on the word.

Convenient (beneficial) things for me:

1. I use examples in my writing that makes me seem somewhat sophisticated.
2. I use a lot of defensive language ("X is great” -> “I think X might be great"). This makes it harder to attack my writing.
3. I occasionally champion issues that my friends/colleagues like seeing championed.
4. If I criticize someone, it's typically someone I know my community would have an easy time being criticized.
5. I post a lot of writing on Facebook, where it’s more likely to be seen by people who I socially interact with. If these people think well of me, that’s particularly valuable to me.
6. I present certain vulnerabilities to my community that I’m confident would get them to generally like me more, not less.
7. I’ve found that my well-being correlates with my status among peers much more than I’d like. When I see my work be well-liked, that just makes me feel good, even without thinking of the instrumental impacts.

Inconvenient things I (mostly) avoid:

1. Discuss controversial+heated ideas or issues. These occasionally bring trolls or online mobs and can cause me a lot of grief.
2. Use examples in blog posts that would be personally embarrassing or make me look weak.
3. Present content that's so rough or unpolished (given expectations, in its given format), that others would likely think much less of me.
4. Reveal early drafts of work publicly, in situations where I'd expect that they would get more attention if they were hidden until the final release.
5. I often don't cross-post my Facebook posts to the EA Forum. I find the comments on Facebook much more agreeable and less emotionally taxing.
6. Negatively critique of groups with much power over QURI.
7. Posting or talking about expensive items I own. I don’t want to be seen as someone who is flashy.
8. Posting information that would be inconvenient for my colleagues, friends, and family. This means I have a hard time being fully straightforward about some of the work at QURI, for instance. I think this is the pro-social thing to do for my group, but not always for the greater world.

Beliefs that are convenient for me.

I genuinely believe these to be true. At the same time, I should also flag that these beliefs seem pragmatically beneficial to me (at least in the short term). I can only have active beliefs about so many things (even true things), so I’m incentivized to focus on those that would be more valuable.

1. QURI has a good shot at (some) success.
2. The lightcone/humanity has a good shot at success.
3. The effective altruism ecosystem is generally healthy and valuable.
4. I'm on the net, producing value, in expectation.
5. I’m not going to have too many disasters happen to myself or ones close to me in the near future.
6. My friends and colleagues are mostly friendly, moral, and effective.
7. It’s cost-effective for me to have various comforts in my life, because they make me more productive in total.
8. I have many left-leaning views, that make me less likely to get canceled or shunned by my colleagues.
9. This blog post might be genuinely useful for me to write.

Convenience Accounting

As for estimating things not meant to be estimated, we can imagine estimating the costs and benefits of a think tank producing a research document.

There are many actors in such a situation. There’s the organization, but then there are individuals within the organization. Outside the organization are donors, potential recruits, potential collaborators, and all the other relevant roles.

Aside: More generally, whenever an action happens in the world, every existing and future actor, globally, could be impacted, in a way that’s net-positive or net-negative. The vast majority of these impacts will be infinitesimal, but they still exist and can be (poorly) estimated. Deciding what to pay attention to really just depends on interest and existing capabilities.

Let’s imagine that this hypothetical piece of research most directly provides value to a certain greater community. We’ll make a simple example of the costs and benefits for the nonprofit organization and for the community.

When both turn out positive, that means that producing the document is in the interests of both the nonprofit and the community. In an aligned environment, their incentives will be heavily correlated.

It’s possible to get arbitrarily complicated with this if you have the capacity. You could outline complicated network structures of many actors, add probability distributions, adjust for the fact that some items probably overlap, and more.

I imagine that this work is generally too much work for humans, but maybe with AI, we’ll be able to implement it on a routine basis eventually.

Content Producer Ledger

Basics:

• Time Cost: -$4,000
• Cost of editing + feedback: -$500
• Cost discount, due to this post being entertaining to write: +$500
• Learning value: +$200

Conveniences:

• Impress potential donors (through demonstrating consumer value): +$4,000
• Impress potential employees: +$2,000
• Impress friends/colleagues: +$200
• Flow-through: Helping customers, in ways that will later benefit the producer: +$400
• Achieving a “win” as an organization, and feeling more productive: +$300
• Achieving a “win” for the author, making them personally feel better: +$200

Inconveniences:

• Risk of upsetting specific donors: -$600
• Risk of causing a PR crisis: -$300
• Risk of generating online harassment or abuse: -$200
• Risk of upsetting Xavier2320, a certain commenter who often complains loudly on our work: -$200
• Will upset some other actor, which this content portrays in a negative light: -$200
• Organization members other than the author might feel jealous that they lose relative status and attention. -$200

Producer Profit: $1,600

Content Consumer Ledger

Time Costs

• Reading and processing content: -$5,000
• Related comments, and the reading/processing of those comments: -$1,000

Learning (Decision Value)

• Direct learning: Using the information to directly make better decisions: +$4,000
• Technique learning: Learning about the techniques and styles used in the post: +$1,000
• Groundwork learning: Having ideas that can be built upon in future work: +$4,000
• Producer learning: The information reveals things about the producer. It might show that the producer is productive (useful for funders), but it might also reveal other useful information. +$500

Signaling / Conveniences

• Direct Signaling: Having more evidence to support existing community beliefs: +$1,000
• Technique Signaling: Your community can gain reputability by showing it does quality work: $1,000
• Flow-through: The information producer is helped, and this correlates to benefits for the consumer down the line: $200

Inconveniences

• Risk of causing a PR crisis: -$1,000
• Alienation of some of the audience: -$800
• Some group’s work is negatively impacted by content, for some reason: -$500
• Status equilibria: The bar for research content is slightly raised, meaning that other competing organizations look worse in comparison: -$500

Consumer Profit: $2,900

Note that you can’t simply add these profits to get the “total profit to both parties.” There’s some overlap. Also, it’s unclear if any actor would care about the utility of these two parties equally.

Formalizing the terminology

I’ve so far given a handwavy definition and some examples. Can we try to produce a more formal definition?

Here’s one attempt:

Conveniences are private, intangible, tangental benefits.

Private

We’re interested in agent-specific benefits, not total benefits. Often a belief or communication will result in a wide array of winners and losers. The specific benefits to a certain agent are often barely correlated with the total net benefit. For example, one person might have incentives to lie, even though this harms society.

A beneficial belief is one that’s good for society as a whole. But a convenient belief is one that is good for a specific party. It might be good on the whole or not - this is beside the point.

Private benefit is an existing term in economics and law. From Economics Online,

Private benefit is the benefit derived by an individual or firm directly involved in a transaction as either buyer or seller. The private benefit to a consumer can be expressed at the utility, and the private benefit to a firm is profit. Private benefit can be contrasted with external benefit.

I think “agent-specific” or “local” can be clearer but they have other downsides.

Intangible

The examples I have in mind of conveniences are intangible. I don’t think I would classify “this transaction produced several effects, including clearly giving me 1000 widgets” as a convenience, but I would if my benefit were in terms of social capital, or if the benefit to myself were obfuscated.

Tangental

Conveniences are not the clear primary public goal of their corresponding activities. Communication is typically seen as being about conveying information, not about anyone signaling allegiances or trying to gain status. These aspects might be decisive motivators, but they are kind of tacked on.

Economics sometimes uses the terms secondary benefits, co-benefits, and ancillary benefits to mean similar things. But I see the primary communication or belief as more of an orthogonal vessel or delivery mechanism for conveniences to attach to. The mechanism often itself doesn’t have a clear benefit, its use is almost orthogonal or tangental.

Use in the Wild

When an inconvenience is significant enough, I find it easier to say uncomfortable instead. It doesn’t feel right to say, “It’s inconvenient for you to learn you [did something terrible you didn’t realize].” It feels better to say, “It’s uncomfortable for you...”

If the word inconvenience sounds insulting or distrustful, you can substitute other words, just point to a definition somewhere.

Trying to set formal definitions ahead of use is often wrought with failure. If you find yourself wanting to use convenient, feel free to adjust the definition if it helps your use case. If you use a definition that might confuse readers, I suggest being explicit about that.

Adjusting Sentences to Use “Convenience”

As stated in the introduction, I’ve found “convenience” to be a useful handle in many situations.

Here are some example phrases with and without the term. I think the term's use either shortens them or adds clarity. I’m sure readers might be divided here, so see which ones you prefer.

"It is difficult to get a man to understand something when his salary depends upon his not understanding it."
“It’s difficult to get someone to understand something incredibly inconvenient for them to understand.”

“I think that you saying that is very specifically advantagous to you.”
“I think that’s a very convenient thing for you to say.”

“You believing that comes with you gaining some personal benefits.”
“That’s a convenient thing for you to believe.”

“There are some topics we avoid discussing because there are several incentives for us not to.”
“There are some topics we avoid discussing because of inconveniences.”

“There’s an incentive gradient for group X to avoid discussing scary topics.”
“There are many inconveniences around group X discussing certain topics.”

“I think Samantha believes that because of motivated reasoning.”
“I think this is a very convenient thing for Samantha to believe, and I also think that those conveniences are counterfactually responsible for that belief.”

“This organization is biased.”
“This organization is wrong in ways that are very convenient.”

“The reasons they gave were very suspiciously beneficial to themselves.”
“The reasons they gave were suspiciously convenient.”

“Senator Dodd’s bill comes with a bunch of provisions that are specifically useful for his state.”
“Senator Dodd’s bill comes with a bunch of provisions that are convenient for his state.”

Benefits and Risks

There are two main clarifications that I like about this term.

1. Clarity that the benefit is only for one specific agent. The impact on society is an entirely different matter. I find that many statements about local benefits tend to get mistaken for ones about global benefits, so extra clarity is helpful.
2. Clarity that you are discussing incentives, not decisions, beliefs, or actions. Words like motivated reasoning, cognitive dissonance, bias, and flexing imply that incentives have actively changed decisions, beliefs, or actions. It should be possible to discuss incentives without accusations - there’s a very different question on where these incentives might actually change behavior.

Unfortunately, discussions about incentives tend to be heated. In everyday use, convenience is often meant to imply bias, which doesn’t help here.

Meaningful discussion of convenience probably requires very high decoupling norms. I’d expect online discussions around convenience to very commonly get hijacked by mistakes or accusations around these two points, but I hope that the term, when taken with a definition like what I provided, helps more than it hinders.

Critical Responses

I imagine that some potential critics of this might say things like,

Is it immoral to take convenience into account when producing content?

Critic #1 says,

Personal conveniences shouldn’t be considered when writing content. You should only be thinking about the impact on the world. Each piece you write should be as useful as that piece can potentially be, and nothing else should matter. It’s immoral to change what you write based on what’s useful to you.

I adhere to something like the rational actor model of humans, which assumes that humans generally do things that are positive-EV. Bad actions are often done for selfish reasons, but so are good actions.

Emphasizing personal incentives shifts some amount of responsibility from individuals to environments. But people have the ability to shape environments. Instead of focusing completely on blaming people who do things for selfish reasons, we can also focus on making sure their incentives align with what we care about.

Also, writers often need to plan ahead several steps. If they burn all of their goodwill in one piece that would limit their work in the future. They need to be at least somewhat strategic with their work in the short term in order to maximize their impact in the long term.

Applying “convenience” to the side I don’t like

Critic #2 says,

Since learning about conveniences, I’ve realized that all the intellectuals I don’t like are really just saying whatever is convenient for them. I’ve made big lists of all the ways their content is convenient to both themselves and convenient to their annoying audiences. This proves they how useless and harmful their work is.

I’m happy that you’ve learned to consider convenience as it applies to one side. However, I’ll make two recommendations.

1. Apply convenience analysis to your side first. There are conveniences everywhere.
2. Be cautious and measured when making assumptions about the motives of others. It’s very easy to mess this up. Deeply empathizing with friends is challenging; with enemies, it’s much more so.

This terminology sucks

Critic #3 says,

I think it’s dangerous to try to redefine commonplace words like this. Instead you should use something more specific, like “localized benefit”. Having technical words that are really similar but not exactly like their common uses will create massive headaches down the road.

Plus, who are you to propose new terminology anyway? Almost no one will actually take this up, and those that do will just create confusion. We have too much hastily-presented jargon as it is.

First, my main interest here is the general topic, not the name. If we both discuss the same thing but use different names, that’s not that harmful compared to us simply not discussing that thing. I’d be curious to see others use alternative names for the same thing, as long as it’s clear that’s what’s going on. I’d be happy to see this sort of concept become popular under any name.

Second, I guess that many of my readers will prefer and remember the term convenience more than anything like localized benefit. I’ve found it much easier to use and remember. I think the word is close enough to colloquial use that errors here won’t be that bad. These are empirical claims, I guess we’ll get evidence if anyone else actually reads this.

But I am unsure. I wish we had some controlled vocabulary service with serious experience and authority on this issue, but we don’t. I’ve tried raising this discussion with other people, but most treat terminology discussions as incredibly tedious. Thoughts and advice in the comments are appreciated.

Isn’t this just media theory?

Critic #4 says,

Isn’t this just media theory and sociology, disguised for rationalists? You really need to understand Slavoj Žižek and Derrida before trying to make your own theories here.

Slavoj Žižek and media theory were definitely inspirations for this. I’m not as well-versed in continental philosophy as I’d like. If anyone reading this is interested in helping out by occasionally providing perspectives here, do reach out!

Also, this content is very similar to writing on status and signaling. In my circle, some prominent figures here are Robin Hanson, Diana Fleischman, and Geoffrey Miller.

Discussing convenience is itself inconvenient

Critic #5 says,

Dude, you’re not supposed to say this stuff out loud. If you talk about the incentives behind your writing, people will assume that you’re biased. You’re just giving your enemies ammunition to use against you.

I think that our epistemic environment will be healthier if people develop a better understanding of bias and convenience.

If you’re in an environment where it’s incredibly unusual to disclose your potential biases and conveniences, then it will seem weird and potentially disadvantageous to do this yourself. However, if you’re in an environment where doing so is a strong norm, then it actually could become convenient for you to be honest about these things. That’s the kind of world I’d like us to move towards.

This is uncomfortable!

Critic #6 says,

I find it uncomfortable to think about. It feels like you’re attacking me.

Finding a topic uncomfortable is a promising sign! Many real epistemic issues are uncomfortable to think about and talk about. Because of this, I think we should be particularly empathetic and understanding when discussing them.

I’ll do my best to keep my discussions at least friendly and non-threatening (I hope the cute illustrations help). My point is to understand our environment better, not to attack anyone.

I’m guilty of being biased around convenience too, I think it’s just how humans and organizations work. I think it’s important to watch out for your self-interest, you just should generally be honest about your motives (especially to yourself) and avoid situations where you’ll do harm to those around you.