Why big organizations don’t take risks proportionate to their size (a theoretical argument)

TLDR version: People optimize E(Log(x)). Big organizations should optimize E(Log(Σ x)), but because they are composed of many people, they optimize ΣE(log(x)).

Logarithmic Utility Curves

Suppose someone offers you a 50% chance of a $1M prize, or a 10% chance of a $10M prize. Most people who don’t already have a lot of money would take the first, more certain option, even though the expected value of the second is double that of the first. ($0.5M vs $1M). $1M would improve my life a lot. $10M would also improve my life a lot, but not so much more than $1M that I’m willing to risk getting nothing. Economists call this “risk aversion.” and model it using a “utility curve.” Though quite abstract, it corresponds pretty well to intuition. The idea is that $10M isn’t “worth” 10x as much to me as $1M, so I have to account for that before I take the expected value. The most common utility curve used in simple models is logarithmic, both because it’s mathematically simple, and close enough to reality for most purposes. Intuitively, no matter how much money I make, I’m willing to put in a linear amount of additional effort in order to raise my income level by 10%, so you end up with logarithmic utility.

To compare these two offers, before we take the expected value, we’ll take the log of the outcomes.
log(1M) ~ 6,
log(10M) ~ 7,
0.5 * 6 > 0.1 * 7, so we take the first offer as observed.

Rather than optimizing E(x), we optimize E(log(x)) and get behavior somewhat like what a real person would do.

Organizations

It makes sense that an organization should have a utility curve looking like a log. Going out of business (log 0) is -infinity as observed, and companies care about percentage improvements rather than absolute improvements just like individuals do. I apologize for not having better arguments that this is the way organizations should behave. This is a blog post, not a research paper. :)

This means that if the output of each employee is x, the outcome for the company as a whole is Σ x, and the company should be optimizing E(log(Σ x)).

How do they actually behave? The company will behave in the way that the aggregate of its individuals behave. If individuals are rewarded in proportion to their personal outcomes rather than the company’s outcomes, they’ll behave by optimizing their personal utility function. As a result, the company as a whole will optimize Σ E(log(x)).

Big organizations apply the risk-aversion of an individual to losses that are teeny relative to the size of the organization, because people care more about their individual risk than the risk to the company. Ideally, a company should have many people working on huge improvements with a small probability of success. In reality, it's almost impossible to structure incentives so that doing things with a small chance of success is a good personal strategy. It’s hard to reward competence and not outcomes. If you reward outcomes, the company will be risk averse. Having a few engineers work for years on something that doesn’t finally work is a teeny risk to the company, but potentially a huge risk to the careers of those engineers.

History through Google Books Ngrams

Each of these graphs plots over time the fraction that one word constitutes of all words in books mined from the Google Books project.

Westward expansion of the U.S.

One invested in railroads, the other, canals.

New religions of the last 200 years.

Race.

Note how people didn't start capitalizing "black" until the mid '60s, even though it had been used as a descriptor as often as "colored" for a long time. Note the different curves for "negress" and "Negress." Capital "Negress" rose in usage alongside "Negro woman" and was nearly as common in the '40s, but half as common in the '70s. Lowercase "negress" was mostly replaced by capitalized versions, which were then replaced as well.

Industrialization of food.

Pawpaws were more popular than blueberries, and gooseberries were more popular than raspberries!

Communism

Presidents

Marijuana

Industry

Music

AIDS

Notice how HIV lags AIDS, and condom is correlated more with HIV than with AIDS.

Feminism

Scurvy

Ideology

Israel and Palestine as a people and as a place

The names of Muslims

It's 2:00 AM, and there's still worlds to explore just off the top of my head. This is a treasure trove. Google Books Team, this is an amazing accomplishment. I used to envision that this is how history would be done in 50 years when the digital age has ensured that anything that someone cares enough about to transfer is preserved. Now we're getting to do that sort of history, today.

Inception

The movie is great. Go see it. What I'm about to write will be entirely spoiler free.

Inception will leave you distrusting your own perceptions in the manner of Descartes's Demon, more so than The Matrix ever did. It does this I think because the experience of the characters in the movie entering and leaving dreams is so similar to our experience entering and leaving the movie. "Waking up" from a good movie is a very similar feeling to waking up from a dream.

When we talk about "suspending disbelief," it isn't supposed to be something you do, but rather something that happens automatically if the movie succeeds in drawing you in. When dreaming, we tolerate wild violations of physics and causality without it ever decreasing our emotional involvement. The objective of a good movie is to tap into that level of credulity, and it wouldn't surprise me if someone discovered that the same pathways in our brains are involved.

When Christopher Nolan was writing the dialogue about how addictive it is to be an architect of dreams, he must have been really talking about is how addictive it is to be an architect of shared dreams: movies.

Men's Wearhouse is missing a major business opportunity.

Every male in the U.S. who goes to a prom or is someone's groomsman has likely been fitted for a tuxedo at the Men's Wearhouse. Theoretically, the Men's Wearhouse should have the most comprehensive database of men's sizing information of any company. There have been a lot of weddings among my close friends this year, and I assumed that after the first fitting I'd be able to rent subsequent tuxes without going to the store in person. Amazingly, according to one of the attendants, they throw away all information about the customer after 6 months.

This loss goes beyond convenient tuxedo rentals. With this much information about me, they could be delivering clothes custom-tailored overseas to my doorstep at a price no other company could match. That much opportunity is certainly worth the cost of a few dozen more machines. Ancillary data produced by your business can become more valuable than the profits from the business itself. Don't throw anything away.

Janelle Monáe (Wow)

Imagine if Lauryn Hill became a geek, teamed up with Outkast, and wrote two concept albums about an android uprising in the futuristic city of Metropolis. The result would sound something like Janelle Monae's inaugural two albums, Metropolis, and The ArchAndroid, the first three acts of a six act story. The songs flow seamlessly (literally) from one to the next, usually jumping genres in the process. She can wail, she can scat, she can croon, she can rap, she can dance (oh she can dance), and she can sound like GLaDOS when necessary. Orchestral interludes bookend the chapters; In one, she puts words to a snippet of Claire De Lune.

Pitchfork loves her too. Historically, the intersection of the albums I like and the albums they bother to review has been approximately 0, so these two recommendations are about as independent as they come. Go have a listen! Every track can be heard for free at the links above.

My favorites so far: Sincerely, Jane, Tightrope, and Oh, Maker.

Tax-time gripes

California Use-Tax

It's really not reasonable for me to try to add up all the money I've spent online (but only on items used in California, not those purchased as gifts to someone out of state.) This is the largest expenditure for me outside of rent and travel, and I don't remember everything I've bought and from where. Mint.com and searching for "receipt" in gmail helps, but it's really a lost cause. I do my best, but it seems most people I talk to don't bother, despite it being the law. This is by far the most time-consuming part of filing my return, and only hurts me to do so.

A few things confuse me about it.

1. Is shipping and handling included in the taxable portion? It's not "used" in California.
2. Do online retailers have enough information themselves that they could provide some sort of automated assistance in tallying these?
3. Have the courts decided whether use-taxes are constitutional despite being a tax on inter-state trade?
4. Am I a chump for bothering?

Passwords

Filing my taxes inevitably involves getting data from financial institutions that I haven't had any interaction with all year, and thus, remembering my various passwords. If you are designing a password system, please don't place restrictions on the content of the password. It's your responsibility as an administrator to keep your users' passwords from being brute-forced. Do not offload this responsibility to them.

Arbitrary restrictions I've seen today alone while trying to get forms from various financial agencies:

• Must have at least 1 number.
• Must have at least 2 numbers.
• Must have at least one capital letter and 1 number.
• Cannot have special characters.
• Must be greater than 6 characters, but no more than 15.
• Must be greater than 6 characters but no more than 8.

If I can't have a small secure set of passwords to use everywhere that I can reasonably remember, I'm just going to email them to myself, thus defeating every high-minded security best practice you are trying to implement.

Google can render your equations for you!

In my last post I mentioned that Knol and Google Docs now have equation editing. What I didn't mention is that this is an undocumented feature of the public Google Chart api, and it's easy to use. For instance, if I wanted to include the Schrödinger equation on this blog. I would construct an url like this:

http://chart.apis.google.com/chart?cht=tx&chf=bg,s,FFFFFF00&chco=AACCFF&chl=i\hbar\frac{\partial}{\partial t} \Psi(\mathbf{r},\,t) = \hat H \Psi(\mathbf{r},t)

Put your code in the chl parameter. The chf parameter lets you specify a background color in RGBA, chco lets you set the foreground color in RGB. When you drop it in inside of an image tag you get this:



If you anticipate making 250,000 calls to the chart server a day, contact Google first at chart-api-notifications@google.com. There's no limit to how much you can use it, but they reserve the right to turn you off.