Overcoming Bias

For many decades I’ve thought about how to reform areas of life via finding ways to measure the long term outcomes people want from each area, and then paying providers for achieving those outcomes. As soon I’ll be at an event where we will be talking about how to reform news, let me take a stab at doing that for news.

If what news customers want is to read the articles that others read, so they can discuss them together, then readers could pay proportional to how many others will read the same article.

If what customers want from news is a feeling of enjoyment from reading, we might just frequently give consumers two new articles, have them rate which they liked more, estimate personal ELO ratings from such tests, and pay news providers more for higher rated articles.

If what news customers want from news is info to predict the big picture future of humanity, we might test LLMs on their ability to predict such things, then pay for each article based on how much such LLM predictions improve by reading that article.

If, in addition to the above, news customers just want accurate articles, that make fewer false claims, we could just evaluate random articles for accuracy, and pay more for more accurate sources.

Sure there are many details re making each of these approaches work better. But the main problem seems to me to be that customers just don’t like such approaches. Most would feel ashamed to make cultural choices using more mechanical numerical mechanisms. Especially if explicit strong financial incentives were involved. Re culture, self-respecting folks follow their vibes.

For example, few seem interested in my many proposals to reform crime, health, career planning, and other areas of life via strong incentives tied to numerical metrics. And I’ve seen many visibly show me how much less they think less of me from learning that I rely heavily on MetaCritic to pick movies and TV shows.

To solve cultural drift, we are going to have to somehow recruit the same level of intense effort and accuracy that modernity has achieved in tech, science, and business practice to other areas of life now more dominated by norms and vibes. But a big obstacle to that is our norms and vibes against such things.

Added 27Feb: Of course I should mention a big way that news might change soon: it might include far more prediction market prices.

“Info institutions” solicit contributions, aggregate them into info summaries, and distribute such summaries to audiences. Examples include gossip, courts, journalism, academia, social media, speculative markets, and official reports of orgs (such as govts and churches). Such institutions often dis their competitors. For example, most have long dissed gossip, the oldest. Early journalists were dissed by governments and churches. Recently, academics and journalists have dissed social media.

Lately, journalists have been dissing prediction markets, with complaints that can be made about most info institutions. For example:

• Prestige - Its bad if people get info they enjoy, vs what prestigious folks say is good for them.

• Waste - People might enjoy it so much they waste time and money on it.

• Money - This involves money, which could change incentives.

• Privacy - Sometimes it is bad to spread more info. For example, info on candidate chances on election day.

• Secrets - People who had promised to keep secrets might be induced instead to reveal them.

• Sabotage - Participants might push changes to the world to make their takes more accurate.

On these complaints I say: treat the various info institutions alike. For example, if you want to ban govt officials from trading in prediction markets, for fear they’d reveal secrets, then also ban them from talking to reporters, or from gossiping. If you want to ban sports betting due to possible waste, then ban sports news and entertainment too. If you want to promote democracy by protecting political speech in gossip, journalism, and social media, then protect political prediction markets also.

For some kinds of complaints, we have good evidence that prediction markets are in fact superior to other info institutions:

• Errors - In particular cases, predictions have been wrong.

• Vagueness - In particular cases, it was unclear to some what exactly was being claimed.

• Manipulation - Folks might offer biased contributions to distort audience actions.

Prediction markets have been consistently more accurate than other sources on the same topics at same time with similar resources. And an expectation of manipulation attempts on average makes such markets more accurate. If these issues are important, we should be willing to tolerate doing worse on other problems, to do better on these.