Seán Ó hÉigeartaigh on FHI and CSER

This is the last part of the Cause Prioritization Shallow, all parts of which are available here. Previously in this series, conversations with Owen Cotton-BarrattPaul ChristianoPaul PenleyGordon Irlam, and Alexander Berger, and Robert Wiblin.

Nick Beckstead interviewed Seán Ó hÉigeartaigh on the Future of Humanity Institute (FHI) and the Center for the Study of Existential Risk (CSER). The notes are here.


Robert Wiblin on the Copenhagen Consensus Center

This post summarizes a conversation which was part of the Cause Prioritization Shallow, all parts of which are available here. Previously in this series, conversations with Owen Cotton-BarrattPaul ChristianoPaul PenleyGordon Irlam, and Alexander Berger.



Katja Grace: Research assistant, Machine Intelligence Research Institute

Robert Wiblin: Executive Director, Center for Effective Altruism


Robert talked extensively with the Copenhagen Consensus Center (CCC) while investigating them as a potential Giving What We Can recommendation for funding[1]. This is Katja’s summary of relevant things Robert learned, from a conversation on the 16th of January 2014, supplemented with facts from the CCC website.

Activities and impact

At a high level, CCC’s main activity is prioritizing a broad variety of altruistic interventions based on cost-effectiveness. They do this by commissioning top economists to research and write about good spending opportunities, using a cost-benefit analysis framework. They do secondary research, assembling existing academic evidence into actionable priorities for governments and philanthropists. An important feature of this work is that they squeeze the analysis of a wide range of topics into the same framework, so one can make reasonable comparisons, given a lot of assumptions.

CCC also devotes substantial efforts to encouraging people to use this decision-support, and in general to prioritize based on good data analysis. In the Millenium Development Goals project for instance, money is probably divided between one third on research and two thirds on dissemination of that research.

CCC usually has around one main project at a time, and as one finishes they dovetail into another. They have around 4-5 core staff, and bring in extra contractors for a lot of the work. The annual budget is $1-2M, and the cost of core staff is probably only around a couple of hundred thousand dollars annually.

The value from CCC’s work does not usually come from finding unsuspected good interventions. It is rather from linking together evidence to make a strong cases for activities that are already believed to be good among experts, but which aren’t widely supported. CCC has for instance highlighted the high value of health interventions relating to nutrition and contagious disease. The notion that these are very good interventions is not unusual among development people, but most of the money is spent elsewhere, so there is a lot of value in making such cases. That CCC usually reaches such plausible conclusions suggests their research method is sensible. Their view on climate change is an exception to this trend; it is quite unusual.

CCC have provided a number of documents on the impact of their work[2]. They have numerous examples of people listening to them and doing things. They can also point to media coverage and a modest number of cases where they said to do something, and talked about it and soon afterwards the person did something like that. It is hard to establish causation, but this is suggestive evidence.

Very few people do anything similar to CCC. Cause prioritization is rare. Doing comparison work at all is a somewhat unique selling point, as is asking for quantitative estimates on things that are not often quantified. Talking about why climate change is not the best cause is also a niche activity.

Contributing to CCC

When Robert spoke to them, CCC was looking for funding for their post-2015 (Millenium Development Goals) project[3]. Their website suggests they still are, along with an American Prosperity Consensus 2014 project and a Global Consensus 2016 project. If the post-2015 is not completely funded there will be less outreach than hoped. They will engage less with the media, and won’t be able to afford some events with officials, where they intend to describe the research and try to persuade them.

Their other recent work includes a book on how much problems have cost the world[4]. Much of the data in it had never been published before, since economic models are seldom run “backwards” – into history.

CCC is currently looking for two summer interns for their back-office in Budapest, Hungary. The desired profile for these positions includes graduate education in an area relevant to CCC’s work (in particular, relating to research project management and outreach) and an interest in and aptitude for digital media (including social media, search, video, web sites). Interns will be assigned tasks and mini-projects within the post-2015 project and the general outreach program, and report to the post-2015 engagement manager and/or the post-2015 project manager. Good mutual match could lead to a permanent position.

[1] Their report is Smart Development Goals: A promising opportunity to influence aid spending via post-MDGs? Giving What We Can.

[2] See Smart Development Goals: A promising opportunity to influence aid spending via post-MDGs? Giving What We Can, p11-12



Alexander Berger on GiveWell Labs

This post summarizes a conversation which was part of the Cause Prioritization Shallow, all parts of which are available here. Previously in this series, conversations with Owen Cotton-BarrattPaul ChristianoPaul Penley, and Gordon Irlam.


Alexander Berger: Senior Research Analyst, GiveWell

Katja Grace: Research Assistant, Machine Intelligence Research Institute


Since this conversation, GiveWell Labs has become the Open Philanthropy Project.

This is a summary made by Katja of points made by Alexander Berger during a conversation about GiveWell Labs and cause prioritization research on March 5th 2014.


GiveWell Labs


GiveWell Labs is trying to answer the same basic question as GiveWell: “what’s the best way to spend money?” However GiveWell Labs is answering this question for larger amounts of money, which is less straightforward. Causes are a more useful unit than charities for very large donors. So instead of trying to figure out which charity one should give to this year, they are asking which program areas a foundation should work on.

Givewell’s relationship with Good Ventures is a substantial reason for focussing on the needs of big donors, and GiveWell Labs research has been done in partnership with Good Ventures. The long term vision is to have ongoing research into which areas that should be covered are not, while providing support for a wide range of foundations working on problems they have previously identified as important.

Approach to research

GiveWell Labs primarily aggregates information, rather than producing primary research. It also puts a small amount of effort into publicizing its research.

Their research process focuses on answering these questions:

  • how important is the issue?
  • how tractable is it?
  • how crowded is it?

They attempt to answer the questions at increasing levels of depth for a variety of areas. It is not certain that these are key criteria for determining returns through a program, but they seems correct intuitively.

Most research is done through speaking to experts (rather than e.g. reading research papers). The ‘importance’ question is the only one likely to have academic research on it.

The learning process

GiveWell Labs is prioritizing learning value and diversification at the moment, and not aiming to make decisions about cause priorities once and for all. Alexander would guess that the impact of GiveWell Labs’ current efforts is divided roughly equally between immediate useful research output and the value of trying this project and seeing how it goes.

In the time it has existed, GiveWell Labs has learned a lot. A big question at the moment is how much confidence to have in a cause before making the choice to dive into deeper research on it.

Spending money

Starting to spend money is probably a big part of diving deeper. Spending money is useful for learning more about an area for two reasons. Firstly, it makes you more credible. Secondly, it encourages people to make proposals. People don’t tend to have proposals readily formulated. They respond to the perception of concrete available funding. This means you will get a better sense of the opportunities if you are willing to spend money.

Transferability of learning

Alexander doesn’t know whether methodological insights discovered in one cause prioritization effort are likely to be helpful to others. One relevant factor is that people at GiveWell Labs have priors about what’s likely to be successful that are partly based on what they have learned before starting the process. But if you didn’t share the starting priors, you might not end up with the same current beliefs. This might be true regarding explicit expected value calculations, and how to weigh robustness or reliability against a high upside, in particular. If you don’t share the same prior, the lessons learned may not be very communicable.

Funding cause prioritization

Adding resources to GiveWell

An outside funder trying to donate to GiveWell Labs couldn’t change the distribution from GiveWell’s conventional research to GiveWell Labs. It would also be hard to change the total work done by donating. Donating would mainly change the amount of time GiveWell would spend raising other funds, and the amount to which they depend on Good Ventures.

Other cause prioritization efforts

Projects like Katja’s cause prioritization shallow investigation are unlikely to be done by Givewell.

Katja’s Structured Case on AI project is also unlikely to overlap based on GiveWell’s current plans. If Alexander were working on something like this, he would typically initially try to effectively aggregate the views of credible people, rather than initially forming object level views. For instance, he would like to know what would happen if Eliezer could sit down with highly credentialed AI researchers and try to convince them of his view. The AI Structured Case on the other hand is more directed at detailing object level arguments.

Cause prioritization work can become fairly abstract. Givewell Labs tries to keep it grounded in looking for concrete funding opportunities. Others may have comparative advantages in more philosophical investigations, which avoids overlapping, but is also less likely to be informative to GiveWell Labs. GiveWell is unlikely to focus on prediction markets, though it’s not out of the question.

General considerations for funding such research

If others were going to do more concrete work, it is a hard question whether it would be better at this point for them to overlap with GiveWell Labs to provide a check, or avoid overlapping to provide broader coverage.

Answering high level questions such as ‘how good is economic growth?’ doesn’t seem very decision relevant in most cases. This is largely because these issues are hard to pin down, rather than because they are unlikely to make a large difference to evaluations if we could pin them down, though Alexander is also doubtful that they would make a large difference. For instance, Alexander doesn’t expect indirect effects of interventions to be large relative to immediate effects, while Holden Karnofsky (co-executive director of GiveWell) does, but their views on this do not seem to play a big role in their disagreements over what GiveWell Labs should prioritize.

When deciding what to do on cause prioritization, it is important to keep in mind how it will affect anything, such as who will pay attention, and what decisions they will change as a result.

Similar projects

Nick Beckstead and Carl Shulman do similar work in their own time.

Alexander’s understanding is that Copenhagen Consensus Center is doing something a bit different, especially around modeling cost effectiveness estimates. They also seem to be less focussed on influencing specific decisions.

Alexander is not aware of any obvious further people one should talk to that Katja has not thought of.

Gordon Irlam on the BEGuide

This post summarizes a conversation which was part of the Cause Prioritization Shallow, all parts of which are available here. Previously in this series, conversations with Owen Cotton-BarrattPaul Christiano, and Paul Penley.



Gordon Irlam: Philanthropist and Founder of Back of the Envelope Guide to Philanthropy

Katja Grace: Research Assistant, Machine Intelligence Research Institute


This is a summary of points made by Gordon Irlam during an interview with Katja on March 4th 2014. The summary was made by an anonymous author.

The Basics

The Back of the Envelope Guide to Philanthropy (BEGuide) is a one-man project that aims to quantify the impact of philanthropic causes and share information, helping philanthropists to prioritise spending in a way that delivers the highest value for money.

Gordon Irlam has been running the BEGuide for about 10 years and estimates that he’s spent between 200 and 600 hours on it in total, working on it in short sprees whenever promising new information catches his attention.

Gordon chooses which causes to investigate by gut instinct, looking at anything that he feels might turn out to be worthwhile. Almost every cause that he’s evaluated has turned out to promise a positive net impact.

Work normally begins with a seed in a newspaper article or radio program and grows within a few hours to an evaluation that can be published on the BEGuide website.

Gordon also runs the Gordon R Irlam Charitable Foundation and his work on the BEGuide is now the basis for around 99% of that foundation’s spending, around $200,000 annually in coming years.

What Gets Done

Gordon does no original research, but spends his time compiling available information and processing it to get results for the BEGuide.

Gordon considers this kind of work important, and laments that academic researchers get funded to go out and get information, but once they write a paper on it, seldom manage to direct it anywhere that it can be compared to other papers, so the knowledge doesn’t come into use and the papers are quickly forgotten.

Uploading the work to the BEGuide also takes time, but almost all of Gordon’s effort goes into the research. He muses that it would take a lot of time if he were ever to update the website, which currently has a simple design.

Gordon continues to update the BEGuide as new information emerges, both adding new causes and revising those he’s already worked on. He still revisits a lot of organisations that he’s been following for so many years, because a lot of those organisations that he looked into initially are still doing relevant work and expanding, as has the BEGuide, though Gordon’s own approach hasn’t changed since he began.

How It Works

Gordon’s method is, he says, quick and dirty, just as the name suggests. The back of the envelope calculation estimates the value of an activity as:

                           How big is the problem?                       
How much would it cost to do something about?

Once Gordon knows what the problem is, he searches the web, looks through journals and takes in information from other organisations until he has enough facts and figures to fill out the equation.

Gordon says that his background in physics has helped him to recognise that a lot of the problems in philanthropic evaluation can be addressed with Fermi estimates. He thinks this is a good approach because the field doesn’t have access to the detailed, accurate data that is necessary for more detailed calculations. Having studied pure mathematics and science, Gordon feels that it’s deep in his nature to appreciate and use these tools. He believes that quantifying everything is the right way to make philanthropic decisions.

Because of the complexity of many of the causes that he’s worked on, Gordon tries to hone in on the main thing that any funding decision would achieve, ignoring any flow on effects, which are, in his experience, harder to quantify and because of his selection process, inherently less important. If he becomes aware of good information on the long term effects of a funding decision, he would use the data, but believes it would be highly unusual for such information to be available.

Who Uses It?

The BEGuide website gets one or two visitors per day and Gordon doesn’t believe it’s used by any philanthropists outside his own organisation.

While smaller parties could find the BEGuide useful, Gordon posits that there is too much followup work for anyone but large philanthropic organisations to do in finding out which organisations best pursue what the BEGuide finds to be the best causes.

Gordon initially expected more interest in the website. He made it a wiki at one point, but it was quickly overrun with vandals, forcing him to retreat from that initiative.

Gordon says that he’s done very little to publicise the BEGuide. He initially wrote to Engaged Donors for Global Equity, who published a blurb for the website in their newsletter, but he has not maintained contact with any publication or actively advertised to other philanthropic organisations since then.

Gordon regrets that he has no real contact with anyone who might find the BEGuide useful and says he’d like to talk with other organisations about his methodology, however he hasn’t taken any steps to do so. The Laura and John Arnold Foundation, approached him and expressed an interest in using similar techniques to quantify the value of some 800 nonprofit CEOs.

The Results

Causes are organised on the BEGuide by what it gives as the upper limit of their Leverage Factor, which is, in simple terms, value for money.

A Leverage Factor of 15 is equivalent to $15 worth of societal value per dollar of input. One virtue of this unit is that it deemphasizes the necessary fact that in quantifying and comparing various philanthropic causes, the BEGuide puts dollar values on such things as human lives. Exactly what that dollar value is, for various results where it might be controversial, can be found on the website, but not on its main page.

If the BEGuide has found one trend in cause evaluations, it’s that high value causes tend to be high risk.

The biggest example so far is hostile artificial intelligence, which the BEGuide ranks as the most fundable cause, with a leverage factor between 100,000 and 11,000,000.

Gordon thinks philanthropic organisations tend to be unwilling to take risks, probably because they feel an obligation to have every cent spent on something that will have a result. He doesn’t see that this reasoning should be more true of philanthropists than of anyone else, comparing the opportunities in philanthropy to those of ordinary finance.

Another tendency that the BEGuide has found is that popular problems, such as solving global warming, are low value, because, the associated costs are so high.

This means that the highest value funding opportunities turn out to be high risk endeavours that aim to reduce problems that not many people know about, which is a problem, because this type of funding is understandably very unpopular with philanthropists.

Expanding the Audience

The most important barrier between the BEGuide and philanthropic organisations may be publicity. If Gordon had resources to add to any part of the processes that enable the BEGuide vision, he would use them to market to philanthropists.

Because philanthropists are regularly swamped by organisations looking for funding, they tend to develop a barrier that makes them very difficult to reach directly.

For this reason, Gordon suggests that the most efficient way to improve interest in projects like the BEGuide would be to deliver workshops at philanthropy conferences, such as the upcoming EDGE conference in Berkeley or those listed by the Chronicle of Philanthropy. These are places where philanthropists might have their guards down and be willing to find out that methodological cause valuation exists and to learn how it might benefit them.

What Else Is There?

Gordon has been in contact with GiveWell and is a member of Giving What We Can, but he feels that these organisations are limited to examining things like developing world issues.

Gordon believes that the BEGuide is unique in indiscriminately assessing the possible impacts of causes and feels that there is a need for a lot more aggregating and sorting of the existing evaluations that are produced by disparate organisations.

Other bodies that Gordon has used to source data for the BEGuide include the Copenhagen Consensus Centre and the Disease Control Priorities Project, both of which produce publications.

Gordon also stresses that it’s important to know the market. The Centre for Effective Global Action at UC Berkeley, for instance, provides data that’s useful for the US Agency for International Development and might at first glance like a good place for philanthropists or philanthropic data agglomerators to look for information. However it investigates problems like the comparative value of giving $10 in aid, or $5 and a chicken, while philanthropists often only want to know which organisation to support.

Without knowing much about other methodological approaches to evaluating causes, Gordon, expresses the sentiment that philanthropy as a whole is probably missing any methodology and that perhaps work on something like a marketing statistics approach would be of more value to the field than simply doing more research.

If other people are interested in contributing to research, Gordon feels that while there may be high value causes that have escaped his attention so far, newcomers’ time would probably best be spent looking in more detail at the highest leverage factor interventions on the BEGuide, as those causes could be much more attractive to philanthropists if their value were clearer.

On top of that, some of the causes on the BEGuide need regular updating and others haven’t been fully explored. Gordon worries that for his hostile AI evaluation, he has so far only been able to look at advocacy, but there are other possible solutions to that problem which might be more effective.

AI: is research like board games or seeing?

‘The computer scientist Donald Knuth was struck that “AI has by now succeeded in doing essentially everything that requires ‘thinking’ but has failed to do most of what people and animals do ‘without thinking’ – that, somehow, is so much harder!”‘
- Nick Bostrom, Superintelligence, p14

There are some activities we think of as involving substantial thinking that we haven’t tried to automate much, presumably because they require some of the ‘not thinking’ skills as precursors. For instance, theorizing about the world, making up grand schemes, winning political struggles, and starting successful companies. If we had successfully automated the ‘without thinking’ tasks like vision and common sense, do you think these remaining kinds of thinking tasks would come easily to AI – like chess in a new domain – or be hard like the ‘without thinking’ tasks?

Sebastian Hagen points out that we haven’t automated math, programming, or debugging, and these seem much like research and don’t require complicated interfacing with the world at least.

Crossposted from Superintelligence Reading Group.