barkbellowroar

305 karmaJoined

Comments
23

Hi David,

I’m intrigued by what you are trying to do with this LLM project. Prompt-engineering is very much a trial and error process - here’s what I got from walking through it a few times. 

To clarify my understanding of your objective here is: 

  • Given the 80,000 hours podcast episodes
  • Identify the episodes discussing quantitative social sciences, economics and policy
  • Within those episodes identify questions brought up in conversation
  • Filter those questions by if they are specific and measurable (operationalizable)
  • From there, identify the questions that would impact a donor’s decision-making in deciding whether or not to fund further research
  • (gleaning from your examples) add a direct quote and link to relevant literature

Does that sound right? If so here, here’s what I got (a few different threads) (top to bottom)

1st attempt: 

  • Make a research agenda for one specific episode
  • identify donor relevant funding questions

2nd attempt: 

  • first identify all the relevant podcasts (but 10 seems to be the limit for any kind of listing) 
  • I narrowed it to just one (quant social). 
  • I then tried using 80k podcast filter to help narrow it but perplexity says it can’t read the filterable listings of the transcripts.

3rd attempt: 

  • I asked it to pull all episodes related to economics (caps it at 10)
  • Pull questions from each episode (only does it for 3 episodes)
  • Identify questions that are measurable and influence donor decision-making
  • Asked it to add quotes but it lost the thread I guess and didn’t connect to the above list of questions. 
  • So I tried narrowing it down to one episode and filter through again adding a quote and a link to a piece of related research. 

Curious to know if these results are along the lines of what you are trying to achieve? 

My guess for systematizing this would be defining a prompt flow that gets the results you want and then replicating it across each individual podcast episode. You could use a separate thread to get it to create a list of episodes relevant to topics to narrow the amount you have to search through. 

Long-term it would be cool to see 80k have something like tim ferriss’ blogbot given how much content they have (and the information that is buried in each podcast episode). 

Two general notes on UXD: 

1) It could be worth conducting user tests of whether people find the site's landing page being the forum feed overwhelming. It’s hard to get your bearings on that page versus say the “best of the forum” page. People typically like to be guided initially in an experience and get a feel for what’s available, then explore. Or even just a pop-up with “learn about the forum” (it takes a minute to find the link for that page on the sidebar and these days people bounce within seconds). 

2) In the spirit of the intranet comment above, I’d love to see the CRM hidden behind a logged-in view; this could just be my “safety” lens but having a giant list of confirmed EAs in public view seems problematic (sadly, something to consider these days). Maybe ask community health team for their view on it? but when it rolled out I was a little irked to see it created without first asking users if they want to be placed on such a definitive, public listing. 

Thanks for writing this Sarah; nice to have leaders sharing thinking/requesting feedback!

My quick ‘off-the-cuff’ thoughts in response: 

I definitely agree the Forum is critical as a “space” in EA, particularly for those of us who aren’t living in a hub, have access to conferences, nor work directly in an organization. It can often feel at times like being a buoy at sea without that direct connection - the handful of times I’ve drifted from EA over the years it has been the Forum pulling me back in (by helping me get up to speed on what’s going on especially in cases like the FTX stuff … and contests are often one of the few ways I feel I can participate - i.e. open phil’s worldview prize, red teaming, future fund RFP). 

I have at times seen established EAs “punch-down” on the “extremely online” EAs and I’ve felt that to be a shallow judgement of those of us not privy to direct engagement opportunities. It definitely gave me the impression the movement and community are two separate entities not always aligned. I’d like to think the Forum could better blur that distinction to avoid hubs becoming silos with strong views (which I think contributes to the confusion around the public perception of EA). 

That aside, the Forum right now is confusing because it’s providing multiple services in one; a newspaper, an opportunities/classifieds board, a library (the wiki) and a discussion space though not as free flowing as slack/discord. Now with groups and CRM and event tracking it’s becoming a catch-all for EA information. Given this… 

 Has the forum team considered reframing the Forum as an intranet? (Note, the intranet model is often associated with corporations but it’s used in a lot of contexts for brand community sites, colleges and social groups).

 I’m thinking one of the limitations of the Forum is that the name implies one feature, but now the Forum is so much more than that - if we were to shift away from centralizing “the Forum” aspect and make it the “Hub” or “EA Online”... whatever it’s called, the idea is to step back and reframe the online space as more than just a “forum” because that’s what it’s becoming, as evidenced by CEA taking on the EA Hub and the Opps Board recently, it’s clear the goal is for CEA to manage the movement’s digital infrastructure.

I see the intranet design helping resolve issues like low engagement. If people can accomplish multiple things like search/apply for jobs, join topic threads, and chat with each other, complete surveys and more, they may be more likely to login and engage. This can help on the backend with things like surveys, community health team work (ticket system), and maybe down the road a common app or work trials. It also helps draw a boundary around community conversations that don’t need to be as public (which I think is a huge deterrent for engagement right now). 

I’d be happy to discuss more in-depth and share some examples of what I’m visualizing here but I’ve long thought the Forum was heading in the “intranet” direction and could provide a lot of benefit for both members of the movement and those running infrastructure for it. 

Hi David, I'm working on a project that might be relevant to this - sending you a DM. 

EA in the wild: I'm having trouble adding a screenshot but I recently made an online purchase and at the bottom of the checkout page was a "give 1% of your purchase to a high-impact cause" - and it was featuring Giving What We Can's funds! 

Always fun to see EA in unexpected places. :) 

In case you haven't seen it, CEA has redone their website. I like the new look and the content makes it much easier to understand the scope of their work. Bravo to whomever worked on this! 

EDIT: just confirmed that FHI shut down as of April 16, 2024

It sounds like the Future of Humanity Institute may be permanently shut down. 

Background: FHI went on a hiring freeze/pause back in 2021 with the majority of staff leaving (many left with the spin-off of the Centre for the Governance of AI) and moved to other EA organizations. Since then there has been no public communication regarding its future return, until now... 

The Director, Nick Bostrom, updated the bio section on his website with the following commentary [bolding mine]: 

"...Those were heady years. FHI was a unique place - extremely intellectually alive and creative - and remarkable progress was made. FHI was also quite fertile, spawning a number of academic offshoots, nonprofits, and foundations. It helped incubate the AI safety research field, the existential risk and rationalist communities, and the effective altruism movement. Ideas and concepts born within this small research center have since spread far and wide, and many of its alumni have gone on to important positions in other institutions.

Today, there is a much broader base of support for the kind of work that FHI was set up to enable, and it has basically served out its purpose. (The local faculty administrative bureaucracy has also become increasingly stifling.) I think those who were there during its heyday will remember it fondly. I feel privileged to have been a part of it and to have worked with the many remarkable individuals who flocked around it."

This language suggests that FHI has officially closed. Can anyone at Trajan/Oxford confirm? 

Also curious if there is any project in place to conduct a post mortem on the impact FHI has had on the many different fields and movements? I think it's important to ensure that FHI is remembered as a significant nexus point for many influential ideas and people who may impact the long term. 

In other news, Bostrom's new book "Deep Utopia" is available for pre-order (coming March 27th). 

On a related note to my other comment on this post: 

A lot of organizations are acknowledging the impact of FTX on their work which is important but I would also like to see an EA organization try to evaluate the positive or negative impact switching to longtermism has had on their ability to attract talent, donors etc.

You say you want to diversify assets but Open Phil still holds a commanding 80% of your bottom line - and both orgs have become much more longtermist in recent years. If OP is going to just prop up 80k for the next several years because it serves their aims, why would I fund 80k when there is now a considerable gap in the landscape for a new career service org that caters to the other cause priorities of EA? 

Equally curious about the push to grow the team if not seeing significant increase in impact, especially given the $2M marketing push this past year.

In 80K’s 2021-2022 Review it mentioned: 

(1) “we seem to be hitting diminishing returns in outreach encouraging more people to apply to advising...” (page 7 under current challenges)

and again 

(2) “overall, we’d guess that 80,000 Hours continued to see diminishing returns to its impact per staff member per year.” (on page 10 under impact evaluation)

What is the strategy/argument for “expand the team” being the best intervention for increasing organizational outreach and subsequent impact? Is it really just a capacity issue or could it be a scope issue? 

Load more