You know, you don't have to oscillate between the extremes of fundamentalist Christianity and atheism. I find the materialist account of reality doesn't actually make that much sense when you start poking at it, leaving open the possibility of spirituality. Perhaps you would get something out of reading things like the Tao Te Ching, the Bhagavad Gita, and the Dhammapada, to balance out rationalistic atheism.

There is no one posture that has all the answers.

Thank you for this post!

I've been kinda following the 80,000 hours career guide, but I don't think it's what I'm looking for. Ultimately, it clarified that what I really want is to work for an org with a good mission. I'm a software developer, and I'm very motivated, signed the Giving What We Can pledge and everything.

I checked out their profile on software engineering, but surely there are more orgs out there that need software devs no?

I've also been thinking I could be good at an operations role, but it's very unclear how to get on that ladder.

The privacy concerns seem more realistic. A rogue superintelligence will have no shortage of ideas, so 2 does not seem very important. As to biasing the motivations of the AI, well, ideally mechanistic interpretability should get to the point we can know for a fact what the motivations of any given AI are, so maybe this is not a concern. I guess for 2a, why are you worried about a pre-superintelligence going rogue? That would be a hell of a fire alarm, since a pre-superintelligence is beatable.

Something you didn't mention though: how will you be sure the LLM actually successfully did the task you gave it? These things are not that reliable: you will have to double-check everything for all your use cases, making using it kinda moot.

You might want to read this is as a counter to AI doomerism: https://www.lesswrong.com/posts/LDRQ5Zfqwi8GjzPYG/counterarguments-to-the-basic-ai-x-risk-case

This for a way to contribute to solving this problem without getting into alignment:

https://www.lesswrong.com/posts/uFNgRumrDTpBfQGrs/let-s-think-about-slowing-down-ai

this too:

https://betterwithout.ai/pragmatic-AI-safety

and this for the case that we should stop using neural networks:

https://betterwithout.ai/gradient-dissent

I'm looking for statistics on how doable it is to solve all the problems we care about. For example, I came across this: https://www.un.org/sustainabledevelopment/wp-content/uploads/2018/09/Goal-1.pdf from the UN which says extreme poverty could be sorted out in 20 years for $175 billion a year. That is actually very doable, in light of the fact of how much money can go into war (in 1945, the US spent 40% of its GDP into the war). I'm looking for more numbers like that, e.g. how much money it takes to solve X problem. 

I intend to use them for a post on how there is no particular reason we can't declare total war on suffering. We can totally organize massively to do great things, and we have done it many times before. We should have a wartime mobilization for the goal of ending suffering.

Hi Felix, thanks for the recs! What I mean by  giving to charity not being exactly rational, is that giving to charity doesn't help one in any way. I think it makes more sense to be selfish than charitable, though there is a case where charity that improves ones community can be reasonable, since an improved community will impact your life.

And sure, one could argue the world is one big community, but I just don't see how the money I give to Africa will help me in any way.

Which is perfectly fine, since I don't think reason has a monopoly on truth. There are such things as moral facts, and morality is in many ways orthogonal to reason. For example, Josef Mengele's problem was not a lack of reason, his was a sickness of the heart, which is a separate faculty that also discerns the truth.

Nice to meet you! Also a new guy. Good to see you're a witch, I'm a mystic! A burn event is a copy of Burning Man? Definitely would like to go to one of those.

Hello everyone!  My name is Carlos. I recently realized I should be leading a life of service, instead of one where I only care about myself, and that has taken me here, to the place that is all about doing the most good.

I'm an odd guy, in that I have read some LessWrong and have been reading Slate Star Codex/Astral Codex Ten for years, but am for all intents and purposes a mystic. That shouldn't put me at odds here too much, since rationality is definitely a powerful and much needed tool in certain contexts (such as this one), it's just that it cannot do all things.

I wonder if there are others like me here, since after all, the decision to give to charity, particularly to far-off places, is not exactly rational.

Hoping to learn a lot, and to figure out a way to make my career (been a software developer for 11 years) high impact, or at least, actually helpful.

You guys are the Rebel Alliance from Star Wars, and I am ready to be an X-Wing pilot in it!