LWMap/A Map That Reflects the Territory
08 Mar 2022 - 17 Jun 2023
- A review of a set of books from the LessWrong community
Introduction
- The Rationalism community has packaged up some of the best of LessWrong into book form, and when I saw that one of the five focus topics was agency I could not resist asking for a review copy, that being something of a pet subject of mine. Now I have to follow through with a review, and I'm taking the opportunity to also completely rebuild my writing and publishing stack.
- My relationship to Rationalism is kind of problematic; I basically think they are nice, smart, well-intentioned folks; and while I respect their earnestness and intelligence, their actual ideas, not so much. It's hard to pinpoint exactly where my thinking diverges from theirs; one of the goals of writing this is to uncover those differences. So I will end up doing a lot of carping and criticizing, but with respect and with the goal of trying to articulate my own point of view.
- Also should note that I am writing this in Roam, which I have not really used before, so this is partly a test drive of a whole new writing and publishing toolchain. The end product will no doubt be hypertextish and/or open-notebook-ish to whatever degree seems appropriate.
- Eg: In some of the pages I've included a Further reading section; unlike so these are more instructions to myself than a traditional list of citations. This convention emerged during the process of writing in part because Roam makes bidirectional linking ridiculously easy, it's not something I planned out.
Aesthetics
- The first thing to say is that this set of books is very attractively designed. printed, and packaged. Compared to the last free book I got from rationalists, which pretty much screamed "nerds nerding out for other nerds", this seems quite tasteful. I'm not sure if this represents a maturing of the rationalist movement or something else, but it works for me.
- Focusing on aesthetics might seem a bit shallow in the context of rationalism; that's the kind of stuff they usually disdain. But in fact one of the essays, LWMap/Naming the Nameless, was a critique of this specific practice and a defense of aesthetics and design. This suggests that the community is learning to appreciate the importance of the superficial.
- In that vein, by far my strongest negative reaction to anything in these books did not come from any of the big ideas, but from a tossed-off aside by Yudkowsky to the effect that he was raised on and is apparently still in thrall to the classic (John Campbell era) science fiction, Asimov and Heinlein and the like, while he finds more contemporary writers like William Gibson somewhere between mystifying and appalling.
- Everyone is entitled to their own literary taste I suppose, but, sorry: a preference like that indicates an immature mind (at best). Yudkowsky (admirably) has disavowed the strain of political reaction that you can find in the rationalist community, but this sort of reactionary aesthetics is almost as bad.
- The whole movement is kind of retro in a way that is sometimes appealing but just as often appalling. Peter Sloterdijk labelled rationalists as "the Amish of postmodernism" and it often does seem like an effort to be staunchly and cluelessly devoted to ideas that nobody really takes seriously any more.
- Of course they and/or the Amish might be right, and clinging to old-fashioned forms of thinking may be the only thing that saves us.
Reactions to individual essays
- The set contains five small books, each touching on a different theme related to rationalism: Epistemology, Agency, Curiosity, Alignment, and Coordination. I originally was going to just review the essays in the Agency volume but that plan did not survive contact; instead I picked the ones that seemed to spark interesting disagreements.
- LWMap/Naming the Nameless (Sarah Constantin)
- LWMap/What Motivated Rescuers During the Holocaust? (Martin Sustrik)
- LWMap/Varieties of Argumentation Experience (Scott Alexander)
- LWMap/The Rocket Alignment Problem (Eliezer Yudkowsky)
- LWMap/Embedded Agency (Demski & Garrabant)
- LWMap/Being a Robust Agent (Raymond Arnold)
- LWMap/Meta-honesty (Eliezer Yudkowsky)
- Some essays were more like introductions to or surveys of interesting topics, which were good reads and things I will probably refer to in the future, but I didn't have a lot to say about them so they don't get pages of their own:
- Specification Gaming Examples in AI (Victoria Krakovna)
- A listing of known instances where AI systems unexpectedly managed to "sneak" around their specifications and find a solution to their problem that fit the literal specifications, but not the actual goals of the programmer. A surprisingly long list!
- A Voting Theory Primer (Jameson Quinn)
- This was a really thorough introduction to the issues surrounding various voting systems. The author works on actual voting reform, so refreshingly practical and down-to-earth.
- Inadequate Equilibria and Governance of the Commons (Martin Sustrik)
- Rationalists discover Elinor Ostrom, who brings some actual sociological observation to the abstract game-theoretic problem of cooperation and coordination. A positive development. I talk some about Ostrom in Nobel Prize in Anarchy and Two talks on trust
