“Human Values ≠ Goodness” by johnswentworth

LessWrong (Curated & Popular)

There is a temptation to simply define Goodness as Human Values, or vice versa.
Alas, we do not get to choose the definitions of commonly used words; our attempted definitions will simply be wrong. Unless we stick to mathematics, we will end up sneaking in intuitions which do not follow from our so-called definitions, and thereby mislead ourselves. People who claim that they use some standard word or phrase according to their own definition are, in nearly all cases outside of mathematics, wrong about their own usage patterns.[1]
If we want to know what words mean, we need to look at e.g. how they’re used and where the concepts come from and what mental pictures they summon. And when we look at those things for Goodness and Human Values… they don’t match. And I don’t mean that we shouldn’t pursue Human Values; I mean that the stuff people usually refer to as Goodness is a coherent thing which does not match the actual values of actual humans all that well.
The Yumminess You Feel When Imagining Things Measures Your Values
There's this mental picture where a mind has some sort of goals inside it, stuff it wants, stuff it [...]
---
Outline:
(01:07) The Yumminess You Feel When Imagining Things Measures Your Values
(03:26) Goodness Is A Memetic Egregore
(05:10) Aside: Loving Connection
(06:58) We Don't Get To Choose Our Own Values (Mostly)
(09:02) So What Do?
The original text contained 2 footnotes which were omitted from this narration.
---
First published:
November 2nd, 2025
Source:
https://www.lesswrong.com/posts/9X7MPbut5feBzNFcG/human-values-goodness
---
Narrated by TYPE III AUDIO.