Harmony Explained: Progress Towards a Scientific Theory of Music (2012)

I played in a gamelan ensemble in college. The music was completely foreign. A piece that sounded like a funeral procession was played at weddings. I have a hard time believing the arrangement of music is not cultural - or, at least partially culturally bound.

I've been working with some people building a music model from scratch and it really surprises me how much correlation there is across different features. It's been difficult applying that to general sound effects, because music is highly correlated in ways more general sound isn't.

Does metal sound "sad"? Opinions differ. :)

So there must be more to it.

Your brain is still just pattern matching “minor = sad” based on thousands of hours of examples from the western canon consumed throughout your life regardless of how you dress it up with rhythmic variations.

FTFY. Notice when you conflate subjective experiences with objective ones. You have been molded by the culture you grew up with. Trying to disentangle yourself from it is like for a monolingual person to express thoughts without using a language.

I had a similar dream as a freshman in physics, I had learned that the organ of Corti was a spiral-shaped transducer in a little special bone and I dreamed of snail shells and proving that harmonies formed interesting geometric patterns on the darned thing, and how maybe people who seemed chronically tuneless might be shown to have malformed ear bones -- until I finally got to see images of them in a bio textbook and realized that my dreams had to be crushed, this blobby thing, whatever else you say, is not working geometrically.

So a science of how music works physiologically and psychologically, is able to explain why rock and later certain forms of ambient and trance "felt good" to folks -- it took them right back into the womb: the distortion on the guitars felt like the swishing amniotic fluid, the four-on-the-floor sounded like a heartbeat, the indecipherable high-pitched vocals sounded like Mama's voice peeking through the noise. But beyond that, it's our experience with twanging strings and air blowing through wind instruments and the like, which informs what sounds familiar to our grown ears. And the proof is how robotic and alien pure sine waves sound: this is a part of your organ of Corti complaining, "hey, when these hairs are tickled usually those ones also tickle -- but they're just standing flat, what the hell is going on."

Lots of everyday things tickle the organ of Corti in patterns that resemble a major chord, so that sounds familiar to it. When you play a minor chord you deprive it of stimulation to some of those hairs but stimulate the ones a little more outward, and the organ says "there's something off, this thing is lower than I'd expect" and it's some sociology that you can do to say "when people are hurt they can communicate this most effectively to other pack animals through lower more mournful cries" and so there are semi-biological semi-cultural expectations that govern "lower = sadder" especially at low tempos. But you can do the same in reverse, play a sus4 chord, and it sounds weirdly brighter and you feedback through those semi-biological expectations in reverse, weird but with more energy than expected. But you can also just play minor scales uptempo and then they also change, the minor scale at a higher tempo has somehow more energy and less sadness than the major scale -- indeed rock music is full of minor pentatonic licks that nobody would ever describe as sad or painful or discomforting.

You've got an awful lot of assumptions in there that I don't think are at all supported, and this is one. It's also contradicted by 3.5.2 in the article: if the brain were actually doing this, then you could, say, replace the 9th by the 2nd.

> Ok, but where do the expectations come from? Your assertion that it is all culturally relative ("nurture") and none from the structure of the most efficient algorithm for recognizing the harmonic series ("nature") has no basis in fact; you just assert it without proof.

I interpreted "mostly meets expectations but surprises you" as referring to things like chord progressions creating tension and resolving it, for which you don't need major triads. (and of which there's no discussion in the article).

> In contrast, there is strong evidence for the absolute nature of auditory processing

I'm perfectly happy to grant that an owl experienced the sensation of a pitch at a frequency that was not actually present, but there's something missing here that's also totally missing from the article: the role of the cochlea. The brain doesn't experience sounds, i.e. the pressure waves in the air, the brain experiences electrical signals transmitted to it that, tinnitus aside, come from excitement of the cochlea by those pressure waves. How do we know that there isn't some kind of mechanical resonance somewhere in the ear that fills in the missing root? And if we know there isn't in the human ear, how do we know the same for the owl?

> The explanation of the experience of the minor triad in "Harmony Explained" requires the concept of an inconsistency arising from a redundant recognizer algorithm.

Once upon a time the explanation of the propagation of light required the concept of a luminiferous ether.

> Your assertion that it is all culturally relative ("nurture") and none from the structure of the most efficient algorithm for recognizing the harmonic series ("nature") has no basis in fact

I never said that. I said "Though it's built on harmonics and overtones which can be explained objectively, you cannot make a fully scientific theory of harmony since cultural expectations factor in just as much (and maybe even more) than the expectations around harmonics." To put it in your terms, "you cannot make a fully scientific theory since nurture factors in just as much (and maybe even more) than nature". The world is not black and white, just because I believe sociology to be critical to understanding the emotional effect of music on people, does not mean I'm claiming there is no objective, measurable component.

> If so, then an octave is too simple to fire the recognizer"

Ok so the happiest music is perfect fifths? Or is 3/2 ratio also too simple? So when does this recognizer get complicated enough to fire? 5/4? For someone claiming to have a scientific theory, you are making a lot of assumptions to make this theory work. Scientific theory is about testing hypothesis to reveal truth and not making hypothesis up to match what you already believe. You make an assumption that the brain wraps/aliases frequencies, create a theory of the recognizer, and then make that recognizer have a very convenient threshold that it makes a physical process suddenly the reasoning for "happy". To the "fully scientfic theory" above, I absolutely agree that ratios play a part, simple ratios are boring, and get more interesting as they get complex, then get too complex and it sounds dissonant. That part I think we, and most people, can agree is objectively true, and fits in the theory of expectation vs surprise at a microscale. I just don't agree this translates to "happy" or "sad". Please show me an experiment that indicates the kind of recognizer that you're talking about, or some kind of cross cultural link between ratios and emotional quality. I'd actually be very curious about that, as it's been a while since I've been reading papers about this.

> The music was played into the brain with the root removed; when coming out of the brain of the owl, brain had restored the root

Yes, I'm sure you know this, I'm just showing I know this too - this is called psychoacoustic bass, and happens as part of the feedback loop between the brain processing and the actual ear - the ear actually generates the root tone! This could also be a product of pattern matching though, as we're used to hearing related harmonics (say 200, 300hz) with a root (100hz), so if we just get the harmonics without a root (200, 300hz) the psychoacoustic part of your brain helps out by filling in the 100hz - this is an alternate theory to the aliasing/halving/doubling theory you put out, not saying either one is absolutely true, just that there are other explanations for the owl thing.

It also has nothing to do with the emotional effect of the music.

> So at Berklee they teach recognizer algorithms and artifacts thereof?

Again, I did not say that. I'm not talking about the speculative, non-experimentally backed parts of your theory, just the parts of your essay that are part of the common scientific consensus. I said "ratios and how chords derive from that was absolutely part of my education at Berklee". Nothing more.

It's a challenging read, but extremely thorough and enlightening. Tymoczko wrote a follow up that extends and reformulates his theories, but I haven't gotten through it yet.

Also, other traditions focus on melody over harmony (e.g. Carnatic and Hindustani), although they have to consider harmony as a by-product of polyphonic music (i.e. more than 1 instrument).

Harmony is derived from composing music with multiple melodic parts. This is most obvious with counterpoint, but even pop/rock music can be seen this way. The essence of pop/rock is a melody and bass line as the two primary parts, and the rest of the instrumentation often plays harmonies that imply inner voices.

I suppose that is literally true but I think of timbre (especially after reading Sethares who I mentioned earlier) as telling me about the notes that are brought along for the ride but aren't shown on the sheet music. It's well known that a distorted powerchord introduces a major 3rd (and not the 12TET major third) as well as a phantom or missing bass note and other higher notes which may or may not be perceptible in a given piece. Quite a bit different from what two flutes playing a root and fifth will give you.

You can have a whole set developing motifs, then multiple minutes of a single note being played in the middle, and somehow that sounds awesome (if you’re into it)

Maybe go on Youtube and listen to music that you might not be familiar with?

Cage, Schaffer, Stockhousen, Boulez are some composers in the tradition of European classical music.

If you go outside that tradition, there’s even more. Endlessly more.

My biggest overall critique is that you start with the assumption that everything is an algorithm, that the brain works like a computer, and the world can be derived from that.

"To the intuition of anyone who has seen hardware designed it seems very likely that the brain is halving/doubling frequencies by many different powers of two in parallel and then running all of the results through the frequency recognizer at once. If any one matches, the harmonic has been found. If this were so, then tones (and notes) that differ from each other by a factor of two would sound very much alike. "

I don't see why I should care about a hardware designer's opinion on the matter any more than a clockmaker's or a dog trainer's.

"1.5.1 Timbre: Systematic Distortions from the Ideal Harmonic Series" I don't believe you ever mention how you want the intensities in your "ideal harmonic series" to be distribute, so why does changing the relative intensities to get timbre matter? If I have an ideal harmonic series with peaks at 220Hz, 440Hz, 880Hz, etc, what should its intensities be? Timbre also has to take into account the full ADSR envelope.

"That is, two notes (series-es of overtones) made by the same (kind of) instrument will be distorted from the ideal Harmonic Series in the same (or similar) way. This must be the case in order for an instrument or instrument kind to have a uniform, recognizable timbre"

If you get a guitar in standard tuning and play, say, the 5th fret on the 6th string it sounds different than the open 5th string, despite them both being the same 'A'.

"But The Nasca People Of Peru Use A Linear, Not A Logarithmic, Scale!" "Who knows what is going on with the linear scales of the Nasca. I suspect the following: Linear-scale flutes are easy to make ... This culture was small and isolated and so no one ever noticed harmony"

I checked the cited article (Joerg Haeberli (1979). Twelve Nasca Panpipes: A Study. Ethnomusicology, 23(1), 57–74. doi:10.2307/851338 ). Among the things I learned: (1) At least some of the pipes were ceramic (picture here: https://artmuseum.princeton.edu/art/collections/objects/1374...) (and the Brooklyn Museum claims that "hundreds" have been found: https://www.brooklynmuseum.org/objects/51700), this is not consistent with a culture that is just figuring things out (2) "Intervals close to the fifth, major third, minor third, and major second occur".

You don’t get the perfect fourth until you go all the way around the circle of fifths.

You cannot stack perfect fifths and get a major scale.