Jim's Reviews > Human Compatible: Artificial Intelligence and the Problem of Control

I'm a computer guy, but I don't know much about how AI works. Russell added a lot to my knowledge & did so in basic steps that I appreciated. As in any field, there's a vocabulary to pick up & while it often resembles typical speech, there can important differences that need to be spelled out. This holds especially true with concepts that we don't understand well in ourselves & yet are trying to build into machines, specifically "intelligence". Just what that is only partially answered in the second chapter because it carries a lot of caveats with it.

Even fuzzier are all the assumptions we can make when defining a goal. He uses some great examples that show we don't always literally want what we ask for. If we tell a car to get us to the airport ASAP, we really don't want it to break speed records or play demolition derby simply to save a few minutes. By the same token, if we want to make sure we get to the airport on time, we don't want to leave the day before & camp out overnight, but both might make sense in a purely logical manner.

Many seemingly simple commands need to use the uncertainty principle, probabilities, & a host of data on real world conditions including our preferences. Of course, our preferences can change over time & we're often irrational. No one is identical in their preferences either, so a robot serving 2 masters has to resolve priorities, too. And on and on down a rabbit hole of complexities. He kept them sorted out very well.

I was really impressed by the control problem in regards to self preservation. If we don't specify it, a program would quite reasonably conclude that allowing itself to be turned off would mean it couldn't fulfill its function. He mentioned Hal 9000 in this regard.

Early on, he promises not to mention SF in the section & that was a mistake, at least in communicating to me. About halfway through the book he starts mentioning SF & has some decent examples. He even says that SF authors are the one group that have given many scenarios a lot of thought. He missed some great books & examples, unfortunately. I was able to fill them in mentally. Maybe it's just me, but the dry recitation of the wire-head experiments (A rat with a wire stuck in its pleasure center & the ability to pull a lever to cause pleasure will do so until it dies.) is sad & disgusting, but it doesn't viscerally affect me the way it did when Gil the ARM dealt with it in Niven's short stories. It's been several decades since I read one of those short stories & I still have a vivid memory.

I loved the audiobook & highly recommend it, but get a print copy, too. Look at the Appendices & skim them early on, maybe even read them. They fill in some gaps that help with the text. I read them & listened to them. I think understand them now. There's a lot to think about, though. I think I could reread this right now & not be bored, but get even more out of it. Now that's a recommendation since anyone who has read my reviews knows how much I detest repetition.