As others have said, game AI doesn't draw from "real" or academic AI or Machine Learning.

That said, the references for AI and machine learning are quite old. Particularly the machine learning parts. The only ML texts on the list are Mitchell and Duda and Hart. The former is extremely outdated at this point. That's not Mitchell's fault -- it was a nice book for learning the basics of machine learning in 1997 when it was published, but all the developments that have made ML a hot subject have occurred since then and in areas that the book simply didn't predict coming. Duda and Hart, similarly, was the bible for certain subfields of ML for a long time, but it won't tell you what everyone's been doing in the past 15 years when ML exploded onto the wider scene.

If I were to add one book, it would be Kevin Murphy's excellent text (https://www.amazon.com/Machine-Learning-Probabilistic-Perspe...). There's no one book that will give you a complete picture of the field, but his is I think the closest available and does a solid job of preparing you with enough fundamentals that you can extend your knowledge from there on your own.

ML isn't really used in game programming. These books focus on more practical techniques such as State Machines, Pathfinding, Planning, Scripting etc.

Exactly. Game AI is not so much about making AI smart, but more about making it fun.

I haven't played it, but there is a game called Smart Kobold that tries to up the AI difficulty as far as it can reasonably go. In some sense, I think it satisfies your criteria. The punishing difficulty is what makes it rewarding, and it accomplishes it through AI rather than making creatures harder to kill or do more damage.

I do agree with your statement in the vast majority of cases. Ultimately, fun trumps everything else when it comes to games (though what counts as fun is subjective). Even Dark Souls, which many (most) gamers consider difficult, is not difficult because of unbeatable AI. It's full of patterns (indeed, that's how you get better at the game: you recognize and respond correctly to those patterns).

This book list is about game AI programming. Game AI and ML don't overlap much.

In particular, I would like to see something on OpenAI Universe