Machine Learning Books

Another book that Dr. L had us read. During the 2008 presidential debate I see one party is trying to buy votes even though the failures of socialism have been proven time and time again throughout history. This is the singular short work on the failures of socialism.

Friedrich Hayek's The Road to Serfdom was written at a time when the Labour Party of Britain was openly socialist. Although modern social democrats renounce the 's' word, socialism is indeed the root of their thinking, and in this exposition, his magnus opus, Hayek unabashedly sends socialism to the gutter where it belongs. Hayek's thesis, that socialism and totalitarianism are two birds of the same feather, has stood the test of time, and it continues to show up today in the cases of Venezuela or Bolivia. Hayek was arguably responsible for Labour's (and the Democrats') turn to the right, set in stone by former PM Tony Blair (and former President Clinton). This book is, however, still very relevant, exemplified by the Democrats' plan to invade the health care sector, the countless bureaucracies located in Washington, and President Bush's reckless invasion of privacy (which is related to Hayek's arguments about war time and peace time). Although Hayek often comes off as soft on a number of issues, he could not be nearly as dedicated as Milton Friedman to absolute freedom because the intelligentsia was on the far-left in the 40's.

For classical liberals, modern leftists, and conservatives alike, The Road to Serfdom is extraordinarily eye-opening.

The only, effective way to reject socialism is by attacking it's fundamental philosophical ideas. That collectivism is good and the individual must be sacrificed for the "good of the people". Attacking a philosophy such as communism or socialism, because it is not "practical" is a contradiction and undercuts any argument against such a corrupt philosophy. These ideas are not good in theory but bad in practice. They are evil in theory and therefore evil in practice.

I would like to also recommend Ayn Rand's, "The Virtue of Selfishness". This is THE work to understand Man's Individual Rights based on His Rational Nature. It is from these fundamental Truths that the ONLY proper function of a legitimate government is derived - The protection of Individual Rights.

Considering it my duty as an economics major, I took it upon myself to read this book, with little expectations as to its brilliance, and was completely swept away. Not only is Hayek extremely eloquent in articulating the case for free trade, he supports his arguments with facts (e.g. what was then going on in Nazi Germany) and with theory (e.g. why without even the historical evidence that we do have we must conclude that a centralized system cannot equally favor everyone).

Since it is my tendency to look at the 1 star reviews before making a 5 star one, I recognize that some people don't like Hayek because he doesn't recognize the great things about socialized medicine (like how a guy in Canada signed up for a CAT scan under his dog's name because animals are not covered under their highly efficient centralized health care...true story by the way) or the kind thoughts of socialist thinkers (please don't make me choose my selection of Marx quotes). But what Hayek does is present a prima facie case against socialism; before anyone can advocate socialism, they MUST address Hayek's arguments.

This is why I think before any socialist and libertarian face each other in a squabble, both must have read The Road to Serfdom so that they can hit on the applicable issues instead of babbling on about poverty statistics. Are you a socialist and disagree with Hayek? Fine, but read the book so that you know where your opponents stand. I really think that socialists think lovers of capitalism are greedy and have no ethics. But if you read our spokesman Hayek, you'll see why we think that the free market is actually BETTER for society.

Let's change the scope of the argument. Socialists should stop arguing about how some people are poor...yes, some people are poor...and demonstrate how a centralized system can make people BETTER than they would be under the free market system. How planning the systems of production would be more efficient and prosperous than under the system of competition. How giving all our freedoms to one entity would guarantee them for all. If you can effectively address these issues and the many more that Hayek brings up, we will soon see a blessed change in the current headache of debates on socialism.

Friedrich August von Hayek was an Austrian-British economist and political philosopher known for his defense of classical liberalism and free-market capitalism against socialist and collectivist thought in the mid-20th century. Since 1920s, he worked in Austria. Unwilling to return to Austria after its annexation to Nazi Germany, Hayek became a British citizen in 1938, a status he held for the remainder of his life. It was during this time that "The Road to Serfdom" originated, originally published by Routledge Press in March 1944 in the UK and then by the University of Chicago in September 1944.

Hayek's central thesis of this book is that all forms of collectivism lead logically and inevitably to tyranny, and he used the Soviet Union and Nazi Germany as examples of countries which had gone down "the road to serfdom" and reached tyranny.

The book has many worthy observations. For example, all people are different by their mental development (which is also influenced by family environment and education, not counting the physical differences of the brain and endocrine system) and thus the classes of the society are needed at least to give more developed people to fully put into action their potential. Liquidation of social classes will also liquidate the abilities of more developed individuals. The same is on the international level. Consider international planning. Whichever honest and democratically open panning system will be adopted, it will be opposed by less developed and poorer nations, because they will see it as ignorance or oppression of their interests. This is obvious - the needs and goals of poor or underdeveloped countries cannot match the goals of rich or developed countries; as the interests of more educated people cannot match the interests of less educated ones.

Many people came to a conclusion that the wealth, in some extent, depends on a level of education. The problem is that not all the people in equal extend incline to the education, to their self-improvement. This is because of the differences of their needs, habits, abilities, capabilities, and so on. Leo Tolstoy in his novel "Resurrection" arose a question of how to improve the level of education: from inside of each individual or from outside? Which came first, the chicken or the egg? Should first the level of education in the society be risen which yields a revolution (dialectic transition of quantity into quality) or the revolution should make the environment to foster the education. Hayek doesn't explicitly raise this issue, but brings parallel between delegation of decision making in managing an enterprise and managing the state. Hayek thought that if a company boss makes all decision making solely by himself and doesn't give the work (of decision making) back to the people (see Ronald Heifetz's publications), it is similar to the states with totalitarian government. Such a dictatorship, enterprise-wide or country-wide, can be used in particular circumstances, but should not be used in all cases as the absolutely correct way of management, according to Hayek.

I would recommend this book to anyone looking to learn the basics of computation theory and formal languages or for someone looking to revisit this material after an absence.

The book is very readable and covers the basics in a systematic fashion. I haven't looked at this material since my university days, but found it very easy to read and digest.

I had read a few other books on automata, but I was not able to get clear notion about pumping lemmas, decidability and so on. However, when I read the 1st edition of this book, finally, I got those concepts. One of great things in this book is proof ideas. This book has very clear and concise explanations about proofs.
I bought 2nd edition of this book, since it has lots of solved problems and exercises. those solved problems will boost your understanding the text and they contains a few things you should know, but omitted in the text.
If you don't understand many concepts in automata and computability with other textbooks, I strongly recommend this book.

This is a great book. The topics are covered in a clear and interesting way. I came to this book after having been exposed to NFA's and DFA's in a compiler course and this exposition is much more enlightening. The proofs in this book are very well written in my opinion, very clear. Studying proof techniques in a book such as Solow's "How to Read and Do Proofs" will prepare you well to understand the proofs in this book.

I recently encountered this book at a publisher's booth at a computer conference and read it on the ride back home. This morning I made a trip to the college bookstore and notified them that it is the textbook that I will be using in my computation theory class this spring.
The chapter titles are:

0) Introduction - this chapter contains the fundamental mathematical background of sets, functions, graphs and proofs. For most students, it could be skipped or skimmed.
1) Regular languages - this chapter is an introduction to deterministic and nondeterministic finite automata and regular expressions.
2) Context-free languages - an introduction to context-free grammars and pushdown automata.
3) The Church-Turing theses - an introduction to Turing machines and the variants, such as multiple tapes and nondeterministic Turing machines.
4) Decidability - the definition of decidability and how Turing machines and finite automata are used to prove or disprove if a language is decidable.
5) Reducibility - the definition of reducible and how Turing machines can be used to execute reductions.
6) The recursion theorem - an introduction to the recursion theorem and some applications to formal theories.
7) Time complexity - the first chapter in the coverage of algorithmic complexity, in this case execution time.
8) Space complexity - an examination of the complexity of algorithms from the perspective of the amount of memory required.
9) Intractability - an examination of the problems that can be solved in principle but not in practice.
10) Advanced topics in complexity theory - approximation algorithms, probabilistic algorithms, alternation, interactive proof systems, parallel computation and cryptography.

There is less coverage of grammars than most books, which is replaced by more in the area of algorithmic analysis. In my opinion, that is an appropriate tradeoff, the analysis of algorithms gives the students some understanding of how automata are applied in computer science.
Another excellent feature of this book is the solutions to selected exercises that appear at the end of the chapters. My estimate is that reasonably detailed solutions to approximately one-third of the problems are included. This allows the students to work extra problems by themselves, and helps the instructor if they are asked to do another example in class that they have not already worked through.
The exposition is very good; I am convinced that the students will be able to read the material on their own, which is one more reason why I adopted this book for my course.

Go buy an international version which is a lot cheaper than this, and they have the same contents. This version is also printed in Black and White and the paper is really cheap. Don't make a mistake like me buying same product for 90$ more. Again, the only difference between hardcover and softcover(international ver) is the price.

This book is an interesting (i.e. light, not too technical) introduction to Genetic Algorithms, Neural Networks, and potential applications of both. The book covers an appropriate amount of history, and a pretty extensive set of notes for those that care to dive deeper. The core of the story is about applying GA and NN to the game of checkers and creating a computer program (NN) that can play checkers without any pre-learned (human) knowledge about how "best" to play the game.

Overall it was a good read. The author argues that the current exmaples of AI success (e.g. DeepBlue in chess) are actually failures as they are brute-force solutions to specific problems with no true "learning" taking place. I would agree, but am also not sure that NN+GA represent "learning" either. They are another method for searching a solution space [in my VERY simple/humble opinion]. All the "learning" happens through solution mutation from generation to generation. Once the mutation/generation process is halted the "learning" also stops.

Anyway, worth the read if you are interested in NN and/or GA, but not already well-educated on the topics.

This is a well written and entertaining book about the author's quest to use neural networks to "teach" a computer how to play checkers. What makes the author's efforts unique is his steadfast refusal to use "human" knowledge in the design or development of the software. Instead his approach is to infuse the software with only the basic mechanics of the game. He then uses neural networks and pits various mutations against each other with only the strongest advancing to the next round. By mimicking the process of natural selection, the author is able to evolve a program that is able to play checkers at a surprisingly high level.

My one criticism of the book is that too much page space is dedicated to describing the intricacies of the various matches. I imagine the readers to be only superficially interested in the actual checkers components of the book and more interested in the AI mechanics. Some of the page space which the author devotes to match description could have been more profitably used to describe the detail behind the neural networks. The author does give descriptive overviews of the process but details of the algorithms themselves are rather scant.

Checkersjock logged in for his daily internet game of checkers and saw a new player. "Hmmm," he thought, "Blondie24 (yrs old), described as a grad student in math who is athletic...yea, I'd like to play her."

Blondie enticed many players. Unbeknownst to them, Blondie was a computer program administered live by the author of this book, David Fogel, and his crew. They played internet checkers for over a year, letting Blondie learn by doing. Unlike "Big Blue" for chess which processed massive amounts of information fed into it by programmers, Blondie was programmed with 3 layers of parallel circuitry and feedback loops, mimicking the human brain. Through trial and error, Blondie taught herself how to play checkers.

As Blondie progressed up the ladder to compete with the best players, Fogel & crew learned how to flirt, reject dates and deal with trash talk online, tweaking the software during downtime. They kept meticulous records and followed rigorous rules so as to satisfy the demands of the scientific method.

For anyone interested in AI, this is your book. For those just interested in a good story, skip the programming & neuroscience shoptalk. This is a dynamic and enchanting memoir appropriate for both AI experts and the general public. Recommended.

I've read Blondie24 more than twice. It's a great and inspirational book about what the future of artificial intelligence is going to be like when computers teach themselves how to figure things out. I was prompted to write here because of a review claiming that Blondie24's playing strength is exaggerated in the book and that it is based only on the ratings of people on a checkers website. On the contrary, the book is straight and upfront about the playing capability of the program. Blondie24 is a great story about how David Fogel and Kumar Chellapilla created a program that learned how to play checkers using very little checkers information. Like, where are the pieces, and what are they. The program learned by playing against itself. Blondie made it to the top 500 of people rated on www.zone.com, but Fogel didn't stop there and claim that Blondie was an expert. Instead, he competed Blondie against a version of Chinook, the world champion program, that played at the expert (so-called "novice") level and won 2, lost 4, and drew 4 games with it. Coincidently, Blondie24's rating after those 10 games was about the same as it was on the website (an expert level). Chinook's web site has a "hall of fame" of people who have beat it at this level. Fogel's name is on that list because Blondie24 beat it. (Probably it should be Blondie24's name on the list.) Funny thing is that Fogel treated the objection about ratings on [...] directly in the book. It's "Objection #12" in the objections section on pages 315-317.

Not only did Blondie24 advance from being basically a random player into playing at the level of experts (but not masters or grand masters, and Fogel never claimed that it was a master), it achieved this without using an endgame database of moves or opening book moves. Imagine how good the combination of human knowledge and Blondie24 could be. I understand that Fogel and others have now moved on to learning in chess, achieving results that are at the grand master level.

Dr. Fogel did serious work in a true scientific manner, and the book is just fine, but potential readers should be aware that the playing strength of Blondie24 is greatly exaggerated. While it will beat the average checker player, that is no accomplishment; and to say it is an "expert" or "master" player based on ratings at some of the larger on-line sites means almost nothing.

In testing against other checker playing computer programs Blondie24 is completely undistinguished and cannot play an even match with any sort of serious checker program.

This points to the fallacy of the implied premise of the book. In fully deterministic games, neural networks to date have been a remarkable failure. In probabalistic games such as Backgammon, on the other hand, they have been an incredible success, as the world's top Backgammon programs are based on neural nets.

But there is all the difference in the world between a probabalistic game and a deterministic one and I don't believe the book "comes clean" on this.

This is not my favorite machine learning book, but Tom Mitchell did us all a favor by writing it. It covers the breadth of topics that make up the machine learning discipline fairly completely. Since this book is about completely, there is also a shallowness, but that shallowness does not trim out complete descriptions of the algorithms covered. Oh no, all the gory math is there, what isn't there are simple examples.

My first time through the book, what gave me the biggest headache was trying to understand back propagation from the algorithm pseudo code and the proof of correctness. I really really wanted one simple example at that point to make sure I understood the correct use of all the greek terms.

So good book, but I really wanted "Machine Learning Examples" to go along with it back when I first picked it up. But once you understand, the book is a great reference.

The book machine learning by Mitchell provides a systematic overview of important concepts in the field. This is rather rare finding because most books present first of all algorithms but fall short communicating systematic insights that would help the reader to creatively develop methods by themselves.


It is needless to say that any book with the title 'machine learning' is inherent incomplete due to the incompletenss of the field itself. For this reason this book is not state of the art of current algrithms. Instead, again, concepts are at the center of focus.

Overall, well writen and a very good selection of examples and explanations. I recommend this to anyone for a general overview.

This book is a redaction of many different white papers on the topic of machine learning. The material is very credible and accepted in the field, with very little (if any) temporal information (short term at least). With that said, it is also very dry and academic, and requires a solid background in mathematics to understand. Even if you are in the field, you're likely to read certain pages several times to embrace a concept... but once you embrace it, you will have some of the best foundational knowledge there is on the subject. If you're in the machine-learning field, you'll benefit from revisiting some of these subject, and probably learn a new thing or two.

I read this book about 7 years ago while in the PhD program at Stanford University. I consider this book not only the best Machine Learning book, but one of the best books in all of Computer Science. It covers every branch of ML I know of and it covers it really well. I found Mitchell's chapter on Neural Networks more insightful than an entire book on NN's that I read. I also found his chapter on Reinforcement Learning more useful and better explained than an entire book on Reinforcement Learning that I also read. The other chapters cover other ML topics at the same level of quality and rigor.

The author did an amazing job in covering the breadth and depth of ML in less than 500 pages. I hope he will write a new edition to cover the advances that happened in the last decade.

I have used this book during my masters and found it to be an extremely helpful and a gentle introduction to the thick and things of machine learning applications. The various chapters are nicely paced with helpful problems at the end. Another great thing about the book is treatment of detailed examples with each concept and that the author carefully ties various concepts as they arise, with not just new, but also examples from previous chapters, which helps the user to understand different concepts applied to same problems thereby making clear difference between different methods. Also the author has a dedicated website with updated errata and notes, which is also very helpful! Having said that, I think the book is an introduction to various machine learning methods and one can easily follow on the references listed for detailed treatment of relevant topics.

If you are interested in this topic, you should read this book. Disclaimer: I am new to the topic but appreciate when it is done well and need to understand how to implement it for my job. I was blown away by both the conceptual coverage and the implementation details. This book will allow you to cover the concepts on a first pass then come back and actually build the approaches you are most interested in. Even if you ultimately use a vendor product for recommendation, you will understand the algorithms being used and their proper application and where they are deficient.

Programming Collective Intelligence is a book about applying data mining techniques to analyse collections of data. There is submerged information in Ebay prices, in Facebook profile networks, in collections of movie reviews, in news sites, in the stockmarket; this book by Toby Segaran shows ways to extract, visualise, understand, and predict that information.

Each chapter explains and explores a different data mining algorithm, and builds up a working example in Python, while presenting different methods and parameters of the implementation. I hadn't really worked with Python before, but found the code easy to follow, and picked up some interesting Python idioms that I haven't seen in other languages before. Chapters end with a set of exercises to follow that build your understanding.

As you follow the examples you build up a reasonably generic code base that allows you to swap in and out different implementations, and reuse previous code to add to new applications.

The examples use live examples from the web: sites like Ebay, Facebook, and Yahoo Finance, and this makes the book more interesting and the results more visceral than some other books on the subject which use more contrived or obscure examples. Even though there is a strong web (or web 2.0) focus on the examples, the methods and the understanding is useful for a whole range of applications.

Some of the topics covered:

* Bayesian classifiers to detect spam, or to file news articles into site sections
* Hierarchical and k-means clustering to discover groups of similar items in massive sets
* Euclidiean distance, Pearson Correlation Coefficient, Tanimoto Coefficient: ways to measure the distance (or difference) between items
* Neural networks to predict user behaviour and improve search result ordering
* Optimisation methods like hill climbing, simulated annealing, and genetic algorithms
* Non-negative matrix factorization
* Support vector machines and kernel methods to go where linear regression can't

I found it exciting to read -- it's one of those books that give you a whole bunch of new ideas for things to build as you read it. The presentation is very good: no background is assumed, and it doesn't talk down to those more experienced.

Recommended.

I have just about finished reading this book, and I'm really enjoying it. It's loaded with great information and examples. I like how the author gives the reader tips on when certain techniques are better than others. The python examples are clear and easy to read. I'd love to see more books follow this one's style and structure.

most people have shared their thoughts on the good of this book. I like to point out some of the bad as I read through:

- first, too many typos - both the author and oreilly should do a better job on proof read the materials. the typos are so much that it can easily wreck otherwise good materials.

- second, arcane solution and coding style. Many first step to the solution of machine learning is to represent the problem at hand well. The author's brain apparently wired different from mine so the opinion is personal. For example: chapter 5 on "optimization for preference", he chose to represent a solution as vector form like [0,0,0,0,0,0,0,0,0,0], there is no way I can relate this solution to the real meaning (you want to allocate 10 students into 5 rooms each with two slots) - if there is an easy explanation, the book didn't say so.

thus the 3 star. I believe a second edition is warranted and should be much better.
just my 2c.

Once I got past the initial shock of finding several glaring grammar and spelling errors in the introduction, I have been pleased with this purchase ever since.

The author gives a good overview the many different approaches to machine language (with great examples in Python). However, it's just that - an overview. While the explanations are very clear and the concepts are presented in a very accessible manner, I found myself having to look elsewhere for more detail on the various algorithms. Yes, with the level of understanding presented in this book you should be able to create functional code for your particular data set. However, I felt that to really get the best results from the algorithms I needed to study them a bit further in order to best apply them to my data.

As a recent CS graduate, I would certainly recommend this book to anyone looking for a basic understanding of machine learning and data mining techniques.

This is a great book to begin your journey on Genetic Algorithms (GA). The author is a pioneering authority on the subject and has explained the basics of a GA in a very gentle and easy to understand manner. The book has a great variety of specific but diverse examples, which may not be useful at first glance, but gives an insight to where all the technique has been applied!

However, some aspects of the book perhaps need an edition, like the more recent advances in GA operators, specifics of chromosomal representation schemes, non-linear optimization functions, etc. I have read several, well written books on the subject, but this one has a very distinct and sometimes interesting style of writing! The best would be to quickly read this one to get a fairly good understanding of the basics and then take up a recent book that addresses other aspects like Mitchell's book, for example.

Having said that, I think the book is a great and inspiring start to using genetic algorithms.

Excellent book for Graduate students and instructors. Highly recommend!

This book gives a good introduction to genetic algorithms for a general undergraduate audience. However, it is important to note that it does not cover Evolutionary Strategies, an approach to evolutionary computing that I have found quite useful since it is specifically designed for Euclidean space optimization problems where many if not most interesting optimization problems are formulated in (take for example the problem of determining the weights of a neural network that minimizes the network's overall classification error). Nor does it cover evolutionary programming (not to be confused with genetic programming). So after reading this book, I recommend (for the mathematically adventurous) Thomas Back's "Evolutionary Algorithms in Theory and Practice: Evolution Strategies, Evolutionary Programming, Genetic Algorithms"
ISBN: 0195099710

Happy reading and enjoy the fascinating world of evolutionary computation!

OK, I agree with the previous reviewers: it's the classical textbook for GAs. But it definitely needs updating, as it's a 15-year old book and much has been done in the area. Niching methods, for example, are just outlined. I'd recommend Melanie Mitchell's book instead of this one.

I agree with another reviewer who said the book was unnecessarily long. Genetic Algorithms are a great programming tool, and there are some tips and tricks that can help your programs converge faster and more accurately, but this book had a lot of redundant information.

If you are interested in using GA for solution-finding, I doubt you'll find much useful in this book beyond the first chapter or so. Many of the examples later in the book were so specific that I couldn't see how they could be usefully generalized. Really optimizing a GA approach for a specific problem domain takes a fair amount of tuning, and this book won't help much with that.

I think time spent surfing siteseer or other publication sites would be better spent than reading this book.

This book helped me accomplish what I set out to do; namely to obtain an overview of the field of natural language processing, with an emphasis on language understanding (as opposed to recognition). And I can recommend it on that level. The weakness of the book however is that it left me asking, "OK, now what?". The book started off strong with a number of dynamic-programming algorithms, finite automaton models, and N-grams that one could sink his/her teeth into from an algorithmic point-of-view. But when it came to actual techniques for natural-language understanding (chapters 14-17) the goods were not delivered. The algorithms disappeared, and the best I could find was in Chapter 15 an incomplete, and unconvincing treatment of Hiyan Alshawi's semantic parsing techniques which fueled the Core Language Engine last decade. Chapter 16 dealt with lexical semantics and was almost entirely devoid of algorithms.

My gut feeling after reading this text is that parsing techniques will likely give way to statistical and probabilistic learning methods that will in some sense bypass the need to correctly or accurately parse language. I cannot fault the authors for not exploring this in more depth,as this represents the cutting edge for both NLP and artificial intelligence. In any case, I'm off to read Schutze and Manning's book which will hopefully provide a bit more focus on that perspective. What intrigues me is that most people can understand some language, but very few people understand the grammar of their own language, especially if they have been deprived of a formal education. So why should computers need to know all about grammar rules and parsing? Could they instead be trained by simply being exposed to enough interactions between language and objects? I teach in a department dominated by both foreign and immigrant students. I understand them most of the time, but I would estimate that half the time their sentences or utterances would not fail to be parsed correctly.

This book is a great general introduction to NLP, covering a broad range of topics. Unfortunately there are many errors in the mathematical formulae and the algorithm descriptions, so do make sure to download the errata list from the book's home page.

something which I can use - I am a linguist - and found it immensly readable and useful

GENERAL IDEA: Broad coverage, it lacks depth and details - particularly practical details. That is, the presentation is often sketchy, mainly because it approaches too many subjects for its available space. I would not say that this book is strong on theory either. It is quite obvious that it avoids getting too formal and precise, probably to remain attractive for non-specialists too.

CASE STUDY: One specific problem I had with the Hidden Markov Models, that are supperficially presented (or spread I could say) in several separate sections of the book, so it's not been a pleasure trying to actually understand them properly and completely as a fundamental concept, to make them work in my particular application.

TITLE: The book's title IS misleading because it starts with "Speeech" and this book's main subject is not speech but (written) language. Actually there are only a few chapters on speech.

CONCLUSION: Get this book if you are looking for a good overview of the field. The book will introduce you to a thousand of topics. As soon as you need in-depth coverage of some particular topic, you will look for additional resources.

This book is by now an accepted classic in the field. It is basically the only textbook that covers so much of computational linguistics, so I have had no choice but to use it for the past several years. Just the same, I'd rather not use it for teaching linguistics students. While the book has much to offer the professional, including a broad range of topics extensively researched, it is much more useful in this "handbook" capacity than as a textbook for the uninitiated. The chief reasons for this are: 1) It is pedagogically very poor; the majority of concepts are either explained in a confusing and obfuscatory manner or are not explained and are simply left in algorithmic form. This is not usually edifying to the linguistics student with no computer science background. 2) There are too many mistakes in its algorithms and method overviews. So far as I can see, even the famed Earley parsing algorithm is wrong here, it will not yield the correct output. 3) It is not written in a language that linguistics students can understand. With no background in mathematics, computer science, or pseudocode, such students need much more coddling than is provided by this book, and they are virtually unable to read it. Basically, as the title to this review states, what is called for now is a book to explain the contents of this book. Perhaps if my students keep encouraging me to write it. . .

I agree with reviewer Frank "Good introduction but not well structured, May 8, 2005" the authors are over-anxious to establish the credentials of RL in older research traditions. Much of the talk about optimal control for instance is confusing because this is a vast field and its assumed you know it. I found myself looking up some of the technical terms from other fields. In the end learning about these concepts didnt help my understanding. This is a pity because the concepts behind RL are relatively simple/

However in general I really enjoyed this book and this is the most accessible (while still being comprehensive) RL introduction out there.

Reinforcement Learning is an exceptionally clear introduction to a field that also goes under names such as approximate dynamic programming, adaptive dynamic programming and neuro-dynamic programming. The book is written entirely from the perspective of computer science, where problems tend to have discrete states (although potentially large state spaces) and (typically) small action spaces.

The book provides numerous step-by-step algorithms that makes it relatively easy to get started writing algorithms. The presentation uses minimal mathematics, and avoids the difficult theory supporting the convergence proofs, making it a nice introduction for undergraduates and graduates alike. But throughout the presentation is evidence of extensive experience with applying these methods to a range of classical problems in artificial intelligence.

Students interested in a stronger theoretical foundation should look at Neuro-Dynamic Programming (Optimization and Neural Computation Series, 3). My recent book, Approximate Dynamic Programming: Solving the Curses of Dimensionality (Wiley Series in Probability and Statistics), puts far more emphasis on mathematical modeling, and presents the field more from the perspective of the operations research community. For an edited volume with a number of contributions from both artificial intelligence and control theory, see Handbook of Learning and Approximate Dynamic Programming (IEEE Press Series on Computational Intelligence).


Warren Powell
Professor
Operations Research and Financial Engineering
Princeton University

As a subfield of artificial intelligence, reinforcement learning has shown great success from both a theoretical and practical viewpoint. Taking the form of numerous applications in finance, network engineering, robot toys, and games, it is clear that his learning paradigm shows even greater promise for future developments. The authors summarize the foundations of reinforcement learning, some of this coming from their own work over the last decade.

The authors define reinforcement learning as learning how to map situations to actions so as to maximize a numerical reward. The machine that is indulging in reinforcement learning discovers on its own which actions will optimize the reward by trying out these actions. It is the ability of such a machine to learn from experience that distinguishes it from one that is indulging in supervised learning, for in the latter examples are needed to guide the machine to the proper concept or knowledge. The authors emphasize the "exploration-exploitation" tradeoffs that reinforcement-learning machines have to deal with as they interact with the environment.

For the authors, a reinforcement learning system consists of a `policy', a `reward function', a `value function', and a `model' of the environment. A policy is a mapping from the states of the environment that are perceived by the machine to the actions that are to be taken by the machine when in those states. The reward function maps each perceived state of the environment to a number (the reward). A value function specifies what is the good for the machine over the long run. A model, as the name implies, is a representation of the behavior of the environment. The authors emphasize that all of the reinforcement learning methods that are discussed in the book are concerned with the estimation of value functions, but they point out that other techniques are available for solving reinforcement learning problems, such as genetic algorithms and simulated annealing.

The authors use dynamic programming, Monte Carlo simulation, and temporal-difference learning to solve the reinforcement learning problem, but they emphasize that each of these methods will not give a free-lunch. An entire chapter is devoted to each of these methods however, giving the reader a good overview of the weaknesses and strengths of each of these approaches. The differences between them usual boil down to issues of performance rather than accuracy in the generated solutions. Temporal difference learning in fact is viewed in the book as a combination of Monte Carlo and dynamic programming techniques, and in the opinion of this reviewer, has resulted in some of the most impressive successes for applications based on reinforcement learning. One of these is TD-Gammon, developed to play backgammon, and which is also discussed in the book.

The authors emphasize that these three main strategies for solving reinforcement learning problems are not mutually exclusive. Instead each of them could be used simultaneously with the others, and they devote a few chapters in the book illustrating how this "unified" approach can be advantageous for reinforcement learning problems. They do this by using explicit algorithms and not just philosophical discussion. These discussions are very interesting and illustrate beautifully the idea that there is no "free lunch" in any of the different algorithms involved in reinforcement learning.

In the last chapter of the book the authors overview some of the more successful applications of reinforcement learning, one of them already mentioned. Another one discussed is the `acrobot', which is a two-link, underactuated robot, which models to some extent the motion of a gymnast on a high bar. The motion of the acrobot is to be controlled by swinging its tip above the first joint, with appropriate rewards given until this goal is reached. The authors use the `Sarsa' learning algorithm, developed earlier in the book, for solving this reinforcement learning problem. The acrobot is an example of the current intense interest in machine learning of physical motion and intelligent control theory.

Another example discussed in this chapter deals with the problem of elevator dispatching, which the authors include as an example of a problem that cannot be dealt with efficiently by dynamic programming. This problem is studied with Q-learning and via the use of a neural network trained by back propagation.

The authors also treat a problem of great importance in the cellular phone industry, namely that of dynamic channel allocation. This problem is formulated as a semi-Markov decision problem, and reinforcement learning techniques were used to minimize the probability of blocking a call. Reinforcement learning has become very important in the communications industry of late, as well as in queuing networks.

This book is undoubtedly the standard book on the topic of reinforcement learning by the two leading researchers in this field. Different from many other AI or maching learning books, this book presents not only the technical details of algorithms and methods, but also a uniquely unified view of how intelligent agents can improve by interacting with the environment. Besides, it is very readable, without much math or theory. The exercises are challenging and interesting, and will force you to understand the stuffs in the book!

This book provides an easy to read introduction in reinforcement learning. It covers several approaches (dynamic programming, monte carlo, temproal differnce) and gives a lot of examples.

However, in my opinion it is neither well structured nor written. The book has no clear separation between theory and examples given to demonstrate the applications of the theory. Due to this, the theoretical ideas are blured instead of clearified. After going through the examples it is always possible to find out how it work, but this should not be necessary.

After reading this book you will definetely know the basics (even more) about reinforcement learning. However, I somehow expected more because of the names of the authors. Perhaps this is not only a problem of this book but of the field of reinforcement learning itself.

Only buy this book if you want a very technical book about this subject. I bought this book because I was generally interested in this research field... and I never read it. If you are a researcher or a student studying this field, then this might be a good book. Otherwise, there are books that you will probably enjoy more.

This is a good book for people who are interested in computational linguists, machine
learning experts who are looking for new application domains and in general for someone who wants an introduction to statistical computational linguistics.

The book is self contained and very well written. It treats most of the general statistical approaches to language processing such as language models, smoothing, etc.. in an excellent, but introductory manner. The book is a good start for any one looking to enter statistical nlp, however for advanced readers who would like to see the cutting edge of statistical computational linguistics they should look somewhere else.

I have never received anything so quick buying off the internet. Few days and I had the book in my hand. I was pleasantly surprised.

this is an import pre-req to any research/inquiry into this field.

Compared to the slightly overrated Jurafsky and Martin's classic, this book aims less targets but hits them all more precisely, completely and satisfactory for the reader. That is, just to give you an idea on what to expect, instead of attacking 200 problems on 2 pages each, this book attacks only 40 problems on 10 pages each.

So, read the TOC before you buy the book: if you find your topics there, you're done, you are saved, buy it and be happy. In contrast, you can buy Jurafsky's book without caring to read the TOC: your problem is likely to be mentioned there but it's quite unlikely to be detailed enough to satisfy you.

Some introductory chapters take too much space and some advanced topics are missing. But the book is actually named "Foundations of..." so it seems to deliver precisely what it promisses, which is a precious and rare accomplishment by itself. I recommend this book.

This book The Hockey Machine is about a kid named Steve. After an ordinary game Steve gets kidnaped but he doesn't know it. The kidnapers[Kenneth] took him to a nice dinner. Kenneth takes Steve to what looks like a hockey camp. After a few weeks, Steve wants to get out and go home. He tries to escape, but guards catch him. He then has to sit in a cell for 48 hours. Now he tries to escape again, and he gets to a phone booth. Then Steve remembers his uncle lives in the area, so he calls him. His uncle finds him immediately. Next Steve finally goes home.
I thought this was one of the best books because I like mystery books.

By: Tyler

The Hockey Machine is about this kid who is taken to the Indians to play hockey. So, if you want to find out what happens to Steve, read The Hockey Machine!

This book is a good book, I recommend it to others. It has adventures and is about hockey.
If you like hockey, adventure or mystery's you would like this book.
It is a fairly easy book to read and not difficult to follow the story line.

This book The Hockey Machine by Matt Christopher is a verry interesting book to the people who really like hockey. In this book Steve a starting center was kiddnapped by his coach of the chariots. He thought his dad signed the paper but Mr agagard forged his fathers name.He played as much as 13 games until something suprises him. I recomend this book to the people who are into hockey and the people who like hockey because this book is perfect. If I like this book you will too.

By: Matt Christopher

Have you ever been away from your parents or someone you loved for a long period of time? Well if you have I bet you don't like it. In the book "The Hockey Machine" by Matt Christopher, the protagonist, Steve, plays ice hockey for hockey team called the Bobtails. Steve lives with his parents, or used to anyway. One day after hockey, just as Steve was leaving, he heard a voice say, "Hi Steve, I'm Mark," he announced. Steve did not know who this was.
Then Mark added, "Me and my older friend have been watching you play hockey". " Come on he is in the car waiting to meet you." Marks older friend is Kenneth. Kenneth is a coach of a ice-hockey team in Indianapolis. When Steve and Kenneth met they talked, and after that Kenneth asked Steve if he wanted to go and get something to eat for lunch. Steve told him no because he had to get home. Kenneth said that he would take him home right after they ate. Without saying anything Mark pushed Steve in. Steve got in and Steve, Mark, Kenneth, and the driver all went out to lunch.

On the ride back to Steve's house, Steve fell asleep. When Steve awoke he was still in the car. Then he asked in a worried voice, " Where are we going?"
Kenneth answered, "To Indianapolis."
Steve shrieked, "What, you said that you would take me home right after lunch." Kenneth ruled, "You're going to play on my hockey team, the Chariots.
I already asked your parents when you were asleep."
Steve said "Prove it." Kenneth pulled out a letter and gave it to Steve.
The letter was typed out. Steve thought in his head that anybody could have typed this letter. After Steve was finished reading the letter he finally questioned, " how come I didn't even get a chance to say good-bye to my parents?" No one answered.
As soon as they reached the airport they got on to an airplane to fly to Indianapolis. After they got there Steve met the hockey team. They stayed in their own hotel. After Steve met the guys he asked Mark if Mark would talk to Kenneth because Steve was already feeling home sick. Mark told him "quit thinking about your home. It just makes you more worried."
Then he advised "all of the guys and I did." Steve just bowed his head in sadness, and all he could do was think about home! If you've ever felt like Steve has, all you want to do is go home and see your parents. See what happens to Steve if you get the book The Hockey Machine