Neural Networks Books

I can't believe I waited so long to read this book -- it truly is a classic. This is the way AI should be done: by focusing on the right level of abstraction, situated above the level of neuroscience but below the level of simple input-output function mapping. Finally, a computer model that makes those first steps towards doing the same thing that people do.

True believers in those original goals of artificial intelligence take heart -- this book gives new hope to a field that has come to be dominated by engineering approaches that only work in special cases like the logic behind the cruise control switch in a car. Mitchell's model provides the fluidity and flexibility that is lacking from classical machine learning techniques.

Melanie Mitchell's analogy-making as perception is a remarkably original book. It documents an artificial intelligence project known as copycat, which was implemented as the author's PhD project under Douglas Hofstadter.

Copycat is unlike anything in artificial intelligence. It is not a symbolic system, neither a connectionist one. The major goal of the project is to study the nature of concepts. Concepts, as we all know, are flexible, context-sensitive creatures. For instance, DNA has nothing to do with a computer program, but there is a sense on which we can see DNA as a computer program that guides embrionary development. DNA can also be seen as a zipper, as it "zips down" in two parts (for cell reproduction). Still another view would be DNA as a will, for it carries valuable hereditary "property". Now, DNA is in truth just a molecule, and nothing else. The question is, how can we see the same thing (such as DNA) as so many different things? Moreover, how can these fluid context-sensitive concepts be implemented in rigid, rule-obeying computers?

To which the answer is: what we view is the abstract roles that DNA plays in embrionary development, cell division, and in individual reproduction. And this is the very idea of "Analogy-making as perception".

Well, not so fast. The copycat project is not designed to grasp such extremely complex subjects as DNA, but, on the other hand, it presents a computational architecture that suggests what the nature of concepts is like, and how flexible concepts may emerge from inflexible mechanisms.

Copycat can solve analogy problems such as abc->abd:ijk-> ?. But it is not restricted to trivial ones. Consider the following analogy: abc ->abd:xyz->?. How would you solve it? How do you think that copycat solves it?

Obviously, this project doesn't fit in very easily in classical artificial intelligence, as it attacks some of the most pervasive ideas of the field, such as the separation of perception and cognition. In fact, I think this book redefines the major questions of artificial intelligence (and although Mitchell does not state it, I think the copycat model does not fall prey to either the frame problem or to the symbol grounding problem).

It is very unfortunate that this is not one of the best-selling books in AI. But I believe that it will ultimately make its mark on the History of the field, if for no other reason than it simply is the right approach to genuine intelligence and authentic understanding.

Should one day Amazon.com let me give a 6-star to a book, but charge me a dollar for giving it, this is one that would definitely deserve to be such a 6-star.

============================================

PS. I would also recommend Hofstadter's Fluid Concepts and Creative Analogies; and Robert French's Subtlety of Sameness.

Since AI researchers are generally engineers, they historically did what engineers do: they broke up the mind in very clear-cut divisions, one for the perception of the things out there in the world, and another, symbolically, to do "abstract cogitation".

For deep reasons, this was an invalid move, but only a few could see it. Melanie surely could, for her highly original copycat project exhibits some of the best insights in Artificial Intelligence ever.

AI is still so much pervaded with the wrong ideas that this book will need to take some time to make its definitive mark on the history of the field.

If genuine understanding is ever to be built into a machine, understanding of the kind that Searle's gang will be forever denying, then it will come from an architecture similar to that proposed in this book.

Then again, I could turn out to be wrong. But let us let History decide on this issue.

Our consulting firm has successfully collaborated with Dr. Powell for years and I have seen first hand how ADP solves large scale, real world problems that would frankly be intractable by many traditional traditional operations research or optimization techniques. While consulting firms and other business jealously guard their intellectual property, it is terrific for all of us that academics are rewarded for precisely the opposite. I would highly recommend for any serious practitioner to grab a copy of this book and study it. Probably one of the best $100s you will have spent in a while.

This book represents a paradigm shift in the presentation of dynamic programming/stochastic optimization. Classical treatments of dynamic programming/neuro-dynamic programming/reinforcement learning typically assume small "action spaces," and often assume the presence of a one-step transition matrix. By contrast, authors working with decision vectors in the presence of uncertainty often turn to stochastic linear programming. But these techniques typically struggle when applied to multistage applications. It is extremely hard to solve most of these problems without taking advantage of the presence of a state variable that captures previous history.

I have adopted the notational style where S is the state of the system, and x is a decision, using the language of math programming. x may have many thousands of dimensions for some problem classes (although the book considers many classical problems where decisions are relatively simple).

The challenge that arises when x is a vector when we use dynamic programming is the expectation within the max/min operator. Bellman's equation is typically written

V(S_t) = max (C(S_t,x) + discount * E{V(S_{t+1})|S_t} )

If x is a vector, we generally need the power of math programming to solve the maximization problem. The challenge is the expectation. We avoid this using the post-decision state variable, which is the state immediately after we have made a decision, but before any time has passed (bringing new information). Denoted S^x_t, the post-decision state variable is a deterministic function of S and x. If V^x(S^x_t) is the value function around the post-decision state variable, we obtain

V(S_t) = max (C(S_t,x) + discount * V^x(S^x_t)

The book provides a number of practical examples of this, but the key is that the maximization problem is now a deterministic problem. The final step is that we have to replace V^x() with a suitably chosen approximation. If our maximization problem is a linear, nonlinear or integer programming problem, we have to choose an approximation for V^x() that allows these algorithmic tools to be used.

In this book Warren nicely blends his practical experience in modeling and solving complex dynamic and stochastic problems occurring in a variety of industries (transportation, the financial sector, energy, etc) with algorithmical and theoretical aspects of approximate dynamic programming. The book can be either used as a textbook in undergraduate or graduate courses, or for practitioners to learn about recent advances in this exciting area. Indeed, I have already used it twice as a textbook for a graduate course, and on the other hand, I have recommended it to several practitioners. Without doubt, this is an important contribution in approximate dynamic programming.

I strongly recommend the book for all practitioners facing large-scale complex dynamic programs. It is also an excellent textbook.

Very impressed. I am glad I read it again after all these years. The story of the Mad Man is very funny. I was cracking up while reading it.

Did you know that free schooling was only briefly offered in Nigeria? There's a poignant story about it here.

I learned a lot about Nigeria from these stories. Sometimes, the stories seemed to end a little too abruptly, but I guess that's part of the story format: it has to end sooner than a short novel, anyway.

Mr. Achebe is a fine storyteller and he has many interesting things to say about the people and customs of Nigeria. I recommend this book, but only after first reading his classic novel about 19th century Ibo tribe people, Things Fall Apart.

After reading these stories, I was both attracted to Nigeria and repelled by it (I've never been to Africa). Achebe does a good job of capturing the ambivalence aroused by Nigeria's exotic nature (to Americans) mixed with its societal dysfunctions.

Diximus.

This is an impressive collection of short stories that covers a twenty-year period of Achebe's writing. They also cover a period of history in his native Nigeria that spans from the late colonial period to the Biafran war. In them Achebe explores various aspects of a predominant theme in his work, i.e. tradition vs. modernism in his country (as introduced by British colonial administration). The various stories offer glimpses into the lives of people from various classes and walks of life. Achebe has a concise and eloquent writing style; he has an almost singular talent for making very pertinent observations in an extremely pithy fashion. Thus, for example, in the few pages of a story like "Dead Man's Path," Achebe brings to life the problems which ensue from the drive for quick modernization, the desire to adhere to tradition and the hypocrisy of Nigeria's colonial administrators. Also impressive is Achebe's mastery of narrative styles, i.e. first person, omiscient, etc. These stories can be read on their own, or as a supplement to Achebe's similarly powerful novels.

Vitalism is a profoundly science-ejected concept, though many CAM or 'natural health' cabals falsely claim that vitalism survives scientific scrutiny.

One of my favorite passages from this book:

"the achievements of molecular biology in the twentieth century proved conclusively that it is not necessary to propose that life processes arise from some nonmaterial vital principle and cannot be explained entirely as physical and chemical phenomena. [E.g.] biological neural networks are created by nature, and the laws of nature should be applicable to them [p.003]."

-r.c.

Karl A. Greene in the foreword asserted that after reading this book, one will never look at neurobiology and the human brain quite the same again, and I fully concur. Baev introduces a modular framework that fuses neurobiology with control theory and opens the portals for artificial intelligence to enter. Unleashing the powers of hierarchical modeling, his monograph presents a thoroughly conceptualized and truly captivating approach to understanding the functioning of the human brain. Two thumbs up, I had to read this book five times to fully understand it, but I enjoyed it every single time.

Karl A. Greene in the foreword asserted that after reading this book, one will never look at neurobiology and the human brain quite the same again, and I fully concur. Baev introduces a modular framework that fuses neurobiology with control theory and opens the portals for artificial intelligence to enter. Unleashing the powers of hierarchical modeling, his monograph presents a thoroughly conceptualized and truly captivating approach to understanding the functioning of the human brain. Two thumbs up, I had to read this book five times to fully understand it, but I enjoyed it every single time.

Kosko done a nice job by bring in the best fuzzy application design potentials by telling us what is fuzzy (additive fuzzy system) and why fuzzy, get it for yourself, if your are working on fuzzy system.

This book is great, it covers from "What is Fuzzy" to "Fuzzy Chaos" ... in the book you can find all the information needed ...

Kosko done a nice job by bring in the best fuzzy application design potentials by telling us what is fuzzy (additive fuzzy system) and why fuzzy, get it for yourself, if your are working on fuzzy system.

What strikes me each time I open this book is Mr Kecman's sense of pedagogy: it is a lesson in the matter. Not only his book delivers the - sometimes complex - techniques in a highly readable manner, but the concepts behind each of the main tools (SVM, NN & FL) he chose to highlight are always brilliantly put in context. One comes out of the reading with more than a set of equations but rather with a clearer picture of the field.
Mr Kecman is - without a doubt - a great teacher.

This effort to deliver a clear message is furthermore underlined through the numerous original figures: if you are like me and feel that a (good) picture speaks more than a thousand words, you will sure appreciate the way the illustrations complement the text and truly help the understanding.

I have read several other books on the subject but if I had to chose one for teaching purposes, this would be the one. I you want to build a better understanding of the field, get this book: it will pay on the long term.

This is a very good book. Another reviewer has commented on Vojislav Kecman being an excellent teacher. I whole-heartedly second that opinion. Often times, while reading this book, you will pause with a doubt or question. What you will find surprising is that almost certainly the author has answered that question in the next paragraph. Many times, the author's answers will tally your own answers.

The first chapter of the book (entitled: Learning and Soft Computing: Rationale, Motivations, Needs, Basics) is 119 pages long. It is an essential reading. By the time you finish reading this chapter the things will start falling into place and you will be more motivated and ready to read the remaining chapters. Until you are highly aware of this topic, do not skip this chapter.

A book is made up of a lot of things other than the text that it covers. Does it contain many/any stupid jokes? Is it printed on the highest quality paper? Is the font size good? Is it printed too dense? Is the cover page inviting enough? Are the dimensions/weight of the book correct? On all these counts the book scores high.

Consistent with the subject matter that it covers, this is not an easy book. You will perhaps like to read it with paper and pencil. But if you are willing to spend time with this book, this book will do a lot of good to you. This is a very good book.

This book is a nice and, I would say, a successful attempt to provide a unified survey of important theoretical and practical machine learning tools: neural networks (NN), support vector machines (SVM) and fuzzy systems (FS).

Book consists of nine chapters, covering SVMs, one- and multi-layer perceptrons and radial-basis function networks, as variants of neural networks, and basics of fuzzy theory. This is followed by interesting case-studies (in financial, control and computer graphic applications) and concluded by basics of optimization theory and an overview of necessary mathematical tools. All the MATLAB programs needed for the simulated experiments are available on the book web site.

Authored by Vojislav Kecman, a prominent researcher in the field of soft computing and previous MIT visiting professor, this book is an excellent material for advanced undergraduate and introductory graduate courses in machine learning applications and soft computing....

Anthony Zaknich wrote a book that provides the reader with a very broad knowledge about neural networks especially for signal processing. Fundamental facts are extracted and presented in a form that is very easy to read, such as listings, keywords, main formulas, diagrams and results from experiments. Zaknich avoided going into too much theoretical details which might only confuse beginners. But also for advanced and professionals his book is a good source of well-structured basic knowledge. This is the difference compared to other books about neural networks since the reader finds the essence that is crucial to understanding the basics. A huge selection of literature references enables the reader to quickly continue deeper studies about interesting topics. Furthermore, Zacknich' provides examples and results from experiments and simulations which he has done by himself for the most parts. He compares different neural network types and configurations for various real-world examples. These comparisons and results were a very valuable help for me in order to understand the differences and characteristics among the vast number of various network types.
Finally, Zaknich's book helps me very much in my job for that it is very important to have application results and "benchmark-like" comparisons. His explanations are often with reference to his own developed Modified Probabilistic Neural Network (MPNN) and Advanced MPNN that animated me to carry out some tests based on his MPNN. Another idea in his book is the Integrated Sensory Intelligent System (ISIS) that he introduces. I would highly appreciate if Zaknich releases new books especially on the field of signal processing and his idea of the ISIS.
Zaknich knows how to guide the reader through the different topics and not to bore him by long theoretical sequences and formulas. There is already a wide range of available literature that explains topics, such as classifications, control systems, robots etc. in more detail. Interesting quotations from different sources of literature attached to each chapter makes the book even more than merely a scientific book.

Claas Richter
Silicann Technologies (Japan/Germany)

Zaknich's book provided me with the necessary theory and detail to enable me to develop a neural network application as part of my phd. I thoroughly recommend this text for anyone wishing to develop and use neural networks, particularly for engineering applications. I found that while most books focussed on either the theory of neural networks or selected applications, Zaknich's text provided a comprehensive coverage of both theory and application providing a sound basis for understanding and applying neural networks. Topics ranged from the applicability of neural networks through to data collection, data conditioning and the final implementation.

***** Stanley McGibney, Consultant Mechanical Engineer

I found this book to be significantly different in its treatment of neural networks for signal processing and pattern recognition. It deals very ably not only with essential theory but also with basic practical issues, often missing from other books on the subject, that significantly enhance understanding and application. Zaknich has included a nice guide and design approach to successful application of neural networks, which is supplemented by frequent tips and a variety of worked application examples.

The book is much more than a good introduction to neural networks. It also includes a class of neural networks that Zaknich has developed and worked on over a decade that he refers to as common bandwidth spherical basis function neural networks. This is based on a generalization called the Modified Probabilistic Neural Network (MPNN) that encompasses Donald Specht's Probabilistic and General Regression Neural Networks. He has continued to develop the MPNN in a number of very practical directions that allows it to used for a wide range of engineering problems. He seems to favour applications related to underwater acoustic signal processing but the methods and approaches that he offers are suited to many other non-linear problems found in engineering and other disciplines.

The book includes a very interesting discussion on intelligent signal processing. Zaknich talks about what he calls hyperspace signal processing in the context of the MPNN and other classical filtering structures that gives an interesting view of some of the basic issues involved. He suggests at least one possible generic approach to non-linear signal processing based on Vapnik's Support Vector Machine that has a structural similarity to the MPNN.

This book is a gem that shines in its clarity beyond many other books on neural networks that I have struggled with in an attempt to understand the subject well enough to apply it.

very well written, easy to understand, walks you through the logic of each part of each equation. builds up more and more complex models based upon the previous models. You'll learn a lot of practical neurobiology stuff other than just modeling too.

I have used this book as an introduction and reference book for modeling neurons since I started my thesis work in computational neuroscience two years ago. It covers various types of spiking neuron models (e.g. Hodgkin-Huxley, Morris-Lecar, Integrate&Fire, Spike-Response-Model), noise in neuron models, population models, and plasticity/learning.
It is a very useful book, clearly written and comprehensive, providing sufficient detail and background information. Derivations of the equations are clearly presented and understandable to anyone with a decent knowledge of mathematics. A degree in physics is not required in order to read this book ;-) With this book and some programming skills, one has a solid foundation for modeling neurons on various levels.
I also like the literature recommendations at the end of each chapter, they give a good overview over important original papers and further reviews.
I would strongly recommend this book to undergraduate and PhD-students in computational neuroscience, as well as to anyone interested in modeling neurons.

This is a very impressive book. It covers in a systematic manner a broad portion of the field of theoretical neuroscience. It covers topics from models of single spiking neurons, through networks of interconnected neurons and up to neuronal plasticity. This book is also written very well. The style of this book reflects the background of the authors as Physicists; it therefore strives for simplicity wherever possible.
I used chapters from this book as a basis for some of my lectures in a course I teach: Introduction to Theoretical/Computational Neuroscience, a graduate level course. I especially liked the systematic approach they have adopted for describing various simplifications of the Hodgkin-Huxley equations.

This book is an excellent reference book on current approaches to some of the foundational questions in artificial intelligence, philosophy of mind, and engineering of intelligent systems. The editors as amply demonstrated by their other publications, have the rare ability to interrelate an amazing diversity of perspectives on Artificial Intelligence - from symbolic methods to neural networks and evolutionary apporaches, to uncover the shared principles and common foundations, and show how different paradigms can be brought together in synergistic ways to advance our ability to design and analyze intelligent systems. This book includes chapters written by some of the leading experts in the field. The book should be especially useful as a reference to graduate students and researchers interested in artificial intelligence, machine learning, cognitive science, software agents, and philosophy of mind. It should also be useful as supplementary reading for a graduate or upper level undergraduate course in artificial intelligence or cognitive science, or as a primary text for a seminar course. All in all, a great book!

Symbolic (e.g., logic based) and connectionist (e.g., neural network) models are often viewed as separate, and perhaps incompatible approaches to artificial intelligence and cognitive modelling. Far too many young researchers (including myself) have been swept away by the exaggerated claims of each camp. This collection of chapters is an excellent medicine for such a malaise. The chapters in this collection demonstrate, with pursuasive theoretical and philosophical arguments as well as empirical data that symbolism and connectionism can, and perhaps should, be reconciled. At the time this book came out, this was not a popular position. However, recent developments in AI and cognitive science have more than vindicated the views expressed in this book. Now (6 years later), some of the chapters are a bit dated. However, I recommend the book strongly to those who are interested in foundational issues in artificial intelligence and cognitive science.

This book laid the foundations of integrative neuroscience independent of silly metaphors and reductionistic assumptions. This in turn led to a new understanding of the functioning of the brain by showing that integration is feasible. The contents dwell on the biophysical properties of neural networks. A highly recommended book!

The contents dwell on the biophysical properties (e.g. dendritic structure, ionic channels, and synaptic transmission) of neural networks. The overall motivation is to show that biophysics of computation is an inconsistent paradigm when dealing with neural networks by showing that integration from microscopic, mesoscopic and macroscopic levels is feasible. Therefore this book laid the foundations of integrative neuroscience independent of silly metaphors. Chapters 1-2 constitute an introduction to the subject and a historical survey of neuronal modeling, respectively. Chapters 3-5 encompass the cellular level with an emphasis on nonsynaptic transmission, and computational models of single neurons from the retina and amygdala. Chapters 6-7 characterize the physico-chemical mechanisms underlying synaptic plasticity in biochemical networks and synaptic transmission in axonal networks. Chapters 8-10 form the hard core of basic theory of small-scale biophysical neural networks.Chapters 11-12 deal with mesoscopic theories of hippocampal neural networks. Chapter 15 serves as an advanced overview of the numerical methods in neural modeling of networks.A number of problems are included at the end of each chapter to give a more in-depth appraisal of the contents, and each chapter ends with some predictions in relation to future developments. A highly recommemded book!