Neural Networks Books

Uniting information theory and inference in an interactive and entertaining way, this book has been a constant source of inspiration, intuition and insight for me. It is packed full of stuff - its contents appear to grow the more I look - but the layering of the material means the abundance of topics does not confuse.

This is _not_ just a book for the experts. However, you will need to think and interact when reading it. That is, after all, how you learn, and the book helps and guides you in this with many puzzles and problems.

I am a PhD student in computer science. Over the last year and a half this book has been invaluable (and parts of it a fun diversion).

For a course I help teach, the intoductions to probability theory and information theory save a lot of work. They are accessible to students with a variety of backgrounds (they understand them and can read them online). They also lead directly into interesting problems.

While I am not directly studying data compression or error correcting codes, I found these sections compelling. Incredibly clear exposition; exciting challenges. How can we ever be certain of our data after bouncing it across the world and storing it on error-prone media (things I do every day)? How can we do it without >60 hard-disks sitting in our computer? The mathematics uses very clear notation --- functions are sketched when introduced, theorems are presented alongside pictures and explanations of what's really going on.

I should note that a small number (roughly 4 or 5 out of 50) of the chapters on advanced topics are much more terse than the majority of the book. They might not be of interest to all readers, but if they are, they are probably more friendly than finding a journal paper on the same topic.

Most importantly for me, the book is a valuable reference for Bayesian methods, on which MacKay is an authority. Sections IV and V brought me up to speed with several advanced topics I need for my research.

This is fantastic book. Really takes an intuitive approach to the material. The explanation of occam's razor is worth the price of the whole book. Highly recommended.

I find it interesting that most of the people reviewing this book seem to be reviewing it as they would any other information theory textbook. Such a review, whether positive or critical, could not hope to give a complete picture of what this text actually is. There are many books on information theory, but what makes this book unique (and in my opinion what makes it so outstanding) is the way it integrates information theory with statistical inference. The book covers topics including coding theory, Bayesian inference, and neural networks, but it treats them all as different pieces of a unified puzzle, focusing more on the connections between these areas, and the philosophical implications of these connections, and less on delving into depth in one area or another.

This is a learning text, clearly meant to be read and understood. The presentation of topics is greatly expanded and includes much discussion, and although the book is dense, it is rarely concise. The exercises are absolutely essential to understanding the text. Although the author has made some effort to make certain chapters or topics independent, I think that this is one book for which it is best to more or less work straight through. For this reason and others, this book does not make a very good reference: occasionally nonstandard notation or terminology is used.

The biggest strength of this text, in my opinion, is on a philosophical level. It is my opinion, and in my opinion it is a great shame, that the vast majority of statistical theory and practice is highly arbitrary. This book will provide some tools to (at least in some cases) anchor your thinking to something less arbitrary. It's ironic that much of this is done within the Bayesian paradigm, something often viewed (and criticized) as being more arbitrary, not less so. But MacKay's way of thinking is highly compelling. This is a book that will not just teach you subjects and techniques, but will shape the way you think. It is one of the rare books that is able to teach how, why, and when certain techniques are applicable. It prepares one to "think outside the box".

I would recommend this book to anyone studying any of the topics covered by this book, including information theory, coding theory, statistical inference, or neural networks. This book is especially indispensable to a statistician, as there is no other book that I have found that covers information theory with an eye towards its application in statistical inference so well. This book is outstanding for self-study; it would also make a good textbook for a course, provided the course followed the development of the textbook very closely.

I have used this to get a good background in the topics covered, especially inference theory, and in general I found it to be great book which fills a market gap. The only sins I see are sins of omission. I personally would have enjoyed seeing a more task driven organization. I seem to need these methods periodically but I never seem to need the same method twice. Also, many of the techniques are heavily iterative, i.e., monte carlo, neural networks, etc. This is fine but much of what I do is in the context of simulations where 100,000 step iterative methods don't work so well because of resource constraints. Historically, that has been the problem with many of these methods. They are useful for relatively small domains but don't necessarily work that well for "real" problems. That is probably why more task oriented books are not available. Of course the author is following the outline of the current research into the subject manner which in turn is largely determined by "interesting" and "doable" problems. The real progess in this field will come when the problems are formulated more by what is needed in the nontraditional domains of application. A good example of a useful compression (and identification in some cases) technique that is not covered is Principal Component Analysis. Technically, it is in none of the technique domains covered in this book, but it would have been nice to see some of the methods in the book compared with PCA. The author does make the statement at one point that image recognition is an interesting problem for which the method being discussed at the time is used. Nevertheless, this is a great overview of the subject manner and is very entertaining. That in the long run probably explains the problem: it is a textbook.

This is an important book, but it would be more useful if it didn't have so many errors in the equations. Chapter 5, for example, states that the rates are a part of the model, but the equations on pages 210-213 only define the rates, but never use them. Equation 5.2-11 should have det(Cbar) in the numerator. There are many more errors like these in just the MLANS description in Chapter 5.

Artificial intelligence research goes back to the 1940s as do the first developments in electronic computers. Perlovsky traces the history of AI in a thoughtful and scholarly manner, emphasizing his philosophy and his own generalization of the theory which he calls Modeling Field Theory (MFT). He also traces the study of intelligence back to the Greek philosophers beginning with Plato and Aristotle some 2300 years ago.
The book however, provides more than just a philosophy for artificial intelligence. It mixes in some very important mathematics from the disciplines of engineering, statistics and computer science. Mathematical techniques and models have been particularly useful in the solution to problems in classification, clustering, pattern recognition, rule-based expert system development, multiple-target tracking, orbit determination and Kalman filtering, and time series prediction.

Perlovsky, over the course of his career, has had a great deal of involvement in the development of this research in both his consulting work and his work at Nichols Research Corporation. I know a lot about this because, in 1980 I began working at the Aerospace Corporation in El Segundo California as a MTS (statistician). I worked on statistical problems including Kalman filtering, image processing, image recognition, orbit determination, rule-based expert systems, multiple-target tracking and target discrimination. When I moved over to manage the tracking and discrimination algorithm development for the Air Force's Space Surveillance and Tracking System I became familar with the work being done for us by contractors that included Nichols Research Corporation (NRC). I became very familar with the work of Perlovsky and his colleagues at Nichols Research who supported us from the Newport Beach, Colorado Springs and Boston offices of the company on the multiple-target tracking algorithms and the target classification algorithms. I found the work to be so interesting and of such high quality that I joined NRC in 1988.

Perlovsky sees artifical intelligence as a very practical discipline and believes that computing machines can do a good job of at least mimicking human intelligence through the use of a priori knowledge (as rules preprogrammed into the computer or a priori probability distributions) along with experience (collected data from observational or statistical designed studies) combined using algorithms (Bayes theorem, adaptive neural networks) based on the mathematical foundation of uncertainty incorporated through probability theory and/or fuzzy set theory.

In my experience, I have found rule-based expert systems to be one of the major successful developments in the field of artificial intelligence. At the heart of these systems lies the tools of mathematics and statistics, including the discriminant or classification algorithms based on multivariate Gaussian models (linear and quadratic classifiers) and the nonparametric classification algorithms (kernel discriminant algorithms and classification tree algorithms). Also, patterns can be discovered by computers through the use of clustering algorithms based on Gaussian mixture models or nonparametric techniques like nearest neighbor rules.

The Bayesian approach to statistical analysis has been useful in many areas including the Kalman filter. In Kalman filtering prior knowledge plus current data is used to update the estimate of the current state and for the prediction of the future state of a dynamic system using a simple recursive algorithm that is easily updated in the computer. Many of these developments are well characterized and developed from first principles in this text.

Perlovsky emphasizes his own work including the MLANS system which is a neural network system that incorporates important statistical ideas such as maximum likelihood, the Cramer-Rao inequality and statistical efficiency along with the neural network architecture.

Some of this work was developed by Perlovsky under an Army contract that was coincidentally managed by my brother Julian. I have always viewed this research as being successful because it applied appropriate statistical models to the real problems. I think the crucial aspects of this work are the appropriate use of the Bayesian paradigm and teh indentification of appropriate models for construction of the likelihood equations. The fundamental and well established tools of probability and statistics are the keys. In his proposals, Leonid also included ideas from fuzzy set theory and embedded his methods in an artificial neural network framework. I always thought that these modern theories (fuzzy set theory and neural networks) were gimmicks to get military funding. This may not have been a fair assessment on my part as a careful reading of this book indicates that Perlovsky honestly views these tools as important.

There are subtleties to concepts such as fuzzy set theory. Although I do not yet see its value as a substitute for measure theoretic probability theory for characterizing human uncertainty, it is possible that I just haven't thought hard enough about it. Maybe a continued reading and rereading of Perlovsky's book will help me.

This is a very interesting and unique book on artificial intelligence from a perspective that is quite different from what one find in the standard books written by computer scientists (who often do not have the deep understanding of probability and statistics that Perlovsky possesses).

Artificial intelligence research goes back to the 1940s as do the first developments in electronic computers. Perlovsky traces the history of AI in a thoughtful and scholarly manner, emphasizing his philosophy and his own generalization of the theory which he calls Modeling Field Theory (MFT). He also traces the study of intelligence back to the Greek philosophers beginning with Plato and Aristotle some 2300 years ago.

The book however, provides more than just a philosophy for artificial intelligence. It mixes in some very important mathematics from the disciplines of engineering, statistics and computer science. Mathematical techniques and models have been particularly useful in the solution to problems in classification, clustering, pattern recognition, rule-based expert system development, multiple-target tracking, orbit determination and Kalman filtering, and time series prediction.

Perlovsky, over the course of his career, has had a great deal of involvement in the development of this research in both his consulting work and his work at Nichols Research Corporation. I know a lot about this because, in 1980 I began working at the Aerospace Corporation in El Segundo California as a MTS (statistician). I worked on statistical problems including Kalman filtering, image processing, image recognition, orbit determination, rule-based expert systems, multiple-target tracking and target discrimination. When I moved over to manage the tracking and discrimination algorithm development for the Air Force's Space Surveillance and Tracking System I became familar with the work being done for us by contractors that included Nichols Research Corporation (NRC). I became very familar with the work of Perlovsky and his colleagues at Nichols Research who supported us from the Newport Beach, Colorado Springs and Boston offices of the company on the multiple-target tracking algorithms and the target classification algorithms. I found the work to be so interesting and of such high quality that I joined NRC in 1988.

Perlovsky sees artifical intelligence as a very practical discipline and believes that computing machines can do a good job of at least mimicking human intelligence through the use of a priori knowledge (as rules preprogrammed into the computer or a priori probability distributions) along with experience (collected data from observational or statistical designed studies) combined using algorithms (Bayes theorem, adaptive neural networks) based on the mathematical foundation of uncertainty incorporated through probability theory and/or fuzzy set theory.

In my experience, I have found rule-based expert systems to be one of the major successful developments in the field of artificial intelligence. At the heart of these systems lies the tools of mathematics and statistics, including the discriminant or classification algorithms based on multivariate Gaussian models (linear and quadratic classifiers) and the nonparametric classification algorithms (kernel discriminant algorithms and classification tree algorithms). Also, patterns can be discovered by computers through the use of clustering algorithms based on Gaussian mixture models or nonparametric techniques like nearest neighbor rules.

The Bayesian approach to statistical analysis has been useful in many areas including the Kalman filter. In Kalman filtering prior knowledge plus current data is used to update the estimate of the current state and for the prediction of the future state of a dynamic system using a simple recursive algorithm that is easily updated in the computer. Many of these developments are well characterized and developed from first principles in this text.

Perlovsky emphasizes his own work including the MLANS system which is a neural network system that incorporates important statistical ideas such as maximum likelihood, the Cramer-Rao inequality and statistical efficiency along with the neural network architecture.

Some of this work was developed by Perlovsky under an Army contract that was coincidentally managed by my brother Julian. I have always viewed this research as being successful because it applied appropriate statistical models to the real problems. I think the crucial aspects of this work are the appropriate use of the Bayesian paradigm and teh indentification of appropriate models for construction of the likelihood equations. The fundamental and well established tools of probability and statistics are the keys. In his proposals, Leonid also included ideas from fuzzy set theory and embedded his methods in an artificial neural network framework. I always thought that these modern theories (fuzzy set theory and neural networks) were gimmicks to get military funding. This may not have been a fair assessment on my part as a careful reading of this book indicates that Perlovsky honestly views these tools as important.

There are subtleties to concepts such as fuzzy set theory. Although I do not yet see its value as a substitute for measure theoretic probability theory for characterizing human uncertainty, it is possible that I just haven't thought hard enough about it. Maybe a continued reading and rereading of Perlovsky's book will help me.

This is a very interesting and unique book on artificial intelligence from a perspective that is quite different from what one find in the standard books written by computer scientists (who often do not have the deep understanding of probability and statistics that Perlovsky possesses).

While this book is not mainstream and would not be a good place to try to begin to learn AI it is thought provoking and presents a number of new ideas. Where Rod Brooks seaks to eliminate internal models Perlovsky makes them a part of his definition of intelligence. I rather liked the philosophical slant of the book. Perlovsky expects to overcome complexity using fuzzy logic. Looked at from Allen Newell's multiple levels or bands perspective fuzzy logic is an extension of classical logic and in any real computer is implemented by a lower level (underlying level) of ordinary Boolean logic. It would seem to me that different granularities at each level would accomplish the same thing. Rules in first order logic can represent subsymbolic entities just as easily as they represent macroscopic objects and actions. Perlovsky presents an interesting MLANS architecture but gives rather less detail on a hierachical version. The computational complexity of these recurrent networks looks rather high to me. Perlovsky's book is a good challenge to those who have already studied some traditional AI and connectionist literature. He criticizes nearest neighbor (and CBR) systems but it seens to me that they offer some very fast parallel implementations such as Kanerva's distributed memory and Stanfill and Waltz's memory-based reasoning.

The book is a fascinating review of concepts in philosophy, semiotics, and mathematics of intelligence; it is interesting for a wide audience, not just for mathematicians. The author relates mathematics and philosophy and finds detailed and specific connections, suggesting that the evolution of mathematics of intelligence during the last fifty years paralleled the evolution of philosophy of mind beginning from Plato. The author develops mathematical techniques for semiotics, which reconciles the differences among concepts in classical semiotics. He argues that *symbol* as understood in general culture is a process relating conscious and unconscious, and he proposes modifications in semiotic terminology needed to bring in correspondence varying usages of this term. A most intriguing is a chapter that connects the mathematics to Kantian theory; an ability for perceiving beauty, concludes the author, is associated with learning. A useful feature of the book is the proposed course outlines for mathematically prepared students and for non-mathematicians.

Profound piece of work. I am not a cognitive scientist, and this book is a bit technical, but it is still within reach of the motivated lay person.

Gardenfors puts forward a a model to explain cognition that he calls "conceptual spaces." These conceptual spaces are at a level of abstraction in between the symbolic (used by AI types) and connectionist (Neural Nets). But what makes his conceptual spaces interesting and plausible is the position he takes that in this conceptual space, most reasoning is done by evaluating the analog of a distance between two aspects of a perception. Or, we find things to be similar if they are "geometrically" (measurably) closer on some limited number of dimensional scales.

This is easy to follow for things like colors, but he doesn't stop there. He goes on to describe how this explains a wide variety of perceptions, as well as how we form and reform categories and concepts, and shows how this informs semantics and the process of induction.

My only criticism is that some of the illustratios would have been more powerful in color.

If one is to design a machine that can formulate concepts and engage in such things as inductive inference and its corollary scientific discovery, then one must be able to quantify the notion of a concept in such a way that it can be implemented into the cognitive structure of the machine. One must be able to distinguish one concept from another, be able to tell when one concept is similar to another, and understand in detail how concepts are related across domains. It would not be enough to have qualitative notions of these distinctions or similarities, since they must be able to be formatted in such a way, either via coding, language, or electronically, so as to be used by the machine.

This book gives an interesting approach to the problem of concept classification, but it does so only from a qualitative point of view. It is a good start in this regard, and readers will gain a lot of insight into the problems that it addresses. It does not however give any advice on how to implement its ideas into a real thinking machine. Mathematical concepts are brought in order to talk more meaningfully about spaces of concepts, but they are really restricted to metric spaces and not general enough to deal with the plethora of concepts that could present themselves in typical environments. The book should be considered more as a work in philosophy, so those interested in this field might enjoy the book more than those who were expecting a book more geared towards artificial intelligence and computer science. Those readers interested in automated theorem proving or automated mathematical discovery might find the discussion on geometric categorization models of interest, and will find an interesting application of Voronoi tessellations, namely that of accounting for the varying sizes of concepts in a categorization.

By far the most interesting chapter in the book is chapter 6, wherein the author gives a highly original discussion of inductive inference. The ability of human cognition to generalize from a limited number of observations is viewed (correctly) by the author as very impressive, but he is careful to note that inductive inference cannot be done free of side constraints. Quoting the philosopher J.S. Peirce and his evolutionary explanation of why induction is so effective, the author uses his theory of conceptual spaces to develop a theory of constraints for inductive inferences. The main notion in this theory is that of "projectability", which attempts to delineate the properties and concepts that are may be used in inductive inference. The author wants to arrive at a computational model of induction, and he offers interesting proposals for doing so, even if they lack immediate empirical justification.

Central to the problem of induction the author argues is how observations are to be represented. This has been neglected in the history of philosophy he says, and so he then proceeds to outline his ideas on how to represent observations, distinguishing three levels, namely the `symbolic', the `conceptual', and the `subconceptual.' At the symbolic level, observations are represented by describing them in a specified language. At the conceptual level, observations are characterized relative to a conceptual space. At this level induction is viewed as concept formation. At the subconceptual level observations are characterized by inputs from sensory receptors. Induction is then viewed as the attaining of connections between various inputs. The author views the processing taking place in artificial neural networks as an example of modeling at the subconceptual level.

The problem of induction is more complicated than is typically presented in the literature, the author argues. Inductive inference will look different depending on which approach to observations is taken. In his elaborations on the processes of induction, one of the key issues that arises is the how discovery takes place across different domains. The process of conceptualizing across different domains takes place, as expected, at the subconceptual and conceptual levels. The symbolic level is delegated to formulating laws.

The essence of conceptual spaces, as I understand it, is that we can define concepts as regions in conceptual spaces. A conceptual space is defined by axes representing qualities. For example, color spaces are conceptual spaces, as is the tasting combos of sweet, bitter, salty.

Your choice of qualitative measures deeply affects how you understand the world. 'Spose reality is an infinitely dimensional, then we have lots of choices for axes. We simplify and correlate by using all that coordinate transformation and axis projection stuff from 3D graphics! Heck Gardenfors even uses Delauney Triangulation (or polyhedralization).

Criterion P, page 71

A natural property is a convex region of a domain in a conceptual space.

Criterion C, page 105

A natural concept is represented as a set of regions in a number of domains together with an assignments of salience weights to the domains and information about how the regions in the different domains are correlated.

Concept Combination, page 122

The combination CD of two concepts C and D is determined by letting the regions for the domains of C, confined by D replace the values of the corresponding regions for D. (contrast class p. 119), for example the "stone lions" outside the NYC library.

Six Tenets of Cognitive Semantics, page 160

i) Meaning is a conceptual structure in a cognitive system (not truth conditions in possible worlds)
ii) Conceptual Structure are embodied (meaning is not independent of perception or of bodily experience).
iii) Semantic elements are constructed from geometrical or topological structures (not symbols that can be composed according to some system of rules).
iv) Cognitive models are primarily image-schematic (not propositional). Image-schemas are transformed by metaphoric and metonymic operations (which are treated as exceptional features on the traditional views).
v) Semantics is primary to syntax and partly determines it (syntax cannot be described independently of semantics).
vi) Concepts show prototype effects (instead of showing the Aristotelian paradigm based on necessary and sufficient conditions).

Process of Abstraction, page 191 - Start with a collection of things. Identify and quantify individual objects. The determine the clusters. Step three: abstract the clusters into dimensions. Simple!

I especially liked the notion that a metaphor is taking the spatial relationship of a cluster of concepts in one domain and using them in a new domain to help understand the new domain.

Gardenfors introduces his theory of concept-formation, and at the same time presents a survey of the competing theories and research. He shows a high level of professionalism by accepting that the theories can coexist, presenting the competing theories in their strongest light and letting you decide where to apply each theory. This book is not only a good argument for why his theory deserves a permanent place in your toolbox, but also a good education for anyone wanting to know the tradeoffs in representing concepts -- especially for knowledge representation or machine learning systems. He presents the material in a very logical order so that the subtopics can be consumed individually. And although some of the material is well-known, each chapter presents a series of contrasting pros and cons and synthesizes the information in ways that are thought-provoking and novel. It was well worth the time and money.

For anyone interested in the cognitive topics, machine learning and artificial intelligence, this book is an eye opener. The point of view it presents attempts to put an order in what "meaning" really means.

Drawbacks of the book? The lack of conceptualization when it comes to dynamic concepts (treated very superficially). Also, the theory is deficient when modeling the functional aspects of concepts (a "sin" already recognized by the author).

But considering the pioneering character of this piece of art, these drawbacks are just compelling invitations for further research in the field.

Rodney, neuroscience explorer, returns from a trek into the unknown chart area, "terra incognita". Consciousness and mind hide there and now this book yields up their secrets and treasures. Well illustrated, apt quotations, written with beautiful expression and constantly rigorous in thought and argument. Science writing now days divides into the noisy and the hidden gems. The first is smart agents, pushy PR and personalities that constantly self promotion but write work that never lives up to its flash and advertising. Would that Pinker was as good as the ads tell us he is. In the second group are those that never get into the stream of success because they are too good natured for the game. But they write the science that is worth reading. Professor Cotterill - at least after reading this book -- is at the top of that group. Having read Pinker's Blank Slate, I wished I had read this first - better is a personal judgment but this is. Not a book for indenting from the library but buying for a holiday and pleasure.

This is vastly better than the meretricious commercial books that clutter up this area. Cotterill is (a) well-informed; (b) writes compellingly and clearly, with analogies and new ideas that will make even neuroscience professionals sit up and take notice. It is emphatically not just another piece of popularisation.

This is an extremely comprehensive book. It covers many aspects of neuroscience and neural networks. Among a lot of information, there is his theory of consciousness. He bases his view of the mind as action centered, and this is to my mind, a good move. It is no surprising that his model includes sensimotor areas. He also includes the prefrontal, premotor, and the thalamus intralaminar nuclei, forming a loop, in his theory of consciousness. He supports it quite well, and it gives rise to predictions that can be experimentaly tested. The data considered is overwhelming, so even if the consciousness theory end up not being totally right, the book as a whole is still a very important piece of literature in the neurosciences. Qualia as essentialy the effects of muscle-spindles in the loop at first seems confusing, if not implausible, but maybe deserves further consideration. Not a lot of neural network talk, but enough to complement nicely.

Daniel Dennett doesn't Explain Consciousness and Steven Pinker doesn't really tell us How the Brain Works. Cotterill does both: at a level of detail which allows the expert (which I am not) to evaluate his claims, yet in a style which is always accessible to the scientifically aware general reader. The evidence, getting down to the individual neuron and its dendrites, builds up to an overall picture which shows consciousness to be the outcome of a sort of time-lapse pattern matching process in the brain. This book really tells you how it works: it's not just a bunch of philosophising -- it's all (almost all, 'cos he does allow himself a speculation or two) based on experiment. Cotterill concludes by telling us that he and his students are now working on computer neural networks which should result in a computer which (convincingly) simulates consciousness. [Maybe I shouldn't have given away the ending.]

If you are, like me, an "amateur" when it comes to the study of the mind, you have probably sought to balance your reading of philosophers like Dennett with something more solid from the science of mind. "Enchanted Looms" is a fine place to do that.

This is not a book to sweep through in a few days. You will want to pause and digest. Although Cotterill is clearly aiming at an educated layperson as a reader, he bows, stylistically, to an academic audience. This interfered with my reading of the book. Dozens of times per chapter, he cites sources parenthetically or within the text. Too many sentences begin in the form "The work of _x_ and _y_ has shown..." For the longest time I kept thinking that noting and remembering those names would help me in following a line of argument. This was rarely the case. But then, at times, a backward reference to "_x_" would stump me. Once I learned to glide over these I found it much easier to read the book.

The tie-in with "neural networks" was an interesting process since I had little sense of their importance in cognitive science. Cotterill does a nice job, initially, of showing how such structures might work in both the abstract and at the level of neural anatomy. But, interestingly, he moves on to make a convincing case that such structures cannot adequately model all the functionality of the human brain. I came away from this book with the sense that neural nets are the "Ptolemaic epicycles" of brain science - a paradigm that with growing complexity and constant tweaking can just barely model what we know about a physical phenomenon, but which are not up to the ultimate task.

Cotterill does a nice job of making the macro-anatomy of the brain a part of a meaningful whole. Too many neuro-anatomy-focused books seem to just carve out the various regions and leave a sense of oddly unconnected "vision centers" and "speech centers." "Enchanted Looms" presents much more of the sense of the interconnectedness of those zones that we have chosen to isolate as anatomical pieces. He goes into some depth about how these connections might themselves function as a layer in the processing that we call thinking or sensation, ... or consciousness.

Which brings me, in the end, to the grail in my own "brain-book" search - "consciousness." Sure its fascinating to realize how interesting the study of, for instance, vision, might be, but its that "me" in there, in HERE, that wants some explaining. Although this is not the focus of Cotterill's book, he does propose a very different model for consciousness from any that I have seen - seemingly centered around neuro-motor systems; an odd twist on the notion of a "muscle-head" ! I say "seemingly" because it was really only upon reading this concluding section of the book that I realized I might not have understood enough of the prior 500 pages. Cotterill's argument for this unusual underpinning of consciousness seemed somewhat unconvincing, to me, only to the degree that it built upon elements of his model for brain that I had only partially grasped.

So I will reread this book... a very unusual thing for me, for this topic. It bespeaks the power of the ideas it presents that I know "Enchanted Looms" will be worth that second effort.

The articles in this work are written by a who's who list of authors from the cognitive and computational neuroscience community. Each article is useful for getting an initial bearing on a topic from this dynamic field. The references for each article serve as useful "jumping off points" for further learning. It should be noted that this text is not a typical college textbook -- it is a reference work. As such, a beginner to the field should consider one of the other introductory textbooks (perhaps "The Cognitive Neurosciences").

Review of Second Edition (January 2008):

This sizable collection of articles updates the first volume with many discoveries and conceptual developments that were unknown at the time. Meant of course for reference, a typical reader, such as this reviewer, would probably not read every article in the collection but would instead concentrate on the ones of primary interest. The editor however does offer advice on "how to use this book" at the beginning of the book, for those readers who intend to use it as their primary source of information, or for instructors who will use it as a supplement to such classes as brain theory, artificial intelligence, computational neuroscience, and cognitive neuroscience. All of these topics are represented, with emphasis of course on those that the editor finds important. Time constraints will of course play in role in any sampling algorithm for the articles, but every article that was studied by this reviewer was well worth the time spent.

One of these articles, written by the editor, gave an overview of his work on the `mirror system hypothesis' (MSH). This work has been widely discussed in the literature on evolutionary linguistics since the first edition of this book, and when confronting it for the first time may seem like a radical hypothesis. Such skepticism is aggravated by the lack of any historical record for the structure of the brain, and so any theories on language evolution will remain more tentative as compared to other scientific theories. The editor though wants the reader to consider evidence for the mirror system hypothesis that is drawn from existing life forms. Thus he proposes that we examine the "mirror system" for grasping in monkeys, which he asserts contains `mirror neurons" that are activated when the monkey performs a specific hand action and when it only observes a human or other monkey performing a similar action. The MSH is the assertion that the matching in the neural code between observation and execution occurs in the common ancestor of monkey and human. Further, this matching explains the notion of language `parity', which asserts that a spoken utterance has essentially identical semantics between speaker and listener. The editor reviews his ideas on what brain mechanisms are responsible for language and grasping, and whether a mirror system is indeed present in humans. Experiments using proton emission topography support his thesis to some extent, but he cautions that the a lot more work needs to be done before one can make definitive conclusions. His thesis though is a plausible one on the surface, and interesting in that it proposes that language originally evolved not from a need for communication but from a need to recognize a set of actions. "Language readiness" then, resulted from an extension of the mirror system from being able to recognize single actions to being able to imitate compound actions. A natural question to ask here is why sophisticated grammatical constructions, some of them semantically awkward and of no practical value, would evolve from the mere need to imitate, which itself is not really complex from any reasonable measure of complexity. The editor is aware of these kinds of objections, for in the article he addresses them under the guise of `protospeech', wherein he postulates two evolutionary stages for its development. His assertions in this regard are interesting for they involve the need for cooperation between two or more areas of the brain. Along these same lines, and even more fascinating, is the editor's discussion on neuronal models for the mirror system, for when he proposes a canonical structuring for sentences he is actually asserting a kind of "entanglement" (he does not use this terminology in the article) between the F5 area and its mirror.

Review of First Edition:
This complilation of articles by leading experts in the field gives an excellent overview of studies in cognitive theory and the theory and applications of neural networks. The first two parts of the book give an overview and background of the properties of neurons and gives guidance to the reader on what sequence the articles are to be read. This reviewer did not read all of the articles, but only those that piqued his interest. such as the following articles which are particularly well-written and informative: 1. "Applications of Neural Networks": Outlines the diverse applications of neural networks to signal processing, time series, imaging, etc. 2. "Astronomy": Neural network applications in astronomy, such as adaptive optics and telescope guidance. 3. "Chains of Coupled Oscillators": Their connection with the lamprey central pattern generator. 4. "Chaos in Axons": An excellent review of chaos experimentally in squid axons and numerically with nerve equations. 5. "Collective Behavior of Coupled Oscillators": A study of the phase and complex Ginzburg-Landau model. 6. "Computer Modeling Methods for Neurons": Good overview of numerical modeling of neurons. 7. "Computing with Attractors": Overview of omputing and feedback networks with attractors and a fascinating discussion of the possible existence of attractors in the brain. 8. "Constrained Optimization and the Elastic Net": Useful discussion of application of neural networks to optimization problems. 9. "Data Clustering and Learning": Good discussion of parameter estimation of mixture models by parametric statistics and vector quantization of a data set by combinatorial optimization. 10. "Diffusion Models of Neuron Activity": Discusses 1-dimensional stochastic diffusion models for the neuron membrane potential. 11. "Disease: Neural Network Models": Interesting overview of neural net computational models of various mental illnesses. 12. "Dynamics and Bifurcation of Neural Networks": Discussion of neural nets and their behavior as dynamical systems. 13. "Emotion and Computational Neuroscience": Fascinating discussion of computational models of emotion. 14. "Investment Management": A discussion of tactical asset allocation neural network methods in asset management. 15. "Learning and Centralization: Theoretical Bounds": Overview of computational learning theory. 16. "Locust Flight": Interesting neural network study of the locust flight system. 17. "Neural Optimization": Discussion of combinatorial optimization using Ising and Potts neural networks. 18. "PAC Learning and Neural Networks": Overview of the Valiant "probabilistically correct learning paradigm in neural networks. 19. "Protein Structure Prediction": Neural network applications to prediction of protein secondary structure. 20. "Schema Theory": Extremely interesting overview of schemas. 21. "Speech Recognition: Pattern Matching": Excellent discussion of the applications of hidden Markov models to speech recognition. 22. "Statistical Mechanics of Neural Networks": Discussion of the use of the Hopfield model in neural networks. 23. Vapnik-Chervonenkis Dimension of Neural Networks": Very interesting discussion of the VC-dimension of neural networks.

Look through this book to convince yourself that an exact brain theory does not exist. The arrangement of the articles by the first letter of their title tells it all (consider classifying animals by the first letter of their name). The editors wrongly assume that mathematical methods equal theory; actually, theory is a small conceptual tent under which a large number of experimentally established facts can be gathered. In most cases, mathematics is a very useful tool in pitching this tent, but it has little to do with the tent itself.

An exact theory of the brain may be possible and we are in dire need of it. Unfortunately, nobody has come up with it yet. This book is an encyclopedia of various mathematical methods that have been used to solve various neuroscience problems. These methods and solutions are as diverse as the problems themselves. Don't look for common themes in this book. If you are looking for a unified brain theory, you'll be much better off reading standard neuroscience textbooks. I do hope one day we'll be able to cast these vague ideas into something precise and, most likely, mathematical. Sadly, not today. I own a copy of this book and use it to remind me why and how we have failed so far.

It should be kept in mind that it is not at all clear that "neural" networks can emulate consciousness. They may or they may not. Firstly, a single neuron resembles a computer processor in its complexity and is a constantly evolving entity. Secondly, only 10% of brain cells are neurons and the remaining 90% (glial cells) now too appear to be involved in information processing. At a more fundamental level, consciousness may be less algorithmic and computational than we expect. Finally, the brain and the reality "outside the brain" are a two-way street. As the great neuroscientist Cajal put it, "As long as our brain remains an arcanum, the Universe, a reflection of its structure, will also be a mystery". If we assume the brain analyzes something, we need to define a reality independent of this analysis -- a hardly possible task if standard "input-output" approaches are used.

If the title of this book were "Current Mathematical Methods in Neurosciences", I'd have no problem giving it five stars.

November 2005: The chapters in the second edition are still arranged alphabetically. I refuse to believe neuro-mathematicians cannot think more coherently.

One final note for those looking for serious conceptual advances on the theoretical front: do no miss "Spikes: Exploring the Neural Code" (edited by F. Rieke) and "Decisions, Uncertainty, and the Brain: The Science of Neuroeconomics" by Paul Glimcher.

Research is tedious, but if you want to know the nitty-gritty of mind-brain theory and neural networking, this book is an invaluable resource for basic, relevant, and accessible papers on the subjects. Encompassing seminal works from an unusually broad range of disciplines, here is an outstanding reference for those concerned with the mechanisms of intelligence.

This is THE neural network and brain theory reference. Owning it is like owning an entire library, though much more compact.

If you take a look at the table of contents, you'll see the massive value in this book. If you're into neural nets and brain theory, or want to be, you need this book.

For the analyst wanting to get into image recognition, Davies offers a detailed look at the many methods used in the last 30-40 years. These include neural networks, support vector machines, and the Hough transform.

If you are tempted to use [or are using] the OpenCV code base for image research, then the book can be a vital theoretical framework. OpenCV is about the best open source image code out there on the net, but it is poorly documented. It does come with many methods for basic and vital operations like make a grayscale image from a colour image, and making a binary image from a grayscale image. But why the code does certain things (actually many things) is rarely explained. Try using this book for understanding. Plus, the text lets you get an idea of how to modify OpenCV for your purposes.

And if you are going to use this book with OpenCV, look closely at the section on using multiple classifiers for training and then testing against unknown images. It is the basic idea for the cascading classifiers used by OpenCV.

Along these lines, one improvement for a future edition of the book could be an analysis of code packages that are currently available for image processing. Just a thought. But it would greatly help people wanting an expert assessment on the efficacies of available packages. Or, on a more basic level, it would aid simply in delineating what is out there.

To begin with, the latest edition of this book was published in 2004, so all reviews dated earlier than that are referring to a previous edition. This book is a good one on issues and algorithms as they pertain to machine vision versus general computer vision. If you want a good general textbook on computer vision try "Computer Vision" by Linda Shapiro. It has all of the background material and a firm foundation in all of the topics you would expect in a course on computer vision. This book also has a section on introductory computer vision topics, I just don't think it is as clear and as comprehensive as Shapiro's book, especially for students.

However, if you want an excellent treatment of the kinds of problems specific to machine vision - the detection of lines, holes, corners, circles, elipses, and polygons, for example, along with specific algorithm details, this book is very good. It also has good sections on pattern matching, motion estimation, and 3D machine vision. I would recommend it especially for those individuals who are already familiar with the basics of computer vision and would like a book on algorithms for solving specific problems in machine vision. I notice that Amazon only shows the table of contents for the previous edition, so I show the table of contents for the new edition next:

1. Vision, The Challenge

PART 1 - LOW-LEVEL VISION
2. Images and Imaging Operations
3. Basic Image Filtering Operations
4. Thresholding Techniques
5. Edge Detection
6. Binary Shape Analysis
7. Boundary Pattern Analysis
8. Mathematical Morphology

PART 2 - INTERMEDIATE-LEVEL VISION
9. Line Detection
10. Circle Detection
11. The Hough Transform and Its Nature
12. Ellipse Detection
13. Hole Detection
14. Polygon and Corner Detection
15. Abstract Pattern Matching Techniques

PART 3 - 3D VISION AND MOTION
16. The Three-Dimensional World
17. Tackling the Perspective n-Point Problem
18. Motion
19. Invariants and their Applications
20. Egomotion and Related Tasks
21. Image Transformations and Camera Calibration

Part 4 - TOWARDS REAL-TIME PATTERN RECOGNITION SYSTEMS
22. Automated Visual Inspection
23. Inspection of Cereal Grains
24. Statistical Pattern Recognition
25. Biologically Inspired Recognition Schemes
26. Texture
27. Image Acquisition
28. Real-Time Hardware and Systems Design Considerations

PART 5 - PERSPECTIVES ON VISION
29. Machine Vision, Art or Science?

Covers many aspects of vision, from basic image processing through high level scene analysis. It doesn't always go down to the nitty-gritty source code level for every topic, but it does provide the direction to handle most every common machine vision problem. Of the ten or so general machine vision books on my easy-access shelf, this is the one I seem to pull down the most.

A very clearly structured book which is useful as a reference. Covers a lot of subjects (filtering, detection of shapes [lines, circles, holes and more], pattern matching/recognition, motion, invariants, ...), including the implementation aspects (hard/software). The chapters sometimes do not go much into deep but provide further references. Recommended book!

First of all I like this book very much. This book provides a solid and concrete foundation to computer vision from engineering point of view. The basic issues are treated very well in the conceptual and practical levels (e.g. edge detection). I came from a photogrammetry background, which means that the geometric aspects are very dominant in my thinking, and this book emphasize many geometric concepts in computer vision specially the treatment of Hough Transform as a main theme in the book. I recommend this book to the practitioners in spatial sciences (GIS, Remote sensing, Photogrammetry, etc) as well as the general community of computer vision.

At this time I am about halfway through the book and finding it very informative and very interesting. It covers a lot of technical information which is normally pretty boring but I am having a hard time putting it down. Highly recommend this as require reading for the ameture forensic computer examiner.

CR Flowers CCE

Are you a law enforcement officer, system administrator, IT professional, legal professional or a computer forensics student? If you are, this book is for you! Author Christopher LT Brown, has done an outstanding job of writing a great book by focusing on the first two phases of the computer forensics process: computer evidence collection and preservation.

Brown, begins by introducing the reader to the essential elements of computer forensics.
Next, the author discusses the rules of evidence, existing computer-related case law, and regulation as a basis of understanding the nature of computer evidence in court. Then, he provides information about evidence dynamics, which is defined as anything that effects evidence in any way. The author continues by presenting the key components to knowing where data can be found within an organization's infrastructure. In addition, the author shows you how an organization's information architecture can be as diverse as a city's street's. He also examines the volatility of digital data in physical memory and storage. Next, the author explains the key components of the IDE,SIDE, and SCSI standards as they pertain to evidence collection. Then, he describes advanced physical storage methods in use today. The author also examines some of the many types and formats of removable media including flash cards and optical media. In addition, the author next describes one of the most important components of any computer forensics investigation: tools preparation and documentation. He also shows you how volatile data can be difficult to capture in a forensically sound fashion. Next, the author describes how methodologies used in computer forensics can be as varied as the systems being imaged. Then, he shows you how the collection of evidence from large computer systems can be challenging to any investigator. The author continues by walking the reader through different design options to get the most out of their hardware configuration in the field and back in the lab. In addition, he shows you how today's computer evidence investigators rarely work from a single forensics workstation. Finally, he discusses areas for further study in computer forensics such as analysis and presentation of evidence in court.

This most excellent book uses evidence dynamics at the center of its approach to show the reader what forces act on data during evidence identification, collection and storage. What's most important though, is that this book will help guide the computer forensics investigator in ensuring case integrity during the most crucial phases of the computer forensics process.

This is a timely book as we are hearing more and more about the U.S. military and intelligence agencies collecting the computers used by terrorists. This same trend is appearing in conventional law enforcement. The amount of information that can be stored on a computer is, of course huge, also important is the transient: What web site is the computer viewing? What e-mail system is on-line? What can be gotten from the router being used?

This book goes into every aspect of getting forensics information off of a computer. It starts with examining the computer, if it is on, then extracting the information from places like temporary internet storage. Of course there's a lot that needs to be done with the hard drive, and if you can find back up disks, tapes or memory devices.

In addition, there are hardware and software tools that can be used to extract information from the system. A general coverage of these is given, along with sources. Some of these are included on the CD-ROM included with the book.

This book is intended for use in a legal environment, so there is discussion on maintaining the chain of evidence to ensure that it doesn't get thrown out of court. Should you be on the other side in a trial, this gives you something to ask of the investigators to be sure that they have followed the rules.

Basically this is the most complete, most thorough book on the subject written by one of the experts in the business.

It seems that a lot of books on forensics concentrate on making a disk image of the hard drive being examined, filtering the information on the disk, and presenting it in proper format for court use. However, collecting and preserving the evidence is much more than imaging the hard disk. If the computer is still on then evidence may be in memory, potential evidence may be on routers, proxy servers, etc. This book details this part of forensic evidence gathering, an area often just skimmed over in other computer forensics texts. This is a critical aspect of investigation because it does not matter how well your filtering works and how much evidence you obtain if your data preservation was not done correctly and the evidence is inadmissible in court.

Evidence dynamics is covered in detail and the author does a better job of this than any other forensics book I have read. Evidence dynamics is how to keep the evidence from disappearing or changing. Just the act of shutting down a computer changes temporary files, open processes, swap file information, and many other items that may be necessary for a thorough investigation. Even the appendixes are valuable and contain several excellent sample forms including chain of custody, evidence collection, and evidence access worksheets. If you are involved in either the collection or the maintenance of data for a potential court case then you will be interested in this book. Alternatively, if you are trying to discredit an expert witness then the information presented here may also provide areas of attack. Either way Computer Evidence Collection and Preservation is highly recommended.

I bought this book a while ago, and I must say that it has been a source of immense help in deepening my understanding of neural networks. Yes, if you buy this, you will clearly realize that it has 'severe' amounts of math on every page, but again, if you can figure the math out, other aspects of the book (especially coverage) generally make it worth the deal.

I feel it is a very good book over-all for Neural Networks. It is one of the very few books that I came across with an excellent description of Time Delay Neural Networks (TDNN) and the associated learning algorithms.

Prof. Hassoum's book is very good to introduce the reader in the mathematics of Artificial Neural Nets (ANN), including an interesting item explaining how to integrate Genetic Algorithms (GA) with Artificial Neural Networks (ANN) not found in this kind of work. Nevertheless, this is not a book for computing professionals because its necessary one to have a solid background on math to understand the ANN concepts along the chapters. Well written for mathematicians, it lacks pratical examples for better understanding the concepts explained in the book.

Prof. Hassoun's book is almost the most complete book that builds a clear and broad foundation of neural networks. His unified approach to cast the problems of neural networks in a mathematical optimization models is excellent. The book is full of challenging and drill-like problems. The references cited blasts the door before the reader's eyes to explore worlds of applications. Prof. Hassoun's contribution to the field of Neural Networks is remarkable. After more than three years of taking two graduate courses using this book (and being lectured by Prof. Hassoun), I can hardly forget any detail. A excellent book which ideas get inscribed in your head. In a few word, The Bible of Neural Networks ...

Very nice introduction to NeuralNetworks and how to implement them in Java.
If you're looking for deep concepts on NeuralNetwork this isn't the best choice.
But if you're looking to figure out how NeuralNetwork works and how to begin codeing them that's it.

I have been reading through the book. Actually it provides very clear explanations, but I had the impression the author talks too much and keep saying the same things over and over again. The book could be half its volume with the same content of knowledge. Besides the provided examples are a bit too simple and obvious.
Nothing much to put under the tooth. After reading it I felt left with my hunger for something deeper and more consistent. The algorithms provided also merely implement and stick to the few examples introduced. On the course of the book, the author wanders from the main point which is first and foremost to discuss neural networks under all angles. He unexpectedly brings up Fuzzy logic and Genetic algorithms which is not what the book title purports to talk about: a bit of confusion.
Overall there is a bit of deception, but indeed the book does what its title says : it is really just an "introduction" to Neural Networks with Java and nothing more. I would recommend it to somebody seeking to embrace the field and who is really a beginner in the domain.

Programming Neural Networks in Java will show the intermediate to advanced Java programmer how to create neural networks. This book attempts to teach neural network programming through two mechanisms. First the reader is shown how to create a reusable neural network package that could be used in any Java program. Second, this reusable neural network package is applied to several real world problems that are commonly faced by programmers. This book covers such topics as Kohonen neural networks, multi layer neural networks, training, back propagation, and many other topics. The content of the book is as follows:
Chapter 1: An Introduction to Neural Networks
The structure of neural networks will be briefly introduced in this chapter. Also discussed is the history of neural networks, since it is important to know where neural networks came from, as well as where they are ultimately headed. Finally, there is a broad overview of both the biological and historic context of neural networks.
Chapter 2: Understanding Neural Networks
A neural network can be trained to recognize specific patterns in data. This chapter will teach you the basic layout of a neural network and end by demonstrating the Hopfield neural network, which is one of the simplest forms of neural network.
Chapter 3: Using Multilayer Neural Networks
You will see how to use the feed-forward multilayer neural network and two ways that you can implement such a neural network. The chapter begins by examining an open source neural network engine called JOONE. JOONE contains a neural network editor that allows you to quickly model and test neural networks.
Chapter 4: How a machine learns
Every learning algorithm involves somehow modifying the weight matrices between the neurons. This chapter examines some of the more popular ways of adjusting these weights.
Chapter 5: Understanding Back Propagation
This chapter examines one of the most common neural network architectures-- the feed foreword back propagation neural network.
Chapter 6: Understanding the Kohonen Neural Network
The Kohonen neural network contains no hidden layer. The Kohonen neural network differs from the feedfroward back propagation neural network in several important ways. This chapter examines the Kohonen neural network and how it is implemented.
Chapter 7: Optical Character Recognition
This chapter develops an example program that can be trained to recognize human handwriting. It is not a program that can scan pages of text. Rather this program will read character by character, as the user draws them. This function will be similar to the handwriting recognition used by many PDA's.
Chapter 8: Understanding Genetic Algorithms
A chapter on an AI technology unrelated to neural networks.
Chapter 9: Understanding Simulated Annealing
A second AI technology that can be used to train neural networks.
Chapter 10: Eluding Local Minima
One of the most fundamental flaws is the tendency for the backpropagation training algorithm to fall into a "local minima". A local minimum is a false optimal weight matrix that prevents the backpropagation training algorithm from seeing the true solution. This chapter shows how to use certain training techniques to supplement backpropagation and elude local minima.
Chapter 11: Pruning Neural Networks
This chapter examines several algorithms that modify the structure of the neural network. This structural modification will not generally improve the performance of the neural network, but makes it more efficient. If a particular neuron's connection to other neurons does not significantly affect the output of the neural network, the connection will be pruned.
Chapter 12: Fuzzy Logic
Fuzzy logic is a branch of AI not directly related to the neural networks examined so far. Fuzzy logic is often used to process data before it is fed to a neural network, or to process the outputs from the neural network. Fuzzy logic is examined in reference to removing SPAM from emails.
Appendix A: JOONE Reference
Appendix B: Mathematical Backgrounder
Appendix C: Using the Examples on a Windows System
Appendix D: Using the Examples on a UNIX System
This book is currently available online. Since Amazon throws out reviews with web addresses in them, suffice it to say that you just need to type "HeatonResearch" into Google. The 2nd address is the one you want. This book couples accessible instruction with plenty of code that you can lift to make your own neural network applications. I highly recommend it.

I have received my copy of the book and I can't put it down. It has been great help with my AI research at the University. I have the other book from the same author "Programming Spiders, Bots and Aggregators in Java" and I have the same comments for both. Both are easy to read, have precise information and great code. Chapter 7 of this book "OCR with Kohonen Neural Network" makes the book more than worth it. Great stuff. I hope the author does not stop and keep writting books like these. I recommend this book for anyone interested in learning AI and also experienced programmers alike. The author makes though topics seem easy. Highly recommended.

This book is a real treasure chest of ideas for an amateur robot builder. It does not show solutions in enough detail to copy them, but it points you in the right direction.

For me some of the most interesting topics covered were:
- Motorola M68332 based general purpose robot controller board,
- Introduction to different robot competitions,
- Simplified image processing solutions,
- Walking robots and evolutionary programs to control the gait,
- Examples of real life robots.

After I have read the book contents, I think it is a great book fo me. From this book, I can get many new knowledge, including many field. I want this book to guide my student to take part in
robot contest.

Hi, so far the book is a little dissapointeing, because is to much centered to a commercial hardware (EyeCon) that is somewhat hard to adquiere to a group that has a very low budget for that kind of projects... but is important to have a reference. That is why it deserve a 3. But also supply a very important source of information for sensors and actuators design. That is why i gave an average of 4 stars.

Best regards, and thanks for all the effort you made for me enjoying this book.