Neural Networks Books

I think it is a little bit outdated, I didn't find much of innovation inside

Like most powerful theories this one is simple. It describes a mathematical model that mimics some aspects of human memory. The book is also refreshingly concise for an academic work.

I'm biased since I have worked with Pentti on his Sparse Distributed Memories at NASA Ames. I would highly recommend you reading his book since he is very careful and general in his use of statistics of large bit vectors. I am continually amazed at how much can be extracted from such vectors and the richness of their properities. SDM is similar to associative memories but simpler in form but just as general. Work is continuing at the Swedish Institute of Computer Science (SICS) on Dr. Kanerva's ideas.

This book presents the current and mor well-known models of cognition in the area cognitive science. This includes descriptions of both symbolic and connectionist models (e.g. ACT-R, SOAR, ART-MAP, MAC/FAC, etc.), written by the authors who developed them. However, each chapter presents a somewhat condensed version of each model, so some (but not all) of the technical details are ommitted. Overall, the book can function as an extensive introduction to contemporary methods and issues in cognitive modelling, or as a reference book for those more familiar with the field.

This book could be considered to be a collection of articles on the `computational theory of mind.' Although the articles are somewhat out of date, due to the advances in neuroscience and cognitive science that have occurred since the time of publication of the book, it does serve as a good motivation for the understanding of more recent developments. I did not read all of the articles in the book, and so my review will be confined to the ones that I did.

The article on ACT in chapter 2 is basically a theory of cognition that is based on recursion. Referring to ACT as a "simple theory of complex cognition", John Anderson, the author of the article, wants to simulate the manner in which humans develop recursive programs. The machine that is to simulate this makes use of `production rules,' in its knowledge base, which the author claims is exhaustive enough to produce complex cognition. To produce true machine intelligence, all one has to do is to tune these production rules and make use of them as needed. As the author describes it, the original ACT theory was based on human associative memory, but the one described in this article is called ACT-R, and can simulate adaptive behavior in the presence of a noisy environment. The author describes various simulations using ACT-R, and concludes that it is sensitive to prior information and to information about what is appropriate response to the situation it finds itself in. The author stresses more than once the simplicity of the ACT-R system: it is able to encode data from the environment as declarative knowledge, encode the changes in the environment as procedural knowledge, and encode the statistics of this knowledge use in the environment.

Another highly interesting article is the one by Alan Prince and Paul Smolensky on the application of optimization theory to linguistics. Called `optimality theory' by the authors in their extensive research on the topic, in the article they discuss the relations between optimality in grammar and optimization in neural networks. The authors discuss with great clarity the role that constraints play in the construction of linguistic structures, and the fact that these constraints typically conflict with each other. This conflict between grammatical constraints must thus be managed by a successful grammatical architecture. Optimality theory asserts that these constraints are universal in the sense that they are present in every language. The connection of optimality theory with neural networks arises when one is interested in finding out if the properties of optimality theory can be explained in terms of fundamental principles of cognition. The computational theory of neural networks the authors believe holds some clues on these properties. In order to make the connection with grammatical issues, as abstract as they are, and because neural networks are highly nonlinear dynamical systems, one must find a way of encapsulating the complicated behavior of neural networks. The authors accomplish this by the use of Lyapunov functions, which for reasons of consistency of terminology they call `harmony functions.' For those neural networks admitting a harmony function, the initial activation pattern flows through the network to construct a pattern of activity that maximizes "harmony." Most interestingly, the harmony function for a neural network performs the same function as does the mechanisms needed for well-formed grammar. The patterns of activation are thus a mathematical analog of the structure of linguistic representations. However, the authors are careful to note that not every weighting scheme for the neural network will give a possible human language. It is here where the constraints play an essential role in limiting the possible linguistic patterns and relations.

The article by Keith Holyoak and Paul Thagard discusses the construction of a correspondence between a source analog and of a target. This is the so-called analogical mapping, which is constructed using a collection of structural, semantic, and pragmatic constraints. In the view of the authors, the concept of analogy can be broken down into four components, namely the selection of a source analog, the actual mapping, an analogical inference (transfer), and the actual learning that takes place. The authors omit discussion of the last component in this article. The finding of the correspondences between the two analogs can result in a combinatorial explosion, and so use is made of appropriate constraints. These constraints consist of those that exemplify structural consistency, those of semantic similarity, and lastly of pragmatic centrality. The theory of analogical mapping that the authors propose is governed by these constraints. They discuss the ACME (Analogical Constraint Mapping Engine) algorithm as one that constructs a network of units representing mapping hypotheses and eventually converges to a state that represents the best mapping. They list several applications of ACME, such as radiation problems, attribute mappings, chemical analogies, and the classical `farmer's dilemma' problem. ACME was also able to simulate a number of empirical results related to human analogical reasoning. The analogical mapping they discuss is most powerful in a specific domain however. This domain-specificity is a typical restriction for most of the efforts in learning theory and artificial intelligence.

provide a good and easy to understand introduction to the subject

This book gives a good introduction to the mathematical modeling of cognition and does so with a level of mathematics that is very accessible to a typical graduate student in computer science or psychology. The book could have been written using tools from measure theory but luckily it was not for a book at an introductory level. The concept of probably approximately correct is introduced early on in the third chapter of the book with efficient learning given later on in Chapter 5. Chapter 7, the best chapter of the book, discusses the idea of VC dimension, which has had many applications, such as network stability and optimization. VC dimension plays the pre-dominant theme in the rest of the book, with the book ending with an application to neural networks. There are short problem sets at the end of the chapters, and these are useful for more understanding of the concepts in the book. A very interesting book and worth the price.

Here's a self-contained introduction to connectionist modeling. Easy to read and straight-forward, this text provides software and excercises aimed at stepping a novice through the basics of connectionism. Designed to accompany Rethinking Innateness (1996), these examples provide a glimpse into the world of cognitive modeling. The examples can, at times, be frustrating and the text is in need of more debugging hints; yet, the simulations are rewarding and thought-provoking. While those already familiar with connectionism will find the excercises too basic, those curious about connectionism will find the book a great place to start and one that doesn't bog the reader down with technical jargon. It is accessible, enjoyable, and written by two key players in connectionism: Kim Plunkett and Jeff Elman. Well worth reading, but only if the reader is willing to work through the basic simulations and answer the excercises along the way.

Here's a self-contained introduction to connectionist modeling. Easy to read and straight-forward, this text provides software and excercises aimed at stepping a novice through the basics of connectionism. Designed to accompany Rethinking Innateness (1996), these examples provide a glimpse into the world of cognitive modeling. The examples can, at times, be frustrating and the text is in need of more debugging hints; yet, the simulations are rewarding and thought-provoking. While those already familiar with connectionism will find the excercises too basic, those curious about connectionism will find the book a great place to start and one that doesn't bog the reader down with technical jargon. It is accessible, enjoyable, and written by two key players in connectionism: Kim Plunkett and Jeff Elman. Well worth reading, but only if the reader is willing to work through the basic simulations and answer the excercises along the way.

This book is one of the two main books used in Caltech's Neural Computation class (the other is Introduction to the Theory of Neural Computation by Hertz, Krogh, and Palmer). This book covers a wider spectrum of learning models than most books, including Hertz, et al. It is still fairly mathematically rigorous, although not as much as Hertz, et al. It is ideal for somebody who wants a fairly mathematically rigorous description of the subject, but also wants something more comprehensive than Herts, et al.

Fifteen years ago I did research in this area and had just completed a math degree. I thought it would be interesting to get back up to speed.
I found the book fascinating, tremendous work has been done in this field and this is a good broad treatment of it. For anyone who is into computer science but has never studied the brain it will be a tremendous eye opener.
But boy, was it hard work. I found I had to read & re-read sections just to understand some of the math involved. In fairness the book does cover everything you need but if you have never done college level math, or you have forgotten most of it, then don't tackle this book when you are tired!

This book gives an introduction in how to implement neural networks using SAS Enterprise Miner, and is written for those who already have a basic understanding of them. Neural networks are straightforward to understand from a mathematical standpoint but their use in real applications can be awkward, especially if they are required to work in out-of-sample contexts. Enterprise Miner will not alleviate these difficulties, but it does offer a more straightforward way to build the neural network architectures, due to its menu-driven approach. The book is somewhat out-of-date, since it is written for those readers who are using Enterprise Miner 4.3, but most of the book is still relevant for those who are now using SAS Enterprise Miner 5.2. The latter is JAVA-based, and has some additional capabilities that one cannot find in Miner 4.3. Readers who will not be using the new features in Miner 5.2 will therefore find the book useful. The author also discusses some of the foundational aspects of neural networks, and how they compare with other methods for doing prediction and classification. Of course if one has access to Enterprise Miner 5.2, the accompanying documentation will lessen the need for this book.

I just purchased this book, and I must say that I am truly impressed. When I picked this book up at the mailbox, I thought there must be a mistake, there must be two books here. There wasn't, this book is an inch and a half thick by itself, and filled with goodies form the first to last page. This book would be perfect to teach a data mining class on Enterprise Miner and Neural Networks.


We used the neural procedure in UCF's Data Mining 2 class and SAS does not provide any support. This book is at the cutting edge of using the Neural procedure in open code.


The book provides the syntax and statements for using the Neural prodecure and the DMDB procedure. SAS does not support these procedures. If you ever want to write a macro using a neural network you will want to use the Neural procedure in open code. The author also provides numerous code examples with different architectures. He also does a good job of explaining how neural networks get stuck in local minimums, and of explaining the link procedures and what a miner would have to do to score a validation/test data set.

The book also explains the nodes in Enterprise Miner and also guides the miner through building a diagram. If only the author can write a book about decision trees and the Split procedure.

Overall, this is a good book on the various subjects it covers, but I have to wonder - why would you buy this expensive hardcover version when the paperback version costs a little more than half what this hardcover costs? The contents are the same, the publication date is the same, and having had my softcover copy for two years and made heavy use of it, it seems to be standing up to the wear and tear just fine.

This book is an overview of all of the components of nonlinear dynamics. Nonlinear dynamics is a field of study that enables well-constructed predictive modeling of systems that might be difficult to solve otherwise. Such continuous systems were first widely modeled by ordinary and differential equations, but with the passage of time there are now tools and mathematical models at our disposal that make for a much more concise model of many systems. This workbook tries to touch on all of those mathematical tools.

The first six chapters of the book has to do with modeling such complex systems in general, and the rest of the book is a survey of the tools needed to perform complex modeling. The book's format is that of briefly explaining a concept in a few pages, and then presenting a computer program that demonstrates the concept just explained. The explanations are very clear and concise, there are plenty of equations shown, and the accompanying code is well commented. If you want to really drill deeply into any of the concepts then you are going to need some other books. I suggest that for further reading for the mathematically inclined that you pick up "Chaos: An Introduction to Dynamical Systems" by Kathleen Alligood. For scientists that want to see specific problems that can be solved by dynamical systems I suggest the excellent "Nonlinear Dynamics and Chaos: With Applications in Physics, Biology, Chemistry, and Engineering" by Strogatz. The only real complaint I have against this book is that there is uneven coverage of different tools. For example, the author has a great deal to say about neural networks and fuzzy logic, but has very short chapters covering discrete wavelets and cellular automata. More material would have been great, since it is hard to find good books on discrete wavelets and cellular automata in particular. Some readers may also be annoyed that much of the book are code listings of the various demonstration programs.

Overall, I would highly recommend this as one of several books that anyone interested in dynamical systems should definitely own. In particular, those individuals interested in the techniques of algorithmic composition of music might find this book a good jumping off point for studying the tools and techniques that make such compositions possible.

Here is a text of advanced nonlinear dynamics. Geared towards the intensive use of computers to perform the necessary grungework. Steeb hits on many important ideas that have emerged in recent decades. He shows the interrelation between chaotic phenomena and fractals, and how fractals can be used to describe the onset to chaos.

The Hidden Markov Models have proved to be the key idea in current Automatic Speech Recognisers. A tribute to the practical nature of this idea.

Steeb's discussion of neural networks and genetic algorithms is enough to get you started in this field. Ideas like forward and back propagation for feedback are clearly explained.

The sample code should be welcomed. It lets you see for yourself on your own computer, and to tinker with the various parameters. Though I am unsure about the choice of code in Symbolic C++. Unlike C++ or Java, this is a rarely used language.

This book explains the concepts in clear simple language and shows you the source code in C++, class diagrams (unique), and how the algorithms work with flow charts (unique). Often, each chapter contains step-by-step examples of how these algorithms work on some simple input vectors - exactly what I need for exam preparation.

This book comes with C++ source code, and thus provides a nice place to begin for someone who is interested in experimenting with neural nets. The author's focus on the character-recognition problem, so the book is somewhat specialized from this perspective. I would not recommed this book to someone who is primarily interested in a strong theoretical book describing learning with neural nets.

Pulsed Neural Networks (90's), Artificial Intelligence (80's), Cybernetics (60's and 70's) Telephone Switch Board (10's and 20's) Hydrodynamics (1700 and 1800) it is amazing the names put on cognitive science through the years. This book is a symposium (13 small books) on developing hardware devices capable of replacing or enhancing neurological functions. Using modeling techniques to duplicate biophysical neural pathways can take two forms. The first are math models, which obviously show the relationship between the neurons (virtual reality). The second type is models that build-spiking neurons in real time to which this book is directed. In the first part of the book, a summary of current thought, written by the main compilers of the book (Maas and Bishop) is worth the price alone. The book addresses the question of biological electrical (vs. chemical or genetic) coding, in which the method of information is actually transmitted and received. The compilers have emphasized the chronological event of development with the articles so that the reader does not become lost in which came first. Gravy is given the reader in the form of articles written by researchers in other fields (VLSI) to the point that the reader wonders if one is still reading a book on biophysics. The hard-wired neural net components are then compared to their biological predecessors for the purpose of obtaining usable "dry lab" tools for experiments. ("Dry-wet-electrical lab", "electrical-dry-lab-wet-computer-lab"?). Even though the material contains electrical engineering stuff it is still very readable to biological types and if interested, can muscle through this stuff. The math model development in Matlab is mentioned, but the reference to Matlab's current capabilities in this area is dated (95). Most of symmetries run in the book are older 200 Pentium type machines, and with a faster (650 up) and better busing Matlab's new neural net toolbox can build some interesting stuff (remember however it is still virtual). The "home modeler" can use chap. 7 and 12 as a theoretical basis for stochastic resonance models which the writers, while dealing with stochastic bit-stream overlooked this aspect. However, H.Wilson's Spikes, Decision and Actions is much better. (Matlab interactive). This is a really good book for modelers (reason for the review as opposed to `me to' reviews). Most of the neural nets and circuits designs are easily modeled in Matlab's Simulink to give real time results similar to those reported. (Whether the results duplicate reality is always a question with these types of models). Flights of fancy (the reason for modeling in the first place, at least the addictive part) can then be implemented according to the capabilities of the reader. The book also discusses "hard wired" CMOS chips available replicating biological systems with plug in units to standard computer I/O units (Motorola, National, and Fuzzytech). However a larger question comes from this book. How can the output of a non-deterministic system be modeled by deterministic model (hardware or otherwise) inputs (H.Wilson)? Without a specific knowledge of the role that neural architecture plays in the phase modulations and oscillatory behavior, how can information be transferred by digital or analog devices duplication neural transmission. As the author puts it in Chap. 12, "Furthermore it is not even clear what the goal of a learning algorithm for pulsed neural nets should be; the goal to learn a function or a function (operator). This book is not a failing because it cannot answer this question. Indeed, the avenues it reviews and discusses opens up many more fields and sparks new uses for the fields it introduces.

This book presents a general overview to the growing field of spiking neural networks and their VLSI implementation written by many of the major figures in this field. It begins with several clear explications of the spiking response model which has been recently popularized by Gerstner (who has several good entries in this volume) and why looking at such a model might be a good idea (it really is completely fascinating). As with all neural network theory, understanding the model will require a fairly solid mathematical background. The middle portion of the book is dedicated to VLSI implementation (which I am not involved with so won't comment) and the last chapters present a wide variety of articles from the highly mathematical to advice for digital simulation of such networks. Chapter 10 is by far the most mathematical chapter and presents the analytic results that have been derived for a homogeneous fully connected network. Although this is far from a complete reference it provides a clear explanation of the reasons for this direction and enough good references at the end of each section to get you started. I have continually turned to this book while getting started in my research in this area.

For far too long, most books about dreaming have been the realm of TV psychologists, mystics, and charlatans; G. William Domhoff is working to change all of that. This book is an excellent overview of recent breakthroughs and future possibilities in quantitative content analysis and neuroimaging studies. Domhoff also makes a convincing case for abandoning the Freudian and Jungian tenets that strangled dream research for most of the last century -- and he sows the seeds of a new cognitive theory that could guide research in the new millennium.

My area of interest is REM sleep and dreaming and I have read many books and scientific journal articles on the subject. This book is full of statistics and data and is so dry I became dehydrated reading it. It is a good resource for someone who is conducting research on the cognitive aspects of dreams and their content. It is not a good book to get if you are new to the subject. Please read my "So you want to...learn about sleep and dreams" for some great recommendations on the subject.