Parallel Computing Books

Another "A-List" Java book that needs to be in every Java reference library. "Java Concurrency in Practice" provides clear and concise coverage of a nontrivial subject.

In terms of the concurrency pragramming, this book is explaining more comprehensive concept and programming model of the thread programming. Also, it is good for upgrading to Java JDK 1.5 programming skills.

To sum, the author introduces the deeper and well understandable thread theory for Intermediate and advanced java programmers.

For me, the most helpful feature of this book are the advices about what not to do -- Best practices on Java concurrency. The authors addressed all the problems I've had with Java concurrency. For those are not using Java 5 yet, it is interesting to remind that the examples are implemented in Java 5.

It's a great book that everyone should read. You'll definitely wind up stopping yourself from some common mistakes that result in the ever-so-common "WTF?!?!!?" reaction when debugging for a couple hours and seeing things working just fine, but not working on and off in production. Yeah... then it dawns on you that there's a synchronization issue.

This book helps keep those kind of issues in mind much, much better.

The only downside to the book is that it's a complete bear to read. It's just an exceedingly difficult book to work yourself through. I actually finished two other books while reading it. It's just really heavy without any real breaks in there to keep it entertaining.

Again... great book, but in a next revision I hope the authors take some time to just make it a bit of a lighter read.

I've been doing Java development for close to thirteen years now, and I learned an enormous amount from this fantastic book. For example, I knew what the textbook definition of a volatile variable was, but I never knew why I would actually want to use one. Now I know when to use them and when they won't solve the problem.

Of course, JCP talks about the Java 5 concurrency library at great length. But this is no paraphrasing of the javadoc. (It was Doug Lea's original concurrency utility library that eventually got incorporated into Java, and we're all better off for it.) The authors start with illustrations of real issues in concurrent programming. Before they introduce the concurrency utilities, they explain a problem and illustrate potential solutions. (Usually involving at least one naive "solution" that has serious flaws.) Once they show us some avenues to explore, they introduce some neatly-packaged, well-tested utility class that either solves the problem or makes a solution possible. This removes the utility classes from the realm of "inscrutable magic" and presents them as "something difficult that you don't have to write."

The best part about JCP, though, is the combination of thoroughness and clarity with which it presents a very difficult subject. For example, I always understood about the need to avoid concurrent modification of mutable state. But, thanks to this book, I also see why you have to synchronize getters, not just setters. (Even though assignment to an integer is guaranteed to happen atomically, that isn't enough to guarantee that the change is visible to other threads. The only way to guarantee ordering is by crossing a synchronization barrier on the same lock.)

I've seen hundreds of web site crashes. Every single one of them eventually boils down to blocked threads somewhere. Java Concurrency in Practice has the theory, practice, and tools that you can apply to avoid deadlocks, live locks, corrupted state, and a host of other problems that lurk in the most innocuous-looking code.

OK, so perhaps you don't need this book, but you'll enjoy it anyway!
The best review and tutorial on clusters available, this book is also funny
enough that you probably don't want to read it in a quiet environment.

As other reviewers have said, this is an excellent book and is a *must have* for anyone exploring practically any aspect of cluster computing. Even beyond the quality of the information conveyed, the writing style is wonderful and the author makes an otherwise abstract and cumbersome topic quite readable and quite approachable. This book is regarded as the Clustering Bible worldwide, and I've seen copies placed prominently in the bookshelves of individuals in Beijing, Tokyo, Paris, and the US. Microsoft's own Cluster Server was codenamed 'Wolfpack', as an honerable reference to the cover art of this specific book.

Aaron McKee
Clustering Products Manager
TurboLinux Inc.

In a subject which is probably considered dull Pfeister has written a book which is actually entertaining as well as informative.If you are new to high perforamnce computing this book will begin from the basics and teach you all the way.If you are a pro you still will find it a great refernce material worth a read.You are bound to find something new in it.

This book provides a terrific introduction to the hardware, software, and systems architecture of parallel computing, candidly discussing the issues and trade-offs in various approaches. The emphasis is on clusters, but there is lots of information on the whole continuum from single processor machines to SMPs to clusters to distributed computing. Pfister will leave you with a better understanding of things like how SMP machines keep processor caches coherent, what the differences are between SMP, NUMA, and distributed computing, how various cluster products work, real world cluster issues (like system administration), programming models used in parallel computing, and why programming code that runs efficently on these architectures is usually the hard part.

The book is somewhat dated - nothing about Beowolf for example - but the concepts remain valid. Many of the issues are illustrated with reference to mainframe clusters, especially IBM's 390 sysplex, which I found particularly interesting since I don't have much experience with these systems.

The style is highly readable and informal, but not insultingly non-technical. The book is loaded with opinion and insights - it is not a dry textbook of issues related to clustering. Highly recommended for anyone in the business of creating information systems that need to run fast.

While struggling through this book, I spend more time thinking about the characters of this author than about cluster. He has this unique style of telling a subject like a friend telling you a story and it goes like : I am going to tell you this .. but first let me explain why I tell you this .. and now I want to tell you how I am going to tell you this .. and finally this is this, believe you me .. OK, truth is - I lied, it is actually this ... and on and on. He successfully made "this" simple and understandable to a certain degree, but the "how I, why I" portion mix with some high IQ humours did confuse me no end - though it is fun. I find that at some stage I have to use a finger to point to a key sentence and said " OK, Pfister is going to tell me this - lets see how" - then at some stage I will have to judge whether he has finished what he promised and is already on to new stuff. And, he does choose some words that I suspect is more often used in classical literature - rococco, dischotomies, litany, etc - which although harmlessly infrequent, does add some irritation to the already burden mind. All things considered, going through this book is fun. In case you are one of those like me that is quite lost and struggle a fair bit, I did find chapter 6 fairly readable - so perhpas you want to starts from there and work backward/forward if other chapters frustrated you a fair bit.

Materials covered in certain chapters are equivalent to a one semester class at MIT.. though in less detail, the lucid explanation of the fundamental concepts are impressive.. this book is a very good start for someone who has never touched the subject of Parallel Computing before.

This book provides great insight into the _performance_ of hardware. How can one be a good programmer without knowing how it is that data gets to memory, how data flows through the CPU and what it is in the hardware that makes ones code fast or slow? It amazes me that most of the developers that I work with are ignorant or could not care less about the effects of cache misses, what superscalar/superpipelined really means, and how a "improper" stride through their array can make the performance of their code deplorable. This book makes for a great read and is guaranteed to make a programmer a better programmer.

Explains very nicely and precisely the HW fundamentals of the multiprocessors, memory, insights in software and optimization concepts. Very much useful for not to get confused by the buzzwords in the super computer industry. Good book for both the engineers and the sales people of hardware companies. If you are new engineer in processor design or compiler architecture jobs, this book gives lot of insight and learning needed for your job.

It is an excellent book I have read. It includes the latest state of art IA-64 architure, RISC, compiler ...

This is an outstanding summary of research (as of 2001 or so) in leading techniques for compiler optimization in performance computing. There's not much here for the typical software developer, even those who write applications compiled for massively parallel systems. Most compiler writers won't find much of immediate use either - if you haven't mastered Optimizing Compilers for Modern Architectures and Parallel Programming With MPI, you're in for a rough ride. These techniques all apply to parallel systems, with hundreds to hundreds of thousands of processors acting together. And laugh if you want: many of these researchers address the needs of Fortran programmers. Anyone who laughs at Fortran just doesn't understand the performance computing market or the recent advances in that venerable language.

If you're deep into compiler development on that class of machine (or something similar enough) this collection presents 21 chapters, a bit under 800 pages, of cutting edge analysis and algorithms. Topics cover every level, from the micro-level checking of dependencies between one array element and another in a looped computation, up to macro-level OS level constructs for distributing and synchronizing coarse-grained tasks.

Even specialists will find only a few chapters that address their immediate needs. Specialists at this level, however, are used to that. Commercial gold mines today yield one gram of gold per tonne of useless tailings. The ratio is better in this case, but even readers with the greatest interest will skip parts of this goldmine of information. Still, if this is your area of interest, you may well find something of value. Highly recommended to the right reader.

-- wiredweird

Scalable parallel systems or, more generally, distributed memory systems offer a challenging model of computing and pose fascinating problems regarding compiler optimization, ranging from language design to run time systems. Research in this area is foundational to many challenges from memory hierarchy optimizations to communication optimization.
This unique, handbook-like monograph assesses the state of the art in the area in a systematic and comprehensive way. The 21 coherent chapters by leading researchers provide complete and competent coverage of all relevant aspects of compiler optimization for scalable parallel systems. The book is divided into five parts on languages, analysis, communication optimizations, code generation, and run time systems. This book will serve as a landmark source for education, information, and reference to students, practitioners, professionals, and researchers interested in updating their knowledge about or active in parallel computing.

This is deja vu again. We talked about AI and robotics in the same vein. Definitely ground breaking and recommend it. Not sure novices can handle it...

The book is one of the best I have seen in Neural and computer intelligence. Highly recommend if you are a student, academia, computer R&D or hobbyist.

Wish somebody published this before I started following Neural networks or computer intelligence

This book is a wonderful look at the background and future of interaction design. McCullough provides wonderful depth of understanding for the reader on the many discipline that support interaction design: psychology, architecture, cultural anthropology, technology. Not only does McCullough draw the disciplines together nicely, it is done seamlessly to the reader.

My copy is now filled with highlighter marks and it a book I will be returning to for my profession and through time. If you are a fan of well developed end notes to find further information, this book is a charm.

The book digital ground presents new ideas about place and technology. I was particularly struck by the idea of technologies piling up at a place -an interesting problem is how this pile of technologies can be organized into a useful whole - device ecologies, and an extensible system (hardware and software) that can grow over time (and be subject to changes, e.g. devices removed, replaced, added,etc). Another interesting idea is how certain places fulfill or serve different aspects of life or functions, and the technology at a place should then be in accordance with the corresponding aspects of life or functions at that place, or at least be attuned to or be aware of context necessary for such functions and related activities. There are also other interesting ideas and underlying theories in the book which makes it an interesting read, and not only for architects and builders but computer scientists!

This book establishes the foundation mathematics and definitions of what are now called "neural networks". In 1986 these guys (on DARPA grants) figured out the basics of what is (in my opinion) the most significant advance in artifical intelligence since the 1960s. The book is a bit dry, as a fully rigorous academic text usually is, but the results speak for themselves - the techniques and approaches described in this book are used all over in some of the most challenging areas of AI - character, speech, and face recognition, surveilance, applicant screening, and so on.

Read it if you believe artifical intelligence is a bunch of hooey - I do, except this stuff.

Though I believe I am not the first person writing the book review of this historic book, I still feel honored to encourage the new readers to read this one of most important AI research book published in 1986. The book edited by Rumlhart and McClelland was well organized and well written, comprised of a series of independent and interesting topics in neural network researches given by the dedicated authors. The editors themselves are also reputated authors in the connectionist community. The most results in that book never appeared in the past publications and represented the high-quality papers in the state-of-the-art research at that time. Many papers in that book rank the top position of citation rate even today, e.g. the paper about error backpropagation due to Rumelhart, Hinton and Williams. I also got to point out that the importance of the book not only lies in its scientific contribution, but also its philosophical meaning in the AI research (which is somehow influenced by the book 'Perceptrons' by Minsky and Pappert). The successful research results in that book showed people of the potential and new prospect of neural networks in different perspectives. From then on the second connectionist revolution has sprang and lasted today. Nowadays, people still can feel its leading influence by reading it. Upon reading the book again and again, you will always feel inspired at another new way (that is the value of a book!). Try it immediately.

In a word without exaggeration, the importance of this book to connectionist and AI researchers is like the Bible to Christians. Read it, enjoy it, once and again.

The ideas behind Wave-WP, a high-level network-centric programming language presented in this book, are fresh, radical and thought-provoking. The author offers a completely original view on what distributed computing ought to be about. As someone interested in the design of next-generation programming models that make it easy for people to build, manage and control large-scale dynamic network systems, I found Peter Sapaty's work a very worthwhile read.

"Ruling Distributed Dynamic Worlds" describes the World Processing language and the Wave-WP model for implementing large distributed systems. The book will be of interest to anyone concerned with distributed computation, the software of robotic systems and application spaces such as network management and security and other crisis management scenarios.
Wave-WP is an extension of the Wave model of distributed processing expounded in the author's 1999 text "Mobile Processing in Distributed and Open Environments." His new text extends the model to embrace "spatial programming", using a "virtual world" abstraction whose content is assembled using the distributed Knowledge Network concept of the earlier Wave paradigm. Throughout the text, he supplies numerous examples of how Wave-WP interacts with the physical world, in such missions as multi-robot firefighting and hospital maintenance. More traditional applications of distributed processing such as network management are also discussed, as are security applications.
The Wave-WP paradigm superficially resembles the mobile agent paradigm. The author argues that it is qualitatively different to the latter in that mobile agent solutions anticipate emergent behaviour from the specified actions of the mobule agents, whereas Wave-WP operates at a higher level (the "implementation layer"). This allows some complex application behaviour to be generated from remarkably simple and concise Wave-WP code. A number of such examples are presented in the text.
The book includes a detailed description of the World Processing language and of the Wave-WP interpreter, and of the various worlds inhabited by the paradigm (the virtual world, the execution world, the physical world and the "united" world). The range of worlds occupied is (in this reviewer's opinion) the key distinction between Wave-WP and its predecessor Wave architecture.
Readers persuaded of the power of Wave-WP will doubtless be itching to try it out for themselves. Unfortunately, it was not yet available as a product when the book was published (as mentioned by the author in his preface) although earlier versions of Wave have been made available to the research community. Despite this limitation (which may well have been addressed by the time you read this review), the book is a provocative read for anyone interested in innovations in distributed systems.

This is an excellent book especially for students who want to do Masters in Computer Science & Engineering. Concept of Parallel processing, multistage Unix Kernel etc. are excellent. More detailed discussion on SuperComputer Architecture with diagrams are required.

An ALGEBRA for thinking about concurrency and nondeterminism in programs. The foundation of modern designs for communicating between threads.

An elegant and enlightening formalism for what you already know if you write multithreaded stuff that works. Feels just like structured programming did in the 70's: that being then the new formalism what you already knew if you wrote single-threaded stuff that really worked. Quoting from the forward by E. W. Dijkstra: "... the computing scientist's main challenge is not to get confused by the complexities of his own making ...".

Easier to read if you already have experience writing programs that write programs, but readable even if you flatly ignore all the academic computer science terms like "lambda expression" and "static binding". Most fun if you write the programs suggested in the exercises, of course.

Among the top five of my all-time favourite books on programming.