Parallel Computing Books

This book provides a good introduction to Concurrent and Object Oriented Programming in Java. I was fortunate enough to take Dr. Kann's class last semester, while this book was being finalized. This is probably the best book I have been assigned as a class textbook while in school, both in readability, and usefulness.

Reading this book will make programming concurrent applications in Java easy, just by following the design patterns in the text. Use of notification objects and synchronization is well explained. If you'd like to understand object oriented concepts such as composition and classification in an easy to understand way, this book will help. Throughout the course of the book, the reader can watch the construction of a concurrent program from beginning to end. The concurrent program (the animator), is also a handy class which can be used to perform simple multithreaded animations.

The section on distributed computing (Java RMI) implements a simple chat client/server in Java which can be used across the internet. Very easy to write your own chat program in Java using the material in this book.

Code included in the book should have you up and running quickly. If you're ready to start utilizing the full power of java objects and multithreading, buy this book.

Unlike conference proceeding, this book is a collection of papers that were presented at a Workshop, which are usually shorter in length and more focused on a specific area. This Workshop, 4th Austrian-Hungarian Workshop on Distributed and Parallel Systems, covered four specific, and yet very important aspects of Distributed and Parallel Computing:
1) Tools and techniques in Cluster Computing, which covered Load Balancing techniques, tools used for simulation and performance prediction, management tools used for abstraction of the underlying infrastructure
2) Global Grid Computing, which covered topics that concern large Grid management and Load Balancing in global Grids, dynamic resource discovery techniques and virtualization of resources to represent the user with a Single System Image (SSI)
3) Parallel Software Development techniques and tools, which covered the latest trends and available tools for developing applications for the gird, architectural techniques for caching, debugging techniques to shorten development time, and testing and deployment ideas for large grids.
4) Dependable and Fault-Tolerant Systems, which covered topics such as fault-tolerant techniques for storage systems, getting around the unpredictability of creating a fault tolerance distributed and parallel system, and failure detection and recovery techniques

As mentioned, the book/workshop does not try to cover everything under the sun, but what it does cover; it does so with applicable, useful and relevant papers. The researchers are basically presenting their own experiences and best-practices to the readers.

Even though the topics presented are academic for the most part, there are a number of ideas, techniques and tools that the reader can take away. One such tool is P-GRADE. There are three papers covering this tool in this book: two on load-balancing and one in software development and engineering. P-GRADE is a graphical tool that provides programming environment for developing parallel applications using graphical representation for communication, etc... It also has a number of tools for debugging, monitoring and visualization. The tool takes the graphical representation of your application, and develops C code for it, which could be compiled for any system. It is an open-source, free software and it is getting plenty of attention in Europe. The tool makes the complex task of developing software for a Grid Environment simpler, and even though it does not literally write the code for you, it will assist you where it can such as easing the pain of network programming and communication.

I highly recommend this book for architects, researches, professionals interested in papers that are not entirely research-based and there are applicable techniques that can be learned from them.

These are the proceedings of the 8th Heterogeneous Computing Workshop, also known as HCW '99. Heterogeneous computing is a very important research area with great practical impact. The topic of heterogeneous computing covers many types of systems. A heterogeneous system may be a set of machines interconnected by a wide-area network and used to support the execution of jobs submitted by a large variety of users to process data that is distributed throughout the system. A heterogeneous system may be a suite of high-performance machines tightly interconnected by a fast dedicated local-area network and used to process a set of production tasks, where the subtasks of each task may execute on different machines in the suite. A heterogeneous system may also be a special-purpose embedded system, such as a set of different types of processors used for automatic target recogntion. In the extreme, a heterogeneous system may consist of a single machine that can reconfigure itself to operate in different ways (e.g., in different modes of parallelism). All of these types of heterogeous systems (as well as others) are appropriate topics for this workshop series.
---excerpt from book's Message from the Steering Committee Chair

Best documentation of actual applications I have ever seen. This book has helped my students to understand the real world of parallel computing.

An excellent introduction to the world of Computer Networks.

This is a delightful new presentation of Process Algebra, which was much needed since the classic text by Baeten and Weijland is getting outdated. In comparison, I find Fokkink's text more easy to digest, more precise and going significantly further. Indeed, it should be for a theory that has evolved rapidly in the last decade. Process Algebra is on its way to become the tool to build full-proof software that will lead us out of the present day software jungle. I have used this text in my one semester class in Trier and can recommend it to any serious student of computer science.

Dr. Sapaty describes some very far out and very powerful technologies that are coming on line for organizational management of dynamic distributed systems. The Internet will no longer be just for sending emails and basic e-commerce in the very near future. I see WAVE as a general solver to a multitude of distributed systems. WAVE is a new methodology and technology enabling the solution of any reasonable, and some that may not be currently reasonable, problem in distributed computer networks, with code mobility being a key principle. The technology allows for the problem description on a semantic level, shifting traditional coordination, synchronization, and data or agent exchange to automatic implementation, making application programming very compact and simple.

This WAVE paradigm looks very powerful: One issue with it is that it is so radically new and improved from what is currently being done, that it will take a while for people to actually believe it is real! I do, and now I want - no, need - to help convince others...The fact that you are reading this review indicates that you have interest in this field. As such, I recommend that you purchase this book, as well as his new book that discusses advances in the spatial programming of distributed dynamic worlds (which is due to be released soon) and judge for yourself. You will not be disappointed.

Larry M. Deschaine, PE / Systems Optimization Design Engineer / Fortune 500 Company / MIT '84

< One of the most insidious sources of programming problems in the RISKS archives involves concurrent programming. Synchronization, locking, message passing, and other tight-coupling mechanisms are extremely difficult to do properly. Programming languages and operating systems are not necessarily much help by themselves.

Fred Schneider has put together a wonderful book on how to do concurrent programming correctly. Whereas the book is ideal for a one-semester course (and more), it is also very valuable as a reference work. It should be read by everyone deeply involved in writing critical programs. Although its focus is strongly on formal methods, I have long claimed that formal methods can be enormously helpful if you are really concerned about correctness in concurrency, for which most unproved algorithms tend to have flaws (and a few ``proved'' ones may also). Furthermore, the implementations of such algorithms are always in question, and formal methods can help significantly there as well.

[From the Risks Forum, vol 19 no 51]

Excellent book. I agree with much of the editorial review. I have found this book very useful for my work in distributed computation. The authors are thorough and exact in their work and rely heavily on matrix and probabilistic methods for their results. If you are into parallel and distributed algorithms, then this is a must have book.

This is a well-written book aimed at researchers in the field of quantum chemistry -- from graduate students to long-standing experts -- who require a concise and clear description of the most important problems facing efforts to parallelize ab initio quantum chemical programs. Given the rapid emergence of new petascale computer systems containing thousands to even millions of computing cores, the timing of this book is fitting.

Full disclosure: I know the authors of this book well. I have published two peer-reviewed journal articles with Dr. Janssen and one with Dr. Nielsen, and I received lab-directed research and development funding from Sandia National Labs through a Department of Energy project of Dr. Janssen's in 2001-2004. In addition, I reviewed the proposal for the book for Taylor and Francis publishers. However, I was not involved in the writing of the book at all. I purchased it of my own accord, and I am writing this review only because I am very impressed with the finished product.

I found the book to be tremendously enlightening. In the first half, the authors provide an overview of essential aspects and tools of parallel computing: hardware, network topology, message-passing software and methods, threading, load-balancing, etc. In addition, they give a fairly detailed explanation of methods for modeling the parallel performance (speedup and efficiency) of algorithms, as well as aspects of parallel program design. One of the strengths of the book is the way the authors make their points clearer by constantly returning to a few specific examples, including matrix-vector multiplication and the second-order Moller-Plesset perturbation theory (MP2) algorithm.

They then make use of the fundamentals developed in the first half of the book to address several key problems in quantum chemical programs: two-electron repulsion integral evaluation, the integral-direct Hartree-Fock method, as well as canonical and local MP2 energy calculations. These provide fertile soil for discussions of load balancing, collective versus one-sided communication, and hybrid (simultaneous shared- and distributed-memory) parallel methods. Each example is well-supported by performance models that provide a clear analysis of the scalability of each algorithm.

My only criticism of the book is that it stops too soon. The numerous problems associated with parallel implementation of more advanced and complicated methods, especially coupled cluster theory, are not discussed, and I would have enjoyed reading the authors' take on this area of on-going research.

Nevertheless, I believe this book will prove to be extremely valuable to those developing quantum chemical program for emerging massively parallel supercomputers. The authors' perspective on the parallelism problem is state-of-the-art, and our field would be wise to listen carefully to what they have to say.