Parallel Computing Books

This book is subtitled as "An all-inclusive survey of the fundamentals of parallel and distributed computing." It both succeeds and fails on this point. Leopold does indeed cover a wide expanse of technologies and approaches that characterize the space of high performance computing. It is in many ways still an emerging space, so conclusively nailing down every possible thread (no pun intended) in a coherent fashion is eminently difficult. The author's treatment of these different possibilities is uneven, overlooking some important contemporary technologies and implementations. It does cover a wide range of topics within the fields of distributed and parallel computing. Furthermore, within the chapters Leopold treats us to both high-level discussions of approaches and provides a glimpse into some of the implementation challenges involved. On the latter point especially, this book is very useful in that it gives the noninitiate some understanding and appreciation of the peculiarities of parallel programming, without requiring substantial technical background in the technologies. The examples in High Performance C and Parallel Fortran were very enlightening.

Where the book fails is that it is far from "all inclusive". There are a number of prominent and important developments that have not been included. Similarly, there are other interesting newer technologies that have only received cursory treatment. Examples include:

- No mention of SETI@Home. SETI@Home is the poster child of massively distributed computing, and with 15 teraflops of raw computing power, it is more capable than IBM's ASCI White supercomputer.
- No mention of distributed.net, or other notable exercises in public and commercial grid computing.
- Grid computing gets only a glancing reference at the tail end of one chapter. A comparative analysis of this important and still-forming space is glaringly absent from this text.
- JavaSpaces, Sun's answer to tuple-spaces, gets only a few sentences.
- Java RMI similarly gets less than a paragraph.
- Although DCOM is now basically legacy for Microsoft, it represents an important milestone in the evolution of distributed computing. It receives only a paragraph.
- Talk of web services and .Net would have been hitting the airwaves as the writing of this book as progressing, although possibly late in the effort. However, some cursory mention at least should have been made. There is increasing discussion of exposing grid compute services via web services interfaces, and Microsoft has recently announced their intention to port the Globus toolkit to Windows.
- Oh yeah, about Globus. Barely a mention.

It was clear from the text that the author came from a strong UNIX and CORBA background. The text has the feel of a PhD thesis-turned-book, and the areas of concentration are decidedly academic. There are a few other areas of minor complaint. Some of the wording in the text is clumsy, reflecting inadequate editing. Some topics feel like they are introduced in reverse order, assuming the reader already has some context about the given topic.

The author makes a sometimes-clumsy distinction between paradigms and models. The distinction is important in that an understanding of models brings a reader closer to envisioning how they might tackle a given problem themselves. However, reference to various models are sprinkled throughout the book. A comparative analysis, even brief, would have been very useful had it been centralized.

Those complaints may sound harsh, but overall the book is useful. It demystifies the problems of parallel programming, and provides a reasonably concise starting point for researching the distributed computing space. But, consider this book a starting point, and not an ending point.

This book is subtitled as "An all-inclusive survey of the fundamentals of parallel and distributed computing." It both succeeds and fails on this point. Leopold does indeed cover a wide expanse of technologies and approaches that characterize the space of high performance computing. It is in many ways still an emerging space, so conclusively nailing down every possible thread (no pun intended) in a coherent fashion is eminently difficult. The author's treatment of these different possibilities is uneven, overlooking some important contemporary technologies and implementations. It does cover a wide range of topics within the fields of distributed and parallel computing. Furthermore, within the chapters Leopold treats us to both high-level discussions of approaches and provides a glimpse into some of the implementation challenges involved. On the latter point especially, this book is very useful in that it gives the noninitiate some understanding and appreciation of the peculiarities of parallel programming, without requiring substantial technical background in the technologies. The examples in High Performance C and Parallel Fortran were very enlightening.

Where the book fails is that it is far from "all inclusive". There are a number of prominent and important developments that have not been included. Similarly, there are other interesting newer technologies that have only received cursory treatment. Examples include:

- No mention of SETI@Home. SETI@Home is the poster child of massively distributed computing, and with 15 teraflops of raw computing power, it is more capable than IBM's ASCI White supercomputer.
- No mention of distributed.net, or other notable exercises in public and commercial grid computing.
- Grid computing gets only a glancing reference at the tail end of one chapter. A comparative analysis of this important and still-forming space is glaringly absent from this text.
- JavaSpaces, Sun's answer to tuple-spaces, gets only a few sentences.
- Java RMI similarly gets less than a paragraph.
- Although DCOM is now basically legacy for Microsoft, it represents an important milestone in the evolution of distributed computing. It receives only a paragraph.
- Talk of web services and .Net would have been hitting the airwaves as the writing of this book as progressing, although possibly late in the effort. However, some cursory mention at least should have been made. There is increasing discussion of exposing grid compute services via web services interfaces, and Microsoft has recently announced their intention to port the Globus toolkit to Windows.
- Oh yeah, about Globus. Barely a mention.

It was clear from the text that the author came from a strong UNIX and CORBA background. The text has the feel of a PhD thesis-turned-book, and the areas of concentration are decidedly academic. There are a few other areas of minor complaint. Some of the wording in the text is clumsy, reflecting inadequate editing. Some topics feel like they are introduced in reverse order, assuming the reader already has some context about the given topic.

The author makes a sometimes-clumsy distinction between paradigms and models. The distinction is important in that an understanding of models brings a reader closer to envisioning how they might tackle a given problem themselves. However, reference to various models are sprinkled throughout the book. A comparative analysis, even brief, would have been very useful had it been centralized.

Those complaints may sound harsh, but overall the book is useful. It demystifies the problems of parallel programming, and provides a reasonably concise starting point for researching the distributed computing space. But, consider this book a starting point, and not an ending point.

It's not perfect, but it definitely doesn't deserve the 1 star review that another reviewer gave it ( he apparently wanted a book on MPI programming, which I would suggest he then buy a book on MPI and/or MPI-IO rather than one on "Parallel I/O ..." ).
I read this book years ago, and purchased it again today, and am enjoying reading it again. It covers a diverse and broad range of subjects, without being too light or heavy in any particular area. Mostly I like that the author treats the subject comprehensively, rather than just a HOW-TO of parallel I/O programming ( which the first reviewer apparently expected ). There is discussion of everything from system architecture, to APIs, to physical hardware and interconnects.
Well done, if not perfect, and the contents definitely match the label. I would definitely suggest this book for anyone wanting a good introduction to the subject, or someone looking for a comprehensive overview that is enjoyable to read.

I bought this book recently to enhance my knowledge of parallel I/O to improve a climate model built on MPI technology. To my disappointment, the author spent less than 1/10th of the book to actual parallel I/O programming. And the mere effort is a list of a few MPI subroutines. The rest of the book tries to cover everything about high performance I/O but does not convey anything USEFUL. I feel the price is way too high for such a mediocre book that talks about too broad a topic and fail to focus on anything specific. I glanced through the book and read the chaptors that talked a little bit about MPI and scientific data management. And now the book is gathering dust on my shelf.

Actually, I don't want to review this book since I am one
of the book editors. I just wish to point out that the book
is incorrectely referenced. The actual title, publisher and
editors are as follows:

title: Cellular Automata: Proceeding of the 5th International Conference on Cellular Automata for Research and Industry, ACRI 2002.

Editors: S. Bandini, B. Chopard, M. Tomassini

Publisher: Lecture Notes in Computer Science, Vol. 2493, Springer
Verlag, Heidelberg, 2002.

Banned in the UK, the bizarre diatribe attempts to causally link the incidence of child abuse in a country with the age of its record buying public. 265 pages of highly dubious statistical analysis cover Britney Spears alone, with authors perhaps telling us more about their own sexual proclivities than those of the small third world countries they are studying.

I've upped the score to three stars because any book that covers Britney Spears published in 1995 shows a degree of foresight that one would not expect from authors this narrow minded.

This book give you a brief description of parallel computing, simple example for you understand the concept of parallel computing.

It has all the basic bits that a beginner needs to get started in numerical computation: polynomial approximation, numerical integration, and linear systems. The latter is a real strength, since it offer not just exact techniques (like Gaussian elimination), but iterative techniques including sparse-system special cases and more general conjugate gradient techniques. There isn't much cut&paste code here, but theorems usually appear only as conclusions, not as arguments. The last sections, on iterative solutions of linear systems, give more in the way of concepts than guidance, but a determined reader will find value.

Despite strengths, this book has significant weaknesses. Error analysis is thin, and that's what really sets good analysts ahead of the pack. The book predates wide acceptance of symbolic algebra packages, so it favors Taylor series approximations over the superior but tedious orthogonal polynomials. It notes cache:memory penalties under 1:10, where they're typically over 1:100 today. And its discussion of performance processor architecture barely approaches adequate, even by 1993 standards. That dates back 10 generations of Moore's Law, ten doublings of transistor count or 1000x. That's a lot, and a world with a Blue Gene in it is a very different place.

Still, the basics haven't changed. Despite some obscurity in the later chapters, it's still good for the first few things a numerical programmer needs to know, including a little parallelism awareness. If I were teaching a basic course in scientific computation, it would still be in the running when I went to choose a text.

//wiredweird

Parallel processing is an important but rather specialised field in programming. In general, the subject treatment in this book can be useful, especially given that it covers both linux and Microsoft Windows.

But why is the parallel language a variant of Pascal? Ever since the early 80s, C overtook Pascal in usage. Very little programming goes on in Pascal anymore. It is befuddling why the people who came up with parallel Pascal did so. Far more effective, in terms of outreach, to have derived a parallel language from C [or C++]. While this may not have been the fault of the book's author, it greatly hinders its uptake.

I have had bad experiences with cluster books before, this one is no exception. There's no how to in here, just a bunch of lecture. For mega nerds only, average computer people need not apply.

The other reviewers are right, this is a genuinely pathetic book that you should not even consider buying.

This book might be worthwhile if you are starting more or less from scratch and know next to nothing about linux, computers, or networks, but if that's the case your first step should be to hire someone who knows, and they won't want this book. it pontificates in general terms about a number or obvious issues without explaining explicitly how to tackle any of the multitude of configuration issues that come up.

A "Beowulf" is the concept of using a network of low cost personal computers for distributed processing. The book doesn't specify a particular configuration. The author describes the three fundamental parts of a Beowulf system: node hardware, network hardware, and parallel software applications. Three chapters explain how to use the message passing interface (MPI) standard to distribute the work for a program executing on multiple nodes. An example MPI program for sorting is presented. The features of the Linux operating system are covered in Chapter 4 since Linux is a common choice for Beowulf clusters.

Before I purchased this book I was completely intimidated by the idea of building a Beowulf Cluster. I found this book to be a surprisingly thorough and accessible resource, allowing me to get a great understanding of what was needed. I've just placed my order for the equipment for my first cluster and can't wait to get it up and running!

As most other readers have elaborated, the book has little or no real substantive material concerning the actual building of a beowulf. Everyone knows Myrinet is better than gigabit ethernet, bu HOW MUCH better for a given class of problems? For any problem???

The book does not even consider addressing real issues and configuration choices, but babbles with obvious choices like ssh vs rsh etc, for pages and pages. I feel I was ripped off. These guys seem to be just resting on their laurels and apparently it doesn't seem to bother them that they are giving the intended audience of the book no real value whatsoever. I believe they should either research the matter thoroughly and completely rewrite the book, or just withdraw it from circulation. One can get orders of magnitude more relevant and reliable information just by reading HOWTOs and Googling around.

I just wish I had read the other reviews BEFORE buying the book.

If a reviewer found this book useful for preparing for a Microsoft test, then judging from results, it must be a useful book.

I found it, however, to be inaccurate in chapter 6, when it discussed how to cluster DHCP & WINS.

Moreover, having created a real cluster on the basis of research in other Microsoft materials, I don't see how anyone could actually understand, from this book, with its overly concise and telegraphic descriptions of cluster activities, how even something as basic as disk groups work.

I hate the way the authors just dump terms like "domainlet" (page 7 & 35) and "dumpconfig" (page 147 & 148), neither of which is mentioned more than once, into the text and expect that the readers understand them.

I hate the way, as in almost all Microsoft training kits, one passage contradicts another.

I hate the way it seems as if the Microsoft Training Kits are written by flunkouts from remedial English classes.

A good example of this is Kay Unkroth's "Microsoft Exchange Server 5.5 Training Kit." This person isn't a native speaker of English, considering her bizarre sentences.

Moreover, that book is poorly proofread with obvious grammar errors scattered through it.

I do this stuff for a living and poor quality just ticks me off.

If a reviewer found this book useful for preparing for a Microsoft test, then judging from results, it must be a useful book.

I found it, however, to be inaccurate in chapter 6, when it discussed how to cluster DHCP & WINS.

Moreover, having created a real cluster on the basis of research in other Microsoft materials, I don't see how anyone could actually understand, from this book, with its overly concise and telegraphic descriptions of cluster activities, how even something as basic as disk groups work.

I hate the way the authors just dump terms like "domainlet" (page 7 & 35) and "dumpconfig" (page 147 & 148), neither of which is mentioned more than once, into the text and expect that the readers understand them.

I hate the way, as in almost all Microsoft training kits, one passage contradicts another.

I hate the way it seems as if the Microsoft Training Kits are written by flunkouts from remedial English classes.

A good example of this is Kay Unkroth's "Microsoft Exchange Server 5.5 Training Kit." This person isn't a native speaker of English, considering her bizarre sentences.

Moreover, that book is poorly proofread with obvious grammar errors scattered through it.

I do this stuff for a living and poor quality just ticks me off.

Wow, what can I say - this is really a terrible book for preparing for the exam.
My overall impression of the book after working with clusters and taking the exam is it is more introductory and overview than anything else.

First and foremost there is almost 100 pages in the book that the exam doesn't cover. On a 400 page book, including glossary and index, that is pretty bad. Most of what is in the book is straight forward enough taken from the exam objectives, but there was quite a bit covered in the exam that was not included in the book. The end of chapter questions in the book are a minuscule compared to the extremely lengthy questions on the exam.

There is mention of utilities that are required for certain functions of MS clustering but no background or much detail about them. They are mentioned in a "matter of fact" role than anything else. The info is available on the MS site but if a training kit mentions a resource they could have just as easily put some detail from MS instead of searching for it.

It is not all bad, there is a good introduction to some of the concepts of clustering. I gave the book 2 stars because it is not very effective as a study guide for the exam. Some of the info on the MS site is included in the book. Be assured there is little chance of "paper" MCPs in clustering with this as the official MS Study guide for exam 70-223.

This is a difficult exam with little material available on the subject. This book is no where near what it takes to pass the exam. I passed this exam with an excellent LAB, work experience, combing MS site for any and all info I could find, along with this and other books on clustering.

This is a very good book on preparing to implement a microsoft cluster. The writing is concise and clear. Some of the contect maybe a little difficult for those who have never dealt with clusters. Covers everything you might need to know about quorms and troubleshooting. If you are going to use this to pass the exam(70-224) let me warn you you need to get some experience with the product or you will likely fail. Overall I'd recommend this book to anyone who want to implement a MS Cluster.

If a reviewer found this book useful for preparing for a Microsoft test, then judging from results, it must be a useful book.

I found it, however, to be inaccurate in chapter 6, when it discussed how to cluster DHCP & WINS.

Moreover, having created a real cluster on the basis of research in other Microsoft materials, I don't see how anyone could actually understand, from this book, with its overly concise and telegraphic descriptions of cluster activities, how even something as basic as disk groups work.

I hate the way the authors just dump terms like "domainlet" (page 7 & 35) and "dumpconfig" (page 147 & 148), neither of which is mentioned more than once, into the text and expect that the readers understand them.

I hate the way, as in almost all Microsoft training kits, one passage contradicts another.

I hate the way it seems as if the Microsoft Training Kits are written by flunkouts from remedial English classes.

A good example of this is Kay Unkroth's "Microsoft Exchange Server 5.5 Training Kit." This person isn't a native speaker of English, considering her bizarre sentences.

Moreover, that book is poorly proofread with obvious grammar errors scattered through it.

I do this stuff for a living and poor quality just ticks me off.

I agree with others, this at full price is not worth it. But used or at a lower discounted price ( $5.00 ) it's very helpful. Value is in the eye's of the beholder. It's basic but it is was helpful.

Gives a Very basic overview of Clustering. Would be useful for somebody who has no clue about the world of clusters, Otherwise the cost is not worth it. If you are a techie, you can scour the net or look up some Magazine and get the same info without wasting any money. But, if you are a IT manager or Salesman, surely this book is worth decorating your shelf with ;-)

Gives an overview of general basic clustering principles, nothing real substantial. If your looking for the best basic understanding, spend a couple hours scouring the Net. (Time better spent!)

The topic of high-availability in the HP environment is worth understanding in detail for anyone who deals with mission critical systems. Unfortunately, the details and effort behind an implementation and/or configuration of MC/Serviceguard is not addressed. This makes the book essentially a kindergarten level primer of generic solutions, something you could get out of Datamation or Informationweek at no charge. If you don't know anything about redundancy, duplicate I/O paths, UPS, clusters, this may be your book. Otherwise, if you want something you could really use look elsewhere.