Parallel Computing Books

This book covers a lot of ground. Wolfe starts off with a chapter on machine architecture, then moves into language features that can be parallelized. Then there are chapters on graphs and linear algebra then he dives into an encyclopedia of data dependence, data flow, and control flow algorithms. I only made it about 1/3 of the way thru the book as it is oriented toward people actually implementing high performance compilers ie it's a practitioners guide. I gave the book four stars because the type setting (previously mentioned lack of periods) is appalling. Especially in this day and age.

If you want to get into data dependency analysis of compilers, this book is absolutely the best. Many compiler book authors do not have the experience of writing a compiler of industrial strength, but Michael Wolfe does.

This is a modern classic. If you develop compilers for high performance computing (HPC), this must be on your shelves. It has a heavy emphasis on Fortran, the workhorse of HPC, but is widely applicable to other languages as well. The majority of the book's content reflects the large majority of processors that carry today's HPC load: sequential, von Neumann engines, even the computing ensemble as a whole has lots of them.

This book's real contribution is in its analysis of the loops that process arrays. Wolfe presents a number of ways to characterize dependencies, using quantitive techniques that go well beyond the graph-based presentations elsewhere. By casting the dependency problem in terms of integer programming or linear algebra, Wolfe make huge bodies of problem-solving knowledge available to the compiler developer. He also uses these quantitative terms to give new insight into loop transformations that the reader may already understand.

Despite the irreplaceable value of this book, I found it maddening to read. For some reason, the periods dropped off of nearly every sentence in the text. Then, just when I got used to that quirk, the periods appeared again. Their coming and going was so irregular throughout the book that I never really got into a steady reading rhythm. A much worse problem appeared throughout the first chapter, though. It's an exceptional discussion of matrix multiplication, a staple of performance computing. The chapter presents it again and again, to demonstrate differences in looping constructs and the organization of memory access. That part of the discussion was great. The problem is that it's wrong - a systematic error, in seemingly every example, replaced the scalar multiplication at the core of the algorithm with addition. It's hard enough trying to understand the interactions of loops nested five deep, language features, and memory hierarchies. Doing all that while mentally correcting the text's blunders is simply infuriating. Perhaps those problems were fixed after the 1996 edition that I read - for the reader's sake, I hope so. As it was, that first chapter left me wary of everything else in the book.

If you need it, though, you need it. Grit your teeth, brace yourself, and dive in. This is a valuable complement to the Allen&Kennedy text, and essential for anyone creating compilers for HPC architectures.

//wiredweird

I bought this book for a course on Parallel Computing for which we did some basic MPI programming. This book was useful to a great extent in describing the syntax of the MPI routines as well as in providing ample examples. To my knowledge, it doesn't contain advanced MPI concepts but it is good in explaining the basics of MPI if you are a beginner.

This book is an excellent introduction to programming with the MPI. It gradually introduces concepts from the simple to the complex. This is done with examples that illustrate the use of different techniques. The examples include the code to implement them. The programming examples alternate between Fortran 90, C, and C++. However, after giving the example in one language, the bindings for the MPI functions in the other two languages are presented. In addition, the programming examples in either language are easily understood. Although my first choice of programming language is Fortran (and I have very basic knowledge or C), I was able to follow the examples in C and C++ and to write their equivalents in Fortran so I could test them on our computers.

I liked the reference as an introduction to MPI, but it switched between fortran and C so often that I spent a fair amount of my time just trying to interpret the code samples and synchronize to the syntax of the language they were using. You shouldnt have that problem if you have used Fortran in the last ten years but i havnt touched it since college. You have to be able to work with both languages though because many of the key concepts are only demonstrated in one language. Also the examples were usaully spread out into mini snipets as opposed to a larger block that might show a more complete picture.

I'm a big fan of open source and virtualization, so I was probably predisposed to like this book, but it far exceeded my expectations.

By the way, this is a good looking book - the images here don't do it justice. Whoever was responsible for the cover deserves a nice bonus or promotion.

But on to the meat: I learned a lot here, and not just about Xen. Although it's obvious to me now, I never thought about the teaching opportunity presented by writing a paravirtualized kernel - since you aren't dealing with real hardware, a lot of the nastiness is taken away, leaving you free to concentrate on the more general OS issues. As the foreword points out, this book could be used as a text in an OS kernel course.

The author writes very well, and explains difficult concepts with grace and ease. This was very enjoyable to read, with (for me) just the right level of detail. Definitely recommended for anyone with an interest in kernel internals, and of course if you specifically need to know about Xen, this really is "The Definitive Guide".

This book is for anyone who wants to understand the inner workings of the Xen Hypervisor. There are also examples where to look at the Xen source code and some porting hints for bringing Xen to other platforms. If you look for a book about Xen administration or management then it is definitely not for you.

It came as somewhat of a surprise to the industry that coupling together several PC's enabled the construction of what was in effect a supercomputer at a small fraction of the cost.

What began twenty or so years ago has now influenced the design of CPU's and the intereconnection 'LANs' that facilitate the transfer of data between the processors. And this clearly hasn't stopped. The AMD Opteron CPU's and Intel's PCI-Express are simply the latest innovations in silicon, and more is coming.

From a system architecture standpoint, we have (and the book discusses) clusters, Grids, and distributed processor systems -- all of which are fairly loosely defined with plenty of room for very good discussions over several beer.

What this book brings is an excellent introduction into the state of the art in parallel computers as it exists today. As is often the case with books that are pushing the state of the art, it is written by a large numnber of experts and edited together. Each chapter covers a particular area in depth from the design of the hardware to the languages (primarily Fortran and Java), to the solution of a series of common problems that are frequent in several different application areas.

This book is an excellent summary of parallel computing as it exists today. It would be of particular help to the person responsible for writing the proposal for an organization to buy/build one. The book is probably a bit too advanced for a course at an undergraduate level, but would be excellent for first year graduate students in a wide variety of fields from computer science to bio-informatics, data mining, cryptography or any number of other fields requiring heavy duty computation.

A sturdy review of the different geometries of parallel machines, and of how to program them. Examples are given of algorithms that can be efficiently ported to these machines. The handbook is useful in summarising a lot of results scattered over conference proceedings and journal papers. The range of applications described is impressive.

The text is probably suited for a graduate level course. A bit too specialised for most undergrad CS majors.

Forced into the world of denotational semantics of functional languages, I found this book actually quite interesting and informative. Using lattice theory to look at the properties of functional evaluation provides for hours of entertaining reading.

Preface
List of figures
1 Introduction
1.1 Functional Programming Languages and Lazy Evaluation
1.2 The Evaluation Transformer Model of Reduction
1.3 An Introduction to Abstract Interpretation
1.4 Outline of Book
2 Operational and Denotational Semantics of the Typed Lambda Calculus
2.1 The Typed Lambda-Calculus
2.2 Sets with Structure
2.3 Interpretations
2.4 A Result Relating the Operational Semantics and Standard Interpretations of AT

2.5 Drawing it Together
3 A Framework for the Abstract Interpretation of Functional Languages
3.1 The Abstract Interpretation of AT
3.2 Abstraction and Concretisation Maps
3.3 Correctness of the Framework
3.4 Drawing it Together
4 Some Example Abstract Interpretations
4.1 An Abstract Interpretation for Evaluation Transformers
4.2 On Defining Abstract Domains
4.3 A Systematic Approach to Deriving Abstract Domains for Lists and Trees
4.4 Drawing it Together
5 Evaluation Transformers
5.1 Safe Changes to the Evaluation Strategy
5.2 Definition of Evaluators
5.3 Determining Evaluation Transformers
5.4 Drawing it Together
6 Implementing Functional Languages on Sequential and Parallel Machines
6.1 Graph Reduction
6.2 The Spineless G-machine
6.3 Relating the Graph Reduction Model to the Typed Lambda Calculus
6.4 Compiling Code for Evaluation Transformers
6.5 Drawing it Together
7 Relationship to Other Work
7.1 Abstract Interpretation
7.2 Polymorphism
7.3 Other Program Analysis Techniques
7.4 Implementation of Lazy Functional Languages
8 Epilogue
A Proofs Omitted in Earlier Chapters
B The Spineless G-Machine
B.1 Compilation Rules
B.2 Initial G-machine state
B.3 An Interpreter for the Spineless G-Machine
Bibliography
Index of Symbols
Index

Parallel and Distributed Simulation Systems, by Richard Fujimoto, brings together all of the leading techniques for designing and operating parallel and distributed simulations. This is the first book to bring this material in a single source. Previously, simulation developers had to research a library to journal and conference articles to identify, master, and select techniques appropriate to their problem.

Fujimoto divides the material into three sections. The first introduces the reader to the real world problems that have given rise to the need for PADS techniques. He also includes an introduction to the field of discrete event simulation, laying the necessary groundwork for the reader to understand the rest of the book.

In the second section the book begins to describe many of the techniques that are used to accomplish parallel or distributed operations for discrete event simulations. These are presented very clearly and most readers will grasp them immediately. In this section, the reader gains an appreciation for the difficulty involved in creating parallel simulations that maintain the causality of events within the entire simulation. Distributing the application across multiple computers has the potential to greatly increase its execution speed, but must do so without jeopardizing the causal accuracy of the simulation. Topics covered include conservative synchronization, time warp, deadlock avoidance, lookahead, repeatability, global virtual time, scheduling, and performance. Fujimoto also includes a description of the Georgia Tech Time Warp system in which he and others have implemented many of these techniques.

Finally, the third section describes the application of PADS techniques to distributed virtual environments. Fujimoto focuses distributed military training systems and points toward the potential for applying these techniques in entertainment applications. This section is motivated by a description of the evolution of distributed military training using protocols like simulator networking (SIMNET), distributed interactive simulation (DIS), aggregate level simulation protocol (ALSP), and the more recent high level architecture (HLA). Fujimoto takes great care to describe how PADS techniques are used within software implementations of the HLA. Topics include distributed interactive simulation, dead reckoning, the high level architecture, network technologies, communication protocols, data distribution management, and message ordering.

The book is an essential tool for those creating parallel or distributed simulation applications or conducting research in the field. It provides and excellent level of detail and sufficient explanation to allow others to implement the techniques. It is available directly from Wiley-Interscience and from most of the Web-based bookstores.

This book is a very readable introduction to parallel and distributed computing. It presents concepts well, and offers enough examples to allow one to make sure they have a firm grasp on what is being presented. It is NOT a textbook. It is not full of code, or detailed descriptions of exactly how to implement a parallel/distributed system in a given situation. Those already expert in the field or those seeking low-level detail and code should look elsewhere. I recommend this book to everyone else.

Book needs some more content, it gives a good first level intro but does not develop it into more interesting/advanced things.

50% of the book is a 'sales' pitch for StarLogo.

If you have experience with writing code for simulations this book is a bit light technically.

Still has some interesting ideas in it.

On the surface, the book seems focused on computers and even video games. This makes the activities appealing for middle and high school students. However, the ideas in this book are actually relevant to two entirely different fields: 1) genetics, and 2) artificial intelligence. Both these fields have at their centers the concept of complexity arising out of simplicity. Students who work their way though the activites in this book will find that Dr. Resnick offers a window into a deep mathematical world that has applications far beyond computers and games.

Mitchel Resnick shares a lovely sense of wonder, discovery and fascination in this slim, easily read volume about agent based modelling. Using micro-worlds (termites, ants and other modelling metaphors) Resnick shows us how collective behaviours are more than a simple sum of the parts - and through his experiments using StarLogo programming he shows the nature of emergent behaviours that come through decentralised thinking.

His reading list is a great starting point for anyone fascinated by Complexity and Agent Based Modelling, and so too is his elegant list of guiding heuristics that he has learned through toying with various ants, termites, forest fires and traffic jams.

- Positive feedback (in models) often plays a vital role.
- Randomness can help create order. Random isn't always chaotic.
- A Flock is not a big bird. The behaviour of groups should not be confused with the behaviour of individuals.
- A traffic jam is not just a collection of cars. Emergent objects have an ever-changing composition.
- The hills are alive. Don't just focus on the individual objects - look also at their environment.

This volume helped our research team design approach agent-based modelling, and put us in touch with other avenues of decentralised thinking: for example Network Theory. Best of all, Resnick helped us lighten-up. This book (and its subject matter) has profound ideas, but never loses a delicious sense of awe. Recommended.

This book provided the motivating force to write my first, and last, review for Amazon.com.

Over the past 5 years since my first reading Mitchel Resnick's Turtles Turmites and Traffic Jams, the book has come up on numerous occasions related to several topics, two of which most basically:

1) Writing style - Resnick's clear, well-researched, simple yet profound style. His background as journalist and inventor enables TT&T to walk a new line between source material and criticism.

2) Content - Resnick's theoretical application of emergent behavior to education is robust; his practical educational tools (starlogo and later, mindstorms) are a fundamentally clear and wondrous collapsing of idea into artifact.

I will include this book with few others in my life bibliography.

This is a book describing the research of a team at MIT using a version of the educational language "Logo". Running in a simple graphical environment which supports multiple parallel operation of code in the same shared space. Write a few lines of code for an "ant", then let 1000 of them loose. The current version of this "StarLogo" system is written in Java, and available as a free download for anyone to play with.

The use of Logo is both a strength and a weakness of the approach. The strength is that the code is concise and easy to understand. The weakness is that there is only one source of the software, and anyone wishing to try it is limited to the available download. This would not be such a limitation if the book described the same version, but unfortunately things have moved on a lot since the book was written, and few (if any) of the examples will work without alteration.

As well as the development of the StarLogo system, the book covers experiments in emergent behaviour. Typical sections include how parameter and environment changes can affect the growth and development of simulated ant colonies, and a theoretical basis for those "phantom traffic jams" we have all experienced.

This book is certainly interesting if you are interested in developing parallel software simulations, or if you are interested in marginal computer languages, but don't expect the code to work without effort.

Robin Milner doesn't need my recommendation, I'm sure, but I want to respond to reviewers who make ridiculous comments like "not for the faint of heart." This book is simply beautifully written, and could not be more clear. Moreover, Milner's intellectual honesty and humility comes through on every page; he never resorts to hand-waving, or obscurantism, or leaving important but tricky steps to the reader "as an exercise".

I wish all textbooks were like this - this is definitely the most readable work on the subject. I lost my first copy, but had to buy a second one, it is just fun to read.

Basic primitives? Professor Milner, If you are writing a book for students, it helps if you make it comprehensible. We didn't all go to Cambridge.

Process algebra in general is not for the faint of heart. It is therefore no surprise that this book is at times difficult. However, for those interested in the formal analysis of concurrent systems, this is a very good introduction to the algebraic view.

Milner presents a set of useful, complex theories concisely. I agree with the reader above in that this isn't bed time reading. However, the rewards are worth the effort. An audience interested in computabilty, semantics, parallel systems and modelling will find that this book presents new tools and ideas in those subjects. Not for the faint of heart, hence only 4 stars.

I'm a hardware person who have been dealing with RF devices for almost 10 years. Recently I was commissioned to make a MFC based Visual C++ code to open and communicate the serial port based liquid crystal filter. Since I was not a computer science student, this was really beyond what I expect. After I bought this book with Ivor Horton's Visual C++ 2008, my work was like..'a morning coffee.' It has the line to line explanations, old C++ to VC++ conversions...etc. If you need different language like MATLAB, VB, Labview, it will cover them, too.

Some people said this book's example is so specific to follow; I don't think so. If you need to open serial port with VC++, that would mean you also have to make some hardware devices or analyze the hardware stuff by yourself.
Even if I didn't use the example provided by this book, this book gave me the enough example how to test the serial port communication. You can apply this to your own work.

For testing or making some serial devices, you basically need a multimeter, female or male serial ports, some cables..etc.

Also, you may have to configure your computer's bios setup.
Don't miss the CD on the back cover.

I am a graduate student who has a lot of interest on this field. I was searching for a good reference for my research and found this book. This book contains a lot of details so that even beginners can easily follow. Not only the book is helpful but also the Author is so helpful. I have contacted him by email regarding my question on the book, actually without expecting any response. But he has replied my email many times with great answer, suggestion, and following-up. People like him should really be acknowledged.

This book allows you to quickly learn how to setup serial port communication in your code so you can read and write bytes on serial port.

However they don't really give you many examples of how to write real world code to interface with serial devices. So if you think this book will help you write the code to communicate with some kind of serial device think again. The book will only give you the foundation for reading and writing serial ports in different programming languages and operating systems. The section on Windows C++ (protected mode) is very limited and most of the information can be found on MSDN.

The most complicated part of serial programming is the two way communication. They only show examples of single threaded code and a Sputnik's sized blip on OVERLAPPED I\O. Overlapped IO is very important if you need real time two way serial communication where you are reading and writing at the same time your program is performing other tasks. Because you dont always want to block your thread on reads and writes.

This book is one of the only book out there that targets serial communication. Overall the book is good and covers a history of serial devices, how to establish communication, and reading\writing bytes on the port. The examples in the book are clear however they lack what I would consider real world application.

This book tries to cover a large amount of material, and as a result, it only touches on the basics. I was interested in the section on programming serial ports in visual C++ for a windows XP machine. The coverage amounted to 20 pages with no in depth discussion of problems that that might be encountered, and how to troubleshoot them. The book did allow me to do a little bit with serial port communications, but only enough to get me into trouble. I'm still looking for a book that explains how to do this in visual C++ on a Win XP machine. I give it two stars because it is well written, and it does allow the reader to begin to use the serial port.

This book offers a very good and exceptionally practical explanation of how to program PC serial ports. In my job, I have to write interfaces to serial devices, and this book gives me the information I need to make quick work of things no matter what language I am using. Within 2 hours I was able to have a running Visual C++ application reading and writing to a device connected to my COM port. The book goes beyond general concepts,with lots of sample code and detailed line-by-line instruction.
I recommend this book to the student who desires a complete understanding of serial ports, and to the engineer who needs to come rapidly up to speed on how to make use of serial ports in his applications.
The section regarding the hardware aspects of serial ports provides a good basis of understanding the terminology, and is particularly helpful in understanding all the different protocols that are out there. Also helpful was the explanation of the Windows API functions necessary for serial communication. The author also anticipates problems that may arise along the way and suggests ways to avoid them.

The reason I purchased this book was a review I read in the IEEE Software journal, that recommended highly this book as a technical book that even steps the reader through an example implementation GRID service. Well, the first part of the book is the usual "GRID vision" hand-waving how GRID-based computing will realize the concept of "utility computing" where you turn on the switch, and presto, all the computing cycles you were starving for, are there, running your scientific or engineering code or whatever...
The book unfortunately is not well-written. Far too often, sentences are not syntactically correct, obfuscating the authors' intents. The book is definitely not suited as a technical reference, because by reading it there is no way you can implement a GRID "HelloWorld" service. And even when you read Sotomayor's tutorial on GRID services, that actually does guide you through your first GRID service using Java WS-Core, all you've done is figure out how to implement Web-services running on GT4. No mention of distributed computing, how to take advantage of parallelism inherent in a computation etc. etc.
So, overall, the book serves mostly as a layman's (or manager's) introduction into what GRID-computing wishes it will eventually be.

Grid computing is extensively described here as a means of providing high powered utility computing on demand. Currently, its potential is mostly unrealised. Many companies and universities have different grid implementations, as described by the authors. The universities' main motivation is to dragoon enough computing resources for hard research problems. While in the commercial sector, computer companies like IBM want to sell on demand access as a means of entering a hopefully vast new market.

The grid approaches in the book collectively can be contrasted with p2p computing. Grid systems tend to use more diverse and powerful hardware and relatively small number of users. Think of this as the high end, while p2p is low end (e.g. the SETI desktop application). The book describes the vast amount of effort that has gone into devising grid standards and the various toolkits, most notably Globus.

A potential problem which may occur to the reader of this book is the sheer complexity of the grid approach. Its proponents argue that this is necessary complexity. But perhaps a p2p methodology might be easier to understand and use.

An analogy is with the X.400 and X.500 email and directory standards. While these are used by some companies, many have not done so. Due to the complexity and slowness. Too heavyweight. The danger for grid computing is meeting a similar fate. It may end up occupying a small high value niche, but no more.

The writing style is too wordy and reiterates too often. It would be better to have less repetition and more useful summaries at the end of each chapter.

There are several references to IBM's leadership in grid computing, which seems unnecessary. Furthermore, there is no mention of the Plan 9 operating system which solved many of the problems that Globus is attempting to solve.

It starts to get technical around chapter 5. So if you already know you want something resembling a grid, start there.

The authors have written a fine book on the potential, execution and practicality of Grid or Utility Computing. It is large ( 400 pages ) and well written book, technically accurate and blends well with other industry strategies such as on demand and Autonomic Computing. The chapters on open standards are particularly strong, well thought out and presented. The book is designed well and book production, diagrams, layout is nothing short of highest quality - in short, excellent.

The prospect of true utility computing is within reach and technically feasible. The authors bring together best deployment practices, practical guidance on integrating existing resources, and applicable case studies. This book goes a long way to assisting that projection and should become a classic standard in the field.

Full kudos~! - and a doff of the hat to both authors.