Home » Technology » Scientific Computing » A Parallel Universe

A Parallel Universe

© Adam Hughes

The next article in this series was advertised as addressing the use of parallel computers in the field of computational chemistry. However, upon further review, it seems fairly obvious that a short background piece on parallel processing is probably a good idea before we can fully appreciate it's applications. Hence, the title of this piece.

Most people who can access this web site are at least nominally familiar with the parts of a computer and how it operates: the user employs some input device (keyboard, mouse, etc.) to give instructions to the computer. These user actions generally launch some more instructions (software), and the computer responds by working systematically through the commands and then usually let's us know that it's finished by accessing an output device (monitor, printer, etc.). Within this process, the computer had to load up a program into memory and most likely also accessed some form of storage system (floppydrive, CDROM drive, hard drive, etc.). This is all basic stuff that is taught in most introductory computer science classes, and it presents a fairly accurate picture for the most part, at least when we're talking about computers with one processor. To begin to understand parallel compu ting, however, the mindset has to be adjusted a bit.

The casual user of large parallel machines (although this may seem like an oxymoron, many researchers treat even the most sophisticated computer platforms as black boxes that solve their problems without taking the time to gain an understanding of the underlying principles) will not notice much difference in his computing experience when compared to the scenario above. Basically, he can input some data, wait for the machine to do its thing, and then retrieve the results. However, what happens in the middle, when the computer takes over, is quite different.

While there are several different models, or architectures, for the implementation of parallel processing, there are some common overriding characteristics. A parallel machine basically consists of multiple processing units, linked together so that they may communicate information with each other. Usually, the number of processors available on a machine is a power of two - 2, 4, 8, 16, 32, etc. - a situation which will be discussed at a later point when we look at different parallel platforms. To run a job, a user will almost always specify the number of processors he wishes to use, but this is likely the only difference he will notice in going from serial (one processor) to parallel mode.

Once the user hits , the problem is divided up in some fashion over the processors. So, if a problem requires N calculations, each of the P processors must do N/P computations. As you might imagine, this theoretically means the problem will be solved in a time period that is P times shorter than what it takes one processor to complete the same job. That would be the case in a perfect world, but even our high-tech land is not immune to inherent limitations: the imperfection in this case is the communication that must be done between processors.

The copyright of the article A Parallel Universe in Scientific Computing is owned by Adam Hughes. Permission to republish A Parallel Universe in print or online must be granted by the author in writing.