Designing Parallel Programs:

• Identify the program's Hotspots:
• Know where most of the real work is being done. The majority of scientific and technical programs usually accomplish most of their work in a few places.
• Profilers and performance analysis tools can help here
• Focus on parallelizing the hotspots and ignore those sections of the program that account for little CPU usage.
• Identify bottlenecks in the program:
  • Are there areas that are disproportionately slow, or cause parallelizable work to halt or be deferred? For example, I/O is usually something that slows a program down.
  • May be possible to restructure the program or use a different algorithm to reduce or eliminate unnecessary slow areas
• Identify inhibitors to parallelism. One common class of inhibitor is data dependence, as demonstrated by the Fibonacci sequence above.