Parallel Architectures

SIMD

Single instruction multiple data

MIMD

Multiple instructions multiple data

• Distributed memory: Nodes have private memory, communication by message passing

• Shared memory: shared memory space

  • UMA: Unified Memory Access
    • same access latency for each processor to a certain address
  • NUMA: Non Unified Memory Access
    • still shared memory, but different nodes could take different amount of times to get the data back