Domain decomposition

An Arbor simulation requires a Recipes, a (hardware) context, and a domain decomposition. The Recipe contains the neuroscientific model, the hardware context describes the computational resources you are going to execute the simulation on, and the domain decomposition describes how Arbor will use the hardware. Since the context and domain decomposition may seem closely related at first, it might be instructive to see how recipes are used by Arbor:

Recipe
Cell descriptions
gid 1
type lif_cell
description
...
...
gid 37
type cable_cell
description
  • Morphology
  • Ion channels
  • Connection sites
  • Labels
Cell kinds
gid 1 lif_cell
...
gid 37 cable_cell
...
Simulator: kind of gid=1?
Recipe: lif_cell.
...
Simulator: kind of gid=37?
Recipe: cable_cell.
Group implementations for lif_cell, cable_cell
Domain decomposition: group-0, group-1
Simulator requests cell description of gid=37
Recipe: morphology, ....
Simulator:
group-0 make cable_cell
group-1 make lif_cell
Simulator:
group-0 simulate for dt
group-1 simulate for dt
Simulation
Recipe
Context
12 threads
1 GPU
Domain decomposition
group-0
kind: cable
hardware: 1 GPU
gids: 37, ...
group-1
kind: lif
hardware: 12 threads
gids: 1, ...
...
An illustration of the cell-specific portion of the recipe, and how it is used during the lifetime of the simulation: the simulation object will, depending on its configuration, query the recipe for the neuroscientific components it describes. This demonstration also show why the recipe separates cell descriptions from cell types. The latter is, as you might expect, shorthand, and is used in the allocation of the cell to a particular cell group. A cell group implementation is a handler for a certain kind of cell, and Arbor comes with these for all it's included cell kinds. However, users can develop their own specialized cell group implementations. More on that in the internal developer documentation.

A domain decomposition describes the distribution of the model over the available computational resources. The description partitions the cells in the model as follows:

  • group the cells into cell groups of the same kind of cell;

  • assign each cell group to either a CPU core or GPU on a specific MPI rank.

The number of cells in each cell group depends on different factors, including the type of the cell, and whether the cell group will run on a CPU core or the GPU. The domain decomposition is solely responsible for describing the distribution of cells across cell groups and domains.

The domain decomposition can be built manually by the modeler, or an automatic load balancer can be used.

We define some terms as used in the context of connectivity

cell_group

List of same-kinded cells that share some information. Must not be split across domains.

domain

Produced by a load_balancer, a list of all cell_groups located on the same hardware. A communicator deals with the full set of cells of one domain.

domain_decomposition

List of domains; distributed across MPI processes.

Load balancers

A load balancer generates the domain decomposition using the model recipe and a description of the available computational resources on which the model will run described by an execution context. Currently Arbor provides one automatic load balancer and a method for manually listing out the cells per MPI task. More approaches might be added over time.

API