What is neural network modularity?

If a neural network is modular, that means it consists of clusters (modules) of neurons, such that the neurons within the cluster are strongly connected to each other, but only weakly connected to the rest of the network.

Making networks more modular is useful to us if the modules represent concepts which we can understand because this helps us understand the whole system better.

Relevant papers about modularity are Neural Networks are Surprisingly Modular, Clusterability in Neural Networks, and Detecting Modularity in Deep Neural Networks.