Subgroup families controlling $p$-local finite groups by C. Broto, N. Castellana, J. Grodal, R. Levi, B. Oliver A $p$-local finite group consists of a finite $p$-group $S$, together with a pair of categories which encode ``conjugacy'' relations among subgroups of $S$, and which are modelled on the fusion in a Sylow $p$-subgroup of a finite group. It contains enough information to define a classifying space which has many of the same properties as $p$-completed classifying spaces of finite groups. In this paper, we examine which subgroups control this structure. More precisely, we prove that the question of whether an abstract fusion system $F$ over a finite $p$-group $S$ is saturated can be determined by just looking at smaller classes of subgroups of $S$. We also prove that the homotopy type of the classifying space of a given $p$-local finite group is independent of the family of subgroups used to define it, in the sense that it remains unchanged when that family ranges from the set of $F$-centric $F$-radical subgroups (at a minimum) to the set of $F$-quasicentric subgroups (at a maximum). Finally, we look at constrained fusion systems, analogous to $p$-constrained finite groups, and prove that they in fact all arise from groups.