These universes existed within an expanse often conceptualised as an ocean in which all universes floated. A number of universes exist, but it was not known if that number is infinite or merely incalculably huge.
Some exist 'sideways' or 'parallel' to each other, although such spatial language must be understood as simplification given that 'space' is a phenomenon confined to within individual universes. Likewise, some exist 'before' or 'after' each other, although such temporal language must be understood as simplification given that 'time' is a phenomenon also confined to within universes. (The Book of the War)
Most 'parallel' universes are exactly that: parallel. They run alongside each other but exist independently. They may share many common features but do not 'diverge' or 'branch off' from each other at any specific point. (The Cosmology of the Spiral Politic)
The countless worlds in which the Roman Empire never fell or in which the Nazis won World War II (Warlords of Utopia) were not twigs off the branch of some 'main' universe or paths that have split off at some particular juncture, but are entirely separate creations that share many, but not all, of the features of the familiar universe.
Those universes which shared the most features in common were considered as being located 'closest' to each other and those which share the least features in common may be considered as being 'furthest away' from each other. Although, again, the spatial terminology here is a simplification as 'space' only exists within universes.
A healthier way for a universe to reproduce is for it to develop within it a species capable of 'engineering' a new universe from scratch. (The Cosmology of the Spiral Politic) Although the Great Houses apparently never attained this level of accomplishment, an unpopular fringe theory on the Homeworld held that their own entire reality had been engineered in this way by the 21st century humanity of some other universe. (The Book of the War)
The City of the Saved existed 'in between' the end of the universe and the start of the next one. The inhabitants of the City once believed they had penetrated into that universe and established a colony there, but in fact they were deceived and had only reached a simulated environment inside the hostile timeship known as Antipathy. (Of the City of the Saved...)