It's more than oral history, to be honest. It's also that the the tribe all the other Banaadiri tribes maintain was the "oldest" and at one point ruling tribe of Xamar are basically an off-shoot of the Ajuuraan.
See this thread.
Then we also know some of these weren't really oral traditions originally as we see that in the days when European researchers were making contact with
these narrators they were frequently stopping to consult manuscripts that are now sadly either lost or in the possession of families that have yet to come forward (
like with this document).
Finally, classifying this as simply "oral stories" is silly. These aren't Fisherman's wives tales. They actually went to many different tribes and individuals all across Koonfur and noticed they were all telling more or less the same story, even when not really in contact with each other much then tried to connect this to contemporary traditions, dynamics (like that Banaadiri tribe) and even some rudimentary archaeological work. The odds of all of that being some grand conspiracy and made-up is quite low.
I'd read Virginia Luling's book on the Geledi to understand what the "Ajuuraan" was more or less really like, though:
xiv, 296 pages, 4 unnumbered pages of plates : 24 cm
archive.org
I honestly haven't given Koonfur the deep dive its deserves and defer to
@Idilinaa on all this in reality but the impression I get is that they were just a dominant tribe in the interior, like the Geledi, who managed to gain a stranglehold over the riverine south's agriculture and therefore essentially make the Banaadir towns, that were completely dependent on that agriculture, beholden to them and even establish and off-shoot of them as a dominant tribe within the greatest of the three main ports. Geledi before the Geledi, basically.