This is far from an undisputed "fact" - the story goes that the "secrets" were preserved in the form of a game because it was understood that the general public would unwittingly preserve such amusements, thus evading the persecution and destruction of this knowledge by the dominant powers that were coming to the fore at the time (presumably the early part of the Common Era).
As for swords having nothing to do with mental processes, this is also a very questionable conclusion. As polymath Jacob Bronowski observed in his Ascent of Man, "the hand is the cutting edge of the mind." This valuable insight yields many significant implications which are well worth considering in the context of elemental attributions.
Lastly I'll just point out that the Tarot is not a static object that was perfected in the past, but a living system that is evolving towards greater perfection as we interact and grow through our relationship with it.
OM TAT SAT,
BNE1