Taking it back to the Angles and the Saxons, a distinction between thralls and free men was that free men could bear arms, while slaves could not.
Joyce Lee Malcolm has done a wonderful study in her book
To Keep And Bear Arms: The Origins of an Anglo-American Right, try reading it.
Englishmen once had a reputation as "the freest subjects under Heaven", as they had the right "to be guarded and defended from all Violence and Force, by their own Arms, kept in their own hands, and used at their own charge under their Prince".