If you're referring to the famous 8-pointed star of Chaos (actually eight outwards-radiating arrows), it's not real, it's an invention of Michael Moorcock:Symbol of Chaos
. It was, of course, appropriated by Games Workshop for their Warhammer games, and has long since spun out into other pop culture.
For the opposing force of Law, Moorcock chose to use a single upwards-pointing arrow rune, identical to that of the historical rune, Teiwaz.
Always thought this was kind of a dumb opposition symbol. To me, "law" ought to be a SQUARE. Sort of the opposite of the 8-pointed arrow thing.
As far as runes, more interesting designs probably to be found among the Zodiac symbols and various astrological things. Norse/germanic runes are fine but were designed to be cut into stone/wood, so you don't get very interesting shapes really, just a few lines/angles. (Lot of speculation that the Norse runes were borrowed/stolen from Latin or Phoenician writing, probably some truth to that.) You could also look at funkier Asian writing like Mongolian (it's crazy) or some of the Native American alphabets like that of Cherokee, the Cree, etc.
Oh and to answer the original question, I sort of did that once. For "Chaotic" cultures (orcs and whatnot) I was planning on using Mongolian script, Korean Hangul letters for the dwarves, and Cree script for elves.
Hamun Ry (Wiz 4)
Str 10 Agi 15 (+1) Sta 11 Per 11 Int 17 (+2) Luc 10 (Unholy House). Align: C. AC: 14. HP: 13. Melee +1, Ranged +2. Crit: d8, I. Save: Ref +5, Fort +3, Will +4.
Spells: 1: Choking Cloud, Comprehend Languages, Detect Magic (odd crystal growths), Magic Missile (mirror images), Runic Alphabet (Mortal) (ravenously hungry), Ventriloquism (rain of frogs)
2: Detect Invisible, Levitate (extremely difficult, d14), Mirror Image (20% chance to raise/lower luck by 1d3 points).
Equip: Ring of the Sand Djinn: +2 AC/saves, Invisibility for 1min/spellburn point, or unmake for great, unknown effect. Padded Armor, Longsword, Longbow, quiver w/20 steel-tipped arrows, 10 silver-tipped arrows, backpack, spellbook, quill and ink, sturdy parchment (10 sheets), 5 days rations, high leather boots, belt w/ belt pouch, gray robe, dark gray hooded cloak. Also carries 3 small mechanical toys: wind-up mouse, wooden puzzle cube, small jewelry box that plays a little tune.