You can only transmog the same armour type. Plate to plate, mail to mail etc. You can’t make a chestplate look like a robe.
Green, Blue and Epic are the only item classes that can be used. You can’t transmog greys, whites or Legendaries. You can transmog heirlooms, though the transmog disappears if you mail it to another character.
One handed swords, maces and axes can be transmogged into each other. Two handed swords, maces, axes and polearms are the same. Daggers, fists, wands and staves can only be transmogged into that type. Same with Bows, Crossbows and Guns.
You also need to be able to equip the item, so you can’t have a mage with an axe.
As for transmogging heirlooms, you need to be high enough level to equip the item to transmog it. So you can’t have a level 89 or below warrior in the BoA transmog sets from the Garrison War Mill.