A large factor is that the algorythms used by altcoins were deliberately made TOUGHER as a way to make it more difficult for someone to create an ASIC for those coins.
Look at the hashrate out of an Innosilicon A1 vs an Innosilicon A2 for a good comparison - SAME technology used on both chips, essencially identical board-level design, essencially same SIZE chips (identical chip package but the internal chips might vary a hair on area), yet the A1 hashrate is a TON higher 'cause SHA256 is much easier to compute than Scrypt.