I did not see this post while I was editing my last post :-D
Anyway, I think the problem here is either:
OP has entered wrong seed
or
His seed are not Electrum seed.
I am not much clear about this too:
So I have lost the passphrase to my wallet, but I have the seed.
What is he meaning by passphrase and seed. Passphrase and seed are same thing in my knowledge or it's me when I call any of them I actually mean the 12/24 word seeds. It could be extended seed or regular seed.
Edit:
Help me to understand that I am close to imagine what you have done when you created the wallet.
When I was creating the seed I made it an extended one and I put a small combination of 3 numbers at the end of it.
When you created the wallet you added this small combination of 3 numbers at the end and you had it in the extended seed box?
Do you have the seed plus the seed with the small combination of 3 numbers? I hope you have.
Without the small combination of the 3 numbers is your regular seed.
With the small combination of the 3 numbers are your extended seed.
Can you try it now?
I remember when I used Electrum first time, I had the same mistake but somehow I was able to figure it out when I was restoring my wallet.