Hey Ken.
The issue was that I was selecting the whole transaction as the input and not choosing the appropriate indexes, so I was creating a transaction with no signatures for those inputs I didn't have the private keys for.
Damn, you think you get Bitcoin, until you need to create a TX manually.
Haha I get the complexcities but choosing a scripting language is always better than c# or java. Try doing the same in py scripts, much easier.