https://www.amazon.com/dp/B000065DKJ/ref=emc_b_5_t
I have 2 of my rigs set up similar to what you are describing, and I have the rigs stacked below the window and one of these style window fans, but both of fans either intake air or outake air. I leave mine on outake. The one I have is quiet and the room stays cool.
You can also put a box fan, on low, in front of the rigs to help push the hot air away from gpus and towards the window. You can also leave the door cracked to draw across cooler air from the rest of house, unless you are in a dorm and it would leave your rig unsecure.