In other words, is a block seen as invalid, if it contains a transaction that spends a coinbase output?
Exactly.
The reason behind this rule is to limit the damage in the event that a block, or potentially a whole bunch of blocks, get orphaned by a better chain. The problem is that while normal transactions can always make their way back to the chain - they just become unconfirmed and can always be reconfirmed - when a block is orphaned the coinbase transaction simply no longer exists and thus transaction that spent that output are invalid. There could be a heck of a lot of those transactions, and thus a lot of peoples' payments disrupted.
The 100 block rule makes that really unlikely to happen because 100 block getting orphaned in a row is really unlikely to happen. Satoshi probably chose 100 blocks, rather than say 6 or something, to be conservative; making miners wait 16 hours before they can spend their reward is a pretty minor disadvantage compared to the huge amount of harm invalidating transactions can do. Personally I would have picked a week myself, but unlike Satoshi I have to sleep.
The value of this rule was really demonstrated a few years ago with the overflow bug. Basically a flaw was discovered where a special transaction could create an enormous number of coins out of thin air. The flaw was (very) quickly patched and miners upgraded, but it still took a good 50 blocks or so before the patched chain finally overtook the un-patched one. Because of the 100 block rule all the transactions on the un-patched chain eventually were confirmed on the patched one.