So each node requires a gigabyte for their own storage table, plus one gigabyte for each peer they connect to?
One gigabyte (assuming that security level, it was a random number) per peer that they are using a storage proof to get priority access to. So they don't necessarily need to have one per outbound peer. The idea would be the your peers would prioritize inbound connections that were providing a storage proof, everyone else's connectivity would depend on other criteria.
There is no "their own" storage requirement the whole basis of this idea is to be memorless on the server.
Node could pick their favorite couple peers to use storage proofs on, and if they get bumped off other peers during a dos attack, it's not the end of the world.
Would a node only require a storage proof for new nodes and drop the requirement after they have established a certain amount of good behaviour?
The problem there is that an attacker could just slowly build up proof free connections to the whole network and still saturate things.
Long term I expect our behavior with inbound connections to look something like: When a new connection comes in and we're full, randomly drop a peer (including, possibly, the new one) based on a priority scheme. The priority scheme could reserve a number of slots for each of several different kinds of priority. For example, some slots should be reserved for the longest connected nodes— since having a long uptime connection is a kind of scarce resource and some attackers come in bursts. Other slots should be reserved for peers which have the lowest minimum round trip time (because being close to you on the network can't be faked), come from the same subnet as you, come from many distinct subnets, are often the first to relay you new blocks, have an IP that when hashed with a secret is a low value, etc. You can list out a bunch of different groups to protect... then randomly (weighed by goodness or whatever) kick nodes that don't make the cut.
One problem with that kind of litany of criteria is that you'll still end up with clients that just don't meet any of the protected classes— they won't appear to be highly useful good nodes, because they're not highly useful (esp SPV clients). They may not be near any of their peers on the network, they may be freshly started and so have low uptime. So it would be good to have a way for nodes to opt into priority without creating an easy avenue for attackers.... thus ideas like a proof of storage.