Bitcoin has a Merkle Tree implementation that allows old block transactions to be purged or something like that. Read the design paper, it's all a little above my head.
Besides, in 10 years our phones will all have a terabyte of storage and our desktops will have god knows how much.
It will stop. The halving is done as a bitwise right shift instead of division, so the number of coins awarded will eventually be 0. Block #6,929,999 will be the last one to produce coins. This should happen around 2140.