All mining is "offline" - "retrieving the puzzle and submit[ting] the solution if found" is exactly what all miners do. You can mine without including transactions, in which case you won't have to verify anything, but you also won't get any transaction fees. Users will offer transactions fees high enough to cover all costs involved in verifying transactions in order to get their transactions confirmed.
Anybody who wants to verify a transaction needs a full copy of the block chain since the oldest output used in that transaction. So at least miners will always have a copy of the chain. They will also want the public to be able to access this data, otherwise the Bitcoins they make while mining could not be spent anywhere.
Other parties like merchants, service providers, law enforcement, academia, NGOs, websites, etc. will also maintain full copies for various reasons.
Others will be able to answer this better, but from what I understand ECDSA verification is expected to be the bottleneck, not network performance.