Yes, you need to wait with that specific client, however, you can use a different 'light' client like electrum or even an online blockchain.info one that doesn't download the blockchain and you can have funds sent and received instantly.
But do I need to wait until exactly the moment it has downloaded the entire blockchain?
With Electrum and MultiBit, you shouldn't need to store the entire blockchain.
With Bitcoin Core, you only need to synchronize to the block where your received bitcoins were confirmed. After that you should be able to send a transaction without needing to wait for the rest fo the blockchain to synchronize. Of course, you won't see the transaction that you send confirm until you synchronize that block.
And when that happens if my internet speed can't keep up with the growing blockchain size how would I be able to use or withdraw my bitcoins?
If you truly can't keep up, then you will fall farther and farther behind, causing you to wait longer and longer before you can see any bitcoins received in your wallet and before you can send them.
With the current protocol, the blockchain is guaranteed to grow by less than 6 megabytes per hour. That's an internet bandwidth of less than 14 kilobits per second. 14.4 kbit/s dial-up modems have existed since the early 1990's. Are you saying your internet speed is slower than an 1990's dial-up modem? If that's true, then you should definitely NOT use Bitcoin Core and should instead consider one of the lightweight wallets such as Electrum or MultiBit.
It seems like when I tried earlier last year I could never get to complete the sync.
How long did you wait? How many blocks did it receive when you gave up? How many blocks was it receiving per hour?
