What you wrote may happen. And it happens often you are right. But nowadays, lot of big e-mail companies, gmail, yahoo etc .. has very good security measures, if you lost your e-mail password or your account is hacked, there is very high chance you can recover your email, using your telephone number, secondary email, security q/a.
I use gmail and it has great function that it prevents you from loggin in if your IP is different than mine. So I can give you my password and you wont be able to login (Sure thing there is way how to login ..but it was meant that security of e-amils is very high)