Email is an important attack vector, but images are mainly blocked when they are loaded from external URLs (not embedded in emails) so that users can choose to avoid automated tracking of whether/when they open messages. Each email can be created with a unique image URL. When that "image" is fetched, the server sending it can then be pretty confident someone opened the email.
Another reason is to reduce the amount of data downloaded for messages that may well be spam anyway. (But whitelisted senders' mails are often treated differently, with all images fetched & displayed.)
The shocking jpeg buffer overflow vulnerability dates mainly to 2004 and has been addressed by updated software but that's not to say any software can be fully trusted.
It does make good sense to block unnecessary online content and to use different computers (virtual computers, at least) for financial tasks!