The HMAC is only used when going from mnemonic -> seed.
A user may decide to protect their mnemonic by passphrase. If a passphrase is not present, an empty string "" is used instead.
To create a binary seed from the mnemonic, we use PBKDF2 function with a mnemonic sentence (in UTF-8 NFKD) used as a password and string "mnemonic" + passphrase (again in UTF-8 NFKD) used as a salt. Iteration count is set to 2048 and HMAC-SHA512 is used as a pseudo-random function. Desired length of the derived key is 512 bits (= 64 bytes).
The spec however allows the user to add a user defined passphrase. So the key stretching is to protect the user defined passphrase not the raw mnemonic. If there is no user defined passphrase then HMAC is still used for consistency but it has no effect on security.
Still I don't see much utility in this as 2,048 is an insufficient number of iterations to provide more than a cryptographic speed bump. A single GPU cracker can perform 80M+ HMAC-SHA512 operations per second reducing that by a factor of 2,048 still means 40,000 passwords per second and that is just a single GPU.
To put it into context iTunes uses 10,000 rounds of PBKDF2 to protect an mp3 account. The Lastpass password manager defaults to 100,000 rounds of PBKDF2 and it can be increased by the user. Key stretching should be as long as possible (but not so long as to disrupt the user). Making a user wait 30 seconds every time they login isn't viable but creating or restoring a seed is infrequent. The iterations should be at least a hundred thousand if not hundreds of thousands. It could even be variable by using a few bits from the checksum.