It really depends on what type of sport you are referring into.
This would be very boring if it's a contact sport like basketball, boxing, volleyball, or soccer. Commentators on those games make the match more enjoyable. They can make it more entertaining that we get hyped to it and feel like we are watching the game live.
That also depends in which environment you're watching the sports into.
For example, when you are watching a game where you placed your bet and you wanted to watch so badly but you're in the office. It would be necessary to turn off the volume of your phone or laptop to watch the game. I've experienced it many times especially during the NBA playoffs. I'd still enjoyed watching without commentators as the volume is totally turned off, though having it in the background is perfect.
I certainly agree with you. Some sports are more enjoyable without commentators while there are also sports that are better without them. Commentators also play a vital role in the game because audiences especially those who are just streaming will be able to understand the game more appropriately even without focusing well on it. They also add energy and excitement. They make the games more lively and catchy so they're needed in huge sports like basketball, football, and volleyball.
Three are some sports that’s I can only watch when I’m not listening to the commentators. For example I get pretty annoyed by golf commentators. I love playing golf but can’t stand watching it. However when I do watch I (almost only because I’ve got a bet on the match) watch it with the volume off as I just find them annoying and they make me want to sleep. Just me though.