The same is true in your daily life. You trust complete strangers that you have never met to keep you alive at all times - every architect and builder to make sure the buildings you enter don't collapse on you, every city engineer to make sure the gas pipes don't explode under the street, everyone involved in designing and building the car/bus/train/tram/whatever you ride to work to make sure doesn't derail/malfunction/crash/explode, everyone involved in growing, harvesting, preparing, packaging, delivering, and serving your food to not make you ill, etc.
It's all about risk. Does using a hardware wallet bring with it a level of risk? Of course. Does using a cold wallet or paper wallet bring with it a level of risk? Also yes.
If you can set up a paper wallet completely offline, using human generated entropy by flipping a coin, converting your coin flips to a seed manually, leaving no traces, etc., etc., then sure, such a set up would be more secure than a hardware wallet. However, such a set up is also not an option for >99% of crypto users, and while it brings with it a lower risk of trusting third parties, it brings with it a much higher risk of messing up, doing something wrong, and either losing access to your coins or your coins being stolen.
For the vast majority of crypto users, a hardware wallet is the best balance between security and simplicity.