First of all, what are the advantages of using a hardware wallet, and is it really worth it?
I know security is one of them, but other than that, is there anything else?
Yes, they are worth.
Since you are a newbie in this forum and you are asking this question, you probably aren't a person who has a deep knowledge in cyber security.
Hardware wallets are very convenient and easy to use. They are very good options for newbies, as anyone can just read 30min about them and do everything that you need to be safe.
If you try to use a paper wallet or an offline computer you will need to study a lot more. Setting your own offline environment can be very tricky for newbies and advanced users, and a small mistake can be fatal and you may lose all your funds.
I strongly recommend that all newcomers buy a hardware wallet if they want to invest more than $ 1000.00 in cryptocurrency. You will not be hacked and your funds will be safe.
There is one other advantage: they are very convenient and fast to use. you can carry them around if you need. You can lose the device and your funds won't be at risky, for example. You can transfer funds easily and smoothly.
And finally, Is the ColdCard a good choice? I'm looking to store Bitcoin only.
I don't think it is the best option.
There 2 very good hardware wallets in the market, and they are Trezor and Ledger. They are both very tested, there were never big problems with them (as far as I know), and you won't lose money using any of them.
They are pretty cheap, and are worth the investment.