Then apart from your privacy being lost, nothing else. They gain access to anything that is inside that wallet (address(es), public key(s), bookmarks, labels) and your transaction history from blockchain.
With those information, someone could
1. Analyze your habit (e.g. when do you spend your Bitcoin)
2. Link your IRL identity with online account (e.g. linking you with your bitcointalk.org account if you ever posted your Bitcoin address)
3. Know how do you spend your Bitcoin (e.g. deposit to exchange and buy gift card)
I think the worst case is someone could threat you and demand Bitcoin with reason such as
1. Will report to police because Bitcoin isn't legal.
2. Share your personal stuff (most common are masturbation video and personal chat history), even though they actually don't have those stuff.