In your wallet you have most likely multiple addresses, the standard way of BitcoinQT is just to send bitcoin from your wallet, not from a specific address.
So if your wallet contains 3 addresses with 1 BTC each, there is currently no (simple) way for the average joe to predict the address the btc are sent from. (I know that it IS possible, just not for the average every day consumer)
And 90% of the users will expect that it works with any address in their wallet. So unless you have all addresses of your wallet registered with your site, you will get in trouble tracking the correct transactions.
So to make this work with just 1 address, most users would have to create a seperat wallet (maybe somewhere online?), then send BTC to that wallet (which contains only 1 address) and from that special single-address-wallet they then initiate the "real" transaction, that will be tracked by your servers.
As I said, I do like the idea(even if not flawless apart from the address-issue), maybe you can find some ways around this, but I will stick with Reverb on this one: Most users want it easy and hassle-free, the word "security" is just in their minds, they will put more trust in an image that says "Verified Certified Trusted Service by Scamtrustifier XXX" than in some complex payment process they cannot fully understand.
you have some very good observations, which could be fixed by simple wallet where you will be operating on one address. The problem is more people would have to use this method so having such wallet would have a point.
Also I could request to copy transaction tx or sign message with address which was used for transaction but still its not simple for average user.
If I won't be able to find solution I would have to set up second server which will be responsible for transactions only and wont be visible from game server but this is more expensive.