Your formula assumes the odds are absolute which is simply not correct, whether it be on PrimeDice or in a physical casino.
To expand, statistical odds represented as a percentage must add up to 100%, while at the same time, will never reach the true round number of 100% --- Being 99.99999... and so on is as close as one could get. Unless of course you're flipping a double headed coin
Lets use your example to demonstrate...
On 9900x you've calculated/assumed the actual odds of not rolling a win rolling 9900 rolls is (99.99%)^9900 = 37.16%
The flaw here is if your odds of 'not' winning is 37.16% then mathematically speaking your odds of winning must be 62.84%.
Betting 0.00001000 per roll would result in a win of 0.09899000. Rolling 9900 times, a player would wager 0.09900000 - he would win 63 out of every 100 times he played this series. (62.84% Rounded).
So every 100 games the player would win 63 and lose 37...? Thats guaranteed profit.
You can use discrete probability to calculate odds for only a single roll... To calculate the odds of winning or losing in a series, continuous probability must be used and a number of statistical paradoxes must be considered.
Strato
It looks to me that you are a bit confused about conditional probability. You are right that the probability won't change after getting 5000 loss, and indeed that is because each bet is independent to each other, and that is exactly the reason why ^ is used in calculating the unconditional probability.
Also, "you have 62.84% chance to make it" doesn't mean you will win 62.84% of the time for sure, especially when the sample size is small.
Take a simple event as example, you have 16.6667% chance to roll a '6' with a fair dice. Does that mean you will be guaranteed to get it once every 6 rolls? The answer is no, but the probability tells you how likely an event is going to occur.
Yeah, math there can get you some basic understanding where you sit with it. But it can be wayyyy off. Every roll is independent. On each new roll you have exact same chance to get win or lose as you had on previous one. Doesn't matter if you had 100 wins or loses in a row on 50% , your next roll still has the same 50% chance.