Topic: rpietila Wall Observer - the Quality TA Thread ;) - page 229. (Read 907227 times)
u want to generate the gaussian who's two points that are 1 std dev from the center fall on the 2 predicted points. normalize the curve so Y is 1 at those pts. Plug in the actual BTC log value and then divide by the width of the prediction to favor narrow range predictions. highest number wins.
I realized that the sum of all the guesses would give a fancy consensus prior on the price at a given date too...
The Wisdom of Crowds.
Perhaps it works by averaging the weights of all predictors to near 1?
http://heatherlench.com/wp-content/uploads/2008/07/dawes2.pdf
I realized that the sum of all the guesses would give a fancy consensus prior on the price at a given date too...
The Wisdom of Crowds.
I realized that the sum of all the guesses would give a fancy consensus prior on the price at a given date too...
Why not have an exclusive channel to publish analysts' predictions? After a certain delay-period, all predictions would become public. The market price for timely access to an analyst's predictions would be a good metric of the accuracy of that analyst.
Ok in that case people will have to guess distributions rather than intervals, then just compare the probability density at the price. If people want to do intervals, then they can use uniform distributions.
This sounds like a solid, sensible and fair solution.
Loved the illustrations too!
Edit: May be a bit cumbersome for each forecaster, but allows each forecaster to put as much work into it as he wants and still remains fair.
Quote
Actually, it is not clear that the first prediction is better. It depends. Also, to the point 1), consider that the probability density function can be asymmetric within the range, i.e. the mean can be different from 0.5*(max+min).
Yes, it depends on how you define "better". I'd consider the first prediction more useful and therefore better, even though the actual falls outside the range. That's what range is not a good method.
A more precise method would be to assume a normal distribution on a log scale and have the prediction include both mean and standard deviation so one could compute the probability associated with a given price. The prediction with the high probability at the final actual value wins.
Predicting a range is clearly useful, but having the value fall into that range or hit the mean value of the range does not always mean the prediction is better.
Example:
Prediction 1: 90-100; Prediction 2: 50-150. Actual: 101
The first is clearly the better prediction, although the actual is outside the range and further from the mean than the second.
What you really want to do is 1) reward a predicted mean value (i.e. max+min/2) that is close to the actual; 2) reward a smaller range preferentially to a larger range.
Example:
Prediction 1: 90-100; Prediction 2: 50-150. Actual: 101
The first is clearly the better prediction, although the actual is outside the range and further from the mean than the second.
What you really want to do is 1) reward a predicted mean value (i.e. max+min/2) that is close to the actual; 2) reward a smaller range preferentially to a larger range.
Actually, it is not clear that the first prediction is better. It depends. Also, to the point 1), consider that the probability density function can be asymmetric within the range, i.e. the mean can be different from 0.5*(max+min).
No. D is the deviation from the interval edge and is added to the weighted interval width. I think you misread it.
Sorry, I misread your AND as an OR in the conditional and I didn't look too closely at the inequalities.In that case, this proposal would give the same score to two forecasts with the same I, regardless of where within the interval the actual outcome falls. For example: a prediction of 90-110 would be ranked equal to a prediction of 99-119 with an outcome of 100. I don't find that intuitive either.
Ok in that case people will have to guess distributions rather than intervals, then just compare the probability density at the price. If people want to do intervals, then they can use uniform distributions.
Predicting a range is clearly useful, but having the value fall into that range or hit the mean value of the range does not always mean the prediction is better.
Example:
Prediction 1: 90-100; Prediction 2: 50-150. Actual: 101
The first is clearly the better prediction, although the actual is outside the range and further from the mean than the second.
What you really want to do is 1) reward a predicted mean value (i.e. max+min/2) that is close to the actual; 2) reward a smaller range preferentially to a larger range.
Example:
Prediction 1: 90-100; Prediction 2: 50-150. Actual: 101
The first is clearly the better prediction, although the actual is outside the range and further from the mean than the second.
What you really want to do is 1) reward a predicted mean value (i.e. max+min/2) that is close to the actual; 2) reward a smaller range preferentially to a larger range.
Actually, it is not clear that the first prediction is better. It depends. Also, to the point 1), consider that the probability density function can be asymmetric within the range, i.e. the mean can be different from 0.5*(max+min).
No. D is the deviation from the interval edge and is added to the weighted interval width. I think you misread it.
Sorry, I misread your AND as an OR in the conditional and I didn't look too closely at the inequalities.In that case, this proposal would give the same score to two forecasts with the same I, regardless of where within the interval the actual outcome falls. For example: a prediction of 90-110 would be ranked equal to a prediction of 99-119 with an outcome of 100. I don't find that intuitive either.
What you want to do is reward for the prediction being within the interval and penalize for distance from the interval as well as for the width of the interval. For example:
W could even be 1, IDK. Lowest score wins.
Code:
I= upper_limit -lower_limit
W=Weight of Interval width penalty
P= Actual
if( P>lower_limit AND P
}else{
D=min [ abs ( P - upper_limit); abs ( P - lower_limit) ]
}
Score= D+W*I
W=Weight of Interval width penalty
P= Actual
if( P>lower_limit AND P
}else{
D=min [ abs ( P - upper_limit); abs ( P - lower_limit) ]
}
Score= D+W*I
W could even be 1, IDK. Lowest score wins.
Sorry for coming off as very negative today, I really don't mean to smack down everything, but this wouldn't work either. My counterexample: I would choose upper_limit = lower_limit = 0 and win every time (assuming P is not zero)! I would get D=0 and I=0, which would give me Score=0, regardless of W. Furthermore 0 is the lowest possible score since D>=0 and I>=0, so I would win every time!
No. D is the deviation from the interval and is added to the weighted interval width. I think you misread it.
What you want to do is reward for the prediction being within the interval and penalize for distance from the interval as well as for the width of the interval. For example:
W could even be 1, IDK. Lowest score wins.
Code:
I= upper_limit -lower_limit
W=Weight of Interval width penalty
P= Actual
if( P>lower_limit AND P
}else{
D=min [ abs ( P - upper_limit); abs ( P - lower_limit) ]
}
Score= D+W*I
W=Weight of Interval width penalty
P= Actual
if( P>lower_limit AND P
}else{
D=min [ abs ( P - upper_limit); abs ( P - lower_limit) ]
}
Score= D+W*I
W could even be 1, IDK. Lowest score wins.
Sorry for coming off as very negative today, I really don't mean to smack down everything, but this wouldn't work either. My counterexample: I would choose upper_limit = lower_limit = 0 and win every time (assuming P is not zero)! I would get D=0 and I=0, which would give me Score=0, regardless of W. Furthermore 0 is the lowest possible score since D>=0 and I>=0, so I would win every time!
Here is a chart illustrating the Log10 delta of actual bitcoin price from my manually-fitted logistic model trendline. Rpietila has been discussing Log10 deviation from the Log10 trend as an indicator for buy, sell or hold. It is clear from the chart how such limits could be derived from the April 2013 and November 2103 bubbles.
Predicting a range is clearly useful, but having the value fall into that range or hit the mean value of the range does not always mean the prediction is better.
Example:
Prediction 1: 90-100; Prediction 2: 50-150. Actual: 101
The first is clearly the better prediction, although the actual is outside the range and further from the mean than the second.
What you really want to do is 1) reward a predicted mean value (i.e. max+min/2) that is close to the actual; 2) reward a smaller range preferentially to a larger range.
Example:
Prediction 1: 90-100; Prediction 2: 50-150. Actual: 101
The first is clearly the better prediction, although the actual is outside the range and further from the mean than the second.
What you really want to do is 1) reward a predicted mean value (i.e. max+min/2) that is close to the actual; 2) reward a smaller range preferentially to a larger range.
Eureka! It is this simple:
- Every predictor gives two prices in log scale eg. "In 2014-5-16 the price is between 2.7 and 2.85 (roughly 500 and 700)"
- When the actual price is known, you take min [ abs ( actual - upper_limit); abs ( actual - lower_limit) ]
- Whoever has the lowest average error after a reasonable number of predictions (predictions can be renewed as often as you wish regardless of their maturity) is the best!
- Proof omitted
- Every predictor gives two prices in log scale eg. "In 2014-5-16 the price is between 2.7 and 2.85 (roughly 500 and 700)"
- When the actual price is known, you take min [ abs ( actual - upper_limit); abs ( actual - lower_limit) ]
- Whoever has the lowest average error after a reasonable number of predictions (predictions can be renewed as often as you wish regardless of their maturity) is the best!
- Proof omitted
I would be very grateful if you could explain this to a simpleton like myself.
Whoever was the closest to the actual price with the narrowest range was the best. I think he's being a little facetious here, because this is, of course, obvious.
Except that it doesn't actually work.
Proof by counterexample: Imagine a forecast range of 50-100. If the outcome if 95, i.e. within the range, the formula produces a score of 5. However, if the outcome is 105, i.e. outside the range, the formula produces a score of 5. But clearly, the first situation should score better, but with this formula it does not! QED?
Edit: I can think of more examples where it doesn't work too, can I leave those as an exercise to the reader?
Edit 2: For those wondering how to do it properly, I suggest searching the meteorology literature - it's much more comprehensive on this issue than the financial/economic/econometric literature.
Suppose rpietila's formula was amended to yield zero in the case that the prediction is within the range. Then your offered counterexample fails. Did you have others that would prove the amended scoring formula invalid?
Note, to anyone interested, that the reduction of the price to log10 form allows the predictions to be compared on widely differing timescales, in which price values might be 10x larger or smaller. rpietila has been talking about deviation from the log10 trendline in terms of these log10 deltas.
What you want to do is reward for the prediction being within the interval and penalize for distance from the interval as well as for the width of the interval. For example:
Code:
I= upper_limit -lower_limit
W=Weight of Interval width penalty
P= Actual
if( P>lower_limit AND P
}else{
D=min [ abs ( P - upper_limit); abs ( P - lower_limit) ]
}
Score= D+W*I
W=Weight of Interval width penalty
P= Actual
if( P>lower_limit AND P
}else{
D=min [ abs ( P - upper_limit); abs ( P - lower_limit) ]
}
Score= D+W*I
W could even be 1, IDK. Lowest score wins.
Suppose rpietila's formula was amended to yield zero in the case that the prediction is within the range. Then your offered counterexample fails. Did you have others that would prove the amended scoring formula invalid?
Note, to anyone interested, that the reduction of the price to log10 form allows the predictions to be compared on widely differing timescales, in which price values might be 10x larger or smaller. rpietila has been talking about deviation from the log10 trendline in terms of these log10 deltas.
Note, to anyone interested, that the reduction of the price to log10 form allows the predictions to be compared on widely differing timescales, in which price values might be 10x larger or smaller. rpietila has been talking about deviation from the log10 trendline in terms of these log10 deltas.
Fair enough, let's do another one:
Imagine the actual outcome is 95. A prediction of 50-100 would score 5. A prediction of 89-100 would also score 5, although it is obviously much better than the first one. So the amended formula does not correctly rank the predictions either (at least not intuitively).
Let me see if I can think of another one...
Log scale or not doesn't really matter for comparing predictions to each other (by which I mean ranking/ordering them).
Edit: Came to think of another counterexample for the amended formula: if the prediction is 50-100 and the outcome is 101, the amended formula gives score 0. If the outcome is 150, the amended formula also gives score 0, although the first case is obviously better than the second. (You might argue that this is "fair", though, but that is a subjective argument: the amended formula still clearly fails to correctly rank/order the two predictions).
Eureka! It is this simple:
- Every predictor gives two prices in log scale eg. "In 2014-5-16 the price is between 2.7 and 2.85 (roughly 500 and 700)"
- When the actual price is known, you take min [ abs ( actual - upper_limit); abs ( actual - lower_limit) ]
- Whoever has the lowest average error after a reasonable number of predictions (predictions can be renewed as often as you wish regardless of their maturity) is the best!
- Proof omitted
- Every predictor gives two prices in log scale eg. "In 2014-5-16 the price is between 2.7 and 2.85 (roughly 500 and 700)"
- When the actual price is known, you take min [ abs ( actual - upper_limit); abs ( actual - lower_limit) ]
- Whoever has the lowest average error after a reasonable number of predictions (predictions can be renewed as often as you wish regardless of their maturity) is the best!
- Proof omitted
I would be very grateful if you could explain this to a simpleton like myself.
Whoever was the closest to the actual price with the narrowest range was the best. I think he's being a little facetious here, because this is, of course, obvious.
Except that it doesn't actually work.
Proof by counterexample: Imagine a forecast range of 50-100. If the outcome if 95, i.e. within the range, the formula produces a score of 5. However, if the outcome is 105, i.e. outside the range, the formula produces a score of 5. But clearly, the first situation should score better, but with this formula it does not! QED?
Edit: I can think of more examples where it doesn't work too, can I leave those as an exercise to the reader?
Edit 2: For those wondering how to do it properly, I suggest searching the meteorology literature - it's much more comprehensive on this issue than the financial/economic/econometric literature.
Suppose rpietila's formula was amended to yield zero in the case that the prediction is within the range. Then your offered counterexample fails. Did you have others that would prove the amended scoring formula invalid?
Note, to anyone interested, that the reduction of the price to log10 form allows the predictions to be compared on widely differing timescales, in which price values might be 10x larger or smaller. rpietila has been talking about deviation from the log10 trendline in terms of these log10 deltas. Accordingly, I added a column to my own logistic model of bitcoin prices which produces this log10 delta for each day's data that I record.
https://docs.google.com/spreadsheet/ccc?key=0ArD8rjI3DD1WdFIzNDFMeEhVSzhwcEVXZDVzdVpGU2c
Eureka! It is this simple:
- Every predictor gives two prices in log scale eg. "In 2014-5-16 the price is between 2.7 and 2.85 (roughly 500 and 700)"
- When the actual price is known, you take min [ abs ( actual - upper_limit); abs ( actual - lower_limit) ]
- Whoever has the lowest average error after a reasonable number of predictions (predictions can be renewed as often as you wish regardless of their maturity) is the best!
- Proof omitted
- Every predictor gives two prices in log scale eg. "In 2014-5-16 the price is between 2.7 and 2.85 (roughly 500 and 700)"
- When the actual price is known, you take min [ abs ( actual - upper_limit); abs ( actual - lower_limit) ]
- Whoever has the lowest average error after a reasonable number of predictions (predictions can be renewed as often as you wish regardless of their maturity) is the best!
- Proof omitted
I would be very grateful if you could explain this to a simpleton like myself.
Whoever was the closest to the actual price with the narrowest range was the best. I think he's being a little facetious here, because this is, of course, obvious.
Except that it doesn't actually work.
Proof by counterexample: Imagine a forecast range of 50-100. If the outcome if 95, i.e. within the range, the formula produces a score of 5. However, if the outcome is 105, i.e. outside the range, the formula produces a score of 5. But clearly, the first situation should score better, but with this formula it does not! QED?
Edit: I can think of more examples where it doesn't work too, can I leave those as an exercise to the reader?
Edit 2: For those wondering how to do it properly, I suggest searching the meteorology literature - it's much more comprehensive on this issue than the financial/economic/econometric literature.
Eureka! It is this simple:
- Every predictor gives two prices in log scale eg. "In 2014-5-16 the price is between 2.7 and 2.85 (roughly 500 and 700)"
- When the actual price is known, you take min [ abs ( actual - upper_limit); abs ( actual - lower_limit) ]
- Whoever has the lowest average error after a reasonable number of predictions (predictions can be renewed as often as you wish regardless of their maturity) is the best!
- Proof omitted
- Every predictor gives two prices in log scale eg. "In 2014-5-16 the price is between 2.7 and 2.85 (roughly 500 and 700)"
- When the actual price is known, you take min [ abs ( actual - upper_limit); abs ( actual - lower_limit) ]
- Whoever has the lowest average error after a reasonable number of predictions (predictions can be renewed as often as you wish regardless of their maturity) is the best!
- Proof omitted
I would be very grateful if you could explain this to a simpleton like myself.
Whoever was the closest to the actual price with the narrowest range was the best. I think he's being a little facetious here, because this is, of course, obvious.
My attempt at drawing random lines:
and a 1h zoom:
and a 1h zoom:
At the moment, China is just waking up and volume is smoothly climbing on Huobi's 30 minute chart as displayed at Bitcoin Wisdom.
On the daily Bitstamp chart, $550 appears to be the trend line of resistance.
On the daily Bitstamp chart, $550 appears to be the trend line of resistance.
Looks like we hit that ceiling.
Next attempt will probably go through
Jump to: