Price drives difficulty - page 2. | Bitcointalksearch.org

organofcorti

donator

Activity: 2058

Merit: 1007

Poor impulse control.

Quote from: molecular on November 13, 2012, 02:30:40 AM

Quote from: organofcorti on November 13, 2012, 02:10:01 AM

Quote from: molecular on November 13, 2012, 01:57:54 AM

I'm also currently preparing (actually waiting for bitcoin-abe) a "hashrate by day" dataset. Maybe it'd be good to use something higher resolution like that. I don't know what the implications ar of using hashrate estimation vs. difficulty.

This what you're after? Wink

Edit: It's actually every 144 blocks, near enough to daily. It won't be exactly the same time period each "day" though.

ah, nice. Thanks for the pointer.

well, I'll do it myself anyhow (daily). Wanted an up-to-date bitcoin-abe dataset anyhow for other purposes.

If you can get a proper weekly / daily dataset, please post. I'd like to use a time period that doesn't stretch.

molecular

donator

Activity: 2772

Merit: 1019

Quote from: organofcorti on November 13, 2012, 02:10:01 AM

Quote from: molecular on November 13, 2012, 01:57:54 AM

I'm also currently preparing (actually waiting for bitcoin-abe) a "hashrate by day" dataset. Maybe it'd be good to use something higher resolution like that. I don't know what the implications ar of using hashrate estimation vs. difficulty.

This what you're after? Wink

Edit: It's actually every 144 blocks, near enough to daily. It won't be exactly the same time period each "day" though.

ah, nice. Thanks for the pointer.

well, I'll do it myself anyhow (daily). Wanted an up-to-date bitcoin-abe dataset anyhow for other purposes.

organofcorti

donator

Activity: 2058

Merit: 1007

Poor impulse control.

Quote from: molecular on November 13, 2012, 01:57:54 AM

I'm also currently preparing (actually waiting for bitcoin-abe) a "hashrate by day" dataset. Maybe it'd be good to use something higher resolution like that. I don't know what the implications ar of using hashrate estimation vs. difficulty.

This what you're after? Wink

Edit: It's actually every 144 blocks, near enough to daily. It won't be exactly the same time period each "day" though.

molecular

donator

Activity: 2772

Merit: 1019

Quote from: organofcorti on November 12, 2012, 04:48:19 PM

Quote from: molecular on November 11, 2012, 04:10:35 AM

this is probably available from many other sources, but I just compiled the daily volume weighted average mtgox price from some 3 million trades and thought I'd share:

http://pastebin.com/uYcX08yL

EDIT: just discovered organofcortis dataset. Ah, well. Great work, btw.

Actually, your post reminded me that I'd been using a truncated dataset, so it prompted me to go back and use the full dataset of MTGOX prices. I'll post a link to that soon. Also, I spent my day off yesterday learning about ARIMA models, and last night analysing the data and making models.

I found an incredibly good model, much better than in my first post. The correlation coefficient of log(Difficulty) and log(model) is 0.99 . I'm amazed by how good a match this is since I expected the change from CPU to GPU mining to have a much greater effect.

I'll post the results in the next few days, but if you want to try to figure the model out for yourselves, I'll give you a hint: D has an autocorrelation as well as a cross correlation with log(vwap usdbtc), and I used multiple linear regression.

I'm also currently preparing (actually waiting for bitcoin-abe) a "hashrate by day" dataset. Maybe it'd be good to use something higher resolution like that. I don't know what the implications ar of using hashrate estimation vs. difficulty.

organofcorti

donator

Activity: 2058

Merit: 1007

Poor impulse control.

Quote from: molecular on November 11, 2012, 04:10:35 AM

this is probably available from many other sources, but I just compiled the daily volume weighted average mtgox price from some 3 million trades and thought I'd share:

http://pastebin.com/uYcX08yL

EDIT: just discovered organofcortis dataset. Ah, well. Great work, btw.

Actually, your post reminded me that I'd been using a truncated dataset, so it prompted me to go back and use the full dataset of MTGOX prices. I'll post a link to that soon. Also, I spent my day off yesterday learning about ARIMA models, and last night analysing the data and making models.

I found an incredibly good model, much better than in my first post. The correlation coefficient of log(Difficulty) and log(model) is 0.99 . I'm amazed by how good a match this is since I expected the change from CPU to GPU mining to have a much greater effect.

I'll post the results in the next few days, but if you want to try to figure the model out for yourselves, I'll give you a hint: D has an autocorrelation as well as a cross correlation with log(vwap usdbtc), and I used multiple linear regression.

molecular

donator

Activity: 2772

Merit: 1019

this is probably available from many other sources, but I just compiled the daily volume weighted average mtgox price from some 3 million trades and thought I'd share:

http://pastebin.com/uYcX08yL

EDIT: just discovered organofcortis dataset. Ah, well. Great work, btw.

organofcorti

donator

Activity: 2058

Merit: 1007

Poor impulse control.

Quote from: niko on November 11, 2012, 02:53:00 AM

Difficulty changes not only due to more people investing into mining, but also due to changes in the efficiency and price of the available technology. I am trying to eliminate these technological sources of "noise" from the calculation, enabling (semi)quantitative predictions. Ultimately, and ideally, there would be no need for miners to have to guess future difficulty in their plans: difficulty will follow the BTC price anyway.

Has anyone ever collected and shown historical cost and efficiency of mining technologies ($s/Hash and J/Hash)? I'd hate to duplicate the effort.

So you want a better predictor of difficulty than just usdbtc? That could be interesting. After ASICs make up the majority of the hashrate you'll be able to calculate the info you want. It would be very hard to figure out before then though. To many different mining rigs with different cards, fpgas and efficiencies.

bitboyben has collected much of the preorder ASIC data here: https://bitcointalksearch.org/topic/asic-difficulty-curves-121453

niko

hero member

Activity: 756

Merit: 501

There is more to Bitcoin than bitcoins.

Quote from: organofcorti on November 11, 2012, 01:50:18 AM

Quote from: niko on November 10, 2012, 01:01:05 PM

Quote from: organofcorti on November 10, 2012, 03:40:00 AM

Quote from: niko on November 08, 2012, 08:08:04 PM

Ideally, we should be looking not at the difficulty, but at the total USD value of mining hardware, based on network hashrate and historical price for H/s. It's tricky to unmix the contributions from different technologies (FPGA vs GPU), but could be estimated. Ideally the value of electricity should be added too, but then we'll have to assume some ROI period that miners are comfortable with.
I expect the correlation between this "value" of mining hardware+electricity and the BTC exchange rate to be much stronger than difficulty/exchange rate.

I think you missed two important part of this thread: the fact it's about predicting difficulty and the fact that difficulty can be modelled as occurring due to changes in price.

I think your post is implying that exchange rate might be a function of the total US$cost of mining. In that case difficulty would be a function of the total cost of mining. This may be true (and I'd be interested to see a thread on it) but it's not what i'm considering in the blog post. Also, I could be wrong but you seem to imply that difficulty could affect exchange rate? I've tried to dispel the myth that difficulty affects exchange rate in any significant way so I hope I'm wrong.

Sounds like you misunderstood my post, and I can't blame you as it was not very clear. After looking at all the data in several different ways, I arrived at the conclusion that, so far, the USD/BTC exchange rate has been driving the network hashrate. We all seem to agree on that. Mining appears more lucrative, more people start investing into mining.

The point I was trying to make is that, instead of looking at the network hashrate (or difficulty), we could look into total USD value of mining equipment. The USD price of a GHash/s has been evolving over time, going from CPUs to GPUs to FPGAs. Also, for each of these technologies, there were constant improvements of the efficiency of the mining software. All in all, we should try and correlate the USD/BTC exchange rate with the (network hashrate / price of a GHash/s worth of mining equipment).

OK, I misunderstood one part - you're not implying difficulty drives price. But I'm having trouble still figuring what you're doing. I was trying to show that there's a time lagged correlation between changes in usdbtc and difficulty, implying that either usdbtc rate is affecting difficulty, or something else is affecting them both - first price, then difficulty.

What is purpose of the correlation you suggest? To predict price? Or to predict sales of mining equipment? Or just to show a correlation between them?

As btcpokey wrote, "to what end"?

Difficulty changes not only due to more people investing into mining, but also due to changes in the efficiency and price of the available technology. I am trying to eliminate these technological sources of "noise" from the calculation, enabling (semi)quantitative predictions. Ultimately, and ideally, there would be no need for miners to have to guess future difficulty in their plans: difficulty will follow the BTC price anyway.

Has anyone ever collected and shown historical cost and efficiency of mining technologies ($s/Hash and J/Hash)? I'd hate to duplicate the effort.

organofcorti

donator

Activity: 2058

Merit: 1007

Poor impulse control.

Quote from: niko on November 10, 2012, 01:01:05 PM

Quote from: organofcorti on November 10, 2012, 03:40:00 AM

Quote from: niko on November 08, 2012, 08:08:04 PM

Ideally, we should be looking not at the difficulty, but at the total USD value of mining hardware, based on network hashrate and historical price for H/s. It's tricky to unmix the contributions from different technologies (FPGA vs GPU), but could be estimated. Ideally the value of electricity should be added too, but then we'll have to assume some ROI period that miners are comfortable with.
I expect the correlation between this "value" of mining hardware+electricity and the BTC exchange rate to be much stronger than difficulty/exchange rate.

I think you missed two important part of this thread: the fact it's about predicting difficulty and the fact that difficulty can be modelled as occurring due to changes in price.

I think your post is implying that exchange rate might be a function of the total US$cost of mining. In that case difficulty would be a function of the total cost of mining. This may be true (and I'd be interested to see a thread on it) but it's not what i'm considering in the blog post. Also, I could be wrong but you seem to imply that difficulty could affect exchange rate? I've tried to dispel the myth that difficulty affects exchange rate in any significant way so I hope I'm wrong.

Sounds like you misunderstood my post, and I can't blame you as it was not very clear. After looking at all the data in several different ways, I arrived at the conclusion that, so far, the USD/BTC exchange rate has been driving the network hashrate. We all seem to agree on that. Mining appears more lucrative, more people start investing into mining.

The point I was trying to make is that, instead of looking at the network hashrate (or difficulty), we could look into total USD value of mining equipment. The USD price of a GHash/s has been evolving over time, going from CPUs to GPUs to FPGAs. Also, for each of these technologies, there were constant improvements of the efficiency of the mining software. All in all, we should try and correlate the USD/BTC exchange rate with the (network hashrate / price of a GHash/s worth of mining equipment).

OK, I misunderstood one part - you're not implying difficulty drives price. But I'm having trouble still figuring what you're doing. I was trying to show that there's a time lagged correlation between changes in usdbtc and difficulty, implying that either usdbtc rate is affecting difficulty, or something else is affecting them both - first price, then difficulty.

What is purpose of the correlation you suggest? To predict price? Or to predict sales of mining equipment? Or just to show a correlation between them?

As btcpokey wrote, "to what end"?

organofcorti

donator

Activity: 2058

Merit: 1007

Poor impulse control.

Quote from: Beans on November 10, 2012, 01:45:25 PM

The price of btc does not control difficulty. Profits drive difficulty, and it looks similar to the price because they are similar.

Can you post a link to your work on the subject? How exactly are you estimating profits?

Korbman

legendary

Activity: 1064

Merit: 1001

Quote from: organofcorti on November 08, 2012, 09:42:12 AM

http://organofcorti.blogspot.com/2012/11/101-price-drives-difficulty.html

Fantastic post again Organofcorti. If I had bitcoins to donate, I would

Keep up the great work..I've got your website bookmarked! Cheesy

bcpokey

hero member

Activity: 602

Merit: 500

Quote from: niko on November 10, 2012, 01:01:05 PM

Quote from: organofcorti on November 10, 2012, 03:40:00 AM

Quote from: niko on November 08, 2012, 08:08:04 PM

Ideally, we should be looking not at the difficulty, but at the total USD value of mining hardware, based on network hashrate and historical price for H/s. It's tricky to unmix the contributions from different technologies (FPGA vs GPU), but could be estimated. Ideally the value of electricity should be added too, but then we'll have to assume some ROI period that miners are comfortable with.
I expect the correlation between this "value" of mining hardware+electricity and the BTC exchange rate to be much stronger than difficulty/exchange rate.

I think you missed two important part of this thread: the fact it's about predicting difficulty and the fact that difficulty can be modelled as occurring due to changes in price.

I think your post is implying that exchange rate might be a function of the total US$cost of mining. In that case difficulty would be a function of the total cost of mining. This may be true (and I'd be interested to see a thread on it) but it's not what i'm considering in the blog post. Also, I could be wrong but you seem to imply that difficulty could affect exchange rate? I've tried to dispel the myth that difficulty affects exchange rate in any significant way so I hope I'm wrong.

Sounds like you misunderstood my post, and I can't blame you as it was not very clear. After looking at all the data in several different ways, I arrived at the conclusion that, so far, the USD/BTC exchange rate has been driving the network hashrate. We all seem to agree on that. Mining appears more lucrative, more people start investing into mining.

The point I was trying to make is that, instead of looking at the network hashrate (or difficulty), we could look into total USD value of mining equipment. The USD price of a GHash/s has been evolving over time, going from CPUs to GPUs to FPGAs. Also, for each of these technologies, there were constant improvements of the efficiency of the mining software. All in all, we should try and correlate the USD/BTC exchange rate with the (network hashrate / price of a GHash/s worth of mining equipment).

Well let me ask you this, to what end exactly? Say I work out a strong correlative effect between the value of mining hardware on the market, and the price USBTC. Now I know how much mining hardware value is, what does that do for me?

The reason for most of these models (at least from my perspective, perhaps they do not agree) is that people are figuring the profitability under different scenarios for mining (and whether to buy in, or drop out). Knowing what the total hashrate will be does not help, except that from that I can calculate difficulty, via an additional step. So why not cut out the middle man and just model difficulty directly?

molecular

donator

Activity: 2772

Merit: 1019

Quote from: Beans on November 10, 2012, 01:45:25 PM

The price of btc does not control difficulty. Profits drive difficulty, and it looks similar to the price because they are similar.

meh. In the same manner you could say: Profits don't drive difficulty, the submission of valid block to the network does.

Beans

hero member

Activity: 518

Merit: 500

The price of btc does not control difficulty. Profits drive difficulty, and it looks similar to the price because they are similar.

niko

hero member

Activity: 756

Merit: 501

There is more to Bitcoin than bitcoins.

Quote from: organofcorti on November 10, 2012, 03:40:00 AM

Quote from: niko on November 08, 2012, 08:08:04 PM

Ideally, we should be looking not at the difficulty, but at the total USD value of mining hardware, based on network hashrate and historical price for H/s. It's tricky to unmix the contributions from different technologies (FPGA vs GPU), but could be estimated. Ideally the value of electricity should be added too, but then we'll have to assume some ROI period that miners are comfortable with.
I expect the correlation between this "value" of mining hardware+electricity and the BTC exchange rate to be much stronger than difficulty/exchange rate.

I think you missed two important part of this thread: the fact it's about predicting difficulty and the fact that difficulty can be modelled as occurring due to changes in price.

I think your post is implying that exchange rate might be a function of the total US$cost of mining. In that case difficulty would be a function of the total cost of mining. This may be true (and I'd be interested to see a thread on it) but it's not what i'm considering in the blog post. Also, I could be wrong but you seem to imply that difficulty could affect exchange rate? I've tried to dispel the myth that difficulty affects exchange rate in any significant way so I hope I'm wrong.

Sounds like you misunderstood my post, and I can't blame you as it was not very clear. After looking at all the data in several different ways, I arrived at the conclusion that, so far, the USD/BTC exchange rate has been driving the network hashrate. We all seem to agree on that. Mining appears more lucrative, more people start investing into mining.

The point I was trying to make is that, instead of looking at the network hashrate (or difficulty), we could look into total USD value of mining equipment. The USD price of a GHash/s has been evolving over time, going from CPUs to GPUs to FPGAs. Also, for each of these technologies, there were constant improvements of the efficiency of the mining software. All in all, we should try and correlate the USD/BTC exchange rate with the (network hashrate / price of a GHash/s worth of mining equipment).

bitboyben

sr. member

Activity: 330

Merit: 250

I thought his idea would be great to measure potential difficulty. I guess I was already of topic too.

axilla

full member

Activity: 217

Merit: 100

great work sir, you are a scholar and a gentlemen.

organofcorti

donator

Activity: 2058

Merit: 1007

Poor impulse control.

And another thread here: http://forum.bitcoin.org/?topic=4339.0

molecular

donator

Activity: 2772

Merit: 1019

Quote from: organofcorti on November 10, 2012, 04:22:26 AM

Looks like I was late to the party - I missed that thread. Thanks for posting the link though molecular - lots of interesting points there.

Here's another relevant thread, an old one (maybe someone wants to necropost).

Not exactly about the topic, but goes into some detail about the correlation. It's about deciding wether it's more profitable to buy or mine:

https://bitcointalksearch.org/topic/price-vs-difficulty-charts-indicators-for-buying-or-mining-7427

organofcorti

donator

Activity: 2058

Merit: 1007

Poor impulse control.

Quote from: molecular on November 10, 2012, 04:08:37 AM

before everything is repeated here. Here's a recent example of the issue being discussed. The OP was confused about the issue and asked wether there was a correlation. After some explaining he said

Quote from: Este Nuno on October 18, 2012, 08:35:14 AM

So at this point I can see that price and difficulty are in fact highly correlated, and changes in difficulty are primarily caused by changes in price.

EDIT: As far as I remember that thread also goes into possible factors that might constitute an influence from difficulty on the price, but I think it was argued that these effects were much weaker than the effect of price on difficulty.

Looks like I was late to the party - I missed that thread. Thanks for posting the link though molecular - lots of interesting points there.

Topic: Price drives difficulty - page 2. (Read 6457 times)