Topic: [ANN] bitcoinX charts & dashboard - difficulty vs. price, forum top 10 (Read 9717 times)

This project is being taken over by someone else and I won't be able to provide support going forward.

It was fun but I will take a break from working on Bitcoin websites for a while.

what about a second top list that counts +1s in a thread?

or a t+1  (thread+1) feature so that you can specifically upvote a thread? Would you use it?

good idea. and I finally added it, too. 

I think it would be more responsible to annotate* this like so :

The top 15 hot topics from Bitcointalk.org of the last 24h *

not so odd after all: the lending board is banned (board 65)

bannedBoards = [5, 51, 52, 53, 56, 65, #marketplace
                9, # off-topic
                27, 31, 32, 33, # spanish
                30, # chinese
                16, 35, 36, 60, 61, 62, 63, 64,    # german
                13, 47, 48, 49, 50, 54, # french
                28, # italian
                29, 69, 70, # portugiesisch
                10, 18, 19, 20, 21, 22, 23, 55, 66, 72, # russian
                45, # skandinavisch
                11, # other languages
                39, # newbies - much trash
                ]

thanks for letting me know. this is odd, I will look into it...

top 15 seems to be updating but I noticed this one is not in the top 15, yet it has 4 page written in less than a day. : https://bitcointalksearch.org/topic/the-pirate-speaks-110152

k. it looks much better this morning (5/15 ponzi threads instead of 10/15).

Nah, leave it as it is. something interesting might surface, and this is just a temporary thing the Bitcoin ecosystem has to go through.

should we (temporarily) ban the lending subforum from the top15? I am sick of all that ponzi crap.

interesting idea. one problem is that power efficiency differs widely between gpu/fpga/asic. and it is hard to get good data on this. will think about it some more.

also I am considering shading the price chart according to volume similar to mtgoxlive. would that be of interest to anyone?

Have you considered making a graph that plots KWh/USD for mining? Ie. how much USD does the average miner earn per KWh of electricity used.

The challenge here would be gathering power usage data. It won't be as reliable as the USD/MH/s/24hr figure because of this, but a lot more useful, as the average miner's mining speed will change (and thus affect whether the USD/MH/s/24hr figure is profitable), but it always remains true that if a miner spends more money on electricity than he earns in bitcoins, he will not be profitable (unless the burned electricity is used for some other purpose, eg. heating).

Would it make sense to start a thread that asks miners to estimate the power consumption of their rigs, in megahashes/joule (ie. mining speed in megahashes per second divided by the power usage of their rig measured in watts)? As far as I can tell we would be able to create a USD/KWh graph from this data.

sorry, guys this took so long. I am quite busy.

it was an overflow: the code was limited to 100000 topics on the board. it will now grow automatically.

btw: what's the matter with this topic anyway? https://bitcointalksearch.org/topic/--100000 I have always wondered if there are secret boards. Or was it simply deleted?

hmmm. MtGox USD on a logarithmic scale? 

Yes sure is, I really like that function. You got me hovering over this site for a long time now.

The top 10 is stuck again.

Graph bitcoin network vs market logarithmic: what the right numbers represent? Volume of trade?

Cool, good to know that it's being worked on!

sorry, might take a couple of days   

Cross-posting my instructions how to find the bad tick in Sierra Charts - may help isolate it in this application as well.

https://bitcointalksearch.org/topic/m.1089239

Hey Phelix after the Mt Gox glitch your charts are now completely worthless, will you be able to remove the faulty data? Cheers!

finally switched the backend to the new server. let me know if you notice any glitches.

hehe, I felt a little bit blind, too.


site is up again.

for some unknown reason mod_python got deactivated.

also the backend is already living on the vserver in test mode and looks ok, will move soon.

Good lord, I feel like I'm missing something.

Unbearable

So many fad threads, I'm mostly interested by mining hardware / project dev.

Interesting to note too that the forum is well split into three or four main core SUB

• Bitcoin General Discussion 8500 topics <-- Mess I'm getting tired to watch. (6 thread)
• Mining 6700 <--- have interest (6)
• Project Development 1500 AND "Development & Technical Discussion" 2500 AND Meta 650 (forum)<--- I make duty to follow (7)
• The whole of Economy sub 3000 + 1500 + big 13000 marketplace (mostly an archive)<--- Shit I'm tired to see. (3)
• Other = ALL OTHERS <--- Who knows, nobody left behind. Noob = 0.3x  (5)

the backend went down and i only noticed it today because I had been kidnapped for my stag party in prague - oops  

top15 will take 24h to fully populate.

    ok, like the idea.

what about splitting it into seven threads of any start time and seven that started less than a week ago?

"less than 1 week" is "shorter" timeframe.

longer time frame would definitely be interesting.

at the moment I am busy moving the scraping backend to a new vServer home. I sold my rig so currently it is living on a laptop

Suggestion, to remove hive-mind and noobs kind of threads.

New topic of interest. Thread of less than 1 week with more than X (say 5), established member posting.

The rational is that a new noob threads will get replied by few established members who'll point to the obvious, end of the story.

Current results tend to give to much score to old threads that turned into chat room with subject close to 4 cuils of away from the OP.  http://cuiltheory.wikidot.com/what-is-cuil-theory

Gold down, BTC up come to mind.  I was among the first to call bullshit on this relation and never thought it would last for 100 pages.

Heh, thanks for that.  But wow, really?  I don't think I've read even two of them.

thanks.

turns out there were plenty of threads with scores above 10000. Here are the threads created in 2012 (first half of the year) sorted by the highest 24h score ever (above 14000):

50799  60001 wtf is going on?
43297  63925 Speculators causing immense price swings: go fuck yourselves
37173  62696 94% correction?
33785  63068 mtgox.com is down
29975  59967 FUCK THIS
29445  61103 CRASH!!!!!!!!!
27502  63839 Throwing in the towel
25810  56583 WTF
24507  59977 is everybody ok?
22654  59964 Here we go!
21458  63064 Gox is goxxing?
21143  64275 What if Bitcoinica requires identification for all customers?
20134  87290 Donating to the Zimmerman Defense Fund
19811  57411 Clash of the Titans
18586  62099 Map Makers Admit Mistake in Showing Ice Cap Loss in Greenland
17398  75577 Want .75010101 BTC for free? Look inside.....
16308  56594 $6 today?!
16028  76129 Encourage George Zimmerman (Florida self-defense shooting victim) to accept BTC
15894  59997 Ok we're on our way back up!
15369  57034 The rocket is secretly taking off.  $8 price target within 18 hours
15158  82581 Bitcoinica stolen coin returns
14893  58922 Why bitcoin is currently (Jan 16th) crashing and why it may continue
14798  88903 BFL refused to issue refund because it was paid with bitcoin (via Bitpay)
14754  56620 Bed vs watching my position? Periodically waking up?
14727  56735 $7 today, in my opinion
14444  63566 S3052 and Manipulation Techniques
14293  63641 Holy shit look at that wall
14152  74315 I believe there is a Hero member who is scamming your money.
14084  74266 Diamonds for Bitcoins

The score is calculated from a formula similar to: nPosts^2 * nUsers  / ((15000 + nAverageId)/15000) all within the last 24h. Looking at the threads above there might be room for optimization 

Im addicted to this page also.. .  Really good, if I remember correctly the bitcoinica hacks got really high points.

yes. this particular one from the litecoin launch sticked with me:

Topic                                   Score
Bitcoin Savings and Trust               10185
"Book club"                               120
US health care mandate (Obamacare)        105
Look at a pirate, eye to eye if you dare.  77
[ANN] BITCOIN CHIPIN is here!              45
LTCPPT - Litecoin Pirate Pass Through      31

Has there ever been a 5-digit (i..e, 10,000 or higher) score before?

Not sure what your indicator would read for that one though. 

probably I will go for it together with a little site restructuring but will take some time.

I'm happy with it so far, too.


thought about more forum indicators lately...  number of posts or maybe a mood indicator based on a count of positive / negative words ("BUY BUY BUY")

The added 5 topic are very interesting,  The top 10 was mostly just feedback of the hive-mind crazes.

Yes for me.  That's the only page I look at.

Thing can always be changed but I find it better this way.  Doesn't look like a "dashboard" though....

The chart I look at the most regularly is http://bitcoin.sipa.be/speed-lin-2k.png

done. not sure I like the current layout, though, might change.

would it be better if bitcoinx.com would lead directly to the bitcoin charts and dashboard page?

thank you for the feedback! 

depending on forum traffic the lower top threads can get quite random. but let's just give it a try. will make the suggested changes in the next couple of days.

Could you raise TOP 10 topic to 15 (or more)

I watch your website about 5 time a day and still feel like I'm missing a lot of interesting thread.

I also suggest that you move the top 15 discussions to the top instead of the news section that seems to get outdated very fast.

Thanks for being my morning front page.

All network data values now have a fall back source.

thanks for letting me know. blockexplorer.com api is partially down. I have set up a fix so that in case of error data will be taken from blockexplorer.info.

will have to check on this tonight/tomorrow

Something seems to be off with days/hours for next difficulty...it's been at 3 days for 3 days 

actually it was quite generous, thanks! 

The math seems to make sense and the chart looks good.

Thanks for fixing this and for the acknowledgement.

I sent you another small tip, I really appreciate !

Difficulty estimate fixed: For quite a while the difficulty estimate on bitcoinx.com/charts was broken (data and graphs). Thanks to Jonathan for notifying me. It should be fixed now and be relatively accurate right after retargeting, too.

The estimate from blockexplorer.com is used: indirectly for the first 504 blocks after retargeting, otherwise direct.


please peer review:

Here you go: http://allchains.info/

Yes it is. And I'd like to know without having to download the whole blockchain.

is it so important? you can always use a thin client.

I would expect the size to correlate with the number of tx.

thinking about it 

I really wish at least one of these statistics sites would show how big in terms of memory space is the current blockchain.

finally blocked the newbies board because of "post for coins" threads

these boards are blocked from the top10:

bannedBoards = [5, 51, 52, 53, 56, 65, #marketplace
                9, # off-topic
                27, 31, 32, 33, # spanish
                30, # chinese
                16, 35, 36, 60, 61, 62, 63, 64,    # german
                13, 47, 48, 49, 50, 54, # french
                28, # italian
                29, 69, 70, # portugiesisch
                10, 18, 19, 20, 21, 22, 23, 55, 66, 72, # russian
                45, # skandinavisch
                11, # other languages
                ]

should new boards be created and show up in the top10 I will block them, too

somebody asked me how the score for the hot topic scraper is calculated:

the score is calculated by the number of posts and the number of different users that posted in the thread. also the average userid is taken into account such that gavin for example has ~~3.5 times more weight than a new user.

added a 90 day chart and an awkward fit curve

nice! I had hoped someone would do the hash growth diagram - much better than the pie.

careful with the histograms, their data has yet to grow as with the other charts.

The charts on http://www.l0ss.net/ are very nice (so are the ones from OP).  What are the pool luck charts showing?  Is it a comparison of the actual blocks they are finding in relation to what their total hash rate suggests they should be finding?

Beautiful! Thanks for putting these charts together. I note that mining profitability is misleading early on when nobody knew how to do GPU mining. Also, it should probably be called "mining revenue"

I wonder if it would be possible to graph mining costs, pre and post GPU revolution. If you subtract mining costs from mining revenue, you get the actual mining profitability.

It would also be interesting to see some scatterplots, like:

mining profitability vs % network growth over the next X days

Anyone who finds these graphs useful should also check out:
http://bitcoin.sipa.be/
http://www.l0ss.net/hist.php

thanks! 

Nice data, appreciate the site.

zlol. thanks for the hint, I put it on the site.

another way of choosing the scales was to get the curves as close together as possible on a minimum squares base. but then the charts would get cluttered with double grid lines...

On this forum? Uh... yeah? 

the linear charts are automatically scaled whereas I chose some arbitrary scalings for the log charts.

of course the relation of the two has no direct absolute meaning. You think it's necessary to point that out?

it might be interesting to choose a scaling where intersecting lines would equal to estimated average mining energy costs.

In the first chart, change the difficulty scale from 0-2mil to 0-10mil.
The chart will look like Bitcoin is WAY overpriced compared to difficulty. Why? Because my scale is just as arbitrary as yours.

maybe it's the thieves dumping stolen coins...

We have been underselling ourselves for months. It's time for everyone to WAKE UP!

linear and logarithmic charts, MtGox USD vs. Difficulty, difficulty estimate in chart, mining factor....

check it out:

http://bitcoinx.com/charts/


Anyone know a (free) API for historic gold prices?

What about an API for historic number of transactions? I guess I'll have to crawl the block chain for this one myself