Your full node info page | Bitcointalksearch.org

shorena

copper member

Activity: 1498

Merit: 1499

No I dont escrow anymore.

Quote from: tnkflx on December 12, 2014, 08:28:59 PM

Shorena, how do you get the stats from bitcoind?

I just "ask" bitcoind what I want to know. Had to start a reindex yesterday because the DB got corrupted, thus they currently look bad.

Code:

n="$(bitcoind getinfo | grep connections | egrep -o [0-9]*)"
rrdtool update connections.rrd N:$n

tnkflx

sr. member

Activity: 349

Merit: 250

Shorena, how do you get the stats from bitcoind?

statoshi

newbie

Activity: 25

Merit: 66

Yes, Grafana is highly customizable and all of the graphs are editable. You can even build your own charts from scratch, though I have it locked down so that you need the admin password in order to permanently store them. If you compose a chart on statoshi.info that you think would be useful for the general public, feel free to save it as a JSON payload and send it to me; I'll add it as a permanent chart.

As for machine specs, I'm running on a single core VPS with 30 GB of disk (that I'll need to bump up soon) with 4 GB of RAM, though bitcoind only uses about 1.5 GB. My node uses ~150 GB of (mostly upstream) bandwidth per month. I hope to eventually make it easier for people to install the entire software stack required to run a Statoshi instance; at the moment it requires a fair amount of Linux admin skills to install & configure.

shorena

copper member

Activity: 1498

Merit: 1499

No I dont escrow anymore.

Quote from: statoshi on November 06, 2014, 09:54:04 AM

My full node metrics: http://statoshi.info

Very nice, Now I want to get another server running to play with your fork.
There is a strange button on the webpage though, lower right corner: "add a row". When I click it (chrome) it adds an empty row. Not sure what thats about
...ooh, wow. Nevermind I got it.

What is the minimum in terms of disk, cpu and ram you would recommend?

Quote from: statoshi on November 06, 2014, 09:54:04 AM

I'm running a fork of Bitcoin Core that emits metrics to Statsd, which are then collected by Graphite and rendered with flot graphs by Grafana.

https://medium.com/@lopp/announcing-statoshi-realtime-bitcoin-node-statistics-61457f07ee87

https://github.com/jlopp/statoshi

statoshi

newbie

Activity: 25

Merit: 66

My full node metrics: http://statoshi.info

I'm running a fork of Bitcoin Core that emits metrics to Statsd, which are then collected by Graphite and rendered with flot graphs by Grafana.

https://medium.com/@lopp/announcing-statoshi-realtime-bitcoin-node-statistics-61457f07ee87

https://github.com/jlopp/statoshi

shorena

copper member

Activity: 1498

Merit: 1499

No I dont escrow anymore.

Quote from: deepceleron on September 15, 2014, 05:24:37 AM

Quote from: shorena on September 15, 2014, 12:50:49 AM

... not sure what to add since I dont want to publish IP addresses or peer informations

Bitcoin users know that their IP addresses are public if they participate in the peer-to-peer network without specifically choosing their peers. The most offending demonstration: https://blockchain.info/ip-log

If you don't want to show the IPs of connections, you can dump your port 8333 connection IPs through a salted hash to make an identifier; bash example:

Code:

ipaddr="127.0.0.1"
salt="mysecret"
hash=$(echo $ipaddr$salt | sha256sum)
id=${hash:0:8}
echo $id

gives an identifier "b10ea208", nearly as unique as IP addresses.

Privacy is not a concern in that regard. I dont think IP addresses are personal information anyway. Behind most IP's are several users, sometimes companies with 1000's of employees and if someone wanted to hide his/her IP address there are services like tor. But IP addresses connected to is nothing I can make a picture of with rrdtool and I dont want to add php/mysql. Im perfectly fine with a number of connections in that regard. Frankly I think a list of IP addresses isnt that interesting to look at. A full dump of getpeerinfo (see other sites linked above) is a different matter, but I will restrict the information shown to number of bits/connections etc.
# of connections by version might be something to consider, but that would require a very sophisticated script. Get different versions, check if data base for version exists if so update with current count, if not make a new one and add a cronjob to update the resulting picture as well as the html page. Depending on the name of the version used that could probably result in problems along the way.

deepceleron

legendary

Activity: 1512

Merit: 1028

Quote from: shorena on September 15, 2014, 12:50:49 AM

... not sure what to add since I dont want to publish IP addresses or peer informations

Bitcoin users know that their IP addresses are public if they participate in the peer-to-peer network without specifically choosing their peers. The most offending demonstration: https://blockchain.info/ip-log

If you don't want to show the IPs of connections, you can dump your port 8333 connection IPs through a salted hash to make an identifier; bash example:

Code:

ipaddr="127.0.0.1"
salt="mysecret"
hash=$(echo $ipaddr$salt | sha256sum)
id=${hash:0:8}
echo $id

gives an identifier "b10ea208", nearly as unique as IP addresses.

shorena

copper member

Activity: 1498

Merit: 1499

No I dont escrow anymore.

I tamed rrdtool now to do what I want. The below pictures are from this morning. The 24h one is updated every 30 minutes and the 30d one is updated every 12h. Full page is here: http://213.165.91.169/ not sure what to add since I dont want to publish IP addresses or peer informations

shorena

copper member

Activity: 1498

Merit: 1499

No I dont escrow anymore.

Its working

Apparently I did nothing wrong I was just not patient enough. I just used the same script from last time and made a cronjob so it runs every 5 minutes. After waiting some hours rrdtool generates pictures

Edit: moved pic to imgur, since I changed it.

To update I use:

Code:

#!/bin/sh
n="$(bitcoind getinfo | grep connections | egrep -o [0-9]*)"
#echo n
rrdtool update connections.rrd N:$n

and the above picture is generated by:

Code:

#!/bin/sh

rrdtool graph /../pic/con24.png
--start -86400 --title "connections 24h" -v "# connections" -w 500
--x-grid MINUTE:30:HOUR:1:HOUR:2:0:%R
DEF:min=connections.rrd:connections:MIN
DEF:avg=connections.rrd:connections:AVERAGE
DEF:max=connections.rrd:connections:MAX
LINE1:min#ff0000:"minimum"
LINE1:avg#00ff00:"average"
LINE1:max#0000ff:"maximum"

Note that the connections.rrd is in the same dir as "canhazpic.sh" and I added linebreaks to make it easier to read.

Its not yet updating via cron as I want to play around some more with the colors and make it look more like this:

shorena

copper member

Activity: 1498

Merit: 1499

No I dont escrow anymore.

Quote from: zvs on September 10, 2014, 12:15:37 PM

-snip-

haha, that 1st page is making my brain hurt.
-snip-
oh, i thought the charts on #2 looked cool.. just not something I could use, since it's still just taking all traffic & I have many things running heh

Thanks

After I did the tutorial twice I though it should be easy. Maybe cacti [1] is something for you? It builds on top of rrdtool, needs php and mysql and other stuff though, so its a bit overkill for what I need. You could feed it with data directly from bitcoind, thus you have only that traffic as well as other information. And if I am not mistake you can compleltly manage it via the web interface.

[1] http://www.cacti.net/

zvs

legendary

Activity: 1680

Merit: 1000

https://web.archive.org/web/*/nogleg.com

Quote from: shorena on September 10, 2014, 03:54:31 AM

I still cant get rrdtool to accept the number of connections. It allways displays NaN even though I follow the tutorial [1]

Code:

Get the value and put it in variable "$speed"
rrdtool update speed.rrd N:$speed

And when I return $con to std:out it shows as number -_- frustrating.

I modified the apache default page a bit to show some infos for now [2]. Keep in mind its a work in progress. I might have some hours tomorrow to battle with rrd again.

Did any of you ever use it? It looks like a simple and powerfull tool, I probably just miss something.

[1] http://oss.oetiker.ch/rrdtool/tut/rrdtutorial.en.html
[2] http://213.165.91.169/

haha, that 1st page is making my brain hurt.

re: mrflibblehat, that rrd thing should be able to convert those ping times to ms as well.

my bitcoin-cli for example shows;

"pingtime" : 0.13562600
"pingtime" : 2.79798200,
"pingtime" : 0.05269900,
"pingtime" : 0.19598100,

etc... which I believe are 135ms, 2.8s, 52ms, and 196ms? the 2.8s one also has a "pingwait" : 582.27381700 ... i'm guessing he's DLing the blockchain and computer is overloaded or something.

oh, i thought the charts on #2 looked cool.. just not something I could use, since it's still just taking all traffic & I have many things running heh

mrflibblehat

sr. member

Activity: 350

Merit: 250

★YoBit.Net★ 350+ Coins Exchange & Dice

Quote from: zvs on September 09, 2014, 09:15:22 PM

Quote from: mrflibblehat on September 07, 2014, 08:02:32 AM

I found this post interesting and thought I would make an info page for my node. I got a few ideas by looking around at different pages and came up with this.

http://188.165.242.83/node/

I created the script in Python and use cron to run it every 1 minute for autoupdate.

If you are interested you can download it from here - https://www.dropbox.com/s/3g6qho1e2q0z6p6/FullNode.zip?dl=0

This contains the python script and the css file for formatting (the css file goes with the create HTML file)

The script uses 2 external dependencies

hurry.filesize (pip install hurry.filesize)
python-bitcoinrpc (https://github.com/jgarzik/python-bitcoinrpc) You can install this by cloning it and running "python setup.py install"

you will need to edit the script and change the following text to what you want.

RPCUSER
RPCPASS
IPADDRESS
LOCATION

In the coming days I plan to add some graphs, history and make it look a little prettier.

If you would like to donate my address is 1QF271jjauFrfojG42b8frfLz4eNRkzpDy

Hope this is useful to someone

ah, looking good. like a clone of 176.10.116.242, but not so polished yet. =p

I guess you were syncing with blockchain, w/ those downloaded amounts... shame about the BQS nodes.. things like that are one of the reasons ppl complain about slow syncs. I think it takes .. 3 minutes? ... before it asks another peer for those blocks.

I sent .01 to 13qDi8k6cPvwAEDVGL7TYrGcAgEtKb22jt a while back (176.10.116.242 fellow).. slick it up some and I'd send $5 or $10 or something. =p

I prefer the gray/white/black color scheme... also, re: 176.10.116.242 and the ping times... if you could have it adjust those to ms that'd be a step up. Another thing it's lacking is total amount sent & total amount received (which would include the peers it has connected to & disconnected from). I *think* that information is available in bitcoin-cli, but not positive.

also, there sure are a lot of dead nodes that just receive info and never send anything out (on 176.10.116.242, 130.253.21.121, 195.113.161.13, 219.243.208.60 amongst others appear to just be wasting inbound slots)... *boggle*

Thanks

Yeah I got the idea from that site, Im currently in the process of upgrading it, I think il add a feature to be able to choose a different css file (so it can have multiple themes)!

also I plan on adding the ping time but all mine are currently 0?!?! I know bitcoind gets it ping value from network ping and processing backlog.

I should have something nicer and clearer up in the next couple of days

shorena

copper member

Activity: 1498

Merit: 1499

No I dont escrow anymore.

I still cant get rrdtool to accept the number of connections. It allways displays NaN even though I follow the tutorial [1]

Code:

Get the value and put it in variable "$speed"
rrdtool update speed.rrd N:$speed

And when I return $con to std:out it shows as number -_- frustrating.

I modified the apache default page a bit to show some infos for now [2]. Keep in mind its a work in progress. I might have some hours tomorrow to battle with rrd again.

Did any of you ever use it? It looks like a simple and powerfull tool, I probably just miss something.

[1] http://oss.oetiker.ch/rrdtool/tut/rrdtutorial.en.html
[2] http://213.165.91.169/

zvs

legendary

Activity: 1680

Merit: 1000

https://web.archive.org/web/*/nogleg.com

Quote from: mrflibblehat on September 07, 2014, 08:02:32 AM

I found this post interesting and thought I would make an info page for my node. I got a few ideas by looking around at different pages and came up with this.

http://188.165.242.83/node/

I created the script in Python and use cron to run it every 1 minute for autoupdate.

If you are interested you can download it from here - https://www.dropbox.com/s/3g6qho1e2q0z6p6/FullNode.zip?dl=0

This contains the python script and the css file for formatting (the css file goes with the create HTML file)

The script uses 2 external dependencies

hurry.filesize (pip install hurry.filesize)
python-bitcoinrpc (https://github.com/jgarzik/python-bitcoinrpc) You can install this by cloning it and running "python setup.py install"

you will need to edit the script and change the following text to what you want.

RPCUSER
RPCPASS
IPADDRESS
LOCATION

In the coming days I plan to add some graphs, history and make it look a little prettier.

If you would like to donate my address is 1QF271jjauFrfojG42b8frfLz4eNRkzpDy

Hope this is useful to someone

ah, looking good. like a clone of 176.10.116.242, but not so polished yet. =p

I guess you were syncing with blockchain, w/ those downloaded amounts... shame about the BQS nodes.. things like that are one of the reasons ppl complain about slow syncs. I think it takes .. 3 minutes? ... before it asks another peer for those blocks.

I sent .01 to 13qDi8k6cPvwAEDVGL7TYrGcAgEtKb22jt a while back (176.10.116.242 fellow).. slick it up some and I'd send $5 or $10 or something. =p

I prefer the gray/white/black color scheme... also, re: 176.10.116.242 and the ping times... if you could have it adjust those to ms that'd be a step up. Another thing it's lacking is total amount sent & total amount received (which would include the peers it has connected to & disconnected from). I *think* that information is available in bitcoin-cli, but not positive.

also, there sure are a lot of dead nodes that just receive info and never send anything out (on 176.10.116.242, 130.253.21.121, 195.113.161.13, 219.243.208.60 amongst others appear to just be wasting inbound slots)... *boggle*

mrflibblehat

sr. member

Activity: 350

Merit: 250

★YoBit.Net★ 350+ Coins Exchange & Dice

I found this post interesting and thought I would make an info page for my node. I got a few ideas by looking around at different pages and came up with this.

http://188.165.242.83/node/

I created the script in Python and use cron to run it every 1 minute for autoupdate.

If you are interested you can download it from here - https://www.dropbox.com/s/3g6qho1e2q0z6p6/FullNode.zip?dl=0

This contains the python script and the css file for formatting (the css file goes with the create HTML file)

The script uses 2 external dependencies

hurry.filesize (pip install hurry.filesize)
python-bitcoinrpc (https://github.com/jgarzik/python-bitcoinrpc) You can install this by cloning it and running "python setup.py install"

you will need to edit the script and change the following text to what you want.

RPCUSER
RPCPASS
IPADDRESS
LOCATION

In the coming days I plan to add some graphs, history and make it look a little prettier.

If you would like to donate my address is 1QF271jjauFrfojG42b8frfLz4eNRkzpDy

Hope this is useful to someone

shorena

copper member

Activity: 1498

Merit: 1499

No I dont escrow anymore.

Quote from: zvs on September 05, 2014, 10:58:50 PM

-snip-

I was looking at that, but doesn't seem to be a way to limit it to show traffic by progamid (or by port, incoming port 8333 + outgoing port 8333).... so it shows all my other traffic too, not just including bitcoind.

May be something other than vnstat out there that will do that, but I couldn't find anything..

ed: well, the internal stats that bitcoin keeps would work. which could probably be added if something like what is on 176.10.116.242 was done

Yeah, but the internal stats are messured per peer, or is there a central data point? Since my server does nothing but run bitcoind I doubt there is much traffic otherwise. Well besides me checking the stats and connecting via ssh.

Newar

legendary

Activity: 1358

Merit: 1000

https://gliph.me/hUF

Quote from: shorena on September 01, 2014, 09:04:05 AM

[...], but it looks like most of them use PHP, not really into that Sad

[..]

They work out of the box. I can personally confirm that no PHP knowledge is needed. Cheesy

Quote from: zvs on September 05, 2014, 02:34:21 PM

Yeah, I posted about http://176.10.116.242/ a while back, but nobody answered (so I guess it's not public domain)...

I'm looking into it.

Quote from: zvs on September 05, 2014, 10:58:50 PM

I was looking at that, but doesn't seem to be a way to limit it to show traffic by progamid (or by port, incoming port 8333 + outgoing port 8333).... so it shows all my other traffic too, not just including bitcoind.

May be something other than vnstat out there that will do that, but I couldn't find anything..

ed: well, the internal stats that bitcoin keeps would work. which could probably be added if something like what is on 176.10.116.242 was done

nethogs shows traffic by PID, I don't know how to put that info on a web page though.

zvs

legendary

Activity: 1680

Merit: 1000

https://web.archive.org/web/*/nogleg.com

Quote from: shorena on September 05, 2014, 03:17:47 PM

Quote from: zvs on September 05, 2014, 02:34:21 PM

Quote from: Newar on September 01, 2014, 03:15:04 AM

Quote from: shorena on August 28, 2014, 07:59:10 AM

[...] I found this script: http://pastebin.com/wKytLZmc but noone running it. [...]

I run it with some modifications, i.e. http://84.200.34.113
I plan to add server uptime and bitcoind uptime to it too, but haven't gotten around it yet.

Morbilas (he wrote the one you linked to, if I'm not mistaken) also uses it on the nodes he sets up: https://bitcointalksearch.org/topic/psa-add-a-full-node-for-just-19year-582817

I came across this which I quite like too: http://176.10.116.242/

Yeah, I posted about http://176.10.116.242/ a while back, but nobody answered (so I guess it's not public domain)... I imagine it's pretty easy to do, but I wouldn't know how to make something like that.

You could try to make a post in the german section.

Im still working on a rrdtool test setup, update on my traffic:

It looks like the peers launch bitcoin-core after work/school/etc and shutdown after the/over night.

I was looking at that, but doesn't seem to be a way to limit it to show traffic by progamid (or by port, incoming port 8333 + outgoing port 8333).... so it shows all my other traffic too, not just including bitcoind.

May be something other than vnstat out there that will do that, but I couldn't find anything..

ed: well, the internal stats that bitcoin keeps would work. which could probably be added if something like what is on 176.10.116.242 was done

shorena

copper member

Activity: 1498

Merit: 1499

No I dont escrow anymore.

Quote from: zvs on September 05, 2014, 02:34:21 PM

Quote from: Newar on September 01, 2014, 03:15:04 AM

Quote from: shorena on August 28, 2014, 07:59:10 AM

[...] I found this script: http://pastebin.com/wKytLZmc but noone running it. [...]

I run it with some modifications, i.e. http://84.200.34.113
I plan to add server uptime and bitcoind uptime to it too, but haven't gotten around it yet.

Morbilas (he wrote the one you linked to, if I'm not mistaken) also uses it on the nodes he sets up: https://bitcointalksearch.org/topic/psa-add-a-full-node-for-just-19year-582817

I came across this which I quite like too: http://176.10.116.242/

Yeah, I posted about http://176.10.116.242/ a while back, but nobody answered (so I guess it's not public domain)... I imagine it's pretty easy to do, but I wouldn't know how to make something like that.

You could try to make a post in the german section.

Im still working on a rrdtool test setup, update on my traffic:

It looks like the peers launch bitcoin-core after work/school/etc and shutdown after the/over night.

zvs

legendary

Activity: 1680

Merit: 1000

https://web.archive.org/web/*/nogleg.com

Quote from: Newar on September 01, 2014, 03:15:04 AM

Quote from: shorena on August 28, 2014, 07:59:10 AM

[...] I found this script: http://pastebin.com/wKytLZmc but noone running it. [...]

I run it with some modifications, i.e. http://84.200.34.113
I plan to add server uptime and bitcoind uptime to it too, but haven't gotten around it yet.

Morbilas (he wrote the one you linked to, if I'm not mistaken) also uses it on the nodes he sets up: https://bitcointalksearch.org/topic/psa-add-a-full-node-for-just-19year-582817

I came across this which I quite like too: http://176.10.116.242/

Yeah, I posted about http://176.10.116.242/ a while back, but nobody answered (so I guess it's not public domain)... I imagine it's pretty easy to do, but I wouldn't know how to make something like that.

Topic: Your full node info page (Read 5576 times)