Pages:
Author

Topic: Re: How (and why) to use the Relay Network - page 8. (Read 695 times)

legendary
Activity: 1260
Merit: 1019
don't get it.. blocks?   the relay always sends me transactions after i've already received them, so it's essentially wasting bandwidth
Your Gangnam-nodes are also a relay-network Smiley
zvs
legendary
Activity: 1680
Merit: 1000
https://web.archive.org/web/*/nogleg.com
Oh, yah, I realized that.  I figured turning it off would save some bandwidth, at least.  Everything is running with the updated code now, though I still wish there was a way to set it to blocks only.. don't need transactions relayed to any of my nodes, really.
THat would entirely defeat the purpose...you cant compress without pre-relaying.
don't get it.. blocks?   the relay always sends me transactions after i've already received them, so it's essentially wasting bandwidth
hero member
Activity: 755
Merit: 515
Oh, yah, I realized that.  I figured turning it off would save some bandwidth, at least.  Everything is running with the updated code now, though I still wish there was a way to set it to blocks only.. don't need transactions relayed to any of my nodes, really.
THat would entirely defeat the purpose...you cant compress without pre-relaying.
zvs
legendary
Activity: 1680
Merit: 1000
https://web.archive.org/web/*/nogleg.com
This seems to be relaying massive amounts of dust transactions, I had over 25000 stored.

Had to turn it off.  Now debug.log is getting spammed with

and a good 50% or more of my peers are sending me an inordinate amount of data

This is not specific to the relay network, this was just someone flooding the Bitcoin network with lots of transactions. The relay network, at least, will no longer do this.
Oh, yah, I realized that.  I figured turning it off would save some bandwidth, at least.  Everything is running with the updated code now, though I still wish there was a way to set it to blocks only.. don't need transactions relayed to any of my nodes, really.

but it still

Code:
----system---- ------sockets------ -net/venet0 ---load-avg---
     time     |tot tcp udp raw frg| recv  send| 1m   5m  15m
09-07 17:24:12|  0 117   7   0   0|   0     0 |0.31 0.30 0.36
09-07 17:24:13|  0 117   7   0   0| 654k   95k|0.29 0.30 0.36
09-07 17:24:14|  0 116   7   0   0| 593k  125k|0.29 0.30 0.36
09-07 17:24:15|  0 116   7   0   0| 495k   83k|0.29 0.30 0.36
09-07 17:24:16|  0 116   7   0   0| 496k   72k|0.29 0.30 0.36
09-07 17:24:17|  0 116   7   0   0| 335k   31k|0.29 0.30 0.36
09-07 17:24:18|  0 116   7   0   0| 402k   32k|0.27 0.29 0.36
09-07 17:24:19|  0 116   7   0   0| 523k   39k|0.27 0.29 0.36
09-07 17:24:20|  0 116   7   0   0| 443k   74k|0.27 0.29 0.36
09-07 17:24:21|  0 116   7   0   0| 474k  138k|0.27 0.29 0.36
09-07 17:24:22|  0 116   7   0   0| 469k   82k|0.27 0.29 0.36
09-07 17:24:23|  0 116   7   0   0| 451k   71k|0.24 0.29 0.35
09-07 17:24:24|  0 115   7   0   0| 533k   98k|0.24 0.29 0.35
09-07 17:24:25|  0 115   7   0   0| 611k   97k|0.24 0.29 0.35
09-07 17:24:26|  0 115   7   0   0| 648k   84k|0.24 0.29 0.35
09-07 17:24:27|  0 115   7   0   0| 652k  149k|0.24 0.29 0.35
09-07 17:24:28|  0 115   7   0   0| 635k   66k|0.30 0.30 0.36
09-07 17:24:29|  0 115   7   0   0| 458k   70k|0.30 0.30 0.36
09-07 17:24:30|  0 115   7   0   0| 514k  124k|0.30 0.30 0.36
09-07 17:24:31|  0 115   7   0   0| 444k   60k|0.30 0.30 0.36
09-07 17:24:32|  0 115   7   0   0| 366k   41k|0.30 0.30 0.36
09-07 17:24:33|  0 115   7   0   0| 375k   51k|0.28 0.29 0.36
09-07 17:24:34|  0 115   7   0   0| 402k   47k|0.28 0.29 0.36
09-07 17:24:35|  0 115   7   0   0| 790k  105k|0.28 0.29 0.36
09-07 17:24:36|  0 115   7   0   0| 502k   72k|0.28 0.29 0.36
09-07 17:24:37|  0 115   7   0   0| 623k   86k|0.28 0.29 0.36
09-07 17:24:38|  0 115   7   0   0| 403k  127k|0.34 0.31 0.36
09-07 17:24:39|  0 115   7   0   0| 595k   56k|0.34 0.31 0.36
09-07 17:24:40|  0 115   7   0   0| 466k  112k|0.34 0.31 0.36
09-07 17:24:41|  0 115   7   0   0| 340k   89k|0.34 0.31 0.36
09-07 17:24:42|  0 115   7   0   0| 655k   71k|0.34 0.31 0.36
09-07 17:24:43|  0 115   7   0   0| 598k   93k|0.31 0.30 0.36
09-07 17:24:44|  0 115   7   0   0| 595k   48k|0.31 0.30 0.36
09-07 17:24:45|  0 115   7   0   0| 465k   95k|0.31 0.30 0.36
09-07 17:24:46|  0 115   7   0   0| 543k   89k|0.31 0.30 0.36
09-07 17:24:47|  0 115   7   0   0| 489k   43k|0.31 0.30 0.36
09-07 17:24:48|  0 115   7   0   0| 492k   50k|0.29 0.29 0.35
09-07 17:24:49|  0 116   7   0   0| 652k  122k|0.29 0.29 0.35
09-07 17:24:50|  0 116   7   0   0| 461k  121k|0.29 0.29 0.35
09-07 17:24:51|  0 116   7   0   0| 601k   57k|0.29 0.29 0.35
09-07 17:24:52|  0 116   7   0   0|1074k   71k|0.29 0.29 0.35
09-07 17:24:53|  0 116   7   0   0|2315k  123k|0.26 0.29 0.35
09-07 17:24:54|  0 116   7   0   0|2372k  213k|0.26 0.29 0.35
09-07 17:24:55|  0 116   7   0   0|2290k  136k|0.26 0.29 0.35
09-07 17:24:56|  0 116   7   0   0|2475k  117k|0.26 0.29 0.35
09-07 17:24:57|  0 116   7   0   0|3616k  174k|0.26 0.29 0.35
09-07 17:24:58|  0 116   7   0   0|3001k  154k|0.24 0.28 0.35
09-07 17:24:59|  0 116   7   0   0|2388k  110k|0.24 0.28 0.35
09-07 17:25:00|  0 116   7   0   0| 392k  110k|0.24 0.28 0.35
09-07 17:25:01|  0 116   7   0   0| 486k   39k|0.24 0.28 0.35
09-07 17:25:02|  0 116   7   0   0|1098k  165k|0.24 0.28 0.35
09-07 17:25:03|  0 116   7   0   0|2137k  239k|0.30 0.30 0.35
09-07 17:25:04|  0 117   7   0   0|3196k  125k|0.30 0.30 0.35
09-07 17:25:05|  0 117   7   0   0|5036k  137k|0.30 0.30 0.35
09-07 17:25:06|  0 117   7   0   0|7243k  120k|0.30 0.30 0.35
09-07 17:25:07|  0 117   7   0   0|5251k  200k|0.30 0.30 0.35
09-07 17:25:08|  0 117   7   0   0|5138k  147k|0.36 0.31 0.36^C

can't be stopped, unless I go through and start firewalling every single one of those nodes.  That's on one of my 100mbit connections too, which is why it only has 100 connections.  I've seen 50MB/s receive amts on 1Gbit

ed:   someone with several thousand $'s to waste could easily get some 10Gbit dedicated lines, and just flood everyone out with trash transactions.

ed2: and i suppose I should point out that this would be different from a typical DoS attack in that they'd have hundreds, or a thousand or two or three helpers, all transmitting as many of those transactions as their bandwidth could handle (turning it into a DDoS, with the quickness!)...   ed3 (last one I hope, wtf):  I've considered upping the minimum version level of connections...  that would get rid of 99%, just the 1% malicious would remain
hero member
Activity: 755
Merit: 515
lol, is your libc REALLY broken??? I'm really not sure how thats possible...is this reproduceable?

 Cheesy I'm buggered if I know - it only happened the once too, so not reproducible I'm afraid.

Can you try with connections to the relay network blocked (I'm assuming this happened when you couldn't connect to any servers)?

I still get a warning when compiling the latest version, but it does run now. I'm still getting plenty of this:

Code:
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
Its having trouble connecting to your local bitcoind it looks like? Anything in your debug.log?

Thanks for adding the extra info to the web page Matt. I'm sure the attack situation must be wrecking your head somewhat, especially with noobs like me constantly asking silly questions!  Cheesy  Your work is very much appreciated, & I for one will be throwing a tip or two your way as & when I can - thanks again  Smiley
Meh, its a change I had been meaning to make for a while, this just forced my hand.
sr. member
Activity: 266
Merit: 250

Kano - thank you for that, I wasn't aware that Matt had changed the start up syntax. However, like you say, it's not responding & as soon as I started it up it locked my system up completely requiring a full reset.

Switched it off & disabled it again  Sad

lol, is your libc REALLY broken??? I'm really not sure how thats possible...is this reproduceable?


 Cheesy I'm buggered if I know - it only happened the once too, so not reproducible I'm afraid.

I still get a warning when compiling the latest version, but it does run now. I'm still getting plenty of this:

Code:
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)

Thanks for adding the extra info to the web page Matt. I'm sure the attack situation must be wrecking your head somewhat, especially with noobs like me constantly asking silly questions!  Cheesy  Your work is very much appreciated, & I for one will be throwing a tip or two your way as & when I can - thanks again  Smiley
hero member
Activity: 755
Merit: 515
This seems to be relaying massive amounts of dust transactions, I had over 25000 stored.

Had to turn it off.  Now debug.log is getting spammed with

and a good 50% or more of my peers are sending me an inordinate amount of data

This is not specific to the relay network, this was just someone flooding the Bitcoin network with lots of transactions. The relay network, at least, will no longer do this.
hero member
Activity: 755
Merit: 515
Seems I'm back to only sending (on west) and rarely getting txns back.
The rare Received ones are always just after, and the same size as, so most likely the same transactions as, the non "(out-of-band)" that it says I send.
This is largely expected. When you're sending out-of-band it means you're running an unmodified bitcoind (more on that later). The send/receive is the same as before, you're echoing the server's sent txn (the printing of what is happenning is out of order, I fixed that now). However, the server is now sending transactions less often (only when its view of what the next few blocks should be changes, instead of on each new transaction).
legendary
Activity: 4634
Merit: 1851
Linux since 1997 RedHat 4
Seems I'm back to only sending (on west) and rarely getting txns back.
The rare Received ones are always just after, and the same size as, so most likely the same transactions as, the non "(out-of-band)" that it says I send.
hero member
Activity: 755
Merit: 515
What are these?

Code:
Sent transaction of size 226 (out-of-band) to relay server

out-of-band transactions are transactions sent but not assumed to be in the next block (ie they will not be compressed if they do appear in the next block). More to come on the website soon.
legendary
Activity: 4634
Merit: 1851
Linux since 1997 RedHat 4
Oh, and I added a donation address to cover the 45USD/month I'm currently shelling out for the servers :p (1NRuqMJAzUGwvFigukLa3UZqcJXix1dETM, feel free to check that address' balance and only donate if the server costs havent already been covered, though if its usually over I'll invest in more servers)
Just updated my main node from the Jul-4 version to now.

All running OK now.

The Jul-4 version regularly disconnects/reconnects and doesn't send block notifications to bitcoind because of that ... so I'd guess update it is necessary.

What are these?

Code:
Sent transaction of size 226 (out-of-band) to relay server
hero member
Activity: 755
Merit: 515
Oh, and I added a donation address to cover the 45USD/month I'm currently shelling out for the servers :p (1NRuqMJAzUGwvFigukLa3UZqcJXix1dETM, feel free to check that address' balance and only donate if the server costs havent already been covered, though if its usually over I'll invest in more servers)
hero member
Activity: 755
Merit: 515
Sorry for the lack of replies, I've been heads-down rewriting the way transactions are handled to work in cases where mempools are reasonably large. That seems to be working now (though much tweaking remains to be done to make it perform well). There is also a new node in Tokyo (you may want to restart your client to auto-detect if its closer for you, or if you had already been connecting to the jpy hostname, you should already be using it).

I'm gonna try to be better at keeping an updated news section in on http://bitcoinrelaynetwork.org, so keep an eye there as things get updated. There will be new operating instructions due to the changes (though if you ignore them things will still work, and probably just as effeciently unless you're using strange transaction policy).

Kano - thank you for that, I wasn't aware that Matt had changed the start up syntax. However, like you say, it's not responding & as soon as I started it up it locked my system up completely requiring a full reset.

Switched it off & disabled it again  Sad
lol, is your libc REALLY broken??? I'm really not sure how thats possible...is this reproduceable?


edit: I fixed it by disabling it attempting to build that file since it's not even included in my binary. Probably shouldn't build all the assembly files unconditionally.
I did this on master now.

Quote from: btcash
Information about when blocks are received is made available publicly
on the stats page
Where do I find this page?
http://bitcoinrelaynetwork.org/stats.html (though that will probably be wiped in not too long as my stats-processing script doesnt handle adding/removing nodes very well :/, if anyone needs old stats I will be glad to provide them).
sr. member
Activity: 266
Merit: 250

He changed the parameter order ...

You now should just:
./relaynetworkclient 127.0.0.1 8333

You can specify which server to use on the end, but it will automatically choose one if you don't, by running a ping test to find the closest.

Edit: note the relay isn't responding at the moment at all ...

Kano - thank you for that, I wasn't aware that Matt had changed the start up syntax. However, like you say, it's not responding & as soon as I started it up it locked my system up completely requiring a full reset.

Switched it off & disabled it again  Sad
legendary
Activity: 4634
Merit: 1851
Linux since 1997 RedHat 4
When I try to run it, I only get this:

Code:
Using server 8333

Running 14.04 64bit.
Hello!

I have the same issue.  Huh

http://s30.postimg.org/43o0fs8w1/relay.png

Prior to update everything worked. Ubuntu Server 64bit 14.04.02
He changed the parameter order ...

You now should just:
./relaynetworkclient 127.0.0.1 8333

You can specify which server to use on the end, but it will automatically choose one if you don't, by running a ping test to find the closest.

Edit: note the relay isn't responding at the moment at all ...
member
Activity: 114
Merit: 10


When I try to run it, I only get this:

Code:
Using server 8333

Running 14.04 64bit.

Hello!

I have the same issue.  Huh



Prior to update everything worked. Ubuntu Server 64bit 14.04.02
sr. member
Activity: 266
Merit: 250
Hi Matt,

I just tried compiling the latest "dirty hack" release:

Code:
rig@rig:~/RelayNode/c++$ make -f Makefile clean; make -f Makefile
rm -f *.o crypto/*.o crypto/sha256_code_release/*.a crypto/*~ *~ *.exe relaynetworkclient relaynetworkterminator relaynetworkproxy relaynetworkoutbound relaynetworkserver relaynetworktest relaynetworkclient.exe
g++ -I. -g -DFORCE_LE -DNDEBUG -O3 -march=native -mtune=native -flto -std=c++11 -Wall -I/usr/include   -c -o flaggedarrayset.o flaggedarrayset.cpp
g++ -I. -g -DFORCE_LE -DNDEBUG -O3 -march=native -mtune=native -flto -std=c++11 -Wall -I/usr/include   -c -o utils.o utils.cpp
g++ -I. -g -DFORCE_LE -DNDEBUG -O3 -march=native -mtune=native -flto -std=c++11 -Wall -I/usr/include   -c -o relayprocess.o relayprocess.cpp
g++ -I. -g -DFORCE_LE -DNDEBUG -O3 -march=native -mtune=native -flto -std=c++11 -Wall -I/usr/include   -c -o p2pclient.o p2pclient.cpp
p2pclient.cpp: In member function ‘virtual void P2PRelayer::net_process(const std::function&)’:
p2pclient.cpp:103:44: warning: large integer implicitly truncated to unsigned type [-Woverflow]
    if (connected.fetch_and(~CONNECTED_FLAGS) & CONNECTED_FLAG_REQUEST_MEMPOOL)
                                            ^
p2pclient.cpp: In member function ‘void P2PRelayer::request_mempool()’:
p2pclient.cpp:191:13: warning: large integer implicitly truncated to unsigned type [-Woverflow]
   connected &= ~CONNECTED_FLAG_REQUEST_MEMPOOL;
             ^
g++ -I. -g -DFORCE_LE -DNDEBUG -O3 -march=native -mtune=native -flto -std=c++11 -Wall -I/usr/include   -c -o connection.o connection.cpp
g++ -I. -g -DFORCE_LE -DNDEBUG -O3 -march=native -mtune=native -flto -std=c++11 -Wall -I/usr/include   -c -o crypto/sha2.o crypto/sha2.cpp
g++ -I. -g -DFORCE_LE -DNDEBUG -O3 -march=native -mtune=native -flto -std=c++11 -Wall -I/usr/include   -c -o client.o client.cpp
g++ -I. -g -DFORCE_LE -DNDEBUG -O3 -march=native -mtune=native -flto -std=c++11 -Wall -I/usr/include flaggedarrayset.o utils.o relayprocess.o p2pclient.o connection.o crypto/sha2.o client.o -Wl,--no-as-needed -pthread -lresolv -o relaynetworkclient

When I try to run it, I only get this:

Code:
Using server 8333

Running 14.04 64bit.
hero member
Activity: 968
Merit: 515
Hmm seems like the relay is pretty much overloaded and not doing it's job at all today.
One of my nodes is permanently sending tnxs to it, getting disconnected every few minutes, and never getting any blocks from it.
Oh well.
Isn't the relay network just for blocks? If not why isn't it?
Because a block needs all the transactions it contains to be able to verify it.
So the relay client has all those transactions, so when a block shows up it can send the block and all the transaction required to the bitcoind ... which is a local network transfer and thus VERY fast.
But it seems like receiving all tx from all nodes is to much for the relay servers to handle. I don't get why not simply focus on blocks. No one cares if a tx needs a few seconds extra till it gets to the miner (tx relay only through p2p network). Since the relay server doesn't verify the tx in a block there is no advantage in pre verifing the tx (signature cache).
When the relay servers receive a new block it cointains all tx. Or does the server only receives the block header an all tx hashes?

BTW:
Quote
Information about when blocks are received is made available publicly
on the stats page
Where do I find this page?
legendary
Activity: 4634
Merit: 1851
Linux since 1997 RedHat 4
Hmm seems like the relay is pretty much overloaded and not doing it's job at all today.
One of my nodes is permanently sending tnxs to it, getting disconnected every few minutes, and never getting any blocks from it.
Oh well.
Isn't the relay network just for blocks? If not why isn't it?
Because a block needs all the transactions it contains to be able to verify it.
So the relay client has all those transactions, so when a block shows up it can send the block and all the transaction required to the bitcoind ... which is a local network transfer and thus VERY fast.
hero member
Activity: 968
Merit: 515
Hmm seems like the relay is pretty much overloaded and not doing it's job at all today.
One of my nodes is permanently sending tnxs to it, getting disconnected every few minutes, and never getting any blocks from it.
Oh well.
Isn't the relay network just for blocks? If not why isn't it?
Pages:
Jump to: