Re: How (and why) to use the Relay Network - page 8.

amaclin

legendary

Activity: 1260

Merit: 1019

Quote from: zvs on July 13, 2015, 08:57:13 PM

don't get it.. blocks? the relay always sends me transactions after i've already received them, so it's essentially wasting bandwidth

Your Gangnam-nodes are also a relay-network

zvs

legendary

Activity: 1680

Merit: 1000

https://web.archive.org/web/*/nogleg.com

Quote from: Matt Corallo on July 10, 2015, 01:33:57 AM

Quote from: zvs on July 09, 2015, 07:28:02 PM

Oh, yah, I realized that. I figured turning it off would save some bandwidth, at least. Everything is running with the updated code now, though I still wish there was a way to set it to blocks only.. don't need transactions relayed to any of my nodes, really.

THat would entirely defeat the purpose...you cant compress without pre-relaying.

don't get it.. blocks? the relay always sends me transactions after i've already received them, so it's essentially wasting bandwidth

Matt Corallo

hero member

Activity: 755

Merit: 515

Quote from: zvs on July 09, 2015, 07:28:02 PM

Oh, yah, I realized that. I figured turning it off would save some bandwidth, at least. Everything is running with the updated code now, though I still wish there was a way to set it to blocks only.. don't need transactions relayed to any of my nodes, really.

THat would entirely defeat the purpose...you cant compress without pre-relaying.

zvs

legendary

Activity: 1680

Merit: 1000

https://web.archive.org/web/*/nogleg.com

Quote from: Matt Corallo on July 09, 2015, 01:15:19 AM

Quote from: zvs on July 06, 2015, 01:42:12 PM

This seems to be relaying massive amounts of dust transactions, I had over 25000 stored.

Had to turn it off. Now debug.log is getting spammed with

and a good 50% or more of my peers are sending me an inordinate amount of data

This is not specific to the relay network, this was just someone flooding the Bitcoin network with lots of transactions. The relay network, at least, will no longer do this.

Oh, yah, I realized that. I figured turning it off would save some bandwidth, at least. Everything is running with the updated code now, though I still wish there was a way to set it to blocks only.. don't need transactions relayed to any of my nodes, really.

but it still

Code:

----system---- ------sockets------ -net/venet0 ---load-avg---
time |tot tcp udp raw frg| recv send| 1m 5m 15m
09-07 17:24:12| 0 117 7 0 0| 0 0 |0.31 0.30 0.36
09-07 17:24:13| 0 117 7 0 0| 654k 95k|0.29 0.30 0.36
09-07 17:24:14| 0 116 7 0 0| 593k 125k|0.29 0.30 0.36
09-07 17:24:15| 0 116 7 0 0| 495k 83k|0.29 0.30 0.36
09-07 17:24:16| 0 116 7 0 0| 496k 72k|0.29 0.30 0.36
09-07 17:24:17| 0 116 7 0 0| 335k 31k|0.29 0.30 0.36
09-07 17:24:18| 0 116 7 0 0| 402k 32k|0.27 0.29 0.36
09-07 17:24:19| 0 116 7 0 0| 523k 39k|0.27 0.29 0.36
09-07 17:24:20| 0 116 7 0 0| 443k 74k|0.27 0.29 0.36
09-07 17:24:21| 0 116 7 0 0| 474k 138k|0.27 0.29 0.36
09-07 17:24:22| 0 116 7 0 0| 469k 82k|0.27 0.29 0.36
09-07 17:24:23| 0 116 7 0 0| 451k 71k|0.24 0.29 0.35
09-07 17:24:24| 0 115 7 0 0| 533k 98k|0.24 0.29 0.35
09-07 17:24:25| 0 115 7 0 0| 611k 97k|0.24 0.29 0.35
09-07 17:24:26| 0 115 7 0 0| 648k 84k|0.24 0.29 0.35
09-07 17:24:27| 0 115 7 0 0| 652k 149k|0.24 0.29 0.35
09-07 17:24:28| 0 115 7 0 0| 635k 66k|0.30 0.30 0.36
09-07 17:24:29| 0 115 7 0 0| 458k 70k|0.30 0.30 0.36
09-07 17:24:30| 0 115 7 0 0| 514k 124k|0.30 0.30 0.36
09-07 17:24:31| 0 115 7 0 0| 444k 60k|0.30 0.30 0.36
09-07 17:24:32| 0 115 7 0 0| 366k 41k|0.30 0.30 0.36
09-07 17:24:33| 0 115 7 0 0| 375k 51k|0.28 0.29 0.36
09-07 17:24:34| 0 115 7 0 0| 402k 47k|0.28 0.29 0.36
09-07 17:24:35| 0 115 7 0 0| 790k 105k|0.28 0.29 0.36
09-07 17:24:36| 0 115 7 0 0| 502k 72k|0.28 0.29 0.36
09-07 17:24:37| 0 115 7 0 0| 623k 86k|0.28 0.29 0.36
09-07 17:24:38| 0 115 7 0 0| 403k 127k|0.34 0.31 0.36
09-07 17:24:39| 0 115 7 0 0| 595k 56k|0.34 0.31 0.36
09-07 17:24:40| 0 115 7 0 0| 466k 112k|0.34 0.31 0.36
09-07 17:24:41| 0 115 7 0 0| 340k 89k|0.34 0.31 0.36
09-07 17:24:42| 0 115 7 0 0| 655k 71k|0.34 0.31 0.36
09-07 17:24:43| 0 115 7 0 0| 598k 93k|0.31 0.30 0.36
09-07 17:24:44| 0 115 7 0 0| 595k 48k|0.31 0.30 0.36
09-07 17:24:45| 0 115 7 0 0| 465k 95k|0.31 0.30 0.36
09-07 17:24:46| 0 115 7 0 0| 543k 89k|0.31 0.30 0.36
09-07 17:24:47| 0 115 7 0 0| 489k 43k|0.31 0.30 0.36
09-07 17:24:48| 0 115 7 0 0| 492k 50k|0.29 0.29 0.35
09-07 17:24:49| 0 116 7 0 0| 652k 122k|0.29 0.29 0.35
09-07 17:24:50| 0 116 7 0 0| 461k 121k|0.29 0.29 0.35
09-07 17:24:51| 0 116 7 0 0| 601k 57k|0.29 0.29 0.35
09-07 17:24:52| 0 116 7 0 0|1074k 71k|0.29 0.29 0.35
09-07 17:24:53| 0 116 7 0 0|2315k 123k|0.26 0.29 0.35
09-07 17:24:54| 0 116 7 0 0|2372k 213k|0.26 0.29 0.35
09-07 17:24:55| 0 116 7 0 0|2290k 136k|0.26 0.29 0.35
09-07 17:24:56| 0 116 7 0 0|2475k 117k|0.26 0.29 0.35
09-07 17:24:57| 0 116 7 0 0|3616k 174k|0.26 0.29 0.35
09-07 17:24:58| 0 116 7 0 0|3001k 154k|0.24 0.28 0.35
09-07 17:24:59| 0 116 7 0 0|2388k 110k|0.24 0.28 0.35
09-07 17:25:00| 0 116 7 0 0| 392k 110k|0.24 0.28 0.35
09-07 17:25:01| 0 116 7 0 0| 486k 39k|0.24 0.28 0.35
09-07 17:25:02| 0 116 7 0 0|1098k 165k|0.24 0.28 0.35
09-07 17:25:03| 0 116 7 0 0|2137k 239k|0.30 0.30 0.35
09-07 17:25:04| 0 117 7 0 0|3196k 125k|0.30 0.30 0.35
09-07 17:25:05| 0 117 7 0 0|5036k 137k|0.30 0.30 0.35
09-07 17:25:06| 0 117 7 0 0|7243k 120k|0.30 0.30 0.35
09-07 17:25:07| 0 117 7 0 0|5251k 200k|0.30 0.30 0.35
09-07 17:25:08| 0 117 7 0 0|5138k 147k|0.36 0.31 0.36^C

can't be stopped, unless I go through and start firewalling every single one of those nodes. That's on one of my 100mbit connections too, which is why it only has 100 connections. I've seen 50MB/s receive amts on 1Gbit

ed: someone with several thousand $'s to waste could easily get some 10Gbit dedicated lines, and just flood everyone out with trash transactions.

ed2: and i suppose I should point out that this would be different from a typical DoS attack in that they'd have hundreds, or a thousand or two or three helpers, all transmitting as many of those transactions as their bandwidth could handle (turning it into a DDoS, with the quickness!)... ed3 (last one I hope, wtf): I've considered upping the minimum version level of connections... that would get rid of 99%, just the 1% malicious would remain

Matt Corallo

hero member

Activity: 755

Merit: 515

Quote from: p3yot33at3r on July 09, 2015, 06:31:29 AM

Quote from: Matt Corallo on July 08, 2015, 10:19:31 PM

lol, is your libc REALLY broken??? I'm really not sure how thats possible...is this reproduceable?

I'm buggered if I know - it only happened the once too, so not reproducible I'm afraid.

Can you try with connections to the relay network blocked (I'm assuming this happened when you couldn't connect to any servers)?

Quote from: p3yot33at3r on July 09, 2015, 06:31:29 AM

I still get a warning when compiling the latest version, but it does run now. I'm still getting plenty of this:

Code:

127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)

Its having trouble connecting to your local bitcoind it looks like? Anything in your debug.log?

Quote from: p3yot33at3r on July 09, 2015, 06:31:29 AM

Thanks for adding the extra info to the web page Matt. I'm sure the attack situation must be wrecking your head somewhat, especially with noobs like me constantly asking silly questions! Cheesy

Your work is very much appreciated, & I for one will be throwing a tip or two your way as & when I can - thanks again

Meh, its a change I had been meaning to make for a while, this just forced my hand.

p3yot33at3r

sr. member

Activity: 266

Merit: 250

Quote from: Matt Corallo on July 08, 2015, 10:19:31 PM

Quote from: p3yot33at3r on July 08, 2015, 08:41:44 AM

Kano - thank you for that, I wasn't aware that Matt had changed the start up syntax. However, like you say, it's not responding & as soon as I started it up it locked my system up completely requiring a full reset.

Switched it off & disabled it again Sad

lol, is your libc REALLY broken??? I'm really not sure how thats possible...is this reproduceable?

I'm buggered if I know - it only happened the once too, so not reproducible I'm afraid.

I still get a warning when compiling the latest version, but it does run now. I'm still getting plenty of this:

Code:

127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)

Thanks for adding the extra info to the web page Matt. I'm sure the attack situation must be wrecking your head somewhat, especially with noobs like me constantly asking silly questions! Cheesy

Your work is very much appreciated, & I for one will be throwing a tip or two your way as & when I can - thanks again

Matt Corallo

hero member

Activity: 755

Merit: 515

Quote from: zvs on July 06, 2015, 01:42:12 PM

This seems to be relaying massive amounts of dust transactions, I had over 25000 stored.

Had to turn it off. Now debug.log is getting spammed with

and a good 50% or more of my peers are sending me an inordinate amount of data

This is not specific to the relay network, this was just someone flooding the Bitcoin network with lots of transactions. The relay network, at least, will no longer do this.

Matt Corallo

hero member

Activity: 755

Merit: 515

Quote from: kano on July 09, 2015, 12:07:09 AM

Seems I'm back to only sending (on west) and rarely getting txns back.
The rare Received ones are always just after, and the same size as, so most likely the same transactions as, the non "(out-of-band)" that it says I send.

This is largely expected. When you're sending out-of-band it means you're running an unmodified bitcoind (more on that later). The send/receive is the same as before, you're echoing the server's sent txn (the printing of what is happenning is out of order, I fixed that now). However, the server is now sending transactions less often (only when its view of what the next few blocks should be changes, instead of on each new transaction).

kano

legendary

Activity: 4466

Merit: 1798

Linux since 1997 RedHat 4

Seems I'm back to only sending (on west) and rarely getting txns back.
The rare Received ones are always just after, and the same size as, so most likely the same transactions as, the non "(out-of-band)" that it says I send.

Matt Corallo

hero member

Activity: 755

Merit: 515

Quote from: kano on July 08, 2015, 10:39:58 PM

What are these?

Code:

Sent transaction of size 226 (out-of-band) to relay server

out-of-band transactions are transactions sent but not assumed to be in the next block (ie they will not be compressed if they do appear in the next block). More to come on the website soon.

kano

legendary

Activity: 4466

Merit: 1798

Linux since 1997 RedHat 4

Quote from: Matt Corallo on July 08, 2015, 10:22:25 PM

Oh, and I added a donation address to cover the 45USD/month I'm currently shelling out for the servers :p (1NRuqMJAzUGwvFigukLa3UZqcJXix1dETM, feel free to check that address' balance and only donate if the server costs havent already been covered, though if its usually over I'll invest in more servers)

Just updated my main node from the Jul-4 version to now.

All running OK now.

The Jul-4 version regularly disconnects/reconnects and doesn't send block notifications to bitcoind because of that ... so I'd guess update it is necessary.

What are these?

Code:

Sent transaction of size 226 (out-of-band) to relay server

Matt Corallo

hero member

Activity: 755

Merit: 515

Oh, and I added a donation address to cover the 45USD/month I'm currently shelling out for the servers :p (1NRuqMJAzUGwvFigukLa3UZqcJXix1dETM, feel free to check that address' balance and only donate if the server costs havent already been covered, though if its usually over I'll invest in more servers)

Matt Corallo

hero member

Activity: 755

Merit: 515

Sorry for the lack of replies, I've been heads-down rewriting the way transactions are handled to work in cases where mempools are reasonably large. That seems to be working now (though much tweaking remains to be done to make it perform well). There is also a new node in Tokyo (you may want to restart your client to auto-detect if its closer for you, or if you had already been connecting to the jpy hostname, you should already be using it).

I'm gonna try to be better at keeping an updated news section in on http://bitcoinrelaynetwork.org, so keep an eye there as things get updated. There will be new operating instructions due to the changes (though if you ignore them things will still work, and probably just as effeciently unless you're using strange transaction policy).

Quote from: p3yot33at3r on July 08, 2015, 08:41:44 AM

Kano - thank you for that, I wasn't aware that Matt had changed the start up syntax. However, like you say, it's not responding & as soon as I started it up it locked my system up completely requiring a full reset.

Switched it off & disabled it again Sad

lol, is your libc REALLY broken??? I'm really not sure how thats possible...is this reproduceable?

Quote from: -ck on July 06, 2015, 08:07:14 PM

edit: I fixed it by disabling it attempting to build that file since it's not even included in my binary. Probably shouldn't build all the assembly files unconditionally.

I did this on master now.

Quote from: btcash

Information about when blocks are received is made available publicly
on the stats page
Where do I find this page?

http://bitcoinrelaynetwork.org/stats.html (though that will probably be wiped in not too long as my stats-processing script doesnt handle adding/removing nodes very well :/, if anyone needs old stats I will be glad to provide them).

p3yot33at3r

sr. member

Activity: 266

Merit: 250

Quote from: kano on July 08, 2015, 08:21:27 AM

He changed the parameter order ...

You now should just:
./relaynetworkclient 127.0.0.1 8333

You can specify which server to use on the end, but it will automatically choose one if you don't, by running a ping test to find the closest.

Edit: note the relay isn't responding at the moment at all ...

Kano - thank you for that, I wasn't aware that Matt had changed the start up syntax. However, like you say, it's not responding & as soon as I started it up it locked my system up completely requiring a full reset.

Switched it off & disabled it again Sad

kano

legendary

Activity: 4466

Merit: 1798

Linux since 1997 RedHat 4

Quote from: igorwhite on July 08, 2015, 05:58:12 AM

Quote from: p3yot33at3r on July 08, 2015, 05:28:42 AM

When I try to run it, I only get this:

Code:

Using server 8333

Running 14.04 64bit.

Hello!

I have the same issue. Huh

http://s30.postimg.org/43o0fs8w1/relay.png

Prior to update everything worked. Ubuntu Server 64bit 14.04.02

He changed the parameter order ...

You now should just:
./relaynetworkclient 127.0.0.1 8333

You can specify which server to use on the end, but it will automatically choose one if you don't, by running a ping test to find the closest.

Edit: note the relay isn't responding at the moment at all ...

igorwhite

member

Activity: 114

Merit: 10

Quote from: p3yot33at3r on July 08, 2015, 05:28:42 AM

When I try to run it, I only get this:

Code:

Using server 8333

Running 14.04 64bit.

Hello!

I have the same issue. Huh

Prior to update everything worked. Ubuntu Server 64bit 14.04.02

p3yot33at3r

sr. member

Activity: 266

Merit: 250

Hi Matt,

I just tried compiling the latest "dirty hack" release:

Code:

rig@rig:~/RelayNode/c++$ make -f Makefile clean; make -f Makefile
rm -f *.o crypto/*.o crypto/sha256_code_release/*.a crypto/*~ *~ *.exe relaynetworkclient relaynetworkterminator relaynetworkproxy relaynetworkoutbound relaynetworkserver relaynetworktest relaynetworkclient.exe
g++ -I. -g -DFORCE_LE -DNDEBUG -O3 -march=native -mtune=native -flto -std=c++11 -Wall -I/usr/include -c -o flaggedarrayset.o flaggedarrayset.cpp
g++ -I. -g -DFORCE_LE -DNDEBUG -O3 -march=native -mtune=native -flto -std=c++11 -Wall -I/usr/include -c -o utils.o utils.cpp
g++ -I. -g -DFORCE_LE -DNDEBUG -O3 -march=native -mtune=native -flto -std=c++11 -Wall -I/usr/include -c -o relayprocess.o relayprocess.cpp
g++ -I. -g -DFORCE_LE -DNDEBUG -O3 -march=native -mtune=native -flto -std=c++11 -Wall -I/usr/include -c -o p2pclient.o p2pclient.cpp
p2pclient.cpp: In member function ‘virtual void P2PRelayer::net_process(const std::function&)’:
p2pclient.cpp:103:44: warning: large integer implicitly truncated to unsigned type [-Woverflow]
if (connected.fetch_and(~CONNECTED_FLAGS) & CONNECTED_FLAG_REQUEST_MEMPOOL)
^
p2pclient.cpp: In member function ‘void P2PRelayer::request_mempool()’:
p2pclient.cpp:191:13: warning: large integer implicitly truncated to unsigned type [-Woverflow]
connected &= ~CONNECTED_FLAG_REQUEST_MEMPOOL;
^
g++ -I. -g -DFORCE_LE -DNDEBUG -O3 -march=native -mtune=native -flto -std=c++11 -Wall -I/usr/include -c -o connection.o connection.cpp
g++ -I. -g -DFORCE_LE -DNDEBUG -O3 -march=native -mtune=native -flto -std=c++11 -Wall -I/usr/include -c -o crypto/sha2.o crypto/sha2.cpp
g++ -I. -g -DFORCE_LE -DNDEBUG -O3 -march=native -mtune=native -flto -std=c++11 -Wall -I/usr/include -c -o client.o client.cpp
g++ -I. -g -DFORCE_LE -DNDEBUG -O3 -march=native -mtune=native -flto -std=c++11 -Wall -I/usr/include flaggedarrayset.o utils.o relayprocess.o p2pclient.o connection.o crypto/sha2.o client.o -Wl,--no-as-needed -pthread -lresolv -o relaynetworkclient

When I try to run it, I only get this:

Code:

Using server 8333

Running 14.04 64bit.

btcash

hero member

Activity: 968

Merit: 515

Quote from: kano on July 07, 2015, 07:24:33 AM

Quote from: btcash on July 07, 2015, 07:20:07 AM

Quote from: kano on July 06, 2015, 10:23:51 PM

Hmm seems like the relay is pretty much overloaded and not doing it's job at all today.
One of my nodes is permanently sending tnxs to it, getting disconnected every few minutes, and never getting any blocks from it.
Oh well.

Isn't the relay network just for blocks? If not why isn't it?

Because a block needs all the transactions it contains to be able to verify it.
So the relay client has all those transactions, so when a block shows up it can send the block and all the transaction required to the bitcoind ... which is a local network transfer and thus VERY fast.

But it seems like receiving all tx from all nodes is to much for the relay servers to handle. I don't get why not simply focus on blocks. No one cares if a tx needs a few seconds extra till it gets to the miner (tx relay only through p2p network). Since the relay server doesn't verify the tx in a block there is no advantage in pre verifing the tx (signature cache).
When the relay servers receive a new block it cointains all tx. Or does the server only receives the block header an all tx hashes?

BTW:

Quote

Information about when blocks are received is made available publicly
on the stats page

Where do I find this page?

kano

legendary

Activity: 4466

Merit: 1798

Linux since 1997 RedHat 4

Quote from: btcash on July 07, 2015, 07:20:07 AM

Quote from: kano on July 06, 2015, 10:23:51 PM

Hmm seems like the relay is pretty much overloaded and not doing it's job at all today.
One of my nodes is permanently sending tnxs to it, getting disconnected every few minutes, and never getting any blocks from it.
Oh well.

Isn't the relay network just for blocks? If not why isn't it?

Because a block needs all the transactions it contains to be able to verify it.
So the relay client has all those transactions, so when a block shows up it can send the block and all the transaction required to the bitcoind ... which is a local network transfer and thus VERY fast.

btcash

hero member

Activity: 968

Merit: 515

Quote from: kano on July 06, 2015, 10:23:51 PM

Hmm seems like the relay is pretty much overloaded and not doing it's job at all today.
One of my nodes is permanently sending tnxs to it, getting disconnected every few minutes, and never getting any blocks from it.
Oh well.

Isn't the relay network just for blocks? If not why isn't it?

Topic: Re: How (and why) to use the Relay Network - page 8. (Read 609 times)