Re: How (and why) to use the Relay Network - page 6.

jonnybravo0311

legendary

Activity: 1344

Merit: 1024

Mine at Jonny's Pool

Quote from: p3yot33at3r on July 21, 2015, 03:49:31 PM

Quote from: Matt Corallo on July 21, 2015, 03:16:24 PM

LOL, I'm blind! That is disconnecting from 127.0.0.1, so, yea, its not able to hold a connection to your bitcoind (because your bitcoind is overloaded). It has little to nothing to do with the relay network.

You & me both! Cheesy

I couldn't understand why it was happening, then I realized that the other merge mined coins were also connecting on port 127.0.0.1 & it hit me when I read about the default rpc being set at 4 - what a plonker I am! Tongue

Still, if nothing else - I've just learned something more & you've tuned the relaynetworkclient even more in your search for a solution....... Wink

Thanks Matt, & sorry to have wasted your valuable time

Edit: I'm still confused as to why it has only recently started causing problems though, it ran fine since the beginning........

Merge mining other coins has no impact on your bitcoind. The impact it has is on your miners... the more coins you are merge mining, the more restarts your miners are subjected to. You aren't connecting to port 127.0.0.1 - that's an IP address. You are connecting to that IP address on multiple different ports (each coin has its own... for example, Bitcoin's RPC port is 8332). What do you have running that could be consuming the RPC connections? That's what you need to investigate. P2Pool is certainly one... anything else?

loshia

legendary

Activity: 1610

Merit: 1000

The real problem is not that your btcd is overloaded. Every btcd will reach max peer count sooner or later. Just make sure that there is always a room for relay client
Can I say that my btcd with 150 peers is overloaded - no of course. I just want it to work that way but I do always have a room for relay client always

p3yot33at3r

sr. member

Activity: 266

Merit: 250

Quote from: Matt Corallo on July 21, 2015, 03:16:24 PM

LOL, I'm blind! That is disconnecting from 127.0.0.1, so, yea, its not able to hold a connection to your bitcoind (because your bitcoind is overloaded). It has little to nothing to do with the relay network.

You & me both! Cheesy

I couldn't understand why it was happening, then I realized that the other merge mined coins were also connecting on port 127.0.0.1 & it hit me when I read about the default rpc being set at 4 - what a plonker I am! Tongue

Still, if nothing else - I've just learned something more & you've tuned the relaynetworkclient even more in your search for a solution....... Wink

Thanks Matt, & sorry to have wasted your valuable time

Edit: I'm still confused as to why it has only recently started causing problems though, it ran fine since the beginning........

loshia

legendary

Activity: 1610

Merit: 1000

Super

Listen to Mat advise ...
You need to hack your Bitcoin or to restart it while relay is running as temporally solution and pray to good it will connect Tongue

Note if relay client disconnects from btcd for some reason other peer will take it's place and same shit will happen again

Matt Corallo

hero member

Activity: 755

Merit: 515

Quote from: p3yot33at3r on July 21, 2015, 06:45:28 AM

Code:

Sent transaction of size 427 to relay server
Received transaction of size 427 from relay server
Sent transaction of size 427 to relay server
public.eu.relay.mattcorallo.com Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
public.eu.relay.mattcorallo.com Disconnect: failed to read message header (Connection reset by peer)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
public.eu.relay.mattcorallo.com Disconnect: failed to read message header (Connection reset by peer)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
Connected to relay node with protocol version sponsor printer
Received transaction of size 426 from relay server
Sent transaction of size 426 to relay server
Received transaction of size 426 from relay server
Sent transaction of size 426 to relay server
Received transaction of size 426 from relay server
Sent transaction of size 426 to relay server
Received transaction of size 426 from relay server
Sent transaction of size 426 to relay server
Received transaction of size 426 from relay server
Sent transaction of size 426 to relay server
Received transaction of size 426 from relay server
Sent transaction of size 426 to relay server
Received transaction of size 426 from relay server
Sent transaction of size 426 to relay server
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
Received transaction of size 426 from relay server
Sent transaction of size 426 to relay server
Received transaction of size 426 from relay server

The amount of Tx sizes of 226/7 & 427/8 flying by at light speed is mind blowing - never seen so many of the same size. However, what ever changes you're making seem to be an improvement, so we appear to be on the right road Wink

Edit2:

I think I've nailed the disconnect problem. I just noticed that the default number of threads to service RPC calls is set at 4 on bitcoind, which, seeing as I'm merge mining 10 other coins plus the relaynetworkclient on my p2pool node - doesn't seem enough. So I increased this value to 10 in the .conf file using:

Code:

rpcthreads=10

...and restarted bitcoind. Since doing this I've not observed a single disconnect - only the constant stream of 226/7 & 427/8 Tx fees flying by at warp speed

I'll keep an eye on it, but hopefully this info will prove useful to other merge miners.

LOL, I'm blind! That is disconnecting from 127.0.0.1, so, yea, its not able to hold a connection to your bitcoind (because your bitcoind is overloaded). It has little to nothing to do with the relay network.

loshia

legendary

Activity: 1610

Merit: 1000

p3yot33at3r,

What is your isp connection?
My impressions are that you are using some kind of DSL with PPPoE or dhcp.
You need a static Routable ip in order to play this game. If you are behind nat your router gw is a private ip or your Routable ip changes too often that will kill your tcp connections for sure

If your ip is public and static you can try kano tcp6 sock hack.

The above is a BULSHIT Forget it

Back to your problem
I do not think that dropped connections have anything in common with bitcoin rpc max threads at all.
Relay client is not using RPC at all. It talks raw to btcd and relay servers
Other thing to check is your BTC max peer count. I had this problem when I reached max peer connections and I have restarted relay client it would never connect.
So one of the hacks to my btcd deamon is to use peer -1 and always to have a room for relay client Wink

I am running btcd 0.11 and relay with kano hack (because it never worked without it-thanks kano) and after block downloading again after mats incident it works just fine no issues

p3yot33at3r

sr. member

Activity: 266

Merit: 250

Quote from: Matt Corallo on July 21, 2015, 04:57:07 AM

Quote from: p3yot33at3r on July 21, 2015, 04:36:52 AM

I can't see any reference to buffer blow-ups in the console, only the disconnects. I'll download wireshark & see what I can see. It just seems weird that these problems have only become apparent since bitcoind v11 - it ran fine previously.

I suspect this has more to do with transaction flooding on the network and new versions of the relay network client than new versions of bitcoin.

PM'd you ping & trace results from my router.

Edit: latest release compiled without issue & running. Still getting disconnects, but far less frequent:

Code:

Sent transaction of size 427 to relay server
Received transaction of size 427 from relay server
Sent transaction of size 427 to relay server
public.eu.relay.mattcorallo.com Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
public.eu.relay.mattcorallo.com Disconnect: failed to read message header (Connection reset by peer)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
public.eu.relay.mattcorallo.com Disconnect: failed to read message header (Connection reset by peer)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
Connected to relay node with protocol version sponsor printer
Received transaction of size 426 from relay server
Sent transaction of size 426 to relay server
Received transaction of size 426 from relay server
Sent transaction of size 426 to relay server
Received transaction of size 426 from relay server
Sent transaction of size 426 to relay server
Received transaction of size 426 from relay server
Sent transaction of size 426 to relay server
Received transaction of size 426 from relay server
Sent transaction of size 426 to relay server
Received transaction of size 426 from relay server
Sent transaction of size 426 to relay server
Received transaction of size 426 from relay server
Sent transaction of size 426 to relay server
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
Received transaction of size 426 from relay server
Sent transaction of size 426 to relay server
Received transaction of size 426 from relay server

The amount of Tx sizes of 226/7 & 427/8 flying by at light speed is mind blowing - never seen so many of the same size. However, what ever changes you're making seem to be an improvement, so we appear to be on the right road Wink

Edit2:

I think I've nailed the disconnect problem. I just noticed that the default number of threads to service RPC calls is set at 4 on bitcoind, which, seeing as I'm merge mining 10 other coins plus the relaynetworkclient on my p2pool node - doesn't seem enough. So I increased this value to 10 in the .conf file using:

Code:

rpcthreads=10

...and restarted bitcoind. Since doing this I've not observed a single disconnect - only the constant stream of 226/7 & 427/8 Tx fees flying by at warp speed

I'll keep an eye on it, but hopefully this info will prove useful to other merge miners.

Matt Corallo

hero member

Activity: 755

Merit: 515

Quote from: p3yot33at3r on July 21, 2015, 04:36:52 AM

I can't see any reference to buffer blow-ups in the console, only the disconnects. I'll download wireshark & see what I can see. It just seems weird that these problems have only become apparent since bitcoind v11 - it ran fine previously.

I suspect this has more to do with transaction flooding on the network and new versions of the relay network client than new versions of bitcoin.

p3yot33at3r

sr. member

Activity: 266

Merit: 250

Quote from: Matt Corallo on July 20, 2015, 07:22:43 PM

Quote from: p3yot33at3r on July 19, 2015, 02:41:07 PM

I'm still getting disconnects I'm afraid, but not constant as before:

Strange, the server logs just shows you disconnecting. Can you run Wireshark? It looks like your outbound bandwidth may be saturated by the uploading of transactions (its quite small), though I would expect your client to print errors about the buffer blowing up.

I can't see any reference to buffer blow-ups in the console, only the disconnects. I'll download wireshark & see what I can see. It just seems weird that these problems have only become apparent since bitcoind v11 - it ran fine previously.

Matt Corallo

hero member

Activity: 755

Merit: 515

Turned on the new node in Beijing provided by f2pool and switched to a more expensive host in Tokyo which has better peering, especially into and out of China.

Matt Corallo

hero member

Activity: 755

Merit: 515

Quote from: p3yot33at3r on July 19, 2015, 02:41:07 PM

I'm still getting disconnects I'm afraid, but not constant as before:

Strange, the server logs just shows you disconnecting. Can you run Wireshark? It looks like your outbound bandwidth may be saturated by the uploading of transactions (its quite small), though I would expect your client to print errors about the buffer blowing up.

kano

legendary

Activity: 4592

Merit: 1851

Linux since 1997 RedHat 4

Quote from: Matt Corallo on July 19, 2015, 02:35:54 PM

Quote from: kano on July 19, 2015, 02:37:17 AM

You wouldn't happen to have an ETA on this?

Yes, it should be resolved by now.

Yep all the numbers looking OK again

Last 9999 messages: Sent: 5752 Received: 4224

p3yot33at3r

sr. member

Activity: 266

Merit: 250

I'm still getting disconnects I'm afraid, but not constant as before:

Code:

127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
Received transaction of size 4027 from relay server
Sent transaction of size 4027 to relay server
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
Received transaction of size 407 from relay server
Sent transaction of size 407 to relay server
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
Received transaction of size 519 from relay server
Sent transaction of size 519 to relay server
Received transaction of size 1057 from relay server
Sent transaction of size 1057 to relay server
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
Received transaction of size 225 from relay server
Sent transaction of size 225 to relay server
Received transaction of size 225 from relay server
Sent transaction of size 225 to relay server
Received transaction of size 226 from relay server
Sent transaction of size 226 to relay server
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
Received transaction of size 226 from relay server
Sent transaction of size 226 to relay server
Received transaction of size 373 from relay server
Sent transaction of size 373 to relay server
Received transaction of size 226 from relay server
Sent transaction of size 226 to relay server
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)
127.0.0.1 Disconnect: failed to read message header (Transport endpoint is not connected)

Matt Corallo

hero member

Activity: 755

Merit: 515

Quote from: kano on July 19, 2015, 02:37:17 AM

You wouldn't happen to have an ETA on this?

Yes, it should be resolved by now.

Peter R

legendary

Activity: 1162

Merit: 1007

Can anyone in this thread help me figure something out?

I'm trying to estimate the block propagation rate using the relay network to make an apples-to-apples comparison with the 17 sec/MB figure from this TradeBlock analysis.

TradeBlock's number is the average time it would take 50% of the nodes that it's watching to receive a solved block over the P2P network.

Between the moment a node on the relay network has a solution to a block, how long until 50% of the entities listening on the relay network have received enough information to verify the block solution?

I've seen these stats but I don't know how to interpret them.

kano

legendary

Activity: 4592

Merit: 1851

Linux since 1997 RedHat 4

Quote from: Matt Corallo on July 17, 2015, 05:10:46 PM

My bitcoind server was killed by the host. The problem has now been resolved, but I'm waiting on it to sync the chain again before transactions will again be relayed within the network. See http://bitcoinrelaynetwork.org/ for latest updates.

You wouldn't happen to have an ETA on this?

I wrote a little script to report the tail 9999 lines of my log every 10s instead of having thousands of lines scrolling by.
It still says 0 txn received.

kano

legendary

Activity: 4592

Merit: 1851

Linux since 1997 RedHat 4

Until he fixes his bitcoind.

If he is re-downloading the blockchain ... that may take a loooooong while.

(The advantage of running mutliple bitcoind without a wallet ...
save config, wipe bad one, rsync good to bad, stop the working one, rsync good to bad again, restore the config, start them both

bicer

newbie

Activity: 23

Merit: 0

Is constant "(out-of-band)" normal?

Code:

Sent transaction of size 815 (out-of-band) to relay server
Sent transaction of size 339 (out-of-band) to relay server
Sent transaction of size 373 (out-of-band) to relay server
Sent transaction of size 226 (out-of-band) to relay server

kano

legendary

Activity: 4592

Merit: 1851

Linux since 1997 RedHat 4

Quote from: Matt Corallo on July 17, 2015, 05:10:46 PM

My bitcoind server was killed by the host. The problem has now been resolved, but I'm waiting on it to sync the chain again before transactions will again be relayed within the network. See http://bitcoinrelaynetwork.org/ for latest updates.

Ah OK - then ignore the 2nd part of my PM

Matt Corallo

hero member

Activity: 755

Merit: 515

My bitcoind server was killed by the host. The problem has now been resolved, but I'm waiting on it to sync the chain again before transactions will again be relayed within the network. See http://bitcoinrelaynetwork.org/ for latest updates.

Topic: Re: How (and why) to use the Relay Network - page 6. (Read 653 times)