Topic: [20 BTC] Multithreaded Keep-alive Implementation in Bitcoind - page 3. (Read 31477 times)

The less ugly optimizations and straight bug fixes should certainly be merge candidates. The hub mode patches and uglier optimizations are not sensible merge candidates.

Obvious merge candidates would be:

1) strRPCUser/strRPCPass variables to avoid extracting them on each RPC call.

2) Native long polling support.

3) ToHex replacing HexStr.

4) Faster Base64 decoder.

5) Leak fix in pthreads version of CreateThread.

Also, some version of the RPC/keepalive fix needs to be merged but it doesn't have to be my version.

Well yes, hub mode probably won't get implemented as its really a hack to get it to work with the current network segmentation, but most, if not all, of the various code optimizations and RPC multithreading/asio/etc absolutely can and will eventually get merged upstream.

I think there's still the getwork skeleton issue to work out, but other than that it's a big improvement.

I'm not sure if the changes to hub mode have made it into the diff yet, as well?

As far as a pull request, it's unlikely it will be mainstreamed due to some of the features/changes are not good for clients.  Though maybe I misunderstood that part?  I'd be happy to see it in the mainstream code.

And then pull request it so it can get a more thorough look-through and get merged upstream.

Is everything stable, yet?  Time to freeze the code?  Release Candidate #1?  No new features planned?  I really want to run this code on my pool and it sounds like you have hit a milestone.

Thanks guys.

3151 XXX      20   0  389m 167m 7556 S    8  1.4  20:04.13 bitcoind

That's about 16 hours of running, so looks good to me.

Yep can confirm fixed here too 

Looks good so far, after running a couple hours.

UPDATE: Looks stable so far, running for 6 hrs.

Compiling and will test today, thank you bunches.

Argh, that's not good. On the bright side, it shouldn't be too terribly hard to find and fix. I'll try to have a fix in the next few hours.

The new 4-diff is the same as the preview diff with a few small fixes and rebased against the latest release version. It's the same as the old 4-diff plus the update patches minus the 'getblock' "precompute skeleton block" patch which was causing problems.


Update: There is no change so simple I can't screw it up somehow!
See if this change fixes it:
--- new/util.h  2011-07-17 09:33:40.826055435 -0700
+++ util.h      2011-07-18 12:49:02.095159533 -0700
@@ -625,9 +625,10 @@ inline pthread_t CreateThread(void(*pfn)
         return (pthread_t)0;
     }
     if (!fWantHandle)
-        return (pthread_t)-1;
-    else
+    {
         pthread_detach(hthread);
+        return (pthread_t)-1;
+    }
     return hthread;
 }


The final CreateThread should look like this:

inline pthread_t CreateThread(void(*pfn)(void*), void* parg, bool fWantHandle=f
{
    pthread_t hthread = 0;
    int ret = pthread_create(&hthread, NULL, (void*(*)(void*))pfn, parg);
    if (ret != 0)
    {
        printf("Error: pthread_create() returned %d\n", ret);
        return (pthread_t)0;
    }
    if (!fWantHandle)
    {
        pthread_detach(hthread);
        return (pthread_t)-1;
    }
    return hthread;
}

PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
26217 XXXX      20   0  9.8g 243m 7664 S    2  2.0  55:09.60 bitcoind

I am seeing similar to Caesium

JoelKatz,

I appear to have some sort of memory/map leak with 0.3.24 + 0.96

bitcoind has grown to the following:

9032M of mapped memory. If I check /proc/24646/smaps, I can see just over  a thousand maps of 8192k but sadly they have no real information, here's a sample of one:


They appear to be increasing at approx one every few minutes.
Any idea where they might be coming from?
(edit: it's not leaking fds by the way, only 123 open descriptors and 108 of those are connections as reported by getinfo)

I had to revert back to the old 3.23 code today. The preview patch + 3.24 locked up the system, used up all available sockets after about 8-10hrs, twice. I had to kill -9 to get the process to end.  Possibly erroring when 2nd block is found.

Any changes v.97 3.24 that might address this?

previewdiff seems to be working fine for me now... just wanted to report back.

I'm a little confused, is the new 4diff update incorporating previewdiff now?  Should I be recompiling at this point?

Yep, very much would be.  In fact it used to absolutely be, its only with the recent block download stuff, plus possibly the IRC split and other factors have made the network very unstable and poorly connected.  My hope is that if people would get off their ass and upgrade to 0.3.24, it would get much better, but as it stands we dont even have a mac build and people are being very, very slow to upgrade.

I see a lot of good ideas in there. But I think people are letting the perfect be the enemy of the good. Nobody will ever agree on the best possible solution and complex solutions have possible complex failure modes. I think a baby step in the right direction might be a good idea.

You might only need 8 hubs/supernodes to handle the network from a technical standpoint, but I think you want vastly more because someone who controls a significant fraction of the network could potentially grab all of at least one client's connections and keep that client in the dark. If we move to concentrate power in a small number of supernodes, we may create new attack possibilities.

There are many other steps we could take specifically to deal with the 'mushroom' attack. For example, well-known DNS names under separate administration could each publish a recent block hash as a TXT record. If you aren't seeing the blocks published by most of those well-known names, you're not caught up to the real network. But ideally, the network would just be dense and diverse enough that this would be impractical.

Yep, that is absolutely something that should be done, and its been tossed around in one form or another quite a few times (typically under the supernode/client or sub-node name).  You get a ton of supernodes which act like nodes do today, except that the client doesn't upgrade itself into a supernode unless it sees that it has good uptime, accept incoming connections, etc similar to the way gtk-gnutella does it.  At that point whether or not hubs need more outgoing connections can be decided.  I have a feeling there will be a small enough number of hubs/supernodes that any more than 8 would be needless, but that depends on any number of factors that can't really be decided now.
See https://forum.bitcoin.org/index.php?topic=12286.0 and http://forum.bitcoin.org/index.php?topic=7972.msg116265#msg116265

Actually, now that I think about it, reducing the outbound connections hub make will weaken the network. My goal was to get a richly interconnected mesh of high-capacity hubs. And I guess I was thinking "we don't need to connect, they'll connect to us eventually". But this is not true -- if neither side is willing to make many outbound connections, then two hubs won't be likely to connect to each other. But if hubs make lots of outbound connections, then hubs will consume the inbound capacity of non-hubs, reducing the diversity of the network.

I've been thinking about a workaround. What I'm thinking is to have a NODE_HUB bit to indicate a large hub (with the same propagation/storage semantics as NODE_NETWORK). That way, hubs will know that nodes that set that bit are also capable of handling large numbers of incoming connections.

Hubs will first follow the normal mechanism for making outbound connections. But then they will preferentially make extra outbound connections strictly to other hubs to which they are not connected to improve network interconnection density and speed network propagation.

Once we attain a decent number hubs, the net effect will be that each hub will have some 32 outbound connections mostly to other hubs, some 32 inbound connections from other hubs, and be able to support about 512 incoming connections from non-hubs.

Oh absolutely not, it is going to solving a legitimate problem.  Hopefully I didn't come across as hostile...
All the work you are doing here is great, really hope we can get the asio/threaded RPC into mainline sometime soon (hopefully for a 4.1)