Topic: Microsoft Researchers Suggest Method to Improve Bitcoin Transaction Propagation - page 2. (Read 17557 times)

Aren't people correct by pointing out that other p2p networks work just fine? And bitcoin works fine at the moment.

Many users will never download the software or will only turn it on when they want to send or receive funds.  I think many people fail to understand the network, computational, and storage requirement of being an active node.  Not at this token <1 transaction per second but at any real sustained transaction volume.

When relaying starting requiring a couple hundred dollars a year in electricity, consumes terrabytes of disk space, produces more heat than a spaceheater, and results in your ISP throttling you due to 'excessive' bandwidth users will learn.  If they don't learn except to see more usage of 'lite wallets' and 'ewallets' and the number of nodes continue to shrink.

Many users that download the software wont know how to disable relaying or care. Not a problem.

There may be alterior motives to leave clients connected. Clients can be created that serve as postage, notary, courier tracking, etc. using Bitcoin for other functions besides simple transactions.

I'm pretty sure this is the way things will go.  The cost of transactions will rise to a point where there is an incentive to minimize the volume of block chain transactions.  I don't think using account based solutions is the best way however.  Instead, I think privately issued coins that have most of the properties of bitcoin, but don't require mining, is the way to go (transactions can just be verified with the issuer(s) rather than replicating a block chain everywhere).  This is similar to the role that gold serves today.  Using gold directly for transactions is cost prohibitive and thus people use various paper instruments for transactions while maintaining gold as reserves.  Debt (as well as equity and various forms of paper) are also a necessary complement to bitcoin to allow the effective monetary supply to expand & contract with market demand (which will also serve to help stabilize the value of bitcoin).

Miners are providing a service: they're memorializing transactions in the block chain, which hardens them against double spending.  They don't have to do it.  If they refuse to, the transactions will sit unconfirmed.


I believe the scenario of a "few" nodes controlling all the flow of information is highly unlikely to happen.  The vast majority of nodes aren't mining and have absolutely no incentive to withhold transactions.  They are just average joe schmoes running the Bitcoin client.  Each one of those links up to at least 8 peers.  So a swarm of 100,000 nodes could, for simplicity's sake, have 400,000 connections between them (100,000 * 8/2).  Fewer than 5% of those nodes are likely to be mining and would see any benefit to refusing to relay valid transactions.  The other 95% have no reason to run what amounts to a crippled client that contributes to network disruption with no benefit to them.  What's the odds that all of any given node's 8+ connections are going to coincidentally go to people who see an advantage to withholding transactions, and are doing it?  Pretty small, in my opinion.

Let's assume that in the future, running the client becomes infeasible and only miners run clients.  So, 100% of the nodes are miners and are refusing to relay transactions.  I, as someone who wants to execute a transaction, could easily send my transaction to each of them myself.  Just the same way your bank separately reports your history to Experian, Equifax, and TransUnion separately.  Given the a transaction size of 0.5 KB, it would be painless for my computer to submit that to a hundred different nodes if necessary.  They could go in UDP packets and be blasted out in the blind.


I don't believe that any of the developers would take this idea seriously because it comes with a large resource burden to solve a problem nobody (other than you guys, of course) believe exists.  If for every big transaction, we need to add (and relay) 9 little transactions just to make sure the miners make a penny, we have suddenly increased the resource cost tenfold, and imposed that increased cost upon every node.  It would be ridiculous and self-destructive.  Each transaction has the same resource cost on the network, whether big or small.  Miners don't want to collectively consume 4.5 KB of disk space - per miner - to make sure that a few of them collect a penny for the service of storing and forwarding a 0.5 KB transaction.


In my view, this is the elephant in the room - in comparison, the "miners won't want to relay" problem is like the distant comet that might or might not hit us.

Presently, all nodes must store and forward all transactions.  If twenty people start their own Bitcoin blockchain and sell each other some snacks, there will never be a problem.

But if scaled to the US size of Visa in its current form, where you could buy sodapop and candybars from vending machines with Bitcoin, every vending machine in the US would need to hear about any time any other vending machine anywhere in the US sold a snack, and would need a gadzookabyte hard drive to store it all.  This would obviously be unsustainable, since I don't know how many snacks are sold every second, but if I needed to download 0.5 KB over my internet connection each time anyone anywhere bought a snack, my internet connection would be useless.  The block chain as we know it now is absolutely unusable for wide scale micropayments - it's only usable that way today because there's plenty of as-yet unused room.  A four-lane highway with a couple cars per mile, if you will.

I refer to this as exponential growth by considering the total number of terabytes of disk space consumed and the total amount of bandwidth consumed in the aggregate passing all these transactions across all nodes, taking into account a vague assumption that transaction volume is correlated with the number of nodes on the network.  That, and the more people that enter the world of Bitcoin and the more potential counterparties they can pay with Bitcoin, the more transactions we will see per participant.

Scaled to Visa proportions, Bitcoin and its blockchain would be useful only the same way Fedwire is useful now - as the backbone that moves large amounts of funds between banks, and which is prohibitive for all but the largest everyday transactions.  The market (miners, that is) would price the transaction fee such that it would be senseless to add entries to the blockchain to buy sodapop - sort of like paying $20 to send a wire for a $1 soda.

On the other hand, everyday transactions with Bitcoin would be possible by using smaller private "banks" that internally accounted for all of the transactions amongst each other, and used Bitcoin as backing.  These wouldn't be cryptographic transactions, they would just be the same sort of internal transactions like me using PayPal to send part of my PayPal balance to you.  Presumably, there could be many such online payment services, with a standardized agreement to push small payments across one another's network and settle up later (the same way competing telcom carriers and long-distance companies handoff phone calls to one another).  The blockchain would only become involved when these internal banks needed to settle their aggregated obligations to one another, or in the event of large transactions where it makes sense for the transactor to cover the fee, just like a bankwire today.

I don't believe Bitcoin would need to get to Visa proportions to suddenly encounter its practical transaction volume limits.  If it suddenly became the preferred way to pay for online gambling, or to buy services to get around a national firewall, or access to adult material, it could find itself bumping up against that limitation rather quickly.

Of course, by that time, your average cell phone might have 200 TB of storage, capable of 20Gbps data communications, powered by nuclear fusion and cost less per month than your average fast food meal (but you'll still have to pay extra for text messaging).

That's true of the default client, but not true of all nodes and is a perfectly valid thing to do (and there are a number of cases I can think of where you'd want/need to do this).  I think overall what I would say is that the cost of relaying a transaction is tiny (I expect the vast majority of the time you're just passing along a small INV message… but I don't have stats to back that up) and all things being equal, staying in sync with the network and having accurate intelligence about what is happening far outweighs the small cost of relaying (especially for certain types of nodes).  But, even if you don't find that to be compelling, the small cost of relaying a transaction is tiny in comparison with the benefit it has for the network and by extension the value of bitcoin itself.  I still don't think there is an incentive problem here…even if all miners never forwarded fee bearing transactions, I don't think we'd have a problem.  

It may even be a simpler method of creating healthy market for transaction processing than the rube goldberg machinery that Microsoft proposes.  You could for example submit fee bearing transactions directly to the mining pools (who don't forward them)…you could create several different versions of a transaction, each with a different fee depending on what different pools are charging…the one that wins collects the fee and the other transactions are discarded (because they are conflicting).  You could also submit a zero fee version to the network at large..the pools would have to beat the rest of the network at large to collect their fee.

At today transaction volume levels sure it is a negligible cost however to validate and relay transaction at something like Paypal level volume would require significant resources. 

Paypal has peak daily processing of about 100 tps.  VISA is roughly 40x as large so just keep that in mind.

Currently 1 Bitcoin transaction is ~0.5KB but with rise of contracts one would expect transactions to become more complex.  Say in future the average transaction is 2KB.  100 tps * 2KB = 200KB/s or 1600 kps or 1.6Mbps to a single peer.  To maintain connections to just 20 peers would require at peak 32Mbps.  To maintain connections to 100 peers would require 160Mbps.

A modern quad-core CPU can validate about 80 transactions per second however the node needs to also re-verify blocks so it potentially verifies each transaction twice making throughput 40 tps    100+ transactions per second would thus require multiple CPUs (not multiple cores either server grade quad CPU systems or multiple systems).  VISA scale transaction volume would require either an entire datacenter of servers or some heavy GPU acceleration.  Imagine mining 24/7/365 with all the heat, noise and electrical cost that involves (plus 1 million fold increase in bandwidth requirements) except you would be just taking a 100% loss on that because it is uncompensated and only miners get compensated for putting transactions in blocks.

At Paypal level transaction volume a block would be about 120MB.  At VISA scale transaction volume a block would be 4.8GB.  One years worth of block chain would require 6TB and 252TB of storage respectively.

So yeah setting up a node for the common good when tps is <1 is one thing.  Doing it without compensation where it requires real resources is another.  Especially under such a scenario miners would be well compensated for their work, work that would be impossible without the compensation free transaction routing.

Before I get flamed: this isn't to indicate I think Bitcoin is dead or can't scale however putting 100% of compensation on one component of the network and making the rest of it free simply means people will be dedicated to pursuing the portion that is compensated.  It is an unsustainable dynamic.  Luckily we have plenty of time to consider the issue as it is likely decades before Bitcoin even reaches Paypal level of acceptance.

I don't see it either…the overall amount of work being performed for a given transactions increases as the number of nodes increases, but it's not exponential.  The mesh topology of node inter-connections means that there is an exponential increase in a subset of the communications protocol, however INV messages are quite small.  The work the network performs as a whole to validate a transaction is also not exponential as the number of transactions increases (given a constant number of nodes in the network).

Why isn't the same argument of "relying on altruists" applied to mining as well? We can take your line of thinking and ask oruselves "Why pay miners"? After all, a small bunch of people interested in the currency will contribute all the resources one needs to verify transactions.

The problem with this argument is that if transaction authentication is left in the hands of a few, they can exploit the system in order to make money. The same applies to transaction propagation: If a few nodes control the flow of information to the entire miner network they can exploit the system for their own benefits.

Bitcoin's great leap is that it incentivizes many miners that keep tabs on each other. We simply argue that it should similarly incentivize a large diversity of transaction distribution channels. Just like anyone can make money by mining, we think that everyone should be able to make money from transaction propagation. This inherently breaks any possible monopoly.

It's true that the solution we offer is not without costs. Most of the increase in block sizes is due to the fact that we esentially "micro-pay" each node for each successful transaction relay, and each payment is saved individually in the blocks. In contrast to that, mining is easier to reward -- only one miner per block. I suspect that it's not impossible to pay for several relaying actions at once as well, and that the associated increase in block size can be reduced (perhaps only reward nodes for relaying in some probabilistic manner using a form of lottery and then there will be less need to remember entire referal chains).


I don't see where there is an exponential increase in resource consumption from a linear increase in volume. Could you elaborate?

I see this as a non-starter, because the resources (disk, bandwidth) required to record all of this splitting of the fee will actually be multiple times more than the original transaction, a resource cost imposed on all nodes on the entire network.  Because instead of recording one entry in the block chain, now you're recording five or ten.

What would be a realistic, practical, nobrainer solution that I am sure will quickly be thought of if this ever becomes a problem, is a few honest nodes with a lot of bandwidth and an interest in the success of Bitcoin as a whole, will publish their IP's and say "Connect to me".  You as a client can connect to as many nodes as you want, but you only really need one well-connected node to talk to the majority of the network, and then...zzzz....business as usual.

I am enthusiastic enough about Bitcoin to be willing to run one, and I have the resources to run one myself, and I am just one guy with far less than 1% of the total bitcoins out there.  It would be drop dead simple too: 1) set up a machine, 2) run bitcoin, 3) publish the IP.  Surely I can't be the only person out there who would be so "altruistic" as to do this.

If you really want to write a paper on a real weakness of Bitcoin as currently implemented, what you really should pursue is the fact that bitcoin's resource consumption increases exponentially as transaction volume increases linearly, and that it's likely to see a scenario where it bursts in popularity for some reason and then suddenly becomes almost completely unusable (Gnutella style) because it is being crushed under its own weight.  Its only means of survival would be to require high fees for transactions so all but the largest transactions would become prohibitive (the same way that fed wires are great for buying a house, but not soda pop), and then become the backbone for a bunch of smaller private payment systems that merely use Bitcoin as backing and as a method to settle aggregated transactions between one another.  This could really happen, and few would disagree.

Hi everyone,

I'm one of the authors of the paper being discussed here.

Following suggestions from some people here in the forum and elsewhere, and in an attempt to clarify some of the points we make in the paper we've posted a summary of the paper aimed at the Bitcoin community. We hope this helps understand our paper a little better, and that this balances some of the more sensational headlines that were out there regarding our work.

http://research.microsoft.com/en-us/people/avivz/bitcoin_red_balloons_summary.aspx

Development takes time.  I've been working on a project part time for about 9 months that will accept bitcoin payments from day one.  I've switched to full time on it but there's still a few months to go before we'll be ready for launch.  Most people who know about bitcoin heard about it more recently than me, so I can only assume they are a little further behind unless they already were working on a project they could easily add bitcoin to.

Agree with you.

Yes, I agree I did seem to be making it appear as that was my understanding. I think I have a bit more refined notion of mining, but probably not by much.

My point was to differentiate mining and transaction processing kinds of activities, even though these are intertwined. So I attempt to draw a bright line, where there really is none.

The reason for trying to even make this distinction is that my impression is that there are (or were) a lot of people interested in getting into mining, while transaction processing has a lot fewer players. Mining is much simpler, requiring less sophistication and design than improving transaction processing. So we see a lot more effort in mining, and less in convenience and acceptance. If there were even a tenth as much progress in building transaction infrastructure as there has been in mining, bitcoins would be more commonly used than they are now.

There are people at Microsoft who are paid full time to do research. Why didn't they discover Bitcoin? There was 7 years of opportunity and the bitgold proposal was released in 2005. That would have been the greatest in-game currency system for MS who is one of the biggest video game companies on the planet.

Their research is being painted as 'discovering a fatal flaw' instead of the reality of 'a bad suggestion to try and improve the bitcoin protocol'. I posted a back of the envelope idea that IMHO would work better than their proposal if bitcoin were to encounter the problem they suggest will happen. I suggest the folks at MS don't understand P2P.

Let me ask a related question, what is the incentive for nodes to broadcast blocks that do not directly include a BTC transfer to an address they posess?

Specifically, there are the incentives to have a P2P currency. The incentive to keep the value of coins you possess. The incentive to know the truth and influence the truth and remotely store the truth on the network. The incentive to not be discovered to be behaving unfairly. These incentives are weighed with the incentive to hoard a transaction.

Perhaps they can go back to the drawing board and work out a math model designed for a graph that matches the graph of the bitcoin network.

Regardless of the intention of the authors of the paper, the premise of the paper is being used as FUD.

My statement is simply that you do not need to relay a transaction not involving you. This was in response to Steve's point that you need to know the "future ledger" in case of it concerning you later, and my responce was that you can worry about it once it is in a block.

No, I'm saying there is no need to know the output until it is included in a block.

No, I get what he's saying now. If you're looking to use the output from a transaction before that transaction is included in a block, you can't do that with the mainline client; therefore, the mainline client has no incentive to relay non-confirmed transactions.

But if you're using the mainline client already, you're relaying. You can modify it to not relay, but you could just as easily modify it to originate transactions with unconfirmed outputs as their inputs.

lol, didn't really want to post but this is to funny to resist... q: how would you relay a sole transaction when it's part of a block ?