Topic: Shardcoin - A Blockchain Partitioning / Sharding proposal (Read 3900 times)

Atomic cross-chain trading is solvable for entirely different alt chains, so cross-shard trading is just a private case.

I must admit I do not fully understand Mike Hearns' explanation as presented, I suggest we use the wiki page to develop and clarify the solution as needed.

How do you do it atomically, if miners aren't monitoring all chains?

I suppose it would be possible that some services would actually ask people to use particular shards, but it's not really clear they'd have much advantage from making their users jump through the extra hoop. I suppose it's possible that people would somehow decide that some shards were worth less than others, and that would become a self-fulfilling prophecy; I remember in Central Asia in the mid-90s people had somehow decided that older US dollars were worth less than new ones, so they were.

But to fend that kind of thing off I think you'd have to design it with the goal of making distribution of coins among the shards as arbitrary as possible. If you were starting with Bitcoin, you'd split the chain when it got too big and assign outputs to one or the other at random, rather than doing what people on this thread suggested and having different shards for different purposes.

It's a good question how the coins will behave economics wise. I mean it's possible some shards will have different kinds of activity - perhaps SatoshiDice like services would only work on a few shards, while other will be reserved for different activities. Or perhaps the shards will mostly just behave the same. Perhaps more coins will be lost on some shards than in others, thus increasing the value of the remaining cords from those shards.

There is no way to guarantee the same value for all shards, I think it's an interesting experiment to perform. If I were into launching altcoins I would perhaps do that myself - it's a simple matter to launch 1000 different shards, all equivalent to bitcoin except the starting nonce.

If I've got this right, since addresses are basically just key pairs, you wouldn't need to send your coins to an _address_ on Shard 9971. All addresses would work on all the shards. That reduces the risk that different shards will have different values, because the vendor asks you to pay X coins to address xyz, they wouldn't know which shard the money would be coming in on. If all the coins on all the shards buy the same amount of stuff, they all have the same value.

So if you had 100 coins on Shard 17, you could just send them to a vendor's address on Shard 17. When the vendor wanted to spend them, they would also send those coins on Shard 17. Most of the time you wouldn't need to make atomic transactions across chains or anything difficult like that. Obviously everybody's client would need access to enough data to do SPV on all the shards it had coins on.

The fact that the money was split into different Shards could mostly be hidden from the end-user, in the same way that it currently hides the fact that your coins are made up of different amounts in different outputs. The only time you'd have a problem would be when you wanted to send 100 coins at once, but you had 40 coins on Shard 17 and 60 coins on Shard 9971. In that case you'd have two options:
1) Send two transactions, one on Shard 17 and the other on Shard 9971. The vendor will give you your stuff when they get both. (Again the client could take care of this without bothering you about it.)
2) If you need the whole payment in one transaction for some reason (maybe it's a fancy escrow contract or something), _then_ you have to swap with somebody on one of the other shards.

(2) may not turn out to be too important if (1) works well, but trading between shards wouldn't _necessarily_ have to be a clever atomic p2p thing. It could just be the equivalent of the current mixing operations - you send someone with a good reputation some coins on Shard 9971, and they send you some back on Shard 17. You might also have the client set to do these operations in the background to defragment itself and consolidate your coins among a smaller number of shards. Obviously having lots of people routinely taking part in mixing operations for routine, non-illegal purposes has some anonymity benefits, too...

Clearly there's a bit of inconvenience there compared to the current arrangement, but the advantage is that it would allow Bitcoin (or ShardCoin, if Bitcoin didn't want to do this) to scale unto infinity. Whereas right now everyone seems to think there's a point _somewhere_  where you can't scale because blocks get too big to transmit fast enough, or you run into some other limitation and fees go up to choke off demand. (People disagree about where this point is, and whether the community should decide by fiat or just let the miners figure it out.)

PS Scuse the thread necromancy.

The problem being attacked is how to scale the blockchain to world scale.

If it could be implemented like a stock split somehow, that would be ideal. Stocks split don't devalue the stocks, they just play with the numbers a bit - owner of $10,000 worth of Apple stocks will remain in possession of $10,000 worth of the stocks after a split.

The problem that it solves would be ONLY transaction volume handling. If bitcoin grows much more, miners will have a lot of churn updating blocks to include the new transactions from the last few seconds. We're not too far off from the max size of a block. The way I envision this happening does not include any new coins for the miners of the shard, and so financing it is one of the problems to solve.

-bgc

I think the question is what problem does this really solve? One reason Bitcoin gains value is because it becomes a more and more scarce over time. Opening up new "shards" to mine might *seem* like it gives a new wave of people a chance to get rich in the future, but it's no different then an alt-chain really. If you can send "shard" coins to the main blockchain, it's just another chain hence, coin. One of the fundamental basics of Bitcoin is that the inflation is predetermined and limited. "Shards" just opens the chain to hyperinflation. The idea is contrary to the whole point, IMHO.

 

I think it would be desirable to specialize the shards. I do like the idea that there could be a micropayments shard. The incentive to keep it alive would be that the fees are lower. And the fees would be lower because the design would be more lightweight than the main shard. Perhaps, on the micropayment shard, unspent balances would need to be carried forwards every 30 days so that the micropayment shard remains compact.

And what is the incentive to keep these shards alive? It seems to me that eventually everyone would end up on the same shard, as that makes life much easier, allowing to avoid the extra "exchanges". Without a monetary incentive encouraging people to move to the lesser utilized shards, I don't think this could work on the long term.

If he knows only the Namecoin block header, then at best he can mine empty namecoin blocks.  That might help him get some namecoins, but it won't help the network confirm any new namecoin transactions.  Mining from a block header would be the cryptocurrency equivalent of rubberstamping whatever transactions were already there without making any efforts to confirm their validity, which is dangerous for the network if it becomes popular.  Somebody has actually got to be validating the transactions so new transactions will confirm, and if that somebody is only one or two or a few people and the rest are rubberstamping the work of a few, then you have the equivalent of deepbit.  May as well just start up a mining pool.

Does a Bitcoin node merged-mining Namecoin need to know all Namecoin transactions? I was under the impression that no, he needs only to know the Namecoin block header, and to include a Bitcoin transaction acknowledging it. If that's true, then a miner of a shard doesn't need to know all transactions of other shards to MM them.

Mining a chain requires a comprehensive and conclusive ability to validate transactions on the chain. One cannot mine a chain with just the headers (unless they are getting them from somewhere else e.g. centralized pool mining).   So, to merge mine all shards, one must be tracking them all - negating the benefit of separating them.

Can you explain? I'm not an expert on MM but the way I see it everyone will have all the block headers of all shards, and Merkle branches of special transactions that have to do with MM. If there's a reorg they know about it from the block headers. They don't need to store all transactions of all shards.

Here is one fundamental problem I could think of with respect to shard chains, and that's the possibility of a chain reorg on one shard invalidating transactions that have been spent on another shard.

Assuming shards are merge mined, there are two possibilities: either merged mining all shards is not mandatory (like today), or it is mandatory.

If mining all shards is not mandatory, then a chain reorg is possible, and so would the possibility of cascading invalidations of transactions on other shards.  One would need to track all the shards in order to exclude that as possible.

If mining all shards is mandatory, then the shards aren't offering any real benefit.  Rather, the only benefit is that non-miners can function with no block chain or a subset of the block chain.  But these benefits can already be realized with the simple "meta tree" proposal already being discussed on another thread, without needing to actually move coins between chains; to the extent this is true, the shards would offer no further benefit.

Merged mining.

With my proposal all shards have to recognize each other, so someone can't single-sidedly start a shard.

There can be a direct upgrade path from the current Bitcoin.

If there was partitioning to be done, I think a more sensible way to partition would be based on rough transaction size, rather than geographical location.  Different policies could then be applied to different partitions.

An illustrative example:

Micropayments ought to be partitioned away from transactions that are more like bankwires (funding loans for houses, cars, buying bulk shipments of goods for resale, etc.)

On micropayments you value speed and convenience and low transaction cost over the risk of fraud and double spending.  On bankwires you value irreversibility over confirmation speed and perhaps transaction cost.

Your solution would work.

In fact its probably a much simpler swarm client proposal than mine, there is just a few issues:

You also split the miners hashing power which makes each shard less safe.
Anyone could make a new shard.
Confusion
Low adoption rate.
(You mentioned some of this yourself)


My proposal has the same idea as yours, but nodes trade tx data instead of exchanging coins and block-branches are split instead of the entire chain. That way there is no fork etc..

My belief is that we need both tx-out summaries and sharding in the long term. How we will get these and when (if ever) is the subject of quite a few threads at the moment =)

-bgc

I think shard recommendation is not that useful. The overhead of communicating to a miner far away isn't that large.
I wonder if your idea is feasible, I'm not that versed in the details. Perhaps it could use some more formalization, and I would like some other Bitcoin devs to give feedback. If it works, then it might be the best solution to scalability.