Pages:
Author

Topic: [Whitepaper] The Decrits Consensus Algorithm (Read 2403 times)

Ix
full member
Activity: 218
Merit: 128
November 15, 2014, 10:24:16 PM
#24
I don't know enough about bitcoin's memory cache or the python or pypy implementation's memory cache to comment. It seems silly to be hitting the disk for any reason for transactions that are not very old. Even with bitcoin's bloated transactions, 300 bytes/sec is pitifully small and anyone should be able to keep several hours' worth in memory without an issue.
legendary
Activity: 1008
Merit: 1001
Let the chips fall where they may.
November 15, 2014, 04:48:29 PM
#23
Quote
My (admittedly slow) node spends about 700ms processing generating new Bitcoin blocks, plus another 1.7 seconds processing new P2Pool blocks (which are every 30 seconds). This is with a SSD, BTW.

I assume most of that is verifying transactions? If the two peers have seen all the transactions in a record, there is no need to verify anything but the record's signature. I'm not going to delve too deeply into how my memory cache works, but the transaction order is canonical and once the peer has received a record packet from another peer and converted it into something the local cache can use, it is looking for mostly contiguous chunks of memory which is extremely fast. And it doesn't need to search - it is getting a direct pointer to the location of the transaction.

The 700ms is the time required to build a (500kB) block from already verified transactions. The 1.7s was an estimate based on be expected Dead-on-arrival rate vs. my actual DOA rate. It may include disk/network/hasher latency, but I don't really know.

I am assuming the large spikes in the pypy side of the graph are caused by disk cache misses. The disk is a SSD, so a lag of an extra second would probably have to correspond to over 500 seeks (assuming 2ms each). P2Pool being run by pypy appears to use about 3x the memory as P2Pool run by python (1200 vs 400 MB). My memory usage graph is blank for some reason.
Ix
full member
Activity: 218
Merit: 128
November 15, 2014, 02:11:54 PM
#22
Making the records more frequent does not at all reduce the latency that is the product of: the laws of physics and processing delays.

It doesn't reduce the overall latency, but it certainly does divide it up into smaller amounts. Tongue That is all that matters for getting confirmations. And as I said, the bandwidth factor is significantly reduced so the latency of transmitting the data is all during the original transaction, not during the record transmission.

Edit: Given the aggressive block record times, how are accidental forks and orphans (children of potential forks) handled?

There is no competition - one voice controls one block of time (unless there are so many that multiple control each block of time, then they use a modulo function to determine which transactions they include). Duplicated transactions in subsequent records that did not acknowledge the earlier other are just ignored, and they do not pose any real penalty on data transmission or verification time. Double spent transactions are resolved by ceding to the record that controlled an earlier block of time - most of the time, this is covered in a little bit of detail in the paper. Unfortunately for the sake of SPV nodes this might require a "reversed transactions" log which still won't require much resources. But maybe not, as if they are worried enough to see their balance update, they will notice that it did not update anyway.
legendary
Activity: 1008
Merit: 1001
Let the chips fall where they may.
November 15, 2014, 12:48:47 PM
#21
Records are every 10 seconds, so that would cut latency by 1/60th, but the actual data needed to be sent is only a few bytes regardless of the number of transactions (among well-connected peers). And the memory cache is organized in a way so that it is slightly more difficult to receive the data, but once done it is very easy to convert it into a new packet for each connected peer. It's pretty moot though because in testing 100k tx takes milliseconds to process. 5 second confirmations are very much a possibility, although typical will be more on the range of 10-12 seconds.

Making the records more frequent does not at all reduce the latency that is the product of: the laws of physics and processing delays.

My (admittedly slow) node spends about 700ms processing generating new Bitcoin blocks, plus another 1.7 seconds processing new P2Pool blocks (which are every 30 seconds). This is with a SSD, BTW.

Edit: Given the aggressive block record times, how are accidental forks and orphans (children of potential forks) handled?
Ix
full member
Activity: 218
Merit: 128
November 15, 2014, 10:02:20 AM
#20
As I read it:

First paragraph entails that the only voices on the network are the attackers, therefore the second paragraph is redundant since no other voices have been used to secure it.

I probably did not provide enough exposition here about network history attacks/"nothing at stake" that is a popular theoretical attack against PoS. The question is given a node new to the network, how does it decide which history is correct in the face of two or more competing histories, especially when it is relatively trivial to create them without proof of work. The answer is that honest voices could not have signed out on the dishonest network because their signatures would not be valid in that history. If the attacker controls 99% of the voices at any point in history, a new node can still see that one network has 100% of the signatures available (assuming the attackers did not sacrifice their money on the legitimate network to make the attack look stronger), whereas the other only has 99%.

Quote
Additionally, malicious nodes could simply reject other peoples tenure ending transactions therefore locking their funds in a denial attack.

Is this correct?

Yes, they can, just as they can reject an honest voice's legitimate record which accomplishes the same thing - the tenure tx is actually part of the record data. However, each record has at minimum 2 days to be accepted by the network (20% of the voices), so the attackers must control at least that amount and every single one of those records to avoid an unresolvable fork. If there is an honest record anywhere in between, it will accept that transaction and the attackers must reject that record as well to continue the attack - they are building a fork.

This will be heavily expanded on in the whitepaper on the network protocol. Not only will there be voices' money at stake, but there will also be the stakes of elevated network nodes risking their money on the fork they believe to be true (or not risking their money on a fork they believe is malicious) so that the current status of the network is visible even when the attacker controls an overwhelming amount of voices and has not created an outright fork.
sr. member
Activity: 294
Merit: 250
Bitmark Developer
November 15, 2014, 02:15:49 AM
#19
Quote
The primary attack vector on the VL is to create an alternate network
history in which the Voices under the attacker’s control become the only Voices
securing the network. This attack is fairly simple to execute and could even
rewrite the entire history of the network if the attackers control any of the
Voices in the genesis CB.

However, a key concept of the VL is that when a Voice chooses to end his
tenure, he will sign a transaction stating so, and this transaction is added to the
VL. This transaction is the key that unlocks the Decrits that were originally
risked to protect the network. Without it, the Voice does not retrieve his
money. Any rewriting of history cannot possibly include these transactions for
Voices which the attacker does not control.

As I read it:

First paragraph entails that the only voices on the network are the attackers, therefore the second paragraph is redundant since no other voices have been used to secure it.

Additionally, malicious nodes could simply reject other peoples tenure ending transactions therefore locking their funds in a denial attack.

Is this correct?
Ix
full member
Activity: 218
Merit: 128
November 15, 2014, 12:32:08 AM
#18
I recently stumbled across one site offering such statistics

While recent blocks tend to propagate faster than older ones, the latency still a appears to range from 4-15 seconds. Older blocks from about a year ago took 5-120 seconds to propagate.

My personal Bitcoin node is set to make 500kB blocks (half the current maximum size). At 5Mbps of upload bandwidth (mid-range to high for VDSL and cable), it will take at least 1 second to send a completed block to a peer. ~10 seconds to send it to 10 peers.

Records are every 10 seconds, so that would cut latency by 1/60th, but the actual data needed to be sent is only a few bytes regardless of the number of transactions (among well-connected peers). And the memory cache is organized in a way so that it is slightly more difficult to receive the data, but once done it is very easy to convert it into a new packet for each connected peer. It's pretty moot though because in testing 100k tx takes milliseconds to process. 5 second confirmations are very much a possibility, although typical will be more on the range of 10-12 seconds.
legendary
Activity: 1008
Merit: 1001
Let the chips fall where they may.
November 14, 2014, 11:38:00 PM
#17

Quote
you can hardly propagate a tx in 5 secs much less "confirm" -propagate a tx and mint and propagate a valid block ("record").

5 seconds is certainly the best case scenario as it considers that a transaction has propagated the network in 2.5 seconds (a reasonable assumption which can be tested against Bitcoin), arrives at the appropriate voice just before creating the next record, and the record is issued and propagates the network in another 2.5 seconds. The network protocol already has O(1) record propagation, in code. Among well connected peers, records will propagate faster than transactions because of the priority system in place.

I recently stumbled across one site offering such statistics

While recent blocks tend to propagate faster than older ones, the latency still a appears to range from 4-15 seconds. Older blocks from about a year ago took 5-120 seconds to propagate.

My personal Bitcoin node is set to make 500kB blocks (half the current maximum size). At 5Mbps of upload bandwidth (mid-range to high for VDSL and cable), it will take at least 1 second to send a completed block to a peer. ~10 seconds to send it to 10 peers.
sr. member
Activity: 248
Merit: 250
November 07, 2014, 10:18:10 AM
#16
have a ipo?
Ix
full member
Activity: 218
Merit: 128
November 05, 2014, 10:20:11 PM
#15
Basically the same stuff refined. The most difficult parts of the network code are coded and working, although some of it still needs to be cleaned up a bit. Go is actually a really fun language to program in, it's more similar to the scripting languages I used to hack as a kid than to C++. I have one more relatively big section of decrits-specific network stuff to get coded, then the RPC interface along with a cheap web GUI and dummy database, and then some open alpha testing will be useful. Maybe about 2 months away? I have some vacation time coming up so it will be interesting to see how much I can do in that time. Tongue
legendary
Activity: 1050
Merit: 1016
November 05, 2014, 06:37:42 PM
#14
Plan to check out the paper when I've got some free time, are there any significant changes from what you explained in the original thread or is the basic idea still the same just refined?

Also, just wondering if you've cut any code for this idea yet, you've had it floating around for some time now.
Ix
full member
Activity: 218
Merit: 128
November 05, 2014, 06:20:06 PM
#13
:shrug: Your points are answered in the paper, so what else am I supposed to do?
sr. member
Activity: 364
Merit: 250
SpainCoin.org
November 05, 2014, 06:14:13 PM
#12

You're just repeating the stuff in your paper again and I can't waste more time on this, sorry. Good luck with decrits
Ix
full member
Activity: 218
Merit: 128
November 05, 2014, 05:59:35 PM
#11
- what motivates signers to sign blocks on only one chain if it's costless to stake? aka "nothing at stake" Nobody has no reason not to mine 2 (or n) forked chains at the same time at no extra cost

2.2:
"A Voice who creates two Records for the same window of time or includes an invalid transaction is considered in violation of the network protocol and will have its risked Decrits destroyed."

Quote
- Block signer selection, how is it done so that it can't be exploited in any way. The "DCA" isn't described anywhere

5.1:
"Each CC, a new order of Records is created by using a random number (see section 5.4) as a seed. Via this mechanism, the Voices have no control over when they will be tasked to create Records."
5.4:
"During each CC, each Voice will include in a Record an encrypted string and a decryption key. The decryption key will decrypt the string from the Voice's Record in the prior CC, and this string will be hashed together with all of the other Voices' strings to generate a random set of data. This random hash will be used to create a new, random order of Records in the next CC."

Quote
- Mining in the past at no extra cost (aka long-range attacks)

All of section 3.

Quote
You're also introducing unnecessary (potentially exploitable, or clearly exploitable) complexities CC/CB sound like plain checkpointing,

It should sound like checkpointing because it is, although I happened to use the phrase "snapshot" instead. It is not a developer or otherwise centralized checkpoint though, it is a consensus of the network, proven by the signatures of all the Voices. Of course it's possible to get all of the Voices to collude, then have every peer watching the network drop the proof of that collusion, and rewrite the CB however they'd like. It's also possible to rewrite the entire Bitcoin blockchain with enough hashing power, although developer checkpoints do prevent recently downloaded clients from being fooled.

Quote
"voice ledger" is another blockchain? how is it secured?

It is part of the consensus block, as with every other piece of network data. It is secured by the ongoing consensus of voices creating records.

Quote
what prevents anyone from forking in the past using a voice's signed transactions to invalidate that voice's later blocks by telling the network that voice is no longer mining.

I'm not sure I understand the question. What is to prevent someone else from using a voice's signature to create a transaction that the voice did not make? Public key cryptography and the extreme difficulty of deriving a private key from a public key. What is to prevent someone that owns the key from pretending it signed out in the past? Nothing, but you can't create a fork if you aren't a voice.

Quote
How is the liquidity/opportunity cost different from taking coin days destroyed into account when doing PoS? (both do basically the same thing: the longer you keep the coins locked down, the more you can stake).

In Decrits, your money is staked up front and there is no taking that decision back without severe penalties (outside the scope of the inital whitepaper). With the coin-age model, you can do whatever you please with your money, which includes the possibility of staking. Decrits is an active stake where as coin-age is a completely passive, opportunistic stake. Consider how this would apply to an exchange or a bank.

Quote
That opens more attack vectors why does your approach not suffer from that (I'm not going into that but you clearly should).

It might, but I haven't thought of any. So could you humor me and go into it at least a little bit?

Quote
you can hardly propagate a tx in 5 secs much less "confirm" -propagate a tx and mint and propagate a valid block ("record").

5 seconds is certainly the best case scenario as it considers that a transaction has propagated the network in 2.5 seconds (a reasonable assumption which can be tested against Bitcoin), arrives at the appropriate voice just before creating the next record, and the record is issued and propagates the network in another 2.5 seconds. The network protocol already has O(1) record propagation, in code. Among well connected peers, records will propagate faster than transactions because of the priority system in place.

Quote
Your notions of confirm and secure in the paper are arbitrary too.

I think they're pretty well-defined.

4.2:
"Confirmed means that a Voice has wagered all of the Decrits he has on deposit that the transaction is valid based on his view of the network, and Secured means that reversing the transaction could only be achieved by an unresolvable network fork."

If you have more concerns about how that works, feel free to ask.
sr. member
Activity: 364
Merit: 250
SpainCoin.org
November 05, 2014, 10:19:25 AM
#10
We could start with this, generically for PoS
- what motivates signers to sign blocks on only one chain if it's costless to stake? aka "nothing at stake" Nobody has no reason not to mine 2 (or n) forked chains at the same time at no extra cost
- Block signer selection, how is it done so that it can't be exploited in any way. The "DCA" isn't described anywhere
- Mining in the past at no extra cost (aka long-range attacks)

Just read the papers you've been referred to, and Andrew Poelstra's treatise on altcoins, Vitalik Buterin's posts on the ethereum blog and so on -to cite some recent material. Address all their concerns and then come back. We shouldn't have to tell you what's already well known, you should be telling us how you're fixing the issues we know. The scenarios you list in section 4 are only of tangential importance and badly addressed anyway.

[for other people reading this Voice=miner who stakes; record=block,DCA ="distributed consensus algorithm" ]

You're also introducing unnecessary (potentially exploitable, or clearly exploitable) complexities CC/CB sound like plain checkpointing, "voice ledger" is another blockchain? how is it secured? what prevents anyone from forking in the past using a voice's signed transactions to invalidate that voice's later blocks by telling the network that voice is no longer mining.

How is the liquidity/opportunity cost different from taking coin days destroyed into account when doing PoS? (both do basically the same thing: the longer you keep the coins locked down, the more you can stake). That opens more attack vectors why does your approach not suffer from that (I'm not going into that but you clearly should).

Quote
In a network where Voices are present and timely, transactions will take
between 5-15 seconds to confirm and 105-115 seconds to secure.
you can hardly propagate a tx in 5 secs much less "confirm" -propagate a tx and mint and propagate a valid block ("record"). Your notions of confirm and secure in the paper are arbitrary too.


Ix
full member
Activity: 218
Merit: 128
November 05, 2014, 08:04:48 AM
#9

He probably hasn't commented because there isn't much to comment on. I've read your paper and it's missing many critical aspects about PoS that you should have covered. That's what I was asking about

Could you provide an example of a critical aspect that I am missing?
sr. member
Activity: 364
Merit: 250
SpainCoin.org
November 05, 2014, 05:12:45 AM
#8

He probably hasn't commented because there isn't much to comment on. I've read your paper and it's missing many critical aspects about PoS that you should have covered. That's what I was asking about
Ix
full member
Activity: 218
Merit: 128
November 04, 2014, 09:14:00 PM
#7

Was it really necessary to switch all terms everybody is familiar with? Anyway, have you read this? Also, you're not providing enough details about how minting goes exactly, or how you plan on avoiding the well-known PoS shortcomings.

It seems like you're responding to the thread not the paper, which is fine but the thread has its own thread. Tongue The paper only introduces like three new terms. I'll edit the OP to make it more obvious. I have read Andrew's paper, and I invited him to comment on mine.
hero member
Activity: 742
Merit: 500
I'm dying.
November 04, 2014, 07:28:09 PM
#6
where is the Whitepaper
It's in the OP. Find the link.
It's not going to scream "I'M THE WHITE PAPER, I'M HERE".
sr. member
Activity: 364
Merit: 250
SpainCoin.org
November 04, 2014, 07:24:43 PM
#5

Was it really necessary to switch all terms everybody is familiar with? Anyway, have you read this? Also, you're not providing enough details about how minting goes exactly, or how you plan on avoiding the well-known PoS shortcomings.
Pages:
Jump to: