Author

Topic: [XMR] Monero - A secure, private, untraceable cryptocurrency - page 1607. (Read 4670972 times)

hero member
Activity: 518
Merit: 521
Your perspective is lacking depth of understanding.

The math I posted applies to any algorithm that uses randomized lookups from reading and writing to a table.

There are 1,000,000 random accesses of the inner loop of CryptoNight.

There are 131,072 individual 128 bit slots in the lookup table.

...

Your approach:  Dynamic recomputation.

The first flaw in your analysis:  Your l3scrypt seems, from what you wrote below, to use 512b (bit?  likely, if scrypt) entries.  CryptoNight uses 128 bit entries, which means that the cost of a 24 bit counter to indicate the last-modified-in round information for a particular value is still fairly significant in comparison to the original storage.

As an example, consider LOOKUP_GAP=2:
  1MB of full cache to store actual values + 64k*4bytes ~= 256KB = 1.25MB of space.

Correct ifIncorrect. You make your hash so slow that you can't deal with DDoS attacks (which is the case for CryptoNote), the size of the 'values' table needed to walk back each path of computation to trade computation for space, becomes larger than the space to store the values normally.

Whereas in L3crypt I have 512B entries in order to make the hash fast enough and still cover 1MB of cache to keep it in L3 cache in order to defeat economies-of-scale with Tilera cpus, GPUs, and ASICs.

So agreed CryptoNote in that case (1MB/16B table with 128-bit, i.e. 16B, elements with 1M writes) defeats the dynamic lookup gap strategy, but at the cost of making the hash too slow to defeat DDoS attacks.

If you actually design hash that won't subject your coin to the threat of being DDoS destroyed in the future, then the dynamic lookup gap strategy can't be avoided. Do some calculations to verify.

On further thought, I remember why I had dismissed the genre of parameters CryptoNote chose ("1MB/16B table with 128-bit, i.e. 16B, elements with 1M writes"), because it doesn't stop the lookup-gap strategy I outlined. The reason is because if the random access is uniformly distributed (and if it isn't you have other attack vectors), then after each quantity of random r/w accesses that exceed ~2X the number elements in your memory table (for the case of reducing the storage by a factor of 2), then the lookup 'values' table can be pruned, because older entries are no longer directly pointed to by any latter entry that corresponds to an entry in the 'elements' table.

Thus the CryptoNote parameters chosen don't necessarily defeat the strategy for trading computation for smaller cache requirement. They do make the proof-of-work much slower than it needs to be.


Also I noticed you ignored my upthread point about cache set associativity and the TLB. When we know all the accesses are 128-bit aligned and they do not span a larger space in main memory, then the ASIC can do optimizations on the design of the cache making faster and or use less logic gates. The CPU's cache is generalized. ASICs can always kick butt because they take advantage of specialized optimizations.
legendary
Activity: 2968
Merit: 1198
An attacker, lets call him Mr X, who could make a significant financial gain by disrupting the Monero network and delaying transaction confirmations would not have to spend a huge amount to obtain the use of a 2M node botnet (interesting aside, from another rough calculation an average botnet could make $0.75/hr per 1000 nodes mining XMR currently) and use it for a DDoS attack against mining pools by sending bogus shares. The pools will ban each node for submitting bogus shares, but at their maximum rate that could only be 1000 banned every second

Another approach is for pools to limit the number (per time interval) of new addresses from which they will accept connections (and/or shares or blocks) at all. A DDOS as you described would lock out new users until countermeasures were deployed but existing users and the existing network would be unaffected.
legendary
Activity: 1624
Merit: 1008
Poloniex reports that they have worked with the Monero devs to improve transaction speeds for deposits and withdrawals.

Any idea what improvements are to be expected?

I've never transacted XMR on Poloniex, actually I tried to withdraw once and it didn't happen as they had an issue.

Today I finished my purchases of XMR on Poloniex and will start withdrawing a significant part.

sr. member
Activity: 462
Merit: 500
sr. member
Activity: 252
Merit: 250

One minor technical correction though:

Quote
In the future those key factors protecting Monero may well change, with more pools online and more decentralisation, plus a much higher tx generation rate resulting in significantly larger blocks requiring longer to validate.

As things stand today larger blocks don't take much longer to validate. The PoW hash is performed not directly on the entire block but on a much faster hash of the block, so it is essentially independent of the block size. The transactions within a block also need to be validated, but currently that is much faster than the PoW. That could change if database lookups are required. Note that if the dominant portion of block validation time is database lookups, then the speed of the PoW becomes unimportant to validation speed.



Thanks, I'd assumed the block validation time was directly related to the number of txes involved, so I see that wouldnt actually be a significant factor.

As I read it most of the arguments for a shorter block time seem to be commercial, e.g. "faster tx validation", whereas the justifications for longer block time are generally technical. I don't really see Monero being used in the future for the sort of transactions where fast confirmation is an important requirement, so I'm not sure there is much to debate. Others may well disagree of course Smiley
legendary
Activity: 1722
Merit: 1217
Correct if you make your hash so slow that you can't deal with DDoS attacks (which is the case for CryptoNote), the size of the 'values' table needed to walk back each path of computation to trade computation for space, becomes larger than the space to store the values normally.

Whereas in L3crypt I have 512B entries in order to make the hash fast enough and still cover 1MB of cache to keep it in L3 cache in order to defeat economies-of-scale with Tilera cpus, GPUs, and ASICs.

So agreed CryptoNote in that case (1MB/16B table with 128-bit, i.e. 16B, elements with 1M writes) defeats the dynamic lookup gap strategy, but at the cost of making the hash too slow to defeat DDoS attacks.

If you actually design hash that won't subject your coin to the threat of being DDoS destroyed in the future, then the dynamic lookup gap strategy can't be avoided. Do some calculations to verify.

This is very interesting. If you are right though it should be too terribly disruptive to hard fork away from cryptonight (if i remember correctly that is what the hashing function for the pow is called) POW and into something less taxing to verify.

*edit* is anyone here particularly committed to cryptonight? would anyone care if we had to switch to scrypt or something similar?
legendary
Activity: 1596
Merit: 1030
Sine secretum non libertas
...Mr X...

Mr X is female, and her name is Eve (because she's "evil").  Just sayin'
legendary
Activity: 2968
Merit: 1198
Okay but I spent considerable time designing what the CryptoNote designers were attempting to design

Actually as far as I can tell, they delivered what you claim you were attempting to design. You have delivered no work product at all. No code. No white papers. Nothing. Except maybe delivered to yourself, which does not count. It is all hot air.

At this point I would ask that you take this discussion elsewhere. There may well be (theoretical, unidentified and undemonstrated) cryptographic or other issues with Cryptonight, but that is a tiny subset of topics of general interest to the user base and potential user base of Monero. Now we have the past several pages of this thread being taken over by the discussion, and the way the last few exchanges have gone, every appearance this back-and-forth will continue for several more pages. That is inappropriate. Please stop, or just go ahead and create your own thread focused on that particular subtopic.



legendary
Activity: 2968
Merit: 1198

Your post makes some good points and these issues are definitely being looked it. There is wide agreement that the one minute block time was a mistake absent other changes in the design of the block chain (such as GHOST or similar ideas).

One minor technical correction though:

Quote
In the future those key factors protecting Monero may well change, with more pools online and more decentralisation, plus a much higher tx generation rate resulting in significantly larger blocks requiring longer to validate.

As things stand today larger blocks don't take much longer to validate. The PoW hash is performed not directly on the entire block but on a much faster hash of the block, so it is essentially independent of the block size. The transactions within a block also need to be validated, but currently that is much faster than the PoW. That could change if database lookups are required. Note that if the dominant portion of block validation time is database lookups, then the speed of the PoW becomes unimportant to validation speed.

newbie
Activity: 24
Merit: 0
Is there a place where one can discuss technical issues and project development, and get timely replies from devs?
I'm working on a rather interesting project and need some help getting Monero RPC to work.

#monero-dev is the best place. You left the channel right before busoni offered to help regarding your RPC issues.

 Grin thanks

I'll try again
legendary
Activity: 2968
Merit: 1198
CryptoNote employs AES encryption as a random oracle so that all possible cache table elements should be equally probable at each random access. But AES encryption isn't designed to be a random oracle. Thus there may exist attacks on the structure of the probabilities of random accesses in the table.

Can this be quantified somehow? Saying AES may be exploitable isn't that much more interesting than saying that SHA3 or some other hash function may some day be found to have collisions. While no one may disagree with that, no one will care either if the possibility is extremely remote.

Reasonably good info here: http://security.stackexchange.com/questions/8048/why-aes-is-not-used-for-secure-hashing-instead-of-sha-x
sr. member
Activity: 252
Merit: 250
Most of that long Anonymint post I don't enough about to debate with any authority, but the issue of block time, DDoS and orphan rate has been of more interest to me recently. The section in question was:

Quote
If DDoS attacker sends bogus proof-of-work blocks, the calculation time around 1/100 of second for an average node, or 1/1000 second for a high powered node.

This impacts on how many IP addresses you can blacklist per second, and also the propagation time of new blocks which affects the orphan rate[1], which thus impacts how fast transactions can be. DDoS could drive orphan rate skyhigh.

[1] https://bitcointalksearch.org/topic/reasons-to-keep-10-min-target-blocktime-260180
     http://bitcoin.stackexchange.com/a/4958
     https://bitcointalksearch.org/topic/m.3647346
     https://eprint.iacr.org/2013/881.pdf#page=11

Of course you can resolve it by reducing decentralization and having everything go through pools that trust each other, a la Bitcoin which now has 1 pool with 50% of hashrate.

I won't disagree with the figure of 1ms for a high powered node to verify a share, my rough figures suggest its within that order of magnitude for the average pool server.

Interpreting your argument into more easily understood language (doing my best not lose the essence of it but it does allow more readers to participate):

An attacker, lets call him Mr X, who could make a significant financial gain by disrupting the Monero network and delaying transaction confirmations would not have to spend a huge amount to obtain the use of a 2M node botnet (interesting aside, from another rough calculation an average botnet could make $0.75/hr per 1000 nodes mining XMR currently) and use it for a DDoS attack against mining pools by sending bogus shares. The pools will ban each node for submitting bogus shares, but at their maximum rate that could only be 1000 banned every second. With the current distribution there are only about 5 pools the botnet would need to attack (although there are at least 3 other "dark" pools of 1-3MH/s, not sure how Mr X identifies them). If they spent all of their processing power banning botnet nodes its going to require around 7 minutes to blacklist the entire botnet, but after 4 minutes over half of it will have been blocked. With an average blocktime of 1min there could be several found blocks waiting to be submitted, so we can be pretty sure that one or more will have been accepted by a pool and started to propagate over the network.

Which takes us to the other part of the argument, by hammering the pools Mr X slows down network communication leading to segmentation and mini-forks being created which will subsequently be orphaned. With its fast block time Monero is more at risk of divergence - ideally you want the majority of the network to have validated a block before the next one is found, although it can cope with a few "quick blocks" in a row because there will always be a longer one to provide time for the network to agree (converge) again. Incidentally this is why exchanges tend to require a large number of confirmations before accepting a tx, to ensure they arent invalidated by an orphaned chain. Most of the theory here about problems with convergence seems to be based on bitcoin and a large array of decentralised network nodes, meaning you only have to cause a small increase in propagation time to get an exponential effect over the entire network. I would actually describe the Monero mining network as semi-centralised, because most of the pools configure each other as priority nodes so they are all connected by 1-3 network hops, meaning new blocks only in fact require 4 hops to propagate on average.

So it must be easy for Mr X then - he only has to DDoS those core pools and he can take down the whole network surely? Well not really, we've already seen that it would take a large botnet to kill the pools for a few minutes, and just putting them under 50% load for twice as long will have even less effect. It also ignores the fact that the pools usually run their daemons on separate servers to the pool code, so he may delay a new block for a few minutes but it will be able to propagate quickly once found. Currently the best Mr X can probably achieve is a few minutes of disruption and some agitated pool ops before the network sorts itself out and the blockchain continues.

In the future those key factors protecting Monero may well change, with more pools online and more decentralisation, plus a much higher tx generation rate resulting in significantly larger blocks requiring longer to validate. This could make it possible for Mr X to launch a more disruptive attack so it is worth considering methods of mitigating that risk, one of the more obvious being to increase the block time, which has been discussed before and I dont believe it has been ruled out by the dev team yet. Monero is still at the stage where major changes to the protocol could be made if deemed necessary, it is not set in stone yet.
hero member
Activity: 560
Merit: 500
Is there a place where one can discuss technical issues and project development, and get timely replies from devs?
I'm working on a rather interesting project and need some help getting Monero RPC to work.

#monero-dev is the best place. You left the channel right before busoni offered to help regarding your RPC issues.
hero member
Activity: 560
Merit: 500
CryptoNote employs AES encryption as a random oracle so that all possible cache table elements should be equally probable at each random access. But AES encryption isn't designed to be a random oracle. Thus there may exist attacks on the structure of the probabilities of random accesses in the table.

Can this be quantified somehow? Saying AES may be exploitable isn't that much more interesting than saying that SHA3 or some other hash function may some day be found to have collisions. While no one may disagree with that, no one will care either if the possibility is extremely remote.
newbie
Activity: 24
Merit: 0
Is there a place where one can discuss technical issues and project development, and get timely replies from devs?
I'm working on a rather interesting project and need some help getting Monero RPC to work.
hero member
Activity: 700
Merit: 500
Seriously until you get some qualified cryptanalysis on your proof-of-work, you are just blowing hot air.
Says the guy blowing more hot air then anybody, and refusing to back up his statements...
hero member
Activity: 560
Merit: 500
Otherwise having to read 1 page thesis about what appears to be who thinks who is god is going to turn off others who are the investing in this coin to use it , and just turn it in another shitty ass Altcoin.

No one is obligated to read any post. It's precisely because XMR isn't "another shitty ass Altcoin" that these sort of discussions take place. The level of intelligence on display in most altcoin threads is just depressing.

The ones who may be scared off are not investors -- it would be the legions of small-time day traders who are cluelessly hemorrhaging their own money.
full member
Activity: 145
Merit: 100
Alright all you "smart people" arguing who has the biggest nutsack or brain, could you all just work together on getting a Bitcoind type system up or an official GUI wallet so we can store this coin elsewhere other than Poloniex and we can use it to buy shit and gamble. This adds value to the coin.

Otherwise having to read 1 page thesis about what appears to be who thinks who is god is going to turn off others who are the investing in this coin to use it , and just turn it in another shitty ass Altcoin.
hero member
Activity: 518
Merit: 521
What gives is very simple:  You're wrong;

I still believe you are wrong. See below...

you're also being needlessly insulting, in a discussion that need not become personal.

You claimed authority with "professional opinion" instead of publishing analysis of all possible attacks. Sorry peer review requires we publish analysis not authority.

If you'd like to engage in a credential pissing match, fine, but that seems like a waste of time.

No I'd like to engage in published analysis instead of claiming the blackbox (closed source) called "professional opinion".

Let's settle for me pointing out that I'm the original source of the code that's now used in the inner loop of the CPU cryptonight mining and block verification code, so I will claim some familiarity thereby.

Okay but I spent considerable time designing what the CryptoNote designers were attempting to design (even designing a 512 BYTE version of the ChaCha ARX style hash to make it fast enough) and wrote a very detailed set of whitepapers on the L3crypt and the Shazam! hash with around 30 citations. Also thought about the math and wrote it down, even for example studying cryptanalysis attacks on the design of ARX hashes.

You haven't posted enough details about your L3scrypt design to determine if your analysis actually applies to CryptoNight, but let's walk through the math a little:

The math I posted applies to any algorithm that uses randomized lookups from reading and writing to a table.

There are 1,000,000 random accesses of the inner loop of CryptoNight.

There are 131,072 individual 128 bit slots in the lookup table.

...

Your approach:  Dynamic recomputation.

The first flaw in your analysis:  Your l3scrypt seems, from what you wrote below, to use 512b (bit?  likely, if scrypt) entries.  CryptoNight uses 128 bit entries, which means that the cost of a 24 bit counter to indicate the last-modified-in round information for a particular value is still fairly significant in comparison to the original storage.

As an example, consider LOOKUP_GAP=2:
  1MB of full cache to store actual values + 64k*4bytes ~= 256KB = 1.25MB of space.

Correct if you make your hash so slow that you can't deal with DDoS attacks (which is the case for CryptoNote), the size of the 'values' table needed to walk back each path of computation to trade computation for space, becomes larger than the space to store the values normally.

Whereas in L3crypt I have 512B entries in order to make the hash fast enough and still cover 1MB of cache to keep it in L3 cache in order to defeat economies-of-scale with Tilera cpus, GPUs, and ASICs.

So agreed CryptoNote in that case (1MB/16B table with 128-bit, i.e. 16B, elements with 1M writes) defeats the dynamic lookup gap strategy, but at the cost of making the hash too slow to defeat DDoS attacks.

If you actually design hash that won't subject your coin to the threat of being DDoS destroyed in the future, then the dynamic lookup gap strategy can't be avoided. Do some calculations to verify.

You furthermore haven't dealt with the issue of potential cycles in the recomputation graph, which requires a somewhat more sophisticated data structure to handle:  A depends on B depends on C which depends on an earlier-computed version of A.  (Keeping in mind that there's a non-negligible chance of A immediately modifying A!  It happens, on average, a few times per hash).

Each entry in the 'values' table can index to another entry in the 'values' table enabling to trace back.

I missed the part of your proposal that handled that.  Furthermore, there's some internal state associated with the mixing that happens at each round -- it's not simplify a crank-through of X iterations of a hash on a static data item.  That state is carried forward from the previous half-round (the multiply or the AES mix, respectively), so you have to have a way to backtrack to that.

I believe what I quoted from my rough draft whitepaper is incorrect on that (hadn't looked at that for some months), and each entry in the 'values' table should point to another entry in the table until it is traced back to a stored value.

As I said in my post, there are possibly some weaknesses involved in the use of a single round of AES as a random number generator, but I *suspect* they're not exploitable enough to confer a major speed advantage.  That's not an expert part of my conclusion, because I'm not a cryptographer.

Single round can only diffuse over 32-bits (and that doesn't even mean all the 32-bit space is randomized), and there are other attacks such as on the key scheduling.

The GPU has more FLOPs and can mask away the latency by running sufficient threads but it lacks an AES circuit. ASICs can add the AES circuit to eliminate that CPU advantage (and even accelerate the computational portion) and apply the GPU style advantage for masking the random access latency.

The CryptoNote hash keeps GPUs at power efficiency parity (and MemoryCoin 2.0 did that too) and it doesn't defeat ASICs dominance, rather it only delays due to more complexity of implementing an ASIC. And I have stated that making it complex but not impossible to make a superior ASIC is a big risk because it could mean when they do come, they won't be ubiquitous (and this is the reason I aborted my L3crypt design).

And these design choices come at the cost of making your coin DDoS attackable (even worse for MemoryCoin at 10 hpm, not per second) and also the slow proof-of-work hash eliminates the opportunity to solve the anonymity correctly (i.e. not using Tor or I2P) but I won't reveal that to you.

I think you're being overly optimistic about the success of your own approach based upon the flaws in your (completely unexplained) l3scrypt.

I know of no design flaws in L3crypt. It achieves the goal of being fast enough and making it complex to implement an ASIC, leveraging Intel's economy-of-scale with L3 caches (which is even superior to Tilera). Note at 512B writes, the write-back bandwidth becomes a factor in the design.  However in attaining that speed, it is vulnerable to the aforementioned lookup-gap approach of trading computation for space. The concept of reading and writing over a memory table is the same in L3crypt and CryptoNote. The difference is the size of the r/w elements, the number of random access iterations relative to the table size, and the resultant speed of the hash. And the design choices in those variables for CryptoNote makes it DDoS attackable because the hash is slow.

If DDoS attacker sends bogus proof-of-work blocks, the calculation time around 1/100 of second for an average node, or 1/1000 second for a high powered node.

This impacts on how many IP addresses you can blacklist per second, and also the propagation time of new blocks which affects the orphan rate[1], which thus impacts how fast transactions can be. DDoS could drive orphan rate skyhigh.

[1] https://bitcointalksearch.org/topic/reasons-to-keep-10-min-target-blocktime-260180
     http://bitcoin.stackexchange.com/a/4958
     https://eprint.iacr.org/2013/881.pdf#page=11

Of course you can resolve it by reducing decentralization and having everything go through pools that trust each other, a la Bitcoin which now has 1 pool with 50% of hashrate.

You're missing way too many CryptoNight-specific details to be convincing at all.  I think that underlying this is an important difference:  Your PoW design didn't carry as much information forward between rounds as CN does.  Your approach isn't crazy, but you've left way too many important parts out of the analysis.

It is carrying state forward just the same. The differences are the variables I stated above.

Regarding the bandwidth-intensive approach, you're still wrong about where the time is being spent in the GPU.  It's about 50/50 in random memory access and AES computation time.  Amdahl's law gets you again there -- I'll certainly grant something like a 4x speedup, but it starts to decline after that.

That is because you aren't running enough threads on the GPU to mask away all the latency with the coalescing of memory accesses on the GPU. As the number of threads increase, this will improve.

Update:  I also read your linked thread's comments about the use of AES.  You're not looking at the big picture.  In the context of a proof-of-work scheme (NOT as the hash to verify integrity), the limitation of 128 bits at each step is unimportant.

In terms of you missing the 'big picture' see my points up-post.

CryptoNote employs AES encryption as a random oracle so that all possible cache table elements should be equally probable at each random access. But AES encryption isn't designed to be a random oracle. Thus there may exist attacks on the structure of the probabilities of random accesses in the table.

Note the AES vulnerability isn't required to implement an ASIC that out peforms. It is an orthogonal potential attack. There might be a way to trade computation for space within some structure that deviates from uniform random distribution given by the misuse of AES encryption.

More to the point, your post has absolutely no substantiation of your claim and has a link to a stackexchange article that in no way suggests any easy-to-exploit repeating pattern of the output bits that could be used to shrink the scratchpad size.  If you'd care to actually provide a substantive reference for and explanation of your claim, then perhaps the Monero developers (or bytecoin developers) might take it a little more seriously.

I don't have to do your work for you. Ask a cryptographer that knows about AES, and they can explain this to you in more detail.

Seriously until you get some qualified cryptanalysis on your proof-of-work, you are just blowing hot air.
newbie
Activity: 24
Merit: 0
Hey, can someone help me with RPC commands? I get empty "result": { } on get_payments method.

Quote
Request:
array (
  'jsonrpc' => '2.0',
  'method' => 'get_payments',
  'id' => 1378931342,
  'params' =>
  array (
    'payment_id' => '64-char string',
  ),
)

Response JSON:
'{
  "id": 1378931342,
  "jsonrpc": "2.0",
  "result": {
  }
}'

However incoming_transfers method works fine

Quote
Request:
array (
  'jsonrpc' => '2.0',
  'method' => 'incoming_transfers',
  'id' => 1998971379,
  'params' =>
  array (
    'transfer_type' => 'all',
  ),
)


Response JSON:
'{
  "id": 1998971379,
  "jsonrpc": "2.0",
  "result": {
    "transfers": [{
      "amount": 90000000000,
      "global_index": 86923,
      "spent": false,
      "tx_hash": "",
      "tx_hash_proper": "AAAAAAAAAAAAAAAAAAAAA"
    },{
      "amount": 200000000000,
      "global_index": 242013,
      "spent": false,
      "tx_hash": "",
      "tx_hash_proper": "AAAAAAAAAAAAAAAAAAAAA"
    }]
  }
}'
Jump to: