Topic: [ANN] ParallelCoin - DUO - SHA256 + Scrypt | Community Takeover - page 13. (Read 61171 times)

List of stolen tokens on Cryptopia:
https://github.com/elementus-io/cryptopia-hack/blob/master/README.md

DUO is not on the list.

Thank you for the update.
I would make a stand alone .exe to run and the user will know where to copy their stuff to run it again and again.
The old Parallelcoin wallet is also stand-alone .exe and comes with no installer....
So I guess to make an installer for the new one would be a total extra, which could be done some time after the initial release....

As @marcetin started working on the wallet GUI, it became clear I needed to clean up the logging system and launch system to enable one binary to run both node and wallet alongside the GUI.

It took some searching to find the simplest and cleanest solution, but I had already written a logging framework, so step by step I will be stripping out the old  logger, putting in the new one, and everythting launches from one nice CLI with sub-commands, automatic self-configuration with sane defaults and later some more utilities and extensions of the components.

The new logging system is based on goroutines and channels, so it minimises context switching, reducing the processing cost of logging.

The all-in-one structure will mean that distribution can be done just with the binary, at least to start with, and with a installation script for each platform, it can be installed where it will be immediately accessible. Possibly I will make installer packages that download and place the binary and create any necessary user interface shortcuts rather than contend with many different installers. It's usually just a few files in the user's profile and the GUI would be accessible by the system launch menu.

It may take a little while for me to finish the launcher/log changes, it's pretty heavy going repetitive stuff, lots of details to account for.

I have been thinking about installations and defaults... I think it will default to running the GUI with no options given, and prompt you to install it, it is easy to put different parts in for different operating systems.

I'm not sure about how to copy the binary in windows but maybe it has some 'do this when I finish' special thing that lets you do exactly that, move the binary to its proper place. Or for windows maybe just download it again and put the file in the right place. On mac the dmg package is the whole thing. Writing installers for multi-platform is a pita, but hey if we can have a GUI launch on all platforms we can be the installer. Also later on such things as running at startup and whatnot, which also has implications with the server versions as well.

Also note that it will have integrated explorer webserver and extended GUI that links to the wallet GUI. No more wondering where to check if a transaction has made it into a block!

Just a little, super-short update:

I have been finding it very confusing trying to work with the KCP protocol for this task, so I have been learning how to program UDP sockets directly.

I now have an important part of it worked out, the 'reliability' part in the Reliable UDP - a fully working Reed Solomon error correction codec. It uses a 9/3 scheme, where you send out 9 separate packets, but only 3 have to make it to the other side intact to deliver the message.

This is very pessimistic about corrupted packets and dropped packets, but it should ensure that almost never a retransmit is required. Being that transmission can be up to 400ms on a good day from the opposite ends of the internet, the extra 5-10 ms for encoding and the triple amount of data sent hardly seems like a bad price to pay.

This reliable UDP protocol is for pushing work to miners in the case of a mining farm, since at 9 second average blocks, every millisecond counts towards finding a solution before someone else. But later on, I will be adding this messaging protocol also to the chatter between nodes, and nodes that have this facility will be able to identify each other and switch to this ultra-low latency gossip protocol transport. This means that Parallelcoin will be one of the fastest proof of work blockchains on the internet.

Oh! In other news, our lead web developer marcetin is back on the case, I had to update some of the wallet server parts that I hadn't updated to match the new hard fork block version scheme, but he has the required wallet server to go with the full node. Oddly, it turned out that the parts that needed fixing were not directly in the 'mod' wallet code at all, but rather, in the 'sac', which is an SPV node based on the SPV wallet being developed by the Lightning people. I didn't recall there being a dependency between them, but there you go.

The SPV wallet is something that I will be also working on after the release of Plan 9 hardfork. It's not so important now since the chain is only about 150Mb in size but it will grow faster after the hard fork and having a light version will be essential especially for mobile devices.

I'm going a bit slow at the moment for various reasons, but just wanted to  make an update as I have achieved something...

I am building a miner worker controller that will interface with the kopach miner to coordinate work for anyone running more than one machine (If you only mine with one machine you can use the built-in CPU miner).

It took a few tries to figure out what way would be the best to work it, and forking the cpu miner code was the simplest, as basically what it does is exactly what I need, except for to instead of run miners directly inside the main server process, it runs a server that clients send a subscription request, and then the worker tries to solve it, when a new block is received from the network, it pushes out new work to the clients, who then should abort current jobs and start on the new.

The miner controller service will be using KCP, a reliable UDP protocol with forward error correction and optional encryption. For miners running on a lan, the small extra reduction of latency not using encryption is an option, and if the workers are connected over insecure paths, it can use AES encryption to prevent mischief, and the pre-shared key used for the encryption also acts to prevent snooping and spoofing from a potential attacker.

Of course I don't advise using the internet as the transport for mining, as it will cost equal to several solution attempts in latency. This is because with 9 second average block time there is only a small window of opportunity to solve a block and submit it and have it accepted before other miners, and this acts as security against both pool miners and botnets, both of which will have a significant disadvantage in potentially up to 2 seconds delay in assigning new work after their main node receives a new block forcing a work restart.

CPU mining has different issues to deal with compared to other types of mining, and long block times allows a lot more scope for attackers to corrupt the data on the network. Plus, it is generally considered in the web application business that after 10 seconds delay, users feel that the remote system is unresponsive, and thus first confirmations typically being under 10 seconds means that real applications, and not just monetary ledgers can be used from this same base.

It's back online again... I am gently pushing to get the infrastructure in focus now, since I am closing in on a release complete with the hardfork and new mining protocols, it will be 100% reliable soon but for the moment it is working and you'll be pleased with how fast it is compared to most explorers. @marcetin is really good at writing super fast web apps using Go and AMP...

https://explorer.parallelcoin.info/

Just so you know, I am an advisor to Bearshares and I was the first after the founder to set up a witness. It's really just one guy working on it right now, but we now have another 3 witnesses and the crypto-social celebrity Halo will be online with her witness shortly also.

I was just informed today that EasyDex, a BTS based DEX that friends of mine started up in second quarter 2018 will be listing the Bearshares tokens, which is nice for me because it means I might have a little extra capital for funding my work. I will also be working towards gluing all the things together, hopefully with a new asynchronous PoW blockchain system that I have just started designing.

And about the actual project at hand - I am currently building the stand-alone miner app for the new chain, and for this I need to write an extension to the websockets notification API that the miner subscribes to to be notified when a new block has been found on the network, and provides the caller with all of the data (prevblock hash, merkle root, timestamp and difficulties for all 9 algos). It is possible to do this without the extension but the response time for the data is much too slow. It needs to basically be instant, chain connects a new block, and then miner has what it needs to directly build and solve a block. Because of the 9 second block target,  every millisecond counts towards a miner finding and submitting a block to get to all peers before another miner does.

I had hoped the websockets latency would be better than what it is, but them's the breaks. In the future I want to abandon the HTTP and TCP protocol completely, they just have so much handshake overhead compared to UDP streams. After the release of the hardfork I plan to be working on everything non-consensus related. Particularly the peer to peer protocol, which I would much prefer if it was multicast UDP, so nodes just stream new gossip to each other without requiring acks (but accepting resend requests) to further tighten the timing and decrease coinciding block solutions.

Orphan rate is somewhere around maybe one in 60 in testing so far. Considering that there is 400 blocks per hour, this does not work out to a lot of wasted hash power really, but I know that improvements in message propagation would cut that down dramatically, maybe as much as 10x, as miners far less frequently have stale blocks to work on after a new block has arrived.

Transaction latency and throughput are important metrics in this technology, and it's my opinion that it is still an unresolved question. There is better gossip network protocols existing than the quite aging system used by most bitcoin forks and most anything else, but the technology to leapfrog past the limitations of TCP is not widely deployed yet, and maybe a little immature, and to some extent, the existing network systems are not *so* friendly with these. If we had IPv6 available to everyone, that would change a lot, enabling many to many multicast routes. This can be achieved to a significant degree with UDP, however.

As I see it, the real winner in the tech race hasn't really shown up yet. Bitcoin and most bigger forks are still on laggy, long delay, narrow channels, and bitcoin's incumbency has stifled its development. Smaller coins with better devs working on them are going begging for funding to improve the work rate and put the better programmers in the job. Even though crypto engineering work has quite a decent pay rate against the market in general, the reason why is in part due to the fact that even programmers involved in developing distributed, centralised systems look down their noses at this tech, and it's quite ridiculous because actually the need for resilient distributed database applications is greater than ever. Especially in payments, where especially social network systems are being 'herded' by players like Mastercard and Visa into supporting their political backers agendas... But I am quite sure that if there was an open protocol that could run rings around Mastercard's capacity, the situation would be very different. It's this that they are trying to stave off and continue to milk their investors as long as possible. There's more conspiracy theory junk I could talk about related to that, but it's quite simply self-interest on the part of those with the biggest existing stake in the market who don't want to lose their top position, everything else is just evidence.

The link:
https://bearshares.com/whitepaper/@loki/parallelcoin-whitepaper

Best of luck to Parallelcoin, great to see them on other platforms like Bearshares.

only in your eyes... 
Block explorer will return soon.
Check out discord for news on the project, too... https://discord.gg/N8hfCX

the blockchain explorer is not working anymore. The project looks dead

I just want to report another, a bit more minor but important change. I am in the early steps building the stand-alone miner app now, and first step was to get an ops/second benchmark against all the hash functions. I can't really change this on the current fork, but the work calculation of the old chain does not account for the nearly 1000x difference between the time to do two SHA256D hashes, versus Scrypt.

The code is there that could change it but I have set the multipliers for the old hash functions to 1 so they don't conflict with the current consensus. But the new functions all have their (approximate) relative hash function times used as a multiplier before the 'work' value is computed, meaning now that as well as comparing the absolute value of each block's hash, first the numbers are multiplied by their hash/ns rate as measured on my machine on a single thread, so now in the case of two blocks arriving close together and having to be separated based on their work value, now it is relative to the time per hash, relative to each other, so this decision is more equitable, as the difficulty targets for the different algorithms is largely based on their relative hash rates.

The difficulty on the Lyra2REv2, for example, is almost always double as compared to most of the others, so its difficulty tends towards being double over a longer period.

The time to solution is entirely proportional to the number of hash passes performed. There is a definite point at which over some number of repetitions, the chance of already having found a solution is over 50%. So the relative work values should reflect this.

I don't know if this is the first multi-algo chain to fix this probable error in many of their consensus algorithms, but it is fixed here. Long block times mean these coinciding solutions happen more rarely, but when it does happen, it doesn't make sense to measure the work by the same exact value because the value is entirely a product of the number of tries, in a probabalistic sense.

Experimenting with shorter block times and being able to watch the appearance of orphans and trying several different multiple averages, I am now close to something that seems fully functional. But the main thing I learned is that the connection protocol on the p2p network has several arbitrarily long delays that are coded in that cause blocks in the propagation of new blocks, sometimes by up to 5 seconds, and since I am testing on loopback, that delay is very visible.

So I have tweaked the timeouts and delay timers in the network protocol, and it has all by itself reduced orphans significantly, and smoothed out the block production timing. The current master works fine on the main net and I have now updated it to have these better connection parameters, and it will definitely improve the performance of the network.

The averaging now uses an exponential weighted per-algo interval target, an all-history average block time, to help it tend towards long term being the target time, and a trailing target that sits back approximately 9 hours before the current head block, to help reduce short term variations. The cubing of the per-algo interval target gives it more weight in the calculation and adjusts to keep the algorithms distributed evenly.

After a little more testing of this I think I will be building a miner app that works with the getwork RPC command that I reinstated. I'm not sure it is properly compatible with regular miners, it sorta looks like it pads out some of the fields wrongly. But it doesn't really matter so much at this point. There will not be a wide distance between smaller and larger miners in the network for some time, and hopefully, GPU miners won't be so much faster that solo CPU mining is not viable.

The biggest reason to get rid of the long block time, and shorten it as much as possible, is that this helps with keeping the distribution broad. Solo mining loses viability when chance of getting a block extends into the range of days, and necessarily, less blocks means more hash power is used per block. There is other ways to get efficiencies and I expect that possibly GPU miners will be more efficient, but it should not be as big as with more usual, non-pipelined multi-algo mining compared to this. Specialisation has less viability on a chain where you can't squeeze significantly more out of your mining by focusing on one algorithm, and thus raises the complexity of the getting better efficiency.

The other thing is that shorter block times mean overall lower time of clearance, which is a desirable feature. Of course it has to be balanced with security and throughput, but why should it be any slower than it has to be? Network timeouts are entirely non-consensus related, and variations usually won't cause nodes to be banned, so long as they are not significant, but since the network will fully upgrade to the new node, it will improve it even before the fork takes effect.

To improve the time it takes for me to see the block times balancing properly, I have set delays for the first 15 blocks of 30 seconds each. I don't think I can really enforce that on the live network, but the all-time and trailing averages will immediately take effect once the first block of each new algorithm comes in, and the weighting begins being added to the calculation from the second block for each algorithm, so probably it will converge quickly. If the currently being tested speed passes, it will be 15 second blocks, 240 blocks per hour, 5760 blocks per day, and the trailing average length is at 640 blocks per algorithm, which will be approximately one day of averaging window, combined with the all-time average.

Plan 9:
9 second blocks
9 complex new algorithms

I am just updating this to report that I have found even with a random submission latency between 256 and about 4384 milliseconds of delay sending to two other nodes in a 3 way testnet, roughly similar to the same happening to two thirds of the network, that it does not make excessive numbers of orphan blocks, hardly any. I will have to test it with a testnet running with one node on a 4G connection and two VPS nodes one in asia one in america, with the testnet submission delay disabled, see if it is similar on a realistic global network.

Seeing this is reinforcing my intuition that most chains are not really exploring the issue of the underlying network protocol as a factor in delays and wasted work on orphans. I estimate that if that is averaged out, the typical delay on my test is about 2 seconds, which, if we give a wide margin of 300ms to propagate to 8 nodes, that's 8^5 (32k) nodes that can receive the messages in under 1/4 of the block time, so it would be a pretty rare situation that the whole network lags longer than this.

I suspect also that the per-algo block time average helps as well, tending to reinforce an even delay time between blocks over a 9 block period.

Because of the greater amount of complexity of the PoW, I am finding that the 30 second testing block time is too short, and every so often it slows down enough that adjustment hits the minimum difficulty and a series of short blocks occurs, so I am going to leave it at 5 minutes. I am inclined towards the strategy adopted by Bitcoin and Litecoin and implement a form of Lightning for faster clearance. Many functions don't require realtime confirmation, and even 30 seconds is too slow for physical point of sale anyway.

I am a firm believer in the Proof of Work scheme, and I absolutely do not want to put anything involving staking at the base level. I think stake can be useful at higher levels of the protocols, like social networks, but not so much good on the replication protocol level.

A thing that is on my mind is eliminating the tcp spin-up/teardown overhead... Currently, even all running on localhost, nodes disconnect and then reconnect for no reason I can discern, and when this happens as a block is found, the other nodes can spend quite a long time (up to 5 seconds sometimes) mining a block that is stale but *even on localhost* they don't know this yet! insane!

So, it is not on my agenda for this hard fork, which I am getting close to a settled difficulty and mining architecture, but I am very much in favour of adding a new connection method between peers, and yes, I think even I should make RPC extensions using KCP reliable UDP connections - a decent server with a 100mbit connection can handle 5000! open sessions and has zero overhead for keeping connections open and low retransmission rates when reed solomon erasure coding is added to the data. With this kind of connection between peers, at least, a block discovery should mean the most time nodes will keep digging on a stale block will be under 400ms for the worst connection at opposite sides of the planet.

Right now, and for most crypto network protocols, this is the best case even in a loopback testnet. I think, if it works as well as is required for game servers, which is how most of them work, using RDP streaming, the same benefits could be realised for blockchain. If the TCP overhead latency were eliminated, I think that 30 second blocks would be hardly pushing, and since, as I have seen, on the Steem network, 19 nodes can hit a 3 second block cycle, who cares about the super tight clocks that staking chains can do, if you can make PoW chain run at 5 second blocks... One of the interesting things about this is that no matter how much hashpower you have, a 5 second block time on a chain would mean you barely even get started digging a block before someone else finds it. It's an interesting thing I have noticed and with 30 second block times now and then I can see, because each node only has one algo mining, that the fact it goes straight to digging on the block it just made...

As for progress, I think I am only a few days away from being confident I have built a good difficulty adjustment mechanism, I also have the issue of the early blocks of the fork and inevitable process of retargeting. I think, looking at how it works so far, that the chain will be fully on its proper clock within 6-12 hours, so there will be a 'fair hardfork' period of 288 blocks, at the same 0.02 as the initial 998 blocks. Thereafter, the closely fitted exponential decay curve will be implemented and start from 2 and decay at approximately 75% per year, or, about 25% annual supply expansion rate, which roughly matches the 250,000 block halving cycle, but without the stair steps.

Just a note about a further improvement - the use of an all-time average necessarily means eventually the factor will greatly decrease in its adjustment precision, so I am adding a second factor, a 'half time' average, which is computed from the block at (integer rounded) half the block height to the first block (which will be set as the hard fork activation height). This will add a further, half-decaying precision adjustment that pegs to a long time range to help reduce oscillation a bit further.

I may add a quarter or 1/3 or other, smaller long run average but I suspect that half will be sufficient.

Some results!

https://github.com/parallelcointeam/pod/blob/master/isolatedtestnet/run1.csv

The all-time (since block one) average for this is about 28 seconds per block. The variance seems to generally stay between 0.5 and 2.0 most of the time, and if the all-time average is shown progressively advancing you can see it oscillates randomly between about 25 and 35 seconds.

This data set was generated using an exponential weighting for the per-algorithm blocks multiplied by the all-time average with a factor of 0.75, I am going to run it with 0.9 (meaning it will average from a much longer range). You can see just by looking at the sequence of algorithms that they don't often run too short but there is periods where it diverges more. I think the bigger window will smooth this out, and I will be putting that data set in the repo also afterwards.

I have debug prints of the retarget parameters in the code currently, not perfectly pretty or anything, but it looks like this:


retarget: blake14lr height 36, old 1f1b1fa9 new 1f333493 average 38.79 weighted 43.797 blocks in window: 3 adjustment 1.887848271
retarget: keccak height 36, old 1f1b1fa9 new 1f2e7629 average 38.79 weighted 39.739 blocks in window: 3 adjustment 1.712952696
retarget: sha256d height 36, old 1f1b1fa9 new 1f23131b average 38.79 weighted 5.532 blocks in window: 2 adjustment 1.293137255
retarget: skein height 36, old 1f1b1fa9 new 1f23131b average 38.79 weighted 18.295 blocks in window: 3 adjustment 1.293137255
retarget: lyra2rev2 height 36, old 1f1b1fa9 new 1f23131b average 38.79 weighted 12.725 blocks in window: 2 adjustment 1.293137255
retarget: stribog height 36, old 1f1b1fa9 new 1f23131b average 38.79 weighted 3.957 blocks in window: 4 adjustment 1.293137255
retarget: scrypt height 36, old 1f1b1fa9 new 1f2c8107 average 38.79 weighted 38.065 blocks in window: 3 adjustment 1.640781434
retarget: cryptonight7v2 height 36, old 1f1b1fa9 new 1f223f51 average 38.79 weighted 29.292 blocks in window: 2 adjustment 1.262636357
retarget: x11 height 36, old 1f1b1fa9 new 1f23131b average 38.79 weighted 20.641 blocks in window: 4 adjustment 1.293137255
retarget: stribog height 37, old 1f23131b new 1f2ae479 average 39.43 weighted 27.914 blocks in window: 5 adjustment 1.222891696
retarget: keccak height 37, old 1f23131b new 1f3d1053 average 39.43 weighted 39.739 blocks in window: 3 adjustment 1.740966977
retarget: cryptonight7v2 height 37, old 1f23131b new 1f2d02c2 average 39.43 weighted 29.292 blocks in window: 2 adjustment 1.283285993
retarget: sha256d height 37, old 1f23131b new 1f2e191c average 39.43 weighted 5.532 blocks in window: 2 adjustment 1.314285714
retarget: lyra2rev2 height 37, old 1f23131b new 1f2e191c average 39.43 weighted 12.725 blocks in window: 2 adjustment 1.314285714
retarget: scrypt height 37, old 1f23131b new 1f3a7db1 average 39.43 weighted 38.065 blocks in window: 3 adjustment 1.667615399
retarget: x11 height 37, old 1f23131b new 1f2e191c average 39.43 weighted 20.641 blocks in window: 4 adjustment 1.314285714
retarget: blake14lr height 37, old 1f23131b new 1f434c6a average 39.43 weighted 43.797 blocks in window: 3 adjustment 1.918722861
retarget: skein height 37, old 1f23131b new 1f2e191c average 39.43 weighted 18.295 blocks in window: 3 adjustment 1.314285714


'average' is the all-time average, 'weighted' is the result of the exponential weighting per algorithm, and the two numbers and their ratio to the target (currently in testing for 30 seconds but I am not 100% sure it will stay this way) you can see that in this case the time has stretched to 39 seconds per block, and so all of the adjustments are over 1. The more the 'weighted' value is, the higher, and vice versa.

Tweaking the per-block weighting may improve things, so I am testing several variants. I think 0.9 will probably be quite good, as it should stretch at least to 20 previous blocks. The width of variance this allows depends a lot on this factor. Obviously the ideal is that very very rarely blocks come in at under 10 seconds or over 40 seconds, being the a variance of 33.3% and it does seem to do this quite well, the average rarely goes under 20 seconds and rarely over 40, so this is very much satisfactory for ensuring regular block times.

In the event of one algorithm being mined at 10x the rate of others, the algorithm will aggressively adjust the difficulty for this algorithm. It's important to note that the multiple algorithms are being used as part of the security against block time manipulation. It also acts as a limiter on how well an ASIC can perform on the chain, or more specifically, requiring that the ASIC mines all of the algorithms at the same time. This is the only way to force a shortening of the overall block time that will not spring back hard against it. Complexity, and the requirement of a parallel pipeline for each algorithm should make ASICs less than 20% efficiency advantage, and GPUs are also similarly hamstrung by this pipeline, so maybe they will gain 10% once implemented.

But the main thing I'm interested in is that mining remains accessible to most people, as the power cost and load of this will not add a lot to an enthusiast's computer's load. Yes, I think it would be good to make what essentially could be injected into people's machines to mine by botnet miners.

Botnets will of course be able to mine this coin. But ultimately that's none of our business, and if it's worth coordinating such mischief then it's good for us also, as obviously the coin is in demand. In the future possibly there may be countermeasures against botnets. Botnets tend to have high latency, and I think that 30 second blocks naturally limits how productive a botnet farm can be, as it will tend to have higher latency in its control channels so this means that it will dig on blocks longer than proper miners will, once a new block comes in, because proper miners will have fast, low latency connections.

There is no practical way at this point to further improve the speed of the chain, as the peer to peer network code has to be added to, improved, and the only way I can see to tighten things up is, one: node operators need to make sure their time never drifts more than about 10 seconds from the NTP clocks, really, I would say they don't want to be more than 3 seconds skewed, and the other thing that would be required would be to write new peer gossip network protocol that does not, as it does now, disconnect all nodes at once when moving them around, and the connection should not be a high-overhead (in initiation) TCP connection, but instead, streaming duplex UDP with FEC to reduce retransmit latency, for announcing the arrival of new blocks, as well as an effective 'epidemic' transmission where nodes pick the lowest latency peers nearby to stream their updates to.

I would want to improve this down the track anyway. The same error corrected UDP streaming protocol has of course many more uses beyond accelerating and tightening the convergence of the mempool and new blocks, it is well suited to of course audio and video and instant messaging as well.

I will make a few runs for decent numbers of hours with different exponential weighting factor, but I think that this regime should work quite well.

Oh, so, currently, if I made no further changes, it will average block time based on the first block, directly after the genesis in early 2014, and this means that likely actual block times will be shorter for a long time. I'm not sure if this is a good thing, as 30 seconds is pretty much as tight as I think you can get with the peer protocol they have currently, and with the all-time block time average of 12m45s the new algorithm will pull them down to about equivalent of 2-3 minutes.

I'm not completely decided about this question. I'm inclined towards instead setting the 'all time' starting block to be the first block of the fork. The reason why I am saying this is that naturally if it was based on the original target time, and the reward was computed on height, and not on timestamps (which is a bad idea anyway) then supply will swell at a rate double currently, and this is frankly way too fast. What I think will be percieved as most fair is to start at exactly 2 duo per block from the fork block onwards. I want to constrain the supply rate, I think the current (theoretical) 25% inflation rate is terribly high. I would want it to be at most 10%.

With 30 second blocks, the reward would thus be starting at 0.2 duo per block, and the exponential decay means that always 90% of the supply expansion is in the first year, and this is why I talk about increasing the precision of the token. Recalibrating this, so, for example, to raise the post-fork reward to 2 DUO per 30 seconds would require dividing the pre-fork tokens by a factor of 10 in their current value, but the point is it increases the precision by one decimal place.

So, my current, tentative idea is that indeed I will raise the block reward by 10, as it will fall to the same level in 1 year at a 10% supply expansion rate, but at the same time, it will increase the overall supply. So, it will increase the potential for a bigger sell side, but this would be a temporary situation, as after 1 year it would be at 2 duo per 5 minutes, and in 2 years down to 0.2 duo per 5 minutes. I think the supply is a little too tight anyway, but this way, after the first flush in the first 6 months, supply will be lower, and so, this is a big incentive to put miners on the network in the short term, and this will swell userbase, and so long as we keep doing the right things by our users (miners, traders and e-commerce) we can sustain the userbase as the supply tightens to a rate that keeps in sync better with the economy as a whole.

I will be discussing the subject in the Discord chat, which you can find here: https://discord.gg/nJKts94

Experience and testing are very important... and I don't want to let something on a live network until my confidence is very high, and my criteria are a lot higher than most... so there is a development process, and I am writing about it here in order to allow people a view, as well as, in case anyone wants to contribute, some feedback. But in the absence of that, it is enough to describe what I am doing.

I have spent a lot of hours staring at logs slowly moving up my screen, making changes, resetting the chain, waiting for results... and I have learned many things about different strategies for difficulty adjustment.

One really key thing relating to multi-algorithms is that there can't be total decoupling of an algo from the timing of the rest of the chain. But tying it to the newest block of an algo causes problems of oscillations and keeping them separate leads to oscillations between min diff and ten or more times the target passing before a block can be found.

So, the current thing I am testing combines the two averages.

On one hand, each algorithm computes an exponentially weighted average, based on only the algorithm's blocks. This aims to have the algorithm tend towards one block being of equal distribution, or in other words, it is based on the target multiplied by the number of algorithms.

However, running alone, this leads to the oscillation between stupidly long blocks and too many of one coming all in quick succession.

So now, I am experimenting with adding some weight to the whole chain's block times. Just a simple average of some number of blocks in total, currently about 300. The weighting did not seem to be enough at a 50/50 between one average and the other, so right now I am testing biasing the global block time average as 3 times as much weight as the per-algorithm average, as making it match 1/9th of the average did not respond well. Now I can see also, in my log outputs, the overall average, the per algo average, the divergence and the compounded adjustment. Now when it is over target, the majority of algos are also raising the difficulty, and the ones that are still too short, are lowering it.

It seems to balance better, but I won't know exactly what the best ratios are until I have run enough tests to see how it pans out over a longer time period.

One of the things that I am noticing a lot has to do with the latency of nodes updating other nodes about new blocks. It seems to me that the connection-oriented TCP connections used in conventional blockchains are not nearly as fluid in propagation as they should be. I'm not going to address this issue in this hard fork, as the issue is less important with more hash power and the intervening natural network latency fluctuations, but I for sure would like to see an architecture with full duplex communication by streaming between nodes.

But that's for the future, so I won't dwell too much on that.

Another update, since I've got nothing else to do and no other place to be, and not feeling like sleeping after spending too much time napping today.

I had been observing so far that the simple 'average from last block of algo' scheme was perturbing the average block time, so I am biting the bullet and writing a per-block averaging scheme. But it's not your typical one, and I don't think a single algo difficulty adjustment is relevant to multi algo, even if it is only two, and definitely for more than 3.

So I am testing an exponential weighted average. It still looks only at the previous block per algo, meaning the adjustments won't affect algos that are not before a spike in hashpower, but instead of using an indiscriminate, fixed width window, instead I am using weighting. I haven't quite got the algorithm right yet, as it appears to not be counting as many previous blocks as it should be (and I am getting an odd NaN problem with the adjustments at first), but this is how the scheme is working now:

1. Gather the full set of previous blocks per algorithm, specifically their timestamps.
2. Calculate the time between each of the blocks
3. Adjust the weight of each block time such that they decay at some rate, something like 90%, I think
5. Add the adjusted actual times together, and divide them by the sum of adjusted target times, divide, to get the divergence
6. Plug the divergence value into the 4 term polynomial that defines the adjustment curve, then modify the difficulty.

Even though I haven't quite yet got the algorithm 100% working exactly as described above, it already achieves pretty good convergence on block time target.

Outside of that, I am also encountering issues that I can only assume relate to network timeouts and such on my 3 node testnet. I have reduced the retry period to 1 second from the default of 5, as previously blocks seemed to be coming in at multiples of 5 seconds a lot. It may not be a change that is needed on the live network, but I suspect it might be because 5 seconds was also related to the 10 minute block time and this goes fully 20x shorter than that. Block propagation encounters many bottlenecks due to this, and thus forces also the lengthening of the block time.

Bitcoin's engineers, and more or less any crypto with a big miner userbase doesn't have to deal with this problem. I would love to be a fly on the wall when they are faced with it. In the long run, it's an issue that will have to be engineered into the protocol, but many people in the space protect themselves from this through their sheepish behaviour. Plus, it's not all bad - because the alts that get this right are pretty much guaranteed success, simply by the way that the population of miners shifts from mostly parasites to fans.

Well, it needs some more work yet, I don't think my implementation is correct, or efficient at this point. The thing is not showing me nearly as many past block time samples as I expect, nor are they weighted as I expect. But it's pretty promising so far.

Aside from that, I have a rudimentary randomisation script that enables and disables threads on the miner, to simulate real variance in network hashpower, and each thread automatically randomly selects the algorithm whenever the template gets stale. This will help a lot. My old Ryzen 7 would have been more fun to work with, as this 5 has only 6 cores, but then again, in a way this simulates the fluctuating network latency as caches get full and cause waiting.

I've not really done *that* much today, but I am pretty satisfied with the revised adjustment curve, which is now not flat in the centre, and it converges pretty good on the target, within about 40 blocks from minimum to hitting target on my ryzen5 cpu.

I have now got the randomised mining mode fully working, testing it with 3 nodes in a little testnet, each using two cores each, and cpu utilisation stays around 80%, and there is definitely less cache pressure as a result, so I'd guess the solver is doing more calculations per thread per time. Each mining thread selects a different algorithm, so it is mining with 6, and every time a block is found it restarts and a new set of 6 are used.

The next thing I wanted to look at was eliminating the stair-step halving, and I came up with an exponential decay curve that interpolates smoothly from the original halvings. I estimate that the theoretical supply rate for the token was originally meant to be about 24% increase per year, and due to the small block reward size, I decided not to lower it to a more reasonable 5-10%. I don't think it will be a problem and less tokens can have a negative impact on the market as userbase expands, as it lowers the price precision available.

This is the formula that calculates the reward based on the block height:

// Plan 9 hard fork prescribes a smooth supply curve made using an exponential decay formula adjusted to fit the previous halving cycle
r = int64(2.7 * (math.Pow(2.7, -float64(height)/375000.0)) * 100000000)

https://i.postimg.cc/gkzvCDv9/unknown.png

(note, for illustration purposes I will multiply the height factor by 1000 to show what the reward level is at each 1000 blocks)

The image linked to above shows the existing supply rate, with the orange dotted horizontal lines, the vertical green lines are the halvening heights. The red curve is produced by that formula. You can see that it cuts the orange horizontal lines about the same point each time, and through the green vertical lines also at the same ratio as each previous.

The difficulty adjustment curve used to be this:

adjustment := 1 + (divergence-1)*(divergence-1)*(divergence-1)

It will instead be calculated as follows:

d := divergence - 1
adjustment := 1 + d + d*d + d*d*d

https://i.postimg.cc/7xdjqCst/Screenshot-from-2018-12-23-19-04-52.png

The red curve is the old version, you can see it is very flat between about 0.8 and 1.25. It doesn't always change the difficulty, and as the Digibyte people observed, when devising their 'digishield' adjustment regime, they said that it is important to always move it. So, the purple line represents how it will adjust now, and it will almost always move it at least a couple of bits back and forth.

Currently the dithering basically consists of flipping the last 2 bits of the 'bits' value, representing a difference of about 1/2^21 - very small, and also, maybe not ideal. I am considering changing it to be derived from the last byte of the best block hash, as a signed 8 bit integer, meaning it will vary on average by a factor of 64 (2^6, to put it in perspective with the full precision). I'm not sure if it's necessary, as what happens now is pretty randomised, depending entirely on the timing of the blocks, in the last two bits, 3 becomes zero, 2 becomes 1, 1 becomes 2 and 0 becomes 3. It's quite small, but I meant it to only jitter it a small amount. More than this might make it prone to drift.

The christmas cheer has seen me taking a bit more relaxed work approach in the last couple of days, and as a consequence I rambled back over to the mining algorithms and difficulty parts again, and discovered there is a Go implementation of Cryptonight version 7. It has all three variants, including the new one which I think is yet to go live on the Monero network.

For some reason, I thought the proofs were different, as cryptonote protocol uses different proofs. But the actual hashing algorithm is the standart run of the mill 256 bit hash. So it has been added, and then I looked around a bit more and discovered the Aurora2i Key Derivation Function (Scrypt is a KDF, probably cn7 could be one too, funny enough).

I also played a bit with Desmos to alter the adjustment curve to be less flat in the center. It's being tested now, along with some big changes that mean CPU mining only is going to be the rule for the early days at least, and many impediments to even GPU implementation.

Here's how it works now:

Every algorithm type, being 9, blake14lr, cryptonight7v2, keccak, lyra2rev2, scrypt, sha256d, skein, stribog, x11 also includes a first pass through Aurora, which is set at a buffer size of 64kb, then the algorithm named in that list, except for cn7v2, which does not have another hash required.

The strategy works in this way to impede increasing efficiency - In order to implement this scheme on a GPU, you need to write a kernel that has all the required component hash functions, and it is different for all 9. You could maybe write one kernel with a mode setting, but any way you slice it, the kernel has to be bigger, which is a latency cost for a miner.

Each algorithm requires, in total, about 6 different algorithms, one being different in each of the set of 9, one having one less.

As well as this, to further reduce shortcuts, the block time is being reduced to 30 seconds, which does somewhat increase the chances of orphans being mined, but it also ensures the chain clears transactions quite quickly.

The multiple algorithms also can be mined with an automatic random selection mode, which I will be recommending for users, as this further decreases the possibility of an attacker being able to determine sufficiently quickly which algorithm is the weakest (lowest difficulty), at any given time, as many miners running random selected algorithms on each block round means that each new block has an entirely different and impossible to predict distribution.

So, even if there is someone mining with GPU, they have less of the advantages given by these algorithms, especially cn7v2, which runs best with 2mb of CPU cache per thread. Ryzen processors are pretty much on par with the AMD GPU of equivalent price, when it comes to mining, because the GPU has far less cache and basically cannot mine effectively with all of its cores. The main memory is faster but CPU cache memory is faster than GPU memory.

The requirement of the common standard compute-hard hashes in most of these algorithms in the middle of the pipeline means that the only way to make it more efficient is to set up pipelines so several cores focus on one stage, pass it to another, and then to the final stage. First is Aurora2i, then a compute hard algorithm, and then memory hard cn7v2. This makes the interconnect between stages in such a hypothetical miner a source of serious latency and contention issues, which slows down the solution search in a way that cannot be reduced significantly. On a GPU this would mean you would have to construct multiple kernels with direct interconnect to pass through the result from each processing phase. The CPU will be fastest at cn7v2, an ASIC would do the compute hard hashes better, and probably Aurora (based on blake2b) probably does well on GPU. But as you can see, linking the optimal hardware set in the pipeline, gives you all kind of transport contention issues, how do you stream from a GPU computing Aurora2i, to an ASIC core hashing (for eg, sha256d, or blake14lr), and it then would have to pass its result to the CPU, to make the final cn7v2 hash, and then check if it hits the target.

Obviously, the complexity of this makes engineering a significantly more efficient solver much harder, and compounding this, the random algorithm selection, if used extensively, makes it impossible to have any advantage in selecting a faster arrangement and maximising the latency required for GPU solvers if they try to match this in order to gain an edge.

So, in summary, after hard fork, and for maybe up to 3 months, at least, if not longer, there won't be anyone with an advantage, except maybe people running Ryzen processors, especially Threadrippers, which are basically built to do Cryptonight7 hashes, which would, ideally, and hopefully will always be, the best way to mine DUO after the fork.