Topic: Bamboo: New crypto using ED25519 signing keys - page 8. (Read 4640 times)

Sorry about the windows stuff. The C++ compiler on windows (MingW) is really hard to get working. I've hired some expert devs on Upwork but they still haven't gotten back to me.

Telegram: I have downloaded it. You can reach me @mrpandabear7 -- I'm not familiar with the app, I'm assuming I need to make an open group that allows invitees to invite others?

EDIT: The group is available at t.me/teampandacoin Thanks @Mando

If anyone will have problems with LevelDB on Ubuntu 20.04 you can install it  by

sudo apt-get install libsnappy-dev

export VER="1.20"
wget https://github.com/google/leveldb/archive/v${VER}.tar.gz
tar xvf v${VER}.tar.gz
rm -f v${VER}.tar.gz


cd leveldb-${VER}


make
sudo scp -r out-static/lib* out-shared/lib* "/usr/local/lib"
cd include
sudo scp -r leveldb /usr/local/include
sudo ldconfig

This (Discord mostly). + Create a telegram group pls, so ppl can reach you dear Dev. 

Nice stuff! I'll get this merged in. Great point regarding not needing to generate an entirely new random solution each time.

Have no time atm, here is the code I have chosen for mining:

SHA256Hash mineHashShifu(SHA256Hash target, unsigned char challengeSize) {
    vector concat;
    concat.resize(2*32);
    for(size_t i = 0; i < 32; i++) concat=target;
    // fill with random data for privacy (one cannot guess number of tries later)
    for(size_t i=32; i<64;++i) concat=rand()%256;
    while (true) {
        *reinterpret_cast(concat.data()+32)+=1;
        SHA256Hash fullHash  = SHA256((const char*)concat.data(), concat.size());
        bool found= checkLeadingZeroBits(fullHash, challengeSize);
        if (found) {
            SHA256Hash solution;
            memcpy(solution.data(),concat.data()+32,32);
            return solution;
        }
    }
}

So I do not call the concat function but manipulate right in place, furthermore one does not have to update every entry to get a new hash, I just directly write to the buffer using reinterpret_cast. 64 bits is enough for now but later at some point one has to also manipulate other bits (at every overflow of *reinterpret_cast(concat.data()+32) one should increase *reinterpret_cast(concat.data()+40) by one, then one has 128 bits. This can then be continued to get more bits changed when mining becomes that difficult. So this code is just for now because it only changes 64 bits.

Maybe I have time again in a few days to watch the progress of this project.

When windows miner and discord community channel ?

I get this
But LevelDB is installed. I'll try to build the miner from source.

EDIT:

Works 

You should just be able to download and unzip the binaries and run them.

If you just run "make miner" instead of "make" it should do the trick. That benchmarking code is cruft I need to remove. I'll update the build instructions on github.

I'm trying to compile from source and get a few errors like this one



How to use binaries on Linux?

  Works for me.

EDIT:

I have uploaded new binaries with a more performant mineHash function. If you are mining I highly recommend updating to this version as you will not be competitive in the mining pool with the old version... you might occasionally win a block but your chances are lower than if you have the fast hash version.

Furthermore, now the miner will constantly check to see if the block it is mining has already been solved, and will give up instead of wasting work.

Both these changes will increase the overall hashrate of the network.

Binaries:
https://github.com/mr-pandabear/panda-coin/releases/tag/v0.2-miner-alpha

Source:
https://github.com/mr-pandabear/panda-coin

Sure but is it OK for you if I reregister as Master Shifu then?

Good point. Blockheaders should do the trick.

Could you share some of the mods you made to improve mineHash? I changed concat hashes as follows and it got a lot faster:
SHA256Hash concatHashes(SHA256Hash& a, SHA256Hash& b) {
        char data[64];
        memcpy(data, (char*)a.data(), 32);
        memcpy(&data[32], (char*)b.data(), 32);
        SHA256Hash fullHash  = SHA256((const char*)data, 64);
        return fullHash;
}

Funnily enough, Kung-Fu Panda was the movie that inspired my github handle


@Salamande: I found the culprit of the inaccurate mining fees displayed on the miner:
https://github.com/mr-pandabear/panda-coin/blob/0641629b9ea6e195b99cc8b65e099d76ad934fe8/src/tools/miner.cpp#L65

The miner records payments for any solved block, even if it's not accepted. This means we are counting fees that are earned by rejected blocks which would obviously make the total displayed on the miner much higher.

I will be pushing an updated miner with a faster mineHash function and addressing danglingnullptr's observation that miners should "give up" on blocks once it is impossible for them to be accepted.

Couldn't help it : https://www.youtube.com/watch?v=4uP7O2mM10A&t=18s

8 ) Concerning the consensus problem: The way to go is to separate header and block downloads. Chain of headers are downloaded from peers to a) known who has the longest (most work) chain.and b) to known which peers have which blocks corresponding to this longest chain. Furthermore bad actors need to be banned based on IPv4. Enabling IPv6 is more difficult because there are so many of them, one approach is to block IPv6 subnet wise.

9) I do not completely oversee if MPFR will solve all problems. It is better to do this retargeting computation without floats similar to bitcoin. It is based on 256 bit numbers that are composed of multiple 64 bit numbers.You will need such large numbers for the concept of total work anyways. Programming such a retargeting function without floats is time-consuming but I think it is the only way.

10) The miner continues to mine if others find a block first (because the miner is not informed). This is really unfair for people with smaller hashrate because their mining problem is updated less frequently (only when the mineHash function returns https://github.com/mr-pandabear/panda-coin/blob/a1291fd9c8713818b8ad522e646a1b62a270e7e7/src/tools/miner.cpp#L71) and therefore they mining an already mined block most of the time (more % of time than faster miners). This gives then an additional disadvantage to their bad hashrate.

11) The mineHash function is inefficient. I have easily achieved a 10x speedup of this function in debug mode by rearranging some code. I could also turn on optimization flags, and other tricks to further increase my advantage (if I wanted mined this coin, maybe later). Please optimize this function a bit. But a good thing: One cannot use stratum protocol miners for this coin due to the custom mining function. So no one can just rent SHA256 hashrate.

I wish you all the best to become the dragon warrior but I cannot further contribute because I have to focus on other things.

For all intents and purposes, this is a "live chain". The only restriction that we have now is that only people we know are running nodes exactly so that we can discover any flaws prior to a broader open launch. Distributed systems without trust are hard to get right and this is a brand new codebase (not a fork of bitcoin)

We aren't treating this chain as a testnet or anything like that (for instance when we forked the chain we copied back the mined rewards into the genesis block of the forked chain).

The main goal of the coin is to prove we can do faster transactions with lower fees by optimizing things like the signature scheme, mempool synchronization, and block mint time. Bitcoin has a 10 minute block mint time and 1MB limit of data per block. We have a 3MB max datablock size but blocks are minted every 30 seconds... so theoretically we should have ~60x the transactions per second of Bitcoin. The problem of course is that these blocks must spread through the network quickly, we chose 3MB because this seems like a reasonable amount of data to transfer with high speed (~1-2 sec). The larger problem was actually verifying all the signatures within the 30 second window. With 10k signatures the Bitcoin signing algorithm (SECP256k1) took almost 17 seconds on my machine to verify. With ED5519 that dropped down to a second.

@danglingnullptr: I see your point regarding the consensus problem. It seems the only real way to fix this is to download multiple copies of the blockchain from each host and verify them in parallel, then reject the ones that are fake or have less POW than other chains. What do you think of that idea?

Regarding the float math issue: My thought is to use MPFR and run the float computations on non-native floats : https://www.mpfr.org/

From what I understand the coins will stay as a reward for helping in secure chain and tests

Thanks for the lookup.

Just a question, for now is it just for testing, or you will launch the coin later on that blockchain, or restart over?

What do you want to acheive exactly as a new coin?

Thanks for the tip. Here are some more thoughts:

5) The hash depends on sizeof(int) which is also platform dependent: https://github.com/mr-pandabear/panda-coin/blob/0641629b9ea6e195b99cc8b65e099d76ad934fe8/src/core/block.cpp#L176
6) concerning 1) The fix is not super easy because you have to first rewrite the difficulty adjustment it in terms of integer computations such as in bitcoin's code and then verify that your current chain still satisfies these changes (if it already retargeted until now which I did not check)
Please use uint32_t or uint64_t instead of int for 5) and 6) to have no platform dependence.
7) concerning 2) The delta idea won't help because an attacker can read the code and still lie to have a longer chain within the delta range in order to be selected as block download source.

Thanks for your insights. I'll work to address these this weekend.
1) This was a definite oversight... having different difficulty results computed by different nodes will definitely lead to problems when nodes run on multiple platforms out of control. The fix is pretty simple though.
2) You are right, I can implement a consensus mechanism to look at the total chain work across multiple nodes and reject nodes which have more than some delta from the group.
3)  This is a great find and also hopefully a quick fix: I can change the endpoint /block_count to /total_work and do the chain selection that way.
4) I don't think this is that big of a deal -- the nodes have timeouts and the host manager randomizes the peer node. I agree that introducing lots of dead nodes would seriously slow down because each host is read synchronously. Changing these requests to run in parallel should fix things.

EDIT: @danglingnullptr : sent you a tip, really appreciate your work!


Also just an FYI: These exploits necessitate that a malicious node can add itself to the network. Right now this is impossible because we control the master host list -- this means in the short term mining the coin is still safe. We just need to make the above changes before allowing anonymous actors to run nodes and manage hostlists. This will happen only once the network is more battle tested

@Salamande,

I ran all the mining transactions through my checker and your balance seems correct. The issue is probably the miner's display (?) but I'm not sure how it would be different. One thing to note is that it appears your miner was running prior to the fork -- because you have a total in the genesis block. Unless you restarted the miner the totals on the miner would be wrong by however many blocks you mined while the chain was in an inconsistent state. If you have the raw amounts the totals are off by I can take a look to see if it is reasonable. It shouldn't be more than 20-30 blocks because that is the amount of time the network was down prior to being restored.

Full list of transactions:
https://pastebin.com/dETiqZTV

There are several problems with the current codebase:

1. The difficulty rebalance is based on floating point arithmetic and therefore platform dependent and not portable. Furthermore compiler flags such as --ffast-math will affect the difficulty computation and the log2 function used may also yield slightly different results on different platforms depending on its implementation.
https://github.com/mr-pandabear/panda-coin/blob/0641629b9ea6e195b99cc8b65e099d76ad934fe8/src/core/blockchain.cpp#L185-L187
2. The longest chain is chosen based on the chain lengths reported by the peers. A malicious peer can just lie and bring the whole program down by then sending garbage data.
https://github.com/mr-pandabear/panda-coin/blob/0641629b9ea6e195b99cc8b65e099d76ad934fe8/src/core/api.cpp#L11-L15
3. The longest chain is defined in terms of number of blocks which is a serious mistake, it should be based on the total work.
https://github.com/mr-pandabear/panda-coin/blob/0641629b9ea6e195b99cc8b65e099d76ad934fe8/src/core/host_manager.cpp#L81
4. The peer communication is based on HTTP requests and is not async. A peer that does not respond or purposefully delays its response can disturb the node.

I want to inform you all about these severe problems with the current codebase. I could easily write a program that brings the whole pandacoin node down by exploiting above flaws 2. and 4. The other flaw 1. will manifest sooner or later and 3. is a total nonsense, I could just solo mine a longer chain in terms of number of blocks by keeping my difficulty low by manipulating timestamps. Then I my chain has less total work but more block count and it will be accepted and wipe the real chain. Yay, all coins are mine then.

Please send some monero tip money for my effort to review the codebase: 86Juw7yZ313E8u2nfHBqKGGLKmki3Tg41CAZjwCTgNpk2DqqAMSR2d3KEwu6evrkMQ3eLzwbEfx9thL aYud3JgC7NxTf5eM

Thank you, now everything works