Topic: Is it possible to generate a consensus algorithm using machine learning? (Read 435 times)

please download the latest edition of the whitepaper that contains the proof model too:
https://bitcointalksearch.org/topic/shahin-go-round-proof-of-consistency-poco-and-the-ringchain-5066624

==========

and also there are pretty well defined notes about relationship of a successful selfish-mining scenario with 1) the quality of connectivity among nodes and 2) adjustment of difficulty - available here in this study:

http://www.mixoftix.net/knowledge_base/blockchain/dynamics_of_blockchain_2016.pdf

which means other samples that we had above could indirectly help preventing selfish-mining..

Could you share more details of it?

Totally true.. the topic is about ML and it was about giving some touchy samples.

but the way that we conclude the results would be different from the way that an AI-Agent may does. I mean these all depend on the quality of DSS module that we provide for this part of the protocol. depend on what we saw in the example of litecoin, when the AI-Agent understands the series of incoming nonce values e.g. contain digit 78 at their right side (without any special reason or flaw in hash algorithms / just based on an accident), this opens two situation ahead:

1- our miner node could rapidly try next nonce values with digit 78 at their right side.. (to over come other miners)
2- our miner node gets suspicious that is working in a different fork.. (the protocol may save a timestamp of such security/fraud reports in a parallel blockchain and then punish the selfish nodes)

I have told some pools already do such calculations to find these patterns in nonce values, the least advantages of including such rules and an AI-Agent to the block header, may bring more fairness to the whole ecosystem..

============

P.S.:

AI in Association rule learning talks about this ability in DSS: "Association rule learning is a rule-based machine learning method for discovering interesting relations between variables in large databases. It is intended to identify strong rules discovered in databases using some measures of interestingness."

https://en.wikipedia.org/wiki/Association_rule_learning

============

UPDATE:

in my project I put all generated transactions inside a virtual ring among candidate nodes and ask them to introduce their candidate block - then other nodes could immediately see the fork(s) and wait for the result of finalizing procedure that broadcasts the confirmed block. so this bounce may just fit into that new data/proof model, not classic PoW.

To be honest I think that these patterns, when they occur --- which seems not to be the case with Bitcoin, according to the post linked above -- are implementation dependent, rather than an inherent property of the hashing algorithm. If the nonces were indeed biased (ie. some nonces having a higher chance of being correct than others) we'd actually look at at a broken cryptographic hashing algorithm, as by definition there should be no bias in its outcome no matter how you look at it.

TLDR; Unless SHA256 is inherently broken you will be unable to find a pattern in nonce values that enables you to speed up the mining process.



Even assuming SHA256 (or Scrypt, or the hashing algorithm of your choice) were broken -- how would filtering nonces like this prevent selfish mining?

Filtering like this only works if everyone is aware of the rules. If everyone is aware of the rules, so is the selfish miner. If the selfish miner is aware of the rules, he can apply them just as well.

But, to reiterate: Unless SHA256 itself is flawed, nonces are statistically random without bias. The only thing one could learn from such a data set, is what bias each respective miner implementation brings. However that doesn't help improving ones own mining performance.

while you can get your answer, why does it matter for you? 

Am i looking at a machine talking or you're really a person  ?

the main idea was about finding a pattern in nonce values, then a miner could speed up his mining process. for example, what if we know 90% of nonce values in mined blocks already are EVEN and only 10% of them goes for ODD values? then I found this link that contains more detail information:

https://bitcoin.stackexchange.com/questions/24650/looking-for-nonces-of-even-numbers


therefore after difficulty target, I decided to exclude/enforce some patterns in block header too.. for example, sometime nonce values CAN NOT be even , sometimes nonce values SHOULD be EVEN, etc. and I have called these sort of conditional-nonce-values as BOUNCE..

Thank you for taking some time out to help me with this @mixoftix and @HeRetik. I get where i am going wrong and would go back and try finding solutions. Wouldn't give up on it though

Edit


As long as the format remains the only thing that matters is the amount of computational power (PoW) or the amount of coins (PoS) you have. But once you change the format it would become very hard for you to change the requirements to mine more.

But in the end there still needs to be a consensus about the state of the ledger, right? What if you simply flood the network with malicious nodes? There doesn't seem to be any way to prevent sybil attacks in this scheme. If the majority of the network is malicious, there's no way for nodes to know the "correct" behaviour. And flooding a network with malicious nodes is fairly trivial if the economic cost is negligible (unlike proof of resource schemes such as PoW or PoS).


Edit:


Neat examples, especially #3 makes me curious.

Could you elaborate on that? How would changing the nonce format help prevent selfish mining?

I afraid not. as HeRetik said above, you are going to centralize everything.

When we talk about AI, this means we are talking about a Decision Support System (DSS), not Decision System. don't go back to the idea of giving the whole consensus responsibility to the AI - this is just another kind of centralization and doesn't work.

and if you are going to work with some artificially-living nodes in your laboratory for studying something about Well-Known Algorithms, after training an entity you should let them work and never stop them in run time by any means - otherwise, you just kill your study. you need to see how many times of how much traitors could break down the consensus model and what would be their achievements. if you kill them, you will never understand your answers..

==========

BUT, instead of that, you try to use AI in enhancing the consensus protocol in run time. for example:

1- WHICH nodes would be better for my FULL/SPV node to connect? parameters: ping time, up time, etc..
2- WHAT difficulty target would fit better for the next block? parameters: distribution of hash power among pools, a disaster close to one major pool, maintenance in pools, changing regulation policies regarding to cryptocurrencies in one country, etc..
3- HOW we could arrange a format in nonce values that protect the network from selfish miners? statistically analysis of nonce values in
history of blockchains.. [I use this one in introducing BOUNCE value instead of classic nonce]
4- even WHEN will be the deadline of planning a hard fork for the network? parameters: number of nodes that follow the network, number of miners, volume of trades, destination of payments, etc..

these are living-problems that you could use DSS to better identify them..

The thing is each node makes a decision on it's own and it doesn't know what's happening with the other nodes. It wouldn't even know that there are other nodes. It would just send out it's decision based on the data that is fed into it so i don't see how it would turn bad!

That sounds like we'd be back to square one though.

How to determine the majority (ie. how to prevent the majority vote from being manipulated)? How to tell "good" AI containers from "bad" AI containers?

What if we divide the administrator power equally? The decision would go through if only if majority of them accept it's correct. We could also process the data in multiple containers so that not one entity has control over the process.

That sounds like the end result would be a centralized administrator node run by an AI, rather than a decentralized consensus scheme? Or what am I missing here?

That's very interesting to hear. How about this?
The neural network takes decision based on weights that is given to each and every input. This is what i was thinking when i posted this. I'll just write it down in steps, Please let me know if i'm correct.
 
Step 1 : So at the beginning we set the same amount of weight to all the inputs( Taking good nodes and bad nodes as the inputs).

Step 2 : We make this model the administrator of the network which has equal amounts of good nodes and the bad nodes. and there is a pool of other nodes(good) which could be introduced into the network based on the actions in the network.

Step 3 : Now, We just let the network run. The algorithm is written in such a way that once a bad node tries to manipulate some node he's immediately replaced with a good node from the pool.

Step 4 : We could also give weights to attacks. If it's irrelevant and if it's making no much damage let's just keep the node. If it again repeats the same thing the node would be replaced.

Step 5 : For scalability we could just choose a particular set of nodes and simulate a tremendous amount of transactions and pin point to what kind of nodes are causing this and we could drastically reduce or try applying compression algorithms to that particular node.

I don't know if what i'm saying makes sense. This is what i've been trying to build from the past week

Yes, this is interesting. in theory we could train one miner or node entity as honest and another one as traitor - then SAVE their mind (trained NN) somewhere and CLONE them as much as we need for the sample algorithm in different environmental incentives.

BUT, the applied version of including NN in crypto world could happen in client applications for FRAUD DETECTION purposes.. specially when they run in SPV mode and they are more vulnerable to attack..

Your perspective is interesting, and even as a non-expert I understand the problem that we maybe do not really understand what's going on inside the "black box". So machine learning may not be (in its present state) the way to "create" a totally new algorithm.

But it could be interesting not to let the NN "create" the algorithm but to use it to select an algorithm from a group of "humanly constructed" algos, creating simulations in the same way AlphaGo or AlphaZero did, only using Bitcoin as the "game" and the attacker(s) as one party and the honest participants as the other one.

The idea would be to simulate different complex attacks, and then compare the values of the different algorithms. This way, one could create inputs based on the incentive structure one wants to achieve, and play around with the values. One of the NNs, for example, could simulate to be an attacker using a "51% + short-sell" attack with different algo configurations, trying to profit from it, with several other NNs playing the roles of different market participants.

your topic subject was interesting - refer to engaging ML in consensus process, but I found nothing about it in content. so I really need to focus to the subject and discuss it more in depth. do you know, the human brain is the best practice that ever developed about node-consensus in the universe? every decision that we make as human, is coming from a perfect consensus algorithm that we try to emulate it by Neural Networks (NN) in AI. just with Bio-NN there is always one stake-holder (the owner of the brain) that the whole process belongs (is centralized) to him, but in Artificial-NN the outcome of the whole process has its own impact on all stake-holders and this makes it complicated.

NN always provides a Black-Box processing unit, so nobody really could understand what would be the next step of one person and if you put a cryptocurrency in the hand of an Artificial-NN, you may have a blockchain that after many years, somehow may gives its vote to a traitor node and nobody could stop it!!

BUT, you could learn something really useful if you ask: "what would be the main characteristics of an advanced consensus algorithm?". so, just simply take a look at the human brain, which is drastically DECENTRALIZED and powerful by its PARALLEL-PROCESSING-UNITS that SHARE their jobs (not in the race of "who is doing it first..") including HUGE-STORAGE-SPACE and MULTI-ROUTES among nodes including HIGH-BANDWIDTH between them, with the least POWER-CONSUMPTION rates!

because of the black-box part that I have mentioned above, we can not provide a consensus algorithm that works fully by ML, but based on lessons that we could learn from human brain, now we know what kind of characteristics for a consensus algorithm we should look for.

Why?

If it's hard fork no one cares about the market impact is close to nil.

If there's a market impact then people care about the hard fork, in which case we can assume that there's likely some merit to their approach.

Historically speaking hard forks caused price increases at least as often as they lead to price deprecation, if not moreso (e.g. the impending BCH and B2X hard forks lead to a run up on BTC's price, the impending BSV BCH hard fork... not so much). In conclusion the market impact is pretty much unpredictable, if there is one (which in the case of the numerous Bitcoin hard forks was relatively seldom).



If a committee of people could decide on who forks off and who doesn't we'd lose a major aspect of the freedom and permissionlessness of cryptocurrencies. No one is in control, and that's the whole point of it.

In practice it's the free market as an extension of our community that decides which fork becomes successful and which doesn't. You don't need a committee for that.

Note that I'm talking about "independent" hard forks. "Sanctioned" upgrade hard forks are a wholly different thing (e.g. the one that Vitalik has in store for Ethereum). In this case you don't just "randomly hard fork" off the main chain but rather have a developer team that orderly decides on a course of action (e.g. the switch from PoW to PoS) and -- optimally -- a community that supports this decision. Or not, in which case parts of the community can fork off and see whether their approach is more viable.

Won't forking the chain randomly decrease the value of a particular asset? Don't you think we should have like a committee of people who gets to decide if a blockchain has to be forked or not?