Topic: Technical questions about blockchain (Read 1545 times)

Because the hash has to be lower than the target value.

During the time in which that block was mined, the target was:
0x000000000000000000c1bd000000000000000000000000000000000000000000

And this value:
0x0000000000000000008664c500e67e68ecc775868eaad375183a9b92b121abf1

is less than that target.

Note that the following value:
0x000000000000000000e673b61ff56f79fbd866759f9be284292b8ca3a230bce0

has the same number of zeros, but is NOT valid, because it is higher than the target value.


Full nodes validate transactions and then send the transaction to all connected peers that have not yet received the transaction.

This is why you see a transaction that was sent to you immediately, before it is confirmed.  The sender didn't necessarily connect directly to your node (how would they know where to connect to?).  Instead they sent the transaction to the peers they were connected to, and those peers sent it to the peers they were connected to, and those peers sent it to the peers they were connected to, and so on until it got sent to a peer that you are connected to, and that peer sent it to you.

Note that this is also how the miners and mining pools find out about the transactions so that they can add them to their blocks.  Nodes and wallets don't need to know the IP address of EVERY miner on the system.  For that matter, they don't need to know the IP address of ANY miners on the system. They just send the transaction to the peers they happen to be connected to.  The transaction gets relayed throughout the network and eventually miners hear about the transaction from the peers that they are connected to.  Then they can add the transaction to the block they are working on if they want to.


No stupid questions.  We aren't born understanding the Bitcoin protocol.  It's something that we learn.  Sometime we learn it through reading documentation or the source code itself, and sometimes we have others clarify and explain the pieces that we don't understand.

I would suggest, however, that you take some time to read about the protocol.  If you ONLY learn from the things that you think to ask, then you are NEVER going to learn the things that you aren't aware that you need to ask about.

Since you are a "software developer since many years", then I STRONGLY suggest you take a look at the source code of a full node implementation.  You'll get a much better understanding of exactly what it is doing.


You are saying that it took 7 seconds to mine a block, and then after increasing the difficulty you were able to mine the blocks even faster (2 seconds)?

There are a couple of possibilities that come immediately to mind here...

1.  Perhaps you are miscalculating the amount of time that your software spent actually calculating hashes?  If you are including the amount of time that it takes for your software to load up into RAM and to build the block, and to output that block to you, then you are including a LOT of variable time that could be different every time you run the program.  On such a small timescale those variable times have a larger influence on total time than the block hashing has.  Possible solutions would be:

• Try timing how long it takes to solve 2000 blocks, and then adjust your difficulty
• Start with a MUCH lower target so that hashing is a larger percentage of the processing time
• Iterate multiple adjustments to see if it eventually starts growing appropriately as hashing becomes an ever larger percentage of processing time

2. You are not properly comparing target to hash?  It is possible that you have a bug in your code that is causing it to exit your hashing loop before it has found a low enough hash value. Possible method to determine this would be:

• Print out both the target and the solution hash when your software finishes processing a block so you can compare the values yourself and see if they are correct

Hello Again,

So Danny, you said that miners don't search hashes with 0 at the start, but why then all block hash have 0's before? for example:

https://blockexplorer.com/block/0000000000000000008664c500e67e68ecc775868eaad375183a9b92b121abf1

And:

When a node receives a transaction, if the transaction is correct, does the node broadcast the transaction? before being added to a block? or the node just wait till resolving the block and broadcast the block ?

Thanks man, sorry for my stupid questions... I thought that i knew how bitcoin architecture is, but every time I've got more questions, I'm software developer since many years, and this project is for pure learning purpose, so I just want to try to make this the most "correct" possible.

I have problems with difficulty adjustments


Following what achow101 has said to calculate time for target:

"you multiple the last block's target by the amount of time it actually took, then divide it by the time you want it to take."

Following this, I have:

lastTarget = 2^256 = 115792089237316195423570985008687907853269984665640564039457584007913129639936

newTarget=(lastTarget*7)/120 = 67545385388434447330416407921734612914407491054956995689683590671282658956 (where the 7 is the time it took to mine 2 blocks)

difficulty = newTarget/LastTarget = 0.00058%

I'm getting a new target, but it's too easy... I can mine with this target in less than 2s, it should be atleast 1m or so...

Best regards

In that case you were a participant in a mining pool.  The pool operated the node for you.


Technically, the pool was mining, they were just using your hash power (with your permission) to do it. You were just renting the hash power of your computer to the pool.  The pool then paid you for the use of your hash power.  This is why you got paid even though you never solved a block yourself.


It's your software.  You can do whatever you want with it.

It's possible that your ideas will be better than anything anybody else has ever thought of.

It's also possible that your ideas will be much worse than existing coins such as Bitcoin.


In my personal opinion, Bitcoin is the ONLY viable cryptocurrency.  The rest are all either scams or interesting experiments.

However, there are many people that have different opinions than mine.  There is no "official" business or agency in charge of setting standards. There are some common design patterns, but if you can come up with an idea that you want to implement, then you can implement it.


The data that is being hashed is the block header.  Therefore, the header MUST be constructed before you can start hashing.

The header includes a merkle root that commits the transaction list to the header. Therefore, the merkle root MUST be constructed before you can construct the header.

The merkle root is constructed from the block transaction list. Therefore the block transaction list MUST be constructed before you can construct the merkle root.

Once you have constructed the transaction list, merkle root, and block header...  You have constructed the block header.  There is nothing left to do.

Thanks a lot to all.

So, as I understand from this post:
https://bitcointalksearch.org/topic/difference-between-miners-and-nodes-1734235

Miners are nodes but they do additional tasks.

But how can this possible, I have mined bitcoins using a client that has less then 20mb of installation size, the blockchain is around 165.30GB

Between, as it is my coin, i can do it the way I like, or just try other methods, right?

There is no "cryptocurrency standard" or is there any?

And about "miners usually construct the block and send it back" Why this way? and not:

The node sends the miner a hash (target) and difficulty and the miner finds a hash that fits the difficulty and send it back to the node, and the node then constructs the block.

Best regards!

Usually mining software connects to a miners own node, not just any node. Any node likely does not have the required interfaces exposed to the general internet so you can't just connect to any node to get the information.

No, that's not how mining works.

The mining software will ask the node for the information necessary to construct a block. This is usually the block version, current difficulty, previous block hash, and branches of the merkle tree (including leafs, i.e. transactions themselves). The mining software then constructs the block by assembling the block header and filling in the details that it needs to, e.g. the coinbase transaction with the payout to his own address. When the mining software has completed a block, it then submits the entire block (header and transactions) to the node. It does not give it a hash or many hashes and an address; it gives it the full block usually.

I highly suggest that you read the source code in Bitcoin Core that calculates the difficulty for each block: https://github.com/bitcoin/bitcoin/blob/master/src/pow.cpp#L13

To calculate the next proof of work, you multiple the last block's target by the amount of time it actually took, then divide it by the time you want it to take. You have a formula that looks like this:
Then you just solve for X. In your example, you would do
Because we are dealing with integers here and not decimals, this is just the floor of that, so your target would be 1985007244068277635832645457291792706056056879982409669247844297278510793827474 .

Ok, right, yes there are no servers, but in solo mining, the mining software (the client) must connect to any available bitcoin node to ask for difficulty and a challenge, the miner search a hash that meets the requirements and send it back to a node or many with the hash and his address.

Between, I cant figure out how to adjust the difficulty, right now I implemented the way you told me,

A node sends the miner a target, and the miner has to find a hash below the target.

So just for testing purpose my current target is:

target = 115792089237316195423570985008687907853269984665640564039457584007913129639936 (2^256)

Every 2 block i adjust difficulty, for every 2 block it should need 2 mins, so my ratio would be:

ratio = 2/120 (0,016666667)

So, right now with my current target, 2^256 block are created instantanly, the time what i need to press "mine" button, 2 block are created in 7~ seconds, so actual ratio would be:

actualRatio = 2/7 (0,285714286) ( a lot easier )

At this step i can't figure out how to relation ratio and actualRatio and target to set target to my ratio, to adjust the difficulty.

What do i need to adjust the target with ratio and actualRatio?

Any suggestion?

Many thanks!

That website is wrong.

There is a LOT of incorrect information about bitcoin on the internet.  You should be careful about what information you trust.



If you are mining in a mining POOL, then you will configure your software so that the pool server and your client software agree on the difficulty target that you will work with.

If you are solo mining, then there is no SERVER.  Bitcoin doesn't have any servers, bitcoin is a peer-to-peer system.  In the case of Bitcoin, each node calculates the difficulty target themselves.  They are all using the blockchain as the source of their information, and they are all performing exactly the same calculations, so they all end up calculating the same difficulty.

The difficulty of a current block can be calculated from the "bits" field in the block header.

Here is a more reliable source of information about the difficulty target:
https://en.bitcoin.it/wiki/Difficulty

Hello! Thanks again,

So I think I understood now, but i found this article:

https://samsclass.info/141/proj/BitDiff.htm

where it says:

"Bitcoin miners must find a SHA-256 hash that starts with many zeroes"

So I'm confused now, i though you told me that the server sends you a challenge (a hash), and the target,  and you (the miner) have to find:

if (sha256(challenge + randomStuff) < target)

Best regards!

You can't use nodejs integers (such as returned from the parseInt method) with SHA256, because the maximum integer value that nodejs can safely hold natively is 9007199254740991:

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number/MAX_SAFE_INTEGER

You'll need to include some big integer module if you want to literally compare values as decimal integers.

That being said, SHA256 has a maximum value of 115792089237316195423570985008687907853269984665640564039457584007913129639936

So, a target of 57896044618658097711785492504343953926634992332820282019728792003956564819968 would give every hash a 50% chance of being successful.

A target of 11579208923731619542357098500868790785326998466564056403945758400791312963994 would give every hash a 10% chance of being successful.

A target of 1157920892373161954235709850086879078532699846656405640394575840079131296399 would give every hash a 1% chance of being successful.

A target of 1157920892373161954235709850086879078532699846656405640394575840079131296 would give every hash a 0.001% chance of being successful.

Perhaps you can figure out how to do that math?

Hey! Thanks again a lot!

I understand better now, so this would be a basic implementation of Bitcoin's HashCash adaption:




But I doesn't understand how to calculate the difficulty, so for example, I start with this target (it's the start ):


target = 9999999999999999999999999999999999999999999999999999999999999999999999

How could i calculate the difficulty/probability? I dont know the max integer outcome of:
let responseAsInt = parseInt(response, 16);

Between, to do it more difficultier to mine, would i just remove nine's at the end, or just rest, for example:

999999999999999999999999999999999999999999999999999999999999999999 (removed for 4 nine's)

or

9999999999999999999999999999999999999999999999999999999999999999990001 (removed 9998 )


Should have the same effect, but what is better?

Thanks a lot really!

Effectively, yes.

Technically, bitcoin uses a 32 bit number to store the target. So, it doesn't have a full range of 2²⁵⁶ target values.  Each miner and node uses the same formula for calculating what the target value is from the 32 bit representation.

You can read the technical details at the following link to learn about the conversion from the 32 bit "nbits" value stored in the block header to the actual target value:
https://bitcoin.org/en/developer-reference#target-nbits


The target is not "random".  The target started a a minimum value set by Satoshi.  At that point, there wasn't enough hash power on the network yet for blocks to be completed every 10 minutes, so mining was a bit slower in the beginning.

Once every 2016 blocks, EVERY node, EVERY solo miner, and EVER mining pool performs the exact same difficulty re-target calculation for them selves.  Since they are all working with the same data, and they are all doing the exact same calculation, they all end up at the exact same answer.

The process is effectively to do the following:

• Look at the timestamp of the current block.
• Look at the timestamp of the block that was created 2016 blocks ago.
• Calculate the difference between the two timestamps.
• If that difference is less than 20160 minutes, then the blocks are being created too quickly.  Calculate the ratio between the actual difference and the intended 20160 minutes.  Adjust the difficulty target to be more difficult by that same ratio to get the average time between blocks back closer to 10 minutes.
• If that difference is more than 20160 minutes, then the blocks are being created too slowly.  Calculate the ratio between the actual difference and the intended 20160 minutes.  Adjust the difficulty target to be less difficult by that same ratio to get the average time between blocks back closer to 10 minutes.
• If the calculated adjustment ratio is more than 400% then only adjust by 400% this time, and check again in 2016 blocks.

No.

Miner has to find hash:

If SHA256(currentBlockHeader) < target
  proof = currentBlockHeader

The block header for bitcoin contains 6 pieces of information:

• 4 byte version number
• 32 byte hash of previous block
• 32 byte merkle root (commitment to the block transaction list)
• 4 byte timestamp
• 4 byte nbits (difficulty target representation)
• 4 byte nonce

Since each nonce value has the same probability of success, typically the miner would try all 4,294,967,296 possible nonce values in order rather than trying random nonce values. If they try all possible nonce values and they haven't found a solution, then they need to change something else in the header (which would allow them to try all 4,294,967,296 possible nonce values again with the new, different, header.  An easy thing to change is the timestamp, however they could also change the transaction list (which would result in a new merkle root), or the version number.


It has a relation to the amount of time that it takes to create 2016 blocks.  The difficulty is not adjsuted after every block.  It is only adjusted once every 2016 blocks.  It is adjusted to keep the average amount of time between blocks close to 10 minutes.

So, bitcoin basically says to the miner,

this is the target: 5000 (between is it a random number?? that the node generates?)

And the miner has to find a hash also:

if sha256(prevHash + randNumber) < target
   proof = randNumber

Ok,  where do target comes? is it a random number or has it some relation with previous blocks?

Note that the concept of requiring "consecutive 0's" is a generalization that is often used when introducing the "proof-of-work" concept to people.

In reality, bitcoin doesn't require a specific number of 0's.

Instead, bitcoin sets a target value, and requires that the value of the hash be less than the target.

The result of SHA256 is NOT a string.  It is a number. That number can be represented in binary, hexadecimal, decimal, or any other base that you like.  When it is represented as a hexadecimal number, it looks a bit like a "string" because it has the letters A, B, C, D, E, and F in it.

Because it is actually a number, which is just being displayed to you in hexadecimal (or base₁₆), it can be easily compared to a target value to determine if it is greater than, equal to, or less than that target.

If you use a binary representation of a 256 bit hash, and you require a specific number of 0's then each increase in difficulty is DOUBLE the previous difficulty.  This isn't very useful when you only want to increase the difficulty by 10% or 15%.

Calculating the probability is as simple as dividing the quantity of valid values by the quantity of possible values.

Here's an example...

---

Lets imagine instead of using a 256 bit hash, we use an 8 bit hash.
With 8 bits you can represent a total of 256 "possible values" (0 through 255).
If you set the difficulty target at a value of 128, then any value less than 128 is a "valid value".
Therefore, the probability of a hash being less than the target is:
128/256 = 0.5 = 50%

Now lets imagine that we increase the difficulty by lowering the target value to 95.
In that case, the probability of a hash being less than the target is:
95 / 256 = 0.371 = 37.1%

Notice that if you choose to require that the binary representation of the hash start with a number of consecutive zeros, then you ONLY have 8 possible target values:
Zero 0's: xxxxxxxx = 256 valid values out of 256 possible values = 100%
One 0: 0xxxxxxx = 128 valid values out of 256 possible values = 50%
Two 0's: 00xxxxxx = 64 valid values out of 256 possible values = 25%
Three 0's: 000xxxxx = 32 valid values out of 256 possible values = 12.5%
Four 0's: 0000xxxx = 16 valid values out of 256 possible values = 6.25%
Five 0's: 00000xxx = 8 valid values out of 256 possible values = 3.125%
Six 0's: 000000xx = 4 valid values out of 256 possible values = 1.5625%
Seven 0's: 0000000x = 2 valid values out of 256 possible values = 0.78125%
Eight 0's: 00000000 = 1 valid value out of 256 possible values = 0.390625%

Notice that if you want a probability of 10%, you can't do it with "consecutive leading zeros".  However, if you use a "target value", and an 8 bit hash then you can increase or decrease difficulty by as little as 0.390625% if you simply increment or decrement the target value by 1.

---

A 256 bit hash with a "target value" gives you even more control over the probability. Each time you increase or decrease the target value by 1, you increase or decrease the probability by:
1 / 2²⁵⁶ = 0.00000000000000000000000000000000000000000000000000000000000000000000000000086%

How could I calculate the probability of finding X consecutives '0's in a sha string?

Nice vid, it helped a lot.

When I have a Beta of the project could you check it out? I'm going to publish the code at github.

Thank you!

Are you asking how Bitcoin handles this? Or are you asking what YOU should do to handle it?

In Bitcoin, a node will accept the FIRST valid block it receives at a given height.  It will only switch to a different block at that height if it is part of a longer chain.  The abandoned block is then called an "orphaned" block.

Lets imagine there is a blockchain with 3 blocks in it (blocks A, B, and C):

A -> B -> C

Now imagine that you are a miner and you mine a new block (block D1) on top of block C.
At the exact same time someone else mines a new block (block D2) on top of block C.

You both broadcast your blocks to the peers that you are connected to.  The blocks are relayed throughout the network and eventually I receive block D1 (your block) from one of my peers.  My node verifies that your block is valid. It is, so I add it to my blockchain, and relay it to all the peers that I am connected to. A millisecond later, one of the peers that I'm connected to sends me block D2 (the one you did NOT create).  I see that I already have a block at that height, and that both chains are the same length, so I just keep block D1.

Meanwhile someone else receives block D2 first from one of their peers.  Their node verifies that block D2 is valid. It is, so they add it to their blockchain, and relay it to all the peers that they are connected to.  A millisecond later, they receive block D1. They see that they already have a block at that height, and that both chains are the same length, so they just keep block D2.

Since, on average if the blocks were broadcast at EXACTLY the same time, approximately half of the nodes and miners on the network will hear about block D1 first, and the other half will hear about block D2 first, the network will disagree on what the blockchain looks like right now.  They will NOT have consensus.

Those that hear about block D1 first will have the following blockchain:
A -> B -> C -> D1

Those that hear about block D2 first will have the following blockchain:
A -> B -> C -> D2

Eventually a miner will solve a new block. Lets imagine that it is a miner that was working with the D2 chain. We'll call his block E2. He will broadcast his block to all the peers he is connected to.  If they were working with the D2 chain, then they will just add the E2 block to their chain after they validate it.  If they were working with the D1 chain, then they will see that the D2 chain is now longer than their chain.  They will abandon their chain and switch to the now longer D2 chain.  As block E2 spreads through the network eventually all nodes will switch to the, now longer, D2 chain.  All nodes will have a chain that looks like this:
A -> B -> C -> D2 -> E2

The D1 block will be abandoned as an "orphaned" block.



Here's a great video that explains how bitcoin works, and why it works the way that it does.  It generalizes some of the technical details about how the block hashing and transactions work, so don't take it as a technical explanation of exactly what is happening in bitcoin, but it clarifies some of the general concepts and helps explain why bitcoin is structured the way that it is:

https://www.youtube.com/watch?v=bBC-nXj3Ng4

Hello, thanks again for the great reply.

I have another question, about miners:

What would exactly happen when 2 miners find the next Block at the same time?

I already read about orphane blocks, or miners 51% ... but i don't really understand what really is happening, some basic explanation?

When this happens, can i just catch the longest available valid BlockChain and replace it with all other nodes?

Best Regards!

The term "cryptocurrency" is a generic term that refers to any system that uses cryptographic functions (such as hashes, digital signatures, and/or encryption) as part of a framework that allows users reliably exchange value.

Each cryptocurrency has its own protocol.  In may cases, those protocols are created by lazy individuals that start by simply copying an existing protocol and then make minor changes without any consideration for the overall effects on their system.

There are a few have introduced innovative ideas, but it is not yet clear if those ideas are better or worse than they pre-existing systems.

Prior to Satoshi's introduction of the blockchain as a method of decentralized consensus, all cryptocurrencies needed a trusted entity to determine the ordering of transactions.  The blockchain was a significant innovation that allowed cryptocurrencies to operate in a decentralized way.

Generally a cryptocurrency would likely need a method for the user to prove that they are authorized to spend some value.  Digital signatures are a common way to accomplish this.

They also need a method to prevent double-spending and to determine transaction order.  The blockchain appears to be the best way to accomplish this if you want the system to be decentralized and trustless.  Let us know if you come up with a better way.

So... I think i'm really confused, if I can do thinks like I want, what makes other coins a "crypto currency", so what should have/rules follow every crypto currency to be even a cryptocurrency?

Is it the Blockchain? The decentralized block structure?

What makes a cryptocurrency a cryptocurrency, i thought this was a protocol that i should to follow so my coin can be a "real" cryptocurrency.

I've read that bitcoin uses ECC for generating public address/private key. What alternatives could I use for my basic Coin?