[empty] | Bitcointalksearch.org

lucasholder

jr. member

Activity: 38

Merit: 19

Quote from: BoofBitcoin on April 07, 2020, 02:26:44 PM

As a thought experiment, I was wondering what would be the best way to securely store some data on the blockchain. The data itself as an example could be some text, maybe a few kilobytes. Lets say I don't want anyone to know what that text is and since its on the blockchain, anyone can find and access it so encryption would likely be required. Is there any sort of best practice for this scenario? Lets say a whole system was built around the idea of storing small bits of text on the blockchain in a secure manner, would it be something you trust? It would be something not far off from a company having a database of encrypted sensitive info but everyone having access to that database. Ethical debates of using bitcoin as an arbitrary store of data aside, I am just curious how in this sort of environment you could guarantee the security and privacy of that data. To be clear I don't really care about the ethical hesitation against this, and I also am specifically only interested in what can be done on the bitcoin blockchain not others.

Can you elaborate a bit on your use case? I'm going to guess that the actual reason for wanting to write the data to Bitcoin's blockchain is not actual storage (there are plenty of distributed and centralized systems for storing data) but to have a sort of proof that the data is authentic/not tampered with? In that case you can simply hash the data and put that small hash in Bitcoin's blockchain using an OP_RETURN transaction. There are plenty of existing tools and libraries for doing that.

A few KBs of data would be prohibitively expensive to store on Bitcoin (and pretty much any altcoin worth its salt).

Wind_FURY

legendary

Activity: 2898

Merit: 1823

Quote from: ABCbits on April 08, 2020, 07:13:57 AM

If you only need to make sure your data is distributed, using another P2P protocol such as IPFS and BitTorrent are better option.

Quote from: BoofBitcoin on April 07, 2020, 04:13:03 PM

I think the "bitcoin isnt made for this" begins to step into the ethical debate which isn't really my focus, but I also believe that as long as bitcoin allows for such a use case even though it wasn't intended for that it does not matter very much. Ultimately, if it becomes a problem for the bitcoin ecosystem as a whole then it will force a solution to be created or bitcoin will fail. Relying on everyone to abide by some ethical standard isn't a great practice so instead of saying "bitcoin should not be used for this" I think the argument should be focused around "If or when bitcoin is used for this..how do we solve for x?". In a way, relying on everyone to stick to what bitcoin should be used for means you're trusting humans to do behave a certain way which goes entirely against the grain of what bitcoin is about at its fundamental core, which is taking away the need to trust humans to behave a certain way.

IMO the community don't rely on ethics to prevent people from doing something unethical on Bitcoin. In this case, it's because :
1. the cost to store arbitrary data on Bitcoin's blockchain is quite expensive (due to big transaction size)
2. there's no standard to do it (aside from OP_RETURN)
3. small amount of Bitcoin might be lost if you use don't use OP_RETURN
4. currently there's no user-friendly software/service to put arbitrary data on Bitcoin's blockchain.

He wants to encrypt some random data, and store it on the blockchain. Why "the blockchain"?

The solution is to encrypt the data with GPG, then upload "the data" on some website, or what you said.

ABCbits

legendary

Activity: 2870

Merit: 7490

Crypto Swap Exchange

If you only need to make sure your data is distributed, using another P2P protocol such as IPFS and BitTorrent are better option.

Quote from: BoofBitcoin on April 07, 2020, 04:13:03 PM

I think the "bitcoin isnt made for this" begins to step into the ethical debate which isn't really my focus, but I also believe that as long as bitcoin allows for such a use case even though it wasn't intended for that it does not matter very much. Ultimately, if it becomes a problem for the bitcoin ecosystem as a whole then it will force a solution to be created or bitcoin will fail. Relying on everyone to abide by some ethical standard isn't a great practice so instead of saying "bitcoin should not be used for this" I think the argument should be focused around "If or when bitcoin is used for this..how do we solve for x?". In a way, relying on everyone to stick to what bitcoin should be used for means you're trusting humans to do behave a certain way which goes entirely against the grain of what bitcoin is about at its fundamental core, which is taking away the need to trust humans to behave a certain way.

IMO the community don't rely on ethics to prevent people from doing something unethical on Bitcoin. In this case, it's because :
1. the cost to store arbitrary data on Bitcoin's blockchain is quite expensive (due to big transaction size)
2. there's no standard to do it (aside from OP_RETURN)
3. small amount of Bitcoin might be lost if you use don't use OP_RETURN
4. currently there's no user-friendly software/service to put arbitrary data on Bitcoin's blockchain.

Wind_FURY

legendary

Activity: 2898

Merit: 1823

OP, does your "experiment" need censorship-resistance? Or do you need protection from adversarial changes done to your database?

You don't need a blockchain, if you don't.

pooya87

legendary

Activity: 3472

Merit: 10611

by the nodes with enforcing the standard rules that would be rejecting any transaction that is non-standard or has large OP_RETURN data (which is what you would use to push arbitrary data to the blockchain). and that is enforced per transaction basis since there is no other way.

Quote from: BoofBitcoin on April 08, 2020, 12:36:34 AM

somewhere around 1 to 2kb tops.

again, bitcoin blockchain is not for data storage. use an altcoin that is designed for that. there are actually a couple of them: SIA, storej, filecoin,...

BoofBitcoin

newbie

Activity: 12

Merit: 25

[empty]

pooya87

legendary

Activity: 3472

Merit: 10611

the only size limit is the block size, in other words you can inject a garbage data a little less than 1 MB in one transaction's output to the bitcoin blockchain. 20 byte (which is actually 40+OP_RETURN+push-byte size) is the standard rule that the bitcoin nodes are enforcing to reduce the chances of abuse. there is also transaction fees that will prevent such shenanigans to some extent.

the thing is that bitcoin is a payment system so it's "ledger" should only have to store payment data not arbitrary messages hence the "The best practice is don't."

BoofBitcoin

newbie

Activity: 12

Merit: 25

Quote from: jackg on April 07, 2020, 07:46:12 PM

I mean there are probably a few bits you could scrape from the lock time value but that'll also be very limited and it'll have to be a block height that has already passed...

20 bytes is what would be needed to store unique identifiers on the chain afaik so it's what you're going to have to put up with... But why you think others should download whatever junk you dump on there confuses me... Try it once or twice don't use 12+ kb of space that no one else can use for anything...

That's not really my problem to deal with. Its not that I think they should, its that I know they have no choice anyways. If the issue comes down to "Just don't do that, its not right" then its not a good argument because someone will eventually abuse that. Even worse, if what I want to do can scale it will be an application that multiple end users can take advantage of so it wouldn't even be just me putting "junk data" on the blockchain. I love bitcoin but it does not progress without people testing the outer bounds of its capabilities. If it becomes a major problem like causing transaction fees to skyrocket then I guess people will have to figure out a solution to that instead of urging people to not stress test weaknesses within the system. Either way, I have something I want to do that uses bitcoin's blockchain, so I would like to see what happens for better or worse. Considering the resilient nature of bitcoin and the incredible amount of brilliant engineering that goes into bitcoin to resolve its shortcomings such as the lightning network, I figure worst case is bitcoin adapts. Best case is my use case has no significant impact.

jackg

copper member

Activity: 2856

Merit: 3071

https://bit.ly/387FXHi lightning theory

I mean there are probably a few bits you could scrape from the lock time value but that'll also be very limited and it'll have to be a block height that has already passed...

20 bytes is what would be needed to store unique identifiers on the chain afaik so it's what you're going to have to put up with... But why you think others should download whatever junk you dump on there confuses me... Try it once or twice don't use 12+ kb of space that no one else can use for anything...

BoofBitcoin

newbie

Activity: 12

Merit: 25

[empty]

jackg

copper member

Activity: 2856

Merit: 3071

https://bit.ly/387FXHi lightning theory

Are you sure it was the whitepaper? He published the title of a headline in the first block that was made (can't remember what it said now) if you open the first blk file bitcoin core downloads you can see what he wrote.

If you're after a way of storing data there are some solutions such as storj that link you up with people renting out hard drive space...

You can have other solutions as well such as only checking message that are from, to or contain a certain address... But you'd also have to check no one sabatages that so it might be better to just look at transactions FROM if you did try to go through with something like this.

BoofBitcoin

newbie

Activity: 12

Merit: 25

Quote from: jackg on April 07, 2020, 05:12:33 PM

Quote from: BoofBitcoin on April 07, 2020, 04:13:03 PM

I was not aware of the 20 byte limit. That would also be problematic for storing text that you wanted to keep private or secure, since encryption could inflate the text so you would probably have to use some combo of encryption and encoding to get the job done it sounds like.

I think the "bitcoin isnt made for this" begins to step into the ethical debate which isn't really my focus, but I also believe that as long as bitcoin allows for such a use case even though it wasn't intended for that it does not matter very much. Ultimately, if it becomes a problem for the bitcoin ecosystem as a whole then it will force a solution to be created or bitcoin will fail. Relying on everyone to abide by some ethical standard isn't a great practice so instead of saying "bitcoin should not be used for this" I think the argument should be focused around "If or when bitcoin is used for this..how do we solve for x?". In a way, relying on everyone to stick to what bitcoin should be used for means you're trusting humans to do behave a certain way which goes entirely against the grain of what bitcoin is about at its fundamental core, which is taking away the need to trust humans to behave a certain way.

Merging encoding and encryption functions is probably a good idea. As well as trying to symmetrically encrypt any private data, storing keys elsewhere...

Blockchain isn't designed for data storage. It can facilitate it but shouldn't be used directly for it... For the experienced - it's incredibly slow to access at least and for the novice it's just difficult to implement... Even not looking at the ethics of it it's still something that most consider.

I agree that ethics aside, it still has technical challenges because that's not what its intended to do but I think that's part of the fun. So far it seems the use case of data storage would be limited to an extremely small amount of data and there is a chance it is not fast. Given the limitations I think I can come up with something that would be functional. I have seen examples of things like the bitcoin whitepaper being embedded into a transaction, how is that done despite there being a 20 byte limit..or am I misunderstanding something?

jackg

copper member

Activity: 2856

Merit: 3071

https://bit.ly/387FXHi lightning theory

Quote from: BoofBitcoin on April 07, 2020, 04:13:03 PM

I was not aware of the 20 byte limit. That would also be problematic for storing text that you wanted to keep private or secure, since encryption could inflate the text so you would probably have to use some combo of encryption and encoding to get the job done it sounds like.

I think the "bitcoin isnt made for this" begins to step into the ethical debate which isn't really my focus, but I also believe that as long as bitcoin allows for such a use case even though it wasn't intended for that it does not matter very much. Ultimately, if it becomes a problem for the bitcoin ecosystem as a whole then it will force a solution to be created or bitcoin will fail. Relying on everyone to abide by some ethical standard isn't a great practice so instead of saying "bitcoin should not be used for this" I think the argument should be focused around "If or when bitcoin is used for this..how do we solve for x?". In a way, relying on everyone to stick to what bitcoin should be used for means you're trusting humans to do behave a certain way which goes entirely against the grain of what bitcoin is about at its fundamental core, which is taking away the need to trust humans to behave a certain way.

Merging encoding and encryption functions is probably a good idea. As well as trying to symmetrically encrypt any private data, storing keys elsewhere...

Blockchain isn't designed for data storage. It can facilitate it but shouldn't be used directly for it... For the experienced - it's incredibly slow to access at least and for the novice it's just difficult to implement... Even not looking at the ethics of it it's still something that most consider.

BoofBitcoin

newbie

Activity: 12

Merit: 25

[empty]

Last of the V8s

legendary

Activity: 1652

Merit: 4393

Be a bank

time to wheel out the old lad

BoofBitcoin

newbie

Activity: 12

Merit: 25

[empty]

jackg

copper member

Activity: 2856

Merit: 3071

https://bit.ly/387FXHi lightning theory

No matter what chain you use, you're almost certainly better trying to compress the data you want to store so it becomes cheaper.

I don't know about other chains but bitcoin only allows 20 bytes of optional universal data per transaction. Some encoding strategies like run length encoding or other methodologies can help maintain data in a way. Also storing text as an alphabet or alphanumeric characters can be stored in 5 bytes or 6 bytes respectively and can use sub versions of bytes to store even more.

But bitcoins blockchain isn't made for handling this and will just push fees up for everyone else...

gmaxwell

staff

Activity: 4284

Merit: 8808

The best practice is don't.

Pmalek

legendary

Activity: 2730

Merit: 7065

Take a look at this thread. https://bitcointalksearch.org/topic/storing-data-in-bitcoin-blockchain-5094063
Some other blockchains might be a better fit for this, like Ethereum. Someone suggested DeepOnion as well.

BoofBitcoin

newbie

Activity: 12

Merit: 25

[empty]

Topic: [empty] (Read 238 times)