Author

Topic: How Can/Do You Store Files on a Blockchain? (Read 863 times)

brand new
Activity: 0
Merit: 0
November 18, 2018, 10:37:22 AM
#42
It's important to make the distinction between storing actual files and the ability to write 80 arbitrary bytes to the OP_RETURN. To utilize the blockchain to store a pdf would require you to create many OP_RETURN outputs containing the raw bytes.  Then when ready to fetch the file you would have to combine all the output OP_RETURN data in order and then compile the raw bytes into an actual usable PDF. It's terribly inefficient and creates outputs that can never be spent, forever littering the UTXO set.
Hi...Need a pm please..
sr. member
Activity: 310
Merit: 727
---------> 1231006505
Is the only possible workaround for these projects to have the Data encrypted or stored separately, and have the blockchain and it's respective ledger work as a Key in the access/decryption of that information.
Well in my opinion it is for public blockchains, the ones where everybody can run a node. However if you are looking at private blockchains you could also store the files itself onchain. It all depends on the use case if you ask me.
legendary
Activity: 1568
Merit: 2037
I fully admit I'm out of my depth, when it comes to the nuances of file sizes and blockchain capabilities. After reading through this though, it seems to me that 99% of the real world applications people want to adapt blockchain technology for are doomed to fail. That in the end it's original purpose of operating as a ledger for currency/asset exchange is viable.

Some of the ideas i had been curious about where, medical records being stored and accessed in this manner. I've heard pitches of governements trying to use it for census data, or taxation purposes. On top of the original file size it would need to have the option to update/ad further information creating larger file sizes.

Is the only possible workaround for these projects to have the Data encrypted or stored separately, and have the blockchain and it's respective ledger work as a Key in the access/decryption of that information.
full member
Activity: 812
Merit: 108
I don't know about Storing file on blockchain but i wanna add one thing in this as we know blockchain is super secure why not some one invented a drive or memory space which is encrypted by blockchain and works like same as wallet works on private key. It can be more secure feature to store your files because no can access yours files without private key.
newbie
Activity: 36
Merit: 0
Quote
As mentioned here - I have heard of Sia and Storj, but what are the best usecases/approach services for storing files ?
SIA and Storj do NOT store any files in a blockchain themselves. That wouldn't make any sense since it would mean all nodes should have a copy of every single file. Which in turn would mean the blockchain itself could become Petabytes big. Instead, they store smart contracts on a blockchain which describes an agreement between a hoster and a party who needs to store files. Just to be sure files will be stored redundantly on a couple of hosters.

I'm not enough confident about Storj to say there are good use cases but I know for sure SIA is an absolute no go at the moment for a couple of reasons:
  • Storage is limited to 5TB and below 2TB it's way too expensive
  • Smallest size for a file is 40MB, which means you pay for 40MB even if you store a file of a couple of bytes. Backing up a lot of small files is a no-go area
  • You can only restore your files from a computer where you uploaded them from. So if the HDD fails you can't even restore your own "backup"
  • Development team is a bunch of kids who have no idea what they are doing.


Thank you for explaining. I am sure Storj team has some plan in mind on their way, but it is not perfect for the moment. If we presume SSD capacities grow exponentially - I think in some near future file storage is possible on a blockchain.
sr. member
Activity: 742
Merit: 395
I am alive but in hibernation.
I think this is solid answer, IPFS is one of the examples.
Its great that it works even today and you can see great community there.


That is 195.90 USD, not 100,000.00!

ok no $1000 or $100K
Do you think still $195 for an MB is a cheap option?  What about if you need to store a GB then.
It is better to use the technology that is intended for the purpose. Storage is getting cheaper and faster day by day but it is not true for the blockchain.
legendary
Activity: 1904
Merit: 1074
Can you imagine what the Blockchain size would be, if you stored large files on it? Nope, I would rather use a cloud storage

site and then link a URL in the Blockchain to that file. Yes, it will only store the file until the cloud storage site close down, but

you can always move the file when that happens and then re-link the new URL with a new transaction on the Blockchain.  Roll Eyes
sr. member
Activity: 310
Merit: 727
---------> 1231006505
Quote
As mentioned here - I have heard of Sia and Storj, but what are the best usecases/approach services for storing files ?
SIA and Storj do NOT store any files in a blockchain themselves. That wouldn't make any sense since it would mean all nodes should have a copy of every single file. Which in turn would mean the blockchain itself could become Petabytes big. Instead, they store smart contracts on a blockchain which describes an agreement between a hoster and a party who needs to store files. Just to be sure files will be stored redundantly on a couple of hosters.

I'm not enough confident about Storj to say there are good use cases but I know for sure SIA is an absolute no go at the moment for a couple of reasons:
  • Storage is limited to 5TB and below 2TB it's way too expensive
  • Smallest size for a file is 40MB, which means you pay for 40MB even if you store a file of a couple of bytes. Backing up a lot of small files is a no-go area
  • You can only restore your files from a computer where you uploaded them from. So if the HDD fails you can't even restore your own "backup"
  • Development team is a bunch of kids who have no idea what they are doing.
newbie
Activity: 36
Merit: 0
We know blockchain is not only limited to cryptocurrencies. But how exactly do you store files, say a one page pdf, in a blockchain? Especially without using any third-party services like IPFS.
Are you asking about how to abuse the blockchain? lol.
You should not!

Abuse? Clearly you should read more. File storage is one of the best usecase for blockchain, and there's already a lot of sites/services that provides it.

As mentioned here - I have heard of Sia and Storj, but what are the best usecases/approach services for storing files ?
newbie
Activity: 27
Merit: 0
I think this is solid answer, IPFS is one of the examples.
Its great that it works even today and you can see great community there.

Sorry, but that is just not entirely true.

First of all, storing a hash is not storing the file. The OP specifically asked for storing a file and not just recording it's existence. What if someone wants to store a file that will be there even if all his computer equipment is destroyed?

Also, you can save all those files on the Blockchain fairly cheaply - no 100,000$ needed.

1 MB data can be split up on 8192 1-of-3 transactions using 2*64 byte from the second and third public key.

I am not sure how much overhead comes with each of those ... let's say it's 300 byte per transaction to calculate easier. That would make the 8192 transactions a total size of 2,4MB including overhead. Currently, 0.00001 BTC/kB is enough to get your TX confirmed. That would be 0.024576 BTC to store your 1 MB file incl. overhead.

That is 195.90 USD, not 100,000.00!

We are trying to run a project which every member will have about 500mb or 1gb files. Is it possible to store on blockchain?
or can you give some advice how to store every big files of members anonymously?
legendary
Activity: 1260
Merit: 1168
I think this is solid answer, IPFS is one of the examples.
Its great that it works even today and you can see great community there.

Sorry, but that is just not entirely true.

First of all, storing a hash is not storing the file. The OP specifically asked for storing a file and not just recording it's existence. What if someone wants to store a file that will be there even if all his computer equipment is destroyed?

Also, you can save all those files on the Blockchain fairly cheaply - no 100,000$ needed.

1 MB data can be split up on 8192 1-of-3 transactions using 2*64 byte from the second and third public key.

I am not sure how much overhead comes with each of those ... let's say it's 300 byte per transaction to calculate easier. That would make the 8192 transactions a total size of 2,4MB including overhead. Currently, 0.00001 BTC/kB is enough to get your TX confirmed. That would be 0.024576 BTC to store your 1 MB file incl. overhead.

That is 195.90 USD, not 100,000.00!
newbie
Activity: 56
Merit: 0
Now imagine using that kind of technology on other day to day transactions from buying and selling of assets (real estate, stocks) to the government's activities it will give you more transparency on your deals and their actions. Someone will input/execute the transaction and everybody can look it up in the blockchain. Just think of it as an tamper proof online ledger where once you successfully inputted a data it can never be change or altered inside the chain.
OP wanted to store a PDF, at $1000 (today) to $100,000 (last December) per MB, that's rediculously expensive to do.
What you can do, is just store the hash of the file. This proves the file existed at the moment the hash was stored, and can't be altered anymore.
Dooglus did this to store Bustabit's hash here.

I think this is solid answer, IPFS is one of the examples.
Its great that it works even today and you can see great community there.
legendary
Activity: 1260
Merit: 1168
It's important to make the distinction between storing actual files and the ability to write 80 arbitrary bytes to the OP_RETURN. To utilize the blockchain to store a pdf would require you to create many OP_RETURN outputs containing the raw bytes.  Then when ready to fetch the file you would have to combine all the output OP_RETURN data in order and then compile the raw bytes into an actual usable PDF. It's terribly inefficient and creates outputs that can never be spent, forever littering the UTXO set.

You could also store them in the 2nd and 3rd public key of a 1-of-3 multisignature output script and spend these outputs whenever you like Smiley
legendary
Activity: 1442
Merit: 1189
It's important to make the distinction between storing actual files and the ability to write 80 arbitrary bytes to the OP_RETURN. To utilize the blockchain to store a pdf would require you to create many OP_RETURN outputs containing the raw bytes.  Then when ready to fetch the file you would have to combine all the output OP_RETURN data in order and then compile the raw bytes into an actual usable PDF. It's terribly inefficient and creates outputs that can never be spent, forever littering the UTXO set.
newbie
Activity: 27
Merit: 0
sure we can store files on blockchain, but it will be very inefficient and bloated in size
it would be better to have a blockchain for storing "references" to actual file location
the file can be stored in multiple locations, and the blockchain serves as a pointer to them
I'm thinking of "napster meets blockchain", we have distributed file sharing and then blockchain to keep all valid references

I like this idea
We have a new project and it is necessary to store files on blockchain, we are not at that stage but it is good to know that we have some options Smiley
copper member
Activity: 85
Merit: 122
If storing file in blockchain is so expensive then all the altcoin developers are scamming by telling that there project aims to put documents in Blockchain ?
Either that, or they mean something else, or they innocently believe it's possible, but haven't really tried. I always roll my eyes when a new client comes in with an idea to create a social network on blockchain, with photos and pictures and video etc.

Remember, objects in the blockchain are stored by the whole network. If you want to store some data, it must now be propagated to all thousands of full nodes and stored by them indefinitely. So the high price of doing that is justified.

Quote
or it is feasible to create a different flavor of blockchain that is capable of storing large volume of data?
Decentralized storage is better, see the projects mentioned above. You can just pay 50 people to store the file for you (much cheaper), then put a file's hash on the blockchain as a proof-of-existence, proof-of-sequence, or a pointer. Nodes interested in getting this information will ask around, and the storage providers will give them the file.

We're already suffering from enormous blockchain sizes and low throughput. Don't try to make our lives even harder. Hopefully, breakthroughs in technologies like sharding and Lightning Network will help with some of these issues.
member
Activity: 169
Merit: 18
Look like , I am late on this discussion but I read the whole thread.

If storing file in blockchain is so expensive then all the altcoin developers are scamming by telling that there project aims to put documents in Blockchain ?
or it is feasible to create a different flavor of blockchain that is capable of storing large volume of data?
newbie
Activity: 23
Merit: 0
We can store files, or anything on blockchain definitely. But one you should also consider is the usage of blockchain. Why storing such a document on blockchain is good, considering its cost and its benefits are drawbacks are important too.
newbie
Activity: 16
Merit: 0
> Blockchain is a source of truth.
> Bitcoin's blockchain is for made for recording transactions, not random messages.
> Blockchain can be used for storing other types of messages. For instance, one common use case is to do voting.
> Storing any arbitrary data on blockchain would simply not scale
member
Activity: 168
Merit: 47
8426 2618 9F5F C7BF 22BD E814 763A 57A1 AA19 E681
There was a project called freenet that attempted a decentralized network to store files there, im not sure if it died or not but that may make more sense that using a blockchain.
Freenet is not died. It don't use the blockchain, it is a totally different approach more similar to a decentralized caching. It is very cool and there are a lot of plugins take a look at this simple guide
http://freesocial.draketo.de/index_en.html
This is tutorial is designed for anyone who wants to use Freenet primarily for anonymous and censorship-resistant social networking and truly private email.
legendary
Activity: 1372
Merit: 1252
Hashing a file and putting it on the blockchain is as far as it's going to get to "store files on the blockchain". The actual files will never be able to be held in a blockchain. Blockchain space is a scarce resource, we have already problems fitting in transactions, imagine if people start uploading GB's worth of crap.

There was a project called freenet that attempted a decentralized network to store files there, im not sure if it died or not but that may make more sense that using a blockchain.
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
Now imagine using that kind of technology on other day to day transactions from buying and selling of assets (real estate, stocks) to the government's activities it will give you more transparency on your deals and their actions. Someone will input/execute the transaction and everybody can look it up in the blockchain. Just think of it as an tamper proof online ledger where once you successfully inputted a data it can never be change or altered inside the chain.
OP wanted to store a PDF, at $1000 (today) to $100,000 (last December) per MB, that's rediculously expensive to do.
What you can do, is just store the hash of the file. This proves the file existed at the moment the hash was stored, and can't be altered anymore.
Dooglus did this to store Bustabit's hash here.
hero member
Activity: 1680
Merit: 655
Think of the Blockchain as an online database or a virtual ledger of some sort, with the technology it has given it has simply removed the use of third party services as you only need your cryptocurrency wallet in order to send and receive from the other party you are dealing with, it simply has removed the used of banks for their bank transfers which includes a more expensive fee compared to what network fees you are paying today.

Now imagine using that kind of technology on other day to day transactions from buying and selling of assets (real estate, stocks) to the government's activities it will give you more transparency on your deals and their actions. Someone will input/execute the transaction and everybody can look it up in the blockchain. Just think of it as an tamper proof online ledger where once you successfully inputted a data it can never be change or altered inside the chain. The positive benefit I see is that this tool will be a cost-effective way of reducing the potential frauds happening in our transactions.
member
Activity: 168
Merit: 47
8426 2618 9F5F C7BF 22BD E814 763A 57A1 AA19 E681
I think you can create a Blockchain for storing files.  Question is how can this be achieved? Would miners be necessary in this case? Why not a different kind of Blockchain?
Who says storing duplicated files on thousands of Nodes is not a good thing... Don't forget that the purpose of Blockchain is to eliminate censorship, enable people have full control over their money, Transparency and eliminate any Point of Failure.    How can we not want these wonderful features for files on Blockchain?

I love the idea of duplicating files across thousands of public Nodes.

blockchain is to prevent doublespend. if you want persistence, you already have freenet and it is far way more efficent than a blockchain.
Ucy
sr. member
Activity: 2744
Merit: 404
Compare rates on different exchanges & swap.
I think you can create a Blockchain for storing files.  Question is how can this be achieved? Would miners be necessary in this case? Why not a different kind of Blockchain?
Who says storing duplicated files on thousands of Nodes is not a good thing... Don't forget that the purpose of Blockchain is to eliminate censorship, enable people have full control over their money, Transparency and eliminate any Point of Failure.    How can we not want these wonderful features for files on Blockchain?

I love the idea of duplicating files across thousands of public Nodes.
hero member
Activity: 1232
Merit: 738
Mixing reinvented for your privacy | chipmixer.com
sure we can store files on blockchain, but it will be very inefficient and bloated in size
it would be better to have a blockchain for storing "references" to actual file location
the file can be stored in multiple locations, and the blockchain serves as a pointer to them
I'm thinking of "napster meets blockchain", we have distributed file sharing and then blockchain to keep all valid references
sr. member
Activity: 257
Merit: 343
...
The blockchain is the only way we have to prevent doublespend. Any other use is an abuse. Doublespending files is a no sense.
Using the blockchain only as a database is an abuse. It is not. What happen if using the blockchain to store files become more valuable than using it to prevent the doublespend?
Yes, I agree to this point of view. I was thinking in the sense of being „just a good citizen“, and using the blockchain only for monetary transactions. And everyone would do so (be a good citizen). And then someone finds out, that you can store data in the blockchain, it get‘s known to the broader community, and suddenly we can‘t transact monetary values any more, cause the system is flooded with data. This would be a denial of service (attack or spamming?). So better we know early about this, than too late. Hence important to shed light in every unknown area.
But never mind, Bitcoin is doing good for ten years now, and everything it proved to be resistant  Wink
member
Activity: 168
Merit: 47
8426 2618 9F5F C7BF 22BD E814 763A 57A1 AA19 E681

I think it’s not about abuse or reading more. This type of discussion is controversial. On the other hand side, if the opportunity exists, to use the blockchain in a different way, then possibilities should be explored. From a security point of view it doesn’t make sense to say “it is possible, but do not use it”.


The blockchain is the only way we have to prevent doublespend. Any other use is an abuse. Doublespending files is a no sense.
Using the blockchain only as a database is an abuse. It is not. What happen if using the blockchain to store files become more valuable than using it to prevent the doublespend?
sr. member
Activity: 257
Merit: 343
...
Abuse? Clearly you should read more. File storage is one of the best usecase for blockchain, and there's already a lot of sites/services that provides it.
I disagree to the statement, that (bitcoin’s) blockchain is best use case for storing files. It is simply too expensive, to store data.
I think it’s not about abuse or reading more. This type of discussion is controversial. On the other hand side, if the opportunity exists, to use the blockchain in a different way, then possibilities should be explored. From a security point of view it doesn’t make sense to say “it is possible, but do not use it”.

The bitcoin blockchain has already data stored in it, and a typical way to do it is using OP_RETURN, with 40 and/or 80 bytes limit. So you would have to create many transactions, to split the file over the transactions.
An alternative approach is to create a P2SH tx, which allows for ~500 bytes. Then again, splitting the files and creating the many tx and paying the fees doesn’t make it a good opportunity. One can clearly see, that bitcoin blockchain is not designed to store data.
Hence the proposals to use a bridge to another system, or going directly to Factum, SIA or STORJ. Hopefully bridges will come into play one day.
member
Activity: 336
Merit: 12
I don't see what would be the benefit of storing files on a blockchain since they must all be identitical to be valid the files should be duplicated as many times as there are blockchain nodes running and validating new blocks.

Maybe it could have a limited use yes, to have a global file mirror system for critical informations but the price to upload the files should really super high in order to maintain the blockchain, I don't see a need for it in the current world.

Now check DeepOnion ( $Onion) Deepvault which is storing hashes of digital files in its blockchain in order to validate their integrity, I think it's a more realistic and working approach. They are releasing a easy to use web version of this feature  Cheesy
member
Activity: 168
Merit: 47
8426 2618 9F5F C7BF 22BD E814 763A 57A1 AA19 E681


"The blockchain, sell blockchain"? That's amusing. People like you who seem to think that blockchain's sole purpose is for cryptocurrency is what hinders the development of the technology. Most researchers on this field doesn't care about your cryptocurrency transactions clogging up. Blockchain is still in research and experimental state, so what the hell do you do with it, you experiment.

the difference between us is that I know how to spam the blockchain. Given enought money I can spam any blockchain.
You don't know, and I hope you will not find out.




Everyone knows how to spam a blockchain, it's a common known issue/attack. And by the way, no, you don't need that much money. Don't flatter yourself with such simple knowledge kid.
If you know how to spam the blockchain(storing file on it), why you opened this thread?
full member
Activity: 336
Merit: 112


"The blockchain, sell blockchain"? That's amusing. People like you who seem to think that blockchain's sole purpose is for cryptocurrency is what hinders the development of the technology. Most researchers on this field doesn't care about your cryptocurrency transactions clogging up. Blockchain is still in research and experimental state, so what the hell do you do with it, you experiment.

the difference between us is that I know how to spam the blockchain. Given enought money I can spam any blockchain.
You don't know, and I hope you will not find out.




Everyone knows how to spam a blockchain, it's a common known issue/attack. And by the way, no, you don't need that much money. Don't flatter yourself with such simple knowledge kid.
member
Activity: 168
Merit: 47
8426 2618 9F5F C7BF 22BD E814 763A 57A1 AA19 E681


"The blockchain, sell blockchain"? That's amusing. People like you who seem to think that blockchain's sole purpose is for cryptocurrency is what hinders the development of the technology. Most researchers on this field doesn't care about your cryptocurrency transactions clogging up. Blockchain is still in research and experimental state, so what the hell do you do with it, you experiment.

the difference between us is that I know how to spam the blockchain. Given enought money I can spam any blockchain.
You don't know, and I hope you will not find out.


full member
Activity: 336
Merit: 112
We know blockchain is not only limited to cryptocurrencies. But how exactly do you store files, say a one page pdf, in a blockchain? Especially without using any third-party services like IPFS.
Are you asking about how to abuse the blockchain? lol.
You should not!

Abuse? Clearly you should read more. File storage is one of the best usecase for blockchain, and there's already a lot of sites/services that provides it.
Clearly you should read more there are better ways to share files than spamming the blockchain.
the blockchain is very inefficent for these stuff. Of course there are a lot of scammers trying to sell the blockchain to steal some money from people like you.


"The blockchain, sell blockchain"? That's amusing. People like you who seem to think that blockchain's sole purpose is for cryptocurrency is what hinders the development of the technology. Most researchers on this field doesn't care about your cryptocurrency transactions clogging up. Blockchain is still in research and experimental state, so what the hell do you do with it, you experiment.

That depends on the blockchain protocol,
1. does the protocol let anyone submit/store file?
2. does the protocol have soft/hard-limit about file which can be submitted/stored?
3. does the protocol force everyone who's on network/nodes to store all data or not?

But, i think another kind of P2P protocol such as Freenet or BitTorrent protocol would be more efficient for store/sharing files. Maybe you should check how Storj/Sia works.

I can't really rely on any third-party like Storj or Sia.
member
Activity: 168
Merit: 47
8426 2618 9F5F C7BF 22BD E814 763A 57A1 AA19 E681

Output #0 script data: j(Olivia,Will you marry me?Love,Paul

oh, that's the worst thing to do, I know, no one can stop abuse and spam, but at least use script(PUSHDATA), so your spam will go to witness.
legendary
Activity: 2870
Merit: 7490
Crypto Swap Exchange
That depends on the blockchain protocol,
1. does the protocol let anyone submit/store file?
2. does the protocol have soft/hard-limit about file which can be submitted/stored?
3. does the protocol force everyone who's on network/nodes to store all data or not?

But, i think another kind of P2P protocol such as Freenet or BitTorrent protocol would be more efficient for store/sharing files. Maybe you should check how Storj/Sia works.
legendary
Activity: 3010
Merit: 3724
Join the world-leading crypto sportsbook NOW!
Yes, you can. I'm not sure I'd call it abuse of the blockchain (at least, on Bitcoin's), but as goddog says, it's extremely inefficient to do so. People still do it for fun, sending/storing messages and files esp on Ethereum lately. There are some services still out there that'll do it for casual users (cryptograffiti is one). You can see a marriage proposal from two weeks ago on Bitcoin blockchain: https://blockchair.com/bitcoin/transaction/52465ad6a5b3742f83f74699b1d6a4a378ac940bebe60183f4a19d1641d464fe#o=0
Output #0 script data: j(Olivia,Will you marry me?Love,Paul

There are a few blockchain projects specifically intended for file storage. I'm not familiar with their specific use cases, but Storj is an old one, and Sia is still I suppose viable, given the merged mining with Bitcoin but neither actually store files on the blockchain, I think, rather on a decentralised network of unused disk space, blockchain is just the method to transact.
member
Activity: 168
Merit: 47
8426 2618 9F5F C7BF 22BD E814 763A 57A1 AA19 E681
We know blockchain is not only limited to cryptocurrencies. But how exactly do you store files, say a one page pdf, in a blockchain? Especially without using any third-party services like IPFS.
Are you asking about how to abuse the blockchain? lol.
You should not!

Abuse? Clearly you should read more. File storage is one of the best usecase for blockchain, and there's already a lot of sites/services that provides it.
Clearly you should read more there are better ways to share files than spamming the blockchain.
the blockchain is very inefficent for these stuff. Of course there are a lot of scammers trying to sell the blockchain to steal some money from people like you.
full member
Activity: 336
Merit: 112
We know blockchain is not only limited to cryptocurrencies. But how exactly do you store files, say a one page pdf, in a blockchain? Especially without using any third-party services like IPFS.
Are you asking about how to abuse the blockchain? lol.
You should not!

Abuse? Clearly you should read more. File storage is one of the best usecase for blockchain, and there's already a lot of sites/services that provides it.
member
Activity: 168
Merit: 47
8426 2618 9F5F C7BF 22BD E814 763A 57A1 AA19 E681
We know blockchain is not only limited to cryptocurrencies. But how exactly do you store files, say a one page pdf, in a blockchain? Especially without using any third-party services like IPFS.
Are you asking about how to abuse the blockchain? lol.
You should not!
full member
Activity: 336
Merit: 112
We know blockchain is not only limited to cryptocurrencies. But how exactly do you store files, say a one page pdf, in a blockchain? Especially without using any third-party services like IPFS.
Jump to: