Topic: [ANN] Sia - Decentralized Storage - page 490. (Read 1382189 times)

Fascinating system. Thank you for the details.




What do you mean by downloads will cost siacoins.

Scalability on blockchains is always a tricky issue. In our case, we're bottlenecked by storage proofs. A storage proof takes around 1kb. If we assume that 90% of Sia is going to be storage proofs, that's ~900 storage proofs per block. 1008 blocks in a week means ~900,000 storage proofs per week.

900,000 storage proofs per week does not sound like a lot, especially when files are being uploaded to 200 places at a time. If you assume each file lasts for 2 years though (100 weeks), we're all the way up to 450,000 files. The beta only puts files in 12 places, and they each last 1 week, so that's a total of 75,000 concurrent files for the beta.

We can improve on this number by using payment channels. Sia has support for payment channels through file contracts, which means you can store multiple files on the same file contract and you can keep adding files as you go. When you add files, you don't need to add transactions to the blockchain. When you first join the network, you'll have to make a bunch of setup transactions with hosts. Assuming each user wants to connect to 200 hosts, that's 200 contracts that will need to be made. We're also going to assume that the average host relationship lasts 6 months. (Instead of having them all expire at the same time though, we'll stagger it. So you'll renew contracts with 4 hosts per week). It comes down to 200 setup transactions, followed by 4 per week for the rest of the time you are using Sia. A setup transaction is really small, but closing the setup will result in a storage proof. Since you're storing multiple files, the storage proof is going to be bigger. We'll call this whole thing 1.5kb per host per renew, or about 6kb per week per user.

6kb per week per user, and ~900mb per week of available space on the blockchain. That comes to a total of 150,000 users, each triggering 4 storage proofs on the blockchain per week. Some users might want more storage proofs per week, some might be comfortable with less. Also, the % of storage proofs might be higher or lower than 90%. So we get a full range of 20,000 to 1,000,000 users on the blockchain. If we increase the block size to 20mb, and decrease the storage proof rate per user to 1 per month, we get all the way up to 80,000,000 users.

The security of this system depends on a few things. What percentage of hosts stay on the network for 6 months or longer? If it's a high percentage, than it's more secure to do it this way. If most hosts aren't on the network for at least 6 months, then you have either reduce the number of hosts you use (which reduces security), or you need more frequent storage proofs (which decreases the number of users that fit on the network). Hard drives typically last around 18 months though, so it seems reasonable to assume that most hosts will be interested in sticking around for at least 6 months. (Housing leases last 12 months, etc.).

Using 6 month long contracts means that the host is only checked on every 6 months. Hosts also don't get paid until the end of 6 months. But, the long term incentives remain the exact same as if the time period were shorter. A host who is not storing all of the data will have an amortized profit that is lower than if they were storing all of the data. (the penalty from getting caught * likelihood of getting caught > money saved on storage by cheating). From this perspective, things are fully secure.

Does that mean hosts have to wait 6 months before getting paid? It's a bit nicer than that. If people are only making 6 month contracts, then a host will have to wait 6 months before getting their first storage paycheck. Hosts however are going to have many clients, and these clients are going to have expirations that are spread out. Once a host is getting paid, they will be getting paid much more often than every 6 months. Additionally, downloads will cost siacoins. Even if the storage contract has not expired yet, the hosts will be making money by serving downloads. So again it does not seem to be a huge problem.

It's now important to mention that all of these numbers are chosen by the users and by the market. Users can choose, if they desire, to only make contracts with hosts that last 1 week. They will need to spend a lot more on transaction fees though, and this will get expensive once there are many users and the blockchain is always full. And if users feel comfortable only having 50 hosts, and having 2 year contracts with each host, then they will pay substantially less on transaction fees.

But I feel confident that 6 month contracts and 200 hosts is around the optimal number. It'll be easier to know once there are more people on the network. As always, we will start conservatively (when there are fewer users, using more storage proofs is okay because they will still be cheap), and we will switch to more aggressive numbers as it becomes clear that more aggressive numbers are also secure.

=====

To summarize, the way we're setting up the defaults means that the network will have room for around 150,000 users. The settings required to make this happen should be quite secure. By reducing the security settings (which may still be extremely secure), we can get up to around 1,000,000 users. By increasing the block size, we can increase the number of users. In the absolute worst case scenario, we should still be able to support 20,000 users under our current model.

There's another nice feature though. Each user can have up to unlimited data. You can fit an infinite amount of data under each contract, at the cost of 32 bytes of storage proof size every time you double the amount of data. Proofs for 1GB are 960 bytes. Proofs for 1TB are 1280 bytes. Proofs for 1EB are 1920 bytes.

If users are willing to share storage contracts, scaling increases substantially, because storage contracts can have infinite size. This is one way we might achieve scalability beyond our initial 150k users. Large users are almost exactly the same weight on the blockchain as small users.

We might also be able to do cryptographic layering. For example, Dropbox would only count at 1 user. All of Dropbox! There might be a way to create large auditable services that use the blockchain, and then thousands or millions of people can use these auditable services instead of using the blockchain correctly. We're not sure yet how the security would work here, but there's a good chance that this is an avenue for further scalability (If we can figure that out, scalability will go up to probably several hundred million users). Solutions such as sidechains, treechains, and other innovations in the space also provide options for scalability.

The scaling I'm telling you about today (150k-ish users) is only the scaling that we already know how to achieve. We are confident that we can use additional tools to push the boundaries even farther. One example is storage proof 'skipping'. When a file contract expires, a host can submit a storage proof that, instead of being a full storage proof, contains a signature from the user that says "I tested the host off of the blockchain, and the host passed the test.". This reduces the size of the storage proof from 1kb to about 50 bytes (but it does require the user to be online when the contract expires).

=====

Hopefully this explains where my size numbers come from. 20k - 1m users is the number I'm currently using, but by the time we get there, we should have more options for increasing the scalability of the blockchain.

I'm happy to answer any further questions.

I believe the problem with 1.5.9 is you have to update to java 8. Get java 8 then "install" 1.5.9.

I'm assuming its the data pruning problem bitcoin faces as well as all blockchain coins do.

But of course I would like some elaboration too if you could.

Learn something everyday. Cool answer, thanks.

Yea, you mentioned this before. By the time 1 million users come there will definitely be a solution. Thats not a worry imo. Hardest part is and always was

to get users and hosts. Crypto is still not a very popular complicated thing. But I think Sia has a better chance than most.

Can't wait to see the GUI!  keep up the great work.  I keep thinking... just wait untill all torrents are hosted on Sia.  Say goodbye to megadownload!!!  And hello to Sia!  Anyone who has a sianote will be rich.

Can you elaborate on this?

I can confirm that the information here mostly correct. You can also learn more by visiting siacoin.com/siafunds.html and siacoin.com/sianotes.html.


So, I recently talked to Shawn (the Storj founder) and from what I understand, we are a lot farther along in the development process than they are. We use a proof-of-work blockchain to enforce all of our contracts, they don't - they only use the blockchain when it is necessary. We have our completely separate blockchain, where they are using counterparty on Bitcoin. One of the biggest elements is scaling. The amount that we can scale is limited by the amount of data we can put on the blockchain. Using a lot of payment-channel style optimizations, we can get somewhere between 20k and 1 million users to fit on Sia given the current technology stack. Once we're over 1 million users, we'll have to find something else.

I'm not sure how well Storj will scale, but their test group A showed that they needed to improve a lot of things. They believe that test group B is still a few months away. We are launching on June 7th.

One of the biggest differences between Sia and Storj is that Sia intentionally stayed off the radar (we've actually been around longer than Storj - Sia was originally invented at Hack MIT in 2013) until we had something to show everybody. We surfaced briefly to do a crowd fund because we were broke. We're now ready to rock and roll, and we have a client that is functional. We also have a very clear roadmap for the next 3 months, and I believe we'll be a satisfactory replacement for Bittorrent by the end of the year. The current 500mb filesize limit should be raised to well over 1TB by August.

If you are curious, you should try out the beta and tell us what you think.

I'm not sure if Storj plans on having an API, but Sia has an API that other apps and decentralized apps can hook into for decentralized storage. Imagine if XBMC started streaming movies and music off of Sia. And imagine if Streamium used Sia to put streams in an archive. These are both things that are already possible using our API, so long as you keep files below 500mb. (and again, this limit is going to be lifted in the next few months).

I hope this helps. We think that there's a lot to love about Sia.


Everything that we release has been and will continue to be cross-platform. We support Windows, Linux, and Mac for all official releases.

yes. You can even try the client and mining now, more info at siacoin.com.
but the coins mined now are play coins only, you could help testing. After 7th june then you can mine the real ones.

Is there a way I can mine Sia on a mac when it launches? Thanks

This has been addressed already in this thread.

Dev technical answer:


My non technical answer:

Nobody knows and nobody can say which will be better or if either will even be completed. All anyone can do is try their best

in their belief as to how a storage coin can work best. I decided long ago not to bother even following Storj cuz I liked Sia's

2 devs and what they're trying to do. If I'm right I'm rich, if I'm wrong oh well then Storj coin holders will be. Que sera sera.

By the way, an obvious difference is Sia has a launch date around the corner.

What makes this coin supperior to storj ?

NXT AE is simple.  I got my sianotes there no problem.

Can't wait for launch!!!

Guess you're not a regular Nxt user. Still it should be simple. Nxt does not need to be installed. Go to https://nxtforum.org/nrs-releases/
and choose 1.4.18. I have had problems with 1.5.9 as well. Verify the hash with something like onlinemd5.com and extract the folder to your
desktop. Double click run.bat and leave the command window open until blockchain is fully dled. Depending how fast your computer and
connection is it could take a couple hours to half a day. Open a browser and go to http://localhost:7876/. Type your passphrase and there
really should not be any single problem.

Tried using online wallets listed on nxt.org site - one requires registration, the other won't let me sign on with passphrase only, seems to want some other ID from wallet, the third link goes to a site which does not show a wallet anywhere.

Tried downgrading NXT client install from 1.5.9 to 1.4.18. Now the NSR server crashes when I try to launch wallet. I'll probably have to manually clear everything off the computer for NXT and do a clean install. I estimate I'm already 4 hours into trying to redeem these Sianotes overall, between reading on how to do it, downloading, installing, and creating the key, and trying to send the transaction to redeem the notes.

I do onsite computer service as my main business, and can say for a fact not one of my customers would be able to use this for any kind of business.

Interesting project, will follow it.

First to make it clear for others, David and Luke created Siacoin not to compete with Bitcoin or litecoin as a currency.
The sole purpose of Siacoin is to be an open source decentralized distributed cloud storage service to compete with
centralized cloud storage services. Siacoins should not have speculative value like all other coins. You mine for
consensus but the coins you get mining have value only because you sell them to people who want to store their files
on the Sia network and Siacoins are the only way to rent storage on Sia. Now anybody with storage can make money
as "mom and pop" datacenters.

The important thing to remember is the two devs created all this for profit just as Microsoft and Amazon provides cloud
storage for profit. The Siacoin project is a revenue stream for David and Luke's corporation they formed Nebulous inc.

How does Siacoin provide revenue for them? Through Siafunds. Siafunds are built into Sia and automatically receive
Siacoins as a 3.9% fee only and I mean only when a storage contract successfully completes. Siafunds do not get any
fees for regular transactions. Think of it this way. The Sia network allows anybody to buy some hard drives and provide
them as cloud storage without red tape or having to form a company. Whenever these hosts get paid for their
storage services, siafund takes 3.9% of it. Thats a damn good deal for everybody.

The devs turned down high paying jobs at Google to chase this. They were broke, so crowd funded on Nxt Asset Exchange
to get enough money to work on Sia for a year at least. It was successful and Nxt investors received 10% of Siafunds.
Nebulous owns the other 90%.

Its been almost a year since and a lot of ups and downs occurred including different iterations of Sia.

The fees Siafund collect are also not for buying yachts. Just like any corporation Nebulous needs the income to pump
back into Siacoin by hiring more people to make Sia as strong as needs to be to replace all centralized cloud services.
Ideally Nebulous will have hundreds or thousands of employees and Sia will be battle tested and trusted as the go
to storage service. If that happens then you can see how valuable Siafunds will be. And Nxt Siafund buyers would
deserve it with the risk they've been through. Buy some while you can on the Nxt AE if you want but now they are
pretty expensive. Siafunds will also be tradable within Sia according to the devs.

Read the whitepaper on http://www.siacoin.com/sia.pdf.
Read the history of this project https://bitcointalksearch.org/topic/m.6486684

watching this one...

Regards

Goldmaxx

This project look promising. I've never seen a project like this... design is superior to Storj.

Adding to my list.

I am considering acquiring sianotes over on the NXT AE.  What is your projected average-case ROI for siafund investors?

ok - im lost ... what did i miss? ...

claim? ...

#crysx