Topic: [IDEA] Dirt cheap online storage - page 6. (Read 14501 times)
The client (doing the paying) should keep some small amount of the data being hosted. Then, at random, request random bytes of the hosted files and compare them. If they're OK, pay the money.
Can someone TL;DR this thread? I am not going to read so many walls of text.
It's a good idea, but there are a variety of ways to implement it, and the best route to go is still being discussed.
I actually really like that idea you just came up with.
It seemed almost implied, but I'll go ahead and state it: The hosts who hold the valid data at the end of the given time period (which could be as short as a minute or as long as a month) split the fee paid by the customer. So if the customer is paying $0.05/day for storing 100 GB, and 5 different hosts hold the data, then each host would receive $0.01/day as long as they are holding the data.
It also allows the customer to pay exactly what they want to pay, and the free market on the hosts side determines how many "backups" would be provided for that payment. A customer paying only $0.01/day for a certain amount of data might only get 2 backups, while a person paying $0.04/day might get 8 backups.
The biggest downside is bandwidth. There would be no reasonable way (that I can see) for a host to cap bandwidth on any data that they hold. So they could potentially be required to serve the data 24/7 indefinitely. I suppose this is workable - it mainly just means that hosts will have to take into account their bandwidth limits, and only host that space accordingly.
This is starting to sound more like paid torrenting than anything.
It seemed almost implied, but I'll go ahead and state it: The hosts who hold the valid data at the end of the given time period (which could be as short as a minute or as long as a month) split the fee paid by the customer. So if the customer is paying $0.05/day for storing 100 GB, and 5 different hosts hold the data, then each host would receive $0.01/day as long as they are holding the data.
It also allows the customer to pay exactly what they want to pay, and the free market on the hosts side determines how many "backups" would be provided for that payment. A customer paying only $0.01/day for a certain amount of data might only get 2 backups, while a person paying $0.04/day might get 8 backups.
The biggest downside is bandwidth. There would be no reasonable way (that I can see) for a host to cap bandwidth on any data that they hold. So they could potentially be required to serve the data 24/7 indefinitely. I suppose this is workable - it mainly just means that hosts will have to take into account their bandwidth limits, and only host that space accordingly.
This is starting to sound more like paid torrenting than anything.
Can someone TL;DR this thread? I am not going to read so many walls of text.
Thanks, I think we are getting somewhere.
However I am still tempted to try for competition and redundancy.
Assuming the market/escrow has the funds secured then how about not committing at the start of the month as to which storage provider to download the data back from during or at the end of the month? Instead, solicit from storage providers up to date indexes of which data blocks they do claim to have stored a copy of, while providing to them up to date counts of how many of their competitors claim to already have copied that block.
The storage providers would thus not be competing on price but on which blocks to store. They should prefer blocks none of their competitors have copies of yet. The more of their competitors who claim to have already stashed a copy of that block, the less desirable choosing that block as the next one to save a copy of ought to be.
The storage providers thus would be trying to get the newest blocks copied to their storage first instead of grabbing a copy of some older block, but also to grab a copy of any blocks that lost a lot of copies due to a lot of storage providers fading into the night.
The testing of whether they really have a block could be ongoing; each time one wants to host a block, they get it from one that has claimed to be storing a copy of that block. If we incite churn somehow, between them they will be in effect constantly checking each other's claims as to having a certain block stored. So lets list for each block not only how many claim to have a copy stored but also exactly when they made that claim or exactly when they made (according to their claim) that copy, and have some accounting mechanism along the lines of any copy that hasn't been copied by another provider in X amount of time is stale. It needs to be checked.
Since barring some clever proofs system the most complete check would be to download the entire block from them, maybe when it is stale it should also expire completely as soon as it has been verified, due to the verifying being simply to take possession of it. That is, maybe we should have the one that "verifies" it become a new holder of it on the list of who has a copy and since when and let the previous holder of it relinquish their claim so on the list it still shows the same number of claims-of-having-it even though someone else just checked one of the earlier claims in the process of grabbing a copy themselves to get themselves on the list of that block's current holders.
There is probably still room for gaming in this, as someone could set up large numbers of virtual machines that would grab blocks and claim not to have received a valid copy of the block, in order to attack a competitor's reputation. Maybe such a claim should therefore be a black mark against both parties: one seems possibly unable to copy a block correctly, the other seems possibly unable to provide a block correctly.
Hmm...
Also, why pay developers in devcoins? Well, devcoins are a side-effect of mining bitcoins. Admittedly namecoins and several other types of coin also are similar side-effects but the point is it costs no bitcoins to create them but they are sellable for bitcoins and fiat on markets so since no one seems to want to pay developers in bitcoins maybe these various side effect coins that happen while you mine bitcoins could be put to use in paying developers. People might feel more open-handed with things they get free as side-effects than with the main ore they are mainly mining for.
Also, the devcoins go out automatically. There is no need for a majority of devcoins minted to go to miners since miners already got paid for their work in bitcoins and in namecoins and maybe also in ixcoins and i0coins and coiledcoins, heck some might also have been paid in rucoins and/or geistgeld too, I certainly was for a while.
Why sell, or even pay people with, any ore you happen upon along side your gold in your goldmine, or alongside your platinum in your platinum mine, etc?
-MarkM-
However I am still tempted to try for competition and redundancy.
Assuming the market/escrow has the funds secured then how about not committing at the start of the month as to which storage provider to download the data back from during or at the end of the month? Instead, solicit from storage providers up to date indexes of which data blocks they do claim to have stored a copy of, while providing to them up to date counts of how many of their competitors claim to already have copied that block.
The storage providers would thus not be competing on price but on which blocks to store. They should prefer blocks none of their competitors have copies of yet. The more of their competitors who claim to have already stashed a copy of that block, the less desirable choosing that block as the next one to save a copy of ought to be.
The storage providers thus would be trying to get the newest blocks copied to their storage first instead of grabbing a copy of some older block, but also to grab a copy of any blocks that lost a lot of copies due to a lot of storage providers fading into the night.
The testing of whether they really have a block could be ongoing; each time one wants to host a block, they get it from one that has claimed to be storing a copy of that block. If we incite churn somehow, between them they will be in effect constantly checking each other's claims as to having a certain block stored. So lets list for each block not only how many claim to have a copy stored but also exactly when they made that claim or exactly when they made (according to their claim) that copy, and have some accounting mechanism along the lines of any copy that hasn't been copied by another provider in X amount of time is stale. It needs to be checked.
Since barring some clever proofs system the most complete check would be to download the entire block from them, maybe when it is stale it should also expire completely as soon as it has been verified, due to the verifying being simply to take possession of it. That is, maybe we should have the one that "verifies" it become a new holder of it on the list of who has a copy and since when and let the previous holder of it relinquish their claim so on the list it still shows the same number of claims-of-having-it even though someone else just checked one of the earlier claims in the process of grabbing a copy themselves to get themselves on the list of that block's current holders.
There is probably still room for gaming in this, as someone could set up large numbers of virtual machines that would grab blocks and claim not to have received a valid copy of the block, in order to attack a competitor's reputation. Maybe such a claim should therefore be a black mark against both parties: one seems possibly unable to copy a block correctly, the other seems possibly unable to provide a block correctly.
Hmm...
Also, why pay developers in devcoins? Well, devcoins are a side-effect of mining bitcoins. Admittedly namecoins and several other types of coin also are similar side-effects but the point is it costs no bitcoins to create them but they are sellable for bitcoins and fiat on markets so since no one seems to want to pay developers in bitcoins maybe these various side effect coins that happen while you mine bitcoins could be put to use in paying developers. People might feel more open-handed with things they get free as side-effects than with the main ore they are mainly mining for.
Also, the devcoins go out automatically. There is no need for a majority of devcoins minted to go to miners since miners already got paid for their work in bitcoins and in namecoins and maybe also in ixcoins and i0coins and coiledcoins, heck some might also have been paid in rucoins and/or geistgeld too, I certainly was for a while.
Why sell, or even pay people with, any ore you happen upon along side your gold in your goldmine, or alongside your platinum in your platinum mine, etc?
-MarkM-
For sure the storage providers should not to be able to read the files without the permission of the owners of the files.
There is plenty of free open source code that does that, in fact I have had the impression that part of what drove open source developers was often precisely the tendency of commercial service providers to set services up in such a way as to allow the providers access to as much of the customers' data as possible.
Just recently I was told that one share of devcoin generation is worth about 15 bitcoins. To me fifteen bitcoins is a nice amount to have flowing in regularly just as a side of effect of the fact I develop free open source stuff, and even a majority of the bitcoins I have made directly as bitcoins I received as bounty for working on devcoin and devcoin-related stuff.
It is true that each individual devcoin is worth far less than each bitcoin, but that was a deliberate design decision; bitcoins were skyrocketing back then, there was talk of having to move the decimal-point because users would find numbers with too many decimals awkward, so devcoin deliberately set out to be worth a thousandth or less of a bitcoin. 50,000 are minted per block accordingly.
You still haven't answered my question though: Why pay a developer in Devcoin instead of Bitcoin?There is plenty of free open source code that does that, in fact I have had the impression that part of what drove open source developers was often precisely the tendency of commercial service providers to set services up in such a way as to allow the providers access to as much of the customers' data as possible.
Just recently I was told that one share of devcoin generation is worth about 15 bitcoins. To me fifteen bitcoins is a nice amount to have flowing in regularly just as a side of effect of the fact I develop free open source stuff, and even a majority of the bitcoins I have made directly as bitcoins I received as bounty for working on devcoin and devcoin-related stuff.
It is true that each individual devcoin is worth far less than each bitcoin, but that was a deliberate design decision; bitcoins were skyrocketing back then, there was talk of having to move the decimal-point because users would find numbers with too many decimals awkward, so devcoin deliberately set out to be worth a thousandth or less of a bitcoin. 50,000 are minted per block accordingly.
Quote
The people who want to make coin by renting out space might be potential customers for the developers, but they are not potential customers for the users who supply the coins in the normal mainstream manner of thinking. Yes, bitcoin owners sometimes want to sell bitcoins, so any merchant willing to accept bitcoins is a customer of the person who sells them bitcoins in return for other goods or services, but if we take your view of wanting this project to be a consumer of bitcoins rather than a producer of bitcoins we want to be buying bitcoins not buying storage, right?
I have no idea what you mean by this. Again, the goal of the project is to match up users with too much storage with users with too little storage.Quote
The people who have bitcoins do sometimes look for customers to sell them to, so maybe this is about how many of their offered bitcoins we can afford to buy given the amount of development ability and storage we can offer in return for their bitcoins.
Maybe we have to look at this in two layers: one, the person who actually wants storage. Two, the person who wants to sell them some. Three, the facilitator(s), which presumably amount to being markets, offering to bring buyers and sellers together.
To operate such a market, maybe a whole new asset-type, OSA (Online Storage Asset) would be useful. Persons wanting to sell storage could look at a prospective customer's OSC balance to decide whether that customer even has enough OSC to buy the storage the storage-vendor wants to sell, and the people wanting to buy storage could look at vendors BTC (BiTCoin) balance to see if the vendor has enough BTC balance to recompense any loss of data purportedly being stored. The market could act as escrow both for both the data stored and the coins paid. And yes I am thinking I mean here escrow of the the actual data, not the abstract OSA asset. If the storage provider fails to provide the storage, the data stored on it can be recovered from the escrow aka the operator of the market? Hmm. Shoot that down for me, I am surely not seeing the problems with it clearly.
-MarkM-
I don't like the idea of assuming the loss of data can have a set price on it. Lots of people would deem their personal files that they hope to never lose as "priceless", especially pictures, videos, etc.Maybe we have to look at this in two layers: one, the person who actually wants storage. Two, the person who wants to sell them some. Three, the facilitator(s), which presumably amount to being markets, offering to bring buyers and sellers together.
To operate such a market, maybe a whole new asset-type, OSA (Online Storage Asset) would be useful. Persons wanting to sell storage could look at a prospective customer's OSC balance to decide whether that customer even has enough OSC to buy the storage the storage-vendor wants to sell, and the people wanting to buy storage could look at vendors BTC (BiTCoin) balance to see if the vendor has enough BTC balance to recompense any loss of data purportedly being stored. The market could act as escrow both for both the data stored and the coins paid. And yes I am thinking I mean here escrow of the the actual data, not the abstract OSA asset. If the storage provider fails to provide the storage, the data stored on it can be recovered from the escrow aka the operator of the market? Hmm. Shoot that down for me, I am surely not seeing the problems with it clearly.
-MarkM-
Having a facilitator manage a market of storage vendors would be interesting. Again, the issue of having 3 different metrics (size, speed, and bandwidth) would be present. Paring that down to a single metric, size, would be potentially doable, so long as a minimum speed and bandwidth are prerequisites to becoming a host in the first place.
So, I would imagine a storage asset would represent storage in the following manner:
- 1 storage asset = 1 GB stored for 1 month.
- The host is under contract to provide the stored data on demand one time at any point in that 1 month period in order to be paid. In other words, bandwidth down and bandwidth up would exactly equal the storage amount.
- The facilitator would act as escrow and hold the funds to be paid to the host. If a dispute is generated, then the facilitator finds out whether the host still has the files or not, and distributes the funds back to the buyer if the host does not. Simply put, the host does not get paid if they cannot provide the files at the end of the month long period, or at any period within that month (if the user requests their files for download prior to the end of the month).
- It is up to the user to pay for and store their files on multiple hosts, if they so desire, in order to provide for redundancy.
This would aid in providing a marketplace for storage, where each storage asset is more or less equal, and the user knows what they are getting.
I think it would be possible to eventually expand the marketplace to account for all three metrics (size, speed, and bandwidth) for purposes of hosting large files for other people to download, but obviously, it adds complexity to the accountability of the services, and would perhaps mandate the use of a feedback-like system for host rating.
For sure the storage providers should not to be able to read the files without the permission of the owners of the files.
There is plenty of free open source code that does that, in fact I have had the impression that part of what drove open source developers was often precisely the tendency of commercial service providers to set services up in such a way as to allow the providers access to as much of the customers' data as possible.
Just recently I was told that one share of devcoin generation is worth about 15 bitcoins. To me fifteen bitcoins is a nice amount to have flowing in regularly just as a side of effect of the fact I develop free open source stuff, and even a majority of the bitcoins I have made directly as bitcoins I received as bounty for working on devcoin and devcoin-related stuff.
It is true that each individual devcoin is worth far less than each bitcoin, but that was a deliberate design decision; bitcoins were skyrocketing back then, there was talk of having to move the decimal-point because users would find numbers with too many decimals awkward, so devcoin deliberately set out to be worth a thousandth or less of a bitcoin. 50,000 are minted per block accordingly.
The people who want to make coin by renting out space might be potential customers for the developers, but they are not potential customers for the users who supply the coins in the normal mainstream manner of thinking. Yes, bitcoin owners sometimes want to sell bitcoins, so any merchant willing to accept bitcoins is a customer of the person who sells them bitcoins in return for other goods or services, but if we take your view of wanting this project to be a consumer of bitcoins rather than a producer of bitcoins we want to be buying bitcoins not buying storage, right?
The people who have bitcoins do sometimes look for customers to sell them to, so maybe this is about how many of their offered bitcoins we can afford to buy given the amount of development ability and storage we can offer in return for their bitcoins.
Maybe we have to look at this in two layers: one, the person who actually wants storage. Two, the person who wants to sell them some. Three, the facilitator(s), which presumably amount to being markets, offering to bring buyers and sellers together.
To operate such a market, maybe a whole new asset-type, OSA (Online Storage Asset) would be useful. Persons wanting to sell storage could look at a prospective customer's OSA balance to decide whether that customer even has enough OSA to buy the storage the storage-vendor wants to sell, and the people wanting to buy storage could look at vendors BTC (BiTCoin) balance to see if the vendor has enough BTC balance to recompense any loss of data purportedly being stored. The market could act as escrow both for both the data stored and the coins paid. And yes I am thinking I mean here escrow of the actual data, not the abstract OSA asset. If the storage provider fails to provide the storage, the data stored on it can be recovered from the escrow aka the operator of the market? Hmm. Shoot that down for me, I am surely not seeing the problems with it clearly.
EDIT: I see one problem already: I seem to have shifted somewhere in that paragraph from OSA as a thing one buys storage with from storage vendors to OSA as actually stored data. I think that the confusion is that the OSA would amount to being coin/value that is commited to buying storage. Looking at someone's OSA balance is different from looking at their bitcoin balance because with bitcoins they might buy anything, so could spend it on something else. Their OSA balance though is purpose-specific. They already spent bitcoins or dollars or icecreamcones or sticks of gum or whatever to buy OSA units, now they are looking at vendors to decide which vendors to give them to. Something like that. I am getting confused.
...Maybe because of the layers. Coin vendors gave coins to the market/escrow, then the market/escrow is looking at storage vendors to choose which ones to obtain storage from...
-MarkM-
There is plenty of free open source code that does that, in fact I have had the impression that part of what drove open source developers was often precisely the tendency of commercial service providers to set services up in such a way as to allow the providers access to as much of the customers' data as possible.
Just recently I was told that one share of devcoin generation is worth about 15 bitcoins. To me fifteen bitcoins is a nice amount to have flowing in regularly just as a side of effect of the fact I develop free open source stuff, and even a majority of the bitcoins I have made directly as bitcoins I received as bounty for working on devcoin and devcoin-related stuff.
It is true that each individual devcoin is worth far less than each bitcoin, but that was a deliberate design decision; bitcoins were skyrocketing back then, there was talk of having to move the decimal-point because users would find numbers with too many decimals awkward, so devcoin deliberately set out to be worth a thousandth or less of a bitcoin. 50,000 are minted per block accordingly.
The people who want to make coin by renting out space might be potential customers for the developers, but they are not potential customers for the users who supply the coins in the normal mainstream manner of thinking. Yes, bitcoin owners sometimes want to sell bitcoins, so any merchant willing to accept bitcoins is a customer of the person who sells them bitcoins in return for other goods or services, but if we take your view of wanting this project to be a consumer of bitcoins rather than a producer of bitcoins we want to be buying bitcoins not buying storage, right?
The people who have bitcoins do sometimes look for customers to sell them to, so maybe this is about how many of their offered bitcoins we can afford to buy given the amount of development ability and storage we can offer in return for their bitcoins.
Maybe we have to look at this in two layers: one, the person who actually wants storage. Two, the person who wants to sell them some. Three, the facilitator(s), which presumably amount to being markets, offering to bring buyers and sellers together.
To operate such a market, maybe a whole new asset-type, OSA (Online Storage Asset) would be useful. Persons wanting to sell storage could look at a prospective customer's OSA balance to decide whether that customer even has enough OSA to buy the storage the storage-vendor wants to sell, and the people wanting to buy storage could look at vendors BTC (BiTCoin) balance to see if the vendor has enough BTC balance to recompense any loss of data purportedly being stored. The market could act as escrow both for both the data stored and the coins paid. And yes I am thinking I mean here escrow of the actual data, not the abstract OSA asset. If the storage provider fails to provide the storage, the data stored on it can be recovered from the escrow aka the operator of the market? Hmm. Shoot that down for me, I am surely not seeing the problems with it clearly.
EDIT: I see one problem already: I seem to have shifted somewhere in that paragraph from OSA as a thing one buys storage with from storage vendors to OSA as actually stored data. I think that the confusion is that the OSA would amount to being coin/value that is commited to buying storage. Looking at someone's OSA balance is different from looking at their bitcoin balance because with bitcoins they might buy anything, so could spend it on something else. Their OSA balance though is purpose-specific. They already spent bitcoins or dollars or icecreamcones or sticks of gum or whatever to buy OSA units, now they are looking at vendors to decide which vendors to give them to. Something like that. I am getting confused.
...Maybe because of the layers. Coin vendors gave coins to the market/escrow, then the market/escrow is looking at storage vendors to choose which ones to obtain storage from...
-MarkM-
I'm intrigued... Sign me up for both sides of the fence if you ever get it up and running.
Also, make damn sure every file uploaded is on more than one or two drives and encrypted with keys that the storage providers don't have
Also, make damn sure every file uploaded is on more than one or two drives and encrypted with keys that the storage providers don't have
Devcoins are virtually worthless - I don't know why a developer would work to gain them. And why not use Bitcoins? How is Devcoin ANY better suited to pay developers than Bitcoin, aside from name? I didn't mention developers getting paid because it absolutely depends on who is willing to develop something like this, and whether they would like to be paid or not. If there is a central entity involved, then it is fairly likely that they would take a cut of the hosting fees gathered, and would be willing to front the costs of having a platform like this developed. If it is a truly decentralized project with no central entity involved, then either some developer needs to believe in it enough to put it together without pay, or some bounty needs to be posted by users interested in seeing it happen.
I made it all about the users because, ultimately, a platform developed without the users in mind is worthless. So, first figure out what the users want, then decide on how it will be developed and managed. Right now, we're still in the "figure out what the users want" bit, which means how the platform is developed and whether developers are paid or not is largely irrelevant. All I was saying in my previous comment is, I'd like to see a platform that allows developers to be paid, either via a central entity taking a cut of the fees, or via a bounty. That's not a necessity, and obviously, it depends on what users want and need.
I don't know why home users should be ignored, either. If they can provide storage at a lower cost than VPS providers (which are typically very expensive), then why not include them? Reselling professional hosting is not the goal here.
I made it all about the users because, ultimately, a platform developed without the users in mind is worthless. So, first figure out what the users want, then decide on how it will be developed and managed. Right now, we're still in the "figure out what the users want" bit, which means how the platform is developed and whether developers are paid or not is largely irrelevant. All I was saying in my previous comment is, I'd like to see a platform that allows developers to be paid, either via a central entity taking a cut of the fees, or via a bounty. That's not a necessity, and obviously, it depends on what users want and need.
I don't know why home users should be ignored, either. If they can provide storage at a lower cost than VPS providers (which are typically very expensive), then why not include them? Reselling professional hosting is not the goal here.
There is already at least one commercial pass-through app, or something very similar to it:
http://smestorage.com/?p=static&page=about#about-clouds
Google Drive does not mention mounting it as a drive in unix/linux using webdav protocol, but the chap who gave me the above link seemed to be saying that using that service he could use webdav to mount his google drive space.
Unthinkingbit too is concerned that developers of free open source stuff often are not paid well; that is why DeVCoins exist. Maybe DeVCoins would be more suitable than BiTCoins for developing this concept since implicit in DeVCoins is developers get coins. So far in your concept this is the first I have heard of any hint of a whisper of a suggestion that developers should get paid, it was all about users (of the software and the network of users it marshals) getting paid.
I wonder whether we even need random home-users though, maybe we should be looking at operators of virtual machines as our target nodes for acting as host nodes. Home users can be target consumers of storage, no need to use any of their own on site storage at all, what we would want them to contribute is coinage, not storage. Even if they do have lots of storage available, so what, that just means they are not likely to be a paying customer. No need to concern ourselves with them until they do not have enough storage and thus go shopping around for more.
In fact if we could estimate very firmly how many terrabytes of storage a large ISP's home-users would offer to sell and how much bandwidth that would cost the ISP maybe we could even go directly to the ISPs suggesting they sell us the storage cheaper than their users could due to not having the bandwidth cost of sending our data all the way to their users and back...
-MarkM-
http://smestorage.com/?p=static&page=about#about-clouds
Google Drive does not mention mounting it as a drive in unix/linux using webdav protocol, but the chap who gave me the above link seemed to be saying that using that service he could use webdav to mount his google drive space.
Unthinkingbit too is concerned that developers of free open source stuff often are not paid well; that is why DeVCoins exist. Maybe DeVCoins would be more suitable than BiTCoins for developing this concept since implicit in DeVCoins is developers get coins. So far in your concept this is the first I have heard of any hint of a whisper of a suggestion that developers should get paid, it was all about users (of the software and the network of users it marshals) getting paid.
I wonder whether we even need random home-users though, maybe we should be looking at operators of virtual machines as our target nodes for acting as host nodes. Home users can be target consumers of storage, no need to use any of their own on site storage at all, what we would want them to contribute is coinage, not storage. Even if they do have lots of storage available, so what, that just means they are not likely to be a paying customer. No need to concern ourselves with them until they do not have enough storage and thus go shopping around for more.
In fact if we could estimate very firmly how many terrabytes of storage a large ISP's home-users would offer to sell and how much bandwidth that would cost the ISP maybe we could even go directly to the ISPs suggesting they sell us the storage cheaper than their users could due to not having the bandwidth cost of sending our data all the way to their users and back...
-MarkM-
I'd gladly free up and manage some gigs to be paid some Bitcoin. I'd probably also back up the the data, if the pay were enough I'd invest in a RAID.
I am not typically interested in any project that isn't "in it for the money". Namely because those projects usually fail, either due to lack of developer incentive (who wants to put a bunch of time and effort into something that doesn't give them something in return?), or lack of updates, or lack of moderation, or lack of users, etc. Money gives project organizers and users incentives to use the project, which is why I like projects in which the developers (and/or users) receive monetary compensation. As you've seen with retroshare, not many people are inspired to continue to use a product that is virtually useless to them unless they are being paid to do so.
Also remember that a lot of people using Bitcoin are only in it for the money too - either in it for the long run, because they believe Bitcoins will be more valuable in the future, or in it to make money mining, etc. Some use it because they believe in the fundamentals of decentralized and deflationary currency, but I doubt that the majority of Bitcoin users fit that profile...
I don't think insurance for users' data would be necessary if all customers paid the same price, and all hosts received the same compensation. Make 2 the minimum number of hosts at which data can be stored with, and if one of those hosts drops offline (which would likely happen very rarely, if 99%+ uptime hosts are chosen), then the 2nd host would simply copy the data over to a 3rd host.
A pass-through app utilizing multiple smaller free hosting solutions would be interesting, but I wonder what the developer incentive would be to create and continually update such an application. I also wonder if various hosts with free storage would simply block that application from using their storage since it could be seen as an abuse of their free accounts.
Also remember that a lot of people using Bitcoin are only in it for the money too - either in it for the long run, because they believe Bitcoins will be more valuable in the future, or in it to make money mining, etc. Some use it because they believe in the fundamentals of decentralized and deflationary currency, but I doubt that the majority of Bitcoin users fit that profile...
I don't think insurance for users' data would be necessary if all customers paid the same price, and all hosts received the same compensation. Make 2 the minimum number of hosts at which data can be stored with, and if one of those hosts drops offline (which would likely happen very rarely, if 99%+ uptime hosts are chosen), then the 2nd host would simply copy the data over to a 3rd host.
A pass-through app utilizing multiple smaller free hosting solutions would be interesting, but I wonder what the developer incentive would be to create and continually update such an application. I also wonder if various hosts with free storage would simply block that application from using their storage since it could be seen as an abuse of their free accounts.
I forgot to take into account the fact that of Nimbus.io's ten storage nodes per cluster only any two of the ten can vanish without losing data.
So each such cluster only really amounts to three copies of your data in a way since if three nodes go down ouch, pow, data loss.
They may well customarily keep people's data on more than one cluster at a time though.
I prefer watching the RetroShare folk looking out for potentially reliable ones to advertising a bribe because the bribe might bring all the bottom-feeders swarming in, all desperately trying to find a way to game the system. If I actually planned to attract those people I would try to design a system that would get as much use out of them as it can during the time they spend trying to game the system and failing, basically in effect planning to game those that try to game the system.
Part of why I was attracted to the RetroShare thread was they were not ostensibly just in it for money; it is even possible that what failed really was the backup bitcoin-forum the thread initially proposed. Actually it might not even be a failure, since maybe if bitcointalk forum does go down some day, all those RetroShare people will all fire back up their RetroShare, reconnect, and start actually using the bitcoin forums they created there. So maybe the real test of their reliability would be to turn off bitcointalk and see how many of the people hosting the backup forum on RetroShare actually do bring RetroShare back up.
Amazon's spot instances do use an auction where you only pay the minimum price needed to buy as much as people want-at-that-price. Everyone who gets an instance-hour in a given hour pays the same price per hour.
The why not dump all your users if someone else offers more could maybe be something insurance companies could cover? Post a bond with an insurance company, it then insures your users against you losing the users' data.
However I think maybe we are basically starting to glimpse *why* online storage *is* so darn expensive.
Maybe some pass-through apps would be useful, that let people go get their free five gigs of dropbox, google, etc, etc, all kinds of free for life storage, then do passthrough to others...
-MarkM-
So each such cluster only really amounts to three copies of your data in a way since if three nodes go down ouch, pow, data loss.
They may well customarily keep people's data on more than one cluster at a time though.
I prefer watching the RetroShare folk looking out for potentially reliable ones to advertising a bribe because the bribe might bring all the bottom-feeders swarming in, all desperately trying to find a way to game the system. If I actually planned to attract those people I would try to design a system that would get as much use out of them as it can during the time they spend trying to game the system and failing, basically in effect planning to game those that try to game the system.
Part of why I was attracted to the RetroShare thread was they were not ostensibly just in it for money; it is even possible that what failed really was the backup bitcoin-forum the thread initially proposed. Actually it might not even be a failure, since maybe if bitcointalk forum does go down some day, all those RetroShare people will all fire back up their RetroShare, reconnect, and start actually using the bitcoin forums they created there. So maybe the real test of their reliability would be to turn off bitcointalk and see how many of the people hosting the backup forum on RetroShare actually do bring RetroShare back up.
Amazon's spot instances do use an auction where you only pay the minimum price needed to buy as much as people want-at-that-price. Everyone who gets an instance-hour in a given hour pays the same price per hour.
The why not dump all your users if someone else offers more could maybe be something insurance companies could cover? Post a bond with an insurance company, it then insures your users against you losing the users' data.
However I think maybe we are basically starting to glimpse *why* online storage *is* so darn expensive.
Maybe some pass-through apps would be useful, that let people go get their free five gigs of dropbox, google, etc, etc, all kinds of free for life storage, then do passthrough to others...
-MarkM-
I think you're still missing the mark on the retroshare - those people didn't stay online namely BECAUSE there was no incentive, present or future, offered. If they knew they could make money by staying online (or similarly, if they could build up host reputation to then rely upon to make money at some point in the future), then many more of them would likely still be running the software.
You do bring up a good point about a person being "outbid". After all, what host wouldn't just drop their current users in favor of users offering more money for the same storage space? One way to mitigate that problem might be to use a central entity to receive payments, then pay out those payments equally to all hosts offering storage. The cost to store would be equal to the highest minimum price paid for storage available. Expanding on my previous 5 hosts example then:
Host 1 charges $0.03/GB/month
Host 2 charges $0.04/GB/month
Host 3 charges $0.05/GB/month
Host 4 charges $0.05/GB/month
Host 5 charges $0.06/GB/month
Host 6 charges $0.07/GB/month
Host 7 charges $0.07/GB/month
Host 8 charges $0.08/GB/month
Host 9 charges $0.09/GB/month
Host 10 charges $0.09/GB/month
If all of the customers combined only needed enough storage to warrant the use of 6 of those hosts, then the lowest-priced 6 hosts would be used to determine price. The customer would be charged the rate of host 6 for ALL of their storage (for example, for 3 hosts, they'd be charged $0.07*3 = $0.21/GB), and the payout would be equal to all hosts ($0.07/GB/month). In this way, there would be no incentive for a host to change whose files they are storing, though it would make it more expensive and variable-cost for the customer. Again, it'd be like the GPUMax of storage.
Aside from that solution or doing the "pay to recover/download" model, I can't think of any other way to make it work. I do like the pay to recover method for its simplicity though.
You do bring up a good point about a person being "outbid". After all, what host wouldn't just drop their current users in favor of users offering more money for the same storage space? One way to mitigate that problem might be to use a central entity to receive payments, then pay out those payments equally to all hosts offering storage. The cost to store would be equal to the highest minimum price paid for storage available. Expanding on my previous 5 hosts example then:
Host 1 charges $0.03/GB/month
Host 2 charges $0.04/GB/month
Host 3 charges $0.05/GB/month
Host 4 charges $0.05/GB/month
Host 5 charges $0.06/GB/month
Host 6 charges $0.07/GB/month
Host 7 charges $0.07/GB/month
Host 8 charges $0.08/GB/month
Host 9 charges $0.09/GB/month
Host 10 charges $0.09/GB/month
If all of the customers combined only needed enough storage to warrant the use of 6 of those hosts, then the lowest-priced 6 hosts would be used to determine price. The customer would be charged the rate of host 6 for ALL of their storage (for example, for 3 hosts, they'd be charged $0.07*3 = $0.21/GB), and the payout would be equal to all hosts ($0.07/GB/month). In this way, there would be no incentive for a host to change whose files they are storing, though it would make it more expensive and variable-cost for the customer. Again, it'd be like the GPUMax of storage.
Aside from that solution or doing the "pay to recover/download" model, I can't think of any other way to make it work. I do like the pay to recover method for its simplicity though.
There seemed no point in even considering offering any deals/offers/incentives to the RetroShare people, since long before enough time had gone by for me to start thinking "hey, some of these people seem to have been online 24/7 for a month or more maybe that could be useful" all but one of them had already demonstrated total lack of reliability. Heck it was days I think, though my sense of time is poor so maybe over a week, I suppose it may even have hit two weeks.
You make a good point though, good enough maybe to motivate me to turn on RetroShare every week-or-few and make deliberate note, maybe even written not merely a mental note, if I do happen to see a neighbor, of which one it is; so as to notice over time whether in fact it is the same one every time. If so that one might be worth thinking of for a scheme like this.
Basically the evidence was so overwhelming that the people were not reliable that the idea of offering to buy reliable service from them never even came up.
However I recently read somewhere that the length of time a host has reliably been online isn't actually much use in predicting whether they will stay online. Maybe some insight into whether they are actually making enough to cover expenses could help make such predictions though.
It seems a little like Amazon spot instances though in a way; any time the current bids outbid you, you lose your instances. Maybe the moment someone outbids you for storage, poof as much of your data as a financial-attack attacker chooses to displace with their own could be gone.
So maybe another metric to track would be what happens when you are outbid? Which hosts retain your data for how long in spite of higher bids being made bidding for the space they are providing to you?
Maybe we need time-scaled markets, covering for how long they will retain your data to give you a chance to move it or to increase your bid in the event you are outbid by how much? (If they are offered double the price you offer, how long grace time do you get? What if someone offers 10 times as much? 100 times as much? The ethical/moral equivalent of pool-hoppers will presumably be out there...)
-MarkM-
You make a good point though, good enough maybe to motivate me to turn on RetroShare every week-or-few and make deliberate note, maybe even written not merely a mental note, if I do happen to see a neighbor, of which one it is; so as to notice over time whether in fact it is the same one every time. If so that one might be worth thinking of for a scheme like this.
Basically the evidence was so overwhelming that the people were not reliable that the idea of offering to buy reliable service from them never even came up.
However I recently read somewhere that the length of time a host has reliably been online isn't actually much use in predicting whether they will stay online. Maybe some insight into whether they are actually making enough to cover expenses could help make such predictions though.
It seems a little like Amazon spot instances though in a way; any time the current bids outbid you, you lose your instances. Maybe the moment someone outbids you for storage, poof as much of your data as a financial-attack attacker chooses to displace with their own could be gone.
So maybe another metric to track would be what happens when you are outbid? Which hosts retain your data for how long in spite of higher bids being made bidding for the space they are providing to you?
Maybe we need time-scaled markets, covering for how long they will retain your data to give you a chance to move it or to increase your bid in the event you are outbid by how much? (If they are offered double the price you offer, how long grace time do you get? What if someone offers 10 times as much? 100 times as much? The ethical/moral equivalent of pool-hoppers will presumably be out there...)
-MarkM-
Cool idea but seems like way too many points of failure
MarkM, I respectfully disagree with your assessment that fly-by-night random hosts would require more copies of the data. The whole idea of host uptime is to ensure that fly-by-night random hosts aren't used. Right from the start, files would only be stored on hosts with more than, say, 99% uptime, and priority of storage would be given to hosts with the greatest uptime beyond that.
Retroshare did not offer your friends from Bitcointalk enough incentive to stay online, so they went offline. But if someone is paying me to keep my computer online, especially when my computer is already online 24/7 anyway, then I'll keep running the software, even if it is only giving me pennies a day. That's still more than I had the day before.
Does Google drive charge only for what it used? That is another manner in which this could be a useful service. Not forcing users to pay for more storage than they actually use. That said, GDrive pricing is very competitive!
I'd be willing to go even lower on my personal computer to compete too. Heck, I'd rent out 100GB for $1 a month! That'd make use of my currently-unused 1TB WDG drive, to the tune of $10/month if fully rented.
Retroshare did not offer your friends from Bitcointalk enough incentive to stay online, so they went offline. But if someone is paying me to keep my computer online, especially when my computer is already online 24/7 anyway, then I'll keep running the software, even if it is only giving me pennies a day. That's still more than I had the day before.
Does Google drive charge only for what it used? That is another manner in which this could be a useful service. Not forcing users to pay for more storage than they actually use. That said, GDrive pricing is very competitive!
I'd be willing to go even lower on my personal computer to compete too. Heck, I'd rent out 100GB for $1 a month! That'd make use of my currently-unused 1TB WDG drive, to the tune of $10/month if fully rented.
Subscribed so i can come back and read everything later 
very interested.
very interested.
Google Drive actually beats your example quoted price, something like $4.95 or $4.99 per month for 100 gigs of storage, no mention of any upload/download fees on top of that, at least not that I noticed. Like dropbox I guess they are going for simple instead of frightening the common people off with nickel and dime stuff that could add up to some unknown in advance total. If you want larger quotas, prices per terrabyte keep going down in more steps as you upgrade to larger quotas.
Nimbus.io offers $6 for 100 gigs, with also $6 per 100 gigs of transfer. ($0.06 per gig for each), and they are free open source software so that anyone who wants to set up storage servers in clusters of ten for parity-striping of data onto ten servers at a time can do so. (So we know they are based on ten hosts at a time, so lets figure we'd have to charge no more than a tenth what they do if we want to offer one host at a time prices.) Their software obviously is set up to do just the kind of pricing in your example quote so maybe if you set up ten virtual machines on your machine you could use it even for just one machine. It has the website, customer info handling and everything by the look of it.
I have found out more about GNUnet. It actually does not "publish" files out onto the grid at all, replication is only an efficiency thing, in that if anyone actually does ever ask for a copy of your file, then it will tend to persist some time at the mode that requested it and possibly even at places along the way to the requesting node. So you don't "push" your files out, instead someone out there has to request from you. Still, given a way of telling a particular node a file to request from you, it could be useful.
Unfortunately though I think most fly by night random hosts that fire up some software to see if it will earn some pennies are not going to be useful for storage. They could provide nice random topology for complicated routing of transfers for anonymity purposes but there is so much chance of the user turning off the machine when they go to bed, or killing the app after half an hour to a few hours when they get bored of playing with it and have not seen any pennies trickling in, that that there seems little point sending them any blocks to store except maybe a canary - a test block to ask for next week if they even manage to stay online a week, which they likely won't.
Did you see the Retroshare thread? I had 28 or more bitcointalk users as "friends" in that friend to friend network, and within a few days tended to have usually no connections, occassionally seeing one connection and maybe catching a glimpse of two for a moment. And this is bitcoiners, a population among whom many have machines that are running 24/7. Among "normal" people I wouldn't be surprised to see even worse results.
I would have needed every single one of those 28 people to have a copy of my data in order to have found one copy still reachable ofter a few days or maybe a week or two, the short time it took for the vast majority of them to fade into the night.
So your idea of lets buy three sites worth seems tragically low. Better buy 30, and still recognise that just gives you a chance one might survive to the end of the month maybe, if the people involved are as used, on average to keeping their machines on 24/7 as bitcoiners (or bitcointalk members anyway) on average are.
-MarkM-
Nimbus.io offers $6 for 100 gigs, with also $6 per 100 gigs of transfer. ($0.06 per gig for each), and they are free open source software so that anyone who wants to set up storage servers in clusters of ten for parity-striping of data onto ten servers at a time can do so. (So we know they are based on ten hosts at a time, so lets figure we'd have to charge no more than a tenth what they do if we want to offer one host at a time prices.) Their software obviously is set up to do just the kind of pricing in your example quote so maybe if you set up ten virtual machines on your machine you could use it even for just one machine. It has the website, customer info handling and everything by the look of it.
I have found out more about GNUnet. It actually does not "publish" files out onto the grid at all, replication is only an efficiency thing, in that if anyone actually does ever ask for a copy of your file, then it will tend to persist some time at the mode that requested it and possibly even at places along the way to the requesting node. So you don't "push" your files out, instead someone out there has to request from you. Still, given a way of telling a particular node a file to request from you, it could be useful.
Unfortunately though I think most fly by night random hosts that fire up some software to see if it will earn some pennies are not going to be useful for storage. They could provide nice random topology for complicated routing of transfers for anonymity purposes but there is so much chance of the user turning off the machine when they go to bed, or killing the app after half an hour to a few hours when they get bored of playing with it and have not seen any pennies trickling in, that that there seems little point sending them any blocks to store except maybe a canary - a test block to ask for next week if they even manage to stay online a week, which they likely won't.
Did you see the Retroshare thread? I had 28 or more bitcointalk users as "friends" in that friend to friend network, and within a few days tended to have usually no connections, occassionally seeing one connection and maybe catching a glimpse of two for a moment. And this is bitcoiners, a population among whom many have machines that are running 24/7. Among "normal" people I wouldn't be surprised to see even worse results.
I would have needed every single one of those 28 people to have a copy of my data in order to have found one copy still reachable ofter a few days or maybe a week or two, the short time it took for the vast majority of them to fade into the night.
So your idea of lets buy three sites worth seems tragically low. Better buy 30, and still recognise that just gives you a chance one might survive to the end of the month maybe, if the people involved are as used, on average to keeping their machines on 24/7 as bitcoiners (or bitcointalk members anyway) on average are.
-MarkM-
I do not know how many different sites dropbox has with redundant data. A cursory google search reveals no further details on it either.
On pricing, I don't think it needs to be a managed item as long as it is built in to the software. For example, there might be 5 hosts available with 100 GB of free storage space.
Host 1 charges $0.03/GB/month
Host 2 charges $0.04/GB/month
Host 3 charges $0.05/GB/month
Host 4 charges $0.05/GB/month
Host 5 charges $0.06/GB/month
I go to rent 100 GB of space. I am given 3 options: 2 hosts, 3 hosts, or 4 hosts, with prices for each level. 2 hosts will (currently) cost me $0.07/GB/month. 3 hosts will cost me $0.12/GB/month. And 4 hosts will cost me $0.17/GB/month. More hosts will protect my data more securely. But, if one host goes offline, then another host will take its place, although possibly at a higher price (if no further lower priced hosts are available).
I think it would certainly be possible to utilize a modified version of Tahoe-LAFS, but I do not have the knowledge or ability to make the code changes necessary myself. That's why I haven't really spoken to what software should be used much - it would be more up to the programmer creating the software than anything else, as to whether he'd like to start from scratch or build on an existing project, and which project that might be.
On pricing, I don't think it needs to be a managed item as long as it is built in to the software. For example, there might be 5 hosts available with 100 GB of free storage space.
Host 1 charges $0.03/GB/month
Host 2 charges $0.04/GB/month
Host 3 charges $0.05/GB/month
Host 4 charges $0.05/GB/month
Host 5 charges $0.06/GB/month
I go to rent 100 GB of space. I am given 3 options: 2 hosts, 3 hosts, or 4 hosts, with prices for each level. 2 hosts will (currently) cost me $0.07/GB/month. 3 hosts will cost me $0.12/GB/month. And 4 hosts will cost me $0.17/GB/month. More hosts will protect my data more securely. But, if one host goes offline, then another host will take its place, although possibly at a higher price (if no further lower priced hosts are available).
I think it would certainly be possible to utilize a modified version of Tahoe-LAFS, but I do not have the knowledge or ability to make the code changes necessary myself. That's why I haven't really spoken to what software should be used much - it would be more up to the programmer creating the software than anything else, as to whether he'd like to start from scratch or build on an existing project, and which project that might be.
Jump to: