Topic: Proof of Storage (Read 1416 times)

P2P Protocol

BitTorrent Sync synchronizes your files using a peer-to-peer (P2P) protocol. This protocol is very effective for transferring large files across multiple devices, and is very similar to the powerful protocol used by applications like µTorrent and BitTorrent. The data is transferred in pieces from each of the syncing devices, and BitTorrent Sync chooses the optimal algorithm to make sure you have a maximum download and upload speed during the process.

The devices you setup to sync are connected directly using UDP, NAT traversal and UPnP port mapping. We also provide such additional methods of ensuring connectivity as relay and tracker servers. If your devices are on the same local network, BitTorrent Sync will use your LAN for faster synchronization.
Security

BitTorrent Sync was designed with privacy and security in mind. All the traffic between devices is encrypted with AES cypher and a 256-bit key created on the base of the secret—a random string (20 bytes or more) that is unique for every folder.

It’s our priority to make sure that nobody has unauthorized access to your folders. That’s why there are no 3rd party servers involved when syncing your files. All the files are stored only on your trusted devices, controlled and managed solely by you.

For the same reason we provide you with a quick and easy way to manage secrets. You can regularly change them and invite people by sharing a one-time secret instead of distributing a permanent one.
Secret

The secret is a randomly generated 21-byte key. It is Base32-encoded in order to be readable by humans. BitTorrent Sync uses dev/random (Mac, Linux) and the Crypto API (Windows) in order to produce a completely random string. This authentication approach is significantly stronger than a login/password combination used by other services. That's why using a secret generated by BitTorrent Sync is very safe and secure.

If you want even more security, BitTorrent Sync gives you a way to use a custom secret. Just create your own secret, encode it with Base64, and enter in the secret field for BitTorrent Sync. Note that a custom secret should be more than 40 characters long.
Peer Discovery

In order to find proper peers that have the same secret, Sync uses:

    Local peer discovery. All peers inside local network are discovered by sending broadcast packets. If there are peers with the same secret they respond to the broadcast message and connect.
    Peer exchange (PEX). When two peers are connected, they exchange information about other peers they know.
    Known hosts (folder settings). If you have a known host with a static ip:port, you can specify this in Sync client, so that it connects to the peer using this information.
    DHT. Sync uses DHT to distribute information about itself and obtain the information about other peers with this secret. Sync sends SHA2(Secret):ip:port to DHT to announce itself and will get a list of peers by asking DHT for the following key SHA2(Secret)
    BitTorrent tracker. BitTorrent Sync can use a specific tracker server to facilitate peer discovery. The tracker server sees the combination of SHA2(secret):ip:port and helps peers connect directly. The BitTorrent Sync tracker also acts like a STUN server and can help do a NAT traversal for peers so that they can establish a direct connection even behind a NAT.

We recommend that you use a tracker server instead of DHT for reasons of faster response and NAT traversal, so peers have a higher probability of networking directly.

Alternatively concatenate the timestamp/signature of the last block onto the file contents, and hash that? Would that work?

Yes that will require disk access, but we're assuming that disk rate is faster than network rate.

Then verification consists of actually sending it. Not too bw efficient indeed

The data and the hash would have to be the same size.  Or, if you like, the data *is* the hash.

Sure, you can verify authenticity of data (if i understand correctly, this is just glorified merkle tree/blockchain). But how does http://upload.wikimedia.org/wikipedia/commons/8/8c/Hashlink_timestamping.svg actually prevents people just storing the hash of data?

Yep, but you can verify somebody else stores the same data only if you have it already as well. What you do is simple cookie authentication:

a: generates a cookie like this: { r: random_bytes , sig: ecc_sign(random_bytes) }
b: cookie.r == ecc_verify(cookie.sig), if ok then computes sha256(cookie + data_checked_ + cookie) and sends it to a
a: does the same sha256 to verify b does indeed stores the data

Imho it's impossible to check somebody owns the data without having it as well - if you have only hash of the data, they could have only the hash too and so on. If you can come up with scheme to actually verify storage w/o having it too, I'd be very interested in how to do that!

There's also Bytecoin mentioned on the wiki, but its description was too wague for me.

An amazing amount of date is stored but rarely fetched. Your approach might not know until too late. Also, it requires needless (slow) network traffic to test. Since I'm supposing your proof-of-stake is related to (number of blocks stored) * (time stored/block). Gaming the system might be to a malicious node's advantage.

Sorry, I'm not sure I follow you.  The proof of work is retrieving the file piece asked for, so you can't delete file pieces otherwise you won't be able to provide the proof of work and receive your coins.  Since random file pieces are asked for, you can't delete any pieces.

That is an interesting concept. Thanks for the reference to a Bloom Filter!

The problem I see is that while you know the stored a data. You don't know if that store displaced something else. (They deleted the data afterwords)

Would you know on the next store? Does every byte stored need to go through the Bloom Filter each time? Article was TL;DR; (yet)

An alternate solution is just to pose a proof-of-storage piece of work. Obviously there would need to be redundant storage of each data block. Just ask each redundant node to re-hash the block using a random seed that you choose. They should all match.

The blockchain stores one small number for every file.  It is an accumulator which has all the file pieces added to it.  To verify file segment storage nodes retrieve the accumulator from the blockchain and check that the file segment is in the accumulator.