Author

Topic: [ANN] Sia - Decentralized Storage - page 420. (Read 1382189 times)

hero member
Activity: 543
Merit: 501
August 18, 2015, 05:51:43 AM
Currently there are 8 full copies stored on 8 different hosts. The future plan is to have dozens of hosts holding fragments with each fragment replicated a few times.

Starting with 0.4.0, which is just days away, files will be fragmented. 6 full copies will be stored across 12 hosts, erasure coded such that the original file can be rebuilt from any 2 of the hosts. The files will be (as they have been up to this point) encrypted, so even if the hosts try to talk to eachother, they still can't determine the original content of the files.

Also, we setup the slackin on our own servers now!

slackin.siacoin.com
legendary
Activity: 2087
Merit: 1015
August 17, 2015, 10:11:51 PM
In a later version you won't even be storing the entire file but a fragment of it

So with the current version, files arent fragmented in more than one hosts?

Currently there are 8 full copies stored on 8 different hosts. The future plan is to have dozens of hosts holding fragments with each fragment replicated a few times.
legendary
Activity: 1401
Merit: 1008
northern exposure
August 17, 2015, 07:48:12 PM
First at all thank you @bitspill for your reply, i really apreciate it.

In a later version you won't even be storing the entire file but a fragment of it

So with the current version, files arent fragmented in more than one hosts?
legendary
Activity: 2087
Merit: 1015
August 17, 2015, 06:38:45 PM
i have a noob question that maybe have been already answered, but i will take the risk and ask it.

I care about if there is some type of protecction about what type of archives can be stored if we rent our HDD space, i mean, imagine that some type of hacker or pedofilo use our HD space to store his virus/photos, are we protected againts that?

because i dont want to rent my space if "users" will use it with malicious intent, dont know if you guys understand what im trying to say.

All files are encrypted so you have no way of knowing what the user uploads. At the same time though all you're storing is random numbers since it's encrypted and no agency can prove it's cp/viruses or anything like that to get you in trouble.

In a later version you won't even be storing the entire file but a fragment of it
legendary
Activity: 1401
Merit: 1008
northern exposure
August 17, 2015, 11:46:36 AM
i have a noob question that maybe have been already answered, but i will take the risk and ask it.

I care about if there is some type of protecction about what type of archives can be stored if we rent our HDD space, i mean, imagine that some type of hacker or pedofilo use our HD space to store his virus/photos, are we protected againts that?

because i dont want to rent my space if "users" will use it with malicious intent, dont know if you guys understand what im trying to say.
sr. member
Activity: 432
Merit: 250
August 17, 2015, 11:24:54 AM
We need a low cost source for large 1 Tb+ drives for it to be competitive with external drives. Our current source only goes to 320 GB.

Do you mean like a cloud storage provider? Or more like a VPS?

--------

If anyone is interested in running a "bootstrap node", please PM me. If you are running a bootstrap node, it means that you'll be in the list of nodes that people connect to first. It means  that you need 98-99% uptime for your node, and it also means that people will probably be downloading a lot of blocks from you and using up a small amount of bandwidth. If Sia grows by 10x in the next month, you can expect to upload around 20-50GB from your node - it's not that much but it's also not trivial.

You will need forwarded ports. Ideally I think we'd like to have 10-20 people step forward as volunteers. The more volunteers that we get, the less strain that each volunteer will experience on their node.

Thanks!

This is for a stand-alone unit with internal storage which plugs into a LAN like a network attached storage (NAS) box, but runs as a personal server. You mentioned using RAID previously, but we have only one SATA port. There are 2 external USB ports so if these can be configured to use a pair of 3TB external drives then the unit could use the existing internal drive to store the blockchain and use the pair of external drives in RAID 0 for storage. Problem with external drives is they tend to have poor cooling though for continuous operation.

Part of it depends on the direction Sia wants to go - a large network of drives running in the home or a smaller network of clustered drives running in server farms.
legendary
Activity: 2870
Merit: 1091
--- ChainWorks Industries ---
August 17, 2015, 02:18:42 AM
We need a low cost source for large 1 Tb+ drives for it to be competitive with external drives. Our current source only goes to 320 GB.

Do you mean like a cloud storage provider? Or more like a VPS?

--------

If anyone is interested in running a "bootstrap node", please PM me. If you are running a bootstrap node, it means that you'll be in the list of nodes that people connect to first. It means  that you need 98-99% uptime for your node, and it also means that people will probably be downloading a lot of blocks from you and using up a small amount of bandwidth. If Sia grows by 10x in the next month, you can expect to upload around 20-50GB from your node - it's not that much but it's also not trivial.

You will need forwarded ports. Ideally I think we'd like to have 10-20 people step forward as volunteers. The more volunteers that we get, the less strain that each volunteer will experience on their node.

Thanks!

im happy to step forward taek ...

ill have a dedicated system ( or systems if we need more ) up and running specifically for this ...

let me know ...

#crysx
hero member
Activity: 543
Merit: 501
August 17, 2015, 02:09:31 AM
We need a low cost source for large 1 Tb+ drives for it to be competitive with external drives. Our current source only goes to 320 GB.

Do you mean like a cloud storage provider? Or more like a VPS?

--------

If anyone is interested in running a "bootstrap node", please PM me. If you are running a bootstrap node, it means that you'll be in the list of nodes that people connect to first. It means  that you need 98-99% uptime for your node, and it also means that people will probably be downloading a lot of blocks from you and using up a small amount of bandwidth. If Sia grows by 10x in the next month, you can expect to upload around 20-50GB from your node - it's not that much but it's also not trivial.

You will need forwarded ports. Ideally I think we'd like to have 10-20 people step forward as volunteers. The more volunteers that we get, the less strain that each volunteer will experience on their node.

Thanks!
sr. member
Activity: 432
Merit: 250
August 16, 2015, 06:40:17 PM
6. Siad uses a databse. Previously, the whole blockchain and output set was kept in memory. It has now been moved to disk. Siad's memory footprint is now around 40mb on Linux. Moving everything to disk made siad slower, but there are some pretty clear avenues to improve speeds. On an HDD (SSDs will be faster), it took me about 90 minutes to download the first 13,000 blocks.

This is very good news. We have had Sia running on the Bitseed Server, bitseed.org, to test it for storage coins. It has been running fine with 1 Gb of RAM, 4 Gb eMMC onbaord flash, 160 Gb hard drive, and dual core ARM processor, but for future growth, having the data on the HDD will work much better. Crypti had problems using up the RAM on the unit before the blockchain was moved to the HDD. Now it runs fine with 768 Mb or more of RAM, and can run multiple instances on a single CPU.

We need a low cost source for large 1 Tb+ drives for it to be competitive with external drives. Our current source only goes to 320 GB.
sr. member
Activity: 280
Merit: 250
August 16, 2015, 01:49:31 PM
Taek,

That's a whole lot of improvements and new features! You got the extra week or more if needed  Smiley lol, this is some significant progress over the previous versions.. larger file uploads is great..  that in itself should increase demand for those hosting SIA. Look forward to folders.

Is the port forwarding issue (Host Announce)  being addressed in the 4.0?

Ready to see everyone rallying back in here to upgrade when released.

Thanks for the update.
legendary
Activity: 2870
Merit: 1091
--- ChainWorks Industries ---
August 15, 2015, 08:48:32 AM
Hey guys, sorry for the let-down, but the new wallet is going to need another week. Here's a list of what's going to be available:

1. Erasure coding! All files are now uploaded to 12 hosts, and the full file can be recovered from any 2 hosts. The redundancy overhead is 6 (compared to the previous redundancy overhead of 8 ). 6 is still very high, twice the industry average. We're doing this because we don't want anyone losing their files, and we still feel that the network is in an early stage. As the network matures, we will continue to reduce the amount of redundancy overhead.

2. Parallel downloads, smarter uploads! Previously, files were downloaded from a single host, and speeds were limited to that host's upload bandwidth. Now, downloads are done in parallel.

3. Encrypted wallet! Previously, the wallet secret keys were kept unencrypted on disk. The keys are now encrypted on disk, and the wallet can be locked and unlocked to enable spending. When the wallet is unlocked, the keys are kept decrypted in memory. When the wallet is locked again, the memory gets wiped.

4. Wallet recovery seeds. All new addresses are now made from a single seed, which means that, as long as you have the seed, you can recover your addresses if you lose your  wallet. Note that previously existing wallet addresses cannot be regenerated by a seed. If you want to enjoy the full protection of wallet seeds, you are going to need to send all of your coins to a seed-generated address.

5. Increased wallet safety. Previously, people had been losing coins because the wallet would sometimes get confused and obliterate addresses it had previously sent money to. This only happened when multiple instances of siad were running, or if the user started moving files around without closing siad first. But it still happened sometimes, and coins were lost as a result. Now, the wallet will never overwrite an existing file, and will never destroy addresses that have been saved to disk. The wallet will never give an address that isn't already saved to disk. Upgrading siad no longer puts your coins at risk.

5. Full wallet API, which means that Sia is now ready to be put on an exchange!

6. Siad uses a databse. Previously, the whole blockchain and output set was kept in memory. It has now been moved to disk. Siad's memory footprint is now around 40mb on Linux. Moving everything to disk made siad slower, but there are some pretty clear avenues to improve speeds. On an HDD (SSDs will be faster), it took me about 90 minutes to download the first 13,000 blocks.

7. Increased max filesize. The maximum filesize allowed by the client has been increased to 5GB. This is because of the more intelligent uploading and downloading algorithms, and the more intelligent host protocols.

8. Refreshed UI. While the UI looks very similar to how it did previously, the architecture has been completely re-written. Most importantly, strong plugin support has been added, which makes it easy for developers to write their own plugins for the UI that use Siad or even talk to other daemons. People who have been excited about things like a Sia-based video streaming now have a platform on which to build such a thing.

I think that covers all of the major changes that will be available in the next release. There have also been a handful of bugfixes that are being released. We are planning on adding support for tracking and uploading folders in 0.4.1, which should be released a week or two after 0.4.0.

Thanks for hanging tight.

As a reminder, everything that we've written to date is open source. The code for Siad can be found here: https://github.com/NebulousLabs/Sia

The related projects and libraries that we've built (such as the UI) can be found here: https://github.com/NebulousLabs

and THIS is why sia has the support it does ...

tanx a great deal taek ...

amazing bloody work mate ...

#crysx
hero member
Activity: 543
Merit: 501
August 15, 2015, 07:40:25 AM
Hey guys, sorry for the let-down, but the new wallet is going to need another week. Here's a list of what's going to be available:

1. Erasure coding! All files are now uploaded to 12 hosts, and the full file can be recovered from any 2 hosts. The redundancy overhead is 6 (compared to the previous redundancy overhead of 8 ). 6 is still very high, twice the industry average. We're doing this because we don't want anyone losing their files, and we still feel that the network is in an early stage. As the network matures, we will continue to reduce the amount of redundancy overhead.

2. Parallel downloads, smarter uploads! Previously, files were downloaded from a single host, and speeds were limited to that host's upload bandwidth. Now, downloads are done in parallel.

3. Encrypted wallet! Previously, the wallet secret keys were kept unencrypted on disk. The keys are now encrypted on disk, and the wallet can be locked and unlocked to enable spending. When the wallet is unlocked, the keys are kept decrypted in memory. When the wallet is locked again, the memory gets wiped.

4. Wallet recovery seeds. All new addresses are now made from a single seed, which means that, as long as you have the seed, you can recover your addresses if you lose your  wallet. Note that previously existing wallet addresses cannot be regenerated by a seed. If you want to enjoy the full protection of wallet seeds, you are going to need to send all of your coins to a seed-generated address.

5. Increased wallet safety. Previously, people had been losing coins because the wallet would sometimes get confused and obliterate addresses it had previously sent money to. This only happened when multiple instances of siad were running, or if the user started moving files around without closing siad first. But it still happened sometimes, and coins were lost as a result. Now, the wallet will never overwrite an existing file, and will never destroy addresses that have been saved to disk. The wallet will never give an address that isn't already saved to disk. Upgrading siad no longer puts your coins at risk.

5. Full wallet API, which means that Sia is now ready to be put on an exchange!

6. Siad uses a databse. Previously, the whole blockchain and output set was kept in memory. It has now been moved to disk. Siad's memory footprint is now around 40mb on Linux. Moving everything to disk made siad slower, but there are some pretty clear avenues to improve speeds. On an HDD (SSDs will be faster), it took me about 90 minutes to download the first 13,000 blocks.

7. Increased max filesize. The maximum filesize allowed by the client has been increased to 5GB. This is because of the more intelligent uploading and downloading algorithms, and the more intelligent host protocols.

8. Refreshed UI. While the UI looks very similar to how it did previously, the architecture has been completely re-written. Most importantly, strong plugin support has been added, which makes it easy for developers to write their own plugins for the UI that use Siad or even talk to other daemons. People who have been excited about things like a Sia-based video streaming now have a platform on which to build such a thing.

I think that covers all of the major changes that will be available in the next release. There have also been a handful of bugfixes that are being released. We are planning on adding support for tracking and uploading folders in 0.4.1, which should be released a week or two after 0.4.0.

Thanks for hanging tight.

As a reminder, everything that we've written to date is open source. The code for Siad can be found here: https://github.com/NebulousLabs/Sia

The related projects and libraries that we've built (such as the UI) can be found here: https://github.com/NebulousLabs
sr. member
Activity: 280
Merit: 250
August 13, 2015, 10:56:57 PM
Maybe the OP or the website should add some info and DIY guide for new miners that come here..

Hmmm......I am starting to realize this is no ordinary cryptocoin.  Where is the market for Siafunds?  Or is there one?

As far as I'm aware (I'm lead dev btw, nice to meet you!), no siafunds have been bought or sold since they were distributed  Tongue

There is a strong lack of beginner's tutorials right now, mainly because we're still in the rapid development + prototyping phase. But with this next release I think we'll be graduating out of that into the growth+features phase. That will include a heavy emphasis on tutorials and getting people (users, developers, and enterprises) involved with Sia.

I'm happy to answer any questions  Smiley

Yes, I would love to see a full tutorial on the website, how to buy siacoins, how to use wallet, how to pay for storage, how to maintain payment for storage, how to retrieve files.  Assume that users know absolutely nothing about how Sia works.  
Since the beta it has been a learning process for most of us with lot of good old Q&A with the devs along the way, to become fluent at its use. You are coming in at a good time. Soon the 4.00 beta should be out and should prove to be a lot more user friendly. Lots to read in the thread, its all here believe it or not!  Wink
But well worth studying. Its all not so difficult, like anything else a little trial and error.. and then you got it
legendary
Activity: 938
Merit: 1000
August 13, 2015, 04:58:29 PM
The SIA slack channel is where most of the discussion takes place and where most people can get any help needed to get started. The invite link is in the OP and here:

https://siatalk-slackin.herokuapp.com/
legendary
Activity: 1470
Merit: 1004
August 13, 2015, 10:46:46 AM
Maybe the OP or the website should add some info and DIY guide for new miners that come here..

Hmmm......I am starting to realize this is no ordinary cryptocoin.  Where is the market for Siafunds?  Or is there one?

As far as I'm aware (I'm lead dev btw, nice to meet you!), no siafunds have been bought or sold since they were distributed  Tongue

There is a strong lack of beginner's tutorials right now, mainly because we're still in the rapid development + prototyping phase. But with this next release I think we'll be graduating out of that into the growth+features phase. That will include a heavy emphasis on tutorials and getting people (users, developers, and enterprises) involved with Sia.

I'm happy to answer any questions  Smiley

Yes, I would love to see a full tutorial on the website, how to buy siacoins, how to use wallet, how to pay for storage, how to maintain payment for storage, how to retrieve files.  Assume that users know absolutely nothing about how Sia works. 
legendary
Activity: 1036
Merit: 1000
August 12, 2015, 06:37:14 PM
What is the difference with storj please?
hero member
Activity: 543
Merit: 501
August 12, 2015, 05:47:18 PM
Maybe the OP or the website should add some info and DIY guide for new miners that come here..

Hmmm......I am starting to realize this is no ordinary cryptocoin.  Where is the market for Siafunds?  Or is there one?

As far as I'm aware (I'm lead dev btw, nice to meet you!), no siafunds have been bought or sold since they were distributed  Tongue

There is a strong lack of beginner's tutorials right now, mainly because we're still in the rapid development + prototyping phase. But with this next release I think we'll be graduating out of that into the growth+features phase. That will include a heavy emphasis on tutorials and getting people (users, developers, and enterprises) involved with Sia.

I'm happy to answer any questions  Smiley
legendary
Activity: 938
Merit: 1000
August 12, 2015, 03:41:18 PM
Maybe the OP or the website should add some info and DIY guide for new miners that come here..

Hmmm......I am starting to realize this is no ordinary cryptocoin.  Where is the market for Siafunds?  Or is there one?
legendary
Activity: 2898
Merit: 1017
August 12, 2015, 02:41:41 PM
Maybe the OP or the website should add some info and DIY guide for new miners that come here..


miners- https://github.com/NebulousLabs/Sia-GPU-Miner/releases/tag/v1.0.1

explorer - http://explore.siacoin.com
legendary
Activity: 938
Merit: 1000
August 12, 2015, 02:14:14 PM
I come from Crypti.  Take me to your leader.

Seriously, I am still trying to wrap my head around what has just happened with this partnership.  So far it sounds like a great match for both of us.  I'm still trying to absorb the whitepaper and understand the economics of your enterprise.  In any event, it shows great imagination so good luck!

I see you have a SIA trading thread.  Where do I find out the basics of mining SIA?  Where are you tracking / posting the current hash rate etc for SIA? 
Jump to: