Pages:
Author

Topic: [ANN] Oyster Shell (SHL) - Dawn of a New Era - page 2. (Read 4002 times)

full member
Activity: 770
Merit: 156
Quote
Here is a video of one of Oyster's developers, Rebel, explaining how the payment system works in the Oyster infrastructure: https://youtu.be/uZ5ZnFjV23o
full member
Activity: 770
Merit: 156
The Oyster development team is pleased to announce that the initial file size limit of 5MB associated with the mainnet launch of the Oyster Storage platform has been increased to 25MB.

The development team optimized several parts of the platform to increase the original file size limit from 5MB to 25MB. For example, previously the client would process the entire file before sending all the chunks. For extensive files, this would cause memory crashes in the browser because the client had to be holding the entire processed file. Now, Oyster Storage handles the file in batches, and sends the chunks in batches, avoiding the client-side out-of-memory crashes. Additionally, all the data for a chunk was stored in a MySQL row, including the “message” field which is 2187 characters long and was burdensome on Oyster’s queries. Now, the development team has moved this large field to Badger, which is a key-value database. The key for the message field is still stored in the MySQL row, but it is considerably smaller than the message field. The team plans to move further and further away from MySQL for data map chunks to further improve optimization. Another change made was that chunk data is now stored in the Badger database and other data about a chunk (status, address, etc.) is stored in MySQL. Previously, these two databases were somewhat coupled; the badger data could not be added to the broker node until the MySQL data was added. Now, these two processes are independent of one another, which means the client can go ahead and start sending chunk data as soon as the session has been created, instead of waiting for the broker node to build the data map (which can take a long time for large files).

We ask our community to note that the team is aware of concurrency limitations. If many users upload large files simultaneously, uploads will slow down considerably. We are working on addressing this issue.
In addition to this file size increase, the development team already has some ideas to push the file size limit to an even larger limit. Currently, Oyster Storage is capable of handling file uploads of ~100MB; however, there is a concurrency issue in the infrastructure, such that if many individuals try to upload files of this magnitude at the same time, it is likely that the upload will timeout before it is finished. Once the development team has implemented their ideas to deal with these concurrency issues, they will begin further testing of larger file uploads (~100Mb) to ensure that many users can upload files of this magnitude concurrently without a timeout occurring.

Some of the changes that the development team are exploring to reach larger upload sizes and higher overall throughput are:

implement round-robinning of uploads so that every user’s upload receives a little attention
continue to depend less on MySQL
switch to Hercules when it is released
use AWS Lambdas (a form of serverless functionality) to offload the Proof-of-Work burden elsewhere
add more broker nodes to Oyster’s network

The development team will be working tirelessly to improve the file size limit even further, and we will continually update our community regarding the progress made towards this goal.

Try out the new file size limit here: https://oysterstorage.com/

https://medium.com/oysterprotocol/oyster-storage-now-supports-25mb-uploads-bbf9f2deec5d?source=user_profile---------2------------------

 Wink
legendary
Activity: 2590
Merit: 1236
I wonder... Why did the team create SHL token they won't start the development of that part of the Oyster network for some time now. This way the price is loooow and the volume is practically non-existent  Sad

When websites install the one line Oyster code to monetize their website, they will earn both PRL and SHL.

I understand that. But the team said that they plan to develop PRL part of the project first and they won't develop SHL until they polish OysterStorage. They could have airdrop SHL tokens later when they start to develop it.
full member
Activity: 770
Merit: 156
I wonder... Why did the team create SHL token they won't start the development of that part of the Oyster network for some time now. This way the price is loooow and the volume is practically non-existent  Sad

When websites install the one line Oyster code to monetize their website, they will earn both PRL and SHL.
legendary
Activity: 2590
Merit: 1236
Wish many investor support your business when ICO will start.

There won't be an ICO. Oyster airdropped 100% of all SHL tokens to PRL holders some time ago. Oyster had unsuccessful ICO with PRL tokens last year in November but after that the community showed their support when the lead dev (Bruno Block) sold tokens to fund the project. It wasn't a dump because he explained everything to the community and all agreed it was the right thing to do.
legendary
Activity: 2590
Merit: 1236
I wonder... Why did the team create SHL token they won't start the development of that part of the Oyster network for some time now. This way the price is loooow and the volume is practically non-existent  Sad
full member
Activity: 770
Merit: 156
Support:

Quote
Dear KuCoin Users,
We are currently updating our system. We understand that some of you cannot see all of your assets or markets. Rest assured that your assets are safe. The system update should be complete within the next few hours. Thank you for your understanding and patience.

 Smiley
full member
Activity: 770
Merit: 156
full member
Activity: 770
Merit: 156
Development Update July 20th, 2018

Today’s update focuses on the work completed from July 15th through July 20th.

Much of the work completed this week focused around deploying upgrades to Oyster Storage’s backend. One of these upgrades included optimizing the backup script that currently creates emergency backups of the data maps presently stored on Oyster’s private tangle. When attempting these upgrades, the problems arose when trying to move recent changes into Oyster’s primary database. The Go framework the development team is using includes a migration tool called Fizz which developers can use to write files which will make changes to a database. Unfortunately, Fizz is not sophisticated enough to refrain from trying to create a table if it already exists or refrains from adding a column if it already exists, which caused errors anytime the development team attempted to pull changes from the master branch and then run migrations on the production machines. Thus, the development team needed to rewrite the migrations in SQL, so that the team could check to see if a table already exists and subsequently not create a duplicate of that table, as well as not adding a column to a table if it already exists. The team had to go through several iterations of SQL migrations before the file would work on both the developer broker databases and the Aurora DBs. Oyster’s development machines use MariaDB Docker images, while the production machines use AWS Aurora DBs with MySQL 5.7 server engines, and an early iteration of SQL migrations worked on the docker image databases but not on Aurora; thus, these SQL migrations had to be re-written. These changes led to SQL migration files that would work on both the production and development machines.

Furthermore, upon attempting to perform the migrations on the production databases, it became apparent that the size of the completed_data_maps table (which stores the uploaded user data submitted via Oyster Storage) would prevent the migrations from running successfully because of the size of the table. The table is very large because the developers did not yet delete old completed data maps and because of an indexing error which allowed duplicates into the table. The team is currently working to remove duplicates from this table, which will enable the team to run migrations, subsequently leading to the production brokers being brought back online.

For the time being, the development team will be bringing an additional set of production brokers online so that Oyster Storage can remain functional while these SQL migrations are completed on the master production infrastructure. Also, our community should expect to see a significant increase in file upload size by mid next week.

In addition to the work mentioned above, the team has continued working on integrating the oyster-streamable package into the web interface. This change will allow more memory efficient file upload and downloads, which means Oyster will be able to support larger file sizes. This is in the final stages of QA and testing, and so the team is hoping to deploy this change soon. Additionally, the team has been cleaning up the oyster-streamable package such that it can be released as a software development kit (SDK) for the public to use Oyster’s storage protocol. Finally, further work was completed on installing TypeScript support in both the web node and web interface, and several bugs and issues regarding the Badger DB deployment have been fixed.

https://medium.com/oysterprotocol/development-update-july-20th-2018-5af90805542e

https://mobile.twitter.com/OysterProtocol/status/1020408934258049025
full member
Activity: 770
Merit: 156
Great quotes about life...  Wink

Quote
Programming anything smaller than an operating system should never cost more than $500k. Go research some basic costs for software development. https://www.linkedin.com/pulse/how-much-does-cost-build-software-application-dr-john-flackett/

We are sitting on $3 million at even the current price: dualsig

If a comfort zone of 600% isn't enough then we are doing code development wrong. The remaining funds are to be used for useful things like exchange listings etc, but what you are discussing now is the viability of the protocol launch.

I don't want to hear anymore 'end of Oyster' stupidity. Was it the end of Bitcoin when Satoshi was mining 50 BTC per block which couldn't even buy half a pack of chewing gum? Were Satoshi and Finney freaking out about the economic untenability of mining?

Just like the infancy of the Bitcoin network, Oyster needs to gradually mature from centralized -> decentralized and unprofitable -> profitable. The difference between Bitcoin and Oyster is that Oyster (the company) has $3 million, after a brutal-bear-bashing, to throw at development and jumpstart node topology propagation.
full member
Activity: 770
Merit: 156
legendary
Activity: 1260
Merit: 1001
All the goals with progress level on the reviewed roadmap are awesome. People juts need to stop by to read main objectives of those features to understand what impact of them at successful completion and implementation of those inside the Oyster Pearl system. Each week is taking the project towards more advanced and improved features with more stability.
newbie
Activity: 35
Merit: 0
Hi.
The basic object that in the future it has guarantee the development of the plan!
full member
Activity: 770
Merit: 156
Development Update July 13th, 2018

Today’s development update will focus on work completed between July 8th - July 13th.

Overview of Content

Began planning process for API that will be a key aspect for strategic commercial use cases moving forward
Created proof of concept of fragile loop.
Made significant backend improvements to support larger file sizes.
Continued with frontend improvements to increase file size.
Oyster roadmap updated.
Korean and Chinese translations of PRL whitepaper added to site.
Oyster Storage API

This week, the development team worked on pulling shared functionality that could be useful in both the back-end and the front-end into a shared Golang library, which eventually can be made into a cross-compiled binary that could be distributed via Node Package Manager (NPM). Compiling this shared functionality is a step forward towards Oyster’s ultimate goal of building an application programming interface (API) that will significantly increase the appeal of Oyster Storage as a platform for commercial use. For example, consider Amazon’s S3 service. S3 is a standard cloud storage service that supplies a flexible API, allowing users and developers alike to interact with the Amazon S3 service without having to use the user interface provided by Amazon. If you used Dropbox during the company’s first eight years of production, you were using Amazon’s S3 service to store your files in the cloud. Although the interface users interacted with was completely Dropbox’s creation, the actual storage location of the users’ data was on cloud infrastructure provided by Amazon (Dropbox has since moved to their own cloud infrastructure platform).

The Amazon S3 API has become a sort of standard for cloud storage, and many other cloud providers build their own cloud storage services to be “S3-compliant.” However, despite the efficiency provided by Amazon S3, there is still a glaring issue associated with the service: S3 does not come pre-configured to be completely secure. In fact, some of the prominent data breaches that have occurred in the last few years have been the result of misconfigured Amazon S3 Buckets (An “S3 Bucket” can be thought of as the overarching file directory where your uploaded files are stored). To use Amazon’s S3 service securely, a user has to have some understanding of how to use Amazon’s cloud services and put in the necessary effort to secure their S3 buckets (to Amazon’s defense, they have offered several options to help in this respect via their “Config Rule” service, but that requires even more knowledge of the AWS platform!). Ultimately, Oyster will develop an API that is “S3-Compliant”, such that it provides a user-friendly interface to interact with our storage capabilities; however, unlike Amazon S3, Oyster will be able to offer secure and encrypted file storage without the need of user configuration. Users/Developers that use the Oyster Storage API interact with the very same backend used on Oyster Storage’s web interface. Thus, any file that is uploaded via the API is just as secure as the files uploaded via Oyster’s web interface.

There are three distinct benefits to building an API for Oyster Storage. First, an API will increase the appeal for Oyster to be used for commercial storage. Instead of using our web interface, companies can build wrappers around Oyster’s API to automate uploading large amounts of files, as well as managing Oyster Handles. Secondly, creating an S3-compliant API will allow companies that are currently using Amazon S3 (or some other storage service that uses an S3-compliant API) to very easily switch to Oyster. Creating the API in an S3-compliant way will give Oyster a better chance to get more commercial customers because these customers will not have to do nearly as much work to be able to use Oyster’s service. Thirdly, the Oyster API provides a whole new playground for the developer community as a whole. The Oyster community can use Oyster’s API to build whatever application they want that utilizes Oyster’s encrypted, anonymous storage capabilities. Ultimately, the Oyster Storage API will be a significant step forward in cementing Oyster’s wide-spread adoption as an alternative to current, centralized cloud storage solutions.

Oyster’s Fragile Loop Experiment



Besides the update regarding Oyster’s future storage API, the team has also been working on an experiment that showcases the ‘fragile loop’ that will be used to limit overburdening device CPUs when the Oyster web script is running. Along with the consent message, the concept of the fragile loop is the main design difference between aggressive crypto mining scripts and Oyster’s non-intrusive treasure hunting. The goal is to perform meaningful work without hogging the CPU or draining the battery of the user’s device.

Generally, this is achieved by only utilizing a fraction of the devices CPU, as opposed to going full throttle during the user’s entire stay on a website.

A CPU effectively only has the states of active and inactive. Energy saving modes aside, there is no running at, say, 30%. Instead, the CPU usage you see in the task manager means that the processor was active for a specific percentage of time over the last second or so. Thus, the Oyster team can achieve the goal of a non-intrusive web node by emulating this behavior in the browser. By splitting up the work the script has to perform into small chunks and then pausing the script between these chunks of work, the team gets an easily configurable piece of software that does not drain any batteries.

The team has prepared a demonstration of this concept for you to play around with here. Any input or criticism is welcome!

Miscellaneous Improvement

The development team has been doing some pre-deployment QA before Oyster rolls out the badger DB changes which will reduce bottlenecks on the backend and help increase the maximum file size we can support (note that the upcoming deployment of changes will not include the front-end changes which are also needed to support larger file sizes). The development team was able to upload a 200mb file using Oyster’s backend alone. Further QA is needed on the client-side of the Oyster Storage web interface before the current 5mb file size limit will be increased.

The Oyster development team has also implemented polling Oyster’s private tangle from the oyster-streamable library. Additionally, internal testing has begun with uploading files using streams on the client. Furthermore, the team has finished the rev2 client-side reference implementation, which is published here, along with writing the necessary unit tests for this implementation.

Finally, the development team has completed work on broker node/smart contract interactions with OysterPearl, as well as unit testing and QAing the associated transactions. Furthermore, the team has converted many of the React components and Redux actions in Oyster’s codebase from regular JavaScript to TypeScript.

Updated Roadmap and PRL Whitepaper Translations

The Oyster roadmap has been updated on the site, and the Korean and Chinese translations of the Oyster PRL whitepaper have been added as well.

https://medium.com/oysterprotocol/development-update-july-13th-2018-ac975ca973b6
newbie
Activity: 24
Merit: 0
Greetings team! The idea will be successful.
Impeccable website, a very detailed and profitable vision!
sr. member
Activity: 896
Merit: 251
During last week completed work is awesome reading everything in such details anyone can watch. Whereas goals for the coming week are very interesting where more probably enhancement and making the process lot more easier for adoption and seamless user experience when it comes to interact with Oyster's storage.
full member
Activity: 770
Merit: 156
Development Update

Today’s update will focus on the work completed between July 1st — July 6th, 2018.
This week, the development team finished up treasure index hashing. The treasure index of a sector is now chosen by hashing the average of the indexes selected by the alpha and beta node. The hash ensures that the treasure index is genuinely random within the sector. In addition to the hashing function, the index selection method now supports variable sector size and has guards to ensure the treasure index is valid. The team has also started working on production database migrations.

Additionally, the team continued to QA ETH/PRL transactions triggered by the broker node, made small fixes, and updated unit tests. The development team is now at a stage where they can test the web node ability to claim real PRL on the Ethereum mainnet successfully. For example, this transaction was the result of a web node submitting a claim to a broker node:

https://etherscan.io/tx/0x73dd4c3a2b7d1daa14c0c2a8bc92f9daffc98256e89bc0ee46d784d4aa30e8a0.

The team completed QA on the BadgerDB integration, as well as cleaning up some issues with the unit tests for the broker nodes. Furthermore, the team has been working diligently on fixing Code Climate issues, ultimately leading to a more standardized code base. By using Code Climate, the team can set coding standards for engineers to help keep the code base healthy while the team grows. The team has also been looking into tooling to help engineers configure and setup testing/staging environments efficiently and reliably so that the QA process is more automated and easier to perform.

In addition to the above improvements, the team is finishing up integrating streams for file uploads and will begin working on streams for file downloads. This is intended to allow users to upload or download large files in the browser while being memory efficient. Finally, the team worked on upgrading the web node from Webpack 3 to Webpack 4 and all dependent development libraries to the latest versions and worked on porting to and installing Typescript on the web node. Typescript will allow for strongly-typed JavaScript (i.e., explicitly declaring a variable type as a string, integer, etc.), which will improve Oyster’s coding standards and enhance security within the code base.

Development Goals for the Following Week

Below are a few of the improvements the Oyster development team will be working on in the following week

1. Continued focus on size/speed improvements
2. First design steps of a public api/client lib — the Oyster development team believes an API that other people can consume is needed for adoption. This will be a building block for how everyone, including Oyster’s internal services, upload data and also a building block for higher level wrappers. The team believes in the future the bulk of our storage will come from API users, not necessarily Oyster’s official storage interface.
3. Continue improving development tooling/process — cleaner code, more code coverage with testing, more end-to-end tests.

https://medium.com/oysterprotocol/july-6th-development-update-1eb1f85d7787
full member
Activity: 770
Merit: 156
Quote
PRL is currently ranked 5th overall in the Bibox community vote. Cast your vote now to help push $PRL into the top three and win a listing on the @bibox
exchange! Vote here: https://www.bibox365.com/cinformation?id=700   
Pages:
Jump to: