Subsquid’s mission is to bring scalable data APIs to Web 3.0 by transforming a traditional SaaS model into a DAO with open participation. The Subsquid DAO replaces the standard approach of redundant monolithic network nodes with heterogeneous workers. Workers are incentivised and are subject to on-chain accountability and democracy.
1 Motivation
Decentralised applications require a complex and versatile backend in order to compete with traditional Web 2.0 services and deliver comparable user experience. Popular websites like uniswap.org already see 10M+ monthly visits, totalling to at least 100M+ backend hits, and as such they require a highly scalable infrastructure to serve the actual on-chain data and analytics. Uniswap also offers analytics covering the prices, trading volumes and other metrics, representing another challenge for Web 3.0 developers as the increasing logical complexity of consumer-facing decentralised apps increases. Such data is not readily available on-chain, and so a dedicated indexing/aggregating middleware like Subsquid is needed.
2 Subsquid DAO
Subsquid envisions a distributed infrastructure of loosely coupled services run by bonded workers with social reputation and fully auditable logs. The first iteration is similar to a traditional SaaS service and is fully controlled by Subsquid.io. As the community around the project grows, the key governance and technical decisions will be handed over to the Subsquid DAO with on-chain democracy, with a free market approach envisioned for each service offered by the protocol and no entry barriers to participate in the DAO.
The final goal is to fulfill several the the following high-level guiding principles:
• There is a free market for each service provided by the Subsquid DAO and there are no entry barriers to participate in the DAO
• All service providers are responsible for fulfilling the agreed SLAs, and are subject to bond slashing in case of violating the commitments (subject to arbitration)
• The Subsquid DAO adds value by providing a marketplace for different roles on the platform. SQD token holders control the development, arbitration and execution of the platform via the governance process
3 Decentralisation trade-offs
The trend we see in Web 3.0 departs from the crypto-anarchist fully anonymous and peer-to-peer design and is transitioning into hybrid forms. Supposedly decentralised entities are often now controlled by centralised bodies backed by traditional investment firms but have website backends powered by smart contracts, making them censorship-resistant and decentralised to the extent guaranteed by the blockchain.
On the left end of the spectrum we have ‘hard’ data with a high replication rate, such as any data stored on-chain. Such data is guaranteed to be immutable and available from any node as dictated by the consensus rules. At the right end of the spectrum one can place “soft” data, which may be available only at a single node and is hard to replicate. These can be real time data streams or an output of a non-deterministic algorithm (e.g. content discovery engines).
4 Fat Indexers: serving the long tail of APIs
Hydra introduces a dedicated indexing service that provides access to raw blockchain data. A traditional monolithic approach to building a decentralised query node network assumes that the query nodes are run by the operators as a black box with little to no customisation. Each network node provides an end-to-end pipeline which ingests the blockchain data from a blockchain node, performs the necessary data transformations and also serves the data through a GraphQL endpoint. Scaling such a network is problematic, as such nodes require powerful hardware to host an index at the same time as keeping up with the API traffic.
Left: each node in the network is an end-to-end middleware from data ingestion to a GraphQL endpoint. Right: Multiple lightweight Hydra processors share a smaller pool of Hydra indexers.
5 Query-node as an oracle
Query node APIs typically benefit from accessing external data, such as a price feed. The traditional way that data gets from the oracle to the blockchain is through a pluggable pallet (e.g. ChainLink pallet for Substrate). The external oracle data is then available via event data picked up by the Hydra Indexers and Processors:
Oracle data can be exposed as runtime event data and captured by the Hydra Processor.
This approach is limited however due to the data feeds provided by the oracle services - even aggregating historical oracle data can be problematic.
A dedicated Subsquid Pallet will enable Hydra-based Query Node data on-chain. This gives a lot more flexibility for the end users, as the Query Node data can be enriched with aggregations and any other on-chain data while preserving the integrity provided by the original oracle feeds.
Website: http://subsquid.io/
Discord Server: https://discord.gg/mggygay5XV
Telegram Chat Group: https://t.me/subsquid
Telegram Announcement Channel: https://t.me/subsquidannouncements
Twitter: https://www.twitter.com/subsquid
Medium Blog: https://subsquid.medium.com/
Github: https://github.com/Subsquid
Subsocial: https://app.subsocial.network/@subsquid
Linkedin: https://www.linkedin.com/company/subsquid
Facebook: https://www.facebook.com/subsquidHQ/
Instagram: https://instagram.com/subsquid.io
Reddit: https://www.reddit.com/r/subsquid/
Youtube: https://www.youtube.com/channel/UCmD8pRYcNTN_p42qNUWK72g