As the blockchain ecosystem grows, so does the amount of data it holds and generates. As such, the challenge of storing, querying, and retrieving data becomes exponentially more complex across a more fragmented, multi-layered landscape. However, necessity is the mother of innovation, and the web3 community is rising to the challenges. 

Here are four key trends currently shaping the web3 data segment and some thoughts on how they may play out.

1. Data availability is the next frontier of solving Ethereum’s scalability trilemma

As the Ethereum community has solidified its move to a rollup-centric ecosystem, data availability (DA) has become front and center as the next challenge in solving the scalability trilemma. Validators in a network rely on the availability of blockchain data to validate transactions, but block space on Ethereum comes at a premium compared to many other platforms. The planned Ethereum upgrade to proto-danksharding will go some way to addressing the issue, but the implementation has been delayed until later in 2024. 

Modular blockchains such as Celestia, Avail, and Eigenlayer DA have emerged to address the problem, providing a platform for rollups to publish transactions at a lower cost but with Ethereum-like security guarantees. 

However, having data distributed across less-proven blockchains may end up introducing risks. If a data layer fails to deliver accurate or timely data, a dapp’s entire infrastructure could collapse since all of the dapp data will be held in that layer. Therefore, platforms will be gearing up to demonstrate that their offering is faster, more resilient, and verifiably able to return valid data. 

2. Data APIs become a rival to oracles

Bringing off-chain data on-chain used to be one of the key challenges facing blockchain developers. However, the emergence of decentralized oracle networks like Chainlink offered a solution by creating a means whereby off-chain data could be verified trustless by a node network before bringing it on-chain. 

Decentralized APIs are now proving an entire third-party oracle is no longer necessary to get off-chain data into a blockchain environment. A decentralized API service such as API3 or Airstack provides access to on- and off-chain data such as transactions, NFT metadata, social interactions, and more. Each data provider signs submissions on-chain as a verification of its accuracy and authenticity. 

These solutions offer distinct advantages over oracles, such as transparency of data sources, lower costs, and lower latency. Therefore, we can expect to see the rivalry between decentralized APIs and legacy oracle providers heating up over the coming months as both compete for the same audience of dapp developers. 

3. Indexers cut through the complexity

Over recent years, the blockchain data landscape has become increasingly complex as the space has grown. The expansion of dapp development and user activity to platforms such as Solana, Polkadot, and Cosmos was followed by an increasing prevalence of Layer-2 and data availability layers to support Ethereum. While such growth is unarguably good news for the sector overall, it has led to more fragmentation in blockchain data, making it difficult for developers who rely on data availability from multiple ecosystems.

To compound the issue further, as web3 finds more and more use cases, it’s increasingly common that not all data is even stored on-chain. For instance, many interactions on decentralized social networks like Farcaster don’t occur on-chain, but the data is stored in off-chain storage hubs. 

With increasingly vast amounts of data being generated all the time, stored in multiple locations and layers, and often in a non-standard format, there is a growing need for reliable indexing services. The Graph was one of the early entrants to the market, having been live for three years, and has undergone several steps to move towards a more decentralized model. However, since an API (or “subgraph”) only works for a single chain, there are limitations for data users seeking to harness data from multiple chains and sources. 

However, the project paved the way for rivals—namely Subsquid—to enter the market with a decentralized, multichain data lake and query engine, empowering developers and analysts with seamless access to blockchain data from across the web3 sphere using a range of programming languages.

Indexing is, therefore, another critical development area in the web3 data sphere, where projects will differentiate themselves on factors such as multi-chain support, integration with other decentralized parts of the stack, speed, accessibility, and cost. 

4. Inscriptions launches deliver diminishing returns

The launch of Bitcoin Ordinals in February 2023 created one of the biggest commotions in an otherwise relatively muted year. Ordinals enable data to be “inscribed” into units of BTC, enabling users to create their own fungible and non-fungible assets. Within less than 12 months, over 55 million inscriptions were minted on the Bitcoin blockchain, making a significant windfall from miners and prompting many other blockchain communities, including Ethereum, Avalanche, and NEAR, to launch inscriptions on their networks. 

However, it seems likely that the future availability of inscription functionality across smaller networks will follow the law of diminishing returns. On newer platforms that don’t suffer Ethereum’s legacy scalability issuers, there isn’t necessarily any benefit to inscribing as opposed to using established token standards, meaning launches have little to sell themselves on beyond the hype. 

For Bitcoin, Ordinals are another chapter in the long-running Block Size Wars, which are unlikely to resolve any time soon. Nevertheless, given ongoing enthusiasm for increased utility on the granddaddy of blockchains, inscription-based traffic looks set to be a key miner revenue driver for the foreseeable future. 

While developments in Ethereum’s scalability and Bitcoin’s utility have been positive developments for the web3 ecosystem, they’ve nevertheless brought fresh challenges for data consumers. However, far from holding back further progress, these challenges are giving rise to a new wave of innovation and creating healthy competition among pioneering projects to establish market share in their respective segments. In turn, dapp developers and users will benefit from the fruits of this competition, along with the choice it offers. 

Marcel Fohrmann is the co-founder and CFO of Subsquid, a peer-to-peer network. He has over seven years of experience in crypto and has cultivated a robust network within the venture capital and angel investor community. Before that, he attended the Stanford University Graduate School of Business. Subsquid is a distributed query engine and data lake around which the Subsquid ecosystem is built. It offers developers permissionless, cost-efficient access to on-chain data from over 100 chains and is integrated into a large ecosystem of web2- and web3-native developer tools.