Data sourcing
Instead of distributing pre-calculated data feeds, DIA covers the whole data journey from individual trade information collection, and data computation to the last mile of the feed delivery. To create, for instance, a ETH/USD price feed oracle, DIA sources data points from dozens of ETH pairs (ETH/USD, ETH/USDT, ETH/BTC, etc) trades from across a wide range of exchanges.
Granular trade data sourcing
By leveraging a network of WebSockets and decentralised node providers, DIA retrieves token trade data from 80+ exchanges. This, in turn, represents an enormous data availability (≈15bn trades/day) to, later on, enable DIA to capture the highest data precision and build flexible and customizable price feeds.
Trades for every asset pair get recorded and collected into blocks of 120 seconds into the DIA Platform. For each block, a range of filters are applied. These are functions to get a single price point from the collection of trades in a block. For example, the Moving Average filter returns the volume-weighted moving average of the trades block.
Filters are calculated for each asset pair on each exchange individually, as well as for each asset on all exchanges combined. This combined filter result represents the result closest to the true "whole market" that can be determined by this system. Learn more about token filters here and NFT filters here.
Type of data sources and their aggregation
DIA retrieves trading data from as close to the data source as possible in order to ensure the highest precision possible. The data retrieval is processed through a series of asset collectors:
For centralised exchanges such as Coinbase, Kraken or Binance, DIA scrapers collect trades directly from the exchange’s databases. The data collection happens in retrieval periods from 1 to 7 seconds and varies from exchange to exchange. All trading data is retrieved through Rest APIs or WebSocket APIs, developed and maintained by the respective trading platform. More on CEX trade data collection.
For decentralised exchanges, DIA collects data from decentralized exchanges on various blockchains by subscribing to swap events in the corresponding liquidity pools – LPs. In contrast to centralized exchanges, in decentralized exchanges, it is possible to retrieve trading data directly from the respective blockchain by subscribing to the corresponding pool events. Examples of sources are Uniswap, curve.finance or PancakeSwap. More on DEX trade data collection.
For NFT marketplaces, DIA collects live trading data on the blockchain for all NFT transactions happening on the integrated marketplaces and blockchains. Therefore, the data is sourced directly from the origin and allows to capture precise market data. Besides the collection of attributes through OpenSea API and other sources, DIA tracks on-chain NFT transactions from the largest NFT marketplaces to fetch data such as trades, bids and offers. Examples of sources are Blur, X2Y2 and TofuNFT. More on NFT trade data collection.
For cross-chain bridges, we collect the swaps, enabling us to port prices and map assets across blockchains. Cross-Chain bridges allow for swapping a crypto asset on a given blockchain to the corresponding asset on a different blockchain. For instance, swap USDC on Ethereum to USDC on Fantom. Cross-Chain bridges can be seen as ranging somewhere between centralized and decentralized: Parts of the complete bridging transactions can be seen publicly on the respective blockchain. However, by its very nature, the bridging itself is done by a centralized entity. More on Cross-Chain Bridge data.
For 3rd party data aggregation, DIA collects quotations from centralised providers such as Coingecko or Coinmarketcap or traditional finance asset quotation providers such as daily exchange rates from European Central Bank (ECB) or Yahoo Finance. More on 3rd party data aggregation.
On oracle feed manipulation and risk
One of the biggest risks for DeFi and Web3 applications is not having enough sources for its oracle and using a feed that relies on one single market. These single-source oracles, as explained in the beginning, are relatively easily exposed to price manipulations and man-in-the-middle attacks.
DIA is designed to avoid these scenarios by leveraging multiple markets as sources, including CEXs and DEXs and enabling the customisation of methodologies and sources to cater best for the needs of specific use cases. So far in DeFi’s (short) history, a failure in the oracle node network has never been the reason for a loss of funds, but attacks on the markets that provide the price data to the sources. Take for instance the example of the Cream Finance attack in October 2021.
DIA prioritises the transparency aspect of its data and oracle services and enables any dApp developer and user to know precisely how the data feeding the application is built, what data sources are been used, and what methodologies are been applied.
Data source explorer
See real-time data of all sources DIA is integrated with:
Last updated
