Data Sources

Most of the data used through the portal comes from the Filecoin chain. There is also some off-chain data like Datacap Applications or Storage Provider’s Reputation scores that are collected from other places.

Deals

Deals data is available on chain and can be obtained in different ways:

Doing a StateMarketDeals JSON-RPC call and parsing the returned JSON. If you don’t have a node running, you can use Glif nodes StateMarketDeals periodic dump on S3 (direct link).
Using an oracle like fil-naive-marketwatch.
Reconstructing the deals state from Lily tables.

Clients

Clients can be derived from the deals dataset and expanded with the following sources:

Datacap. From Datacap Stats API calling https://api.datacapstats.io/api/getVerifiedClients. You get a JSON of verified clients in the FIL+ program that contains client names, Datacap application data and other self-reported data. Alternatively, this data can be obtained by parsing the relevant GitHub repositories issues and comments.

Storage Providers

Storage Providers be derived from the deals dataset. More information about proviers can be collected in the following sources:

Location using the different provider.quest endpoints/datasets.

Retrieval Data

Retrieval data is available from the Spark API.

Reputation Data

Reputation is both obtained from FilRep (methodology) and augmented with custom metrics around deals. For example, what is the average replication of a deal for the SP?

Energy Data

Energy data is available from Filecoin Green (Model API and Green Scores API)

FVM

Filecoin Virtual Machine data is trickier to get. Some sources:

Directly from the FVM dashboard.
Some metrics are available in the Spacescope API

Messages

A few teams across the ecosystem are indexing Filecoin Messages. The most comprehensive source are Beryx and FilInfo.

Data Indexers

Besides the data sources mentioned above, there are a few data indexers that provide data in a more structured way.

Starboard - FVM
Dev Storage
Beryx
Spacegap
Filecoin Green
Filecoin CID Checker
File.app
Gliff Explorer
DMOB Messages Database powering FilInfo
Filrep
FilecoinGreen
FilFox and API
FilScan and API
Filutils decoding message params (example).
Blockscout and API for FEVM.

JSON-RPC Endpoints

Nodes usually implement all the JSON-RPC methods needed to get the data.

Glif - https://api.node.glif.io
Zondax - https://api.zondax.ch/fil/node/mainnet/rpc/v1
Laconic - https://fil-mainnet-1.rpc.laconic.com/rpc/v1
Provider Quest - https://lotus.miner.report/mainnet_api/0/node/rpc/v0
More at filecoin.io docs!
More at Chainlist!