Datasets / daily_provider_quality

Daily provider quality

Daily Dealbot probe outcomes and onchain PDP proving per service provider. Probes are not client traffic. Check types with no observations for a provider and date are NULL.

Download Parquet Asset code Markdown

Shape

Table: daily_provider_quality
Rows ≈: 6,155 · 39 columns · local snapshot
Grain: Key not declared
Observed, UTC: 2025-11-04 → 2026-10-28 first and last day; days may be missing
Raw sources: raw.filecoin_api_data_sets, raw.filecoin_api_rails, raw.probelab_pull_checks, raw.probelab_retrieval_checks, raw.probelab_sampled_retrieval_checks, raw.probelab_storage_checks, raw.riba_eth_logs, raw.riba_eth_logs_decoded · raw assets

Query

DuckDB, reads the published file

select *
from 'https://data.filecoindataportal.xyz/daily_provider_quality.parquet'
limit 10;

The published file is rebuilt daily; counts and sample here come from the local snapshot built 2026-10-29.

Columns

daily_provider_quality column definitions
Name	Type	Description
date	DATE	UTC date.
provider_id	UBIGINT	Service provider registry identifier; null when a Dealbot check did not record it.
storage_checks	BIGINT	Data storage checks that finished as created or failed.
storage_successes	BIGINT	Data storage checks that created the deal.
storage_median_upload_ms	DOUBLE	Median upload time to the provider of created deals, in milliseconds.
storage_median_confirmation_ms	DOUBLE	Median time from piece added to piece confirmed onchain of created deals, in milliseconds.
retrieval_checks	BIGINT	IPFS retrievals of Dealbot's own deals.
retrieval_successes	BIGINT	Successful IPFS retrievals of Dealbot's own deals.
retrieval_median_ttfb_ms	DOUBLE	Median time to first byte of successful retrievals, in milliseconds.
retrieval_p95_ttfb_ms	DOUBLE	95th percentile time to first byte of successful retrievals, in milliseconds.
retrieval_median_bytes_per_second	DOUBLE	Median download rate of successful retrievals, in bytes per second.
pull_checks	BIGINT	Pull-to-park checks.
pull_successes	BIGINT	Successful pull-to-park checks.
pull_median_completion_ms	DOUBLE	Median time from pull request to completion of successful pulls, in milliseconds.
pull_p95_completion_ms	DOUBLE	95th percentile time from pull request to completion of successful pulls, in milliseconds.
pull_median_bytes_per_second	DOUBLE	Median piece bytes over total completion time of successful pulls, in bytes per second.
sampled_retrieval_checks	BIGINT	Attempted piece retrievals of sampled non-Dealbot pieces.
sampled_retrieval_successes	BIGINT	Sampled piece retrievals returning HTTP 2xx with bytes matching the piece CID.
sampled_retrieval_skipped	BIGINT	Sampled retrieval checks skipped because no candidate piece was found.
sampled_retrieval_median_ttfb_ms	DOUBLE	Median time to first byte of successful sampled retrievals, in milliseconds.
sampled_retrieval_p95_ttfb_ms	DOUBLE	95th percentile time to first byte of successful sampled retrievals, in milliseconds.
sampled_retrieval_median_bytes_per_second	DOUBLE	Median download rate of successful sampled retrievals, in bytes per second.
sampled_car_checks	BIGINT	Sampled-piece checks that attempted CAR parsing. Skips are excluded; malformed CAR data is not a provider-fault measure.
sampled_car_successes	BIGINT	Sampled-piece checks whose fetched bytes parsed as a CAR.
sampled_ipni_checks	BIGINT	Sampled-piece checks that attempted IPNI verification. Skips are excluded.
sampled_ipni_successes	BIGINT	Sampled IPNI lookups resolving every CID to the provider.
sampled_ipni_median_verify_ms	DOUBLE	Median IPNI verification time of successful sampled IPNI lookups, in milliseconds.
sampled_block_fetch_checks	BIGINT	Sampled-piece checks that attempted gateway block fetches. Skips are excluded.
sampled_block_fetch_successes	BIGINT	Sampled-piece checks where every fetched block matched its CID.
proving_proofs	HUGEINT	Accepted onchain PDP proofs of FWSS data sets. Null when onchain logs through the date are not fully decoded or the provider has no proving data set.
proving_periods_closed	HUGEINT	Proving periods closed onchain on the date, faulted or proven. Null under the same conditions as proving_proofs.
proving_periods_faulted	HUGEINT	Proving periods recorded as faulted when closed on the date; missed periods remain uncounted until closure. Null under the same conditions as proving_proofs.
storage_success_rate	DOUBLE	storage_successes over storage_checks.
retrieval_success_rate	DOUBLE	retrieval_successes over retrieval_checks.
pull_success_rate	DOUBLE	pull_successes over pull_checks.
sampled_retrieval_success_rate	DOUBLE	sampled_retrieval_successes over sampled_retrieval_checks.
sampled_ipni_success_rate	DOUBLE	sampled_ipni_successes over sampled_ipni_checks.
sampled_block_fetch_success_rate	DOUBLE	sampled_block_fetch_successes over sampled_block_fetch_checks.
proving_fault_rate	DOUBLE	proving_periods_faulted over proving_periods_closed.

Sample

5 rows from the local snapshot, latest first, exactly as stored.

daily_provider_quality sample rows
date	provider_id	storage_checks	storage_successes	storage_median_upload_ms	storage_median_confirmation_ms	retrieval_checks	retrieval_successes	retrieval_median_ttfb_ms	retrieval_p95_ttfb_ms	retrieval_median_bytes_per_second	pull_checks	pull_successes	pull_median_completion_ms	pull_p95_completion_ms	pull_median_bytes_per_second	sampled_retrieval_checks	sampled_retrieval_successes	sampled_retrieval_skipped	sampled_retrieval_median_ttfb_ms	sampled_retrieval_p95_ttfb_ms	sampled_retrieval_median_bytes_per_second	sampled_car_checks	sampled_car_successes	sampled_ipni_checks	sampled_ipni_successes	sampled_ipni_median_verify_ms	sampled_block_fetch_checks	sampled_block_fetch_successes	proving_proofs	proving_periods_closed	proving_periods_faulted	storage_success_rate	retrieval_success_rate	pull_success_rate	sampled_retrieval_success_rate	sampled_ipni_success_rate	sampled_block_fetch_success_rate	proving_fault_rate
2026-10-28	1	96	88	10906	62443.5	142	142	100.5	198.89999999999998	5054971.148778652	24	24	6738.5	10100.949999999999	1556244	48	48	0	174	391.24999999999994	4667615.675929837	48	48	48	47	1988	48	48	175	176	0	0.9166666666666666	1	1	1	0.9791666666666666	1	0
2026-10-28	2	96	96	9930	56847.5	129	129	662	1400.8	2309750.881057269	24	20	10741.5	224160.40000000002	976440	48	48	0	637	994.6499999999999	3775987.039229097	40	40	40	40	1613.5	40	39	20	20	0	1	1	0.8333333333333334	1	1	0.975	0
2026-10-28	3	null	null	null	null	null	null	null	null	null	null	null	null	null	null	null	null	null	null	null	null	null	null	null	null	null	null	null	0	0	0	null	null	null	null	null	null	null
2026-10-28	5	96	96	6871.5	56149	134	134	505	600.5	1559644.8042868292	24	24	7947	10043.05	1319925.5	48	48	0	572.5	621.25	1792656.353880188	43	43	43	42	2070	43	43	136	140	4	1	1	1	1	0.9767441860465116	1	0.02857142857142857
2026-10-28	6	null	null	null	null	null	null	null	null	null	null	null	null	null	null	null	null	null	null	null	null	null	null	null	null	null	null	null	0	0	0	null	null	null	null	null	null	null