Dash Core Source Documentation (0.16.0.1)

Find detailed information regarding the Dash Core source code.

We want to be able to estimate feerates that are needed on tx's to be included in a certain number of blocks. More...

#include <fees.h>

+ Collaboration diagram for CBlockPolicyEstimator:

Classes

struct  TxStatsInfo
 

Public Member Functions

 CBlockPolicyEstimator ()
 Create new BlockPolicyEstimator and initialize stats tracking classes with default values. More...
 
 ~CBlockPolicyEstimator ()
 
void processBlock (unsigned int nBlockHeight, std::vector< const CTxMemPoolEntry *> &entries)
 Process all the transactions that have been included in a block. More...
 
void processTransaction (const CTxMemPoolEntry &entry, bool validFeeEstimate)
 Process a transaction accepted to the mempool. More...
 
bool removeTx (uint256 hash, bool inBlock)
 Remove a transaction from the mempool tracking stats. More...
 
CFeeRate estimateFee (int confTarget) const
 DEPRECATED. More...
 
CFeeRate estimateSmartFee (int confTarget, FeeCalculation *feeCalc, bool conservative) const
 Estimate feerate needed to get be included in a block within confTarget blocks. More...
 
CFeeRate estimateRawFee (int confTarget, double successThreshold, FeeEstimateHorizon horizon, EstimationResult *result=nullptr) const
 Return a specific fee estimate calculation with a given success threshold and time horizon, and optionally return detailed data about calculation. More...
 
bool Write (CAutoFile &fileout) const
 Write estimation data to a file. More...
 
bool Read (CAutoFile &filein)
 Read estimation data from a file. More...
 
void FlushUnconfirmed (CTxMemPool &pool)
 Empty mempool transactions on shutdown to record failure to confirm for txs still in mempool. More...
 
unsigned int HighestTargetTracked (FeeEstimateHorizon horizon) const
 Calculation of highest target that estimates are tracked for. More...
 

Private Member Functions

bool processBlockTx (unsigned int nBlockHeight, const CTxMemPoolEntry *entry)
 Process a transaction confirmed in a block. More...
 
double estimateCombinedFee (unsigned int confTarget, double successThreshold, bool checkShorterHorizon, EstimationResult *result) const
 Helper for estimateSmartFee. More...
 
double estimateConservativeFee (unsigned int doubleTarget, EstimationResult *result) const
 Helper for estimateSmartFee. More...
 
unsigned int BlockSpan () const
 Number of blocks of data recorded while fee estimates have been running. More...
 
unsigned int HistoricalBlockSpan () const
 Number of blocks of recorded fee estimate data represented in saved data file. More...
 
unsigned int MaxUsableEstimate () const
 Calculation of highest target that reasonable estimate can be provided for. More...
 

Private Attributes

unsigned int nBestSeenHeight
 
unsigned int firstRecordedHeight
 
unsigned int historicalFirst
 
unsigned int historicalBest
 
std::map< uint256, TxStatsInfomapMemPoolTxs
 
std::unique_ptr< TxConfirmStatsfeeStats
 Classes to track historical data on transaction confirmations. More...
 
std::unique_ptr< TxConfirmStatsshortStats
 
std::unique_ptr< TxConfirmStatslongStats
 
unsigned int trackedTxs
 
unsigned int untrackedTxs
 
std::vector< double > buckets
 
std::map< double, unsigned int > bucketMap
 
CCriticalSection cs_feeEstimator
 

Static Private Attributes

static constexpr unsigned int SHORT_BLOCK_PERIODS = 12
 Track confirm delays up to 12 blocks for short horizon. More...
 
static constexpr unsigned int SHORT_SCALE = 1
 
static constexpr unsigned int MED_BLOCK_PERIODS = 24
 Track confirm delays up to 48 blocks for medium horizon. More...
 
static constexpr unsigned int MED_SCALE = 2
 
static constexpr unsigned int LONG_BLOCK_PERIODS = 42
 Track confirm delays up to 1008 blocks for long horizon. More...
 
static constexpr unsigned int LONG_SCALE = 24
 
static const unsigned int OLDEST_ESTIMATE_HISTORY = 6 * 1008
 Historical estimates that are older than this aren't valid. More...
 
static constexpr double SHORT_DECAY = .962
 Decay of .962 is a half-life of 18 blocks or about 45 minutes. More...
 
static constexpr double MED_DECAY = .9952
 Decay of .998 is a half-life of 144 blocks or about 6 hours. More...
 
static constexpr double LONG_DECAY = .99931
 Decay of .9995 is a half-life of 1008 blocks or about 2 days. More...
 
static constexpr double HALF_SUCCESS_PCT = .6
 Require greater than 60% of X feerate transactions to be confirmed within Y/2 blocks. More...
 
static constexpr double SUCCESS_PCT = .85
 Require greater than 85% of X feerate transactions to be confirmed within Y blocks. More...
 
static constexpr double DOUBLE_SUCCESS_PCT = .95
 Require greater than 95% of X feerate transactions to be confirmed within 2 * Y blocks. More...
 
static constexpr double SUFFICIENT_FEETXS = 0.1
 Require an avg of 0.1 tx in the combined feerate bucket per block to have stat significance. More...
 
static constexpr double SUFFICIENT_TXS_SHORT = 0.5
 Require an avg of 0.5 tx when using short decay since there are fewer blocks considered. More...
 
static constexpr double MIN_BUCKET_FEERATE = 1000
 Minimum and Maximum values for tracking feerates The MIN_BUCKET_FEERATE should just be set to the lowest reasonable feerate we might ever want to track. More...
 
static constexpr double MAX_BUCKET_FEERATE = 1e7
 
static constexpr double FEE_SPACING = 1.05
 Spacing of FeeRate buckets We have to lump transactions into buckets based on feerate, but we want to be able to give accurate estimates over a large range of potential feerates Therefore it makes sense to exponentially space the buckets. More...
 

Detailed Description

We want to be able to estimate feerates that are needed on tx's to be included in a certain number of blocks.

The BlockPolicyEstimator is used for estimating the feerate needed for a transaction to be included in a block within a certain number of blocks.

Every time a block is added to the best chain, this class records stats on the transactions included in that block

At a high level the algorithm works by grouping transactions into buckets based on having similar feerates and then tracking how long it takes transactions in the various buckets to be mined. It operates under the assumption that in general transactions of higher feerate will be included in blocks before transactions of lower feerate. So for example if you wanted to know what feerate you should put on a transaction to be included in a block within the next 5 blocks, you would start by looking at the bucket with the highest feerate transactions and verifying that a sufficiently high percentage of them were confirmed within 5 blocks and then you would look at the next highest feerate bucket, and so on, stopping at the last bucket to pass the test. The average feerate of transactions in this bucket will give you an indication of the lowest feerate you can put on a transaction and still have a sufficiently high chance of being confirmed within your desired 5 blocks.

Here is a brief description of the implementation: When a transaction enters the mempool, we track the height of the block chain at entry. All further calculations are conducted only on this set of "seen" transactions. Whenever a block comes in, we count the number of transactions in each bucket and the total amount of feerate paid in each bucket. Then we calculate how many blocks Y it took each transaction to be mined. We convert from a number of blocks to a number of periods Y' each encompassing "scale" blocks. This is is tracked in 3 different data sets each up to a maximum number of periods. Within each data set we have an array of counters in each feerate bucket and we increment all the counters from Y' up to max periods representing that a tx was successfully confirmed in less than or equal to that many periods. We want to save a history of this information, so at any time we have a counter of the total number of transactions that happened in a given feerate bucket and the total number that were confirmed in each of the periods or less for any bucket. We save this history by keeping an exponentially decaying moving average of each one of these stats. This is done for a different decay in each of the 3 data sets to keep relevant data from different time horizons. Furthermore we also keep track of the number unmined (in mempool or left mempool without being included in a block) transactions in each bucket and for how many blocks they have been outstanding and use both of these numbers to increase the number of transactions we've seen in that feerate bucket when calculating an estimate for any number of confirmations below the number of blocks they've been outstanding.

Definition at line 138 of file fees.h.

Constructor & Destructor Documentation

◆ CBlockPolicyEstimator()

CBlockPolicyEstimator::CBlockPolicyEstimator ( )

Create new BlockPolicyEstimator and initialize stats tracking classes with default values.

Definition at line 527 of file fees.cpp.

References bucketMap, buckets, FEE_SPACING, feeStats, INF_FEERATE, LONG_BLOCK_PERIODS, LONG_DECAY, LONG_SCALE, longStats, MAX_BUCKET_FEERATE, MED_BLOCK_PERIODS, MED_DECAY, MED_SCALE, MIN_BUCKET_FEERATE, SHORT_BLOCK_PERIODS, SHORT_DECAY, SHORT_SCALE, and shortStats.

◆ ~CBlockPolicyEstimator()

CBlockPolicyEstimator::~CBlockPolicyEstimator ( )

Definition at line 545 of file fees.cpp.

Member Function Documentation

◆ BlockSpan()

unsigned int CBlockPolicyEstimator::BlockSpan ( ) const
private

Number of blocks of data recorded while fee estimates have been running.

Definition at line 728 of file fees.cpp.

References firstRecordedHeight, and nBestSeenHeight.

Referenced by MaxUsableEstimate(), processBlock(), and Write().

◆ estimateCombinedFee()

double CBlockPolicyEstimator::estimateCombinedFee ( unsigned int  confTarget,
double  successThreshold,
bool  checkShorterHorizon,
EstimationResult result 
) const
private

Helper for estimateSmartFee.

Return a fee estimate at the required successThreshold from the shortest time horizon which tracks confirmations up to the desired target.

If checkShorterHorizon is requested, also allow short time horizon estimates for a lower target to reduce the given answer

Definition at line 756 of file fees.cpp.

References feeStats, longStats, nBestSeenHeight, shortStats, SUFFICIENT_FEETXS, and SUFFICIENT_TXS_SHORT.

Referenced by estimateSmartFee().

◆ estimateConservativeFee()

double CBlockPolicyEstimator::estimateConservativeFee ( unsigned int  doubleTarget,
EstimationResult result 
) const
private

Helper for estimateSmartFee.

Ensure that for a conservative estimate, the DOUBLE_SUCCESS_PCT is also met at 2 * target for any longer time horizons.

Definition at line 795 of file fees.cpp.

References DOUBLE_SUCCESS_PCT, feeStats, longStats, nBestSeenHeight, and SUFFICIENT_FEETXS.

Referenced by estimateSmartFee().

◆ estimateFee()

CFeeRate CBlockPolicyEstimator::estimateFee ( int  confTarget) const

DEPRECATED.

Return a feerate estimate

Definition at line 663 of file fees.cpp.

References DOUBLE_SUCCESS_PCT, estimateRawFee(), and MED_HALFLIFE.

Referenced by estimatefee().

◆ estimateRawFee()

CFeeRate CBlockPolicyEstimator::estimateRawFee ( int  confTarget,
double  successThreshold,
FeeEstimateHorizon  horizon,
EstimationResult result = nullptr 
) const

Return a specific fee estimate calculation with a given success threshold and time horizon, and optionally return detailed data about calculation.

Definition at line 672 of file fees.cpp.

References cs_feeEstimator, TxConfirmStats::EstimateMedianVal(), feeStats, TxConfirmStats::GetMaxConfirms(), LOCK, LONG_HALFLIFE, longStats, MED_HALFLIFE, nBestSeenHeight, SHORT_HALFLIFE, shortStats, SUFFICIENT_FEETXS, and SUFFICIENT_TXS_SHORT.

Referenced by estimateFee(), and estimaterawfee().

◆ estimateSmartFee()

CFeeRate CBlockPolicyEstimator::estimateSmartFee ( int  confTarget,
FeeCalculation feeCalc,
bool  conservative 
) const

Estimate feerate needed to get be included in a block within confTarget blocks.

estimateSmartFee returns the max of the feerates calculated with a 60% threshold required at target / 2, an 85% threshold required at target and a 95% threshold required at 2 * target.

If no answer can be given at confTarget, return an estimate at the closest target where one can be given. 'conservative' estimates are valid over longer time horizons also.

Each calculation is performed at the shortest time horizon which tracks the required target. Conservative estimates, however, required the 95% threshold at 2 * target be met for any longer time horizons also.

true is passed to estimateCombined fee for target/2 and target so that we check the max confirms for shorter time horizons as well. This is necessary to preserve monotonically increasing estimates. For non-conservative estimates we do the same thing for 2*target, but for conservative estimates we want to skip these shorter horizons checks for 2*target because we are taking the max over all time horizons so we already have monotonically increasing estimates and the purpose of conservative estimates is not to let short term fluctuations lower our estimates by too much.

Definition at line 819 of file fees.cpp.

References CONSERVATIVE, cs_feeEstimator, FeeCalculation::desiredTarget, DOUBLE_ESTIMATE, DOUBLE_SUCCESS_PCT, FeeCalculation::est, estimateCombinedFee(), estimateConservativeFee(), FULL_ESTIMATE, HALF_ESTIMATE, HALF_SUCCESS_PCT, LOCK, longStats, MaxUsableEstimate(), FeeCalculation::reason, FeeCalculation::returnedTarget, and SUCCESS_PCT.

Referenced by CTransactionBuilder::CTransactionBuilder(), estimatesmartfee(), GetDiscardRate(), and GetMinimumFee().

◆ FlushUnconfirmed()

void CBlockPolicyEstimator::FlushUnconfirmed ( CTxMemPool pool)

Empty mempool transactions on shutdown to record failure to confirm for txs still in mempool.

Definition at line 983 of file fees.cpp.

References cs_feeEstimator, BCLog::ESTIMATEFEE, GetTimeMicros(), LOCK, LogPrint, CTxMemPool::queryHashes(), and removeTx().

Referenced by PrepareShutdown().

◆ HighestTargetTracked()

unsigned int CBlockPolicyEstimator::HighestTargetTracked ( FeeEstimateHorizon  horizon) const

Calculation of highest target that estimates are tracked for.

Definition at line 710 of file fees.cpp.

References feeStats, LONG_HALFLIFE, longStats, MED_HALFLIFE, SHORT_HALFLIFE, and shortStats.

Referenced by estimaterawfee(), GetDiscardRate(), and ParseConfirmTarget().

◆ HistoricalBlockSpan()

unsigned int CBlockPolicyEstimator::HistoricalBlockSpan ( ) const
private

Number of blocks of recorded fee estimate data represented in saved data file.

Definition at line 736 of file fees.cpp.

References historicalBest, historicalFirst, nBestSeenHeight, and OLDEST_ESTIMATE_HISTORY.

Referenced by MaxUsableEstimate(), processBlock(), and Write().

◆ MaxUsableEstimate()

unsigned int CBlockPolicyEstimator::MaxUsableEstimate ( ) const
private

Calculation of highest target that reasonable estimate can be provided for.

Definition at line 746 of file fees.cpp.

References BlockSpan(), HistoricalBlockSpan(), and longStats.

Referenced by estimateSmartFee(), and processBlock().

◆ processBlock()

void CBlockPolicyEstimator::processBlock ( unsigned int  nBlockHeight,
std::vector< const CTxMemPoolEntry *> &  entries 
)

◆ processBlockTx()

bool CBlockPolicyEstimator::processBlockTx ( unsigned int  nBlockHeight,
const CTxMemPoolEntry entry 
)
private

◆ processTransaction()

void CBlockPolicyEstimator::processTransaction ( const CTxMemPoolEntry entry,
bool  validFeeEstimate 
)

◆ Read()

◆ removeTx()

bool CBlockPolicyEstimator::removeTx ( uint256  hash,
bool  inBlock 
)

Remove a transaction from the mempool tracking stats.

Definition at line 512 of file fees.cpp.

References cs_feeEstimator, feeStats, LOCK, longStats, mapMemPoolTxs, nBestSeenHeight, and shortStats.

Referenced by FlushUnconfirmed(), processBlockTx(), and CTxMemPool::removeUnchecked().

◆ Write()

bool CBlockPolicyEstimator::Write ( CAutoFile fileout) const

Member Data Documentation

◆ bucketMap

std::map<double, unsigned int> CBlockPolicyEstimator::bucketMap
private

Definition at line 257 of file fees.h.

Referenced by CBlockPolicyEstimator(), and Read().

◆ buckets

std::vector<double> CBlockPolicyEstimator::buckets
private

Definition at line 256 of file fees.h.

Referenced by CBlockPolicyEstimator(), Read(), and Write().

◆ cs_feeEstimator

CCriticalSection CBlockPolicyEstimator::cs_feeEstimator
mutableprivate

◆ DOUBLE_SUCCESS_PCT

constexpr double CBlockPolicyEstimator::DOUBLE_SUCCESS_PCT = .95
staticprivate

Require greater than 95% of X feerate transactions to be confirmed within 2 * Y blocks.

Definition at line 165 of file fees.h.

Referenced by estimateConservativeFee(), estimateFee(), and estimateSmartFee().

◆ FEE_SPACING

constexpr double CBlockPolicyEstimator::FEE_SPACING = 1.05
staticprivate

Spacing of FeeRate buckets We have to lump transactions into buckets based on feerate, but we want to be able to give accurate estimates over a large range of potential feerates Therefore it makes sense to exponentially space the buckets.

Definition at line 187 of file fees.h.

Referenced by CBlockPolicyEstimator().

◆ feeStats

std::unique_ptr<TxConfirmStats> CBlockPolicyEstimator::feeStats
private

Classes to track historical data on transaction confirmations.

Definition at line 249 of file fees.h.

Referenced by CBlockPolicyEstimator(), estimateCombinedFee(), estimateConservativeFee(), estimateRawFee(), HighestTargetTracked(), processBlock(), processBlockTx(), processTransaction(), Read(), removeTx(), and Write().

◆ firstRecordedHeight

unsigned int CBlockPolicyEstimator::firstRecordedHeight
private

Definition at line 234 of file fees.h.

Referenced by BlockSpan(), processBlock(), and Write().

◆ HALF_SUCCESS_PCT

constexpr double CBlockPolicyEstimator::HALF_SUCCESS_PCT = .6
staticprivate

Require greater than 60% of X feerate transactions to be confirmed within Y/2 blocks.

Definition at line 161 of file fees.h.

Referenced by estimateSmartFee().

◆ historicalBest

unsigned int CBlockPolicyEstimator::historicalBest
private

Definition at line 236 of file fees.h.

Referenced by HistoricalBlockSpan(), Read(), and Write().

◆ historicalFirst

unsigned int CBlockPolicyEstimator::historicalFirst
private

Definition at line 235 of file fees.h.

Referenced by HistoricalBlockSpan(), Read(), and Write().

◆ LONG_BLOCK_PERIODS

constexpr unsigned int CBlockPolicyEstimator::LONG_BLOCK_PERIODS = 42
staticprivate

Track confirm delays up to 1008 blocks for long horizon.

Definition at line 148 of file fees.h.

Referenced by CBlockPolicyEstimator(), and Read().

◆ LONG_DECAY

constexpr double CBlockPolicyEstimator::LONG_DECAY = .99931
staticprivate

Decay of .9995 is a half-life of 1008 blocks or about 2 days.

Definition at line 158 of file fees.h.

Referenced by CBlockPolicyEstimator(), and Read().

◆ LONG_SCALE

constexpr unsigned int CBlockPolicyEstimator::LONG_SCALE = 24
staticprivate

Definition at line 149 of file fees.h.

Referenced by CBlockPolicyEstimator(), and Read().

◆ longStats

◆ mapMemPoolTxs

std::map<uint256, TxStatsInfo> CBlockPolicyEstimator::mapMemPoolTxs
private

Definition at line 246 of file fees.h.

Referenced by processBlock(), processTransaction(), and removeTx().

◆ MAX_BUCKET_FEERATE

constexpr double CBlockPolicyEstimator::MAX_BUCKET_FEERATE = 1e7
staticprivate

Definition at line 180 of file fees.h.

Referenced by CBlockPolicyEstimator().

◆ MED_BLOCK_PERIODS

constexpr unsigned int CBlockPolicyEstimator::MED_BLOCK_PERIODS = 24
staticprivate

Track confirm delays up to 48 blocks for medium horizon.

Definition at line 145 of file fees.h.

Referenced by CBlockPolicyEstimator(), and Read().

◆ MED_DECAY

constexpr double CBlockPolicyEstimator::MED_DECAY = .9952
staticprivate

Decay of .998 is a half-life of 144 blocks or about 6 hours.

Definition at line 156 of file fees.h.

Referenced by CBlockPolicyEstimator(), and Read().

◆ MED_SCALE

constexpr unsigned int CBlockPolicyEstimator::MED_SCALE = 2
staticprivate

Definition at line 146 of file fees.h.

Referenced by CBlockPolicyEstimator(), and Read().

◆ MIN_BUCKET_FEERATE

constexpr double CBlockPolicyEstimator::MIN_BUCKET_FEERATE = 1000
staticprivate

Minimum and Maximum values for tracking feerates The MIN_BUCKET_FEERATE should just be set to the lowest reasonable feerate we might ever want to track.

Historically this has been 1000 since it was inheriting DEFAULT_MIN_RELAY_TX_FEE and changing it is disruptive as it invalidates old estimates files. So leave it at 1000 unless it becomes necessary to lower it, and then lower it substantially.

Definition at line 179 of file fees.h.

Referenced by CBlockPolicyEstimator().

◆ nBestSeenHeight

unsigned int CBlockPolicyEstimator::nBestSeenHeight
private

◆ OLDEST_ESTIMATE_HISTORY

const unsigned int CBlockPolicyEstimator::OLDEST_ESTIMATE_HISTORY = 6 * 1008
staticprivate

Historical estimates that are older than this aren't valid.

Definition at line 151 of file fees.h.

Referenced by HistoricalBlockSpan().

◆ SHORT_BLOCK_PERIODS

constexpr unsigned int CBlockPolicyEstimator::SHORT_BLOCK_PERIODS = 12
staticprivate

Track confirm delays up to 12 blocks for short horizon.

Definition at line 142 of file fees.h.

Referenced by CBlockPolicyEstimator(), and Read().

◆ SHORT_DECAY

constexpr double CBlockPolicyEstimator::SHORT_DECAY = .962
staticprivate

Decay of .962 is a half-life of 18 blocks or about 45 minutes.

Definition at line 154 of file fees.h.

Referenced by CBlockPolicyEstimator(), and Read().

◆ SHORT_SCALE

constexpr unsigned int CBlockPolicyEstimator::SHORT_SCALE = 1
staticprivate

Definition at line 143 of file fees.h.

Referenced by CBlockPolicyEstimator(), and Read().

◆ shortStats

std::unique_ptr<TxConfirmStats> CBlockPolicyEstimator::shortStats
private

◆ SUCCESS_PCT

constexpr double CBlockPolicyEstimator::SUCCESS_PCT = .85
staticprivate

Require greater than 85% of X feerate transactions to be confirmed within Y blocks.

Definition at line 163 of file fees.h.

Referenced by estimateSmartFee().

◆ SUFFICIENT_FEETXS

constexpr double CBlockPolicyEstimator::SUFFICIENT_FEETXS = 0.1
staticprivate

Require an avg of 0.1 tx in the combined feerate bucket per block to have stat significance.

Definition at line 168 of file fees.h.

Referenced by estimateCombinedFee(), estimateConservativeFee(), and estimateRawFee().

◆ SUFFICIENT_TXS_SHORT

constexpr double CBlockPolicyEstimator::SUFFICIENT_TXS_SHORT = 0.5
staticprivate

Require an avg of 0.5 tx when using short decay since there are fewer blocks considered.

Definition at line 170 of file fees.h.

Referenced by estimateCombinedFee(), and estimateRawFee().

◆ trackedTxs

unsigned int CBlockPolicyEstimator::trackedTxs
private

Definition at line 253 of file fees.h.

Referenced by processBlock(), and processTransaction().

◆ untrackedTxs

unsigned int CBlockPolicyEstimator::untrackedTxs
private

Definition at line 254 of file fees.h.

Referenced by processBlock(), and processTransaction().


The documentation for this class was generated from the following files:
Released under the MIT license