湖州企业做网站,原创网站开发流程,北堂网站制作,成都网站建设需要多少钱一、背景介绍
一些报文在网络传输中#xff0c;会存在丢包重传和延时的情况。渲染时需要进行适当缓存#xff0c;等待丢失被重传的报文或者正在路上传输的报文。
jitter延时计算是确认需要缓存的时间
另外#xff0c;在检测到帧有重传情况时#xff0c;也可适当在渲染时…一、背景介绍
一些报文在网络传输中会存在丢包重传和延时的情况。渲染时需要进行适当缓存等待丢失被重传的报文或者正在路上传输的报文。
jitter延时计算是确认需要缓存的时间
另外在检测到帧有重传情况时也可适当在渲染时间内增加RTT延时时间等待丢失重传的报文
二、jitter实现原理
JitterDelay由两部分延迟造成传输大帧引起的延迟和网络噪声引起的延迟。计算公式如下 其中
estimate[0]信道传输速率的倒数 MaxFrameSize表示自会话开始以来所收到的最大帧size AvgFrameSize表示平均帧大小排除keyframe等超大帧
kNoiseStdDevs 表示噪声系数2.33 var_noise_ms2_ 表示噪声方差 kNoiseStdDevOffset: 表示噪声扣除常数30
实现函数
JitterEstimator::CalculateEstimate 1、传输大帧引起的延迟
传输大帧引起的延迟 这个公式的原理是[milliseconds] [1 / bytes per millisecond] * [bytes]
实现函数
double FrameDelayVariationKalmanFilter::GetFrameDelayVariationEstimateSizeBased(double frame_size_variation_bytes) const {// Unit: [1 / bytes per millisecond] * [bytes] [milliseconds].return estimate_[0] * frame_size_variation_bytes;
}
filtered_max_frame_size_bytes std::maxdouble(kPsi * max_frame_size_bytes_, frame_size.bytes());
constexpr double kPsi 0.9999;
filtered_avg_frame_size_bytes
是每一帧的加权平均值但是需要排除key frame这种超大帧 estimate_[0]参数计算
使用一个简化卡尔曼滤波算法在处理帧延迟变化(frame_delay_variation_ms)的估计考虑了帧大小变化(frame_size_variation_bytes)和最大帧大小(max_frame_size_bytes)作为输入参数。
void FrameDelayVariationKalmanFilter::PredictAndUpdate(double frame_delay_variation_ms,double frame_size_variation_bytes,double max_frame_size_bytes,double var_noise) {// Sanity checks.if (max_frame_size_bytes 1) {return;}if (var_noise 0.0) {return;}// This member function follows the data flow in// https://en.wikipedia.org/wiki/Kalman_filter#Details.// 1) Estimate prediction: x F*x.// For this model, there is no need to explicitly predict the estimate, since// the state transition matrix is the identity.// 2) Estimate covariance prediction: P F*P*F Q.// Again, since the state transition matrix is the identity, this update// is performed by simply adding the process noise covariance.estimate_cov_[0][0] process_noise_cov_diag_[0];estimate_cov_[1][1] process_noise_cov_diag_[1];// 3) Innovation: y z - H*x.// This is the part of the measurement that cannot be explained by the current// estimate.double innovation frame_delay_variation_ms -GetFrameDelayVariationEstimateTotal(frame_size_variation_bytes);// 4) Innovation variance: s H*P*H r.double estim_cov_times_obs[2];estim_cov_times_obs[0] estimate_cov_[0][0] * frame_size_variation_bytes estimate_cov_[0][1];estim_cov_times_obs[1] estimate_cov_[1][0] * frame_size_variation_bytes estimate_cov_[1][1];double observation_noise_stddev (300.0 * exp(-fabs(frame_size_variation_bytes) /(1e0 * max_frame_size_bytes)) 1) *sqrt(var_noise);if (observation_noise_stddev 1.0) {observation_noise_stddev 1.0;}// TODO(brandtr): Shouldnt we add observation_noise_stddev^2 here? Otherwise,// the dimensional analysis fails.double innovation_var frame_size_variation_bytes * estim_cov_times_obs[0] estim_cov_times_obs[1] observation_noise_stddev;if ((innovation_var 1e-9 innovation_var 0) ||(innovation_var -1e-9 innovation_var 0)) {RTC_DCHECK_NOTREACHED();return;}// 5) Optimal Kalman gain: K P*H/s.// How much to trust the model vs. how much to trust the measurement.double kalman_gain[2];kalman_gain[0] estim_cov_times_obs[0] / innovation_var;kalman_gain[1] estim_cov_times_obs[1] / innovation_var;// 6) Estimate update: x x K*y.// Optimally weight the new information in the innovation and add it to the// old estimate.estimate_[0] kalman_gain[0] * innovation;estimate_[1] kalman_gain[1] * innovation;// (This clamping is not part of the linear Kalman filter.)if (estimate_[0] kMaxBandwidth) {estimate_[0] kMaxBandwidth;}// 7) Estimate covariance update: P (I - K*H)*Pdouble t00 estimate_cov_[0][0];double t01 estimate_cov_[0][1];estimate_cov_[0][0] (1 - kalman_gain[0] * frame_size_variation_bytes) * t00 -kalman_gain[0] * estimate_cov_[1][0];estimate_cov_[0][1] (1 - kalman_gain[0] * frame_size_variation_bytes) * t01 -kalman_gain[0] * estimate_cov_[1][1];estimate_cov_[1][0] estimate_cov_[1][0] * (1 - kalman_gain[1]) -kalman_gain[1] * frame_size_variation_bytes * t00;estimate_cov_[1][1] estimate_cov_[1][1] * (1 - kalman_gain[1]) -kalman_gain[1] * frame_size_variation_bytes * t01;// Covariance matrix, must be positive semi-definite.RTC_DCHECK(estimate_cov_[0][0] estimate_cov_[1][1] 0 estimate_cov_[0][0] * estimate_cov_[1][1] -estimate_cov_[0][1] * estimate_cov_[1][0] 0 estimate_cov_[0][0] 0);
}
2、网络噪声引起的延迟
网络噪声引起的延迟 constexpr double kNoiseStdDevs 2.33; //噪声系数
constexpr double kNoiseStdDevOffset 30.0;//噪声扣除常数
var_noise_ms2_ //噪声方差
实现函数 噪声方差var_noise_ms2计算
var_noise_ms2 alpha * var_noise_ms2_ (1 - alpha) *(d_dT - avg_noise_ms_) *(d_dT - avg_noise_ms_);
实现函数JitterEstimator::EstimateRandomJitter 其中
d_dT 实际FrameDelay - 评估FrameDelay 在JitterEstimator::UpdateEstimate函数实现 实际FrameDelay 两帧之间实际接收gap - 两帧之间实际发送gap 在InterFrameDelayVariationCalculator::Calculate函数实现
absl::optionalTimeDelta InterFrameDelayVariationCalculator::Calculate(uint32_t rtp_timestamp,Timestamp now) {int64_t rtp_timestamp_unwrapped unwrapper_.Unwrap(rtp_timestamp);if (!prev_wall_clock_) {prev_wall_clock_ now;prev_rtp_timestamp_unwrapped_ rtp_timestamp_unwrapped;// Inter-frame delay variation is undefined for a single frame.// TODO(brandtr): Should this return absl::nullopt instead?return TimeDelta::Zero();}// Account for reordering in jitter variance estimate in the future?// Note that this also captures incomplete frames which are grabbed for// decoding after a later frame has been complete, i.e. real packet losses.uint32_t cropped_prev static_castuint32_t(prev_rtp_timestamp_unwrapped_);if (rtp_timestamp_unwrapped prev_rtp_timestamp_unwrapped_ ||!IsNewerTimestamp(rtp_timestamp, cropped_prev)) {return absl::nullopt;}// Compute the compensated timestamp difference.TimeDelta delta_wall now - *prev_wall_clock_;int64_t d_rtp_ticks rtp_timestamp_unwrapped - prev_rtp_timestamp_unwrapped_;TimeDelta delta_rtp d_rtp_ticks / k90kHz;// The inter-frame delay variation is the second order difference between the// RTP and wall clocks of the two frames, or in other words, the first order// difference between delta_rtp and delta_wall.TimeDelta inter_frame_delay_variation delta_wall - delta_rtp;prev_wall_clock_ now;prev_rtp_timestamp_unwrapped_ rtp_timestamp_unwrapped;return inter_frame_delay_variation;
}
评估FrameDelay estimate[0] * (FrameSize – PreFrameSize) estimate[1]
评估FrameDelay实现函数
double FrameDelayVariationKalmanFilter::GetFrameDelayVariationEstimateTotal(double frame_size_variation_bytes) const {double frame_transmission_delay_ms GetFrameDelayVariationEstimateSizeBased(frame_size_variation_bytes);double link_queuing_delay_ms estimate_[1];return frame_transmission_delay_ms link_queuing_delay_ms;
}
3、jitter延时更新流程 三、RTT延时计算
VideoStreamBufferController::OnFrameReady函数在判断帧有重传情况时还会根据实际情况在渲染帧时间里面增加RTT值。 JitterEstimator::GetJitterEstimate根据实际配置可以在渲染时间中适当增加一定比例的RTT延时值。 四、参考
WebRTC视频接收缓冲区基于KalmanFilter的延迟模型 - 简书在WebRTC的视频处理流水线中接收端缓冲区JitterBuffer是关键的组成部分它负责RTP数据包乱序重排和组帧RTP丢包重传请求重传关键帧估算缓冲区延迟等功能...https://www.jianshu.com/p/bb34995c549a
