Is this a right way to acheive LTR in av1

[klaus zhang]

PS:Sorry to start a new conversation here,as I can not reply in the previous conversation any more,and I don't know why.


Is this a right way to acheive LTR in av1 ?
With this implemnts,we could have the encoded video with bad visual quality.

# decoder tell encoder which frame has been decoded successfully

'newest_available_ref_poc' is the frame has been decoded successfully.

and then at encoder side,new encoded frame would reference this frame.

# achevie ltr by reference golden frame at encoder side

'''
poc_ is start from zero ,for the first key frame,increase by one each frame encoded.

REF_FRAMES = 8;

// In ltr mode,do not generate keyframe immediately,use the ltr
// pattern to control the keyframe generating.
force_keyframe = (poc_ - newest_available_ref_poc >= REF_FRAMES) &&
                (poc_ - last_keyframe_poc_ >= REF_FRAMES);

void LibaomAv1Encoder::SetSvcRefFrameConfigToAcheiveLTR(
    const ScalableVideoController::LayerFrameConfig& layer_frame,
    int newest_available_ref_poc,
    bool force_keyframe) {
  const int slot_number = 8;  // slots 0 - 7.
  int last_idx = 0;
  int last_idx_refresh = poc_ % slot_number;
  // Moving index slot for last: 0 - (sh - 1)
  if (poc_ > 1)
    last_idx = (poc_ - 1) % slot_number;

  aom_svc_ref_frame_config_t ref_frame_config = {};
  // Set the reference map buffer idx for the 7 references:
  // LAST_FRAME (0), LAST2_FRAME(1), LAST3_FRAME(2), GOLDEN_FRAME(3),
  // BWDREF_FRAME(4), ALTREF2_FRAME(5), ALTREF_FRAME(6).
  for (int i = 0; i < INTER_REFS_PER_FRAME; i++)
    ref_frame_config.ref_idx[i] = i;
  for (int i = 0; i < INTER_REFS_PER_FRAME; i++)
    ref_frame_config.reference[i] = 0;
  for (int i = 0; i < REF_FRAMES; i++)
    ref_frame_config.refresh[i] = 0;

  ref_frame_config.refresh[last_idx_refresh] = 1;

  if (!force_keyframe) {
    ref_frame_config.ref_idx[SVC_GOLDEN_FRAME] =
        newest_available_ref_poc % slot_number;
    ref_frame_config.reference[SVC_GOLDEN_FRAME] = 1;
    if (qoe_experiment_->enable_detail_log()) {
      RTC_LOG(LS_INFO) << __FUNCTION__ << ",poc/ref_poc/ref_index:" << poc_
                       << "/" << newest_available_ref_poc << "/"
                       << (newest_available_ref_poc % slot_number);
    }
  } else {
    RTC_LOG(LS_INFO) << __FUNCTION__
                     << ",the newest received decoded frame was too old to "
                        "reference,force generating a key frame then.current "
                        "poc/newest_available_ref_poc:"
                     << poc_ << "/" << newest_available_ref_poc;
  }

  aom_codec_err_t ret = aom_codec_control(&ctx_, AV1E_SET_SVC_REF_FRAME_CONFIG,
                                          &ref_frame_config);
  if (ret != AOM_CODEC_OK) {
    RTC_LOG(LS_WARNING) << "LibaomAv1Encoder::Encode returned " << ret
                        << " on control AV1_SET_SVC_REF_FRAME_CONFIG.";
  }
  aom_svc_ref_frame_comp_pred_t ref_frame_comp_pred;
  ref_frame_comp_pred.use_comp_pred[0] = 0;  // GOLDEN_LAST
  ref_frame_comp_pred.use_comp_pred[1] = 0;  // LAST2_LAST
  ref_frame_comp_pred.use_comp_pred[2] = 0;  // ALTREF_LAST
  ret = aom_codec_control(&ctx_, AV1E_SET_SVC_REF_FRAME_COMP_PRED,
                          &ref_frame_comp_pred);
  if (ret != AOM_CODEC_OK) {
    RTC_LOG(LS_WARNING) << "LibaomAv1Encoder::Encode returned " << ret
                        << " on control AV1E_SET_SVC_REF_FRAME_COMP_PRED.";
  }
}
'''

[Marco Paniconi]

Hi,

I added your code to the svc_encoder_rtc.cc for testing in this patch (the command line I used to test the quality issue is there too):

https://aomedia-review.googlesource.com/c/aom/+/200501

The uploaded patch has some fixes for the quality issue. Let me know if they work for you.

Thanks,

-marco

--
You received this message because you are subscribed to the Google Groups "AV1 Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to av1-discuss...@aomedia.org.
To view this discussion visit https://groups.google.com/a/aomedia.org/d/msgid/av1-discuss/e0a7ad99-1518-4844-b959-729c93adff8fn%40aomedia.org.

[klaus zhang]

Thanks for your reply! 

I've applied the new code
1. patch you uploaded
2. change cpu speed from 9 to 7

Difference of the configurations between us
1. in realtime mode,we use 4 threads for encoding,and it's one thread in your command line for testing.

The test result
The video quality is improved much better than the previous one,however,some quality issue still exists
1. the video become in a bit bad quality perioudly

2. the video is a little flickering