Active map regions skipped when zero-MV heuristic triggers in real-time encodes

[Haseeb Abdul Qadir]

Hi,

We’re using libaom in realtime mode for screen content. We've noticed that very small screen updates (a few pixels) sometimes never get coded. It seems like the zero-MV SAD heuristic forces skip, even though the active map marks those blocks as active. I used an AI tool to create a patch below. It disables the heuristic whenever active_map is enabled. I'd really appreciate any feedback on the patch to see if this is the right direction for a fix to this problem.

Thank you for your help.

-Haseeb.

diff --git a/av1/encoder/partition_search.c b/av1/encoder/partition_search.c
index 81ac00be2a..b4f1e5d2de 100644
--- a/av1/encoder/partition_search.c
+++ b/av1/encoder/partition_search.c
@@ -2172,8 +2172,15 @@ static void encode_b_nonrd(const AV1_COMP *const cpi, TileDataEnc *tile_data,
 static int get_force_zeromv_skip_flag_for_blk(const AV1_COMP *cpi,
                                               const MACROBLOCK *x,
                                               BLOCK_SIZE bsize) {
-  // Force zero MV skip based on SB level decision
-  if (x->force_zeromv_skip_for_sb < 2) return x->force_zeromv_skip_for_sb;
+  // Force zero MV skip based on SB level decision. Honor user-provided active
+  // maps: when they are enabled we must continue evaluating motion even if the
+  // SAD-based heuristics think the block is static, otherwise we can "forget"
+  // to encode small updates in active regions.
+  if (x->force_zeromv_skip_for_sb < 2) {
+    return cpi->active_map.enabled ? 0 : x->force_zeromv_skip_for_sb;
+  }
+
+  if (cpi->active_map.enabled) return 0;

   // For blocks of size equal to superblock size, the decision would have been
   // already done at superblock level. Hence zeromv-skip decision is skipped.
diff --git a/av1/encoder/var_based_part.c b/av1/encoder/var_based_part.c
index 3027726726..5724f78eb5 100644
--- a/av1/encoder/var_based_part.c
+++ b/av1/encoder/var_based_part.c
@@ -1560,6 +1560,8 @@ static inline bool set_force_zeromv_skip_for_sb(
     unsigned int *uv_sad, int mi_row, int mi_col, unsigned int y_sad,
     BLOCK_SIZE bsize) {
   AV1_COMMON *const cm = &cpi->common;
+  if (cpi->active_map.enabled) return false;
+
   if (!is_set_force_zeromv_skip_based_on_src_sad(
           cpi->sf.rt_sf.set_zeromv_skip_based_on_source_sad,
           x->content_state_sb.source_sad_nonrd))

[Marco Paniconi]

Hi Haseeb,

Thanks for the issue and the patch. As we discussed, you can try disabling the speed feature that controls that logic when the active_maps is used (i.e., set  sf->rt_sf.set_zeromv_skip_based_on_source_sad=0 when cpi->active_maps.enabled=1; in the speed_feautres.c). Let me know if that fix resolves the issue.

Thanks,

-marco

--
You received this message because you are subscribed to the Google Groups "AV1 Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to av1-discuss...@aomedia.org.
To view this discussion visit https://groups.google.com/a/aomedia.org/d/msgid/av1-discuss/CADTa4GDXvUVSJpZyuQC1m1SE9S5iZw2_U0Rz5RiJ554QY7%3D1tw%40mail.gmail.com.

[Haseeb Abdul Qadir]

Hi Marco,

Thanks for your help on this. I've tested your suggested fix on all our problematic scenarios and I can confirm that the fix is working great! It fixes the artifacts we were seeing.

 Please find the patch attached. 

Thanks again!

[Marco Paniconi]

Great, the patch is uploaded here: https://aomedia-review.googlesource.com/c/aom/+/204421