How to dynamically change the palm detection threshold?

[Danylo Ulianych]

I'm working with the Hand Landmarks graph in C++ and need to change the palm detection threshold and the hand tracking threshold. Basically, I need to turn these calculator constants into input side packets. Here is what I've tried:

Modified "modules/hand_landmark/hand_landmark_cpu.pbtxt":

// Presence threshold

input_side_packet: "HAND_PR_THR:hand_pr_thr"

node {

calculator: "HandLandmarkThrLoader"

input_side_packet: "HAND_PR_THR:hand_pr_thr"

output_side_packet: "HAND_THR:hand_thr"

}

# Applies a threshold to the confidence score to determine whether a hand is

# present.

node {

calculator: "ThresholdingCalculator"

input_stream: "FLOAT:hand_presence_score"

output_stream: "FLAG:hand_presence"

options: {

[mediapipe.ThresholdingCalculatorOptions.ext] {

threshold: "HAND_THR:hand_thr"

}

}

}

... In C++:

std::map<std::string, mediapipe::Packet> input_side_packets;

input_side_packets["hand_pr_thr"] = mediapipe::MakePacket<float>(0.9f);

But of course, this doesn't work (double expected, got string).

Full graph proto:

# CPU image. (ImageFrame)

input_stream: "input_video"

input_stream: "input_video_visualize"

input_side_packet: "NUM_HANDS:num_hands"

# CPU image. (ImageFrame)

output_stream: "output_video"

node {

calculator: "ConstantSidePacketCalculator"

output_side_packet: "PACKET:use_prev_landmarks"

node_options: {

[type.googleapis.com/mediapipe.ConstantSidePacketCalculatorOptions]: {

packet { bool_value: true }

}

}

}

# Detects/tracks hand landmarks.

node {

calculator: "HandLandmarkTrackingCpu"

input_stream: "IMAGE:input_video"

input_side_packet: "NUM_HANDS:num_hands"

input_side_packet: "USE_PREV_LANDMARKS:use_prev_landmarks"

output_stream: "LANDMARKS:landmarks"

output_stream: "WORLD_LANDMARKS:world_landmarks"

output_stream: "HANDEDNESS:handedness"

output_stream: "PALM_DETECTIONS:multi_palm_detections"

output_stream: "HAND_ROIS_FROM_LANDMARKS:multi_hand_rects"

output_stream: "HAND_ROIS_FROM_PALM_DETECTIONS:multi_palm_rects"

}

# Subgraph that renders annotations and overlays them on top of the input

# images (see hand_renderer_cpu.pbtxt).

node {

calculator: "HandRendererSubgraph"

# Change to "input_video" to see the changes in

# the colorspace (color glove swaps R-G-B channels)

# input_stream: "IMAGE:input_video"

input_stream: "IMAGE:input_video_visualize"

input_stream: "DETECTIONS:multi_palm_detections"

input_stream: "LANDMARKS:landmarks"

input_stream: "HANDEDNESS:handedness"

input_stream: "NORM_RECTS:0:multi_palm_rects"

input_stream: "NORM_RECTS:1:multi_hand_rects"

output_stream: "IMAGE:output_video"

}

# need to detect packet presense

node {

calculator: "PacketPresenceCalculator"

input_stream: "PACKET:landmarks"

output_stream: "PRESENCE:landmark_presence"

}