Selected frame index in video Auto-Annotation predict method

[zzv2]

I am working on a custom version of the segment_anything_2_video ml backend and I was wondering if there is a way to obtain the currently selected video frame in the annotator in the predict method. I looked in both the context and the tasks objects that are passed to the predict method, but I could not find it.

For example, if I am doing video annotation, and I have frame 439 currently selected in the annotator window, and then I trigger the Auto-Annotation, I want to be able to have 439 passed to the predict method (in the context would make the most sense to me) or call some API or method within the predict method to retrieve the currently selected frame 439.

[heidi-humansignal]

Hello,

Label Studio sends the current frame back to your ML backend in the context parameter. That payload includes a sequence array, where each entry has a frame field—the frame index where the user action occurred.

Something similar to this:

def predict(self, tasks, context, **kwargs): selected_frame = None if context and 'result' in context: # context['result'][0]['value']['sequence'] holds the prompt sequence seq = context['result'][0]['value']['sequence'] if seq: # The first entry is the frame where the annotation was made selected_frame = seq[0]['frame'] # Now selected_frame == 439 (if that was the frame the user was on) print("User selected frame:", selected_frame) # …rest of your prediction logic… return ModelResponse(predictions=[…])

Thank you,
Abu

Comment by Abubakar Saad
Workflow Run