Action Prediction During Human-Object Interaction Based on Dtw and Early Fusion of Human and Object Representations

EasyChair Preprint 6218

Abstract

Action prediction is defined as the inference of an action label while the action is still ongoing. Such a capability is extremely useful for early response and further action planning. In this paper, we consider the problem of action prediction in scenarios involving humans interacting with objects. We formulate an approach that builds time series representations of the performance of the humans and the objects. Such a representation of an ongoing action is then compared to prototype actions. This is achieved by a Dynamic Time Warping (DTW)-based time series alignment framework which identifies the best match between the ongoing action and the prototype ones. Our approach is evaluated quantitatively on three standard benchmark datasets. Our experimental results reveal the importance of the fusion of human- and object-centered action representations in the accuracy of action prediction. Moreover, we demonstrate that the proposed approach achieves significantly higher action prediction accuracy compared to competitive methods.

Keyphrases: Dynamic Time Warping (DTW), Early Action Classification, Global Alignment Kernel (GAK)., Observation ratio, Soft Dynamic Time Warping (Soft DTW), Time series alignment, Trimmed video, action classification, action label, action prediction, action prediction accuracy, computer vision, distance matrix, human-object interaction, msr daily activity dataset, short term action prediction

@booklet{EasyChair:6218,
  author    = {Victoria Manousaki and Konstantinos Papoutsakis and Antonis Argyros},
  title     = {Action Prediction During Human-Object Interaction Based on Dtw and Early Fusion of Human and Object Representations},
  howpublished = {EasyChair Preprint 6218},
  year      = {EasyChair, 2021}}