EquineML: Machine learning methods for recognition of the pain expressions of horses
Recognition of pain in horses and other animals is important, because
pain is a manifestation of disease and decreases animal
welfare. Pain diagnostics for humans typically includes
self-evaluation and location of the pain with the help of
standardized forms, and labeling of the pain by an clinical expert
using pain scales. However, animals cannot verbalize their pain as
humans can, and the use of standardized pain scales is challenged by
the fact that animals as horses and cattle, being prey animals,
display subtle and less obvious pain behavior - it is simply
beneficial for a prey animal to appear healthy, in order lower the
interest from predators. The aim of this project is to develop a
method for automatic recognition of pain in horses. The method
employs an RGB-D sensor mounted in the stable ceiling, and detects
and recognizes behavioral patterns related to pain, in an automated
manner when the horse perceives itself as being alone. The Machine
Learning-based pain classification system is trained with examples
of behavioral traits labeled with pain level and pain
characteristics. This automated, user independent system for
recognition of pain behavior in horses will be the first of its kind
in the world. A successful system might change the concept for how
we monitor and care for our animals.
Via forskning kan datorer se hästens 3D-form,
Hästsverige, January, 2021.
Zhenghong Li, Sofia Broomé, Pia Haubro Andersen, and Hedvig
Kjellström. Automated detection of equine facial action units,
Joonatan Mänttäri*, Sofia Broomé*, John Folkesson, and Hedvig Kjellström. Interpreting video features: A comparison of 3D convolutional networks and convolutional LSTM networks. In Asian Conference on Computer Vision, 2020. (*Joint first authors)
F. M. Serra
S. Broomé, M. Rhodin, S. Björnsdóttir,
V. Gunnarsson, J. P. Voskamp, E. Persson-Sjodin, W. Back,
G. Lindgren, M. Novoa-Bravo, C. Roepstorff, B. J. van der Zwaag,
P. R. Van Weeren, and E. Hernlund. Improving gait classification
in horses by using inertial measurement unit (IMU) generated data
and machine learning. Nature Scientific Reports 10, paper
number 17785, 2020.
Katrina Ask, Marie Rhodin, Lena-Mari Tamminen, Elin Hernlund, and Pia Haubro Andersen. Identification of body behaviors and facial expressions associated with induced orthopedic pain in four equine pain scales. Animals 10(11), 2155, 2020.
Maheen Rashid, Hedvig Kjellström, and Yong Jae Lee. Action graphs: Weakly-supervised action localization with graph convolution networks. In IEEE Winter Conference on Applications of Computer Vision, 2020.
Sofia Broomé, Karina Bech Gleerup, Pia Haubro Andersen, and Hedvig
Kjellström. Dynamics are important for the recognition of
equine pain in video. In IEEE Conference on Computer Vision and
Pattern Recognition, 2019.
Pia Haubro Andersen, Karina B. Gleerup, Jennifer Wathan, Britt Coles, Hedvig
Kjellström, Sofia Broomé, Yong Jae Lee, Maheen Rashid,
Claudia Sonder, Erika Rosenberg, and Deborah Forster. Can a machine learn to see horse pain? An interdisciplinary approach towards automated decoding of facial expressions of pain in the horse. In International Conference on Methods and
Techniques in Behavioral Research, 2018.