Video analytics for stance estimation of walking cows to identify visual signs of lameness

Objectives

The goal of this project is to automate or facilitate the detection of lameness in cattle. To accomplish this goal, the objectives are to collect videos of walking cows, extract spatiotemporal features, and build a deep neural network to identify instances of cows’ head raising, head drooping, and arching of the back.

Materials and methods

We built a deep residual neural network inspired by current algorithms for stance (or posture) estimation of moving humans and animals. Video streams were collected from cows to build an audiovisual data repository for training and testing our model. The recorded videos were saved in .mp4 file format and transformed into skeleton sequences. The video data was preprocessed and cleaned to remove redundant segments. Each video was trimmed into smaller video frames using the ‘TensorFlow Object Detection Application Programming Interface’ and the ‘OpenCV’ programming platform. Image frames were extracted from the training videos and saved in a .csv file along with their corresponding tags. Data labeling was done in presence of an expert. The training dataset was created based on the Pandas data frames that contained the labelled information. The model was trained using the ‘TensorFlow’ open-source environment for subsequent estimation of cows’ stance in the validation data set. [...]