Markerless methods for animal posture tracking have been rapidly developing recently, but frameworks and benchmarks for tracking large animal groups in 3D are still lacking. To overcome this gap in the literature, we present 3D-MuPPET, a framework to estimate and track 3D poses of up to 10 pigeons at interactive speed using multiple camera views. We train a pose estimator to infer 2D keypoints and bounding boxes of multiple pigeons, then triangulate the keypoints to 3D. For identity matching of individuals in all views, we first dynamically match 2D detections to global identities in the first frame, then use a 2D tracker to maintain IDs across views in subsequent frames. We achieve comparable accuracy to a state of the art 3D pose estimator in terms of median error and Percentage of Correct Keypoints. Additionally, we benchmark the inference speed of 3D-MuPPET, with up to 9.45 fps in 2D and 1.89 fps in 3D, and perform quantitative tracking evaluation, which yields encouraging results. Finally, we showcase two novel applications for 3D-MuPPET. First, we train a model with data of single pigeons and achieve comparable results in 2D and 3D posture estimation for up to 5 pigeons. Second, we show that 3D-MuPPET also works in outdoors without additional annotations from natural environments. Both use cases simplify the domain shift to new species and environments, largely reducing annotation effort needed for 3D posture tracking. To the best of our knowledge we are the first to present a framework for 2D/3D animal posture and trajectory tracking that works in both indoor and outdoor environments for up to 10 individuals. We hope that the framework can open up new opportunities in studying animal collective behaviour and encourages further developments in 3D multi-animal posture tracking.

用于动物姿态跟踪的无标记方法最近发展迅速，但仍然缺乏用于 3D 跟踪大型动物群体的框架和基准。为了克服文献中的这一空白，我们提出了 3D-MuPPET，这是一个使用多个摄像机视图以交互速度估计和跟踪最多 10 只鸽子的 3D 姿势的框架。我们训练姿态估计器来推断多只鸽子的 2D 关键点和边界框，然后将关键点三角测量为 3D。对于所有视图中个体的身份匹配，我们首先将 2D 检测与第一帧中的全局身份进行动态匹配，然后使用 2D 跟踪器在后续帧中跨视图维护 ID。在中值误差和正确关键点百分比方面，我们实现了与最先进的 3D 姿态估计器相当的精度。此外，我们对 3D-MuPPET 的推理速度进行了基准测试，2D 时高达 9.45 fps，3D 时高达 1.89 fps，并进行了定量跟踪评估，取得了令人鼓舞的结果。最后，我们展示了 3D-MuPPET 的两个新颖应用。首先，我们使用单只鸽子的数据训练模型，并在最多 5 只鸽子的 2D 和 3D 姿势估计中获得可比较的结果。其次，我们证明 3D-MuPPET 也可以在户外工作，无需来自自然环境的额外注释。这两个用例都简化了向新物种和环境的领域转移，大大减少了 3D 姿势跟踪所需的注释工作。据我们所知，我们是第一个提出 2D/3D 动物姿势和轨迹跟踪框架的公司，该框架适用于最多 10 个人的室内和室外环境。我们希望该框架能够为研究动物集体行为开辟新的机会，并鼓励 3D 多动物姿势跟踪的进一步发展。