Italy

We introduce a real-time method that estimates the 3D human pose directly in the popular Bio Vision Hierarchy (BVH) format, given estimations of the 2D body joints originating from monocular color images. Our contributions include: (a) A novel and compact 2D pose representation. (b) A human body orientation classifier and an ensemble of orientation-tuned neural networks that regress the 3D human pose by also allowing for the decomposition of the body to an upper and lower kinematic hierarchy. This permits the recovery of the human pose even in the case of significant occlusions. (c) An efficient Inverse Kinematics solver that refines the neural-network-based solution providing 3D human pose estimations that are consistent with the limb sizes of a target person (if known). All the above yield a 33\% accuracy improvement on the Human 3.6 Million (H3.6M) dataset compared to the baseline method (MocapNET) while maintaining real-time performance (70 fps in CPU-only execution).

ICPR 2020

Occlusion-tolerant and personalized 3D human pose estimation in RGB images

realtime

kinematics

inverse

network

body

human

pose-estimation

regression

neural

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


poster


Robotics applications in urban environments are subject to obstacles that exhibit specular reflections hampering autonomous navigation. On the other hand, these reflections are highly polarized and this extra information can successfully be used to segment the specular areas. In nature, polarized light is obtained by reflection or scattering. Deep Convolutional Neural Networks (DCNNs) have shown excellent segmentation results, but require a significant amount of data to achieve best performances. The lack of data is usually overcomed by using augmentation methods. However, unlike RGB images, polarization images are not only scalar (intensity) images and standard augmentation techniques cannot be applied straightforwardly. We propose to enhance deep learning models through a regularized augmentation procedure applied to polarimetric data in order to characterize scenes more effectively under challenging conditions. We subsequently observe an average of 18.1% improvement in IoU between non augmented and regularized training procedures on real world data.

Polarimetric Image Augmentation

Reinforcement Learning has been able to solve many complicated robotics tasks without any need for feature engineering in an end-to-end fashion. However, learning the optimal policy directly from the sensory inputs, i.e the observations, often requires processing and storage of a huge amount of data. In the context of robotics, the cost of data from real robotics hardware is usually very high, thus solutions that achieve high sample efficiency are needed. We propose a method that aims at learning a mapping from the observations into a lower-dimensional state
space. This mapping is learned with unsupervised learning using loss functions shaped to incorporate prior knowledge of the environment and the task. Using the samples from the state space, the optimal policy is quickly and efficiently learned. We test the method on several mobile robot navigation tasks in a simulation
environment and also on a real robot.

Low Dimensional State Representation Learning with Reward-shaped Priors

Fashion analysis is an attractive domain for vision research due to its direct applications in e-commerce contexts.
However, fashion datasets are commonly rather demanding, as both objects and attributes tend to be fine-grained and thus result in very long-tailed datasets.
Furthermore, relationships between objects and attributes are often dense, but are crucial for the performance of fashion applications.
In this paper, we propose to generate scene graphs for existing fashion datasets.
By detecting relationships between fashion objects, their parts, and their attributes we gain a better understanding of the scenes.
As no current fashion dataset provides scene graphs, we generate relationships between fashion objects from existing annotations.
The output is post-processed and filtered to generate a meaningful scene graph for each image.
In the experiments, we can show existing applications like image retrieval benefiting from the scene graph understanding.
We first evaluate the accuracy of the generated scene graphs.
Then, we employ scene graphs to fashion image retrieval in order to showcase their performance in real applications.
The results show various benefits for fashion applications by exploiting scene graph knowledge.
The resources and model for the proposed method are available on GitHub (https://github.com/shabnamsadegh/FashionGraph).

FashionGraph: understanding fashion data using scene graph generation


Lecture videos are rapidly becoming an invaluable source of information for students across the globe. Given the large number of online courses currently available, it is important to condense the information within these videos into a compact yet representative summary that can be used for search-based applications. We propose a framework to summarize whiteboard lecture videos by finding feature representations of detected handwritten content regions to determine unique content. We investigate multi-scale histogram of gradients and embeddings from deep metric learning for feature representation. We explicitly handle occluded, growing and disappearing handwritten content. Our method is capable of producing two kinds of lecture video summaries - the unique regions themselves or so-called key content and keyframes (which contain all unique content in a video segment). We use weighted spatio-temporal conflict minimization to segment the lecture and produce keyframes from detected regions and features. We evaluate both types of summaries and find that we obtain state-of-the-art peformance in terms of number of summary keyframes while our unique content recall and precision are comparable to state-of-the-art.

Automated Whiteboard Lecture Video Summarization by Content Region Detection and Representation

The perspective-n-point (PnP) problem is of fundamental importance in computer vision. A global optimality condition for PnP that is independent of a particular rotation parameterization was recently developed by Nakano. This paper puts forward a direct least squares, algebraic PnP solution that extends Nakano's work by combining his optimality condition with the modified Rodrigues parameters (MRPs) for parameterizing rotation. The result is a system of polynomials that is solved using the Gr\obner basis approach. An MRP vector has twice the rotational range of the classical Rodrigues (i.e.

A Globally Optimal Method for the PnP Problem with MRP Rotation Parameterization


This paper tackles automated screening of Retinopa- thy of Prematurity (ROP), one of the most common causes of visual loss in childhood. Clinically, ROP screening per case requires multiple color fundus images capturing different zones of the premature retina. A desirable model shall not only make a decision at the case level, but also pinpoint which instances and what part of the instances are responsible for the decision. This paper makes the first attempt to accomplish three tasks, i.e. ROP case classification, instance selection and abnormality localization in a unified framework. To that end, we propose a new model that effectively combines instance-attention based deep multiple instance learning (MIL) and spatial attention (SA). The propose model, which we term MIL-SA, identifies positive instances in light of their contributions to case-level decision. Meanwhile, abnormal regions in the identified instances are automatically localized by the SA mechanism. Moreover, MIL-SA is learned from case-level binary labels exclusively, and in an end-to-end manner. Experiments on a large clinical dataset of 2,186 cases with 11,053 fundus images show the viability of the proposed model for all the three tasks.

Deep Multiple Instance Learning with Spatial Attention for ROP Case Classification, Instance Selection and Abnormality Localization


Sensitive gravitational wave (GW) detectors such as that of Laser Interferometer Gravitational-wave Observatory (LIGO) realize the direct observation of GW signals that confirm Einstein’s general theory of relativity. However, it remains chal- lenges to quickly detect faint GW signals from a large number of time series with background noise under unknown probability distributions. Traditional methods such as matched-filtering in general assume Additive White Gaussian Noise (AWGN) and are far from being real-time due to its high computational complexity. To avoid these weaknesses, one-dimensional (1D) Convolutional Neural Networks (CNNs) are introduced to achieve fast online detection in milliseconds but do not have enough consideration on the trade-off between the frequency and time features, which will be revisited in this paper through data pre-processing and subsequent two-dimensional (2D) CNNs during offline training to improve the online detection sensitivity.
In this work, the input data is pre-processed to form a 2D spec- trum by Short-time Fourier transform (STFT), where frequency features are extracted without learning. Then, carrying out two 1D convolutions across time and frequency axes respectively, and concatenating the time-amplitude and frequency-amplitude feature maps with equal proportion subsequently, the frequency and time features are treated equally as the input of our following two-dimensional CNNs. The simulation of our above ideas works on a generated data set with uniformly varying SNR (2-17), which combines the GW signal generated by PYCBC and the background noise sampled directly from LIGO. Satisfying the real-time online detection requirement without noise distribution assumption, the experiments of this paper demonstrate better performance on average compared to that of 1D CNNs, especially in the cases of lower SNR (4-9).

Improving Gravitational Wave Detection with 2D Convolutional Neural Networks


Functional MRI based on blood oxygenation level dependent (BOLD) contrast is well established as a neuroimaging technique for detecting neural activity in the cortex
of the human brain. Recent studies have shown that variations of BOLD signals in white matter are also related to neural activities both in resting state and under functional loading. We develop a comprehensive framework of detecting task-specific fiber pathways. We not only study fiber tracts as open curves with different physical features (shape, scale, orientation and position), but also incorporate the BOLD signals associated with them to find stimulus-relevant pathways. Specifically, we propose a novel Riemannian metric, which is a weighted sum of distances in product space of shapes and functions. This metric provides both a cost function for registration and a proper distance for comparison. Experimental results on real data have shown that we can cluster fiber pathways correctly by evaluating correlations
between BOLD signals and stimuli, temporal variations and power spectra of them.

A Riemannian Framework for Detecting Stimulus-Relevant Fiber Pathways


Our work is about simultaneously pixel-level retinal lesion segmentation and image-level disease classification.

Learn to Segment Retinal Lesions and Beyond

A network to solve the segmentation of ultra-high resolution images

Downloads

Next from ICPR 2020

Polarimetric Image Augmentation

Similar lecture

CenterRepp: Predict Central Representative Point Set’s Distribution For Detection