Italy

With recent advances in RGB-D sensing technologies as well as improvements in machine learning and fusion techniques, RGB-D facial recognition has become an active area of research. A novel attention aware method is proposed to fuse two image modalities, RGB and depth, for enhanced RGB-D facial recognition. The proposed method first extracts features from both modalities using a convolutional feature extractor. These features are then fused using a two layer attention mechanism. The first layer focuses on the fused feature maps generated by the feature extractor, exploiting the relationship between feature maps using LSTM recurrent learning. The second layer focuses on the spatial features of those maps using convolution. The training database is preprocessed and augmented through a set of geometric transformations, and the learning process is further aided using transfer learning from a pure 2D RGB image training process. Comparative evaluations demonstrate that the proposed method outperforms other state-of-the-art approaches, including both traditional and deep neural network-based methods, on the challenging CurtinFaces and IIIT-D RGB-D benchmark databases, achieving classification accuracies over 98.2% and 99.3% respectively. The proposed attention mechanism is also compared with other attention mechanisms, demonstrating more accurate results.

ICPR 2020

Two-Level Attention-based Fusion Learning for RGB-D Face Recognition

modality fusion

facial recognition

rgb-d images

attention

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


poster

Real-time HI (Human Interface) systems need accurate and efficient hand detection models to meet the limited resources in budget, dimension, memory, computing, and electric power. In recent years, object detection became a less challenging task with the latest deep CNN-based state-of-the-art models, i.e., RCNN, SSD, and YOLO; however, these models cannot provide the desired efficiency and accuracy for HI systems on embedded devices due to their complex time-consuming architecture. In addition, the detection of small hands (<30x30 pixels) is still a challenging task for all the above existing methods. Thus, we propose a shallow model named Concatenated Feature Pyramid Network (CFPN) to provide above mentioned performance for small hand detection. The superiority of CFPN is confirmed on a HandFlow dataset with mAP:0.5 of 95.6 and FPS of 33 on Nvidia TX2. The COCO dataset is also used to compare with other state-of-the-art method and shows the highest efficiency and accuracy with the proposed CFPN model. Thus we conclude that the proposed model is useful for real-life small hand detection on embedded devices.

Deep Real-time Hand Detection Using CFPN on Embedded Systems

We present a method to reconstruct depth videos of non-rigidly deformable objects directly from a video sequence. The estimation of depth is performed locally on spatio-temporal patches of the video, and then the full depth video of the entire shape is recovered by combining them together. We artificially generate a database of small deforming rectangular meshes rendered with different material properties and light conditions, along with their corresponding depth videos and use such data to train a convolutional neural network based on the 3D U-Net architecture

Learning non-rigid surface reconstruction from spatio-temporal image patches


Knowing the weather and atmospheric conditions from the past can
help weather researcher to generate models like the ones used to
predict how weather conditions are likely to change as global
temperatures continue to rise. Many historical weather records are
available from the past registered on systemic basis. Historical
weather logs were created by ships, when they were on the high seas,
recording daily weather conditions such as: wind, speed,
temperature, coordinates, etc. These historical documents represent
an important source of knowledge with valuable information to
extract climatological information of several centuries ago. This paper
presents a study about the capability of state-of-the-art
handwritten text recognition system in a dataset compiled from a
ship log mainly composed by handwritten tables that contain mainly
numbers.


The HisClima database: historical weather logs for automatic transcription and information

With the development of DNNs, the training of deep models usually consumes massive data and computational resources. Therefore, the well-trained neural networks are often seen as valuable intellectual properties. Backdoor-based watermarking schemes have been proposed for copyright protection, and we want to check how robust are backdoor-based watermarking techniques. We propose a generic framework WILD for watermark removal, and the results show that backdoor-based watermarks can be easily removed.

Backdoor-Based Watermarks in Neural Networks with Limited Data

Weakly-supervised object localization is a challenging task in which the object of interest should be localized while learning its appearance. State-of-the-art methods recycle the architecture of a standard CNN by using the activation maps of the last layer for localizing the object. While this approach is simple and works relatively well, object localization relies on different features than classification, thus, a specialized localization mechanism is required during training to improve performance. In this paper, we propose a convolutional, multi-scale spatial localization network that provides accurate localization for the object of interest. Experimental results on CUB-200-2011 and ImageNet datasets show that our proposed approach provides competitive performance for weakly supervised localization.

Convolutional STN for Weakly Supervised Object Localization


Matching information across image and text modalities is a fundamental challenge for many applications that involve both vision and natural language processing. The objective is to find efficient similarity metrics to compare the similarity between visual and textual information. Existing approaches mainly match the local visual objects and the sentence words in a shared space with attention mechanisms. The matching performance is still limited because the similarity computation is based on simple comparisons of the matching features, ignoring the characteristics of their distribution in the data. In this paper, we address this limitation with an efficient learning objective that considers the \textit{discriminative feature distributions} between the visual objects and sentence words. Specifically, we propose a novel \textit{Adversarial Discriminative Domain Regularization} (ADDR) learning framework, beyond the paradigm metric learning objective, to construct a set of discriminative data domains within each image-text pairs. Our approach can generally improve the learning efficiency and the performance of existing metrics learning frameworks by regulating the distribution of the hidden space between the matching pairs. The experimental results show that this new approach significantly improves the overall performance of several popular cross-modal matching techniques on the MS-COCO and Flickr30K benchmarks.

Beyond the Deep Metric Learning: Enhance the Cross-Modal Matching with Adversarial Discriminative Domain Regularization


Image based analysis of plants is a high-throughput and non-invasive approach to study plant traits. The quantitative estimation of many plant traits (leaf area index, biomass etc.) from plant images is primarily based on accurate segmentation of individual leaves. This is a challenging task due to the presence of overlapped leaves and lack of discernible boundaries between them. To overcome these limitations, state-of-the-art supervised deep learning algorithms have been recently employed. However, the annotations of individual leaf instances is time consuming, in addition the variability in leaf shapes and its arrangement among different plant species limits the broad utilisation of these algorithms. We propose a novel framework that relies on a graph based formulation to extract leaf shape knowledge for the task of leaf instance segmentation. These shape priors are generated based on leaf shape characteristics independent of plant species. Thus, relieving the current bottleneck of annotated data.

A Hierarchical Framework for Leaf Instance Segmentation: Application to Plant Phenotyping


This paper studies the problem of Person Re-Identification (ReID)
for large-scale applications. Recent research efforts have been devoted
to building complicated part models, which introduce considerably
high computational cost and memory consumption, inhibiting its practicability
in large-scale applications. This paper aims to develop a novel learning
strategy to find efficient feature embeddings while maintaining the
balance of accuracy and model complexity. More specifically, we find
by enhancing the classical triplet loss together with cross-entropy
loss, our method can explore the hard examples and build a discriminant
feature embedding yet compact enough for large-scale applications.
Our method is carried out progressively using Bayesian optimization,
and we call it the Progressive Learning Algorithm (PLA). Extensive
experiments on three large-scale datasets show that our PLA is comparable
or better than the state-of-the-arts. Especially, on the challenging
Market-1501 dataset, we achieve Rank-1=94.7%/mAP=89.4% while saving
at least 30\% parameters than strong part models.

Directionally Paired Principal Component Analysis for Bivariate Estimation Problems


We present a multimodal deep learning framework that can generate summarization text supporting the main idea of an information graphic for presentation to a person who is blind or visually impaired. The framework utilizes the visual, textual, positional, and size characteristics extracted from the image to create the summary. Different and complimentary neural architectures are optimized for each task using crowdsourced training data. From our quantitative experiments and results, we explain the reasoning behind our framework and show the effectiveness of our models. Our qualitative results showcase text generated from our framework and show that Mechanical Turk participants favor them to other automatic and human generated summarizations.
We describe the design and results of an experiment to evaluate the utility of our system for people who have visual impairments in the context of understanding Twitter Tweets containing line graphs.

Information Graphic Summarization using a Collection of Multimodal Deep Neural Networks

Learning musical instruments like piano or electronic keyboard on average takes a decade. Currently, learning an instrument requires continuous supervision from a tutor, and it often becomes unrealistic to stay connected with the music tutor for a long time. Online music learning platforms remain unscalable as there is no way for these platforms to verify whether user pressed a key note with the intended finger, which is significant for learning finger based musical instruments. To overcome this, an autonomous system to evaluate and guide in the learning process by continuously tracking finger movements via a non-camera based solution is proposed. Finger press triggers the muscle movements which are detected at the surface of
the forearm in the form of surface Electromyography (sEMG) signals. The paper proposes tracking of finger press for 10seconds on an electronic keyboard using MyoBand [1] wearable device that provided 8 channels of sEMG signals. Eleven time and frequency domain features of sEMG signals were extracted from sEMG signals to classify musical note played by the instrument on corresponding finger press. The feature set was standardized using standard scaler approach, and vector dimensions were reduced by Linear Discriminant Analysis (LDA) method. The
resulting reduced dimension data was applied on Random Forest (RF) classifier to report best classification accuracy for our application. Experiments involved single finger press to render a note in the musical instrument, and multiple finger press to define chord sequence on an electronic musical keyboard. Further analysis was performed to maximize the classification accuracy over the number of trials and optimize the position of electrodes for successful identification of musical note played. The proposed method achieves a classification accuracy of 74.25%
for 5 musical note played on an electronic keyboard instrument with 4 MyoBand electrodes, and an accuracy of 95.83% with one electrode for identifying between four musical events including two major chords sequence and two musical notes.


Downloads

Next from ICPR 2020

Deep Real-time Hand Detection Using CFPN on Embedded Systems

Similar lecture

Transformer-Encoder Detector Module: Using Context to Improve Robustness to Adversarial Attacks on Object Detection