Italy

Multispectral disparity estimation is a difficult task since it includes the challenges found in the visible spectrum, while also having the difficulty of matching corresponding pixels with few similarities. We propose a new CNN architecture that extracts features from each patch without sharing any weights. We combine those features with two operations: correlation and concatenation. These newly merged features are then forwarded to their respective classification heads that determine if the input patches are the same or not. Experiments on the LITIV2014 and 2018 datasets show that our network outperforms other methods.

ICPR 2020

Domain Siamese CNNs for Sparse Multispectral Disparity Estimation

multispectral images

disparity estimation

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


poster


In this paper, we propose high quality image debluring by using discrete cosine transform (DCT) with less computational complexity. Recently, Convolutional Neural Network (CNN) and Generative Adversarial Network (GAN) based algorithms have been proposed for image deblurring. Moreover, multi-scale architecture of CNN restores blurred image cleary and suppresses more ringing artifacts or block noise, but it takes much time to process. To solve these problems, we propose a method that preserves texture and suppresses ringing artifacts in the restored image without multi-scale architecture using DCT based loss named ``DeblurDCTGAN.''. It compares frequency domain of the images made from deblurred image and ground truth image by using DCT. Hereby, DeblurDCTGAN can reduce block noise or ringing artifacts while maintaining deblurring performance. Our experimental results show that DeblurDCTGAN gets the highest performances on both PSNR and SSIM comparing with other conventional methods in GoPro, DVD, NFS and HIDE test Dataset. Also, the running time per pair of DeblurDCTGAN is faster than others.

GAN-Based Image Deblurring Using DCT Discriminator

Given a speech clip and facial image, the goal of talking face generation is to synthesize a talking face video with accurate mouth synchronization and natural face motion. Recent progress has proven the effectiveness of the landmarks as the intermediate information during talking face generation. However, the large gap between audio and visual modalities makes the prediction of landmarks challenging and limits generation ability. This paper proposes a semantic and temporal synchronous landmark learning method for talking face generation. First, we propose to introduce a word detector to enforce richer semantic information. Then, we propose to preserve the temporal synchronization and consistency between landmarks and audio via the proposed temporal residual loss. Lastly, we employ a U-Net generation network with adaptive reconstruction loss to generate facial images for the predicted landmarks. Experimental results on two benchmark datasets LRW and GRID demonstrate the effectiveness of our model compared to the state-of-the-art methods of talking face generation.

Talking face generation via Learning Semantic and Temporal Synchronous Landmarks


This paper studies the problem of Person Re-Identification (ReID)
for large-scale applications. Recent research efforts have been devoted
to building complicated part models, which introduce considerably
high computational cost and memory consumption, inhibiting its practicability
in large-scale applications. This paper aims to develop a novel learning
strategy to find efficient feature embeddings while maintaining the
balance of accuracy and model complexity. More specifically, we find
by enhancing the classical triplet loss together with cross-entropy
loss, our method can explore the hard examples and build a discriminant
feature embedding yet compact enough for large-scale applications.
Our method is carried out progressively using Bayesian optimization,
and we call it the Progressive Learning Algorithm (PLA). Extensive
experiments on three large-scale datasets show that our PLA is comparable
or better than the state-of-the-arts. Especially, on the challenging
Market-1501 dataset, we achieve Rank-1=94.7%/mAP=89.4% while saving
at least 30\% parameters than strong part models.

Single view learning in action recognition

Color is very important feature for image representation, it assumes very essential in the human visual recognition process. Although, several color extension of Local Binary Pattern processes the color texture from the three color channels separately (Marginal way). Aware of the high interaction that exists between different channels in the color image, this work introduces a new vector process to define a local descriptor for color images named Local Binary Quaternion Rotation Pattern (LBQRP). In this LBQRP purpose, the quaternion representation is used to represent color texture. The distance between two color can be expressed as the angle of rotation between two unit quaternions using the geodesic distance. After a LBQRP coding, local histograms are extracted and used as features. Experiments on three challenging color datasets: Vistex, Outex-TC13 and USPtex are carried out to evaluate the LBQRP performance in texture classification. Results show the high efficiency of the proposed approach facing to several stat-of-art methods.

Local Binary Quaternion Rotation Pattern for colour texture retrieval


Local binary pattern (LBP) is sensitive to inverse grayscale changes. In this paper, we propose first- and secondorder sorted LBP (SLBP) features which are robust to inverse grayscale changes and image rotation. The proposed method focuses on encoding difference-sign information and it can be generalized to embed other difference-magnitude features to obtain complementary representations. Experiments demonstrate the effectiveness of our method for texture classification under (linear or nonlinear) grayscale-inversion and rotation changes.

First- and Second-Order Sorted Local Binary Pattern Features for Grayscale-Inversion and Rotation Invariant Texture Classification


Recently, a novel radical analysis network (RAN) has the
capability of effectively recognizing unseen Chinese
character classes and largely reducing the requirement of
training data by treating a Chinese character as a
hierarchical composition of radicals rather than a single
character class.} However, when dealing with more
challenging issues, such as the recognition of complicated
characters, low-frequency character categories, and
characters in natural scenes, RAN still has a lot of room
for improvement. In this paper, we explore options to
further improve the structure generalization and robustness
capability of RAN with the Transformer architecture, which
has achieved start-of-the-art results for many
sequence-to-sequence tasks. More specifically, we propose
to replace the original attention module in RAN with the
transformer decoder, which is named as a transformer-based
radical analysis network (RTN). The experimental results
show that the proposed approach can significantly
outperform the RAN on both printed Chinese character
database and natural scene Chinese character database.
Meanwhile, further analysis proves that RTN can be better
generalized to complex samples and low-frequency
characters, and has better robustness in recognizing
Chinese characters with different attributes.

A Transformer-based Radical Analysis Attention Network for 
Chinese Character Recognition

This work explores the biases in learning processes based on deep neural network architectures. We analyze how bias affects deep learning processes through a toy example using the MNIST database and a case study in gender detection from face images. We employ two gender detection models based on popular deep neural networks. We present a comprehensive analysis of bias effects when using an unbalanced training dataset on the features learned by the models. We show how bias impacts in the activations of gender detection models based on face images. We finally propose InsideBias, a novel method to detect biased models. InsideBias is based on how the models represent the information instead of how they perform, which is the normal practice in other existing methods for bias detection. Our strategy with InsideBias allows to detect biased models with very few samples (only 15 images in our case study). Our experiments include 72K face images from 24K identities and 3 ethnic groups.

InsideBias: Measuring Bias in Deep Networks and Application to Face Gender Biometrics

Image-text matching has made great progresses recently, but there still remains challenges in fine-grained matching. To deal with this problem, we propose an Improved Visual Semantic Reasoning model (VSR++), which jointly models 1)
global alignment between images and texts and 2) local correspondence between regions and words in a unified framework. To exploit their complementary advantages, we also develop a suitable learning strategy to balance their relative importance. As a result, our model can distinguish image regions and text words in a fine-grained level, and thus achieves the current state-of-the-art performance on two benchmark datasets.

VSR++: Improving Visual Semantic Reasoning for Fine-Grained Image-Text Matching

Current compressed video action recognition methods are mainly based on complete data. However, in a real transmission scenario, the compressed video packets are usually disorderly received and even lost due to network jitters or
congestion. To recognize actions in early phases with limited packets, e.g. for quickly forecasting possible potential risks, in this paper, we propose a Temporal Enhanced Multi-Stream Network (TEMSN) towards practical compressed video action
recognition. First, we make use of three modalities in the compressed domain as complementary cues and build a multistream network to capture rich information from compressed video packets. Second, we design a temporal enhanced module
based on an Encoder-Decoder structure, which is applied to each stream to infer missing packets, generating more accurate action dynamics. Thanks to the multiple modalities and their temporal enhancement, our approach better models actions with partial available compressed video packets. Experiments on the HMDB-51 and UCF-101 datasets validate its effectiveness and efficiency.

Towards Practical Compressed Video Action Recognition: A
Temporal Enhanced Multi-Stream Network


This paper strives for action recognition and detection in video modalities like RGB, depth maps or 3D-skeleton sequences when only limited modality-specific labeled examples
are available. For the RGB, and derived optical-flow, modality
many large-scale labeled datasets have been made available. They
have become the de facto pre-training choice when recognizing
or detecting new actions from RGB datasets that have limited
amounts of labeled examples available. Unfortunately, large-scale
labeled action datasets for other modalities are unavailable for
pre-training. In this paper, our goal is to recognize actions from
limited examples in non-RGB video modalities, by learning from
large-scale labeled RGB data. To this end, we propose a two-step
training process: (i) we extract action representation knowledge
from an RGB-trained teacher network and adapt it to a nonRGB student network. (ii) we then fine-tune the transfer model
with available labeled examples of the target modality. For the
knowledge transfer we introduce feature-supervision strategies,
which rely on unlabeled pairs of two modalities (the RGB and
the target modality) to transfer feature level representations from
the teacher to the student network. Ablations and generalizations
with two RGB source datasets and two non-RGB target datasets
demonstrate that an optical-flow teacher provides better action
transfer features than RGB for both depth maps and 3Dskeletons, even when evaluated on a different target domain, or
for a different task. Compared to alternative cross-modal action
transfer methods we show a good improvement in performance
especially when labeled non-RGB examples to learn from are
scarce.

Downloads

Next from ICPR 2020

GAN-Based Image Deblurring Using DCT Discriminator

Similar lecture

Boundary Optimised Samples Training for Detecting Out-of-Distribution Images