United States

Deep neural networks have been successfully deployed in various domains of artificial intelligence, including computer vision and natural language processing. We observe that the current standard procedure for training DNNs discards all the learned information in the past epochs except the current learned weights. An interesting question is: is this discarded information indeed useless? We argue that the discarded information can benefit the subsequent training. In this paper, we propose learning with retrospection (LWR) which makes use of the learned information in the past epochs to guide the subsequent training. LWR is a simple yet effective training framework to improve accuracies, calibration, and robustness of DNNs without introducing any additional network parameters or inference cost, but only with a negligible training overhead. Extensive experiments on several benchmark datasets demonstrate the superiority of LWR for training DNNs.

AAAI 2021

Learning with Retrospection

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


ICPR 2020

technical paper

The purpose of the AAAI conference is to promote research in artificial intelligence (AI) and scientific exchange among AI researchers, practitioners, scientists, and engineers in affiliated disciplines. 

Mobile devices are becoming an important carrier for deep learning tasks, as they are being equipped with powerful, high-end mobile CPUs and GPUs. However, it is still a challenging task to execute 3D Convolutional Neural Networks (CNNs) targeting for real-time performance, besides high inference accuracy. The reason is more complex model structure and higher model dimensionality overwhelm the available computation/storage resources on mobile devices. A natural way may be turning to deep learning weight pruning techniques. However, the direct generalization of existing 2D CNN weight pruning methods to 3D CNNs is not ideal for fully exploiting mobile parallelism while achieving high inference accuracy.

This paper proposes RT3D, a model compression and mobile acceleration framework for 3D CNNs, seamlessly integrating neural network weight pruning and compiler code generation techniques. We propose and investigate two structured sparsity schemes i.e., the vanilla structured sparsity and kernel group structured (KGS) sparsity that are mobile acceleration friendly. The vanilla sparsity removes whole kernel groups, while KGS sparsity is a more fine-grained structured sparsity that enjoys higher flexibility while exploiting full on-device parallelism. We propose a reweighted regularization pruning algorithm to achieve the proposed sparsity schemes. The inference time speedup due to sparsity is approaching the pruning rate of the whole model FLOPs (floating point operations). RT3D demonstrates up to 29.1x speedup in end-to-end inference time comparing with current mobile frameworks supporting 3D CNNs, with moderate 1%~1.5% accuracy loss. The end-to-end inference time for 16 video frames could be within 150 ms, when executing representative C3D and R(2+1)D models on a cellphone. For the first time, real-time execution of 3D CNNs is achieved on off-the-shelf mobiles.

RT3D: Achieving Real-Time Execution of 3D Convolutional Neural Networks on Mobile Devices

Over the past half-decade, many methods have been considered for neural architecture search (NAS). Bayesian optimization (BO), which has long had success in hyperparameter optimization, has recently emerged as a very promising strategy for NAS when it is coupled with a neural predictor. Recent work has proposed different instantiations of this framework, for example, using Bayesian neural networks or graph convolutional networks as the predictive model within BO. However, the analyses in these papers often focus on the full-fledged NAS algorithm, so it is difficult to tell which individual components of the framework lead to the best performance.

In this work, we give a thorough analysis of the "BO + neural predictor framework" by identifying five main components: the architecture encoding, neural predictor, uncertainty calibration method, acquisition function, and acquisition function optimization. We test several different methods for each component and also develop a novel path-based encoding scheme for neural architectures, which we show theoretically and empirically scales better than other encodings. Using all of our analyses, we develop a final algorithm called BANANAS, which achieves state-of-the-art performance on NAS search spaces. We adhere to the NAS research checklist (Lindauer and Hutter 2019) to facilitate best practices, and our code is available at https://github.com/naszilla/naszilla.

BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search

Data mixing augmentation has proved effective in training deep models. Recent methods mix labels mainly according to the mixture proportion of image pixels. Due to the major discriminative information of a fine-grained image usually resides in subtle regions, these methods tend to introduce heavy label noise in fine-grained recognition. We propose Semantically Proportional Mixing (SnapMix) that exploits class activation map (CAM) to lessen the label noise in augmenting fine-grained data. SnapMix generates the target label for a mixed image by estimating its intrinsic semantic composition. This strategy can adapt to asymmetric mixing operations and ensure semantic correspondence between synthetic images and target labels. Experiments show that our method consistently outperforms existing mixed-based approaches regardless of different datasets or network depths. Further, by incorporating the mid-level features, the proposed SnapMix achieves top-level performance, demonstrating its potential to serve as a strong baseline for fine-grained recognition.

SnapMix: Semantically Proportional Mixing for Augmenting Fine-Grained Data

This paper explores the problem of clustering ensemble, which aims to combine multiple base clusterings to produce better performance than that of the individual one. The existing clustering ensemble methods generally construct a co-association matrix, which indicates the pairwise similarity between samples, as the weighted linear combination of the connective matrices from different base clusterings, and the resulting co-association matrix is then adopted as the input of an off-the-shelf clustering algorithm, e.g., spectral clustering. However, the co-association matrix may be dominated by poor base clusterings, resulting in inferior performance. In this paper, we propose a novel low-rank tensor approximation based method to solve the problem from a global perspective. Specifically, by inspecting whether two samples are clustered to an identical cluster under different base clusterings, we derive a coherent-link matrix, which contains limited but highly reliable relationships between samples. We then stack the coherent-link matrix and the co-association matrix to form a three-dimensional tensor, the low-rankness property of which is further explored to propagate the information of the coherent-link matrix to the co-association matrix, producing a refined co-association matrix. We formulate the proposed method as a convex constrained optimization problem and solve it efficiently. Experimental results over 7 benchmark data sets show that the proposed model achieves a breakthrough in clustering performance, compared with 12 state-of-the-art methods. To the best of our knowledge, this is the first work to explore the potential of low-rank tensor on clustering ensemble, which is fundamentally different from previous approaches. Last but not least, our method only contains one parameter, which can be easily tuned.

Clustering Ensemble Meets Low-Rank Tensor Approximation

Recently, the relationship among individual pedestrian images and the relationship among pairwise pedestrian images have become attractive for person re-identification (re-ID) as they effectively improve the ability of feature representation. In this paper, we propose a novel method named Hybrid Relationship Network (HRNet) to learn the two types of relationships in a unified framework that makes use of their own advantages. Specifically, for the relationship among individual pedestrian images, we take the features of pedestrian images as the nodes to construct a locally-connected graph, so as to improve the discriminative ability of nodes. Meanwhile, we propose the consistent node constraint to inject the identity information into the graph learning process and guide the information to propagate accurately. As for the relationship among pairwise pedestrian images, we treat the feature differences of pedestrian images as the nodes to construct a fully-connected graph so as to estimate robust similarity of nodes. Furthermore, we propose the inter-graph propagation to alleviate the information loss for the fully-connected graph. Extensive experiments on Market-1501, DukeMTMCreID, CUHK03 and MSMT17 demonstrate that the proposed HRNet outperforms the state-of-the-art methods.

Learning Hybrid Relationships for Person Re-Identification

State estimation problems without absolute position measurements routinely arise in navigation of unmanned aerial vehicles, 
autonomous ground vehicles, etc., whose proper operation relies on 
accurate state estimates and reliable covariances.
Unaware of absolute positions, these problems have immanent unobservable directions.
Traditional causal estimators, however, usually gain spurious information 
on the unobservable directions, leading to over-confident covariance inconsistent with actual estimator errors.
The consistency problem of fixed-lag smoothers (FLSs) 
has only been attacked by the first estimate Jacobian (FEJ) technique because 
of the complexity to analyze their observability property.
But the FEJ has several drawbacks hampering its wide adoption.
To ensure the consistency of a FLS, this paper introduces the right invariant
error formulation into the FLS framework. To our knowledge, 
we are the first to analyze the observability of a FLS with the right invariant error.
Our main contributions are twofold.
As the first novelty, to bypass the complexity of analysis with the classic 
observability matrix, we show that observability analysis of FLSs can be done equivalently on the linearized system.
Second, we prove that the inconsistency issue in the traditional FLS can be
elegantly solved by the right invariant error formulation without artificially correcting Jacobians.
By applying the proposed FLS to the monocular visual inertial
simultaneous localization and mapping (SLAM) problem,
we confirm that the method consistently
estimates covariance similarly to a batch smoother in simulation and that our method achieved comparable accuracy as traditional FLSs on real data.

Consistent Right-Invariant Fixed-Lag Smoother with Application to Visual Inertial SLAM

Purchase prediction is an essential task in both online and offline retail industry, especially during major shopping festivals, when strong promotion boosts consumption dramatically. It is important for merchants to forecast such surge of sales and have better preparation. This is a challenging problem, as the purchase patterns during shopping festivals are significantly different from usual cases and also rare in historical data. Most existing methods fail at this problem due to the extremely scarce data samples as well as the inability to capture the complex macroscopic spatio-temporal dependencies in a city. To address this problem, we propose the Spatio-Temporal Meta-learning Prediction (STMP) model for purchase prediction during shopping festivals. STMP is a meta-learning based spatio-temporal multi-task deep generative model. It adopts a meta-learning framework with few-shot learning capability to capture both spatial and temporal data representations. A generative component then uses the extracted spatio-temporal representation and input data to infer the prediction results. Extensive experiments demonstrate the meta-learning generalization ability of STMP. STMP outperforms baselines in all cases, which shows the effectiveness of our model.

Robust Spatio-Temporal Purchase Prediction via Deep Meta Learning

Understanding the effect of depth in deep learning is a critical problem. In this work, we utilize the Fourier analysis to empirically provide a promising mechanism to understand why feedforward deeper learning is faster. To this end,  we separate a deep neural network, trained by normal stochastic gradient descent, into two parts during analysis, i.e., a  pre-condition component and  a learning component, in which the output of the pre-condition one is the input of the learning one. We use a filtering method to characterize the frequency distribution of a high-dimensional function.  Based on experiments of deep networks and real dataset, we propose a deep frequency principle, that is, the effective target function for a deeper hidden layer biases towards  lower  frequency during the training. Therefore, the learning component effectively learns a lower frequency function if the pre-condition component has more layers. Due to the well-studied frequency principle, i.e., deep neural networks learn lower frequency functions faster, the deep frequency principle provides a reasonable explanation to why deeper learning is faster. We believe these empirical studies would be valuable for future theoretical studies of the effect of depth in deep learning.

Deep Frequency Principle Towards Understanding Why Deeper Learning Is Faster

Affinity Propagation (AP) is a fundamental algorithm to identify clusters included in data objects.
Given a similarities among objects, it iteratively performs message updates between all data object pairs until convergence.
Although AP yields a higher clustering quality compared with other methods, it is computationally expensive.
Hence, it has difficulty handling massive datasets that include numerous data objects.
This is because the message updates require a quadratic cost of the number of data objects.
Here, we propose a novel fast algorithm, ScaleAP, which outputs the same clusters as AP but within a shorter computation time.
ScaleAP dynamically excludes unnecessary message updates without sacrificing its clustering accuracy.
Our extensive evaluations demonstrate that ScaleAP outperforms existing AP algorithms in terms of running time by up to two orders of magnitude.

Scalable Affinity Propagation for Massive Datasets

Visual saliency has emerged as a major visualization tool for interpreting deep reinforcement learning (RL) agents. However, much of the existing research uses it as an analyzing tool rather than an inductive bias for policy learning. In this work, we use visual attention as an inductive bias for RL agents. We propose a novel self-supervised attention learning approach which can 1. learn to select regions of interest without explicit annotations, and 2. act as a plug for existing deep RL methods to improve the learning performance. We empirically show that the self-supervised attention-aware deep RL methods outperform the baselines in the context of both the rate of convergence and performance. Furthermore, the proposed self-supervised attention is not tied with specific policies, nor restricted to a specific scene. We posit that the proposed approach is a general self-supervised attention module for multi-task learning and transfer learning, and empirically validate the generalization ability of the proposed method. Finally, we show that our method learns meaningful object keypoints highlighting improvements both qualitatively and quantitatively.

Downloads

Next from AAAI 2021

RT3D: Achieving Real-Time Execution of 3D Convolutional Neural Networks on Mobile Devices

Similar lecture

Is the Meta-Learning Idea Able to Improve the Generalization of Deep Neural Networks on the Standard Supervised Learning?

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAAI 2021

RT3D: Achieving Real-Time Execution of 3D Convolutional Neural Networks on Mobile Devices

Similar lecture

Is the Meta-Learning Idea Able to Improve the Generalization of Deep Neural Networks on the Standard Supervised Learning?

Downloads