10 Most Notable AI Papers of 2020

The pandemic in 2020 has caused all major AI conferences to go virtual and thus made the latest research discussions much more accessible to the global community. Here, we put together a list of 10 notable AI papers from this year’s top conferences and publications, covering new research in computer vision, natural language processing, reinforcement learning, recommendation systems, robotics, and more. For each paper, we provide a short summary and link to the original conference talk or explanatory video for a quick grasp of the research highlights.

‍

1. Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wild

Award: CVPR 2020 Best Paper

Authors: Shangzhe Wu, Christian Rupprecht, Andrea Vedaldi (University of Oxford)

In this paper, the authors present a technique that learns the 3D deformable object categories from raw, single-view, and weakly symmetric object category images which can be trained without any manual supervision. The proposed model works on the basis of an autoencoder network which divides the input scene into albedo, illumination, viewpoint, and depth information. It creates a photo-geometric autoencoding pipeline that is trained using the reconstruction loss without any 3D ground truths, multiple views, or prior shape models. The simulation results show that the proposed method can accurately recover the 3D shape of human faces, cat faces, and cars with fine details, and outperforms other unsupervised methods and even the DepthNet model that uses supervision on 2D image correspondences. [Paper Link]

Watch the conference talk video:

‍

‍

2. Beyond Accuracy: Behavioral Testing of NLP Models with CheckList

Award: ACL 2020 Best Paper

Authors: Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, Sameer Singh (Microsoft Research, University of Washington, University of California-Irvine)

The use of held-out accuracy as a performance parameter often leads to overestimation. Although the current evaluation techniques are very useful, they are not sufficient. The authors present CheckList, which is a task agnostic approach for assessing NLP models and comprises a collection of various performance tests based on linguistic abilities. They also propose a software that quickly generates numerous testing measures. The results show that although previous models have extremely effective abilities, they are unable to pass many linguistic tests. The researchers have discovered new issues and actionable bugs in different models using the CheckList. [Paper Link]

Watch the paper explanatory video:

‍

3. Efficiently Sampling Functions from Gaussian Process Posteriors

Award: ICML 2020 Outstanding Paper

Authors: James T. Wilson, Viacheslav Borovitskiy, Alexander Terenin, Peter Mostowsky, and Marc Peter Deisenroth (Imperial College London, St. Petersburg State University, St. Petersburg Department of Steklov Mathematical Institute of Russian Academy of Sciences, University College London)

The Gaussian processes are of great importance in modeling real-world problems. This paper proposes a straightforward and general-purpose methodology for fast posterior sampling for the Gaussian process. Using Matheron’s rule, the authors decompose the posterior as the sum of prior and an update. This decomposition lends itself to scalable sampling by detaching the prior from the data.This sampling pairs easily with the sparse approximations and allow the system to be scalable too. [Paper Link]

Watch the conference talk video:

‍

4. RAFT: Recurrent All-Pairs Field Transforms for Optical Flow

Award: ECCV 2020 Best Paper

Authors: Zachary Teed, Jia Deng (Princeton University)

There are two major hindrances in optical flow: processing speed and hunger for computational resources. The authors propose RAFT architecture comprises three modules, i.e., two encoder modules which generate a 4-dimensional volume by using width and height of two frames and an updater module that comprises GRUs and updates the information of the optical flow. The RAFT architecture varies from the other state-of-the-art models in 4 ways, including architecture, training, reusability of weights, and optimization algorithm. The simulation results show that RAFT achieves F1 error score of 5.1% on KITTI. [Paper Link]

Watch the paper explanatory video:

‍

5. Learning Latent Representations to Influence Multi-Agent Interaction

Award: CoRL 2020 Best Paper

Authors: Annie Xie, Dylan P. Losey, Ryan Tolsma, Chelsea Finn, Dorsa Sadigh (Stanford University, Virginia Tech)

Modeling the interactions between robots is a complex task. The literature focuses on modeling each low-level action of the robot. The authors propose a reinforcement learning-based framework for predicting the latent representation of the ego agent. An encoder is trained on previous interactions, which estimates the next strategy. The decoder takes this strategy and predicts the next interaction. The ego agent then learns a policy accustomed on the predicted strategy. The influencing behavior in the ego agent is achieved by maximizing the long-term rewards in interactions. The simulation results reveal that this method outperforms other techniques. [Paper Link]

Watch the conference talk video:

‍

6. Language Models are Few-Shot Learners

Award: NeurIPS 2020 Best Paper

Authors: Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, Dario Amodei (OpenAI)

The non-trivial lingual tendencies start appearing when the model is scaled up to 100 billion parameters. Researchers from OpenAI propose GPT-3, an auto-regressive language model with 175 billion parameters. It is observed that as the models are scaled up, task agnostic and few-shot performance is greatly improved. The GPT-3 is much more general and has the capability to generate webpage layouts, translating a complex legal text in plain English or performing the other way around, typesetting mathematical equations from plain English, and generating properly structured different types of visual plots on the basis of plain English prompts. [Paper Link]

Watch the explanatory video:

‍

7. Preference-Based Learning for Exoskeleton Gait Optimization

Award: ICRA 2020 Best Paper

Authors: Tucker, Maegan; Novoseller, Ellen; Kann, Claudia; Sui, Yanan; Yue, Yisong; Burdick, Joel; Ames, Aaron (CalTech & Tsinghua University)

Authors present a preference-based learning framework for personalized exoskeleton gait optimization. The proposed work takes its motivation from compass-gait biped. Contrary to other methods, which optimize various numerical objectives, such as the mechanical cost of transport, the proposed technique tries to learn from the preference of the user, such as comfort. In order to implement the preference-based learning, the authors present a new algorithm called COSPAR. This algorithm inspires the user to choose pair-wise preferences between trials. The COSPAR algorithm is also applied to lower-body exoskeleton for optimizing the human walking trajectory features which shows its effectiveness. [Paper Link]

Watch the conference talk video:

‍

8. Progressive Layered Extraction (PLE): A Novel Multi-Task Learning (MTL) Model for Personalized Recommendations

Award: RecSys 2020 Best Paper

Authors: Hongyan Tang; Junning Liu; Ming Zhao; Xudong Gong (Tencent PCG)

In multi-task learning (MTL) systems, the performance of one task is usually achieved at the cost of performance degradation for another task. This is known as seesaw phenomenon. The authors propose progressive layer extraction (PLE) which has a novel sharing architecture. The PLE has the ability to differentiate task distinct components and shared components overtly. This is accomplished by incorporating a multi-layered gate control. On a gradual basis, the PLE extracts and divides the semantic knowledge by using a progressive routing mechanism.  The proposed model is then applied to complicatedly correlated and normally correlated tasks. [Paper Link]

Watch the conference talk video:

‍

9. Towards a Human-like Open-Domain Chatbot

Authors: Daniel Adiwardana, Minh-Thang Luong, David R. So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, Quoc V. Le (Google)

The Google research team introduces Meena, a 2.5B parameter, multi-turn, human-like, and open-domain chatbot. Meena is trained end-to-end on the conversational data extracted from social media platforms. It is a seq2seq model trained using the recall-first technique with the major aim to minimize the perplexity of the next generated token. The authors also present a new evaluation metric - sensibleness and specificity average (SSA) - to encapsulate the essential characteristics of human-like multi-turn conversation. The results show that Meena outperforms existing chatbots by 23% in absolute SSA and is getting closer to human performance. [Paper Link]

Watch the research talk video below:

‍

10. Fourier Neural Operator for Parametric Partial Differential Equations

Authors: Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, Anima Anandkumar (CalTech & Purdue University)

Partial differential equations (PDE) can describe everything from planetary motion to plate tectonics, but they’re extremely difficult to solve. In this paper, the authors present a technique that numerically solves the PDEs with efficiency, creating great potential to speed up many scientific applications. They propose a new neural operator by parameterizing the integral kernel directly in the Fourier space. The architecture of the proposed method comprises a Fourier layer, which is discretization invariant and is able to learn and evaluate the functions discretized in a random fashion. This method demonstrates state-of-the-art performance compared to existing neural network methodologies and is up to three orders of magnitude faster compared to traditional PDE solvers. [Paper Link]

Watch the explanatory video below:

‍

Check out our collection for more impactful AI research talks!