Workshop
Bridge Between Perception and Reasoning: Graph Neural Networks & Beyond
Jian Tang · Le Song · Jure Leskovec · Renjie Liao · Yujia Li · Sanja Fidler · Richard Zemel · Ruslan Salakhutdinov
Keywords: Graph Neural Networks Logic Reasoning Perception and Reasoning Integration
Deep learning has achieved great success in a variety of tasks such as recognizing objects in images, predicting the sentiment of sentences, or image/speech synthesis by training on a large-amount of data. However, most existing success are mainly focusing on perceptual tasks, which is also known as System I intelligence. In real world, many complicated tasks, such as autonomous driving, public policy decision making, and multi-hop question answering, require understanding the relationship between high-level variables in the data to perform logical reasoning, which is known as System II intelligence. Integrating system I and II intelligence lies in the core of artificial intelligence and machine learning.
Graph is an important structure for System II intelligence, with the universal representation ability to capture the relationship between different variables, and support interpretability, causality, and transferability / inductive generalization. Traditional logic and symbolic reasoning over graphs has relied on methods and tools which are very different from deep learning models, such Prolog language, SMT solvers, constrained optimization and discrete algorithms. Is such a methodology separation between System I and System II intelligence necessary? How to build a flexible, effective and efficient bridge to smoothly connect these two systems, and create higher order artificial intelligence?
Graph neural networks, have emerged as the tool of choice for graph representation learning, which has led to impressive progress in many classification and regression problems such as chemical synthesis, 3D-vision, recommender systems and social network analysis. However, prediction and classification tasks can be very different from logic/symbolic reasoning.
Bits and pieces of evidence can be gleaned from recent literature, suggesting graph neural networks may be a general tool to make such a connection. For example, \cite{battaglia2018relational,barcelo2019logical} viewed graph neural networks as tools to incorporate explicitly logic reasoning bias. \cite{kipf2018neural} used graph neural network to reason about interacting systems,
\cite{yoon2018inference,zhang2020efficient} used neural networks for logic and probabilistic inference, \cite{hudson2019learning, hu2019language} used graph neural networks for reasoning on scene graphs for visual question reasoning, \cite{qu2019probabilistic} studied reasoning on knowledge graphs with graph neural networks, and \cite{khalil2017learning, xu2018powerful, velickovic2019neural, sato2019approximation} used graph neural networks for discrete graph algorithms. However, there can still be a long way to go for a satisfactory and definite answers on the ability of graph neural networks for automatically discovering logic rules, and conducting long-range multi-step complex reasoning in combination with perception inputs such as language, vision, spatial and temporal variation.
{\bf Can graph neural networks be the key bridge to connect System I and System II intelligence? Are there other more flexible, effective and efficient alternatives?} For instance, \citep{wang2019satnet} combined max satisfiability solver with deep learning, \citep{manhaeve2018deepproblog} combined directed graphical and Problog with deep learning, \citep{skryagin2020splog}~combined sum product network with deep learning, \citep{silver2019few,alet2019graph}~combined logic reasoning with reinforcement learning. How do these alternative methods compare with graph neural networks for being a bridge?
The goal of this workshop is to bring researchers from previously separate fields, such as deep learning, logic/symbolic reasoning, statistical relational learning, and graph algorithms, into a common roof to discuss this potential interface and integration between System I and System intelligence. By providing a venue for the confluence of new advances in theoretical foundations, models and algorithms, as well as empirical discoveries, new benchmarks and impactful applications,
Schedule
Sat 5:50 a.m. - 6:00 a.m.
|
Opening Remarks: Jian Tang & Le Song
(
Opening
)
>
SlidesLive Video |
Jian Tang · Le Song 🔗 |
Sat 6:00 a.m. - 6:30 a.m.
|
Keynote: Yoshua Bengio
(
Invited Talk
)
>
SlidesLive Video |
Yoshua Bengio 🔗 |
Sat 6:30 a.m. - 6:40 a.m.
|
Keynote: Yoshua Bengio (Q&A)
(
Q&A
)
>
|
Yoshua Bengio 🔗 |
Sat 6:40 a.m. - 7:10 a.m.
|
Invited Talk: Peter Battaglia
(
Invited Talk
)
>
SlidesLive Video |
Peter Battaglia 🔗 |
Sat 7:10 a.m. - 7:20 a.m.
|
Invited Talk: Peter Battaglia (Q&A)
(
Q&A
)
>
|
Peter Battaglia 🔗 |
Sat 7:20 a.m. - 7:25 a.m.
|
Spotlight Talk (1): Generating Programmatic Referring Expressions via Program Synthesis
(
Spotlight Talk
)
>
SlidesLive Video |
Jiani Huang 🔗 |
Sat 7:25 a.m. - 7:30 a.m.
|
Spotlight Talk (2): Closed Loop Neural-Symbolic Learning via Integrating Neural Perception, Grammar Parsing, and Symbolic Reasoning
(
Spotlight Talk
)
>
SlidesLive Video |
Qing Li 🔗 |
Sat 7:30 a.m. - 7:35 a.m.
|
Spotlight Talk (3): Beta Embeddings for Multi-Hop Logical Reasoning in Knowledge Graphs
(
Spotlight Talk
)
>
SlidesLive Video |
Hongyu Ren 🔗 |
Sat 7:35 a.m. - 7:40 a.m.
|
Spotlight Talk (4): Barking up the right tree: an approach to search over molecule synthesis DAGs
(
Spotlight Talk
)
>
SlidesLive Video |
John Bradshaw 🔗 |
Sat 7:40 a.m. - 8:30 a.m.
|
Morning Poster Session
(
Poster
)
>
Jinghan Shi et al. Heterogeneous Graph Neural Network for Recommendation. https://us02web.zoom.us/j/82916453037?pwd=SGlwN3hnM0RKaGJUZ0lZSHA1Kzh5QT09 (Breakout Room 1) Yangyang Hu et al. Enhancing Neural Mathematical Reasoning by Abductive Combination with Symbolic Library https://us02web.zoom.us/j/82916453037?pwd=SGlwN3hnM0RKaGJUZ0lZSHA1Kzh5QT09 (Breakout Room 2) Binbin Hu et al. KGNN: Distributed Framework for Graph Neural Knowledge Representation. https://us02web.zoom.us/j/82916453037?pwd=SGlwN3hnM0RKaGJUZ0lZSHA1Kzh5QT09 (Breakout Room 3) Hao Tang et al.. Towards Scale-Invariant Graph-related Problem Solving by Iterative Homogeneous Graph Neural Networks. https://us02web.zoom.us/j/82916453037?pwd=SGlwN3hnM0RKaGJUZ0lZSHA1Kzh5QT09 (Breakout Room 4) Giuseppe Futia et al. Modeling the semantics of data sources with graph neural networks https://us02web.zoom.us/j/82916453037?pwd=SGlwN3hnM0RKaGJUZ0lZSHA1Kzh5QT09 (Breakout Room 5) Laetitia Teodorescu et al. SpatialSim: Recognizing Spatial Configurations of Objects with Graph Neural Networks. https://us02web.zoom.us/j/82916453037?pwd=SGlwN3hnM0RKaGJUZ0lZSHA1Kzh5QT09 (Breakout Room 6) Binghong Chen et al. Learning Retrosynthetic Planning with Chemical Reasoning https://us02web.zoom.us/j/82916453037?pwd=SGlwN3hnM0RKaGJUZ0lZSHA1Kzh5QT09 (Breakout Room 7) Maxwell Crouse et al. Neural Analogical Matching.https://us02web.zoom.us/j/82916453037?pwd=SGlwN3hnM0RKaGJUZ0lZSHA1Kzh5QT09 (Breakout Room 8) Yuta Kawachi et al. End-to-end permutation learning with Hungarian algorithm. https://us02web.zoom.us/j/82916453037?pwd=SGlwN3hnM0RKaGJUZ0lZSHA1Kzh5QT09 (Breakout Room 9) Mikołaj Sacha et al. Molecule Edit Graph Attention Network: Modeling Chemical Reactions as Sequences of Graph Edits. https://us02web.zoom.us/j/82916453037?pwd=SGlwN3hnM0RKaGJUZ0lZSHA1Kzh5QT09 (Breakout Room 10) Aleksandar Stanic et al.. Hierarchical Relational Inference. https://us02web.zoom.us/j/83436973845?https://us02web.zoom.us/j/82916453037?pwd=SGlwN3hnM0RKaGJUZ0lZSHA1Kzh5QT09 (Breakout Room 11) |
🔗 |
Sat 8:30 a.m. - 9:00 a.m.
|
Invited Talk: Zico Kolter
(
Invited Talk
)
>
|
Zico Kolter 🔗 |
Sat 9:00 a.m. - 9:10 a.m.
|
Invited Talk: Zico Kolter (Q&A)
(
Q&A
)
>
|
Zico Kolter 🔗 |
Sat 10:30 a.m. - 11:00 a.m.
|
Invited Talk: Tommi Jaakkola
(
Invited Talk
)
>
|
Tommi Jaakkola 🔗 |
Sat 11:00 a.m. - 11:10 a.m.
|
Invited Talk: Tommi Jaakkola (Q&A)
(
Q&A
)
>
|
🔗 |
Sat 11:10 a.m. - 11:40 a.m.
|
Invited Talk 4: Luc De Raedt
(
Invited Talk
)
>
SlidesLive Video |
Luc De Raedt 🔗 |
Sat 11:40 a.m. - 11:50 a.m.
|
Invited Talk 4: Luc De Raedt (Q&A)
(
Q&A
)
>
|
Luc De Raedt 🔗 |
Sat 11:50 a.m. - 11:55 a.m.
|
Spotlight talk (5): Modeling the semantics of data sources with graph neural networks
(
Spotlight Talk
)
>
SlidesLive Video |
Giuseppe Futia 🔗 |
Sat 11:55 a.m. - 12:00 p.m.
|
Spotlight talk (6): SpatialSim: Recognizing Spatial Configurations of Objects with Graph Neural Networks
(
Spotlight Talk
)
>
SlidesLive Video |
Laetitia Teodorescu 🔗 |
Sat 12:00 p.m. - 12:05 p.m.
|
Spotlight Talk (7): Enhancing Neural Mathematical Reasoning by Abductive Combination with Symbolic Library
(
Spotlight Talk
)
>
|
yangyang hu 🔗 |
Sat 12:05 p.m. - 12:10 p.m.
|
Spotlight Talk (8): Learning Retrosynthetic Planning with Chemical Reasoning
(
Spotlight Talk
)
>
SlidesLive Video |
Binghong Chen 🔗 |
Sat 12:10 p.m. - 1:00 p.m.
|
Afternoon Poster Session
(
Poster
)
>
Arseny Skryagin et al.. Sum-Product Logic: Integrating Probabilistic Circuits into DeepProbLog. https://us02web.zoom.us/j/82921478440?pwd=VWZSZm1HTWRENEJpQytSL0w5RXdKUT09 (Breakout room 1) Maria Leonor Pacheco et al.. Neural-Symbolic Modeling for Natural Language Discourse. https://us02web.zoom.us/j/82921478440?pwd=VWZSZm1HTWRENEJpQytSL0w5RXdKUT09 (Breakout room 2) Richard Li et al.. Towards Practical Multi-Object Manipulation using Relational Reinforcement Learning. https://us02web.zoom.us/j/82921478440?pwd=VWZSZm1HTWRENEJpQytSL0w5RXdKUT09 (Breakout room 3) Nichita Uțiu et al.. Performance Evaluation of Graph Convolutional Networks with Siamese Training for Few-Shot Classification of Nodes. https://us02web.zoom.us/j/82921478440?pwd=VWZSZm1HTWRENEJpQytSL0w5RXdKUT09 (Breakout room 4) Rajat Koner et al. Scenes and Surroundings: Scene Graph Generation using Relation Transformer. https://us02web.zoom.us/j/82921478440?pwd=VWZSZm1HTWRENEJpQytSL0w5RXdKUT09 (Breakout room 5) Meng Qu et al. RNNLogic: Learning Logic Rules for Reasoning on Knowledge Graphs. https://us02web.zoom.us/j/82921478440?pwd=VWZSZm1HTWRENEJpQytSL0w5RXdKUT09 (Breakout room 6) Aditya Mogadala et al. Sparse Graph to Sequence Learning for Vision Conditioned Long Textual Sequence Generation https://us02web.zoom.us/j/82921478440?pwd=VWZSZm1HTWRENEJpQytSL0w5RXdKUT09 (Breakout room 7) Xinshi Chen et al. Understanding Deep Learning with Reasoning Layer https://us02web.zoom.us/j/82921478440?pwd=VWZSZm1HTWRENEJpQytSL0w5RXdKUT09 (Breakout room 8) Jiani Huang et al. Generating Programmatic Referring Expressions via Program Synthesis. https://us02web.zoom.us/j/82921478440?pwd=VWZSZm1HTWRENEJpQytSL0w5RXdKUT09 (Breakout room 9) Qing Li et al. Closed Loop Neural-Symbolic Learning via Integrating Neural Perception, Grammar Parsing, and Symbolic Reasoning https://us02web.zoom.us/j/82921478440?pwd=VWZSZm1HTWRENEJpQytSL0w5RXdKUT09 (Breakout room 10) Hongyu Ren et al. Beta Embeddings for Multi-Hop Logical Reasoning in Knowledge Graphs. https://us02web.zoom.us/j/82921478440?pwd=VWZSZm1HTWRENEJpQytSL0w5RXdKUT09 (Breakout room 11) John Bradshaw et al. Barking up the right tree: an approach to search over molecule synthesis DAGs https://us02web.zoom.us/j/82921478440?pwd=VWZSZm1HTWRENEJpQytSL0w5RXdKUT09 (Breakout room 12) |
🔗 |
Sat 1:00 p.m. - 1:30 p.m.
|
Invited Talk 5:Ferran Alet
(
Invited Talk
)
>
SlidesLive Video |
Ferran Alet 🔗 |
Sat 1:30 p.m. - 1:40 p.m.
|
Invited Talk 5:Ferran Alet (Q&A)
(
Q&A
)
>
|
Ferran Alet 🔗 |
Sat 1:40 p.m. - 2:10 p.m.
|
Invited Talk 6: Kristian Kersting
(
Invited Talk
)
>
|
Kristian Kersting 🔗 |
Sat 2:10 p.m. - 2:20 p.m.
|
Invited Talk 6: Kristian Kersting (Q&A)
(
Q&A
)
>
|
Kristian Kersting 🔗 |
Sat 2:20 p.m. - 2:30 p.m.
|
Concluding Remarks
(
Conclusion
)
>
|
🔗 |