Introduction

Graphs are ubiquitous in real-world, covering a variety of applications ranging from social networks, recommender systems, knowledge graphs, computer vision, and drug discovery. To analyze graphs, an important prerequisite is to have effective representations of graphs, which largely determine the performance of most downstreaming tasks. In this course, I will introduce the latest progress on learning representations of graphs such as node representation learning, graph visualization, knowledge graph embedding, graph neural networks, graph generation and their applications to a variety of tasks.

Topics

Part I: Machine learning & deep learning preliminary

• Machine Learning Basics
• Feedforward Neural Networks
• Convolutional Neural Networks
• Recurrent Neural Networks

Part II: Fundamental

• Graph representation learning overview & unsupervised node embeddings
• Graph layout and high-dimensional visualization
• Knowledge graph embedding
• Graph neural networks
• Deep generative models for graph generation
• Adversarial attacks on graphs
• Graph representation learning systems

Part III: Applications

• Natural language understanding
• Computer vision
• Drug discovery
• Combinatorial optimization

Evaluation

• Class participation
• Presentation
• Kaggle Competition

Schedule

• Day 1 AM: Introduction & Machine Learning Basics [Jian]
• Day 1 PM: Feedforward Neural Networks & Convolutional Neural Networks [Jian]
• Day 2 AM: Recurrent Neural Networks [Jian]
• Day 2 PM: Graph representation learning overview & unsupervised node embeddings [Jian]
• Day 3 AM: Graph layout and high-dimensional visualization
• Day 3 PM: Knowledge graph embedding
• Day 4 AM: Graph neural networks
• Day 4 PM: Deep generative models for graph generation [Jian] & Adversarial attacks on graphs
• Day 5 AM: Graph representation learning system
• Day 5 PM: Graph representation learning applications

References

1. Books
   • Christopher Bishop. Pattern Recognition and Machine Learning.
   • Ian Goodfellow and Yoshua Bengio and Aaron Courville. Deep Learning.
2. Tutorials
   • Graph representation learning. William L. Hamilton and Jian Tang. Tutorial at AAAI’19.
   • Geometric deep learning.
3. Unsupervised node embeddings
   • DeepWalk: Online Learning of Social Representations.
   • LINE: Large-scale Information Network Embedding.
   • node2vec: Scalable Feature Learning for Networks
4. Graph layout and high-dimensional visualization
   • Accelerating t-SNE using Tree-Based Algorithms.
   • Visualizing Large-scale and High-dimensional Data
   • UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction.
5. Knowledge graph embedding
   • [TransE] Translating Embeddings for Modeling Multi-relational Data.
   • [ComplexE] Complex Embeddings for Simple Link Prediction
   • [ConvE] Convolutional 2D Knowledge Graph Embeddings
   • [DisMult] Embedding Entities and Relations for Learning and Inference in Knowledge Bases
   • [RotatE] RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space
6. Graph convolutional neural networks
   • [scarselli et al. 2009]The graph neural network model
   • [GCN] Semi-Supervised Classification with Graph Convolutional Networks
   • [GraphSAGE] Inductive Representation Learning on Large Graphs
   • [GAT] Graph Attention Networks
   • [Neural message passing] Neural Message Passing for Quantum Chemistry
7. Graph representation learning system [GraphVite] GraphVite: A High-Performance CPU-GPU Hybrid System for Node Embedding [Aligraph] AliGraph: A Comprehensive Graph Neural Network Platform [DGL] https://www.dgl.ai/
8. Adversarial attacks on graphs
   • Adversarial Attacks on Node Embeddings
   • Data Poisoning Attack against Unsupervised Node Embedding Methods
   • Adversarial Attack on Graph Structured Data
   • Adversarial Attacks on Neural Networks for Graph Data
   • Adversarial Attacks on Graph Neural Networks via Meta Learning
   • CHARACTERIZING MALICIOUS EDGES TARGETING ON GRAPH NEURAL NETWORKS
9. Graph Generation
   • GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models
   • NetGAN: Generating Graphs via Random Walks
   • Junction Tree Variational Autoencoder for Molecular Graph Generation
   • Generative Modeling for Protein Structures

10. Graph representation learning for natural language understanding

11. Graph representation learning for computer vision
    • Beyond Grids: Learning Graph Representations for Visual Recognition

12. Graph representation learning for drug discovery

13. Graph neural networks for combinatorial optimization
    • Combinatorial Optimization with Graph Convolutional Networks and Guided Tree Search
14. Graph neural networks for reinforcement learning
    • NerveNet: Learning Structured Policy with Graph Neural Networks
    • Graph Networks as Learnable Physics Engines for Inference and Control