Beyond the Additive Nodes' Convolutions: a Study on High-Order Multiplicative Integration

Graph Convolutional Neural Networks (GCNs) compute representations of graph nodes by exploiting convolution operators based on some neighborhood aggregation scheme. These operators are defined by using several stacked Graph Convolutional (GC) layers. They are usually defined as additive building blocks that fuse multiple information streams. However, when considering information integration in sequences, the flow of gradient has been shown to be more robust by adopting the Multiplicative Integration (MI) technique. Because of that, it is worth investigating the impact of MI in Graph Neural Networks. We propose three different GC layers that exploit MI to improve various aspects of the neighborhood aggregation scheme. We report both a theoretical and empirical comparison of our proposals with respect to the most common GC operators for the graph classification task.