Adaptive Contextual Processing of Structured Data by Recursive Neural Networks: A Survey of Computational Properties

In this section, the capacity of statistical machine learning techniques for recursive structure processing is investigated. While the universal approximation capability of recurrent and recursive networks for sequence and tree processing is well established, recent extensions to so-called contextual models have not yet been investigated in depth. Contextual models have been proposed to process acyclic graph structures. They rely on a restriction of the recurrence of standard models with respect to children of vertices as occurs e.g. in cascade correlation. This restriction allows to introduce recurrence with respect to parents of vertices without getting cyclic definitions. These models have very successfully been applied to various problems in computational chemistry. In this section, the principled information which can be processed in such a way and the approximation capabilities of realizations of this principle by means of neural networks are investigated.