Evolutionary Training of CLP-Constrained Neural Networks

In this paper we study how global optimization methods (like genetic algorithms) can be used to train neural networks. These methods are useful when local (for example gradient-based methods) do not work well. We introduce the notion of regularity for studying properties of the error function. If regularities are present in the error function, then they expand the search space in an artificial way. Regularities are used to generate constraints on the weights of the network. By the introduction of constraints we avoid the expansion of the search space. The main idea is then to consider the training of the network as a constrained optimization problem. Often there are other constraints on the weights of the network (for example domain constraints, shared weights). In order to find a satisfiable set of constraints we use a constraint logic programming system. We also relate the notion of regularity to so-called network transformations.