Ensemble physics informed neural networks: A framework to improve inverse transport modeling in heterogeneous domains

Modeling fluid flow and transport in heterogeneous systems is often challenged by unknown parameters that vary in space. In inverse modeling, measurement data are used to estimate these parameters. Due to the spatial variability of these unknown parameters in heterogeneous systems (e.g., permeability or diffusivity), the inverse problem is ill-posed and infinite solutions are possible. Physics-informed neural networks (PINN) have become a popular approach for solving inverse problems. However, in inverse problems in heterogeneous systems, PINN can be sensitive to hyperparameters and can produce unrealistic patterns. Motivated by the concept of ensemble learning and variance reduction in machine learning, we propose an ensemble PINN (ePINN) approach where an ensemble of parallel neural networks is used and each sub-network is initialized with a meaningful pattern of the unknown parameter. Subsequently, these parallel networks provide a basis that is fed into a main neural network that is trained using PINN. It is shown that an appropriately selected set of patterns can guide PINN in producing more realistic results that are relevant to the problem of interest. To assess the accuracy of this approach, inverse transport problems involving unknown heat conductivity, porous media permeability, and velocity vector fields were studied. The proposed ePINN approach was shown to increase the accuracy in inverse problems and mitigate the challenges associated with non-uniqueness.