A hybrid physics-informed neural network based multiscale solver as a partial differential equation constrained optimization problem
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 17 | |
| Author | ||
| License | CC Attribution 3.0 Germany: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/68251 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
Leibniz MMS Days 202413 / 17
16
17
00:00
Computer animation
00:14
Mass flow rateLiquidVelocityStokes' theoremScale (map)Time domainAlgebraic structureComplex (psychology)Model theoryRandbedingung <Mathematik>ApproximationPartial differential equationPoint (geometry)Residual (numerical analysis)Partial differential equationNichtlineares GleichungssystemCategory of beingOptimale KontrolleSequenceVector spaceSpacetimeParameter (computer programming)Scaling (geometry)Differential equationModel theoryMass flow rateRule of inferenceProjective planeMultiplicationFunctional (mathematics)Process (computing)Partial differential equationBoundary value problemSet theorySocial classTime domainResultantInsertion lossPhysicalismMereologyTerm (mathematics)ModulformPartial differential equationThermal conductivityMathematical optimizationDimensional analysisStrategy gameApproximationWave packetState of matterGroup actionInfinityVariable (mathematics)MathematicsOptimization problemObject (grammar)Complex (psychology)GeometryCartesian coordinate systemPoint (geometry)Feasibility studyTheory of relativityHeat transferFluidPhysical lawAlgebraic structureElement (mathematics)Residual (numerical analysis)Materialization (paranormal)AverageStatisticsPerspective (visual)Volume (thermodynamics)Einbettung <Mathematik>Image resolutionMultiplication signFunction (mathematics)Derivation (linguistics)Axiom of choiceINTEGRALComputer animation
09:02
SequenceCategory of beingOptimale KontrolleScale (map)SpacetimeVector spaceProbability density functionMathematical optimizationGradientPartial differential equationStandard deviationApproximationFinite element methodIterationPoint (geometry)Nichtlineares GleichungssystemRandbedingung <Mathematik>Stokes' theoremScaling (geometry)Nichtlineares GleichungssystemGeometryResultantWave packetWell-formed formulaPhysicalismModulformCoefficientInsertion lossLimit (category theory)Mathematical optimizationConstraint (mathematics)Numerical analysisDifferential equationMaxima and minimaGradientExistenceUniverse (mathematics)Strategy gameTerm (mathematics)Classical physicsThermal conductivityFinite element methodSpacetimeFehlerschrankeFunktionenalgebraDirection (geometry)Partial differential equationRoutingConsistencyTheoremSocial classObject (grammar)Category of beingEnergy levelPartial differential equationOptimization problemApproximationOperator (mathematics)Regular graphFluidProjective planeCalculus of variationsStandard deviationDegrees of freedom (physics and chemistry)Algebraic structureInfinityFree groupMeasurementMass flow rateParameter (computer programming)Cartesian coordinate systemLine (geometry)Residual (numerical analysis)Connectivity (graph theory)Point (geometry)Standard errorAverageFrequencyComputer animation
17:50
Computer animationLecture/Conference
Transcript: English(auto-generated)
00:05
Now I'm not trying to pronounce this title one more time and yes, this is the joint work with Michael Hintermüller from WIES as a part of ongoing project machine learning for simulation intelligence and composite material design between multiple partners including IVW and
00:26
What IVW approached us with what kind of problem so in particular We would like to simulate the flow through a composite material and this is typically a resin flow which you have to use for a composite material reinforcement and
00:44
Then in order to simulate such a flow you need to know the permeability of a material So this is a characterization of a material and in order to resolve it Due to the complex geometries you have to go for multiple scales. So first you go to a micro scale So this is the smallest resolution of your geometry and then on this
01:05
resolution typically you have Very slow simulations due to complicated geometries and very fine meshes and then on such a scale you need to solve the stocks problem and The results of these simulations you have to use further to compute with permeability for the meso scale
01:24
Where you further use stocks Brinkman flow to compute the permeability for the macro scale so there is a multi-scale bridging in this setting and for this multi-scale bridging you need to know permeability of a material and
01:42
In order to figure out what kind of bridging we are doing So first of all, we are using the upscaling technique and this is the technique from the numerical homogenization Which is applied for heterogeneous materials so you have a fine scale domain, which is typically taken as a Statistical volume element and then on this fine scale domain you have to solve
02:05
In 3d you need to solve three stocks problems in 2d. You need to solve two stocks problems respectively and then from these simulations You can use the constitutive material relation And for fluid flows, this is the Darcy's law to determine the permeability for the meso scale
02:26
So this is a very general process for fluid flows We use Darcy's law, but if you are interested in the heat conduction processes, you have to use for real of heat conduction So yeah, this is a very general process which is used in engineering applications
02:41
And in particular if you want to now do that, what kind of challenges do you have? so first of all you have Complex fine-scale geometries and On these geometries you need to do repetitive and expensive computations in order to understand the geometric variability on the micro scale and
03:04
In particular also if you are dealing with fine-scale problems with micro scale problems the quality of models itself is Questionable so this is a rather philosophical issue because we use the stocks flow But the question is how accurate is the stock flow to describe the phenomena on the micro scale?
03:25
And then in order to resolve all these challenges we came up with a so-called hybrid strategy on The fine scale we would like to use physics and form neural networks to compute the solution of our problem and then
03:41
The solution from physics and form neural networks the use as an input for the core scale domain through this upscaling process and What is the novelty is that this core scale solution then further informs a fine scale Optimization problem which comes from the physics and form neural networks
04:02
So we do not only have belief from the micro scale to the meso scale but we also couple all two scales in our hybrid strategy and First of all I would like to give a very brief primer on physics and form neural networks and yeah, so this physics informed parts terms from idea that
04:24
Yeah, you take your physics or more precisely your understanding of physics in the form of PDE model and then This PDE is embedded into the L2 loss so you try to minimize the PDE residual and the PDE boundary residual in the least square sense and
04:46
as a Set over which you want to minimize is where you use the neural network class of suitable Structure and what is suitable I will explain a bit later
05:01
And in this neural network class you have neural network functions Which are parameterized by the parameters and these parameters you would like to determine for the optimization task So you take this objective Yeah, so you take this l2 objective and in order to Do this optimization in practice you have to apply the quadrature rule of choice
05:24
For example, you can use Monte Carlo to discretize all the integrals and then you do a following optimization so you have this allocation points which term from Monte Carlo discretization and this are the inputs of your neural network and as an output you parameterize your PDE solution and then in order to build such a residual you have to compute all the derivatives
05:46
Which are necessary to construct your differential operator and for this to use the algorithmic differentiation and then you perform the optimization on that when you perform the optimization update Unless the residual is small enough. So this is the way to solve the PDEs
06:02
Using neural networks and now what we are doing, so we take our physics and form neural network objective yeah, and then We constrain it by the coarse-scale PDEs of a fine-scale problem We use a pin approach and this pin approach is now constrained by the coarse-scale partial differential equation
06:25
This coarse-scale partial differential equation is informed by our neural network and that's through web scaling process And at the same time the solution enters the pin objective Where we construct a very special term to make this enrichment possible
06:41
And in particular what we are doing We take our neural network and that's and then we do the local averaging of this fine-scale problem and we try to bring this Average neural network to the coarse-scale solution So in essence you are trying to minimize the mismatched between the averaged fine-scale solution and the coarse-scale solution
07:03
Yeah, and this idea from the comorganization perspective It comes from the embedding weak information, weak convergence information into the objective Yeah, and what is the rationale behind is that pins are meshless Which makes them suitable for complex geometries and coarse-scale PDEs are inexpensive to solve which makes this
07:22
hybrid strategy Computationally feasible and the hope now is that this scale coupling improves overall approximation approximation on the fine scale and hopefully potentially when the approximation in the coarse-scale and Yeah, there are also some benefits of neural networks in particular transfer learning which makes them very well suitable for repetitive computations
07:47
Meaning that first of all, you can solve your problem on one geometry but then you can take the parameters which you obtain from this solve to initialize your training on another geometry, which is quite similar and then typically the optimization is much faster and
08:05
Secondly pins can be data-driven which means that you can not only embed your PDE Into the objective but also data which could further correct your micro scale problem and to refine your physics Yeah, and at Weierstrass, I am from the group on
08:24
which specializes on the design of optimization algorithms in infinite dimensional spaces and Just to get the taste of our work. I also would like to give some mathematical details Yeah, and then inspired by the optimal control in optimal control. People are typically interested in control to state maps and
08:45
In our optimization process We are rather interested in the so-called fine to coarse-scale maps because our fine-scale problem now acts as a control and the coarse-scale problem acts as a state and of course the question is and
09:01
Yeah in our context with map fine to coarse-scale map is in essence the composition of our upscaling process which we use to obtain the efficient material properties with a solution operator on the coarse-scale level and the question of course when this map is well defined and first of all You need to be sufficiently close to your truth fine-scale solution because if you're not close
09:25
Then you will get rubbish on the coarse scale which will give you a very bad approximation of the coarse-scale problem which will not be solvable and Secondly, you also need to have good neural networks in the vicinity of your fine-scale solution
09:41
Because you use a neural network ansatz and This is guaranteed by the so-called UA property, which is the abbreviation for universal approximation So you need an appropriate an appropriate universal approximation theorem Which guarantees that your PDE solution could be approximated by neural network to arbitrary precision
10:02
which is not a trivial question in the context of PDEs and Network classes sufficiently large you can well define this fine to coarse-scale map and if you can do that You can simply reduce your PDE constraint optimization problem to a standard optimization problem
10:22
which now depends only on a fine-scale problem, which resembles now the pin structure and Apart from other challenges. We have to deal with minimization over Non-linear spaces and then if you want like to deal with such problems by direct method of calculus of variations
10:41
to show the existence of minimizers is not directly applicable and When you have to go for quasi minimizers Which allows you not to minimize your problem exactly, but rather to find an approximation Of your
11:01
optimal solution and This is a relatively big concept which requires only the infinum existence for your problem and what we can show is that you could approach the infinum as close as possible if the number of Neurons in your neural networks goes to infinity and if you have such fins quasi minimization universal approximation properties and a few other technical assumptions and then what you could show is that your
11:26
hybrid loss In the limit of number of neurons going to infinity can be bounded by our regularization term which we constructed to embed the core scale information and the approximation error in the limit Also bounded by this regularization term and what you could further show is that if epsilon going to zero
11:46
Then this term could be made very very small well, this is nice, but the problem here is that You have an approximation error Which is bounded by the term which is in the loss and
12:00
the loss you use for training and In practice this term must never be zero but never neural networks were a little bit dumb and they will try to make it zero and This will lead to a core scale overfitting and it will Generally mitigate the route to convergence on the fine scale
12:20
nevertheless we propose some upscale and consistent strategies which help to reduce these problems in learning or at least to understand them qualitatively and then if you go for optimization now on the fine scale we use a neural network and that's on the core scale we use the finite elements and that's and
12:44
You hope that the degrees of freedom in your finite element discretization is not that high so that you could solve your problem repetitively And then you have to compute the gradient of objective. Yeah, if you have it you could feed it in your Optimizer of choice, yeah typically in the Adam optimizer, but if you have a better optimizer you can use it based on your taste and
13:04
If you spins, yeah, this is typically done using algorithmic differentiation in particular The second term is a classical term from pin but now they also have some Additional contribution which comes from the constraints and this Contributions this contribution can be computed using the joint approach. So in particular you need to solve a joint equation, which is a PDE and
13:28
This a joint equation requires the knowledge of a core scale solution So now it means that in order to compute the gradient You need to do the additional two additional solves of partial differential equations using finite element method
13:41
But once you have it you have a gradient for optimization and then I Would like to illustrate the concept on when America example and here we have a fine scale equation which is given by the equation of heat conduction with a highly oscillatory coefficient and this highly oscillatory coefficient embeds the information on
14:03
your complex geometry and then in the framework of upscaling you could then compute the formula for the upscaled coefficient, which you use for the core scale equation now Let's look at the numerical results. So in particular
14:24
One problem with physics and form neural networks is that if you use standard feed-forward neural network to approximate a multiscale problem Then the neural network is prone to the so-called low frequency BS it means that it first learns low frequencies and only when high frequencies which makes the training very slow and
14:44
Now in our hybrid strategy We basically embed the core scale information the slow frequencies into objective directly as data By telling to neural network like hey, this is our core scale component which you have to approximate you don't need to approximate it from the PD residual and
15:04
If you do that then so first of all We get our fine scale approximation using the using the hybrid approach and then we compare it with the reference finite element solution by getting relatively small point wise errors in the solution itself in the gradient
15:21
then we get our hybrid core scale solution and this term you get from Doing the local averaging of your fine scale solution. So this is in essence What you are doing in your new regularization term, which is introduced in our Hebrew strategy You are trying to minimize the mismatch between this
15:41
this guy and this guy and Now if you do that We compare it with a standard pin approach and here the red line is the PD residual loss for the pin and the blue line is the PD residual loss for the Hebrew approach and As you can see it decays immediately much faster if you embed core scale information and
16:07
This would result in the faster convergence In this with respect to approximation error So this is a very strong argument to embed the core scale information in training multi-scale problems
16:20
Which isn't generally quite challenging if you would like to apply pins Yeah, and further on of course, we would like to extend this approach to free scales So to build a free scale solver and possibly introduce the corrections to when the reliant physics using measurement data so this is the next step and Yeah as a conclusion what we have developed we have developed a hybrid solver which combines both neural networks and
16:47
Conventional numerical solvers and this feet hybrid framework is cast into a PD constraint optimization using multi-scale ideas when the overall problem is studied in the function space framework and What we could see is that the hybrid framework improves learning with pins on a fine scale and when bedding core scale information
17:06
Helps mitigate for low frequency BS of neural networks possible directions for future extending this ideas to fluid flow equations and porous media and industrial application or at least the scaling which is the still then going project with our ML for SIM partners and
17:24
Then of course to improve improve pins to improve optimization algorithms But rather the question to the audience as well as that can our multi-scale techniques be used within the proposed framework And can be developed coupling ideas between the scales be used for our ML based problem problems
17:44
Yeah, thank you for your attention