Abstract
We consider deterministic mean field games where the dynamics of a typical agent is non-linear with respect to the state variable and affine with respect to the control variable. Particular instances of the problem considered here are mean field games with control on the acceleration (see Achdou et al. in NoDEA Nonlinear Differ Equ Appl 27(3):33, 2020; Cannarsa and Mendico in Minimax Theory Appl 5(2):221-250, 2020; Cardaliaguet and Mendico in Nonlinear Anal 203: 112185, 2021). We focus our attention on the approximation of such mean field games by analogous problems in discrete time and finite state space which fall in the framework of (Gomes et al. in J Math Pures Appl (9) 93(3):308-328, 2010). For these approximations, we show the existence and, under an additional monotonicity assumption, uniqueness of solutions. In our main result, we establish the convergence of equilibria of the discrete mean field games problems towards equilibria of the continuous one. Finally, we provide some numerical results for two MFG problems. In the first one, the dynamics of a typical player is nonlinear with respect to the state and, in the second one, a typical player controls its acceleration.
Similar content being viewed by others
Notes
Consider three sequences \((p_n)_{n\in \mathbb {N}}\subset [0,\infty )\), \((q_n)_{n\in \mathbb {N}}\subset \mathbb {R}\), and \((r_n)_{n\in \mathbb {N}}\subset \mathbb {R}\) such that
$$\begin{aligned} r_{n+1} \le p_{n+1} r_{n} + q_{n+1}\quad \text {for all }n\in \mathbb {N}. \end{aligned}$$Then, setting \(\texttt{P}_{n}= \prod _{j=1}^{n} p_j\) and \(\texttt{P}_{k,n}= \prod _{j=k}^{n} p_j\), we have
$$\begin{aligned} r_{n}\le \texttt{P}_{n}r_0+\sum _{k=1}^{n} \texttt{P}_{k,n} q_k\quad \text {for all }n\in \mathbb {N}. \end{aligned}$$
References
Y. Achdou, F. Camilli, and Italo Capuzzo-Dolcetta. Mean field games: convergence of a finite difference method. SIAM J. Numer. Anal., 51(5):2585–2612, 2013.
Y. Achdou and I. Capuzzo-Dolcetta. Mean field games: numerical methods. SIAM J. Numer. Anal., 48(3):1136–1162, 2010.
Y. Achdou, P. Cardaliaguet, F. Delarue, A. Porretta, and F. Santambrogio. Mean field games, volume 2281 of Lecture Notes in Mathematics. Springer, 2020. Notes from the CIME School held in Cetraro, June 2019, Edited by Cardaliaguet and Porretta, Fondazione CIME/CIME Foundation Subseries.
Y. Achdou and M. Laurière. Mean field games and applications: numerical aspects. In Mean field games, volume 2281 of Lecture Notes in Math., pages 249–307. Springer, 2020.
Y. Achdou, P. Mannucci, C. Marchi, and N. Tchou. Deterministic mean field games with control on the acceleration. NoDEA Nonlinear Differential Equations Appl., 27(3):33 2020.
L. Ambrosio, N. Gigli, and G. Savare. Gradient Flows: In Metric Spaces and in the Space of Probability Measures. Lectures in Mathematics. ETH Zürich. Birkhäuser Basel, 2008.
M. Bardi and I. Capuzzo-Dolcetta. Optimal control and viscosity solutions of Hamilton-Jacobi-Bellman equations. Birkhäuser, Boston, 1997.
G. Barles. Solutions de viscosité des équations de Hamilton-Jacobi. Springer-Verlag, Paris, 1994.
J.-D. Benamou, G. Carlier, and F. Santambrogio. Variational mean field games. In Active particles. Vol. 1. Advances in theory, models, and applications, Model. Simul. Sci. Eng. Technol., pages 141–171. Birkhäuser/Springer, 2017.
A. Bensoussan, J. Frehse, and P. Yam. Mean field games and mean field type control theory. Springer Briefs in Mathematics. Springer, New York, 2013.
C. Bertucci and A. Cecchin. Mean field games master equations: from discrete to continuous state space. Preprint, arXiv:2207.0319, 2022.
G. W. Brown. Iterative solution of games by fictitious play. Activity analysis of production and allocation, 13(1):374–376, 1951.
F. Camilli and F. J. Silva. A semi-discrete approximation for a first order mean field game problem. Netw. Heterog. Media, 7(2):263–277, 2012.
P. Cannarsa and R. Capuani. Existence and uniqueness for mean field games with state constraints. In PDE models for multi-agent phenomena, volume 28 of Springer INdAM Ser., pages 49–71. Springer, 2018.
P. Cannarsa, R. Capuani, and P. Cardaliaguet. Mean field games with state constraints: from mild to pointwise solutions of the PDE system. Calc. Var. Partial Differential Equations, 60(3):Paper No. 108, 33, 2021.
P. Cannarsa and C. Mendico. Mild and weak solutions of mean field game problems for linear control systems. Minimax Theory Appl., 5(2):221–250, 2020.
P. Cannarsa and C. Sinestrari. Semiconcave functions, Hamilton-Jacobi equations, and optimal control, volume 58 of Progress in Nonlinear Differential Equations and their Applications. Birkhäuser Boston, Inc., Boston, MA, 2004.
P. Cardaliaguet. Notes on Mean Field Games: from P.-L. Lions’ lectures at Collège de France. Lecture Notes given at Tor Vergata, 2010.
P. Cardaliaguet and S. Hadikhanloo. Learning in mean field games: the fictitious play. ESAIM Control Optim. Calc. Var., 23(2):569–591, 2017.
P. Cardaliaguet and C. Mendico. Ergodic behavior of control and mean field games problems depending on acceleration. Nonlinear Anal., 203:112185, 2021.
P. Cardaliaguet, A. Mészáros, and F. Santambrogio. First order mean field games with density constraints: Pressure equals price. SIAM Journal on Control and Optimization, 54(5):2672–2709, 2016.
E. Carlini and F. J. Silva. A fully discrete semi-Lagrangian scheme for a first order mean field game problem. SIAM J. Numer. Anal., 52(1):45–67, 2014.
E. Carlini and F. J. Silva. On the discretization of some nonlinear Fokker-Planck-Kolmogorov equations and applications. SIAM J. Numer. Anal., 56(4):2148–2177, 2018.
R. Carmona and F. Delarue. Probabilistic theory of mean field games with applications. I, volume 83 of Probability Theory and Stochastic Modelling. Springer, 2018. Mean field FBSDEs, control, and games.
R. Carmona and F. Delarue. Probabilistic theory of mean field games with applications. II, volume 84 of Probability Theory and Stochastic Modelling. Springer, 2018. Mean field games with common noise and master equations.
I. Chowdhury, O. Ersland, and E.R. Jakobsen. On numerical approximations of fractional and nonlocal mean field games. Found Comput Math, 23:1381–1431, 2023.
F. Da Lio and O. Ley. Convex Hamilton-Jacobi equations under superlinear growth conditions on data. Appl. Math. Optim., 63(3):309–339, 2011.
B. Dacorogna. Direct Methods in the Calculus of Variations. Springer-Verlag, Berlin, Heidelberg, 1989.
S. Dweik and G. Mazanti. Sharp semi-concavity in a non-autonomous control problem and \(L^p\) estimates in an optimal-exit MFG. NoDEA Nonlinear Differential Equations Appl., 27(2):Paper No. 11, 59, 2020.
M. Falcone and R. Ferretti. Semi-Lagrangian approximation schemes for linear and Hamilton-Jacobi equations. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2014.
M. Fischer and F. J. Silva. On the asymptotic nature of first order mean field games. Appl. Math. Optim., 84(2):2327–2357, 2021.
D. Fudenberg and D. K. Levine. The theory of learning in games, volume 2 of MIT Press Series on Economic Learning and Social Evolution. MIT Press, Cambridge, MA, 1998.
D. A. Gomes, J. Mohr, and R. R. Souza. Discrete time, finite state space mean field games. J. Math. Pures Appl. (9), 93(3):308–328, 2010.
D. A. Gomes, E. A. Pimentel, and V. Voskanyan. Regularity theory for mean-field game systems. SpringerBriefs in Mathematics. Springer, 2016.
D. A. Gomes and J. Saúde. Mean field games models—a brief survey. Dyn. Games Appl., 4(2):110–154, 2014.
O. Guéant, J.-M. Lasry, and P.-L. Lions. Mean field games and applications. In Paris-Princeton Lectures on Mathematical Finance 2010, volume 2003 of Lecture Notes in Math., pages 205–266. Springer, Berlin, 2011.
S. Hadikhanloo. Learning in Mean Field Games. PhD thesis, Université, France, 2018.
S. Hadikhanloo and F. J. Silva. Finite mean field games: fictitious play and convergence to a first order continuous mean field game. J. Math. Pures Appl. (9), 132:369–397, 2019.
M. Huang, R. P. Malhamé, and P. E. Caines. Large population stochastic dynamic games: closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle. Commun. Inf. Syst., 6(3):221–251, 2006.
J.-M. Lasry and P.-L. Lions. Jeux à champ moyen. I. Le cas stationnaire. C. R. Math. Acad. Sci. Paris, 343(9):619–625, 2006.
J.-M. Lasry and P.-L. Lions. Jeux à champ moyen. II. Horizon fini et contrôle optimal. C. R. Math. Acad. Sci. Paris, 343(10):679–684, 2006.
J.-M. Lasry and P.-L. Lions. Mean field games. Jpn. J. Math., 2(1):229–260, 2007.
R. M. Lewis and R. B. Vinter. Relaxation of optimal control problems to equivalent convex programs. J. Math. Anal. Appl., 74(2):475–493, 1980.
S. Liu, M. Jacobs, W. Li, L. Nurbekyan, and S. J. Osher. Computational methods for first-order nonlocal mean field games with applications. SIAM J. Numer. Anal., 59(5):2639–2668, 2021.
P. Mannucci, C. Marchi, C. Mariconda, and N. Tchou. Non-coercive first order Mean Field Games. J. Differential Equations, 269(5):4503–4543, 2020.
P. Mannucci, C. Marchi, and N. Tchou. First order mean field games in the Heisenberg group: periodic and non periodic case. Preprint, arXiv:2010.09279, 2021.
P. Mannucci, C. Marchi, and N. Tchou. Non coercive unbounded first order mean field games: the Heisenberg example. J. Differential Equations, 309:809–840, 2022.
G. Mazanti and F. Santambrogio. Minimal-time mean field games. Math. Models Methods Appl. Sci., 29(8):1413–1464, 2019.
L. Nurbekyan and J. Saúde. Fourier approximation methods for first-order nonlocal mean-field games. Port. Math., 75(3-4):367–396, 2018.
R. B. Vinter and R. M. Lewis. The equivalence of strong and weak formulations for certain problems in optimal control. SIAM J. Control Optim., 16(4):546–570, 1978.
J. Warga. Optimal control of differential and functional equations. Academic Press, New York-London, 1972.
Acknowledgements
F. J. Silva was partially supported by l’Agence Nationale de la Recherche (ANR), project ANR-22-CE40-0010, and by KAUST through the subaward agreements OSR-2017-CRG6-3452.04 and ORA-2021-CRG10-4674.6.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Martin Burger.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix I
In this section we prove some technical properties of the semi-discrete value function \(v_{k}:\mathbb {R}^d\rightarrow \mathbb {R}\) (\(k\in \mathcal {I}\)) introduced in Sect. 3.1. In what follows we assume that (H1), (H2) and (H3) are in force.
We start showing the existence of optimal controls for the problem defined by \(v_k\) in (3.7).
Lemma 6.2
Given \(k \in \mathcal {I}^{*}\) and \(x\in \mathbb {R}^d\), there exists \({\bar{\alpha }}\in \mathcal {A}_{k}\) such that \(v_k( x)= J_{k, x}({\bar{\alpha }})\). In addition, there exists \(\tilde{C}>0\), independent of \(\Delta t\), m, k, and x, such that any optimal solution \({{\tilde{\alpha }}}\in \mathcal {A}_k\) satisfies
Proof
Given \(k \in \mathcal {I}^{*}\) and \(x\in \mathbb {R}^d\), let \((\alpha ^n)_{n\in \mathbb {N}}\subset \mathcal {A}_k\) be a minimizing sequence for \(J_{k,x}\). Let \((\gamma ^n)_{n\in \mathbb {N}}\subset \Gamma _{k,x}\) be the sequence of states associated with \((\alpha ^n)_{n\in \mathbb {N}}\) via (3.6) and let \({\bar{\gamma }}^0\in \Gamma _{k,x}\) be the state associated with the null control. By definition of minimizing sequence, (H1)(i), and (H2), for any \(\delta >0\), there exists \(n_0\in \mathbb {N}\) such that
for all \(n\ge n_0\). Let us estimate \(\left| {\bar{\gamma }}^0_{N_t} - \gamma ^n_{N_t}\right| \). For every \(j=k, \dots , N_t-1\) we have
Since \({\bar{\gamma }}^0_{k} =\gamma ^n_{k}\), by the discrete Grönwall’s lemma, there exists \(C>0\) such that
Thus, by Young’s inequality, for every \(\eta >0\) there exists \(C_\eta >0\) such that
Taking \(\eta <\underline{\ell }\) and combining the above equation with (6.8) and (6.9), we deduce the existence of \(\tilde{C}>0\), independent of \(\Delta t\), m, k, and x, such that
Therefore, there exists at least one accumulation point \({\bar{\alpha }}\in \mathcal {A}_k\) of \((\alpha ^n)_{n\in \mathbb {N}}\) and, by the continuity assumptions in (H1)–(H3), we conclude that \(v_k(x)= J_{k, x}({\bar{\alpha }})\). Finally, if \({\tilde{\alpha }}\) is any other optimal control for the problem defined by \(v_k(x)\), the previous argument shows that (6.7) holds. \(\square \)
Proof of the Lipschitz property (3.9)
Given \(k \in \mathcal {I}^{*}\) and \(x\in \mathbb {R}^d\), let \({\bar{\alpha }}\in \mathcal {A}_k\) be an optimal control for \(v_k(x)\) and let \({\bar{\gamma }}\in \Gamma _{k,x}\) be the associated state given by (3.6). Let \(y\in \mathbb {R}^d\) and let \(\zeta \in \Gamma _{k,y}\) be the state associated with \({\bar{\alpha }}\) via (3.6), then (H1)(ii) implies
On the other hand, for every \(j=k,\dots , N_t-1\),
By the discrete Grönwall’s lemma, we have
By (6.7), Hölder’s inequality, and (6.10), there exists \(L_{v}>0\), independent of x, y, such that \(v_k(x)-v_k(y)\le L_v |x-y|\) for all x, \(y\in \mathbb {R}^d\), which implies (3.9). \(\square \)
Lemma 6.3
Given \(k \in \mathcal {I}^*\) and \(x\in \mathbb {R}^d\), there exists \(\alpha _{k,x}\in \mathbb {R}^r\) such that
Moreover, there exists \(\widehat{C}>0\), independent of \(\Delta t\), m, k, and x, such that
Proof
Let \(\bar{\alpha }\) be as in Lemma 6.2. Then, by the dynamic programming principle, \(\alpha _{k,x}=\bar{\alpha }_k\) satisfies (6.11) and hence
Thus, by (3.9), (H1)(i), and (H3)(ii), we have
and the existence of \(\widehat{C}>0\) such that (6.12) holds follows from Young’s inequality. \(\square \)
Proof of the Proposition 3.1
Notice that (H1)(i), with \(a=0\), and (H2) imply that
where \({\bar{\gamma }}^{0,n}\) is defined by (3.6) with \(\alpha _j=0\) for all \(j=k, \ldots , N_t^n-1\) and \(\bar{\gamma }^{0,n}_k=x\). By the definition of \({\bar{\gamma }}^{0,n}\), Remark 2.1(i), and the discrete Grönwall’s lemma, there exists \(C>0\), independent of n, k, and x, such that
which, together with (6.13), implies that \(v^n\) is locally bounded, uniformly with respect to n. Therefore, we can define \(v^*:[0,T]\times \mathbb {R}^d \rightarrow \mathbb {R}\) and \(v_{*}:[0,T]\times \mathbb {R}^d \rightarrow \mathbb {R}\) by
for all \((t,x)\in [0,T]\times \mathbb {R}^d\). By [7, Chapter V, Lemma 1.5] we have that \(v^*\) and \(v_*\) are upper and lower semicontinuous, respectively. We claim that
(a) \(v^*(T,x)=v_*(T,x)=g(x,m(T))\) for all \(x\in \mathbb {R}^d\).
(b) \(v^*\) satisfies, in the viscosity sense (see e.g. [7, Chapter II]),
(c) \(v_*\) satisfies, in the viscosity sense,
If (a), (b), and (c) hold, then by the comparison principle [27, Theorem 2.1] we obtain that \(v^*=v_*=v\) and the result follows from [7, Chapter V, Lemma 1.9]. It remains to prove the claim. The proofs of (b) and (c) being analogous, we only show (a) and (b).
Proof of (a). Let \(x\in \mathbb {R}^d\) and \((k(n), x^n)\in \mathcal {I}^n\times \mathbb {R}^d\) be such that \((t^n_{k(n)}, x^n)\rightarrow (T,x)\) as \(n\rightarrow \infty \). Denote by \({\widehat{\mathcal {A}}}_{k}^n\) the set defined in (3.5) for \(k=0,\ldots ,N^n_t-1\). By Lemma 6.2 and Lemma 6.3, there exists \(\alpha ^n\in {\widehat{\mathcal {A}}}_{k(n)}^n\) such that
where \(\gamma ^n\) is the state associated with \(\alpha ^n\) via (3.6) and \(\gamma ^n_{k(n)}=x^n\). By (H1)(i) we obtain
By (H3) and Lemma 6.3 we have
On the other hand, arguing as in the proof of (6.14), there exists \(C>0\), independent of n, such that
which, together with (6.17) and the boundeness of \((x^{n})_{n\in \mathbb {N}}\), yields
and hence
The result follows from the previous equation and (6.16).
Proof of (b). To check that (6.15) holds in the viscosity sense, following the lines of [38, Proposition 4.3], let \(\phi \in C^1\left( [0,T]\times \mathbb {R}^d\right) \) and \((t^*, x^*)\in (0,T)\times \mathbb {R}^d\) be such that \(v^*-\phi \) has a local maximum in \((t^*, x^*)\). Modifying \(\phi \), if necessary, we can assume that \((t^*, x^*)\) is a strict maximum for \(v^*-\phi \) on \(\overline{\textrm{B}}((t^*, x^*), \delta )\), for some \(\delta >0\). Therefore, by [7, Chapter V, Lemma 1.6] there exists a sequence \(((k(n), x^n))_{n\in \mathbb {N}} \subset \mathcal {I}^{n,*}\times \mathbb {R}^d\) such that \((t^n_{k(n)}, x^n)\rightarrow (t^*, x^*)\), \(v^n_{k(n)}(x^n)\rightarrow v^*(t^*, x^*)\), and
By (6.18), (6.13), (H2), (6.14), and modifying \(\phi \) outside \(\overline{\textrm{B}}((t^*, x^*), \delta )\), if necessary, we can assume that \(\phi \in C^1\left( [0,T]\times \mathbb {R}^d\right) \), \(\partial _t \phi \) and \(\nabla \phi \) are bounded, and \((k(n), x^n)\) is a global maximum of \(\mathcal {I}^n \times \mathbb {R}^d \ni (k,x) \mapsto v^n_{k}(x)-\phi (x)\in \mathbb {R}\). In particular, for all \(y\in \mathbb {R}^d\), we have
Using (3.8), we obtain
Since \(\nabla \phi \) is bounded, (H1)(i) implies that the last infimum above is reached at some \({\bar{\alpha }}^n\in \mathbb {R}^r\) and there exists \(C_{\phi }>0\), independent of n, such that \(|{\bar{\alpha }}^n|\le C_\phi \). Therefore, for any \(\alpha \in \mathbb {R}^r\) we have
Since the sequence \(({\bar{\alpha }}^n)_{n\in \mathbb {N}}\) is bounded by \(C_\phi \), there exists a subsequence, still denoted by \(({\bar{\alpha }}^n)_{n\in \mathbb {N}}\), and \(\alpha ^*\in \overline{\textrm{B}}(0, C_\phi )\) such that \({\bar{\alpha }}^n\rightarrow \alpha ^*\) as \(n\rightarrow \infty \). Passing to the limit in the previous inequality we obtain
from which we deduce that
Finally, by (6.19), we obtain
which proves assertion (b). \(\square \)
Appendix II
Proof of Theorem 2.2
As in Sect. 3.2 we assume that B and A are decomposed as in (3.15) and (3.16), respectively, with \(B_1(t,x)\in \mathbb {R}^{r\times r}\) being invertible for all \((t,x)\in [0,T]\times \mathbb {R}^d\). Let \(\xi ^{1}\) and \(\xi ^{2}\) be two solutions to Problem 2.1 and, for \(i=1,2\), define \(m^i\in C([0,T];\mathcal {P}_1(\mathbb {R}^d))\) as \(m^{i}(t)=e_{t}\sharp \xi ^{i}\) for all \(t\in [0,T]\). Given \(x\in \mathbb {R}^d\) and \(i=1,2\), let us set
Observe that, by Remark 2.1(iii), if \(\gamma \in \text {Opt}^{i}(x)\), then there exists a unique control \(\alpha [\gamma ]\in L^{p}([0,T];\mathbb {R}^r)\), given by (5.24), such that \((\gamma ,\alpha [\gamma ])\) solves \(({OC_{x,m^i}})\). Let \(\mathbb {R}^d\ni x\mapsto \gamma ^{i,x} \in \Gamma \) be a Borel measurable selection of the set-valued map \(\text {Opt}^{i}\). The existence of such a selection follows from exactly the same arguments than those in the proof of [31, Lemma 2.1]. Since \(\xi ^i\) solves Problem 2.1, we have \(\textrm{supp}(\xi ^i)\subseteq \cup _{x\in \textrm{supp}(m_0)}\text {Opt}^{i}(x)\) and, hence, condition (2.4) implies that \(\xi ^i=\gamma ^i\sharp m_0\).
Let us define \(J^i:W^{1,p}([0,T];\mathbb {R}^d)\rightarrow \mathbb {R}\) as
and notice that, by (H5), we have
If \(\xi ^1\ne \xi ^2\), then (2.4) implies that
Using that \(\xi ^i=\gamma ^i\sharp m_0\), the previous condition can be rewritten as
Analogously, we obtain
Combining (7.2) and (7.3), we obtain a contradiction with (7.1). \(\square \)
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Gianatti, J., Silva, F.J. Approximation of Deterministic Mean Field Games with Control-Affine Dynamics. Found Comput Math (2023). https://doi.org/10.1007/s10208-023-09629-4
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10208-023-09629-4
Keywords
- Deterministic mean field games
- Control-affine dynamics
- Lagrangian equilibrium
- Approximation of equilibria
- Convergence results
- Numerical experiences