- Research
- Open Access
An exact viscosity solution to a Hamilton–Jacobi–Bellman quasi-variational inequality for animal population management
- Yuta Yaegashi^{1, 2},
- Hidekazu Yoshioka^{3}Email authorView ORCID ID profile,
- Kentaro Tsugihashi^{3} and
- Masayuki Fujihara^{1}
https://doi.org/10.1186/s13362-019-0062-y
© The Author(s) 2019
- Received: 9 October 2018
- Accepted: 19 June 2019
- Published: 28 June 2019
Abstract
We formulate a stochastic impulse control model for animal population management and a candidate of exact solutions to a Hamilton–Jacobi–Bellman quasi-variational inequality. This model has a qualitatively different functional form of the performance index from the existing monotone ones. So far, optimality and unique solvability of the Hamilton–Jacobi–Bellman quasi-variational inequality has not been investigated, which are thus addressed in this paper. We present a candidate of exact solutions to the Hamilton–Jacobi–Bellman quasi-variational inequality and prove its optimality and unique solvability within a certain class of solutions in a viscosity sense. We also present and examine a dynamical system-based numerical method for computing coefficients in the exact solutions.
Keywords
- Population management
- Threshold control
- Hamilton–Jacobi–Bellman quasi-variational inequalities
- Viscosity solution
- Feeding damage
- Uniqueness and existence of solution
- Optimality of control
1 Introduction
This paper focuses on mathematical analysis of an exact viscosity solution to Hamilton–Jacobi–Bellman quasi-variational inequality arising in an animal population management problem. Our problem, despite it is relatively simple, is important from mathematical, environmental and ecological engineering standpoints since many management problems can be described and/or analyzed with mathematical tools focused on in our paper. Our results are mathematical ones but providing a background of the problem is important for understanding it. Hence, we firstly describe the problem background in this section.
Management of animal population, such as fishery resources and their predators, is an important ecological problem. Such examples include aquaculture of Plecoglossus altivelis (P. altivelis, Ayu) in Japan [1], extermination of its predator bird Phalacrocorax carbo (P. carbo, Great cormorant) [2, 3], agricultural crops damage by a wild boar Sus scrofa in Europe [4], and feeding damage from many insects to soybean seeds [5]. Recently, fish-eating bird P. carbo population has been increasing worldwide, such as in Japan [6], Europe [7], North America [8], and Greenland [9]. The increase of the population causes several problems. In Japan, feeding damage from fish-eating birds, such as P. carbo, to inland fishery resources, has been increasing and is currently one of the most severe problems to be solved [6]. Cost-effective and ecologically-sound bird population management policy is required for effective reduction of the feeding damage.
Stochastic optimal control theory [10] has been applied to population and resource management problems [1, 11–14]. In reality, there are fixed costs besides proportional costs when some interventions are performed for management of animal population. Stochastic impulse control theory serves as an effective mathematical tool for dealing with this issue [14–17] and has been applied to many problems, such as finance and economics [18–20], and animal population management by the authors [21].
Finding an optimal control policy in the context of stochastic impulse control reduces to solving a Hamilton–Jacobi–Bellman quasi-variational inequality (HJBQVI), a degenerate elliptic or parabolic differential inequality. Many researchers have investigated mathematical properties of HJBQVIs. In Cadenillas [17], the performance index is quadratic monomial (convex). He proposed a candidate of the exact solution to the HJBQVI and proved a verification theorem and that the exact solution satisfies the HJBQVI. Existence and uniqueness of exact solutions has not been proved. In Ohnishi and Tsujimura [15] and Øksendal [21], the performance index is quadratic monomial (convex). They proposed a candidate of the exact solution to the HJBQVI and proved a verification theorem, existence and uniqueness of the exact solution, and that it satisfies the HJBQVI. Note that in Ohnishi and Tsujimura [15], the cost function is quadratic unlike those of Cadenillas [17], Øksendal [21], and this paper. In addition, viscosity solutions are appropriate weak solutions for degenerate elliptic and parabolic equations [22–24], and are appropriate solutions to HJBQVIs.
Recently, Yaegashi et al. [3] proposed an optimal control model for cost-effective and sustainable management of P. carbo and a candidate of its exact viscosity solution to a HJBQVI. This model has a different performance index from those in the above-mentioned literature. However, existence and uniqueness of the solution, and its optimality namely the verification, have not been discussed so far. If we could give an answer on the above-mentioned problems about the exact solution, the solution and the associated optimal policy can establish a firm position as a reasonable mathematical tool. This is the motivation of this paper.
The objectives of this paper are thus to formulate the stochastic impulse control model of an animal population management, to present a candidate of the exact solutions to a HJBQVI, and to prove its optimality and unique solvability within a certain class of solutions from a viscosity viewpoint. The novelty of this paper against the previous studies from Cadenillas [17], Ohnishi and Tsujimura [15], and Øksendal [21] is the point that our performance index is based on a concave polynomial as a sum of two monomials. Our performance index is therefore not based on monotone functions as in the above-mentioned models. We also propose and examine a numerical method to compute the coefficients of the exact solution.
The rest of this paper consists of 4 Sections and 3 Appendices. Section 2 introduces our model. Section 3 provides the exact solution. Section 4 concerns mathematical analysis of the exact solution. Section 5 presents and examines the numerical method. Section 6 concludes this paper. Appendix A contains the proofs of lemmas in Sects. 2 and 4. Appendix B contains the proofs of Theorems 4.2 and 4.3. Finally, Appendix C presents the proof of Theorem 5.1.
2 Mathematical model: our method
Our aim is mathematical and numerical analysis on a recent population management problem with an emphasis of viscosity solutions. This section presents our mathematical approach and derives basic properties of the present mathematical model.
2.1 Population dynamics
2.2 Performance index
2.3 Hamilton–Jacobi–Bellman quasi-variational inequality
Lemma 2.1
Proofs of all lemmas and theorems in the paper are in Appendices for the sake of brevity of the main body. In what follows, solutions to the HJBQVI complying with the condition (15) are explored.
2.4 Optimal control
- (A)
If \(X_{t -} < \bar{x}\), then no intervention is performed. If \(X_{t -} = \bar{x}\), the intervention is immediately performed and \(X_{t -} \) is reduced to \(\underline{x}\) (\(X_{t} = \underline{x}\)).
- (B)
If \(X_{0 -} > \bar{x}\), then \(X_{0 -} \) is immediately reduced to \(\underline{x}\) (\(X_{0} = \underline{x}\)) by the intervention, and follows (A).
3 Results and discussion on an exact solution
- (a)
Continuity of \(\bar{V} ( x )\) at \(x = \bar{x}\) (Value matching),
- (b)
Continuity of \(\bar{V}' ( x )\) at \(x = \bar{x}\) (Smooth pasting),
- (c)
Optimality of the thresholds x̄ and \(\underline{x}\) in (14).
For a, we have the following lemma, which immediately follows from the functional form of the exact solution (30) and Lemma 2.1. The following lemma is useful for determining the sign of the unknown coefficient involved in V̄.
Lemma 3.1
For a in (30), we have \(a \ge 0\).
4 Discussion on the system of nonlinear equations
4.1 Existence and uniqueness
In this subsection, we prove unique solvability of the system of nonlinear Eqs. (35) based on an analytical approach, which is inspired from the arguments in the literatures [15, 21]. Because our performance index contains a sum of two monomials of X, which is different from those in the literature, their procedure cannot be directly applied to our problem. The main objective of this section is to prove the following theorem that guarantees unique existence of the triplet of the unknown coefficients, with which we can completely determine V̄.
Theorem 4.1
There exists a unique triplet \(( a,\bar{x},\underline{x} )\) to the system of nonlinear equations (35).
Theorem 4.1 is proved in a step by step approach using a series of lemmas. Lemma 4.1 sharpens the result of Lemma 3.1.
Lemma 4.1
For a in (30), we have \(a \ne 0\).
Lemma 4.2
\(g' ( x ) = 0\) has a unique zero \(x = \hat{x}\) such that \(0 < \hat{x} < \infty\).
Hereafter, we denote the unique x̂ in Lemma 4.2 as \(\hat{x} = \hat{x} ( a )\) to indicate its dependence on a. We show dependence of \(\hat{x} ( a )\) on a in the next lemma. The lemma indicates a monotone property of \(\hat{x} ( a )\).
Lemma 4.3
Lemma 4.4
Profile of \(g ( x )\)
x | 0 | x̂ | ∞ |
---|---|---|---|
\(g' ( x )\) | −∞ | 0 | ∞ |
g(x) | ∞ | \(g ( \hat{x} ) = a\beta \frac{m - \beta}{m - 1}\hat{x}^{\beta - 1} + A_{1}M\frac{m - M}{m - 1}\hat{x}^{M - 1} + k_{1}\) | ∞ |
In the next lemma, we show dependence of \(\underline{x} ( a )\) and \(\bar{x} ( a )\) on a. They are monotone with respect to a.
Lemma 4.5
The next lemma shows that this upper bound in fact exists.
Lemma 4.6
There exists an upper bound of a with which \(\underline{x}\) and x̄ exist.
Hereafter, we represent the range of a as \(0 < a < \hat{a} ( k_{1} )\) to indicate its dependence on \(k_{1}\). Now, we prove that the pended inequality (40) is satisfied without any additional conditions.
Lemma 4.7
The inequality (40) is satisfied without any additional conditions to \(k_{1}\). Thus, there exist unique x̄ and \(\underline{x}\) which solve the system of nonlinear equations (35).
Now, we can prove uniqueness and existence of a.
Lemma 4.8
There exists a unique coefficient a solving the system of nonlinear equations (35) at least if \(k_{0}\) is sufficiently small.
We prove that the uniqueness and existence hold true also for not small \(k_{0}\).
Lemma 4.9
There exists a unique \(a^{ *} \) such that \(F ( a^{ *} ) = 0\) and \(0 < a^{ *} < \hat{a}\) solving (35). Here, \(F ( a )\) is the left hand side of (98).
Finally, from Lemmas 4.7 and 4.9, Theorem 4.1 immediately follows.
4.2 Optimality of the exact solution
The following technical lemma is on the profile of V̄, necessary to consider property of V̄.
Lemma 4.10
With the help of Lemma 4.10, we show that the candidate solution actually satisfies the HJBQVI (12).
Next, we prove a mathematical property of the exact solution (30). We prove that the exact solution (30) is a viscosity solution [10], which is an appropriate weak solution to degenerate elliptic and parabolic differential equations. The definition of viscosity solution is as follows.
Definition 4.1
Viscosity super-solution
Viscosity sub-solution
Viscosity solution
A function \(V \in C [ 0,\infty )\) such that \(V ( 0 ) = 0\) satisfying (15) is a viscosity solution to the HJBQVI (12) if it is a viscosity super-solution as well as a viscosity sub-solution.
Viscosity property of the constructed exact solution is checked through Definition 4.1.
From Theorems 4.1, 4.2, and 4.3, we can prove the following theorem, the most important theorem in this paper.
5 Numerical method for computing the coefficients
In the previous section, we showed that the coefficients \(( a,\underline{x},\bar{x} )\) are found uniquely; however, their exact values cannot be calculated analytically. We found that a dynamical system-based approach can be used for approximating the coefficients in a stable manner. The numerical method is presented and its convergence is analyzed, and its performance is examined in this section.
5.1 Dynamical system
5.2 Stability of the equilibrium point
Based on the following easy but pivotal lemma, local stability of the equilibrium point \(( \bar{x}^{ *},\underline{x}^{ *} )\) is investigated.
Lemma 5.1
By Lemma 5.1, the following theorem shows local stability of our dynamical system, supporting our approach to find the unknown coefficients through solving the dynamical system.
Theorem 5.1
The equilibrium point \(( \bar{x}^{ *},\underline{x}^{ *} )\) in (52) is locally asymptotically stable.
5.3 Numerical experiment
Time evolution of the error, the residuals and the computed thresholds
t (×10^{7}) | Error | Res 1 | Res 2 | Res 3 | x̄ | \(\underline{x}\) |
---|---|---|---|---|---|---|
0.0 | 1.6 × 10^{−1} | 1.6 × 10^{1} | 1.3 × 10^{−2} | 1.3 × 10^{−2} | 7500 | 5500 |
1.0 | 5.7 × 10^{−6} | 5.7 × 10^{−4} | 5.3 × 10^{−4} | 5.3 × 10^{−4} | 7376.14 | 5735.92 |
2.0 | 5.0 × 10^{−7} | 5.0 × 10^{−5} | 4.7 × 10^{−5} | 4.7 × 10^{−5} | 7354.98 | 5755.88 |
3.0 | 4.4 × 10^{−8} | 4.4 × 10^{−6} | 4.2 × 10^{−6} | 4.2 × 10^{−6} | 7353.11 | 5757.65 |
4.0 | 3.9 × 10^{−9} | 3.9 × 10^{−7} | 3.7 × 10^{−7} | 3.7 × 10^{−7} | 7352.95 | 5757.8 |
5.0 | 3.5 × 10^{−10} | 3.5 × 10^{−8} | 3.3 × 10^{−8} | 3.3 × 10^{−8} | 7352.93 | 5757.82 |
5.5 | 9.9 × 10^{−11} | 1.0 × 10^{−10} | 9.5 × 10^{−9} | 9.5 × 10^{−9} | 7352.93 | 5757.82 |
6 Conclusions
This paper formulated the impulse control model of an animal population management, presented a candidate of the exact solutions of the HJBQVI, and proved its optimality and unique solvability within a certain class of solutions from a viscosity viewpoint. A numerical method to compute the coefficients for the exact solution to the HJBQVI was also presented and examined. Our future research will address analysis of time-dependent counterpart of the presented impulse control model. Such an extension would be necessary for dealing with seasonal population dynamics.
Declarations
Acknowledgements
JSPS Research Grant No. 17K15345 and No. 17J09125 support this research.
Availability of data and materials
Not applicable.
Funding
JSPS Research Grant No. 17K15345 and No. 17J09125 support this research.
Authors’ contributions
YY and HY carried out the mathematical analysis. YY carried out the numerical computation. YY, HY, and KT wrote the paper. MF supervised and improved the manuscript. All authors read and approved the final manuscript.
Competing interests
The authors declare that they have no competing interests.
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Authors’ Affiliations
References
- Yoshioka H, Yaegashi Y. Stochastic optimization model of aquacultured fish for sale and ecological education. J Math Ind. 2017;7:8. MathSciNetView ArticleGoogle Scholar
- Yaegashi Y, Yoshioka H, Unami K, Fujihara M. A singular stochastic control model for sustainable population management of the fish-eating waterfowl Phalacrocorax carbo. J Environ Manag. 2018;219:18–27. View ArticleGoogle Scholar
- Yaegashi Y, Yoshioka H, Unami K, Fujihara M. Impulse and singular stochastic control approaches for management of fish-eating bird population. In: New trends in emerging complex real life problems. Berlin: Springer; 2018. p. 493–500. View ArticleGoogle Scholar
- Schley L, Dufrêne M, Krier A, Frantz AC. Patterns of crop damage by wild boar (Sus scrofa) in Luxembourg over a 10-year period. European Journal of Wildlife Research. 2008;54:589. View ArticleGoogle Scholar
- Turnipseed SG, Kogan M. Soybean entomology. Annu Rev Entomol. 1976;21:247–82. View ArticleGoogle Scholar
- Kameda K. Population increase of the Great Cormorant Phalacrocorax carbo and measures to reduce its damage to the fisheries and forests of Lake Biwa. In: Kawanabe H, Nishino M, Maehata M, editors. Lake Biwa: interactions between nature and people. Berlin: Springer; 2012. p. 491–6. Google Scholar
- van Eerden M, van Rijn S, Volponi S, Paquet JY, Carss D. Cormorants and the European environment: exploring cormorant status and distribution on a continental scale. In: INTERCAFE COST action 635 final report I. NERC/centre for ecology & hydrology on behalf of COST. 2012. Google Scholar
- Doucette JL, Wissel B, Somers CM. Cormorant–fisheries conflicts: stable isotopes reveal a consistent niche for avian piscivores in diverse food webs. Ecol Appl. 2011;21:2987–3001. View ArticleGoogle Scholar
- White CR, Boertmann D, Gremillet D, Butler PJ, Green JA, Martin GR. The relationship between sea surface temperature and population change of great cormorants Phalacrocorax carbo breeding near disko bay. Greenland Ibis. 2011;153:170–4. View ArticleGoogle Scholar
- Pham H. Continuous-time stochastic control and optimization with financial applications. Berlin: Springer; 2009. View ArticleGoogle Scholar
- Insley MC. On the timing of non-renewable resource extraction with regime switching prices: an optimal stochastic control approach. 2015. Google Scholar
- Lechuga GP. Optimal logistics strategy to distribute medicines in clinics and hospitals. J Math Ind. 2017;7:8. View ArticleGoogle Scholar
- Marheineke N, Cibis TM, Schiessl S, Pielmeier U. Optimal control of glucose balance in ICU patients based on GlucoSafe model. J Math Ind. 2014;4:3. MathSciNetView ArticleGoogle Scholar
- Øksendal B, Sulem A. Applied stochastic control of jump diffusions. Berlin: Springer; 2005. MATHGoogle Scholar
- Ohnishi M, Tsujimura M. An impulse control of a geometric Brownian motion with quadratic costs. Eur J Oper Res. 2006;168:311–21. MathSciNetView ArticleGoogle Scholar
- Tsujimura M, Maeda A. Stochastic control: theory and applications. Asakura Shoten. 2016. (in Japanese). Google Scholar
- Cadenillas A. Optimal central bank intervention in the foreign exchange market. J Econ Theory. 1999;87:218–42. MathSciNetView ArticleGoogle Scholar
- Bensoussan A, Long H, Perera S, Sethi S. Impulse control with random reaction periods: a central bank intervention problem. Oper Res Lett. 2012;40:425–30. MathSciNetView ArticleGoogle Scholar
- Korn R. Portfolio optimisation with strictly positive transaction costs and impulse control. Finance Stoch. 1998;2:85–114. View ArticleGoogle Scholar
- Øksendal B, Sulem A. Optimal consumption and portfolio with both fixed and proportional transaction costs. SIAM J Control Optim. 2002;40:1765–90. MathSciNetView ArticleGoogle Scholar
- Øksendal B. Stochastic control problems where small intervention costs have big effects. Appl Math Optim. 1999;40:355–75. MathSciNetView ArticleGoogle Scholar
- Crandall MG, Ishii H, Lions PL. User’s guide to viscosity solutions of second order partial differential equations. Bull Am Math Soc. 1992;27:1–67. MathSciNetView ArticleGoogle Scholar
- Pham H. Optimal stopping of controlled jump diffusion processes: a viscosity solution approach. J Math Syst Estim Control. 1998;8:1–27. MathSciNetGoogle Scholar
- Zhang S, Almost PD. Periodic viscosity solutions of nonlinear parabolic equations. Bound Value Probl. 2009;2009:873526. MathSciNetMATHGoogle Scholar
- Øksendal B. Stochastic differential equations. Berlin: Springer; 2003. View ArticleGoogle Scholar
- Kreyszig E. Advanced engineering mathematics. New York: Wiley; 2010. p. 69–72. Google Scholar
- Chong E, Zak SH. An introduction to optimization. Algorithms. 1996;419:75–83. MATHGoogle Scholar