Mathematical and numerical analyses of a stochastic impulse control model with imperfect interventions

Abstract

A stochastic impulse control problem with imperfect controllability of interventions is formulated with an emphasis on applications to ecological and environmental management problems. The imperfectness comes from uncertainties with respect to the magnitude of interventions. Our model is based on a dynamic programming formalism to impulsively control a 1-D diffusion process of a geometric Brownian type. The imperfectness leads to a non-local operator different from the many conventional ones, and evokes a slightly different optimal intervention policy. We give viscosity characterizations of the Hamilton–Jacobi–Bellman Quasi-Variational Inequality (HJBQVI) governing the value function focusing on its numerical computation. Uniqueness and verification results of the HJBQVI are presented and a candidate exact solution is constructed. The HJBQVI is solved with the two different numerical methods, an ordinary differential equation (ODE) based method and a finite difference scheme, demonstrating their consistency. Furthermore, the resulting controlled dynamics are extensively analyzed focusing on a bird population management case from a statistical standpoint.

1 Introduction

Balancing costs and benefits is essential for successful environmental and ecological management [1, 2]. Harvesting invasive species requires measuring invasion dynamics, evaluating costs of control efforts, and tracking invasion damages [3]. Cooperative lake eutrophication management considers the costs for water quality improvement and their benefits under social pressure [4]. Transboundary pollution problems are regional problems where the pollution mitigation policy is critically affected by the costs and benefits evaluation [5]. Smith et al. [6] discussed uncertainties in wildlife management focusing on problems related to waterbird.

Stochastic optimal control [7] is a central mathematical concept for modeling, analyzing, and controlling stochastically-driven system dynamics. Its application areas are widely distributed in both science [8, 9] and engineering [10, 11]. One effective way for solving a stochastic optimal control problem is to derive the optimality equation, often called Hamilton–Jacobi–Bellman (HJB) equation, from which we can construct the optimal control [12, 13]. The optimality equations are solved analytically in simple problems [7, 13, 14], while they must be approximated in complex problems using numerical methods [15–17]. Management problems related to environment and ecology, especially those on population dynamics that are inherently stochastic [18], are the ones that the stochastic control theory can handle. Biological resource exploitation considering renewal dynamics [19], invasive and nuisance species management [20–22], and ecosystem and biodiversity management [23, 24] are such examples.

Management problems of population dynamics through harvesting by human have been efficiently described with the impulse control formalism [7] whose control variables are the timing and amount of harvesting. Impulse control models have been applied to various problems, such as predator-prey population management [25], fishery resources management [26], aquaculture operation [27], tree harvesting [28, 29], and waterbird management [30]. Deep mathematical analysis of the impulse control models of population dynamics and related models have been carried out [31, 32]. Other important industrial problems, such as gene regulation [33] and satellite control [34, 35], have been considered with the impulse control as well.

Models based on the impulse control have thus been attractive mathematical tools for management of population dynamics. However, to the best of the authors’ knowledge, they lack the important fact that the impacts of interventions on the population dynamics are not always realized as planned by the decision-maker, but are often uncertain as seen in the field survey results [36]. Loosely speaking, the same effort of interventions does not always lead to the same impacts on the population dynamics. Korn [37] considered an imperfect impulse control problem of diffusion processes, and Helmes et al. [38] generalized it in an abstract manner for continuous-time Markov processes. Considering the uncertainty, namely imperfectness of interventions, is a natural policy in environmental and ecological management. However, such a mathematical approach has not been made so far. Impulse control models in other research areas [39–41] also do not handle cases where the impulsive interventions have uncertain impacts. This research background motivates us to formulate and analyze an impulse control problem of population dynamics subject to imperfectness interventions.

The objectives of this paper are formulation and analysis of an impulse control problem of population dynamics subject to imperfect interventions, focusing on its application to a recent waterbird management problem [21, 30]. Our model is based on Korn [37], but considers a specific problem where a deeper mathematical and numerical analysis are possible. In addition, our contributions include derivation of a candidate exact solution and numerical computation of the stochastic control problem. A 1-D stochastic differential equation (SDE) describes the population dynamics in a habitat. The population is regulated through costly and imperfect impulsive interventions. Optimizing the interventions is achieved through minimization of a performance index considering both costs and benefits of the interventions. An application of the dynamic programming principle to the system dynamics and the performance index leads to the optimality equation as a nonlinear complementary problem called the Hamilton–Jacobi–Bellman Quasi Variational Inequality (HJBQVI) [7]. The HJBQVI has a different non-locality from the conventional ones due to the imperfectness of the interventions. This gives rise to an optimal policy having a slightly more complicated form than the conventional ones. We give viscosity characterizations of the (HJBQVI) [42, 43] and that uniqueness and verification results hold true for the HJBQVI. A candidate exact solutions is constructed analytically, and a system of nonlinear equations governing its unknown coefficients is derived. Validity of the candidate exact solution is checked through comparison with numerical solutions generated by a steady counterpart of a formally first-order finite difference scheme [44], and a sharper convergence estimate against our HJBQVI is obtained. Optimally controlled population dynamics is extensively analyzed as well from a statistical standpoint. Our model is relatively simple, but potentially serves as a foundation for analysis of imperfect impulse control problems. Our contribution is rather from an engineering side but contains a new application of the impulse control.

The rest of this paper is organized as follows. In Sect. 2, we present the impulse control model and derives the HJBQVI. Mathematical analysis results on the value function, HJBQVI, and the controlled dynamics are presented in Sect. 3. In Sect. 4, the HJBQVI is solved numerically by two different methods, and we see that their results agree well with each other. Sensitivity analysis of the optimal control and the controlled dynamics is carried out in this section as well. Finally, in Sect. 5, we summarize the results of this paper and present our future perspectives. Proofs of Propositions, which are a bit technical, are placed in the Appendix.

2 Mathematical model

An SDE for describing single-species population dynamics with imperfect impulsive harvesting is presented, and the performance index to be minimized through choosing harvesting policies is formulated. The HJBQVI of the present optimal control problem is finally derived. Our mathematical formulation is based on Korn [37], but handles a new specific problem related to ecological and environmental management.

2.1 Stochastic differential equation

The time is denoted as t. A 1-D standard Brownian motion defined on a complete probability space is denoted as $B_{t}$ at time t [7]. A 1-D SDE governs an adapted càdlàg stochastic process $( X_{t} )_{t \ge 0}$ representing the population. Let $\tau _{i}$ ($i = 0,1,2, \ldots , \tau _{0} = 0$) be the time at which the intervention is performed to harvest the population. Without significant loss of generality, we assume that the sequence $\{ \tau _{i} \} _{i = 0,1,2,\ldots}$ is increasing. The amount of population harvested at the time $\tau _{i}$ is denoted as $\zeta _{i}$ with $0 < \zeta _{i} \le X_{\tau _{i} -}$ ($i \ge 1$) and $\zeta _{0} = 0$. The couple of sequences $\eta = \{ \tau _{i},\zeta _{i} \} _{i = 0,1,2,\ldots}$, which is called policy, is optimized so that a performance index is minimized.

We are interested in an imperfect impulse control problem in which the harvesting is not always realized as planned by the decision-maker, the manager of the population, but may be perturbed stochastically. Let $Z = \{ z_{i} \} _{i = 0,1,2,\ldots}$ be a sequence of i.i.d stochastic variables valued in a compact set $S = [ a,b ]$ with $0 < a < b \le 1$ having the probability density function (PDF) q. $Z = \{ z_{i} \} _{i = 0,1,2,\ldots}$ is independent from $( B_{t} )_{t \ge 0}$. The decision-maker can know the value of $z_{i}$ just after the harvesting at $\tau _{i}$.

Remark 1

The assumption $0 < a < b \le 1$ means that over-harvesting such that the population is harvested more than that planned by the decision-maker does not occur. In this sense, the present model is more pessimistic than the conventional impulse control models with perfect interventions.

A policy η is called admissible if it complies with the following conditions, which are natural requirements to well-pose the problem (see, Definition 2.1 of Korn [37]):

The conditions of admissible policy

$$\begin{aligned}& \tau _{0} = 0\quad \text{and}\quad \zeta _{0} = 0, \end{aligned}$$

(1)

$$\begin{aligned}& 0 \le \tau _{i} \le \tau _{i + 1},\qquad 0 < \zeta _{i + 1} \le X_{ ( \tau _{i + 1} ) -},\quad \mbox{a.s.}, i \ge 0, \end{aligned}$$

(2)

$$\begin{aligned}& \begin{aligned} &\tau _{i} \mbox{ is a stopping time with respect to the filtration generated by} \\ &\quad \sigma \bigl\{ X_{s -}, ( \tau _{n},\zeta _{n},z_{n} ), n < i, s \le t \bigr\} , \end{aligned} \end{aligned}$$

(3)

$$\begin{aligned}& \begin{aligned} &\zeta _{i}\mbox{ is measurable with respect to the filtration generated by } \sigma \bigl\{ X_{\tau _{n} -}, ( \tau _{n},\zeta _{n},z_{n} ), n < i \bigr\} , \end{aligned} \end{aligned}$$

(4)

$$\begin{aligned}& \Pr \Bigl[ \lim_{i \to \infty } \tau _{i} \le T \Bigr] = 0,\quad \forall T \in [ 0,\infty ), \end{aligned}$$

(5)

$$\begin{aligned}& \mathrm{E} \Biggl[ \sum_{i = 1}^{\infty } e^{ - \bar{\delta } \tau _{i}} ( 1 + \zeta _{i} ) \Biggr] < + \infty \quad \mbox{for a sufficiently large } \bar{\delta } > 0, \end{aligned}$$

(6)

$$\begin{aligned}& X_{t} \ge 0,\quad \mbox{a.s. } t \ge 0. \end{aligned}$$

(7)

Here, $\Pr [ \cdot ]$ and $\mathrm{E} [ \cdot ]$ present the probability and expectation, respectively. The collection of all admissible policies, namely policies complying with (2) through (7), is denoted as $\mathcal{A}$. This set depends on $X_{0 -}$, and is denoted as $\mathcal{A}_{X_{0 -}}$ when necessary. Here, $X_{t -}$ is the left-limit of X at time t.

For each $\eta \in \mathcal{A}$, the SDE governing $( X_{t} )_{t \ge 0}$ is set as

$$ \textstyle\begin{cases} \mathrm{d}X_{t} = X_{t} ( \mu \,\mathrm{d}t + \sigma \,\mathrm{d}B_{t} ), \quad \tau _{i} \le t < \tau _{i + 1}, \\ X_{\tau _{i}} = X_{\tau _{i -}} - z_{i}\zeta _{i}, \end{cases} $$

(8)

subject to an initial condition $X_{0 -} = x \ge 0$, where the fluctuation term $\sigma X_{t}\,\mathrm{d}B_{t}$ is in the Itô’s sense. Here, $\mu > 0$ is the intrinsic growth rate and $\sigma > 0$ is the stochastic growth rate modulating temporal fluctuation involved in the dynamics [21]. Since we are interested in controlling the population through harvesting, assume $2\mu > \sigma ^{2}$ so that $X_{t} \to + \infty$ a.s. as $t \to + \infty$ if there is no intervention [21]. This condition is utilized in the statistical analysis of Sect. 3.3. Due to the global Lipschitz continuity of the coefficients of the SDE (8), it admits a unique non-negative solution in a path-wise sense in each $\tau _{i} \le t \le \tau _{i + 1}$ when $\tau _{i} < \tau _{i + 1}$. In addition, the SDE (8) admits a unique non-negative solution in a path-wise sense global in time since $X_{\tau _{i}} = X_{\tau _{i -}} - z_{i}\zeta _{i} \ge 0$. The second line of (8) means that the decision-maker choses $\zeta _{i}$ based on $X_{\tau _{i -}}$. Due to the uncertain perturbation $z_{i}$ not controllable by the decision-maker, the realized state transition at the time $\tau _{i}$ is $X_{\tau _{i -}} \to X_{\tau _{i -}} - z_{i}\zeta _{i}$. We assume that the decision-maker knows the PDF q. He/she thus wants to decide a policy $\eta \in \mathcal{A}$ that can minimize a performance index under the imperfectness of harvesting.

2.2 Performance index

Following the previous research of controlling population dynamics [21, 30], the performance index $J = J ( x;\eta )$ ($x \ge 0$, $\eta \in \mathcal{A}_{x}$) is the expected net cost depending on the initial condition:

$$ J ( x;\eta ) = \mathrm{E} \Biggl[ \int _{0}^{\infty } e^{ - \delta s} \bigl( - RX_{s}^{M} + rX_{s}^{m} \bigr) \,\mathrm{d}s + \sum_{i = 1}^{\infty } e^{ - \delta \tau _{i}}K ( \zeta _{i} ) \Biggr] $$

(9)

with

$$ K ( \zeta ) = k_{1}\zeta + k_{0},\quad \zeta > 0 $$

(10)

and

$$ 0 < M < 1 < m \le 2, $$

(11)

where $\delta > 0$ is the discount rate, R, r, M and m are the positive constants. In (10), $K ( \zeta )$ is the cost of harvesting with the amount of $\zeta > 0$, where $k_{1}\zeta$ with $k_{1} > 0$ is the proportional cost and $k_{0} > 0$ is the fixed cost incurred regardless of ζ.

The performance index J in (9) consists of the two terms: the cumulative total disutility (first term, the integral) caused by the existence of the population and the total cost of harvesting the population (second term, the sum). In the first term, $- RX_{s}^{M}$ represents the ecological utility provided by the population, and $rX_{s}^{m}$ represents the disutility caused by the population [21]. The negativity of the first term is due to measuring the cost to be positive in the present model. The sum $- RX_{s}^{M} + rX_{s}^{m}$ is unimodal and convex with respect to $X_{s} \ge 0$, and is bounded from below by a negative constant as

$$ - RX_{s}^{M} + rX_{s}^{m} \ge - r\frac{m - M}{M} \biggl( \frac{MR}{mr} \biggr)^{\frac{m}{m - M}} \quad \text{if } X_{s} \ge 0. $$

(12)

The relationship (11) means that the rate of increase of the disutility caused by the population is higher than that of the utility provided by the population. The condition $0 < M < 1$ in (11) means that the ecosystem services provided by the population [45, 46] drastically change between the states that the population exists ($X_{s} > 0$) and that the population does not exist ($X_{s} = 0$). The assumption $m \le 2$ is to guarantee a square-integrability of the term $rX_{s}^{m}$ in the performance index.

As in the conventional impulse control problems, we assume that δ is sufficiently large so that J is bounded [47–49]. To guarantee this, we assume

$$ \delta > m\mu + \frac{1}{2}m ( m - 1 )\sigma ^{2}. $$

(13)

In fact, by (13), we obtain

$$ \begin{aligned} &\mathrm{E} \biggl[ \int _{0}^{\infty } e^{ - \delta s} \bigl( - RX_{s}^{M} + rX_{s}^{m} \bigr) \,\mathrm{d}s \biggr] \\ &\quad \le \mathrm{E} \biggl[ \int _{0}^{\infty } e^{ - \delta s}rX_{s}^{m} \,\mathrm{d}s \biggr] \\ &\quad \le r\mathrm{E} \biggl[ \int _{0}^{\infty } e^{ - \delta s} \bigl( xe^{ ( r - \frac{\sigma ^{2}}{2} )s + \sigma B_{s}} \bigr)^{m}\,\mathrm{d}s \biggr] \\ &\quad = Ax^{m} ,\quad A = \frac{r}{\delta - m\mu - \frac{1}{2}m ( m - 1 )\sigma ^{2}} > 0 \end{aligned} $$

(14)

for arbitrary $\eta \in \mathcal{A}$. We also have

$$ \mathrm{E} \biggl[ \int _{0}^{\infty } e^{ - \delta s} \bigl( - RX_{s}^{M} + rX_{s}^{m} \bigr) \,\mathrm{d}s \biggr] \ge - r\frac{m - M}{\delta M} \biggl( \frac{MR}{mr} \biggr)^{\frac{m}{m - M}} $$

(15)

for arbitrary $\eta \in \mathcal{A}$ because the function $- Rx^{M} + rx^{m}$ ($x \ge 0$) is convex and has a minimum value.

2.3 Value function and HJBQVI

The value function $\Phi = \Phi ( x )$ is the minimized performance index with respect to $\eta \in \mathcal{A}$:

$$ \Phi ( x ) = \inf_{\eta \in \mathcal{A}}J ( x;\eta ) = J \bigl( x;\eta ^{*} \bigr),\quad x \ge 0, $$

(16)

where $\eta ^{*} = \{ \tau _{i}^{*},\zeta _{i}^{*} \} _{i = 0,1,2,\ldots} \in \mathcal{A}$ minimizing J is referred to as the optimal policy. This Φ is bounded from below and locally bounded from above, and is well-defined. These are because, firstly

$$ \begin{aligned} \Phi ( x ) &\ge - r\frac{m - M}{\delta M} \biggl( \frac{MR}{mr} \biggr)^{\frac{m}{m - M}} + \inf_{\eta \in \mathcal{A}}\mathrm{E} \Biggl[ \sum _{i = 1}^{\infty } e^{ - \delta \tau _{i}}K ( \zeta _{i} ) \Biggr] \\ &\ge - r\frac{m - M}{\delta M} \biggl( \frac{MR}{mr} \biggr)^{\frac{m}{m - M}},\quad x \ge 0. \end{aligned} $$

(17)

Secondly, with the null control $\eta = \eta _{0}$ such that $\tau _{i} = + \infty$ ($i \ge 1$), which is the control with no intervention, $( X_{t} )_{t \ge 0}$ is a classical geometric Brownian motion. Then, we get

$$ \begin{aligned} \Phi ( x ) &\le \mathrm{E} \biggl[ \int _{0}^{\infty } e^{ - \delta s} \bigl( - RX_{s}^{M} + rX_{s}^{m} \bigr) \,\mathrm{d}s \biggr] \\ &\le Bx^{M} + Ax^{m},\quad B = - \frac{R}{\delta - M\mu - \frac{1}{2}M ( M - 1 )\sigma ^{2}} < 0. \end{aligned} $$

(18)

In addition, $\Phi ( 0 ) = 0$ because $X_{t} = 0$ ($t \ge 0$) if $x = 0$ and only the null control $\eta _{0}$ is admissible in this case. The boundedness results of Φ are sharpened later.

We assume the following recursion inspired from the dynamic programming principle of the conventional impulse control problems [7]. Here, τ is any finite stopping time adapted to a natural filtration $\mathcal{F}$ generated by $( B_{t} )_{t \ge 0}$ and $\{ z_{i} \} _{i = 1,2,3,\ldots}$:

$$ \Phi ( x ) = \inf_{\eta \in \mathcal{A}}\mathrm{E} \Biggl[ \int _{0}^{\tau } e^{ - \delta s} \bigl( - RX_{s}^{M} + rX_{s}^{m} \bigr) \,\mathrm{d}s + \sum_{i = 1,\tau _{i} \le \tau }^{\infty } e^{ - \delta \tau _{i}}K ( \zeta _{i} ) + e^{ - \delta \tau } \Phi ( X_{\tau } ) \Biggr], $$

(19)

which formally leads to the HJBQVI governing Φ:

$$ \max \bigl\{ \mathcal{L}\Phi + Rx^{M} - rx^{m},\Phi - \mathcal{M}\Phi \bigr\} = 0,\quad x > 0 $$

(20)

with the degenerate elliptic operator $\mathcal{L}$ and the non-local intervention operator $\mathcal{M}$

$$ \mathcal{L}W = \delta W - \mu x\frac{\mathrm{d}W}{\mathrm{d}x} - \frac{1}{2} \sigma ^{2}x^{2}\frac{\mathrm{d}^{2}W}{\mathrm{d}x^{2}} $$

(21)

and

$$ \mathcal{M}W = \sup_{0 < \zeta \le x} \biggl[ K ( \zeta ) + \int _{a}^{b} W ( x - z\zeta )q ( z ) \,\mathrm{d}z \biggr], $$

(22)

respectively, for generic regular $W = W ( x )$. The boundary condition for the HJBQVI is $\Phi ( 0 ) = 0$. Growth speed of (viscosity) solutions to (20) should be at most polynomial by (17) and (18).

The left part in “max” operator in the HJBQVI (20) corresponds to the situation where no intervention should be performed, while the right part corresponds to the situation where the intervention should be performed immediately. A difference between the present and conventional HJBQVIs is that the present non-local operator $\mathcal{M}$ involves the effect of uncertainty induced by the uncertainties Z, while the conventional ones do not and have the non-local terms like $\mathcal{M}W = \sup_{0 < \zeta \le x} [ K ( \zeta ) + W ( x - \zeta ) ]$ in which no uncertainty is considered [30, 39–41].

3 Mathematical analysis

3.1 Value function

Several key mathematical properties of the value function Φ are analyzed, from which we show that it is a viscosity solution to the HJBQVI (20). A comparison result for a modified problem that has important implications in numerical computation is also presented.

The first result concerns local boundedness of Φ as an extension of the discussion in the previous section.

Proposition 1

$$ \max \biggl\{ - r\frac{m - M}{\delta M} \biggl( \frac{MR}{mr} \biggr)^{\frac{m}{m - M}},Bx^{M} \biggr\} \le \Phi ( x ) \le Bx^{M} + Ax^{m},\quad x \ge 0. $$

(23)

In addition, $\lim_{x \to + 0}\Phi ( x ) = \Phi ( 0 ) = 0$.

The following proposition gives a key inequality characterizing the value function Φ, which is effectively utilized in a proof of viscosity property of Φ.

Proposition 2

$$ \Phi ( x ) \le \mathcal{M}\Phi ( x ),\quad x \ge 0. $$

(24)

In this paper, we assume that the value function Φ is continuous, which is true in the problems without uncertainties in the interventions [30, 50, 51]. The continuity is also justified numerically as the computational results with a finite difference scheme suggest (see, Sect. 4). The following continuity result supports the definitions of viscosity solutions to the HJBQVI (20) presented later. Actually, the candidate of exact solution, which turns out to be the value function under certain conditions, is continuous.

Proposition 3

There exists a constant $C > 0$ such that

$$ \Phi ( x ) - \Phi ( y ) \le C \bigl( ( x - y )^{M} + x^{m - 1} ( x - y ) \bigr),\quad 0 \le y \le x. $$

(25)

In addition, $\Phi ( x )$ is continuous at $x = 0$.

Remark 2

An obstacle to derive the other side of the inequality is the dependence of the set $\mathcal{A}_{x}$ of admissible controls on x. Notice that this problem has not been encountered in the conventional research such as Guo and Wu [48] because their admissible sets are independent from the state variables.

3.2 HJBQVI and optimal control

3.2.1 Viscosity property

Here, we show that the HJBQVI (20) is solvable in a weak sense. The weak solutions employed in this paper are viscosity solutions [42, 43] as appropriate weak solutions to degenerate elliptic problems. We show that the value function Φ is a continuous viscosity solution to the HJBQVI (20) under certain conditions. As noted above, hereafter we assume that the value function is continuous, which is partly supported in Proposition 3.

Viscosity solutions to the HJBQVI (20) are defined as follows based on the monotonicity and continuity result of non-local terms in HJBQVIs (Proposition 3 of Azimzadeh et al. [42]) whose conditions are satisfied by our non-local operator $\mathcal{M}$. In what follows the function space of upper-semi continuous (resp., lower-semi continuous) functions valued in a domain Ω is denoted as $\operatorname{USC} ( \Omega )$ (resp., $\operatorname{LSC} ( \Omega )$). Note that the difference between a test function and a viscosity sub-solution or a super-solution needs not be globally maximized or minimized because the non-local part is defined using super- or sub-solution itself (e.g., [42, Definition 1]).

Definition 1

• Viscosity sub-solution. A function $\varphi \in \operatorname{USC} [ 0, + \infty ) \cap C ( 0, + \infty )$ with $\varphi ( 0 ) \le 0$ is a viscosity sub-solution to the HJVQVI (20) if the following conditions are satisfied at each $x > 0$: for any $\psi \in C [ 0, + \infty ) \cap C^{2} ( 0, + \infty )$ such that $\varphi - \psi$ is locally strictly maximized at x with $\varphi ( x ) - \psi ( x ) = 0$, we have

  $$ \max \bigl\{ \mathcal{L}\psi + Rx^{M} - rx^{m},\varphi - \mathcal{M}\varphi \bigr\} \le 0. $$

  (26)

• Viscosity super-solution. A function $\varphi \in \operatorname{LSC} [ 0, + \infty ) \cap C ( 0, + \infty )$ with $\varphi ( 0 ) \ge 0$ is a viscosity super-solution to the HJBQVI (20) if the following conditions are satisfied at each $x > 0$: for any $\psi \in C [ 0, + \infty ) \cap C^{2} ( 0, + \infty )$ such that $\varphi - \psi$ is locally strictly minimized at x with $\varphi ( x ) - \psi ( x ) = 0$, we have

  $$ \max \bigl\{ \mathcal{L}\psi + Rx^{M} - rx^{m},\varphi - \mathcal{M}\varphi \bigr\} \ge 0. $$

  (27)

• Viscosity solution. A function $\varphi \in C [ 0, + \infty )$ is a viscosity solution to the HJBQVI (20) if it is a viscosity sub-solution as well as a viscosity super-solution in the above-mentioned sense.

The following proposition shows the viscosity property of the value function Φ and supports our approach of finding the optimal policy through solving the HJBQVI (20).

Proposition 4

The value function Φ is a viscosity solution to the HJBQVI (20).

Uniqueness of solutions to our HJBQVI is an important issue as well, from which we can guarantee unique existence of viscosity solutions under certain assumptions, and also convergence of numerical solutions generated by an appropriate discretization method. An interest, especially in numerical computation, is uniqueness of the localized HJBQVI:

$$\begin{aligned}& \max \bigl\{ \mathcal{L}\Phi + Rx^{M} - rx^{m},\Phi - \mathcal{M}\Phi \bigr\} = 0,\quad 0 < x < X_{\max }, \end{aligned}$$

(28)

$$\begin{aligned}& \max \{ \Phi - g,\Phi - \mathcal{M}\Phi \} = 0,\quad x = X_{\max } \end{aligned}$$

(29)

with $\Phi ( 0 ) = 0$ and a sufficiently large constant $g > 0$ such that $g > AX_{\max }^{m}$. Here, $X_{\max } > 0$ is a constant to truncate the unbounded domain $[ 0, + \infty )$ as $[ 0,X_{\max } ]$. This kind of localization has been commonly used in solving degenerate elliptic problems [49, 52, 53]. Here, introducing the term $\Phi - g$ is from a technical reason (see, Proof of Proposition 5) to guarantee uniqueness of the system and actually innocuous because of the estimate (23).

Following Definition 1, viscosity solutions to the localized HJBQVI are defined as follows: A difference between Definitions 1 and 2 is the boundary condition at the right-end. (See, (31) and (33). They are absent in Definition 1.)

Definition 2

• Viscosity sub-solution. A function $\varphi \in \operatorname{USC} [ 0,X_{\max } ] \cap C ( 0,X_{\max } ]$ with $\varphi ( 0 ) \le 0$ is a viscosity sub-solution to the localized HJVQVI (28)–(29) if the following conditions are satisfied at each $x > 0$: for any $\psi \in C^{2} [ 0,X_{\max } ]$ such that $\varphi - \psi$ is locally strictly maximized at x with $\varphi ( x ) - \psi ( x ) = 0$, we have

  $$\begin{aligned}& \max \bigl\{ \mathcal{L}\psi + Rx^{M} - rx^{m},\varphi - \mathcal{M}\varphi \bigr\} \le 0,\quad 0 < x < X_{\max },\ \end{aligned}$$

  (30)
  $$\begin{aligned}& \max \{ \varphi - g,\varphi - \mathcal{M}\varphi \} \le 0,\quad x = X_{\max }. \end{aligned}$$

  (31)

• Viscosity super-solution. A function $\varphi \in \operatorname{LSC} [ 0,X_{\max } ] \cap C ( 0,X_{\max } ]$ with $\varphi ( 0 ) \ge 0$ is a viscosity super-solution to the localized HJBQVI (28)–(29) if the following conditions are satisfied at each $x > 0$: for any $\psi \in C^{2} [ 0,X_{\max } ]$ such that $\varphi - \psi$ is locally strictly minimized at x with $\varphi ( x ) - \psi ( x ) = 0$, we have

  $$\begin{aligned}& \max \bigl\{ \mathcal{L}\psi + Rx^{M} - rx^{m},\varphi - \mathcal{M}\varphi \bigr\} \ge 0,\quad 0 < x < X_{\max }, \end{aligned}$$

  (32)
  $$\begin{aligned}& \max \{ \varphi - g,\varphi - \mathcal{M}\varphi \} \ge 0,\quad x = X_{\max }. \end{aligned}$$

  (33)

• Viscosity solution. A function $\varphi \in C [ 0,X_{\max } ]$ is a viscosity solution to the localized HJBQVI (28)–(29) if it is a viscosity sub-solution as well as a viscosity super-solution.

Now, we prove uniqueness of viscosity solutions to the localized HJBQVI (28)–(29) through a comparison argument.

Proposition 5

For the localized HJBQVI (28)–(29), any viscosity sub-solution u and any viscosity super-solution v satisfy $u \le v$ in $[ 0,X_{\max } ]$. In particular, it admits at most one viscosity solution in the sense of Definition 2.

Remark 3

The value function is the unique viscosity solution to the localized HJBQVI (28)–(29) if $\Phi - \mathcal{M}\Phi = 0$ and $\mathcal{L}\Phi + Rx^{M} - rx^{m} < 0$ for sufficiently large $x > 0$. In fact, if this holds true, then we can choose a sufficiently large $X_{\max }$ such that $\Phi - \mathcal{M}\Phi = 0$ for $x > X_{\max }$.

3.2.2 Verification

In this sub-section, we present a candidate exact solution to the HJBQVI (20). From a management viewpoint [21, 30], we conjecture the following with some $\bar{x} > 0$:

$$ \mathcal{L}\Phi + Rx^{M} - rx^{m} = 0\quad \text{and}\quad \Phi - \mathcal{M}\Phi < 0\quad \text{for } 0 < x < \bar{x} $$

(34)

and

$$ \mathcal{L}\Phi + Rx^{M} - rx^{m} < 0\quad \text{and} \quad \Phi - \mathcal{M}\Phi = 0\quad \text{for } x \ge \bar{x}. $$

(35)

The point $x = \bar{x}$ is a free boundary. The conditions (34) and (35) imply that the optimal policy is of the following form, which is of course admissible, and is referred to as QVI-control [37, 54]:

$$\begin{aligned}& ( \tau _{0},\zeta _{0} ) = ( 0,0 ), \end{aligned}$$

(36)

$$\begin{aligned}& \tau _{i} = \inf \bigl\{ t \ge \tau _{i - 1}:\Phi ( X_{t} ) = \mathcal{M}\Phi ( X_{t} ) \bigr\} ,\quad i \ge 1, \end{aligned}$$

(37)

$$\begin{aligned}& \zeta _{i} = \arg \min_{0 < \zeta \le x} \biggl\{ K ( \zeta ) + \int _{a}^{b} \Phi ( x - z\zeta )q ( z ) \,\mathrm{d}z \biggr\} ,\quad i \ge 0. \end{aligned}$$

(38)

From (36) through (38),we infer that the following threshold-type policy with the threshold values x̄ and $\underline{x}$ ($\bar{x} > \underline{x}$) is optimal for $t \ge 0$ (see, Korn [37]):

• (A) $X_{t -} < \bar{x}$, then no intervention is performed.

• (B) If $X_{t -} \ge \bar{x}$, then the intervention with the magnitude of $\zeta = \bar{x} - \underline{x} ( \bar{x} )$ is immediately performed (and then $X_{t -}$ is actually reduced to $X_{t} = \bar{x} - ( \bar{x} - \underline{x} ( X_{t -} ) )z$, where z is uncertain for the decision-maker). Notice that we may have $X_{t} \ge \bar{x}$. Then, repeat the first sentence of (B) until we get $X_{t} < \bar{x}$.

Notice that multiple interventions are allowed at the same time [37]. This is due to the model simplification that assumes the time scale of interventions is much smaller than that of the population dynamics. Here, $\bar{x} > 0$ is a constant while $\underline{x} ( x )$ is a function of $x > 0$ and is assumed to be $0 < \underline{x} ( x ) < \bar{x}$ ($x \ge \bar{x}$). With the two thresholds x̄ and $\underline{x} ( x )$, the optimal amount of the intervention $\zeta ^{ *}$ becomes

$$ \zeta ^{ *} ( x ) = \textstyle\begin{cases} x - \underline{x} ( x ) &( x \ge \bar{x} ), \\ 0 & ( x < \bar{x} ). \end{cases} $$

(39)

Figure 1 presents an image of sample paths by the above-presented threshold-type policy. The function $\underline{x} ( \cdot )$ is the population just after the harvesting. It is related to equation (38) in the way that $\zeta = \bar{x} - \underline{x} ( x )$ is its maximizer given $x \ge 0$ under the conjectures (A)–(B). An example of $\underline{x} ( \cdot )$ is plotted in Fig. 2 of Sect. 4.2.

Figure 1

A sample path of the controlled population dynamics (blue), the upper threshold x̄ (red), and the uncertainty range $[ \bar{x} - b\zeta ^{ *} ( \bar{x} ),\bar{x} - a\zeta ^{ *} ( \bar{x} ) ]$ (pink)

Note that the conventional 1-D impulse control problems in an infinite horizon assume constant thresholds [44, 50, 54–56], namely x̄ and $\underline{x} ( x )$ are positive constants, while in ours the latter is not a constant. If both the thresholds are constant, then (39) becomes a policy with a constant $\underline{x}$:

$$ \zeta ^{ *} = \textstyle\begin{cases} x - \underline{x} & ( x \ge \bar{x} ), \\ 0 & ( x < \bar{x} ). \end{cases} $$

(40)

However, later we imply that this is not always true in our model, through an exploration of exact solutions. This difference between the conventional and our models is due to the form of the non-local term in the HJBQVI (20). In fact, the previous model without the uncertainty [30, 50, 51] admits a constant $\underline{x}$. We also demonstrate through numerical experiments that the threshold $\underline{x}$ depends on x in the present model.

The final proposition concerns optimality of the exact solution.

Proposition 6

Let ϕ be a viscosity solution to the HJBQVI (20) and S be a finite sub-set in $( 0, + \infty )$. Assume that ϕ satisfies the upper- and lower-bounds (23) and is twice continuously differentiable except at each point of S. Assume that there is $\bar{x} > 0$ such that

$$ \mathcal{L}\phi + Rx^{M} - rx^{m} = 0 \quad \textit{and}\quad \phi - \mathcal{M}\phi \le 0 \quad \textit{for } 0 < x < \bar{x} $$

(41)

and

$$ \mathcal{L}\phi + Rx^{M} - rx^{m} \le 0 \quad \textit{and}\quad \phi - \mathcal{M}\phi = 0 \quad \textit{for } x \ge \bar{x}. $$

(42)

For each given $x \ge 0$, policy $\eta \in \mathcal{A}$, and stopping time τ, suppose that $\phi ( X_{\tau } )$ is uniformly integrable and satisfies the bound (23). Then, we have

$$ \phi \le \Phi \quad \textit{in } [ 0, + \infty ). $$

(43)

In addition, if the QVI-control is admissible, then

$$ \phi = \Phi \quad \textit{in } [ 0, + \infty ). $$

(44)

3.2.3 Candidate of exact solutions

We try to construct a viscosity solution potentially complying with (34) and (35). Set

$$ \beta = \frac{1}{2} \biggl( 1 - \frac{2\mu }{\sigma ^{2}} + \sqrt{ \biggl( \frac{2\mu }{\sigma ^{2}} - 1 \biggr)^{2} + \frac{8\delta }{\sigma ^{2}}} \biggr) > 0. $$

(45)

Let Φ̄ be a candidate of viscosity solutions. Firstly, we consider the case $x < \bar{x}$, under which no intervention should be performed. General solutions to

$$ \mathcal{L}\bar{\Phi } + Rx^{M} - rx^{m} = 0 $$

(46)

are represented as

$$ \bar{\Phi } ( x ) = c_{1}x^{\beta _{ +}} + c_{2}x^{\beta _{ -}} + Bx^{M} + Ax^{m} $$

(47)

with

$$ \beta _{ \pm } = \frac{1}{2} \biggl( 1 - \frac{2\mu }{\sigma ^{2}} \pm \sqrt{ \biggl( \frac{2\mu }{\sigma ^{2}} - 1 \biggr)^{2} + \frac{8\delta }{\sigma ^{2}}} \biggr),\qquad \beta _{ +} > 0\quad \text{and} \quad \beta _{ -} < 0 $$

(48)

and constants $c_{1}$ and $c_{2}$. If $c_{2} \ne 0$, then $\lim_{x \to + 0}c_{2}x^{\beta _{ -}} = \pm \infty$ since $\beta _{ -} < 0$. Hereafter, we write $\beta = \beta _{ +}$. This contradicts with Proposition 1. Therefore, $c_{2} = 0$ and thus

$$ \bar{\Phi } ( x ) = \Phi _{0} ( x ) = cx^{\beta } + Bx^{M} + Ax^{m}, $$

(49)

where the notations $c_{1} = c$ is used and $\bar{\Phi } ( 0 ) = 0$.

Remark 4

By Proposition 1, we should have $c \le 0$.

For $x \ge \bar{x}$ where the intervention must be performed immediately, we should have

$$ \bar{\Phi } ( x ) = \mathcal{M}\bar{\Phi } ( x ) = \inf_{0 \le \zeta \le x} \biggl\{ K ( \zeta ) + \int _{a}^{b} \bar{\Phi } ( x - z\zeta )q ( z ) \,\mathrm{d}z \biggr\} $$

(50)

whose right-hand side is determined based on the information in $[ 0,x ]$. Therefore, theoretically, Φ̄ for $x \ge \bar{x}$ can be constructed from the left toward the right by the equation (50) once we get $\bar{\Phi } ( \bar{x} )$.

We found that the equation (50) is not analytically solvable because of its nonlinearity and nonlocality. Nevertheless, assuming that the population is not large at some time t, as demonstrated below, it is sufficient to solve the HJBQVI (20) for $x \le \bar{x}$ considering $x = \bar{x}$ as a free boundary:

• (a) Continuity of $\bar{\Phi } ( x )$ at $x = \bar{x}$ (Value matching condition).

• (b) Continuity of $\frac{\mathrm{d}\bar{\Phi }}{\mathrm{d}x} ( x )$ at $x = \bar{x}$ (Smooth pasting condition).

• (c) Optimality of the thresholds x̄ and $\underline{x} ( \bar{x} )$ (Optimality condition).

From the standpoint of the optimal policy constructed above, it is sufficient to have the point value $\zeta ^{ *} ( \bar{x} )$ but not whole $\zeta ^{ *} ( x ) ( x \ge \bar{x} )$ when the initial population x is not so large. This is because of the rules (A)–(B) of the optimal policy with which the process $( X_{t} )_{t \ge 0}$ is eventually confined in $[ 0,\bar{x} ]$. We thus only have to determine the three constants $\zeta ^{ *} ( \bar{x} )$, x̄, and c. For the sake of simplicity of presentations, write $\bar{\zeta } = \zeta ^{ *} ( \bar{x} )$. We assume that $\zeta ^{ *}$ is sufficiently smooth.

Now, we derive three equations to find the three unknowns ζ̄, x̄, and c. Assume $x \ge \bar{x}$. By the Value matching condition (a), we get

$$ \Phi _{0} ( x ) = K ( \bar{\zeta } ) + \int _{a}^{b} \bar{\Phi } ( x - z\bar{\zeta } )q ( z )\,\mathrm{d}z. $$

(51)

The optimality condition then gives the implicit governing equation of $\zeta = \zeta ^{*}$, if it is an interior minimizer, as

$$ k_{1} + \frac{\mathrm{d}}{\mathrm{d}\zeta } \int _{a}^{b} \bar{\Phi } ( x - z\zeta )q ( z ) \,\mathrm{d}z = k_{1} - \int _{a}^{b} \frac{\mathrm{d}}{\mathrm{d}x}\bar{\Phi } ( x - z\zeta )zq ( z )\,\mathrm{d}z = 0. $$

(52)

Namely, we have

$$ \int _{a}^{b} \frac{\mathrm{d}}{\mathrm{d}x}\bar{\Phi } \bigl( x - z\zeta ^{*} \bigr)zq ( z )\,\mathrm{d}z = k_{1}. $$

(53)

The Smooth pasting condition (b) gives

$$ \begin{aligned} \frac{\mathrm{d}}{\mathrm{d}x}\bar{\Phi } ( x ) &= k_{1} \frac{\mathrm{d}\zeta ^{ *}}{\mathrm{d}x} + \frac{\mathrm{d}}{\mathrm{d}x} \int _{a}^{b} \bar{\Phi } \bigl( x - z\zeta ^{ *} ( x ) \bigr)q ( z )\,\mathrm{d}z \\ &= \biggl( k_{1} - \int _{a}^{b} \frac{\mathrm{d}}{\mathrm{d}x}\bar{\Phi } \bigl( x - z\zeta ^{*} \bigr)zq ( z )\,\mathrm{d}z \biggr) \frac{\mathrm{d}\zeta ^{ *}}{\mathrm{d}x} + \int _{a}^{b} \frac{\mathrm{d}}{\mathrm{d}x}\bar{\Phi } \bigl( x - z\zeta ^{ *} ( x ) \bigr)q ( z )\,\mathrm{d}z. \end{aligned} $$

(54)

Combining (53) and (54) leads to

$$ \frac{\mathrm{d}}{\mathrm{d}x}\bar{\Phi } ( x ) = \int _{a}^{b} \frac{\mathrm{d}}{\mathrm{d}x}\bar{\Phi } \bigl( x - z\zeta ^{ *} ( x ) \bigr)q ( z )\,\mathrm{d}z. $$

(55)

Now, substituting $x = \bar{x}$ into (51), (53), and (55) with the Optimality condition (c) gives the system of three nonlinear equations to find the three unknowns ζ̄, x̄, and c:

$$\begin{aligned}& \Phi _{0} ( \bar{x} ) = K ( \bar{\zeta } ) + \int _{a}^{b} \Phi _{0} ( \bar{x} - z\bar{\zeta } )q ( z )\,\mathrm{d}z, \end{aligned}$$

(56)

$$\begin{aligned}& \int _{a}^{b} \frac{\mathrm{d}}{\mathrm{d}x}\Phi _{0} ( \bar{x} - z\bar{\zeta } )zq ( z )\,\mathrm{d}z = k_{1}, \end{aligned}$$

(57)

and

$$ \frac{\mathrm{d}}{\mathrm{d}x}\Phi _{0} ( \bar{x} ) = \int _{a}^{b} \frac{\mathrm{d}}{\mathrm{d}x}\Phi _{0} ( \bar{x} - z\bar{\zeta } )q ( z )\,\mathrm{d}z, $$

(58)

where

$$ \frac{\mathrm{d}}{\mathrm{d}x}\bar{\Phi } ( \bar{x} ) = \frac{\mathrm{d}}{\mathrm{d}x}\Phi _{0} ( \bar{x} ) = c\beta \bar{x}^{\beta - 1} + BM \bar{x}^{M - 1} + Am\bar{x}^{m - 1}. $$

(59)

Remark 5

We can expect an asymptotic relationship $\Phi ( x ) \sim dk_{1}x$ with a constant $d > 0$ for x sufficiently large. In fact, substituting $\Phi ( x ) = dk_{1}x$ into (53) leads to

$$ k_{1} = \int _{a}^{b} zq ( z )dk_{1} \,\mathrm{d}z, $$

(60)

namely

$$ d = \biggl( \int _{a}^{b} zq ( z )\,\mathrm{d}z \biggr)^{ - 1}\ ( \ge 1 ). $$

(61)

This is consistent with $\Phi ( x ) = \mathcal{M}\Phi ( x )$ for x sufficiently large because of

$$ \mathcal{M}\Phi ( x ) \sim k_{1}\xi ^{*} + \int _{a}^{b} dk_{1} \bigl( x - \zeta ^{ *} z \bigr)q ( z )\,\mathrm{d}z = dk_{1}x $$

(62)

by (61). From a practical viewpoint, (61) means that the increasing rate d with respect to the population x is not smaller than $k_{1}$ because of the uncertainty of the intervention impacts. We get $d = 1$ if and only if the interventions are perfect ($a = b = 1$ and q is a Dirac’s delta concentrated at $z = 1$). Therefore, the theoretical result implies that imperfect interventions are always less efficient than the perfect ones, and the efficiency can be measured by d. The modulation of the increasing rate is the inverse of the mean value of the uncertainty Z by (61). The coefficient d is determined by the moment of q. The result presented above is consistent with our intuition that the intervention is n (>1) times less efficient if its impact is expected to be n times smaller than that without the uncertainty.

3.2.4 Empirical numerical approach

The nonlinear equations (56) through (58) seem to be not analytically solvable and their numerical approximation in general may not be an easy task. We consider a simplified case where we can, at least empirically, find the unknowns ζ̄, x̄, and c. A similar numerical approach has been successfully applied to finding the coefficients in the exact solution to an HJBQVI associated with a problem with a perfect intervention [51].

We consider the uniform distribution $q ( z ) = \frac{1}{b - a}$ ($a \le z \le b$). In this case, by a straightforward calculation, we get

$$ \int _{a}^{b} \frac{\mathrm{d}}{\mathrm{d}x}\Phi _{0} ( \bar{x} - z\bar{\zeta } )q ( z )\,\mathrm{d}z = \frac{1}{\bar{\zeta } ( b - a )} \bigl( \Phi _{0} ( \bar{x} - a\bar{\zeta } ) - \Phi _{0} ( \bar{x} - b\bar{\zeta } ) \bigr) $$

(63)

and

$$ \begin{aligned} &\int _{a}^{b} \frac{\mathrm{d}}{\mathrm{d}x}\Phi _{0} ( x - z\bar{\zeta } )zq ( z )\,\mathrm{d}z \\ &\quad = \frac{1}{\bar{\zeta } ( b - a )} \bigl( a\Phi _{0} ( \bar{x} - a\bar{\zeta } ) - b\Phi _{0} ( \bar{x} - b\bar{\zeta } ) \bigr) + \frac{1}{\bar{\zeta } ( b - a )} \int _{a}^{b} \Phi _{0} ( \bar{x} - z\bar{\zeta } )\,\mathrm{d}z. \end{aligned} $$

(64)

Then, (57) and (58) are rewritten as

$$ k_{1}\bar{\zeta } ( b - a ) = a\Phi _{0} ( \bar{x} - a \bar{\zeta } ) - b\Phi _{0} ( \bar{x} - b\bar{\zeta } ) + \int _{a}^{b} \Phi _{0} ( \bar{x} - z\bar{\zeta } )\,\mathrm{d}z $$

(65)

and

$$ \frac{\mathrm{d}}{\mathrm{d}x}\Phi _{0} ( \bar{x} ) = \frac{1}{\bar{\zeta } ( b - a )} \bigl( \Phi _{0} ( \bar{x} - a\bar{\zeta } ) - \Phi _{0} ( \bar{x} - b\bar{\zeta } ) \bigr), $$

(66)

respectively. The three unknowns are explored through the following empirically-derived system containing the two ODEs and a nonlinear equation whose equilibrium satisfy (56), (65), and (66):

$$\begin{aligned}& \frac{\mathrm{d}\bar{x}}{\mathrm{d}\rho } = \bar{x}^{\gamma } \biggl[ \Phi _{0} ( \bar{x} - b\bar{\zeta } ) - \Phi _{0} ( \bar{x} - a\bar{\zeta } ) + \bar{\zeta } ( b - a )\frac{\mathrm{d}}{\mathrm{d}x}\Phi _{0} ( \bar{x} ) \biggr], \end{aligned}$$

(67)

$$\begin{aligned}& \frac{\mathrm{d}c}{\mathrm{d}\rho } = k_{0} + k_{1}\bar{\zeta } + \int _{a}^{b} \Phi _{0} ( \bar{x} - z\bar{\zeta } )q ( z )\,\mathrm{d}z - \Phi _{0} ( \bar{x} ), \end{aligned}$$

(68)

$$\begin{aligned}& \bar{\zeta } = \frac{1}{k_{1} ( b - a )} \biggl[ a\Phi _{0} ( \bar{x} - a\bar{\zeta } ) - b\Phi _{0} ( \bar{x} - b\bar{\zeta } ) + \int _{a}^{b} \Phi _{0} ( \bar{x} - z\bar{\zeta } )\,\mathrm{d}z \biggr], \end{aligned}$$

(69)

where $\gamma > 0$ is a constant and the coefficient $\bar{x}^{\gamma }$ in the right-hand side of (67) is chosen for the sake of empirical computational stability of the discretized system. Here, $\rho \ge 0$ is an auxiliary time variable to parameterize ζ̄, x̄, and c, and the ODEs (67) and (68) are integrated with respect to $\rho > 0$ with (69) starting from an initial guesses $( \bar{x},\bar{\zeta },c )_{\tau = 0}$. This is carried out in a standard forward Euler manner. Set the small increment $\Delta \rho > 0$ of ρ and a variable at $\rho = k\Delta \rho$ ($k = 0,1,2,\ldots$) is denoted with the subscript $( k )$. For $k \ge 0$, we discretize (67) through (69) as

$$\begin{aligned}& \begin{aligned} \bar{\zeta }_{ ( k + 1 )} ={}& \frac{1}{k_{1} ( b - a )} \biggl[ a\Phi _{0 , ( k )} ( \bar{x}_{ ( k )} - a\bar{\zeta }_{ ( k )} ) - b\Phi _{0 , ( k )} ( \bar{x}_{ ( k )} - b\bar{\zeta }_{ ( k )} ) \\ &{}+ \int _{a}^{b} \Phi _{0 , ( k )} ( \bar{x}_{ ( k )} - z\bar{\zeta }_{ ( k )} )\,\mathrm{d}z \biggr], \end{aligned} \end{aligned}$$

(70)

$$\begin{aligned}& \begin{aligned} \bar{x}_{ ( k + 1 )} ={}& \bar{x}_{ ( k )} + \bar{x}_{ ( k )}^{\gamma } \biggl[ \Phi _{0, ( k )} ( \bar{x}_{ ( k )} - b\bar{\zeta }_{ ( k + 1 )} ) - \Phi _{0, ( k )} ( \bar{x}_{ ( k )} - a \bar{\zeta }_{ ( k + 1 )} ) \\ &{}+ \bar{\zeta }_{ ( k + 1 )} ( b - a ) \frac{\mathrm{d}}{\mathrm{d}x}\Phi _{0, ( k )} ( \bar{x}_{ ( k )} ) \biggr] \Delta \rho , \end{aligned} \end{aligned}$$

(71)

$$\begin{aligned}& c_{ ( k + 1 )} = c_{ ( k )} + \biggl( k_{0} + k_{1}\bar{\zeta }_{ ( k )} + \int _{a}^{b} \Phi _{0, ( k )} ( \bar{x}_{ ( k )} - z\bar{\zeta }_{ ( k )} )q ( z )\,\mathrm{d}z - \Phi _{0, ( k )} ( \bar{x} ) \biggr)\Delta \rho , \end{aligned}$$

(72)

where $\Phi _{0, ( k )}$ is $\Phi _{0}$ with $c = c_{ ( k )}$. From a stability viewpoint of discretized ODEs [57], we expect that the discretized system (70)–(72) can be solved numerically with a small $\Delta \rho > 0$.

Remark 6

We observed that convergence of solutions to (70)–(72) depends on the initial guess, implying its sensitivity against the initial guess. At the current stage, we have neither unique solvability nor stability results on the continuous system (67)–(69) and discretized system (70)–(72). Therefore, we utilize the ODE-based method as a tool to validate the finite difference scheme because the latter is more computationally stable for our problem.

Remark 7

Construction of the system is based on an empirical approach. Yaegashi et al. [51] addressed this issue for an impulse control problem with perfect interventions. Fortunately, their ODE-based method can compute the unknown coefficients as a locally asymptotically stable equilibrium point of a system. However, such a result has not been found in the present case.

3.3 Statistics of controlled state dynamics

From a practical point of view, analyzing statistics of the optimally-controlled population dynamics is as important as finding the optimal intervention policy. Under the conjectured optimal policy in the previous sub-sections, we can focus on the dynamics in the interval $[ 0,\bar{x} ]$. The threshold-type optimal control established in the previous section is assumed in what follows.

Firstly, we derive the statistical moments of the intervals $\tau _{i + 1} - \tau _{i}$ between each successive intervention. This is a stochastic variable of interest in applications, because the frequency of interventions crucially affects cost-effectiveness of the population management. The present problem is a stochastic optimal control of a time-homogenous process in an infinite horizon, and thus the statistical properties of the optimally controlled dynamics can be studied through stationary distributions.

The first hitting time $\tau _{x}$ of the process $( X_{t} )_{t \ge 0}$ ($t > 0$) with $X_{0 -} = x \in ( 0,\bar{x} ]$ is defined as

$$ \tau _{x} = \inf \bigl\{ t > 0 \mid X_{t} = \bar{x},X_{0 -} = x \in ( 0,\bar{x} ] \bigr\} . $$

(73)

The Laplace’s transformation method (Appendix of Chap. 9, Dixit et al. [58]) gives the following exponential formula because our underlying process is a geometric Brownian motion:

$$ \mathrm{E}^{x} \bigl[ e^{ - \theta \tau _{x}} \bigr] = \biggl( \frac{x}{\bar{x}} \biggr)^{u} $$

(74)

with

$$ \lambda = \frac{2\mu }{\sigma ^{2}} - 1 ( > 0 )\quad \text{and}\quad u = \frac{1}{2}\sqrt{\lambda ^{2} + \frac{8\theta }{\sigma ^{2}}} - \frac{1}{2}\lambda $$

(75)

for each $\theta > 0$, where $\mathrm{E}^{x}$ is the conditional expectation on $X_{0 -} = x$. The formula of the probabilistically distributed x follows directly from (74) as

$$ \mathrm{E} \bigl[ e^{ - \theta \tau } \bigr] = \int _{a}^{b} g ( z )^{u}q ( z ) \,\mathrm{d}z\quad \text{with } g ( z ) = 1 - \frac{z\bar{\zeta }}{\bar{x}}, $$

(76)

where $\tau _{x}$ is rewritten as τ, which is now the first hitting time to the threshold x̄ just after an intervention whose exertion time is 0. We have

$$ \frac{\partial u}{\partial \theta } = \frac{2}{\sigma ^{2}} \biggl( \lambda ^{2} + \frac{8\theta }{\sigma ^{2}} \biggr)^{ - 1/2}\quad \text{and}\quad \frac{\partial ^{2}u}{\partial \theta ^{2}} = - \frac{8}{\sigma ^{4}} \biggl( \lambda ^{2} + \frac{8\theta }{\sigma ^{2}} \biggr)^{ - 3/2} $$

(77)

and thus

$$ u |_{\theta \to + 0} = 0,\qquad \frac{\partial u}{\partial \theta } \bigg|_{\theta \to + 0} = \frac{2}{\sigma ^{2}\lambda },\quad \text{and} \quad \frac{\partial^{2} u}{\partial \theta ^{2}} \bigg|_{\theta \to + 0} = - \frac{8}{\sigma ^{4}\lambda ^{3}}. $$

(78)

By (77) and (78), we get

$$ \mathrm{E} [ \tau ] = - \frac{\partial }{\partial \theta } \mathrm{E} \bigl[ e^{ - \theta \tau } \bigr] \bigg|_{\theta \to + 0} = - \frac{2}{\sigma ^{2}\lambda } \int _{a}^{b} q ( z )\ln g ( z ) \,\mathrm{d}z $$

(79)

and

$$ \mathrm{E} \bigl[ \tau ^{2} \bigr] = \frac{4}{\sigma ^{2}\lambda } \mathrm{E} [ \tau ] + \frac{4}{\sigma ^{4}\lambda ^{2}} \int _{a}^{b} q ( z )\ln ^{2}g ( z )\,\mathrm{d}z. $$

(80)

The equation (79) gives the formula for the mean $\mathrm{E} [ \tau ]$, and the formula for the variance $\mathrm{E} [ \tau ^{2} ] - ( \mathrm{E} [ \tau ] )^{2}$ immediately follows from (79) and (80). The standard deviation is then obtained as $\mathrm{Std} [ \tau ] = \sqrt{\mathrm{E} [ \tau ^{2} ] - ( \mathrm{E} [ \tau ] )^{2}}$. The integrals appearing in (79) and (80) are calculated exactly or evaluated numerically depending on the PDF q. Higher order moments of τ can also be computed based on (76) if necessary.

Another statistical index of interest is the mean harvested amount per unit time [21], which is denoted as H. This quantity can be simply estimated as

$$ H = \frac{\bar{\zeta } \int _{a}^{b} zq ( z )\,\mathrm{d}z}{\mathrm{E} [ \tau ]}, $$

(81)

where the numerator of the right-hand side is the mean unit-time harvesting. Validity of this formula is examined in the next section with a Monte-Carlo method.

4 Numerical computation

The HJBQVI (20) is computed with the two different numerical methods, which is the ODE-based method using (70) through (72) and a verified finite difference scheme having a formally first-order accuracy [44]. In addition, the statistical indices of the controlled state dynamics are calculated.

4.1 Parameter setting

A target species of the model application is the great cormorant Phalacrocorax carbo, which is one of the most widespread waterfowls found over the world whose population management has been serious ecological issues because of their excessively high predation to fishery resources [21, 59, 60]. They have indirect environmental impacts through modulating nutrient cycles around their habitat [61] through complex mechanisms. Nevertheless, exterminating the bird is not be the optimal solution because they are not alien species in many cases. Their population dynamics would present seasonal characteristics; however, from a practical modeling viewpoint, the dynamics can be effectively considered to be time-homogenous with the deterministic and stochastic growth as modelled in the SDE (8). Human interventions to suppress their population has been carried out in many ways, and the most popular one is gun-shooting [21, 36].

In this paper, the nominal model parameter values are specified as follows. For the population dynamics, we set $\mu = 0.50$ (1/year) and $\sigma = 0.50$ (1/year$^{1/2}$), considering the parameter estimation result in Japan [21] where $\mu = O ( 10^{ - 1} )$ (1/year), $\sigma = O ( 10^{ - 1} )$ (1/year$^{1/2}$), and $2\mu > \sigma ^{2}$. The other parameters are specified as follows: $\delta = 2$ (1/year), $m = 2$, $M = 0.5$, $r = 0.005$, $R = 10$, $k_{0} = 0.05$ (1/year), $k_{1} = 0.15$ (1/year), $a = 0.2$ and $b = 1$. The uniform distribution $q ( z ) = \frac{1}{b - a}$ ($a \le z \le b$) is assumed. The specified range $[ a,b ]$ is reasonably consistent with the survey result that the total number of harvested bird population at one harvesting event had different orders ($O ( 10^{0} )$ to $O ( 10^{2} )$) among hunters [36].

The computational domain for the finite difference scheme is $[ 0,X_{\max } ]$ with $X_{\max } = 100$, which is discretized into 1000 cells and 1001 nodes. The threshold $X_{\max }$ is determined so that $X_{\max } > \bar{x}$ in what follows based on preliminary numerical experiments. These parameter values and the computational resolution are used unless otherwise specified.

4.2 Value function and optimal policy

The HJBQVI (20) is discretized with the finite difference scheme [44] that has been found to be monotone, stable, consistent, and thus convergent in the viscosity sense under the strong comparison principle [62]. Proposition 5 supports applicability of the present scheme to discretization of the HJBQVI (20) in the truncated domain, the localized HJBQVI (28)–(29). Through comparing the computational results between the ODE-based method and the finite difference scheme, we examine validity of the functional form of the exact solution to the HJBQVI (20) conjectured in the previous section. Notice that the two numerical methods are based on fundamentally different concepts; the ODE-based method relies on the conjecture of the solution shape and the matching conditions at $x = \bar{x}$, while the finite difference scheme on a direct discretization of the localized HJBQVI (28)–(29). The scheme does not a priori use functional shapes of solutions in the computational domain. In the scheme, x̄ is detected between mid-points of each successive cells.

Table 1 shows a summary of comparison results between the numerical solution with the ODE-based method ($\Delta \rho = 0.00005$) and those with the finite difference scheme against different computational resolution. The auxiliary temporal integration is terminated when the absolute difference of the computed a, x̄, ξ̄ between each successive steps become smaller than 10⁻¹³. The ODE-based method computes $\bar{x} = 59.392$, $\bar{\xi } = 19.262$, and $c = - 0.0001307$, which are consistent with the assumptions $\bar{x} > \bar{\xi }$ and $c < 0$. Here, the value function computed with the ODE-based method is referred to as the numerical value function for the sake of brevity. In the table, the error (Er) between the numerical value function and each numerical solution computed with the finite difference scheme is the maximum error observed at vertices in $0 < x < \bar{x}$. The maximum error in each numerical solution was observed near $x = 0$ at which the value function is not differentiable in the classical sense because of the term $Bx^{M}$. Table 1 demonstrates that the numerical solutions with the finite difference scheme converge toward the numerical value function as the computational resolution increases. Convergence of x̄ and ξ̄ is not monotone, but it seems that the results between the two methods become closer as the resolution of the finite difference scheme becomes fine.

Figure 2

The numerical value function computed by the ODE-based method (green line) and the numerical solution by the finite difference scheme with 1,000 cells (blue line). The left vertical axis represents their values. The values of dashed vertical lines are $x = \bar{x} - \bar{\xi }$ and $x = \bar{x}$ from the left to the right, respectively. The lower-threshold $\underline{x} = \underline{x} ( x )$ ($\bar{x} \le x \le X_{\max }$) is also plotted in the figure (red line). The right vertical axis represents its value

Table 1 Summary of comparison results between the ODE-based method and finite difference scheme. The total number of cells, cell size, the error Er, x̄, and ξ̄ by the finite difference scheme are presented. With the ODE-based method, we have $\bar{x} = 59.392$ and $\bar{\xi } = 19.262$

Figure 2 presents the numerical value function ($0 \le x \le \bar{x}$) and the numerical solution to the finite difference scheme with 1000 cells ($0 \le x \le X_{\max }$). They agree well with each other as observed in the figure. Furthermore, the numerical solution does not contain spurious oscillations, demonstrating its satisfactory ability to handle the non-linear and non-local problem. The computed solution shape is consistent with that assumed in Proposition 6. The lower-threshold $\underline{x} = \underline{x} ( x )$ ($\bar{x} \le x \le X_{\max }$), which equals $\bar{x} - \xi ^{*} ( x )$, is also plotted in Fig. 2, showing that $\underline{x} ( x )$ clearly depends on the population x as suggested in Sect. 3 where a difference between the intervention operators of the conventional and present model has been notified. Figure 3 shows that the convergence of the numerical solutions toward the value function is close to $O ( h^{1/2} )$, where h represents the cell size. In fact, the least-squares best-fitted curve is $\mathrm{Er} = 0.0126 \times h^{0.497}$. This convergence speed agrees with that of the other monotone numerical schemes for HJBQVIs in impulse and related control problems [62, 63]. Furthermore, we examine validity of Remark 5. We get $\frac{\mathrm{d}\Phi }{\mathrm{d}x} = dk_{1} = 0.250$ theoretically, while the finite difference approximation leads to $\frac{ \Phi |_{x = X_{\max }} - \Phi |_{x = 0.999X_{\max }}}{0.001X_{\max }} = 0.251$, demonstrating an agreement between the theoretical and numerical results. In summary, the results with both numerical methods are in good agreement with each other. They agree well with the theoretical results as well. In what follows, parameter dependence of the optimal policy with the finite difference scheme is analyzed.

Figure 3

The error Er and the cell size h: circles represent the computational results and the curve represents the least-squares best-fitted curve

Table 2 presents the parameter dependence of the thresholds x̄, $\underline{x} = \underline{x} ( \bar{x} )$, and the amount of harvesting $\bar{\xi } = \bar{x} - \underline{x}$. Figures 4 through 12 graphically present the parameter dependence of x̄, $\underline{x} = \underline{x} ( \bar{x} )$, and ξ̄ on the parameters μ, σ, δ, r, R, $k_{0}$, $k_{1}$, a, and b, respectively. The computational results in Table 2 and Figs. 4 through 12 demonstrate monotone dependence of x̄, $\underline{x}$, and ξ̄ on the model parameters. The computational results suggest that increasing the stochasticity σ or the deterministic growth rate μ leads to a larger threshold values that potentially result in managing the dynamics with higher population. The increasing/decreasing nature of ξ̄ with respect to these parameters are contrasting, but Figs. 4 and 5 imply relatively smaller changes of ξ̄ compared with the two threshold values. The dependence of the optimal policy on the discount rate δ and the parameters R and r for the utility and disutility shows that the decision-maker with a longer-term perspective maintains the population at a higher level. In addition, the decision-maker who evaluates the utility (resp., disutility) larger (resp., smaller) seems to choose larger (smaller) thresholds and leads to larger (smaller) population. The parameter dependence on the fixed $k_{0}$ and proportional costs $k_{1}$ are qualitatively the same for the upper threshold x̄ and the harvesting amount ξ̄, suggesting that incurring a higher cost result in less frequent but more drastic harvesting. This is due to that executing small harvesting several times are rather costly than choosing a more drastic harvesting. The dependence between $k_{0}$ and $k_{1}$ is different for the lower threshold as suggested in Table 2, implying that incurring a higher fixed cost results in choosing a smaller $\underline{x}$. On the other hand, incurring a higher proportional cost results in choosing a larger $\underline{x}$. This difference can be explained by the different nature of the two costs. Increasing the fixed cost induces less frequent interventions because this cost is the same at each intervention, while higher proportional cost induces smaller amount of harvesting because this cost is positively proportional to the harvested amount of the population. These results are consistent with the model without uncertainty [44], demonstrating a qualitative linkage between the present and previous models.

Figure 4

Dependence of x̄ (red), $\underline{x} = \underline{x} ( \bar{x} )$ (blue), and ξ̄ (green) on μ

Figure 5

Dependence of x̄ (red), $\underline{x} = \underline{x} ( \bar{x} )$ (blue), and ξ̄ (green) on σ

Figure 6

Dependence of x̄ (red), $\underline{x} = \underline{x} ( \bar{x} )$ (blue), and ξ̄ (green) on δ

Figure 7

Dependence of x̄ (red), $\underline{x} = \underline{x} ( \bar{x} )$ (blue), and ξ̄ (green) on r

Figure 8

Dependence of x̄ (red), $\underline{x} = \underline{x} ( \bar{x} )$ (blue), and ξ̄ (green) on R

Figure 9

Dependence of x̄ (red), $\underline{x} = \underline{x} ( \bar{x} )$ (blue), and ξ̄ (green) on $k_{0}$

Figure 10

Dependence of x̄ (red), $\underline{x} = \underline{x} ( \bar{x} )$ (blue), and ξ̄ (green) on $k_{1}$

Table 2 Parameter dependence of the thresholds x̄, $x = x ( \bar{x} )$, and the amount of harvesting $\bar{\xi } = \bar{x} - x$. In this table, the notations “+” and “−” mean means increase and decrease of x̄, x, or ξ̄ with respect to each parameter, respectively

Finally, Figs. 11 and 12 on the parameters a and b suggest that increasing the lower-bound (resp., upper-bound) of uncertainties through increasing a (resp., b) lowers the thresholds x̄ and $\underline{x}$, and further the amount of harvesting ξ̄. This is due to that the harvested amount is the multiplicative form zξ̄, meaning that a larger ξ̄ is necessary to efficiently suppress the population with a possibly smaller z. A practical implication of this observation is that increasing the mean $\frac{a + b}{2}$ would be a key to effectively suppress the population at a lower level with smaller harvesting effort. Collecting empirical relationship between the harvesting effort and the realized impacts of the interventions can potentially make the range $[ a,b ]$ of the uncertainties smaller or its means larger, so that the decision-maker can better manage and predict the controlled population dynamics. However, in practice, such an effort may require costly additional field surveys, like performance tests of the interventions [64] and behavioral analysis of the bird in the habitat [65]. Our computational results suggest that such preliminary surveys should be carried out if they are evaluated as less costly than taking interventions under the uncertainties. If it is not the case, a cost term on a preliminary field survey must be incorporated into the performance index, leading to an advanced optimization problem. Importance of reducing and evaluating the uncertainties has also been suggested in a research with empirical models [66]. Our computational results support their suggestion.

Figure 11

Dependence of x̄ (red), $\underline{x} = \underline{x} ( \bar{x} )$ (blue), and ξ̄ (green) on a

Figure 12

Dependence of x̄ (red), $\underline{x} = \underline{x} ( \bar{x} )$ (blue), and ξ̄ (green) on b

4.3 Statistical indices

Here, we focus on parameter dependence of statistical indices on the controlled state dynamics. Before the analysis, we check their validity through a comparison with a numerical result by a standard Monte-Carlo combined with the classical Euler–Maruyama discretization of the optimally-controlled SDE. For the nominal parameter values, the formulae (79), (80), and (81) lead to $\mathrm{E} [ \tau ] = 0.5887$, $\mathrm{Std} [ \tau ] = 1.0530$, and $H = 19.63$. On the other hand, the numerical counterparts with the Monte-Carlo method (10⁹ sample points, the time increment 0.00001) are $\mathrm{E} [ \tau ] = 0.5898$, $\mathrm{Std} [ \tau ] = 1.0422$, and $H = 19.47$. The obtained results demonstrate a reasonable agreement between the theoretical and numerical results and suggest validity of these analytical formulae.

The parameter dependence of the optimally-controlled dynamics on the other parameters are summarized in Table 3, suggesting monotone dependence of the statistical indices of the dynamics on the parameters. Especially the dependence of $\mathrm{E} [ \tau ]$ and $\mathrm{Std} [ \tau ]$ on each parameter is qualitatively the same, suggesting that increase or decrease of the mean and standard deviation is coherent under the optimal policy.

Table 3 Parameter dependence of the $\mathrm{E} [ \tau ]$, $\mathrm{Std} [ \tau ]$, and H The notations “+” and “−” mean increase and decrease of these indices

The quantitative analysis in what follows puts a focus on parameter dependence of the optimally-controlled dynamics on the ecological parameters μ and σ. Figures 13 and 14 show the computed $\mathrm{E} [ \tau ]$, $\mathrm{Std} [ \tau ]$, and H for different values of μ and σ, respectively. The results imply a critical importance of identifying the model parameters especially when μ is relatively small and/or σ is relatively large. Because of the monotone and convex dependence as plotted in Figs. 13 and 14. Such a situation is in practice encountered when the population dynamics is highly stochastic where the ratio $\mu \sigma ^{ - 2}$ is large. The standard deviation $\mathrm{Std} [ \tau ]$ is more affected by the parameter values than the mean $\mathrm{E} [ \tau ]$, implying that the optimal timing of the harvesting timing may be highly variable when the deterministic growth rate is small.

Figure 13

Dependence of $\mathrm{E} [ \tau ]$ (red), $\mathrm{Std} [ \tau ]$ (blue), and H (green) on μ

Figure 14

Dependence of $\mathrm{E} [ \tau ]$ (red), $\mathrm{Std} [ \tau ]$ (blue), and H (green) on σ

Statistical analysis focusing on the uncertainty of the interventions is also carried out by examining parameter dependence of the statistical indices on a and b. Figures 15 and 16 demonstrate that reducing the potential uncertainties of interventions through increasing of a or decreasing b can effectively reduce both the mean and standard deviation of the harvesting frequency, and further the unit-time harvesting H. This observation combined with Figs. 11 and 12 on the parameter dependence of the management thresholds leads to a key implication that reducing the range and/or mean of the uncertainties is indispensable for cost-effective and less frequent interventions to suppress the population. This observation found through the present model emphasizes crucial importance of accurately estimating the intervention impacts in environmental and ecological management.

Figure 15

Dependence of $\mathrm{E} [ \tau ]$ (red), $\mathrm{Std} [ \tau ]$ (blue), and H (green) on a

Figure 16

Dependence of $\mathrm{E} [ \tau ]$ (red), $\mathrm{Std} [ \tau ]$ (blue), and H (green) on b

5 Conclusions

An impulse control model of a 1-D diffusion process with imperfect interventions was formulated, and the associated HJBQVI was presented with its solvability and verification results. A candidate exact solution to the HJBQVI was found and a system of nonlinear equations to determine its coefficients was derived. The system was solved numerically and the coefficients were found successfully. In addition, the (localized) HJBQVI was discretized with a finite difference scheme, demonstrating that the solutions generated by the determined coefficients and that by the scheme agreed well. Application of the scheme to the HJBQVI revealed both qualitative and quantitative parameter dependence of the optimal policy and the controlled dynamics extensively. The parameter dependence was found to be monotone for the ranges of the examined parameter values. Statistical analysis on the controlled dynamics demonstrated importance of accurately identifying the ecological parameters involved in the SDE.

Our model was possibly the simplest one, and can be extended to generalized models. Population dynamics may be subject to nonlinear [67, 68] and jump disturbances [69, 70]. A more general form of uncertainty in the interventions can be considered. In practical problems, different types of imperfectness would be encountered. An example is the model uncertainty in the state dynamics itself, which has conventionally been discussed in the framework of the differential game between the decision-maker and nature [71, 72]. The modern poorly known dynamics approach can also harmonize with the present framework [73]. The filtering problem as an identification through doing problem would be encountered if the system dynamics has a parameter whose value is not known a priori [74, 75]. It is not always possible to obtain a continuous information flow of the system dynamics but discrete and costly observations may be possible [76, 77]. Delayed execution of control policy is also an important element in realistic problems [78]. The present model is a building block for approaching these advanced issues. Models based on singular-like controls [79] can also be considered in the context of the intervention uncertainty discussed in this paper.

Appendix:  Proofs of propositions

Proof of Proposition 1

With the null control $\eta = \eta _{0}$, we have

$$ \Phi ( x ) \ge - R\mathrm{E} \biggl[ \int _{0}^{\infty } e^{ - \delta s}X_{s}^{M} \,\mathrm{d}s \biggr] = Bx^{m}. $$

(82)

Combining (82), (17), and (18) yields (23). The last statement directly follows from (23). □

Proof of Proposition 2

By (19), considering a policy immediately harvesting the population with a fixed amount of ζ such that $0 < \zeta \le x$, which is admissible and sub-optimal, yields

$$ \Phi ( x ) \le \mathrm{E} \bigl[ K ( \zeta ) + \Phi ( X_{0} ) \bigr] = K ( \zeta ) + \int _{a}^{b} \Phi ( x - z\zeta )q ( z ) \,\mathrm{d}z. $$

(83)

Taking the infimum of the right-hand side of (83) with respect to $0 < \zeta \le x$ yields (24). □

Proof of Proposition 3

Given $( B_{t} )_{t \ge 0}$ and Z, the process $( X_{t} )_{t \ge 0}$ starting from the initial condition is denoted as $( X_{t}^{ ( x )} )_{t \ge 0}$. Set x, y such that $0 \le y \le x$. Choose one $\eta \in \mathcal{A}_{y}$. Notice that we have $\mathcal{A}_{y} \subset \mathcal{A}_{x}$ because of $0 \le y \le x$. The process $Y_{t} = X_{t}^{ ( x )} - X_{t}^{ ( y )}$ is continuous at each $\tau _{i}$ because

$$ \begin{aligned} Y_{\tau _{i}} &= X_{\tau _{i}}^{ ( x )} - X_{\tau _{i}}^{ ( y )} \\ &= X_{\tau _{i} -}^{ ( x )} - z_{i}\zeta - \bigl( X_{\tau _{i} -}^{ ( y )} - z_{i}\zeta \bigr) \\ &= X_{\tau _{i} -}^{ ( x )} - X_{\tau _{i} -}^{ ( y )} \\ &= Y_{\tau _{i} -}. \end{aligned} $$

(84)

Therefore, $( Y_{t} )_{t \ge 0}$ is a classical geometric Brownian motion:

$$ \mathrm{d}Y_{t} = Y_{t} ( \mu \,\mathrm{d}t + \sigma \,\mathrm{d}B_{t} ),\quad t \ge 0 $$

(85)

with the initial condition $Y_{0 -} = x - y \ge 0$. We get

$$ \mathrm{E} \bigl[ Y_{t}^{l} \bigr] = \mathrm{E} \bigl[ \bigl\vert X_{t}^{ ( x )} - X_{t}^{ ( y )} \bigr\vert ^{l} \bigr] \le \exp ^{ ( l\mu + \frac{1}{2}l ( l - 1 )\sigma ^{2} )t} \vert x - y \vert ^{l},\quad x,y \ge 0, t \ge 0 $$

(86)

for any $l > 0$. For any $x,y, 0 \le y \le x$, we have

$$ \begin{aligned} \bigl\vert J ( x;\eta ) - J ( y;\eta ) \bigr\vert &= \biggl\vert \mathrm{E} \biggl[ \int _{0}^{\infty } e^{ - \delta s} \bigl( - R \bigl( X_{s}^{ ( x )} \bigr)^{M} + r \bigl( X_{s}^{ ( x )} \bigr)^{m} \bigr)\,\mathrm{d}s \\ &\quad {}- \int _{0}^{\infty } e^{ - \delta s} \bigl( - R \bigl( X_{s}^{ ( y )} \bigr)^{M} + r \bigl( X_{s}^{ ( y )} \bigr)^{m} \bigr)\,\mathrm{d}s \biggr] \biggr\vert \\ &\le R\mathrm{E} \biggl[ \int _{0}^{\infty } e^{ - \delta s} \bigl\vert \bigl( X_{s}^{ ( x )} \bigr)^{M} - \bigl( X_{s}^{ ( y )} \bigr)^{M} \bigr\vert \,\mathrm{d}s \biggr] \\ &\quad {}+ r\mathrm{E} \biggl[ \int _{0}^{\infty } e^{ - \delta s} \bigl\vert \bigl( X_{s}^{ ( x )} \bigr)^{m} - \bigl( X_{s}^{ ( y )} \bigr)^{m} \bigr\vert \,\mathrm{d}s \biggr]. \end{aligned} $$

(87)

Notice that

$$ \bigl\vert \bigl( X_{s}^{ ( x )} \bigr)^{M} - \bigl( X_{s}^{ ( y )} \bigr)^{M} \bigr\vert \le \bigl\vert X_{s}^{ ( x )} - X_{s}^{ ( y )} \bigr\vert ^{M} = Y_{s}^{M} $$

(88)

by an elementary calculation, and that

$$ \begin{aligned} \bigl\vert \bigl( X_{s}^{ ( x )} \bigr)^{m} - \bigl( X_{s}^{ ( y )} \bigr)^{m} \bigr\vert &\le m\max \bigl\{ X_{s}^{ ( x )},X_{s}^{ ( y )} \bigr\} ^{m - 1} \bigl\vert X_{s}^{ ( x )} - X_{s}^{ ( y )} \bigr\vert \\ &= m\max \bigl\{ X_{s}^{ ( x )},X_{s}^{ ( y )} \bigr\} ^{m - 1}Y_{s} \\ &= m \bigl( X_{s}^{ ( x )} \bigr)^{m - 1}Y_{s} \\ &\le m \bigl( X_{s,0}^{ ( x )} \bigr)^{m - 1}Y_{s}. \end{aligned} $$

(89)

Here, $X_{s,0}^{ ( x )}$ represents $X_{s}^{ ( x )}$ with the null control $\eta = \eta _{0}$, which is a classical geometric Brownian motion. Therefore, $( X_{s,0}^{ ( x )} )^{m - 1}Y_{s}$ is a geometric Brownian motion subject to the initial condition $x^{m - 1} \vert x - y \vert$. By (86), we get with some $C > 0$:

$$ \bigl\vert J ( x;\eta ) - J ( y;\eta ) \bigr\vert \le C \bigl( ( x - y )^{M} + x^{m - 1} \vert x - y \vert \bigr) $$

(90)

and thus

$$ J ( x;\eta ) \le J ( y;\eta ) + C \bigl( ( x - y )^{M} + x^{m - 1} \vert x - y \vert \bigr). $$

(91)

Taking the infimum with respect to $\eta \in \mathcal{A}_{y}$ gives

$$ \inf_{\eta \in \mathcal{A}_{y}}J ( x;\eta ) \le \Phi ( y ) + C \bigl( ( x - y )^{M} + x^{m - 1} \vert x - y \vert \bigr). $$

(92)

In addition, we have

$$ \inf_{\eta \in \mathcal{A}_{y}}J ( x;\eta ) \ge \inf_{\eta \in \mathcal{A}_{x}}J ( x;\eta ) = \Phi ( x ) $$

(93)

and thus

$$ \Phi ( x ) - \Phi ( y ) \le C \bigl( ( x - y )^{M} + x^{m - 1} \vert x - y \vert \bigr),\quad 0 \le y \le x. $$

(94)

The continuity at the origin is the consequence of Proposition 1. □

Proof of Proposition 4

The proof here is based on the technique of Federico et al. [80]. Firstly, we show that Φ is a viscosity sub-solution. The condition at $x = 0$ is obviously satisfied. Fix $x_{0} > 0$ and set a test function ψ for viscosity sub-solutions such that $\Phi ( x_{0} ) = \psi ( x_{0} )$ and $\psi > \Phi$ in $( x_{0} - \varepsilon ,x_{0} + \varepsilon )$ with a small constant $\varepsilon > 0$. Set the stopping time $\bar{\tau } = \inf \{ t \ge 0 | \vert X_{t} - x_{0} \vert > \varepsilon \}$, which is adapted to the filtration $\mathcal{F}$ and satisfies $\Pr [ \bar{\tau } > 0 ] = 1$. Set $\tau = \min \{ \bar{\tau },\varepsilon _{0} \}$ with $\varepsilon _{0} > 0$. Then, by (19), with the null control $\eta = \eta _{0}$, we have

$$ \begin{aligned} \psi ( x_{0} ) &= \Phi ( x_{0} ) \\ &\le \mathrm{E} \biggl[ \int _{0}^{\tau } e^{ - \delta s} \bigl( - RX_{s}^{M} + rX_{s}^{m} \bigr) \,\mathrm{d}s + e^{ - \delta \tau } \Phi ( X_{\tau } ) \biggr] \\ &\le \mathrm{E} \biggl[ \int _{0}^{\tau } e^{ - \delta s} \bigl( - RX_{s}^{M} + rX_{s}^{m} \bigr) \,\mathrm{d}s + e^{ - \delta \tau } \psi ( X_{\tau } ) \biggr]. \end{aligned} $$

(95)

By the Dynkin’s formula, dividing the both-hand sides by $\varepsilon _{0}$ and letting $\varepsilon _{0} \to + 0$ with the dominated convergence theorem yields

$$ \mathcal{L}\psi + Rx^{M} - rx^{m} \le 0\quad \text{at } x = x_{0}. $$

(96)

The inequality

$$ \Phi \le \mathcal{M}\Phi \quad \text{at } x = x_{0} $$

(97)

follows from Proposition 2, and combining (96) and (97) gives the desired inequality. The property of viscosity sub-solution then follows because $x_{0} > 0$ is arbitrary.

Secondly, we show that Φ is a viscosity super-solution. The condition at $x = 0$ is obviously satisfied. Fix $x_{0} > 0$ and set a test function ψ for viscosity super-solutions such that $\Phi ( x_{0} ) = \psi ( x_{0} )$ and $\psi < \Phi$ near $x = x_{0}$. If $\Phi ( x_{0} ) = \mathcal{M}\Phi ( x_{0} )$, then we have nothing to prove and thus assume $\Phi ( x_{0} ) < \mathcal{M}\Phi ( x_{0} )$. Then, there is a constant $\omega > 0$ such that $\Phi ( x_{0} ) + \omega \le \mathcal{M}\Phi ( x_{0} )$. We must show

$$ \mathcal{L}\psi + Rx^{M} - rx^{m} \ge 0\quad \text{at }x = x_{0}. $$

(98)

Assume (98) is not true. Then, there exists a constant $\varepsilon > 0$ such that $\mathcal{L}\psi + Rx^{M} - rx^{m} \le - \varepsilon$ at $x = x_{0}$. Because of the smoothness of ψ and continuity of Φ and $\mathcal{M}\Phi$ (By an application of Proposition 3 of Azimzadeh et al. [42] to $\mathcal{M}$), we have $2\delta _{0} \in ( 0,x_{0} )$ such that

$$ \textstyle\begin{cases} \mathcal{L}\psi + Rx^{M} - rx^{m} \le - \varepsilon /2 \\ \Phi \ge \psi \\ \Phi \le \mathcal{M}\Phi - \omega /2 \end{cases}\displaystyle \quad \text{for all } x_{0} - 2\delta _{0} \le x \le x_{0} + 2\delta _{0}, x \ne x_{0}. $$

(99)

Set the stopping time $\tau = \inf \{ t \ge 0 \mid \vert X_{t} - x_{0} \vert > \delta _{0} \}$ with $\Pr [ \tau > 0 ] = 1$. The third line of (99) implies that undertaking an intervention is not optimal during $( 0,\hat{\tau } )$. By (19), we have

$$ \begin{aligned} \psi ( x_{0} ) &= \Phi ( x_{0} ) \\ &= \mathrm{E} \biggl[ \int _{0}^{\tau } e^{ - \delta s} \bigl( - RX_{s}^{M} + rX_{s}^{m} \bigr) \,\mathrm{d}s + e^{ - \delta \tau } \Phi ( X_{\tau } ) \biggr] \\ &\ge \mathrm{E} \biggl[ \int _{0}^{\tau } e^{ - \delta s} \bigl( - RX_{s}^{M} + rX_{s}^{m} \bigr) \,\mathrm{d}s + e^{ - \delta \tau } \psi ( X_{\tau } ) \biggr]. \end{aligned} $$

(100)

Again, by the Dynkin’s formula and (100), we have

$$ \begin{aligned} &\mathrm{E} \biggl[ \int _{0}^{\tau } e^{ - \delta s} \bigl( \mathcal{L}\psi ( X_{s} ) + RX_{s}^{M} - rX_{s}^{m} \bigr)\,\mathrm{d}s \biggr] \\ &\quad = \psi ( x_{0} ) - \mathrm{E} \biggl[ \int _{0}^{\tau } e^{ - \delta s} \bigl( - RX_{s}^{M} + rX_{s}^{m} \bigr) \,\mathrm{d}s + e^{ - \delta \tau } \psi ( X_{\tau } ) \biggr] \\ &\quad = \Phi ( x_{0} ) - \mathrm{E} \biggl[ \int _{0}^{\tau } e^{ - \delta s} \bigl( - RX_{s}^{M} + rX_{s}^{m} \bigr) \,\mathrm{d}s + e^{ - \delta \tau } \psi ( X_{\tau } ) \biggr] \\ &\quad \ge \Phi ( x_{0} ) - \mathrm{E} \biggl[ \int _{0}^{\tau } e^{ - \delta s} \bigl( - RX_{s}^{M} + rX_{s}^{m} \bigr) \,\mathrm{d}s + e^{ - \delta \tau } \Phi ( X_{\tau } ) \biggr] \\ &\quad = 0. \end{aligned} $$

(101)

By the first line of (99), from (101) we get

$$ 0 \le \mathrm{E} \biggl[ \int _{0}^{\tau } \biggl( - \frac{\varepsilon }{2} \biggr)e^{ - \delta s}\,\mathrm{d}s \biggr] = - \frac{\varepsilon }{2}\mathrm{E} \biggl[ \int _{0}^{\tau } e^{ - \delta s}\,\mathrm{d}s \biggr] = - \frac{\varepsilon }{2}\frac{1 - \mathrm{E} [ e^{ - \delta \tau } ]}{\delta }, $$

(102)

and thus

$$ 1 \le \mathrm{E} \bigl[ e^{ - \delta \tau } \bigr]. $$

(103)

However, the inequality is possible (103) only when $\tau = 0$ a.s., which is a contradiction. Therefore, we have (98). The property of viscosity super-solution is then proven because $x_{0} > 0$ is arbitrary. □

Proof of Proposition 5

The proof is based on that of Theorem 3.1 of Ishii [81] where the comparison principle of a HJBQVI is discussed by constructing a strict viscosity-supersolution that satisfies the inequalities of viscosity sub-solutions (in our case, (30) and (31)) with “<” instead of “≤”.

The boundary condition is different between the two problems, where ours is found to be simpler. The degenerate elliptic part $\mathcal{L}\psi + Rx^{M} - rx^{m}$ is not problematic because its coefficients are bounded and smooth in $[ 0,X_{\max } ]$. Therefore, we can apply the fundamental lemma (Example 1 of Crandall and Ishii [82]). The non-local part $\varphi - \mathcal{M}\varphi \ge 0$ has a somewhat different form from the conventional ones as pointed out above because of the uncertainty involved in the interventions. We can directly check that the followings, which correspond to the conditions (1)–(6) in Proposition 2.3 of Ishii [81]: let u, v be real-valued functions defined on $[ 0,X_{\max } ]$, then we have

• (Monotonicity) In $[ 0,X_{\max } ]$, if $u \le v$ then $\mathcal{M}u \le \mathcal{M}v$,

• (Concavity) $c\mathcal{M}u + ( 1 - c )\mathcal{M}v \le \mathcal{M} ( cu + ( 1 - c )v )$ for all $0 \le c \le 1$,

• (Transversity) $\mathcal{M} ( u + c ) = \mathcal{M}u + c$ for all $c \in \mathbb{{R}}$,

• (Upper-semi continuity) If $u \in \operatorname{USC} [ 0,X_{\max } ]$, then $\mathcal{M}u \in \operatorname{USC} [ 0,X_{\max } ]$,

• (Lower-semi continuity) If $u \in \operatorname{LSC} [ 0,X_{\max } ]$, then $\mathcal{M}u \in \operatorname{LSC} [ 0,X_{\max } ]$, and

• (Continuity) $\Vert \mathcal{M}u - \mathcal{M}v \Vert _{C [ 0,X_{\max } ]} \to 0$ as $\Vert u - v \Vert _{C [ 0,X_{\max } ]} \to 0$.

Set $C = r\delta ^{ - 1}X_{\max }^{m} + 1$ and $\alpha = \min \{ 1,k_{0} \} > 0$. Let u and v be a viscosity sub-solution and a viscosity super-solution, respectively. For each $n \in \mathbb{{N}}$, set $u_{n} = ( 1 - \frac{1}{n} )u - \frac{C}{n}$, which is straightforwardly checked to be a (strict) viscosity sub-solution to

$$ \max \bigl\{ \mathcal{L}\Phi + Rx^{M} - rx^{m},\Phi - \mathcal{M}\Phi \bigr\} + \frac{\alpha }{n} = 0,\quad 0 < x < X_{\max } $$

(104)

and

$$ \max \{ \Phi - g,\Phi - \mathcal{M}\Phi \} + \frac{\alpha }{n} = 0,\quad x = X_{\max }. $$

(105)

From this observation, we see that $u_{n}$ is a strict viscosity sub-solution to the localized HJBQVI. Constructing strict viscosity solution is necessary to handle the non-local term. The remaining part of the proof follows Ishii [81]. □

Proof of Proposition 6

The proof here is based on that of Theorem 1 of Wu [83]. We must modify their proof in several parts because our model does not have jump but has an uncertainty in interventions. By Proposition 1 for any $\eta \in \mathcal{A}$ and $x \ge 0$, we get

$$ B\mathrm{E} \bigl[ e^{ - \delta t}X_{t}^{M} \bigr] \le \mathrm{E} \bigl[ e^{ - \delta t}\phi ( X_{t} ) \bigr] \le A \mathrm{E} \bigl[ e^{ - \delta t}X_{t}^{m} \bigr], \quad t \ge 0, x \ge 0. $$

(106)

Let $( \bar{X}_{t} )_{t \ge 0}$ be the process $( X_{t} )_{t \ge 0}$ with the null control $\eta = \eta _{0}$. Then, we have $\bar{X}_{t} \ge X_{t}$ a.s. for $t \ge 0$. Therefore, we get

$$ \begin{aligned} &\lim_{t \to + \infty } B\mathrm{E} \bigl[ e^{ - \delta t}X_{t}^{M} \bigr] \ge \lim_{t \to + \infty } B\mathrm{E} \bigl[ e^{ - \delta t} \bar{X}_{t}^{M} \bigr] = 0\quad \text{and} \\ &\lim _{t \to + \infty } A\mathrm{E} \bigl[ e^{ - \delta t}X_{t}^{m} \bigr] \le \lim_{t \to + \infty } A\mathrm{E} \bigl[ e^{ - \delta t} \bar{X}_{t}^{m} \bigr] = 0, \end{aligned} $$

(107)

leading to

$$ 0 = \lim_{t \to + \infty } B\mathrm{E} \bigl[ e^{ - \delta t} \bar{X}_{t}^{M} \bigr] \le \mathrm{E} \bigl[ e^{ - \delta t}\phi ( X_{t} ) \bigr] \le \lim _{t \to + \infty } A\mathrm{E} \bigl[ e^{ - \delta t} \bar{X}_{t}^{m} \bigr] = 0 $$

(108)

and thus

$$ \lim_{t \to + \infty } \mathrm{E} \bigl[ e^{ - \delta t}\phi ( X_{t} ) \bigr] = 0. $$

(109)

Here, recall that $B < 0$ and $A > 0$. For any $t \ge 0$, set $\theta _{i + 1} = \max \{ \tau _{i},\min \{ \tau _{i + 1},t \} \}$. The generalized Dynkin’s formula then leads to

$$ \mathrm{E} \bigl[ e^{ - \delta \theta _{i + 1}}\phi ( X_{\theta _{i + 1} -} ) \bigr] = \mathrm{E} \bigl[ e^{ - \delta \tau _{i}}\phi ( X_{\tau _{i}} ) \bigr] - \mathrm{E} \biggl[ \int _{\tau _{i}}^{\theta _{i + 1}} e^{ - \delta s}\mathcal{L}\phi ( X_{s} )\,\mathrm{d}s \biggr] $$

(110)

because $\tau _{i} \le \theta _{i + 1} \le \tau _{i + 1}$. Letting $t \to + \infty$ in (110), using the dominated convergence theorem, and summing $i = 0$ from $i = n \in \mathbb{{N}}$ yields

$$ \begin{aligned} \sum_{i = 0}^{n} \mathrm{E} \bigl[ e^{ - \delta \tau _{i + 1}}\phi ( X_{\tau _{i + 1} -} ) \bigr] &= \sum _{i = 0}^{n} \mathrm{E} \bigl[ e^{ - \delta \tau _{i}}\phi ( X_{\tau _{i}} ) \bigr] - \sum _{i = 0}^{n} \mathrm{E} \biggl[ \int _{\tau _{i}}^{\tau _{i + 1}} e^{ - \delta s}\mathcal{L}\phi ( X_{s} )\,\mathrm{d}s \biggr] \\ &= \phi ( x ) + \sum_{i = 1}^{n} \mathrm{E} \bigl[ e^{ - \delta \tau _{i}}\phi ( X_{\tau _{i}} ) \bigr] - \mathrm{E} \biggl[ \int _{0}^{\tau _{n + 1}} e^{ - \delta s}\mathcal{L}\phi ( X_{s} )\,\mathrm{d}s \biggr]. \end{aligned} $$

(111)

We can rewrite (111) as

$$ \begin{aligned} \mathrm{E} \bigl[ e^{ - \delta \tau _{n + 1}}\phi ( X_{\tau _{n + 1} -} ) \bigr] = {}&\phi ( x ) + \sum_{i = 1}^{n} \mathrm{E} \bigl[ e^{ - \delta \tau _{i}} \bigl( \phi ( X_{\tau _{i}} ) - \phi ( X_{\tau _{i} -} ) \bigr) \bigr] \\ &{}- \mathrm{E} \biggl[ \int _{0}^{\tau _{n + 1}} e^{ - \delta s}\mathcal{L}\phi ( X_{s} )\,\mathrm{d}s \biggr]. \end{aligned} $$

(112)

By (41) and (42), we have

$$ \mathrm{E} \biggl[ \int _{0}^{\tau _{n + 1}} e^{ - \delta s}\mathcal{L}\phi ( X_{s} )\,\mathrm{d}s \biggr] \le \mathrm{E} \biggl[ \int _{0}^{\tau _{n + 1}} e^{ - \delta s} \bigl( rX_{s}^{m} - RX_{s}^{M} \bigr) \,\mathrm{d}s \biggr]. $$

(113)

In addition, we have

$$ \mathrm{E} \bigl[ e^{ - \delta \tau _{i}} \bigl( \phi ( X_{\tau _{i}} ) - \phi ( X_{\tau _{i} -} ) \bigr) \bigr] \ge - \mathrm{E} \bigl[ e^{ - \delta \tau _{i}}K ( \zeta _{i} ) \bigr],\quad i \ge 1. $$

(114)

This follows from (41) and (42) because for $i \ge 1$ we have

$$ e^{ - \delta \tau _{i}}\phi ( X_{\tau _{i} -} ) \le e^{ - \delta \tau _{i}} \int _{a}^{b} g ( z_{i} )\phi ( X_{\tau _{i} -} - z_{i}\zeta _{i} ) \,\mathrm{d}z_{i} + e^{ - \delta \tau _{i}}K ( \zeta _{i} ), $$

(115)

and then taking the expectation yields (114) since

$$ \begin{aligned} \mathrm{E} \bigl[ e^{ - \delta \tau _{i}}\phi ( X_{\tau _{i} -} ) \bigr] - \mathrm{E} \bigl[ e^{ - \delta \tau _{i}}K ( \zeta _{i} ) \bigr] &\le \mathrm{E} \biggl[ e^{ - \delta \tau _{i}} \int _{a}^{b} g ( z_{i} )\phi ( X_{\tau _{i} -} - z_{i}\zeta _{i} ) \,\mathrm{d}z_{i} \biggr] \\ &= \mathrm{E} \bigl[ e^{ - \delta \tau _{i}}\phi ( X_{\tau _{i}} ) \bigr]. \end{aligned} $$

(116)

Substituting (113) and (114) into (112) yields

$$ \begin{aligned} &\mathrm{E} \bigl[ e^{ - \delta \tau _{n + 1}}\phi ( X_{\tau _{n + 1} -} ) \bigr] \\ &\quad = \phi ( x ) + \sum_{i = 1}^{n} \mathrm{E} \bigl[ e^{ - \delta \tau _{i}} \bigl( \phi ( X_{\tau _{i}} ) - \phi ( X_{\tau _{i} -} ) \bigr) \bigr] - \mathrm{E} \biggl[ \int _{0}^{\tau _{n + 1}} e^{ - \delta s}\mathcal{L}\phi ( X_{s} )\,\mathrm{d}s \biggr] \\ &\quad \ge \phi ( x ) + \sum_{i = 1}^{n} \mathrm{E} \bigl[ e^{ - \delta \tau _{i}} \bigl( \phi ( X_{\tau _{i}} ) - \phi ( X_{\tau _{i} -} ) \bigr) \bigr] - \mathrm{E} \biggl[ \int _{0}^{\tau _{n + 1}} e^{ - \delta s} \bigl( rX_{s}^{m} - RX_{s}^{M} \bigr) \,\mathrm{d}s \biggr] \\ &\quad \ge \phi ( x ) - \sum_{i = 1}^{n} \mathrm{E} \bigl[ e^{ - \delta \tau _{i}}K ( \zeta _{i} ) \bigr] - \mathrm{E} \biggl[ \int _{0}^{\tau _{n + 1}} e^{ - \delta s} \bigl( rX_{s}^{m} - RX_{s}^{M} \bigr) \,\mathrm{d}s \biggr]. \end{aligned} $$

(117)

Namely, we get

$$ \phi ( x ) \le \mathrm{E} \Biggl[ \int _{0}^{\tau _{n + 1}} e^{ - \delta s} \bigl( rX_{s}^{m} - RX_{s}^{M} \bigr) \,\mathrm{d}s + \sum_{i = 1}^{n} e^{ - \delta \tau _{i}}K ( \zeta _{i} ) \Biggr] + \mathrm{E} \bigl[ e^{ - \delta \tau _{n + 1}}\phi ( X_{\tau _{n + 1}} ) \bigr]. $$

(118)

Letting $n \to + \infty$ gives $\mathrm{E} [ e^{ - \delta \tau _{n + 1}}\phi ( X_{\tau _{n + 1}} ) ] \to + 0$ by the assumption, and we get

$$ \phi ( x ) \le J ( x;\eta ). $$

(119)

Because $\eta \in \mathcal{A}$ and $x \ge 0$ are arbitrary, we get

$$ \phi ( x ) \le \Phi ( x ), $$

(120)

namely (43) and the equality (44) with the QVI-control. □