Submitted papers

Maximizing the probability of visiting a set infinitely often for a Markov decision process with Borel state and actions spaces
Dufour, François; PrietoRumeau, Tomás

Minimum Contrast Estimators for Piecewise Deterministic Markov Processes
Costa, Oswaldo; Dufour, François; Genadot, Alexandre

Adaptive Average Control for Piecewise Deterministic Markov Processes
Costa, Oswaldo; Dufour, François; Genadot, Alexandre
Books

Numerical methods for simulation and optimization of piecewise deterministic Markov processes
De Saporta, Benoîte; Dufour, François; Huilong Zhang
Mathematics & Statistics. Wiley, 2015 (290 pages).

Continuous average control of piecewise deterministic Markov processes
Costa, Oswaldo; Dufour, François
Springer Briefs in Mathematics. Springer, New York, 2013. xii+116 pp.
Selected Book Chapters

Optimal control of piecewise deterministic Markov processes
Costa, Oswaldo; Dufour, François
A Commemorative Volume to Honor Mark H. A. Davis's Contributions
Stochastic analysis, filtering, and stochastic optimization. Springer, 2022
Edited by George Yin, Thaleia Zariphopoulou

Numerical Approximations for Discounted Continuous Time Markov Decision Processes
Dufour, François; PrietoRumeau, Tomás
Modeling, Stochastic Control, Optimization, and Applications.
The IMA Volumes in Mathematics and its Applications (164). Springer, 2019
Edited by George Yin, Qing Zhang

Continuous control of piecewise deterministic Markov processes with long run average cost
Costa, Oswaldo; Dufour, François
A Festschrift in honor of Robert J. Elliott
Stochastic processes, finance and control. World Scientific Publishing, 2012
Edited by Samuel N. Cohen, Dilip Madan, Tak Kuen Siu and Hailiang Yang

Approximation of infinite horizon discounted cost Markov decision processes
Dufour, François; PrietoRumeau, Tomás
In honor of Onésimo HernándezLerma
Optimization, control, and applications of stochastic systems. Birkhäuser/Springer, 2012
Edited by Daniel HernándezHernández and J. Adolfo MinjárezSosa
Papers
 [75]
Maximizing the probability of visiting a set infinitely often for a Markov decision process with Borel state and actions spaces
Dufour, François; PrietoRumeau, Tomás
To appear in Journal of Applied Probability, 33 pages, 2023
 [74]
Absorbing Markov Decision Processes
Dufour, François; PrietoRumeau, Tomás
To appear in ESAIM: Control, Optimisation and Calculus of Variations, 19 pages, 2023
 [73]
Nash Equilibria for Total Expected Reward Absorbing Markov Games: The Constrained and Unconstrained Cases
Dufour, François; PrietoRumeau, Tomás
To appear in Applied Mathematics and Optimization, 40 pages, 2023
 [72]
Adaptive Discounted Control for Piecewise Deterministic Markov Processes
Costa, Oswaldo; Dufour, François
Journal of Mathematical Analysis and Applications 58(2), 23 pages, 2023
 [71]
Stationary Markov Nash equilibria for nonzerosum constrained ARAT Markov games
Dufour, François; PrietoRumeau, Tomás
SIAM Journal on Control Optimization 60(2), 945–967, 2022
 [70]
Maximizing the probability of visiting a set infinitely often for a countable state space Markov decision process
Dufour, François; PrietoRumeau, Tomás
Journal of Mathematical Analysis and Applications, 505(2): 21 pages, 2022
 [69]
On the equivalence of the integral and differential Bellman equations in impulse control problems
Dufour, François; Piunovskiy, Alexey; Plakhov, Alexander
International Journal of Control, 95(1): 173–186, 2022
 [68]
IntegroDifferential Optimality Equations for the RiskSensitive Control of Piecewise Deterministic Markov Processes
Costa, Oswaldo; Dufour, François
Mathematical Methods of Operations Research, 93(2): 327–357, 2021
 [67]
A convex programming approach for discretetime Markov decision processes under the expected total reward optimization criterion
Dufour, François; Genadot, Alexandre
SIAM Journal on Control and Optimization, 58(4), 2535–2566, 2020
 [66]
PowerofdChoices with Memory: Fluid Limit and Optimality
Anselmi, Jonatha; Dufour, François
Mathematics of Operations Research, 45(3): 862888, 2020
 [65]
On the expected total cost with unbounded returns for Markov decision processes
Dufour, François; Genadot, Alexandre
Applied Mathematics and Optimization, 82(2): 433450, 2020
 [64]
HamiltonJacobiBellman Inequality for the Average Control of Piecewise Deterministic Markov Processes
Costa, Oswaldo; Dufour, François
Stochastics, 91(6): 817–835, 2019
 [63]
Approximation of discounted minimax Markov control problems and zerosum Markov games using Hausdorff and Wasserstein distances
Dufour, François; PrietoRumeau, Tomás
Dynamic Games and Applications, 9(1): 68102, 2019
 [62]
ZeroSum Discounted Reward Criterion Games for Piecewise Deterministic Markov Processes
Costa, Oswaldo; Dufour, François
Applied Mathematics and Optimization, 78(3): 587–611, 2018
 [61]
Computable approximations for average Markov decision processes in continuoustime
Anselmi, Jonatha; Dufour, François; PrietoRumeau, Tomás
Journal of Applied Probability, 55(2): 571592, 2018
 [60]
Controlling IL7 injections in HIVinfected patients
Pasin, Chloé; Dufour, François; Villain, Laura; Zhang, Huilong; Thiébaut, Rodolphe
Bulletin of Mathematical Biology, 80(9): 2349–2377, 2018
 [59]
Stochastic Control of Observer Trajectories in Passive Tracking with Acoustic Signal Propagation Optimization
Zhang, Huilong; De Saporta, Benoîte; Dufour; François; Laneuville Dann, Negre Adrien
IET Radar Sonar and Navigation, 12(1): 112–120, 2018
 [58]
Partially observed optimal stopping problem for discretetime Markov processes
De Saporta, Benoîte; Dufour, François; Nivot, Christophe
4OR A Quarterly Journal of Operations Research, 15(3): 277–302, 2017
 [57]
Optimal strategies for impulse control of piecewise deterministic Markov processes
De Saporta, Benoîte; Dufour, François; Geeraert Alizée
Automatica, 77(2): 219–229, 2017
 [56]
Optimal impulsive control of piecewise deterministic Markov processes
Dufour, François; Horiguchi, Masayuki; Piunovskiy, Alexey
Stochastics, 88(7): 10731098, 2016
 [55]
Computable approximations for continuoustime Markov decision processes on Borel spaces based on empirical measures
Anselmi, Jonatha; Dufour, François; PrietoRumeau, Tomás
Journal of Mathematical Analysis and Applications, 443(2): 1323–1361, 2016
 [54]
Constrained and Unconstrained Optimal Control of Piecewise Deterministic Markov Processes
Costa, Oswaldo; Dufour, François; Piunovskiy Alexey
SIAM Journal on Control and Optimization, 54(3): 1444–1474, 2016
 [53]
Impulsive control for continuoustime Markov Decision Processes: a linear programming approach
Dufour, François; Piunovskiy Alexey
Applied Mathematics and Optimization, 74(1): 129161, 2016
 [52]
Conditions for the solvability of the linear programming formulation for constrained discounted Markov decision processes
Dufour, François; PrietoRumeau, Tomás
Applied Mathematics and Optimization, 74(1): 2751, 2016
 [51]
Approximation of average cost Markov decision processes using empirical distributions and concentration inequalities
Dufour, François; PrietoRumeau, Tomás
Stochastics, 87(2): 273307, 2015
 [50]
Impulsive control for continuoustime Markov decision processes
Dufour, François; Piunovskiy, Alexey
Advances in Applied Probability, 47(1): 106127, 2015
 [49]
A Linear Programming Formulation for Constrained Discounted Continuous Control for Piecewise Deterministic Markov Processes
Costa, Oswaldo; Dufour, François
Journal of Mathematical Analysis and Applications, 424(2): 892–914, 2015
 [48]
Piecewise Deterministic Markov Processes based approach applied to an offshore oil production system
Zhang, Huilong; Innal, Fares; Dufour, François; Dutuit, Yves
Reliability Engineering & System Safety, 126: 126–134, 2014
 [47]
Stochastic Approximations of Constrained Discounted Markov Decision Processes
Dufour, François; PrietoRumeau, Tomás
Journal of Mathematical Analysis and Applications, 413(2): 856–879, 2014
 [46]
NonParametric Estimation of the Conditional Distribution of the Interjumping Times for PiecewiseDeterministic Markov Processes
Azaïs, Romain; Dufour, François; GégoutPetit, Anne
Scandinavian Journal of Statistics, 41(4): 950–969, 2014
 [45]
Optimal stopping for partially observed piecewisedeterministic Markov processes
Brandejsky, Adrien; de Saporta, Benoîte; Dufour, François
Stochastic Processes and their Applications, 123(8): 3201–3238, 2013
 [44]
Finite linear programming approximations of constrained discounted Markov decision processes
Dufour, François; PrietoRumeau, Tomás
SIAM J. Control and Optimization, 51(2): 1298–1324, 2013
 [43]
The expected total cost criterion for Markov decision processes under constraints
Dufour, François; Piunovskiy, Alexey
Advances in Applied Probability, 45(3): 837–859, 2013
 [42]
Nonparametric estimation of the jump rate for nonhomogeneous marked renewal processes
Azaïs, Romain; Dufour, François; GégoutPetit, Anne
Ann. Inst. Henri Poincaré Probabilités et Statistiques, 49(4): 1204–1231, 2013
 [41]
Average control of Markov decision processes with Feller transition probabilities and general action spaces
Costa, Oswaldo; Dufour, François
Journal of Mathematical Analysis and Applications, 396(1): 58–69, 2012
 [40]
The expected total cost criterion for Markov decision processes under constraints: a convex analytic approach
Dufour, François; Horiguchi, Masayuki; Piunovskiy, Alexey
Advances in Applied Probability, 44(3): 774–793, 2012
 [39]
Approximation of Markov decision processes with general state space
Dufour, François; PrietoRumeau, Tomás
Journal of Mathematical Analysis and Applications, 388(2): 1254–1267, 2012
 [38]
Optimal stopping for the predictive maintenance of a structure subject to corrosion
De Saporta, Benoîte; Dufour, François; Huilong Zhang; Elegbede, Charles
Journal of Risk and Reliability, 226(2): 169181, 2012
 [37]
Singularly perturbed discounted Markov control processes in a general state space
Costa, Oswaldo; Dufour, François
SIAM J. Control and Optimization, 50(2): 720–747, 2012
 [36]
Numerical method for impulse control of piecewise deterministic Markov processes
De Saporta, Benoîte; Dufour, François
Automatica, 48(5): 779–793, 2012
 [35]
Numerical methods for the exit time of a piecewisedeterministic Markov process
Brandejsky, Adrien; De Saporta, Benoîte; Dufour, François
Advances in Applied Probability, 44(1): 196–225, 2012
 [34]
Numerical method for expectations of piecewise deterministic Markov processes
Brandejsky, Adrien; De Saporta, Benoîte; Dufour, François
Communications in Applied Mathematics and Computational Science, 7(1): 63–104, 2012
 [33]
On the existence of strict optimal controls for constrained, controlled Markov processes in continuous time
Dufour, Francois; Stockbridge, Richard
Stochastics, 84(1): 55–78, 2012
 [32]
Singular perturbation for the discounted continuous control of piecewise deterministic Markov processes
Costa, Oswaldo; Dufour, Francois
Applied Mathematics and Optimization, 63(3): 357–384, 2011
 [31]
Multiobjective stopping problem for discretetime Markov processes: convex analytic approach
Dufour, Francois; Piunovskiy, Alexey
Journal of Applied Probability, 47(4): 947–966, 2010
 [30]
Average continuous control of piecewise deterministic Markov processes
Costa, Oswaldo; Dufour, François
SIAM J. Control and Optimization, 48(7): 4262–4291, 2010
 [29]
The policy iteration algorithm for average continuous control of piecewise deterministic Markov processes
Costa, Oswaldo; Dufour, Francois
Applied Mathematics and Optimization, 62(2): 185–204, 2010
 [28]
Numerical method for optimal stopping of piecewise deterministic Markov processes
De Saporta, Benoîte; Dufour, François; Gonzalez, Karen
The Annals of Applied Probability, 20(5): 1607–1637, 2010
 [27]
The vanishing discount approach for the average continuous control of piecewise deterministic Markov processes
Costa, Oswaldo; Dufour, François
Journal of Applied Probability, 46(4): 1157–1183, 2009
 [26]
Almost sure stabilization for feedback controls of regimeswitching linear systems with a hidden Markov chain
Bercu, Bernard; Dufour, François; Yin, George
IEEE Trans. Automat. Control, 54(9): 2114–2125, 2009
 [25]
Piecewise deterministic Markov processes and dynamic reliability
Zhang, Huilong; Dufour, François; Dutuit, Yves; Gonzales, Karen
Journal of Risk and Reliability, 222(4): 545551, 2009
 [24]
Stability and ergodicity of piecewise deterministic Markov processes
Costa, Oswaldo; Dufour, François
SIAM J. Control and Optimization 47(2): 1053–1077, 2008
 [23]
Necessary conditions for optimal singular stochastic control problems
Dufour, François; Miller, Boris
Stochastics, 79(5): 469–504, 2007
 [22]
Ergodic properties and ergodic decompositions of continuoustime Markov processes
Costa, Oswaldo; Dufour, François
Journal of Applied Probability, 43(3): 767–781, 2006
 [21]
Maximum principle for singular stochastic control problems
Dufour, François; Miller, Boris
SIAM J. Control and Optimization, 45(2): 668–698, 2006
 [20]
A sufficient condition for the existence of an invariant probability measure for Markov processes
Costa, Oswaldo; Dufour, François
Journal of Applied Probability, 42(3): 873–878, 2005
 [19]
State and mode estimation for discretetime jump Markov systems
Elliott, Robert; Dufour, François; Malcolm, Paul
SIAM J. Control and Optimization, 44(3): 1081–1104, 2005
 [18]
On the ergodic decomposition for a class of Markov chains
Costa, Oswaldo; Dufour, François
Stochastic Processes and their Applications, 115(3): 401–415, 2005
 [17]
Singular stochastic control problems
Dufour, François; Miller, Boris
SIAM J. Control and Optimization, 43(2): 708–730, 2004
 [16]
On the Poisson equation for piecewisedeterministic Markov processes
Costa, Oswaldo; Dufour, François
SIAM J. Control and Optimization, 42(3): 985–1001, 2003
 [15]
Generalized solutions in nonlinear stochastic control problems
Dufour, François; Miller, Boris
SIAM J. Control and Optimization, 40(6): 1724–1745, 2002
 [14]
Necessary and sufficient conditions for nonsingular invariant probability measures for Feller Markov chains
Costa, Oswaldo; Dufour, François
Statistics & Probability Letters, 53(1): 47–57, 2001
 [13]
Discretetime estimation of a Markov chain with marked point process observations
Allam, Sébastien; Dufour, François; Bertrand, Pierre
IEEE Trans. Automat. Control 46 (2001), no. 6, 903–908
 [12]
Invariant probability measures for a class of Feller Markov chains
Costa, Oswaldo; Dufour, François
Statistics & Probability Letters, 50(1): 13–21, 2000
 [11]
Optimal stopping with continuous control of piecewise deterministic Markov processes
Costa, Oswaldo; Raymundo, C.; Dufour, François
Stochastics, 70(12): 41–73, 2000
 [10]
Stability of piecewisedeterministic Markov processes
Costa, Oswaldo; Dufour, François
SIAM J. Control and Optimization, 37(5): 1483–1502, 1999
 [9]
Filtering with discrete state observations
Dufour, François; Elliott, Robert
Applied Mathematics and Optimization, 40(2): 259–272, 1999
 [8]
Discrete time nonlinear filtering with marked point process observations
Dufour, François; Kannan Dan
Stochastic Analysis and Applications, 17(1): 99–115, 1999
 [7]
Adaptive control of linear systems with Markov perturbations
Dufour, François; Elliott, Robert
IEEE Trans. Automat. Control, 43(3): 351–372, 1998
 [6]
On the JLQ problem with uncertainty
Bernard, Fabienne; Dufour, François; Bertrand, Pierre
IEEE Trans. Automat. Control, 42(6): 869–872, 1997
 [5]
Exact hybrid filters in discrete time
Elliott, Robert; Dufour, François; Sworder, Dave
IEEE Trans. Automat. Control, 41(12): 1807–1810, 1996
 [4]
An imagebased filter for discretetime Markovian jump linear systems
Dufour, François; Bertrand, Pierre
Automatica, 32(2): 241–247, 1996
 [3]
The filtering problem for continuoustime linear systems with Markovian switching coefficients
Dufour, François; Bertrand, Pierre
Systems & Control Letters, 23(6): 453–461, 1994
 [2]
Stabilizing control law for hybrid models
Dufour, François; Bertrand, Pierre
IEEE Trans. Automat. Control, 39(11): 2354–2357, 1994
 [1]
Tracking a 3D Maneuvring Target with Passive Sensors
Dufour, François; Mariton, Michel
IEEE Trans. Aero. and Electr. Systems, 27(4): 725739, 1991
