In submission

On the expected total cost with unbounded returns for Markov decision processes

Dufour, François; Genadot, Alexandre

Hamilton-Jacobi-Bellman Inequality for the Average Control of Piecewise Deterministic Markov Processes

Costa, Oswaldo; Dufour, François

Approximation of discounted minimax Markov control problems and zero-sum Markov games using Hausdorff and Wasserstein distances

Dufour, François; Prieto-Rumeau, Tomás

Controlling IL-7 injections in HIV-infected patients

Pasin, Chloé; Dufour, François; Villain, Laura; Zhang, Huilong; Thiébaut, Rodolphe

Books

Numerical methods for simulation and optimization of piecewise deterministic Markov processes

De Saporta, Benoîte; Dufour, François; Huilong Zhang
Mathematics & Statistics. Wiley, 2015 (290 pages).

Continuous average control of piecewise deterministic Markov processes

Costa, Oswaldo; Dufour, François
Springer Briefs in Mathematics. Springer, New York, 2013. xii+116 pp.

Selected Book Chapters

Continuous control of piecewise deterministic Markov processes with long run average cost

Costa, Oswaldo; Dufour, François
A Festschrift in honor of Robert J. Elliott
Stochastic processes, finance and control. World Scientific Publishing, 2012
Edited by Samuel N. Cohen, Dilip Madan, Tak Kuen Siu and Hailiang Yang

Approximation of infinite horizon discounted cost Markov decision processes

Dufour, François; Prieto-Rumeau, Tomás
In honor of Onésimo Hernández-Lerma
Optimization, control, and applications of stochastic systems. Birkhäuser/Springer, 2012
Edited by Daniel Hernández-Hernández and J. Adolfo Minjárez-Sosa

Papers

[61] Computable approximations for average Markov decision processes in continuous-time

Anselmi, Jonatha; Dufour, François; Prieto-Rumeau, Tomás
To appear in Journal of Applied Probability

[60] Stochastic Control of Observer Trajectories in Passive Tracking with Acoustic Signal Propagation Optimization

Zhang, Huilong; De Saporta, Benoîte; Dufour; François; Laneuville Dann, Negre Adrien
To appear in IET Radar Sonar and Navigation, 2017

[59] Zero-Sum Discounted Reward Criterion Games for Piecewise Deterministic Markov Processes

Costa, Oswaldo; Dufour, François
To appear in Applied Mathematics and Optimization, 2017

[58] Partially observed optimal stopping problem for discrete-time Markov processes

De Saporta, Benoîte; Dufour, François; Nivot, Christophe
4OR A Quarterly Journal of Operations Research, 15(3): 277–302, 2017

[57] Optimal strategies for impulse control of piecewise deterministic Markov processes

De Saporta, Benoîte; Dufour, François; Geeraert Alizée
Automatica, 77(2): 219–229, 2017

[56] Optimal impulsive control of piecewise deterministic Markov processes

Dufour, François; Horiguchi, Masayuki; Piunovskiy, Alexey
Stochastics, 88(7): 1073-1098, 2016

[55] Computable approximations for continuous-time Markov decision processes on Borel spaces based on empirical measures

Anselmi, Jonatha; Dufour, François; Prieto-Rumeau, Tomás
Journal of Mathematical Analysis and Applications, 443(2): 1323–1361, 2016

[54] Constrained and Unconstrained Optimal Control of Piecewise Deterministic Markov Processes

Costa, Oswaldo; Dufour, François; Piunovskiy Alexey
SIAM J. Control and Optimization, 54(3): 1444–1474, 2016

[53] Impulsive control for continuous-time Markov Decision Processes: a linear programming approach

Dufour, François; Piunovskiy Alexey
Applied Mathematics and Optimization, 74(1): 129-161, 2016

[52] Conditions for the solvability of the linear programming formulation for constrained discounted Markov decision processes

Dufour, François; Prieto-Rumeau, Tomás
Applied Mathematics and Optimization, 74(1): 27-51, 2016

[51] Approximation of average cost Markov decision processes using empirical distributions and concentration inequalities

Dufour, François; Prieto-Rumeau, Tomás
Stochastics, 87(2): 273-307, 2015

[50] Impulsive control for continuous-time Markov decision processes

Dufour, François; Piunovskiy, Alexey
Advances in Applied Probability, 47(1): 106-127, 2015

[49] A Linear Programming Formulation for Constrained Discounted Continuous Control for Piecewise Deterministic Markov Processes

Costa, Oswaldo; Dufour, François
Journal of Mathematical Analysis and Applications, 424(2): 892–914, 2015

[48] Piecewise Deterministic Markov Processes based approach applied to an offshore oil production system

Zhang, Huilong; Innal, Fares; Dufour, François; Dutuit, Yves
Reliability Engineering & System Safety, 126: 126–134, 2014

[47] Non-Parametric Estimation of the Conditional Distribution of the Interjumping Times for Piecewise-Deterministic Markov Processes

Azaïs, Romain; Dufour, François; Gégout-Petit, Anne
Scandinavian Journal of Statistics, 41(4): 950–969, 2014

[46] Stochastic Approximations of Constrained Discounted Markov Decision Processes

Dufour, François; Prieto-Rumeau, Tomás
Journal of Mathematical Analysis and Applications, 413(2): 856–879, 2014

[45] Optimal stopping for partially observed piecewise-deterministic Markov processes

Brandejsky, Adrien; de Saporta, Benoîte; Dufour, François
Stochastic Processes and their Applications, 123(8): 3201–3238, 2013

[44] Finite linear programming approximations of constrained discounted Markov decision processes

Dufour, François; Prieto-Rumeau, Tomás
SIAM J. Control and Optimization, 51(2): 1298–1324, 2013

[43] The expected total cost criterion for Markov decision processes under constraints

Dufour, François; Piunovskiy, Alexey
Advances in Applied Probability, 45(3): 837–859, 2013

[42] Nonparametric estimation of the jump rate for non-homogeneous marked renewal processes

Azaïs, Romain; Dufour, François; Gégout-Petit, Anne
Ann. Inst. Henri Poincaré Probabilités et Statistiques, 49(4): 1204–1231, 2013

[41] Average control of Markov decision processes with Feller transition probabilities and general action spaces

Costa, Oswaldo; Dufour, François
Journal of Mathematical Analysis and Applications, 396(1): 58–69, 2012

[40] The expected total cost criterion for Markov decision processes under constraints: a convex analytic approach

Dufour, François; Horiguchi, Masayuki; Piunovskiy, Alexey
Advances in Applied Probability, 44(3): 774–793, 2012

[39] Singularly perturbed discounted Markov control processes in a general state space

Costa, Oswaldo; Dufour, François
SIAM J. Control and Optimization, 50(2): 720–747, 2012

[38] Numerical method for expectations of piecewise deterministic Markov processes

Brandejsky, Adrien; De Saporta, Benoîte; Dufour, François
Communications in Applied Mathematics and Computational Science, 7(1): 63–104, 2012

[37] Approximation of Markov decision processes with general state space

Dufour, François; Prieto-Rumeau, Tomás
Journal of Mathematical Analysis and Applications, 388(2): 1254–1267, 2012

[36] Numerical method for impulse control of piecewise deterministic Markov processes

De Saporta, Benoîte; Dufour, François
Automatica, 48(5): 779–793, 2012

[35] Numerical methods for the exit time of a piecewise-deterministic Markov process

Brandejsky, Adrien; De Saporta, Benoîte; Dufour, François
Advances in Applied Probability, 44(1): 196–225, 2012

[34] Optimal stopping for the predictive maintenance of a structure subject to corrosion

De Saporta, Benoîte; Dufour, François; Huilong Zhang; Elegbede, Charles
Journal of Risk and Reliability, 226(2): 169-181, 2012

[33] On the existence of strict optimal controls for constrained, controlled Markov processes in continuous time

Dufour, Francois; Stockbridge, Richard
Stochastics, 84(1): 55–78, 2012

[32] Singular perturbation for the discounted continuous control of piecewise deterministic Markov processes

Costa, Oswaldo; Dufour, Francois
Applied Mathematics and Optimization, 63(3): 357–384, 2011

[31] Multiobjective stopping problem for discrete-time Markov processes: convex analytic approach

Dufour, Francois; Piunovskiy, Alexey
Journal of Applied Probability, 47(4): 947–966, 2010

[30] The policy iteration algorithm for average continuous control of piecewise deterministic Markov processes

Costa, Oswaldo; Dufour, Francois
Applied Mathematics and Optimization, 62(2): 185–204, 2010

[29] Numerical method for optimal stopping of piecewise deterministic Markov processes

De Saporta, Benoîte; Dufour, François; Gonzalez, Karen
The Annals of Applied Probability, 20(5): 1607–1637, 2010

[28] Average continuous control of piecewise deterministic Markov processes

Costa, Oswaldo; Dufour, François
SIAM J. Control and Optimization, 48(7): 4262–4291, 2010

[27] The vanishing discount approach for the average continuous control of piecewise deterministic Markov processes

Costa, Oswaldo; Dufour, François
Journal of Applied Probability, 46(4): 1157–1183, 2009

[26] Almost sure stabilization for feedback controls of regime-switching linear systems with a hidden Markov chain

Bercu, Bernard; Dufour, François; Yin, George
IEEE Trans. Automat. Control, 54(9): 2114–2125, 2009

[25] Piecewise deterministic Markov processes and dynamic reliability

Zhang, Huilong; Dufour, François; Dutuit, Yves; Gonzales, Karen
Journal of Risk and Reliability, 222(4): 545-551, 2009

[24] Stability and ergodicity of piecewise deterministic Markov processes

Costa, Oswaldo; Dufour, François
SIAM J. Control and Optimization 47(2): 1053–1077, 2008

[23] Necessary conditions for optimal singular stochastic control problems

Dufour, François; Miller, Boris
Stochastics, 79(5): 469–504, 2007

[22] Ergodic properties and ergodic decompositions of continuous-time Markov processes

Costa, Oswaldo; Dufour, François
Journal of Applied Probability, 43(3): 767–781, 2006

[21] Maximum principle for singular stochastic control problems

Dufour, François; Miller, Boris
SIAM J. Control and Optimization, 45(2): 668–698, 2006

[20] A sufficient condition for the existence of an invariant probability measure for Markov processes

Costa, Oswaldo; Dufour, François
Journal of Applied Probability, 42(3): 873–878, 2005

[19] State and mode estimation for discrete-time jump Markov systems

Elliott, Robert; Dufour, François; Malcolm, Paul
SIAM J. Control and Optimization, 44(3): 1081–1104, 2005

[18] On the ergodic decomposition for a class of Markov chains

Costa, Oswaldo; Dufour, François
Stochastic Processes and their Applications, 115(3): 401–415, 2005

[17] Singular stochastic control problems

Dufour, François; Miller, Boris
SIAM J. Control and Optimization, 43(2): 708–730, 2004

[16] On the Poisson equation for piecewise-deterministic Markov processes

Costa, Oswaldo; Dufour, François
SIAM J. Control and Optimization, 42(3): 985–1001, 2003

[15] Generalized solutions in nonlinear stochastic control problems

Dufour, François; Miller, Boris
SIAM J. Control and Optimization, 40(6): 1724–1745, 2002

[14] Necessary and sufficient conditions for non-singular invariant probability measures for Feller Markov chains

Costa, Oswaldo; Dufour, François
Statistics & Probability Letters, 53(1): 47–57, 2001

[13] Discrete-time estimation of a Markov chain with marked point process observations

Allam, Sébastien; Dufour, François; Bertrand, Pierre
IEEE Trans. Automat. Control 46 (2001), no. 6, 903–908

[12] Invariant probability measures for a class of Feller Markov chains

Costa, Oswaldo; Dufour, François
Statistics & Probability Letters, 50(1): 13–21, 2000

[11] Optimal stopping with continuous control of piecewise deterministic Markov processes

Costa, Oswaldo; Raymundo, C.; Dufour, François
Stochastics, 70(1-2): 41–73, 2000

[10] Stability of piecewise-deterministic Markov processes

Costa, Oswaldo; Dufour, François
SIAM J. Control and Optimization, 37(5): 1483–1502, 1999

[9] Filtering with discrete state observations

Dufour, François; Elliott, Robert
Applied Mathematics and Optimization, 40(2): 259–272, 1999

[8] Discrete time nonlinear filtering with marked point process observations

Dufour, François; Kannan Dan
Stochastic Analysis and Applications, 17(1): 99–115, 1999

[7] Adaptive control of linear systems with Markov perturbations

Dufour, François; Elliott, Robert
IEEE Trans. Automat. Control, 43(3): 351–372, 1998

[6] On the JLQ problem with uncertainty

Bernard, Fabienne; Dufour, François; Bertrand, Pierre
IEEE Trans. Automat. Control, 42(6): 869–872, 1997

[5] Exact hybrid filters in discrete time

Elliott, Robert; Dufour, François; Sworder, Dave
IEEE Trans. Automat. Control, 41(12): 1807–1810, 1996

[4] An image-based filter for discrete-time Markovian jump linear systems

Dufour, François; Bertrand, Pierre
Automatica, 32(2): 241–247, 1996

[3] The filtering problem for continuous-time linear systems with Markovian switching coefficients

Dufour, François; Bertrand, Pierre
Systems & Control Letters, 23(6): 453–461, 1994

[2] Stabilizing control law for hybrid models

Dufour, François; Bertrand, Pierre
IEEE Trans. Automat. Control, 39(11): 2354–2357, 1994

[1] Tracking a 3D Maneuvring Target with Passive Sensors

Dufour, François; Mariton, Michel
IEEE Trans. Aero. and Electr. Systems, 27(4): 725-739, 1991