Mathematical Methods in Signals, Systems and Control

News

1. Kind reminder: Please send your presentation slides to the teacher the day before the presentation.

2. The presentations will be done on Friday March 28, 08:00-, at Harry Nyquist (floor 7 of Malvinas väg 10). Attendance to all presentations is compulsory. The day before the presentations, please email the teacher a copy of your slides.

3. See below a list of suggestions of papers for the final presentation. Once you and your presentation partner have chosen one, please email the teacher to reserve it.

Credits

8 hp

Objectives

This course introduces some mathematical tools for understanding fundamental results underlying several areas of automatic control, signal processing and communications. In particular, the course focuses in aspects of functional analysis, specifically related to Hilbert and Banach spaces. The theory is complemented with several examples from areas such as robust control, game theory, and estimation/filtering theory.

Examiner and course responsible

Cristian R. Rojas, crro@kth.se

Course registration

If you are interested in taking this course, please register through this form: https://forms.gle/QTJBXHPdJSd9AYo59.

Note. The form is merely to know how many students intend to take the course, but the formal course registration is done at the same time as the grades are up in Ladok (i.e., after the course has finished), so no feedback will be sent after filling in the form.

Eligibility

The course is open to all graduate students (no prerequisite courses).

Literature

The textbook for the course is

• N. Young. An Introduction to Hilbert Space. Cambridge University Press, 1988.

Some excellent complementary texts are

• D. G. Luenberger. Optimization by Vector Space Methods. John Wiley & Sons, 1969.

• E.Kreyszig. Introductory Functional Analysis with Applications. John Wiley & Sons, 1989.

Schedule

Note: The schedule is preliminary. It will be updated with lecture slides as the course advances.

All lectures will be in the Harry Nyquist room, on Malvinas väg 10, floor 7.

Additional (optional) references

• P.R. Halmos. “How to write mathematics”. L'Enseignement Mathématique, vol. 16, 1970.

• J.M. Lee, “Some remarks on writing mathematical proofs”.

• L. C. Kinsey, Topology of Surfaces, Springer-Verlag, 1993.

• A. Megretski and J. Wyatt. Linear Algebra and Functional Analysis for Signals and Systems. 2009.

• J.R. Partington. An Introduction to Hankel Operators. Cambridge University Press, 1989.

• B.A. Francis. A Course in H-infinity Control Theory. Springer-Verlag, 1987.

• J.W. Helton. Operator Theory, Analytic Functions, Matrices, and Electrical Engineering. AMS, 1987.

• P. Regalia. Adaptive IIR Filtering in Signal Processing and Control. CRC Press, 1994.

• A. Barvinok. A Course in Convexity. American Mathematical Society, 2002.

• D.G. Luenberger. “Convex programming and duality in normed space”. IEEE Trans. Syst. Sci. Cybern., 4(2):182-188, 1968.

• J.W. Helton and M. Putinar. “Positive polynomials in scalar and matrix variables, the spectral theorem and optimization”. arXiv:0612103, 2006.

Evaluation (pass/fail)

The evaluation of the course will be based on several assignments (80%) and a project (individual or, preferably, in a group to 2 students) consisting in analyzing a particular application or extension of the theory presented in a recent publication from the areas of control, signal processing or communications, preferably related to the student’s own research, with a 15 min presentation of the main ideas/results of that publication. Once you and your presentation partner have chosen one, please email the teacher your chosen paper to reserve it.

Papers already taken

1. Margarita Guerrero: M. Budisic, R. Mohr and I. Mezic, “Applied Koopmanism”.

2. Gustav Norén: B. Hassibi, A.H. Sayed and T. Kailath, “Linear estimation in Krein spaces”.

3. Aihui Liu and Elisa Bin: R. Hildebrand and G.E. Solari, “Closed loop optimal input design: the partial correlation approach”.

4. Yibo Shi and Ying Wang: D.P. de Farias and B. Van Roy, “Approximate dynamic programming via linear programming”, “The linear programming approach to approximate dynamic programming”.

5. Mengyuan Zhao: Scutari, D.P. Palomar, F. Facchinei and J.-S. Pang, “Convex optimization, game theory, and variational inequality theory”.

6. Simon Lindståhl and Frédéric Zheng: B. Ninness and F. Gustafsson, “A unifying construction of orthonormal bases for system identification”.

7. Andrea Da Col and Gregorio Marchesini: S. Shalev-Shwartz and Y. Singer, “Online learning meets optimization in the dual”.

8. Noè Bernadas: L. Miretti, R. L. Garrido Cavalcante and S. Stańczak, “Channel covariance conversion and modelling using infinite dimensional Hilbert spaces”.

9. Chenyang Yan: G. Cybenko, “Approximation by superpositions of a sigmoidal function”.

10. William Réveillard: L. Vandenberghe, S. Boyd and K. Comanor, “Generalized Chebyshev bounds via semidefinite programming”.

11. Raghav Bongole and Abolfazl Changizi: A. Sinha1 and J. Duchi, “Learning kernels with random features”.

12. Anton Hässler: D. Xiu and G.E. Karniadakis, “The Wiener-Askey polynomial chaos for stochastic differential equations”.

Suggestions of papers

Here is a (biased) list of (recent and not-so-recent) papers to choose from for the presentation:

1. E.J. Balder, “An extension of duality-stability relations to nonconvex optimization problems”. SIAM J. Control and Optim., 15(2):329-343, 1977.
   R.T. Rockafellar, “Augmented lagrange multiplier functions and duality in nonconvex programming”. SIAM J. Control, 12(2):268-285, 1974.
   A. Nedich and A. Ozdaglar, “A geometric framework for nonconvex optimization duality using augmented lagrangian functions”. J. Glob. Optim., 40:545-573, 2008.

2. E.W. Barankin, “Locally best unbiased estimates”. Annals of Math. Stat., 20(4):477-501, 1949.

3. V.S. Borkar, “Convex analytic methods in Markov decision processes”. In E.A. Feinberg et. al. (eds.), Handbook of Markov Decision Processes, Kluwer, 2002.

4. M. Budisic, R. Mohr and I. Mezic, “Applied Koopmanism”. Chaos, 22:047510, 2012.

5. G. Cybenko, “Approximation by superpositions of a sigmoidal function”. Math. Control, Signals, and Systems, 2:303-314, 1989.

6. D.P. de Farias and B. Van Roy, “Approximate dynamic programming via linear programming”. Adv. in Neural Inf. Proc. Syst. (NIPS), 14, 2001.
   D.P. de Farias and B. Van Roy, “The linear programming approach to approximate dynamic programming”. Operations Research, 51(6):850-865, 2003.

7. R. de la Madrid, “The role of the rigged Hilbert space in quantum mechanics”. Eur. J. Physics, 26:287-312, 2005.

8. O. Devolder, F. Glineur and Y. Nesterov, “Solving infinite-dimensional optimization problems by polynomial approximation”. In M. Diehl et. al. (eds.), Recent Advances in Optimization and its Applications in Engineering, Springer-Verlag, 2010.

9. J. Eckstein and D.P. Bertsekas, “On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators”. Math. Prog., 55:293-318, 1992.

10. A.K. El-Sakkary, “The gap metric: Robustness of stabilization of feedback systems”. IEEE Trans. Aut. Control, 30(3):240-247, 1985.

11. A. Ghulchak and A. Rantzer, “Robust controller design via linear programming”. Proc. of IEEE CDC, 1999.

12. B. Hassibi, A.H. Sayed and T. Kailath, “Linear estimation in Krein spaces - Part I: Theory”. IEEE Trans. Aut. Control, 41(1):18-33, 1996.
    B. Hassibi, A.H. Sayed and T. Kailath, “Linear estimation in Krein spaces - Part II: Applications”. IEEE Trans. Aut. Control, 41(1):34-49, 1996.

13. J.W. Helton and M. Putinar, “Positive polynomials in scalar and matrix variables, the spectral theorem and optimization”. ArXiV: 0612103. 2006.

14. R. Hildebrand and G.E. Solari, “Closed loop optimal input design: the partial correlation approach”. Proc. of IFAC SYSID, 2009.

15. F.S. Hover and M.S. Triantafyllou, “Applications of polynomial chaos in stability and control”. Automatica, 42:789-795, 2006.

16. T. Iwasaki and S. Hara, “Generalized KYP lemma: Unified frequency domain inequalities with design applications”. IEEE Trans. Aut. Control, 50(1):41-59, 2005.

17. J. Karlsson, A. Lindquist and A. Ringh, “The multidimensional moment problem with complexity constraint”. Integr. Equ. Oper. Theory, 84: 395-418, 2016.
    A. Ringh, J. Karlsson and A. Lindquist, “Multidimensional rational covariance extension with applications to spectral estimation and image compression”. SIAM J. Control Optim., 54(4):1950-1982, 2016.

18. H. Kwakernaak, “Minimax frequency domain performance and robustness optimization of linear feedback systems”. IEEE Trans. Aut. Control, 30(10): 994-1004, 1985.

19. A. Megretski and A. Rantzer, “System analysis via integral quadratic constraints”. IEEE Trans. Aut. Control, 42(6):819-830, 1997.
    A. Megretsky and S. Treil, “Power distribution inequalities in optimization and robustness of uncertain systems”. J. Math. Syst., Estim., and Control, 3(3):301-319, 1993.

20. S.N. Negahban, P. Ravikumar, M.J. Wainwright and B. Yu, “A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers”. Stat. Science, 27(4):538-557, 2012.

21. B. Ninness and F. Gustafsson, “A unifying construction of orthonormal bases for system identification”. IEEE Trans. Aut. Control, 42(4):515-521, 1997.

22. B. Ninness and H. Hjalmarsson, “Variance error quantifications that are exact for finite-model order”. IEEE Trans. Aut. Control, 49(8):1275-1291, 2004.

23. G. Pillonetto, “Solutions of nonlinear control and estimation problems in reproducing kernel Hilbert spaces: Existence and numerical determination”. Automatica, 44:2135-2141, 2008.

24. A. Rantzer and A. Megretski, “A convex parameterization of robustly stabilizing controllers”. IEEE Trans. Aut. Control, 39(9):1802-1808, 1994.

25. A. Rantzer, “On the Kalman-Yakubovich-Popov lemma”. Syst. & Control Lett., 28:7-10, 1996.

26. H. Sandberg and B. Bernhardsson, “A Bode sensitivity integral for linear time-periodic systems”. IEEE Trans. Aut. Control, 50(12):2034-2039, 2005.

27. C.W. Scherer and C.W.J. Hol, “Matrix sum-of-squares relaxations for robust semi-definite programs”. Math. Prog., Ser. B, 107:189-211, 2006.

28. G. Scutari, D.P. Palomar, F. Facchinei and J.-S. Pang, “Convex optimization, game theory, and variational inequality theory”. IEEE Sig. Proc. Mag., 27(3):35-49, 2010.

29. S. Shalev-Shwartz and Y. Singer, “Online learning meets optimization in the dual”. Proc. Int. Conf. Comput. Learn. Theory (COLT), 423-437, 2006.
    S. Shalev-Shwartz and Y. Singer, “Convex repeated games and Fenchel duality”. Adv. in Neural Inf. Proc. Syst. (NIPS), 2006.

30. L. Vandenberghe, S. Boyd and K. Comanor, “Generalized Chebyshev bounds via semidefinite programming”. SIAM Review, 49(1):52-64, 2007.

31. G. Vinnicombe, “Frequency domain uncertainty and the graph topology”. IEEE Trans. Aut. Control, 38(9):1371-1383, 1993.

32. D. Xiu and G.E. Karniadakis, “The Wiener-Askey polynomial chaos for stochastic differential equations”. SIAM J. Sci. Comput., 24(2):619-644, 2002.

33. V.A. Yakubovich, “The solution of certain matrix inequalities in automatic control theory”. Soviet Math., 620-623, 1962.

34. P.M. Young and M.A. Dahleh, “Infinite-dimensional convex optimization in optimal and robust control theory”. IEEE Trans. Aut. Control, 42(10):1370-1381, 1997.

35. M. Kamgarpour and T. Summers, “On infinite dimensional linear programming approach to stochastic control”. IFAC PapersOnLine, 50(1):6148-6153, 2017.