Mathematical Methods in Signals, Systems and Control

News

• The presentations will be done in two time slots:

1. Wednesday Jan 11, 13:00- (Harry Nyquist, floor 7, Malvinas väg 10)

2. Monday Jan 16, 13:00- (Meeting room in floor 6 of Malvinas väg 10, next to the kitchen)

You are welcome to present in any of those two time slots. The day before the presentation slot you choose, please email the teacher a copy of your slides.

• The teacher will bring in the next lectures some papers that can be considered for the final project, but you can propose a different one (for example, connected to your research). The presentations will be scheduled for an afternoon between Jan 9 and Jan 13; a doodle link will be set up soon for this purpose.

• The 4th (and final) homework has been posted.

• The 3rd homework has been posted.

• The 2nd homework has been posted.

• The first homework has been posted. You can write your solutions by hand (in readable hand-writing) or in a computer, and submit them by email or on the teacher's mailbox (Malvinas väg 10, floor 6).

Credits

8 hp

Objectives

This course introduces some mathematical tools for understanding fundamental results underlying several areas of automatic control, signal processing and communications. In particular, the course focuses in aspects of functional analysis, specifically related to Hilbert and Banach spaces. The theory is complemented with several examples from areas such as robust control, game theory, and estimation/filtering theory.

Examiner and course responsible

Cristian R. Rojas, crro@kth.se

Eligibility

The course is open to all graduate students (no prerequisite courses).

Literature

The textbook for the course is

• N. Young. An Introduction to Hilbert Space. Cambridge University Press, 1988.

Some excellent complementary texts are

• D. G. Luenberger. Optimization by Vector Space Methods. John Wiley & Sons, 1969.

• E.Kreyszig. Introductory Functional Analysis with Applications. John Wiley & Sons, 1989.

Schedule

Note: The schedule is preliminary. It will be updated with lecture slides as the course advances.

All lectures will be in the Harry Nyquist room, on Malvinas väg 10, floor 7.

Additional (optional) references

• P.R. Halmos. “How to write mathematics”. L'Enseignement Mathématique, vol. 16, 1970.

• J.M. Lee, “Some remarks on writing mathematical proofs”.

• L. C. Kinsey, Topology of Surfaces, Springer-Verlag, 1993.

• A. Megretski and J. Wyatt. Linear Algebra and Functional Analysis for Signals and Systems. 2009.

• J.R. Partington. An Introduction to Hankel Operators. Cambridge University Press, 1989.

• B.A. Francis. A Course in H-infinity Control Theory. Springer-Verlag, 1987.

• J.W. Helton. Operator Theory, Analytic Functions, Matrices, and Electrical Engineering. AMS, 1987.

• P. Regalia. Adaptive IIR Filtering in Signal Processing and Control. CRC Press, 1994.

• A. Barvinok. A Course in Convexity. American Mathematical Society, 2002.

• D.G. Luenberger. “Convex programming and duality in normed space”. IEEE Trans. Syst. Sci. Cybern., 4(2):182-188, 1968.

• J.W. Helton and M. Putinar. “Positive polynomials in scalar and matrix variables, the spectral theorem and optimization”. arXiv:0612103, 2006.

Evaluation (pass/fail)

The evaluation of the course will be based on several assignments (80%) and a project (individual or, preferably, in a group to 2 students) consisting in analyzing a particular application or extension of the theory presented in a recent publication from the areas of control, signal processing or communications, preferably related to the student’s own research, with a 15 min presentation of the main ideas/results of that publication.

Papers already taken

• Alberta Longhini and Gustaf Tegnér: R. Antonova, J. Yang, P. Sundaresan, D. Fox, F. Ramos and J. Bohg, “A Bayesian treatment of real-to-sim for deformable object manipulation”. ArXiv: 2112.05068, 2021.

• Paul Häusner: D. Obmann, L. Nguyen,J. Schwab1 and M. Haltmeier, “Augmented NETT regularization of inverse problems”. J. Physics Comm., 5:105002, 2021.

• Borja Rodríguez and Amaury Gouverneur: G. Lugosi and G. Neu, “Generalization bounds via convex analysis”. ArXiv:2202.04985, 2022.

• Malin Andersson: A. Pozzi and D.M. Raimondo, “Stochastic model predictive control for optimal charging of electric vehicles battery packs”. J. Energy Storage, 55A(1):105332, 2022.

• Anh Tung Nguyen: T. Iwasaki and S. Hara, “Generalized KYP lemma: Unified frequency domain inequalities with design applications”. IEEE Trans. Aut. Control, 50(1):41-59, 2005.

• Braghadeesh Lakshminarayanan: B. Hassibi, A.H. Sayed and T. Kailath, “Linear estimation in Krein spaces”. IEEE Trans. Aut. Control, 41(1):18-49, 1996.

• Miguel Aguiar and Mayank Sewlia: S. Boyd and L. O. Chua, “Fading memory and the problem of approximating nonlinear operators with Volterra series”. IEEE Trans. Circ. Syst., 32(11):1150-1171, 1985.

• Jiabao He: B. Ninness and H. Hjalmarsson, “Variance error quantifications that are exact for finite-model order”. IEEE Trans. Aut. Control, 49(8):1275-1291, 2004.

• Maria Charitidou: F.S. Hover and M.S. Triantafyllou, “Applications of polynomial chaos in stability and control”. Automatica, 42:789-795, 2006.

• Daniele Foffano: D.P. de Farias and B. Van Roy, “The linear programming approach to approximate dynamic programming”. Operations Research, 51(6):850-865, 2003.

• Victor Molnö and Kamil Hassan: G. Pillonetto, “Solutions of nonlinear control and estimation problems in reproducing kernel Hilbert spaces: Existence and numerical determination”. Automatica, 44:2135-2141, 2008.

• Yassir Jedra: O. Devolder, F. Glineur and Y. Nesterov, “Solving infinite-dimensional optimization problems by polynomial approximation”. In M. Diehl et. al. (eds.), Recent Advances in Optimization and its Applications in Engineering, Springer-Verlag, 2010.

Suggestions of papers

Here is a (biased) list of (recent and not-so-recent) papers to choose from for the presentation:

1. E.J. Balder, “An extension of duality-stability relations to nonconvex optimization problems”. SIAM J. Control and Optim., 15(2):329-343, 1977.
   R.T. Rockafellar, “Augmented lagrange multiplier functions and duality in nonconvex programming”. SIAM J. Control, 12(2):268-285, 1974.
   A. Nedich and A. Ozdaglar, “A geometric framework for nonconvex optimization duality using augmented lagrangian functions”. J. Glob. Optim., 40:545-573, 2008.

2. E.W. Barankin, “Locally best unbiased estimates”. Annals of Math. Stat., 20(4):477-501, 1949.

3. V.S. Borkar, “Convex analytic methods in Markov decision processes”. In E.A. Feinberg et. al. (eds.), Handbook of Markov Decision Processes, Kluwer, 2002.

4. M. Budisic, R. Mohr and I. Mezic, “Applied Koopmanism”. Chaos, 22:047510, 2012.

5. G. Cybenko, “Approximation by superpositions of a sigmoidal function”. Math. Control, Signals, and Systems, 2:303-314, 1989.

6. D.P. de Farias and B. Van Roy, “Approximate dynamic programming via linear programming”. Adv. in Neural Inf. Proc. Syst. (NIPS), 14, 2001.
   D.P. de Farias and B. Van Roy, “The linear programming approach to approximate dynamic programming”. Operations Research, 51(6):850-865, 2003.

7. R. de la Madrid, “The role of the rigged Hilbert space in quantum mechanics”. Eur. J. Physics, 26:287-312, 2005.

8. O. Devolder, F. Glineur and Y. Nesterov, “Solving infinite-dimensional optimization problems by polynomial approximation”. In M. Diehl et. al. (eds.), Recent Advances in Optimization and its Applications in Engineering, Springer-Verlag, 2010.

9. J. Eckstein and D.P. Bertsekas, “On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators”. Math. Prog., 55:293-318, 1992.

10. A.K. El-Sakkary, “The gap metric: Robustness of stabilization of feedback systems”. IEEE Trans. Aut. Control, 30(3):240-247, 1985.

11. A. Ghulchak and A. Rantzer, “Robust controller design via linear programming”. Proc. of IEEE CDC, 1999.

12. B. Hassibi, A.H. Sayed and T. Kailath, “Linear estimation in Krein spaces - Part I: Theory”. IEEE Trans. Aut. Control, 41(1):18-33, 1996.
    B. Hassibi, A.H. Sayed and T. Kailath, “Linear estimation in Krein spaces - Part II: Applications”. IEEE Trans. Aut. Control, 41(1):34-49, 1996.

13. J.W. Helton and M. Putinar, “Positive polynomials in scalar and matrix variables, the spectral theorem and optimization”. ArXiV: 0612103. 2006.

14. R. Hildebrand and G.E. Solari, “Closed loop optimal input design: the partial correlation approach”. Proc. of IFAC SYSID, 2009.

15. F.S. Hover and M.S. Triantafyllou, “Applications of polynomial chaos in stability and control”. Automatica, 42:789-795, 2006.

16. T. Iwasaki and S. Hara, “Generalized KYP lemma: Unified frequency domain inequalities with design applications”. IEEE Trans. Aut. Control, 50(1):41-59, 2005.

17. J. Karlsson, A. Lindquist and A. Ringh, “The multidimensional moment problem with complexity constraint”. Integr. Equ. Oper. Theory, 84: 395-418, 2016.
    A. Ringh, J. Karlsson and A. Lindquist, “Multidimensional rational covariance extension with applications to spectral estimation and image compression”. SIAM J. Control Optim., 54(4):1950-1982, 2016.

18. H. Kwakernaak, “Minimax frequency domain performance and robustness optimization of linear feedback systems”. IEEE Trans. Aut. Control, 30(10): 994-1004, 1985.

19. A. Megretski and A. Rantzer, “System analysis via integral quadratic constraints”. IEEE Trans. Aut. Control, 42(6):819-830, 1997.
    A. Megretsky and S. Treil, “Power distribution inequalities in optimization and robustness of uncertain systems”. J. Math. Syst., Estim., and Control, 3(3):301-319, 1993.

20. S.N. Negahban, P. Ravikumar, M.J. Wainwright and B. Yu, “A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers”. Stat. Science, 27(4):538-557, 2012.

21. B. Ninness and F. Gustafsson, “A unifying construction of orthonormal bases for system identification”. IEEE Trans. Aut. Control, 42(4):515-521, 1997.

22. B. Ninness and H. Hjalmarsson, “Variance error quantifications that are exact for finite-model order”. IEEE Trans. Aut. Control, 49(8):1275-1291, 2004.

23. G. Pillonetto, “Solutions of nonlinear control and estimation problems in reproducing kernel Hilbert spaces: Existence and numerical determination”. Automatica, 44:2135-2141, 2008.

24. A. Rantzer and A. Megretski, “A convex parameterization of robustly stabilizing controllers”. IEEE Trans. Aut. Control, 39(9):1802-1808, 1994.

25. A. Rantzer, “On the Kalman-Yakubovich-Popov lemma”. Syst. & Control Lett., 28:7-10, 1996.

26. H. Sandberg and B. Bernhardsson, “A Bode sensitivity integral for linear time-periodic systems”. IEEE Trans. Aut. Control, 50(12):2034-2039, 2005.

27. C.W. Scherer and C.W.J. Hol, “Matrix sum-of-squares relaxations for robust semi-definite programs”. Math. Prog., Ser. B, 107:189-211, 2006.

28. G. Scutari, D.P. Palomar, F. Facchinei and J.-S. Pang, “Convex optimization, game theory, and variational inequality theory”. IEEE Sig. Proc. Mag., 27(3):35-49, 2010.

29. S. Shalev-Shwartz and Y. Singer, “Online learning meets optimization in the dual”. Proc. Int. Conf. Comput. Learn. Theory (COLT), 423-437, 2006.
    S. Shalev-Shwartz and Y. Singer, “Convex repeated games and Fenchel duality”. Adv. in Neural Inf. Proc. Syst. (NIPS), 2006.

30. L. Vandenberghe, S. Boyd and K. Comanor, “Generalized Chebyshev bounds via semidefinite programming”. SIAM Review, 49(1):52-64, 2007.

31. G. Vinnicombe, “Frequency domain uncertainty and the graph topology”. IEEE Trans. Aut. Control, 38(9):1371-1383, 1993.

32. D. Xiu and G.E. Karniadakis, “The Wiener-Askey polynomial chaos for stochastic differential equations”. SIAM J. Sci. Comput., 24(2):619-644, 2002.

33. V.A. Yakubovich, “The solution of certain matrix inequalities in automatic control theory”. Soviet Math., 620-623, 1962.

34. P.M. Young and M.A. Dahleh, “Infinite-dimensional convex optimization in optimal and robust control theory”. IEEE Trans. Aut. Control, 42(10):1370-1381, 1997.