Portal de Periódicos da CAPES

Flight Testing Reinforcement-Learning-Based Online Adaptive Flight Control Laws on CS-25-Class Aircraft

2024; American Institute of Aeronautics and Astronautics; Volume: 47; Issue: 11 Linguagem: Inglês

10.2514/1.g008321

1533-3884

Ramesh Konatala, Daniel Milz, Christian Weiser, Gertjan Looye, Erik-Jan Van Kampen,

Adaptive Control of Nonlinear Systems

No AccessEngineering NotesFlight Testing Reinforcement-Learning-Based Online Adaptive Flight Control Laws on CS-25-Class AircraftRamesh Konatala, Daniel Milz, Christian Weiser, Gertjan Looye and E. van KampenRamesh Konatala https://orcid.org/0000-0002-8051-2406German Aerospace Center (DLR), 82234 Weßling, Germany, Daniel MilzGerman Aerospace Center (DLR), 82234 Weßling, Germany, Christian Weiser https://orcid.org/0000-0002-2968-6961German Aerospace Center (DLR), 82234 Weßling, Germany, Gertjan LooyeGerman Aerospace Center (DLR), 82234 Weßling, Germany and E. van KampenDelft University of Technology, 2626HS Delft, The NetherlandsPublished Online:2 Sep 2024https://doi.org/10.2514/1.G008321SectionsRead Now ToolsAdd to favoritesDownload citationTrack citations ShareShare onFacebookXLinked InRedditEmail About References [1] "Loss of Control In-Flight Accident Analysis Report," Tech. Rept., International Air Transport Association (IATA), Geneva, 2019, https://www.iata.org/contentassets/b6eb2adc248c484192101edd1ed36015/loc-i_2019.pdf. Google Scholar[2] Dydek Z. T., Annaswamy A. M. and Lavretsky E., "Adaptive Control and the NASA X-15-3 Flight Revisited," IEEE Control Systems Magazine, Vol. 30, No. 3, 2010, pp. 32–48. https://doi.org/10.1109/MCS.2010.936292 CrossrefGoogle Scholar[3] Annaswamy A. M. and Fradkov A. L., "A Historical Perspective of Adaptive Control and Learning," Annual Reviews in Control, Vol. 52, Jan. 2021, pp. 18–41. https://doi.org/10.1016/j.arcontrol.2021.10.014 CrossrefGoogle Scholar[4] Enns R. and Si J., "Neuro-Dynamic Programming Applied to Helicopter Flight Control," AIAA Guidance, Navigation, and Control Conference and Exhibit, AIAA Paper 2000-4280, 2000. https://doi.org/10.2514/6.2000-4280 Google Scholar[5] Ferrari S. and Stengel R. F., "Online Adaptive Critic Flight Control," Journal of Guidance, Control, and Dynamics, Vol. 27, No. 5, 2004, pp. 777–786. https://doi.org/10.2514/1.12597 LinkGoogle Scholar[6] Helder B., van Kampen E.-J. and Pavel M., "Online Adaptive Helicopter Control Using Incremental Dual Heuristic Programming," AIAA Scitech 2021 Forum, AIAA Paper 2021-1118, 2021. https://doi.org/10.2514/6.2021-1118 LinkGoogle Scholar[7] Heyer S., Kroezen D. and Van Kampen E., "Online Adaptive Incremental Reinforcement Learning Flight Control for a CS-25 Class Aircraft," AIAA Scitech 2020 Forum, AIAA Paper 2020-1844, 2020. https://doi.org/10.2514/6.2020-1844 LinkGoogle Scholar[8] Jacklin S. A., "Closing the Certification Gaps in Adaptive Flight Control Software," AIAA Guidance, Navigation and Control Conference and Exhibit, AIAA Paper 2008-6988, 2008. https://doi.org/10.2514/6.2008-6988 LinkGoogle Scholar[9] Zhou Y., Van Kampen E.-J. and Chu Q., "Incremental Model-Based Online Dual Heuristic Programming for Nonlinear Adaptive Control," Control Engineering Practice, Vol. 73, April 2018, pp. 13–25. https://doi.org/10.1016/j.conengprac.2017.12.011 CrossrefGoogle Scholar[10] Recht B., "A Tour of Reinforcement Learning: The View from Continuous Control," Annual Review of Control, Robotics, and Autonomous Systems, Vol. 2, No. 1, 2019, pp. 253–279. https://doi.org/10.1146/annurev-control-053018-023825 CrossrefGoogle Scholar[11] Dean S., Mania H., Matni N., Recht B. and Tu S., "On the Sample Complexity of the Linear Quadratic Regulator," Foundations of Computational Mathematics, Vol. 20, No. 4, 2019, pp. 633–679. https://doi.org/10.1007/s10208-019-09426-y CrossrefGoogle Scholar[12] Konatala R., Kampen E.-J. V. and Looye G., "Reinforcement Learning Based Online Adaptive Flight Control for the Cessna Citation II (PH-LAB) Aircraft," AIAA Scitech 2021 Forum, AIAA Paper 2021-0883, 2021. https://doi.org/10.2514/6.2021-0883 LinkGoogle Scholar[13] Zhou Y., Van Kampen E. and Chu Q., "Nonlinear Adaptive Flight Control Using Incremental Approximate Dynamic Programming and Output Feedback," Journal of Guidance Control and Dynamics, Vol. 40, No. 2, 2017, pp. 493–496. https://doi.org/10.2514/1.G001762 LinkGoogle Scholar[14] Zhou Y., van Kampen E.-J. and Chu Q., "Incremental Approximate Dynamic Programming for Nonlinear Adaptive Tracking Control with Partial Observability," Journal of Guidance, Control, and Dynamics, Vol. 41, No. 12, 2018, pp. 2554–2567. https://doi.org/10.2514/1.G003472 LinkGoogle Scholar[15] Lewis F. L. and Vrabie D., "Reinforcement Learning and Adaptive Dynamic Programming for Feedback Control," IEEE Circuits and Systems Magazine, Vol. 9, No. 3, 2009, pp. 32–50. https://doi.org/10.1109/MCAS.2009.933854 CrossrefGoogle Scholar[16] Bradtke S. J. and Ydstie B. E., "Adaptive Linear Quadratic Control Using Policy Iteration," Proceedings of 1994 American Control Conference-ACC'94, Vol. 3, Inst. of Electrical and Electronics Engineers, New York, 1994, pp. 3475–3479. https://doi.org/10.1109/ACC.1994.735224 Google Scholar[17] Bellman R., "Dynamic Programming," Science, Vol. 153, No. 3731, 1957, pp. 34–37. https://doi.org/10.1126/science.153.3731.34 Google Scholar[18] Dias P. M., Zhou Y. and Kampen E.-J. V., "Intelligent Nonlinear Adaptive Flight Control Using Incremental Approximate Dynamic Programming," AIAA Scitech 2019 Forum, AIAA Paper 2019-2339, 2019. https://doi.org/10.2514/6.2019-2339 LinkGoogle Scholar[19] Konatala R., Kampen E.-J. V., Looye G., Milz D. and Weiser C., "Flight Testing Reinforcement Learning Based Online Adaptive Flight Control Laws on CS-25 Class Aircraft," AIAA Scitech 2024 Forum, AIAA Paper 2024-2402, 2024. https://doi.org/10.2514/6.2024-2402 LinkGoogle Scholar[20] Konatala R., Müller R., May M., Looye G. and van Kampen E.-J., "Verification & Validation (V&V) of Reinforcement Learning Based Online Adaptive Flight Control Laws on CS-25 Class Aircraft," Proceedings of the 2024 CEAS EuroGNC Conference, Council of European Aerospace Societies, Bristol, U.K., Paper CEAS-GNC-2024-062, 2024. Google Scholar[21] "Flight Control Design—Best Practices," Defense Technical Information Center, Research, and Technology Organization, NATO, 2000, https://apps.dtic.mil/sti/tr/pdf/ADA387777.pdf. Google Scholar[22] Narendra K. S. and Annaswamy A. M., "Persistent Excitation in Adaptive Systems," International Journal of Control, Vol. 45, No. 1, 1987, pp. 127–160. https://doi.org/10.1080/00207178708933715 CrossrefGoogle Scholar[23] Joos H.-D., "A Methodology for Multi-Objective Design Assessment and Flight Control Synthesis Tuning," Aerospace Science and Technology, Vol. 3, No. 3, 1999, pp. 161–176. https://doi.org/10.1016/S1270-9638(99)80040-6 CrossrefGoogle Scholar Previous article Next article FiguresReferencesRelatedDetails What's Popular Articles in Advance Metrics CrossmarkInformationCopyright © 2024 by the American Institute of Aeronautics and Astronautics, Inc. All rights reserved. All requests for copying and permission to reprint should be submitted to CCC at www.copyright.com; employ the eISSN 1533-3884 to initiate your request. See also AIAA Rights and Permissions www.aiaa.org/randp. TopicsAeronauticsAircraft ControlAircraft Flight Control SystemAircraft Operations and TechnologyAircraft Stability and ControlAircraftsArtificial IntelligenceAutomatic Flight Control SystemsAviationAviation SafetyComputing, Information, and CommunicationData ScienceFlight TestGuidance, Navigation, and Control SystemsMachine Learning KeywordsReinforcement LearningFlight Control LawFlight TestingAircraft CategoryAircraft Flight Control SystemAdaptive Control MethodAutomatic Flight Control SystemsAdaptive ControlAcknowledgmentsThe authors express their gratitude to the personnel at the Delft University of Technology, including Alexander in't Veld, Hans Mulder, Menno Klaassen, Ferdinand Postema, Olaf Stroosma, René van Paassen, Fred den Toom, and Isabelle El-Hajj, for providing the framework and expertise that facilitated the successful execution of the flight tests. Special acknowledgment also goes to the DLR (German Aerospace Center) team members involved in the flight test campaign, namely, Richard Kuchar, Marc May, Reiko Müller, Stefan Langen, and Christina Schreppel, for their valuable support and expertise. A special acknowledgment goes to Carsten Oldemeyer for carefully reviewing the manuscript.Digital Received4 March 2024Accepted8 July 2024Published online2 September 2024