With the advent of Generative AI, it has become increasingly critical to monitor data and models (of any type) in production. This need has led to the development of numerous techniques and tools dedicated to the field of Monitoring and Observability.
In this talk, I will explore the most effective methods for monitoring AI solutions, covering different applications that include Large Language Models and classical Machine Learning, both in batch and streaming contexts.
I will analyse the common challenges, such as detecting data drift, managing anomalies and controlling model performance over time.
Finally, I will present the open-source AI monitoring solution developed by Radicalbit, a company with extensive experience in MLOps. This solution not only offers the benefits of continuous and in-depth model monitoring, guaranteeing the reliability and efficiency of AI implementations in production, but it also fosters a collaborative environment in which the community continuously improves the solution through contributions.