Meelis Kull
I’m an associate professor in machine learning, leading the machine learning research group at the Institute of Computer Science, University of Tartu, Estonia. I am also an honorary senior research associate at the University of Bristol, UK.
My research interests are machine learning, artificial intelligence, and data science, with a focus on uncertainty quantification and trustworthiness. I enjoy theoretical research with a clear path to applications and my work is primarily on the statistical side of machine learning.
Selected activities:
Head of the Estonian Centre of Excellence in Artificial Intelligence (EXAI)
Program co-chair of ECML PKDD 2024, Vilnius, Lithuania
Organiser of OpenML Workshop 2023, Tartu, Estonia
Organiser of LMCE 2015 – 2nd International Workshop on Learning over Multiple Contexts at ECML PKDD 2015
Keynote at the WUML 2020 Workshop on Uncertainty in Machine Learning at ECML PKDD 2020
Tutorial at ECML PKDD 2020: “Context-Aware Knowledge Discovery: Opportunities, Techniques and Applications”
Tutorial at ECML PKDD 2016: “Classifier Calibration How to assess and improve classifier confidence and uncertainty”
Editorial board member for Machine Learning journal, since 2021
Reviewing for journals Machine Learning (MLj), Data Mining and Knowledge Discovery (DAMI), Transactions on Machine Learning Research (TMLR), Journal of Artificial Intelligence Research (JAIR), International Journal of Forecasting (IJF), Journal of Machine Learning Research (JMLR),
Reviewing for conferences ICML, NeurIPS, ICLR, IJCAI, AISTATS, KDD, ECML PKDD
Grants and awards:
Head of the Estonian Centre of Excellence in Artificial Intelligence (EXAI, 2024-2030)
PI of “Contextual uncertainty and representation learning in machine perception” (Estonian Research Council ETAg team grant, 2022-2026)
PI of “Making intelligent systems avoid over-confidence: Theory and software to learn and recalibrate probabilistic classifiers under uncertainty about the application context” (Estonian Research Council ETAg starting grant, 2017-2020)
Teacher of the Year 2019 at the Institute of Computer Science, University of Tartu (awarded by the students)
Teacher of the Year 2019 at the Institute of Computer Science, University of Tartu (awarded by the institute)
Supervision of Theodore Heiser to win the First Prize at the Estonian National Contest for Student Research, in the Master thesis category within the field of Natural Sciences and Engineering
ACM Computing Reviews listed the research article “Cost-sensitive boosting algorithms: Do we really need them?” among the most notable items published in the field of computing in 2016
Best Paper Award at the ECML-PKDD 2014 (European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases) for the paper on “Reliability maps: a tool to enhance probability estimates and improve classification accuracy”
Second Award of Estonian National Contest for Student Research (for doctoral thesis) in 2011
Best Poster Award on international conference ISMB 2004 (Intelligent Systems for Molecular Biology)
Second Award of Estonian Academy of Sciences Contest for Student Research in 2004 (for master thesis)
Postdocs:
Bhawani Shankar Leelar
PhD students:
Mari-Liis Allikivi
Karl Kaspar Haavel
Joonas Järve
Viacheslav Komisarenko
Markus Kängsepp
Mihkel Lepson
Novin Shahroudi
Teaching
Andmeteaduse meetodid (Methods in Data Science, in Estonian) (6 ects) – fall 2021, fall 2022, fall 2023
Introduction to Data Science (6 ects) – fall 2018, fall 2019, fall 2020, fall 2021, fall 2022, fall 2023
Machine Learning (6 ects) – spring 2018, fall 2018, fall 2019
Data Mining (6 ects) – fall 2017
Special Course in Machine Learning: Uncertainty in Machine Learning
(3 ects) – spring 2022
Special Course in Machine Learning: M4 Regression Competition (3 ects) – spring 2018
Special Course in Machine Learning: Ensemble methods (3 ects) – fall 2017
Selected publications (see full list at Google scholar):
Generality-training of a Classifier for Improved Calibration in Unseen Contexts
B S Leelar, M Kull
Conference: ECML PKDD 2023 – The European Conference on Machine Learning & Principles and Practice of Knowledge Discovery in Databases
2023
Classifier calibration: a survey on how to assess and improve predicted class probabilities
T Silva Filho, H Song, M Perello-Nieto, R Santos-Rodriguez, M Kull, P Flach
Journal: Machine Learning
2023
Sodium adduct formation with graph-based machine learning can aid structural elucidation in non-targeted LC/ESI/HRMS.
R Costalunga, S Tshepelevitsh, H Sepman, M Kull, A Kruve
Journal: Analytica Chimica Acta
2022
Evaluating Classifiers’ Performance to Detect Attacks in Website Traffic
D Urda, N Basurto, M Kull, A Herrero
Conference: CISIS 2022 – 15th International Conference on Computational Intelligence in Security for Information Systems
2022
Instance-based Label Smoothing For Better Calibrated Classification Networks
M Maher, M Kull
Conference: ICMLA’21 – Proceedings of the 20th IEEE International Conference on Machine Learning and Applications
2021
CRISP-DM Twenty Years Later: From Data Mining Processes to Data Science Trajectories
F Martinez-Plumed, L Contreras-Ochando, C Ferri, J Hernandez Orallo, M Kull, N Lachiche, M J Ramirez Quintana, P Flach
Journal: IEEE Transactions on Knowledge and Data Engineering
2021
Correlated daily time series and forecasting in the M4 competition
A Ingel, N Shahroudi, M Kängsepp, A Tättar, V Komisarenko, M Kull
Journal: International Journal of Forecasting
2020
Non-parametric Bayesian Isotonic Calibration: Fighting Over-confidence in Binary Classification
M Allikivi, M Kull
Conference: ECML-PKDD’19 – The European Conference on Machine Learning & Principles and Practice of Knowledge Discovery in Databases
2020
Shift Happens: Adjusting Classifiers
TJT Heiser, M Allikivi, M Kull
Conference: ECML-PKDD’19 – The European Conference on Machine Learning & Principles and Practice of Knowledge Discovery in Databases
2020
Beyond temperature scaling: Obtaining well-calibrated multi-class probabilities with Dirichlet calibration
M Kull, MP Nieto, M Kängsepp, T Silva Filho, H Song, P Flach
Conference: NeurIPS’19 – Advances in Neural Information Processing Systems
2019
Distribution calibration for regression
H Song, T Diethe, M Kull, P Flach
Conference: ICML’19 – The International Conference on Machine Learning
2019
Releasing eHealth Analytics into the Wild: Lessons Learnt from the SPHERE Project
T Diethe, M Holmes, M Kull, M Perello Nieto, K Sokol, H Song, E Tonkin, N Twomey, P Flach
Conference: KDD Applied Data Science track
2018
Beyond sigmoids: How to obtain well-calibrated probabilities from binary classifiers with beta calibration
M Kull, TM Silva Filho, P Flach
Journal: Electronic Journal of Statistics
2017
Beta calibration: a well-founded and easily implemented improvement on logistic calibration for binary classifiers
M Kull, TM Silva Filho, P Flach
Conference: AISTATS 2017 – Proceedings of the 20th International Conference on Artificial Intelligence and Statistics
2017
Cost-sensitive boosting algorithms: Do we really need them?
N Nikolaou, N Edakunni, M Kull, P Flach, G Brown
Journal: Machine Learning
2016 – ACM Computing Reviews named it as one of the most notable items published in computer science in 2016
Reframing in context: A systematic approach for model reuse in machine learning
J Hernández-Orallo, A Martínez-Usó, RBC Prudêncio, M Kull, P Flach, CF Ahmed, N Lachiche
Journal: AI Communications
2016
Precision-recall-gain curves: PR analysis done right
P Flach, M Kull
Conference: NIPS’15 – Advances in Neural Information Processing Systems
2015
Novel decompositions of proper scoring rules for classification: Score adjustment as precursor to calibration
M Kull, P Flach
Conference: ECML’15 – Joint European Conference on Machine Learning and Knowledge Discovery in Databases
2015
Versatile decision trees for learning over multiple contexts
R Al-Otaibi, RBC Prudêncio, M Kull, P Flach
Conference: ECML’15 – Joint European Conference on Machine Learning and Knowledge Discovery in Databases
2015
Rate-oriented point-wise confidence bounds for ROC curves
LAC Millard, M Kull, PA Flach
Conference: ECML-PKDD’14 – The European Conference on Machine Learning & Principles and Practice of Knowledge Discovery in Databases
2014
Reliability maps: a tool to enhance probability estimates and improve classification accuracy
M Kull, PA Flach
Conference: ECML-PKDD’14 – The European Conference on Machine Learning & Principles and Practice of Knowledge Discovery in Databases – Best paper award
2014