Welcome to the homepage of the Machine Learning research group at the Institute of Computer Science, University of Tartu. Our main research topics are uncertainty in machine learning and more broadly, methods for trustworthy artificial intelligence. We have worked on context, uncertainty and representation learning, supervised learning (classifier adaptation, calibration, evaluation, ensembles, uncertainty estimation) and applications of machine learning and deep learning for machine perception, autonomous driving, neuroscience, biology and health.
People
Academic staff:
Meelis Kull
Raul Vicente Zafra
Ardi Tampuu
PhD students:
Mari-Liis Allikivi
Marharyta Domnich
Karl Kaspar Haavel
Joonas Järve
Viacheslav Komisarenko
Markus Kängsepp
Marek Leibl
Mihkel Lepson
Novin Shahroudi
Alumni (including bachelor, master and Phd students)
Teaching
2022 Fall
Introduction to Data Science (6 ects)- lecturer: Meelis Kull – teaching assistants: Victor Pinheiro, Anna Aljanaki, Markus Kängsepp, Ingvar Baranin, Friedrich Krull
Andmeteaduse meetodid (Methods in Data Science, in Estonian) (6 ects)- lecturer: Meelis Kull – teaching assistants: Joonas Järve, Carel Kuusk, Marilin Moor
Machine Learning (6 ects) – lecturer: Dmytro Fishman – teaching assistants: Lisa Yankovskaya, Victor Pinheiro, Pavel Chizhov, Joonas Ariva, Mohammed Ali
Introduction to Computational Neuroscience (6 ects) – lecturer: Raul Vicente Zafra – teaching assistants: Marharyta Domnich, Farid Hasanov, Kristjan Kaup
Computational Neuroscience Seminar (3 ects) – instructors: Raul Vicente Zafra, Aqeel Labash, Marharyta Domnich
Special Course in Machine Learning: Probabilistic Deep Learning (3 ects) – instructors: Sven Laur
Research Seminar in Data Mining (3 ects) – instructor: Sven Laur
2022 Spring
Machine Learning II (6 ects) – lecturer: Sven Laur
Neural Networks (6 ects)- lecturer: Raul Vicente Zafra – teaching assistants: Marharyta Domnich, Mari-Liis Allikivi, Tarun Khajuria, Victor Pinheiro
Computational Neuroscience Seminar (3 ects) – instructors: Oriol Corcoll, Raul Vicente Zafra
Special Course in Machine Learning: Uncertainty in Machine Learning
(3 ects) – instructors: Meelis Kull
Research Seminar in Data Mining (3 ects) – instructor: Sven Laur
2021 Fall
Introduction to Data Science (6 ects)- lecturer: Meelis Kull – teaching assistants: Victor Pinheiro, Anna Aljanaki, Markus Kängsepp, Farid Hasanov, Friedrich Krull
Andmeteaduse meetodid (Methods in Data Science, in Estonian) (6 ects)- lecturer: Meelis Kull – teaching assistants: Sander Tamm, Carel Kuusk, Simo Sirel
Machine Learning (6 ects) – lecturer: Dmytro Fishman – teaching assistants: Mohammed Ali, Victor Pinheiro, Tetiana Shtym, Tetiana Rabiichuk, Taavi Luik
Introduction to Computational Neuroscience (6 ects) – lecturer: Raul Vicente Zafra – teaching assistants: Marharyta Domnich, Farid Hasanov, Kristjan Kaup
Computational Neuroscience Seminar (3 ects) – instructors: Oriol Corcoll, Raul Vicente Zafra
Special Course in Machine Learning: Geometric Deep Learning (3 ects) – instructors: Raul Vicente, Florian Stelzer, Kallol Roy
Research Seminar in Data Mining (3 ects) – instructor: Sven Laur
Past teaching (since 2017 Fall)
Industrial collaborations
Current:
PerkinElmer (since 2017)
Bolt (since 2019)
Selected publications
Evaluation of Trajectory Distribution Predictions with Energy Score
N Shahroudi, M Lepson, M Kull
Conference: ICML’24 – The International Conference on Machine Learning
2024
Cautious Calibration in Binary Classification
M-L Allikivi, J Järve, M Kull
Conference: ECAI’24 – European Conference on Artificial Intelligence
2024
Improving Calibration by Relating Focal Loss, Temperature Scaling, and Properness
V Komisarenko, M Kull
Conference: ECAI’24 – European Conference on Artificial Intelligence
2024
Classifier calibration: a survey on how to assess and improve predicted class probabilities
T Silva Filho, H Song, M Perello-Nieto, R Santos-Rodriguez, M Kull, P Flach
Journal: Machine Learning
2023
Assuming Locally Equal Calibration Errors for Non-Parametric Multiclass Calibration
K Valk, M Kull
Journal: Transactions on Machine Learning Research
2023
Generality-Training of a Classifier for Improved Calibration in Unseen Contexts
B.S. Leelar, M Kull
Conference: ECML PKDD 2023 – Joint European Conference on Machine Learning and Knowledge Discovery in Databases
2023
Quantifying Reinforcement-Learning Agent’s Autonomy, Reliance on Memory and Internalisation of the Environment.
A Ingel, A Makkeh, O Corcoll, R Vicente
Journal: Entropy
2022
Instance-based Label Smoothing For Better Calibrated Classification Networks
M Maher, M Kull
Conference: ICMLA’21 – Proceedings of the 20th IEEE International Conference on Machine Learning and Applications
2021
Information Bottleneck as Optimisation Method for SSVEP-Based BCI
A Ingel, R Vicente
Journal: Frontiers in Human Neuroscience
2021
Deep neural networks using a single neuron: folded-in-time architecture using feedback-modulated delay loops
F Stelzer, A Röhm, R Vicente, I Fischer, S Yanchuk
Journal: Nature communications
2021
A granular recurrent neural network for multiple time series prediction
S Tomasiello, V Loia, A Khaliq
Journal: Neural Computing and Applications
2021
Evaluating Very Deep Convolutional Neural Networks for Nucleus Segmentation from Brightfield Cell Microscopy Images
MAS Ali, O Misko, SO Salumaa, M Papkov, K Palo, D Fishman, L Parts
Journal: SLAS DISCOVERY: Advancing the Science of Drug Discovery
2021
Correlated daily time series and forecasting in the M4 competition
A Ingel, N Shahroudi, M Kängsepp, A Tättar, V Komisarenko, M Kull
Journal: International Journal of Forecasting
2020
A Survey of End-to-End Driving: Architectures and Training Methods
A Tampuu, T Matiisen, M Semikin, D Fishman, N Muhammad
Journal: IEEE Transactions on Neural Networks and Learning Systems
2020
Perspective Taking in Deep Reinforcement Learning Agents
A Labash, J Aru, T Matiisen, A Tampuu, R Vicente
Journal: Frontiers in Computational Neuroscience
2020
A model for time interval learning in the Purkinje cell
D Majoral, A Zemmar, R Vicente
Journal: PLoS Computational Biology
2020
On a granular functional link network for classification
F Colace, V Loia, W Pedrycz, S Tomasiello
Journal: Neurocomputing
2020
Beyond temperature scaling: Obtaining well-calibrated multi-class probabilities with Dirichlet calibration
M Kull, MP Nieto, M Kängsepp, T Silva Filho, H Song, P Flach
Conference: NeurIPS’19 – Advances in Neural Information Processing Systems
2019
Non-parametric Bayesian Isotonic Calibration: Fighting Over-confidence in Binary Classification
M Allikivi, M Kull
Conference: ECML-PKDD’19 – The European Conference on Machine Learning & Principles and Practice of Knowledge Discovery in Databases
2019
Shift Happens: Adjusting Classifiers
TJT Heiser, M Allikivi, M Kull
Conference: ECML-PKDD’19 – The European Conference on Machine Learning & Principles and Practice of Knowledge Discovery in Databases
2019
Distribution calibration for regression
H Song, T Diethe, M Kull, P Flach
Conference: ICML’19 – The International Conference on Machine Learning
2019
Segmenting nuclei in brightfield images with neural networks
D Fishman, S Salumaa, D Majoral, S Peel, J Wildenhain, A Schreiner, K Palo, L Parts
Preprint: bioRxiv, https://www.biorxiv.org/content/early/2019/09/10/764894
2019
Activations of Deep Convolutional Neural Network are Aligned with Gamma Band Activity of Human Visual Cortex
I Kuzovkin, R Vicente, M Petton, J Lachaux, M Baciu, P Kahane, S Rheims, JR Vidal, J Aru
Journal: Communications Biology
2018
Efficient neural decoding of self-location with a deep recurrent network
A Tampuu, T Matiisen, HF Ólafsdóttir, C Barry, and R Vicente
Preprint: bioRxiv 242867
2018
Releasing eHealth Analytics into the Wild: Lessons Learnt from the SPHERE Project
T Diethe, M Holmes, M Kull, M Perello Nieto, K Sokol, H Song, E Tonkin, N Twomey, P Flach
Conference: KDD Applied Data Science track
2018
Prediction of antibiotic resistance in Escherichia coli from large-scale pan-genome data
D Moradigaravand, M Palm, A Farewell, V Mustonen, J Warringer, L Parts
Journal: PLoS computational biology
2018
Self-regulated learning with approximate reasoning and situation awareness
G D’Aniello, A Gaeta, M Gaeta, S Tomasiello
Journal: Journal of Ambient Intelligence and Humanized Computing
2018
Accurate Classification of Protein Subcellular Localization from High-Throughput Microscopy Images Using Deep Learning
T Pärnamaa, L Parts
Journal: G3: Genes, Genomes, Genetics
2017
Beyond sigmoids: How to obtain well-calibrated probabilities from binary classifiers with beta calibration
M Kull, TM Silva Filho, P Flach
Journal: Electronic Journal of Statistics
2017
Bivariate Partial Information Decomposition: The Optimization Perspective
M Abdullah, DO Theis, R Vicente
Journal: Entropy 19, no. 10, 530
2017
Linear Ensembles of Word Embedding Models.
A Muromägi, K Sirts, S Laur
Conference: NODALIDA (Nordic Conference on Computational Linguistics)
2017
Multiagent cooperation and competition with deep reinforcement learning
A Tampuu, T Matiisen, D Kodelja, I Kuzovkin, K Korjus, Juhan Aru, Jaan Aru, R Vicente
Journal: PloS one 12, no. 4: e0172395
2017
An efficient data partitioning to improve classification performance while keeping parameters interpretable
K Korjus, MN Hebart, R Vicente
Journal: PloS one 11, no. 8, e0161788.
2016
Cost-sensitive boosting algorithms: Do we really need them?
N Nikolaou, N Edakunni, M Kull, P Flach, G Brown
Journal: Machine Learning
2016 – ACM Computing Reviews named it as one of the most notable items published in computer science in 2016
Deep learning for computational biology
C Angermueller, T Pärnamaa, L Parts, O Stegle
Journal: Molecular Systems Biology
2016
EstNLTK – NLP Toolkit for Estonian
S Orasmaa, T Petmanson, A Tkachenko, S Laur, HJ Kaalep
Conference: LREC (International Conference on Language Resources and Evaluation)
2016
Reframing in context: A systematic approach for model reuse in machine learning
J Hernández-Orallo, A Martínez-Usó, RBC Prudêncio, M Kull, P Flach, CF Ahmed, N Lachiche
Journal: AI Communications
2016
Precision-recall-gain curves: PR analysis done right
P Flach, M Kull
Conference: NIPS’15 – Advances in Neural Information Processing Systems
2015
Novel decompositions of proper scoring rules for classification: Score adjustment as precursor to calibration
M Kull, P Flach
Conference: ECML’15 – Joint European Conference on Machine Learning and Knowledge Discovery in Databases
2015
Versatile decision trees for learning over multiple contexts
R Al-Otaibi, RBC Prudêncio, M Kull, P Flach
Conference: ECML’15 – Joint European Conference on Machine Learning and Knowledge Discovery in Databases
2015
Reframing in frequent pattern mining
CF Ahmed, M Samiullah, N Lachiche, M Kull, P Flach
Conference: ICTAI’15 – IEEE 27th International Conference on Tools with Artificial Intelligence
2015
Privacy-Preserving Statistical Data Analysis on Federated Databases.
D Bogdanov, L Kamm, S Laur, P Pruulmann-Vengerfeldt, R Talviste, J Willemson
Conference: APF (Annual Privacy Forum)
2014
Rate-oriented point-wise confidence bounds for ROC curves
LAC Millard, M Kull, PA Flach
Conference: ECML-PKDD’14 – The European Conference on Machine Learning & Principles and Practice of Knowledge Discovery in Databases
2014
Reliability maps: a tool to enhance probability estimates and improve classification accuracy
M Kull, PA Flach
Conference: ECML-PKDD’14 – The European Conference on Machine Learning & Principles and Practice of Knowledge Discovery in Databases – Best paper award
2014