Research Resource I

Abstract

This long-term article contains all resources available and ready to be used, on the study of artificial intelligence and related concepts. ___

0.IXS - Research Journals Collection

• Computational Intelligence (Diana Inkpen) - Computational Intelligence is an artificial intelligence journal publishing novel research on a broad range of experimental and theoretical topics in AI and computer science.
  • Coverage: Machine learning, knowledge mining, web intelligence, AI language, and philosophical implications.

• I - Research Papers Reading

1. An efficient encoder-decoder architecture with top-down attention for speech separation Kai Li,Runxuan Yang, Xiaolin Hu - Arxiv Paper
2. Understanding How Encoder-Decoder Architectures Attend Kyle Aitken et al. - Research Paper .
3. Activation Functions in Deep Learning: A Comprehensive Survey and Benchmark Shiv Ram Dubey, Satish Kumar Singh, Bidyut Baran Chaudhuri - Arxiv Paper
4. Novel mixture allocation models for topic learning - Kamal Maanicshah, Manar Amayri, Nizar Bouguila - doi.org. A very interesting read about topic modelling. Wiley and Original Article.
5. Logic Tensor Networks (Samy Badreddine, Artur d’Avila Garcez, Luciano Serafini, Michael Spranger) - First order logic and neurosymbolic treatment of neural network.
6. The Modern Mathematics of Deep Learning - Julius Berner, Philipp Grohs, Gitta Kutyniok, Philipp Petersen - Arxiv.
7. Towards a Mathematical Understanding of Neural Network-Based Machine Learning: what we know and what we don’t - Weinan E, Chao Ma, Stephan Wojtowytsch, Lei Wu - Arxiv.

II. Articles for Resource Reading

IBM AI Development

• Unsupervised learning for data classification
• Tutorial on Artificial Intelligence
• What is self-supervised learning?

Statistic StackExchange PL

• Which notation and why: \(\text{P}()\), \(\Pr()\), \(\text{Prob}()\), or \(\mathbb{P}()\)? - link.
• How to calculate the likelihood function - Surprisingly, this question has an answer that really, well, explain likelihood than other “answers”.
• Statistical learning theory versus computational learning theory?

Artificial Intelligence PL

• What is the difference between self-supervised and unsupervised learning?
• What is self-supervised learning in machine learning?
• Can we Consider Regularization as a “Constraint”?.

---

LXL - Lectures and Notes

Mixed

1. https://ufal.mff.cuni.cz/~helcl/courses/npfl116/slides/03-encoder-decoder.pdf - On Encoder - Decoder Architecture.
2. What does PAC learning theory really mean?
3. Error and Residue
4. Regression Analysis
5. 8803 Machine Learning Theory - CMU.

Mathematical Aspects of Deep Learning

Main link: here.

Czech Technical University in Prague DokuWiki

Main site: link.

Principles and Techniques of Data Science (UC Berkeley, Summer 2020)

• Syllabus
• Calendar
• Resources
• Staff
• Github Repository Account - Berkeley Data Science

Cambridge MSc Advanced Study - Department of Quantum Physics and Computing

Quantum Computation, Information and Foundations Part III - Quantum Computation Part IB - Quantum Mechanics Part II - Quantum Information and Computation

Anne Sabourin’s Courses on Statistic and Statistical Learning

She is a professor at previously Telecom ParisTech and is now a professor in Université Paris Cité. There are many interesting, high level lecture notes and previous notes on lectures of advanced study on statistics, as well as learning theory and machine learning theory. - Main link: here

Present Courses

• Exploratory Data Analysis (Master’s program Mathematics and Applications (MDA), 1st year )
• Mathematical statistics, jointly with Fabienne Comte (Master’s program Mathematics and Applications (MDA) ), track IMB, 1st year
• Non Parametric Statistics (Master’s program Mathematics and Applications (MDA), 2nd year)
• High Dimensional statistics ( master MDA , 2nd year, track MMA)
• Statistical Learning with Extreme Values , master MVA

Main Past Courses

• Survival analysis (Master’s program Mathematics and Applications (MDA), IMB track, 2nd year, Université Paris-Cité )
• Tail events analysis: Robustness, outliers and models for extreme values (Joint course with Pavlo Mozharovskyi)
• Statistics-Linear Model, MS Big Data, MDI-720 (Telecom Paris)
• Extremes ( Lecture notes ), Master’s program “Mathématiques de l’aléatoire”, Paris-Saclay University- Orsay.
• Numerical methods (MACS-205, Telecom ParisTech, Spring semester)
• Introduction to Bayesian learning , Master Data sciences, (Ecole Polytechnique, Institut polytechnique de Paris)
• Statistics MDI-220 (Telecom Paris)
• Bayesian filtering (SIGMA-203-b, Telecom ParisTech, Spring semester)
• Probability MDI-104 (Telecom ParisTech, Fall semester)
• Computational statistics and optimisation, Master ‘Data Science’, Ecole Polytechnique. ( Lecture notes ): Convex problems in Machine Learning, non smooth convex analysis, Fenchel-Legendre transform, duality.

A Short Course on Nonparametric Curve Estimation - MSc in Applied Mathematics at EAFIT University (Colombia), 2017

Main links to the course: here

Introduction to Programming Synthesis (MIT, Solar-Lezama)

Lectures 1 and Main site - Overall course have 24 lectures. Important.

Dive Deep into Deep Learning (Alex J. Smola et al)

Main link to source: link. This is a very important text, and is used to be the main resource for deep learning study.

AstroML Project

Main page. Ridge and Lasso: Geometric Interpretation - Illustration on Ridge and Lasso Regularized Regression (L1 and L2).

---