ssiltane / math_for_ml_2021 Goto Github PK

This repository is for materials and codes related to the course "Basics of Mathematics in Machine Learning I" given in January-March 2021 at University of Helsinki, Finland.

The lecturers are Prof. Samuli Siltanen and Dr. Fernando Silva de Moura. Teaching assistants are Salla Latva-Äijö and Siiri Rautio.

(1) How to use the Matlab codes in folder ./Weather_examples/

First run "WeatherData_from_Excel_to_mat.m". It will read in Excel files from subfolder ./Weather_examples/data/ and save selected weather measuerements as several .mat Matlab data files. (Of course, you can examine the Excel files and study the weather measurements. The Excel files whose name ends with Long.xlsx contain three tabs with information about the type of measurements. They are in the format provided by the open data service of Finnish Meteorological Institute available on page https://en.ilmatieteenlaitos.fi/download-observations. The Excel files whose name ends with Long2.xlsx have only one tab, identical to the third tab in the file with the same name apart from the "2". This two-file system is for making reading the data into Matlab more straightforwards.)

Second, run "NeuralNet_manual_plots.m" to examine the simple classifications of days based on either just the average temperature, or on temperature and air pressure data together. Here the two-neuron networks are not really trained; their weights and biases are simply chosen explicitly.

Third, examine the classification of April and July days with an automatically trained network. The classification is based on two pieces of measurement data for each day, namely average temperature and average air pressure. Run "NeuralNet_twodata_learned_plots.m". This classifies April and July days almost perfectly. The routine "NeuralNet_twodata_learned_plots.m" takes the weights and biases from the file ./Weather_examples/data/NN_parameters.mat. (Where does that file come from? It was computed by training the network with the code available in the subfolder ./Weather_examples/ML_Higham_applied2weather/. In that folder, you can train the network yourself using Matlab, possibly changing the network architecture, learning rate, and other parameters. Instructions: first run "WeatherData_Higham_AI_prepare.m" to produce file Highamdata.mat, where all weather measurements are normalized to the interval [0,1]. Then run "train_NN.m", which will train the network and save the obtained parameters in the file ./Weather_examples/ML_Higham_applied2weather/NN_parameters.mat. If you like the results, you can move this NN_parameters.mat file into the folder ./Weather_examples/data/ and replace the old one. The machine learning code is adapted from the excellent article [HH2019].)

Finally, you can run "NeuralNet_threedata_learned_plots.m". This routine classifies days of January, April and July based on three pieces of weather information: average temperature, average air pressure, and average humidity. This three-dimensional information is a bit harder to visualize, so the result is provided in the form of a list of classification accuracies output by Matlab. Also, there is a 3D plot of the data points, which we recommend rotating manually using Matlab's rotation tool in the image window. Note that the measurement units are not correct in this plot as it shows the data normalized to the interval [0,1]. The weights and biases of the network are given in the file ./Weather_examples/data/NN_parameters_3seasons.mat. (Where does that file come from? It was computed by training the network with the code available in the subfolder ./Weather_examples/ML_Higham_applied2weather_multi/. In that folder, you can train the network yourself using Matlab, possibly changing the network architecture, learning rate, and other parameters. Instructions: first run "WeatherData_Higham_AI_3seasons_3data_prepare.m" to produce file Highamdata_3seasons.mat, where all weather measurements are normalized to the interval [0,1]. Then run "train_3seasons.m", which will train the network and save the obtained parameters in the file ./Weather_examples/ML_Higham_applied2weather/NN_parameters_3seasons.mat. If you like the results, you can move this NN_parameters_3seasons.mat file into the folder ./Weather_examples/data/ and replace the old one. The machine learning code is adapted from the excellent article [HH2019].)

There are a couple of demo routines as well, used in the lectures for illustrating the action of matrices (linear maps) as parts of neural networks. You can run these two routines: NeuralNet_linearmap_demos.m and NeuralNet_linearmap_demos2.m.

References: [HH2019] Higham, C. F., & Higham, D. J. (2019). Deep learning: An introduction for applied mathematicians. SIAM Review, 61(4), 860-891.