Daily reports for Machine Learning features in Scilab

Project wiki

Introduction

Proposal : Machine Learning features in Scilab

An effort by Soumitra Agarwal under the mentors -

• Aashay Singhal
• Mandar Deshpande

Daily reports

15th May

• Implementation of the Jupyter hub setup initiated on the local machine after the yyintegration of last year's implementation.

16th May

• Jupyter hub only supports python3 so the idea was kept on hold.
• Contacted Simon and Clement for further discussion and python3 guidelines.

17th May

• Set up a virtual instance using google cloud.
• Initialisation of the development and experimentation forking procedure for the summer.

18th May

• Installed anaconda (and other required packages) onto the google vm

19th May

• Transfer of the file to the local machine from the cloud instance completed using scp
• Set up a daily update page for the project
• Set up project wiki

• Set up github repository

21st May

• Set up automated scripts to start the ipython kernel on the remote host from the client
• After the kernel is started, a copy script can be run to automatically get the json config file on the local machine

22nd May

• Initialise development procedure with linear regression scripts

• Add dataset from Kaggle to the github repo.

23rd May

• Formulate addition of a standalone toolbox for machine learning in Scilab
• Formulate threading mechanism to run the scripts from experimentation independently

24th May

• Add polynomial regression to the set of available scripts in the development section

25th May

• Add Kmeans clustering to the development section
• Separate development section into algorithms, pre-processing and visualisation

26th May

• Add Normalisation and train test split to the pre-processing section under the development header

27th May

• Add Scaling to the pre-processing section
• Formulate the authentication mechanism for the experimentation aspect of the project
• Set up GCP machine initiated by mentor

28th May

• Add Naive Bayes to the algorithms section under development (Gaussian)

29th May

• Add decision tree classification to the algorithms section (Training)
• Begin work on the authentication mechanism to prevent a user from doing anything other than initialising a kernel and copying its json

30th May

• Add command to the authorized_keys option for OpenSSH for the GCP server
• Complete decision tree classification with prediction test

31st May

• Update the project wiki
• Send concise report on the mailing list

1st June

• Updated readme with function usage
• Add comments to code for usage

2nd June

• Prepare timeline with agendas to complete to ensure completion of project

3rd June

• Commence process of automation for the experimentation part with the GCP server
• Add mechanism to copy dataset and scripts to the server

4th June

• Figure out starting a kernel on the server using python (nohup) with the same python_local script as earlier

5th June

• Complete automation architecture with copying mechanism to get the attributes file back
• Figure out kernel tagging mechanism to a user of the GCP server
• Start a new kernel everytime a user wants to run a model and discard previous

6th June

• Implement mechanism to generate missing values and add it to the preprocessing macros setup

7th June

• Implement 1 hot encoding mechanism and add it to the preprocessing macros
• Implement binarization mechanism with the preprocessing macros using the earlier implemented normalization

8th June

• Implement principal component analysis for preprocessing

9th June

• Add single layer perceptron to the set of algorithms in the standalone toolbox
• Add K-Nearest Neighbours classification to the algorithms set

10th June

• Add Ridge regression to algorithms toolbox
• Add Lasso regression to algorithms toolbox

11th June

• Add Elastic net to the set of algorithms
• Add spectral clustering to the set of algorithms

12th June

• Add tests for Elastic Net, Kmeans, Spectral Clustering, Lasso regression, KNN classification and decision tree classification

13th June

• Add LARS regression and multinomial Naive Bayes with tests

14th June

• Add Ensemble learning method (with result comparison) for the set of algorithms

15th June

• Add regression tree learning algorithm to the set of given algorithms

16th June

• Added K-Mediods clustering algorithm to algorithms set
• Added affinity propagation to the set of algorithms

17th June

• Add random forest classification algorithm

18th June

• Add support vector regression for set of algorithms
• Add support vector classification for set of algorithms

19th June

• Start neural network initial setup
• Begin reading on building the toolbox

20th June

• Complete neural network training part
• Add oversampling class rebalancing macro

21st June

• Complete neural network prediction

22nd June

• Add class rebalancing using clustering
• Send updates to the mailing list

23rd June

• Complete toolbox build

24th June

• Add demos
• Add readme
• Add tests

25th June

• Resume experimentation. Initiate packet tests

26th June

• Complete tests for packets

27th June

• Initiate script migration to server

28th June

• Complete migration setup

29th June

• Update template files for toolbox

30th June

• Complete url method for server based learning

1st July

• Publish toolbox

2nd July

• Complete authentication mechanism for Experimentation

3rd July

• Complete custom script for Experimentation

4th July

• Complete custom script with URL download

5th July

• Start toolbox structuring for cloud

6th July

• Perform minor corrections ML toolbox

8th July

• Publish a guide to mentors
• Add scripts to server

9th - 12th July

• Complete cloud machine learning toolbox

13th July - 31st July

• Add multiple enhancements to the cloud toolbox (separating URL download, prevent recurrent prompt for username/pass)
• Structure enhancements into toolbox

1st August - 6th August

• Scrub and structure final code
• Complete blog