GSoC 2017 - Matlab Mex Files Wrapper

Project Link

• Matlab MEX-Files management

Student and Mentors

Student Name -

• Siddhartha Gairola

Mentors -

• Clément David
• Dhruv Khattar

Timeline for GSoC 2017

This timeline is for after the students projects were announced on 4th of May, 2017.

• May 5 - 30, 2017 - Community Bonding
• May 30, 2017 - June 30, 2017 - Students Work on their Projects.
• June 26 - 30, 2017 - First Evaluations
• July 1 - 24, 2017 - Students Continue Coding
• July 24 - 28, 2017 - Second Evaluations
• July 28, 2017 - August 21, 2017 - Students Continue Coding
• August 21 - 29, 2017 - Students Submit Code and Evaluations
• August 29, 2017 - September 5, 2017 - Mentors Submit Final Evaluations
• September 6, 2017 - Results Announced

Overview

The main aim of this project is to provide a wrapper of Matlab MEX-Files headers on the current API Scilab. By using the MEX-Files wrapper the toolboxes in Matlab will be very easy to port to Scilab and also existing libraries using Matlab MEX could be directly used in Scilab. A module - mexlib module already exists in Scilab which is compatible with the Matlab-Mex API. Extending the current module and writing tests for each extension will be the major part of the project. Adding tests and a working demo for the already existing MEX functions will be included.

Community Bonding

The community bonding period was from May 5 - 30, 2017.

(1.) Findings and updates from the community bonding period -

• There are a few functions in the existing module which do not have tests. These functions are as follows -

  • mxCreateNumericMatrix

  • mxCreateUninitNumericArray

  • mxCreateUninitNumericMatrix

  • mxDestroyArray

  • mxIsNumeric

  • mxIsLogicalScalarTrue - test is already present in mxIsLogicalScalar.

  • mxSet - Function not implemented yet.

  • mxSetProperty - Function not implemented yet.

(2.) There is no proper help/documentation for the usage of mex in Scilab. There is no list of functions which are present in the "mexlib" module in Scilab. So it is very difficult for users to use mex files without any proper help pages. The bugs related to this are as follows -

• Bug #4286

• Bug #5523

• Bug #8113

(3.) There are many functions which are still not present in the "mexlib" module in Scilab, that are there in the MEX API of MATLAB. (These have been mentioned in my proposal but am re-listing them here for reference)

* MEX Library API:
  * mexCallMATLABWithTrap()
  * mexEvalStringWithTrap()
  * mexGet()
  * mexSet()
  * mexPrintf(.) - size limit not defined currently
  * mexSetTrapFlag()
  * mexErrMsgIdAndTxt()
  * mexWarnMsgIdAndTxt()
  * mexIsLocked()
  * mexLock()
  * mexUnlock()
  * mexMakeArrayPersistent()
  * mexMakeMemoryPersistent()

 * Create or Delete Array:
  * mxCreateUninitNumericMatrix()
  * mxCreateUninitNumericArray()
  * mxCreateSparseLogicalMatrix()
  * mxCreateSparse()
  * mxCalloc()
  * mxMalloc()
  * mxRealloc()
  * mxFree()

 * Validate Data:
  * mxIsScalar()
  * mxIsClass() - does not handle class name and class id
  * mxIsSparse()
  * mxIsFromGlobalWS()

 * Convert Data Types
  * mxArrayToUTF8String()

 * Access Data
  * mxSetClassName()
  * mxSetDimensions() - mxIsSparse() is not handled.
  * mxGetProperty()
  * mxSetProperty()
  * mxRemoveField()
  * mxGetNzmax()
  * mxSetNzmax()
  * mxGetIr()
  * mxSetIr()
  * mxGetJc()
  * mxSetJc()

Coding Period Begins May 30, 2017

My mentor suggested that I should select an already existing toolbox for mex to focus my dev on. One can also refer the mex examples which are distributed with MATLAB, so that we can say that the Scilab mex is fully compatible with atleast all the MATLAB mex modules.

Here are some of the modules which I shortlisted, but I feel they are too "big" for including as a working demo with Scilab. (also they are dependent on external libraries)

• https://kyamagu.github.io/mexopencv/, https://github.com/kyamagu/mexopencv

• https://github.com/alecjacobson/gptoolbox

• http://petercorke.com/wordpress/toolboxes/machine-vision-toolbox

• http://bootstrap-software.com/psignifit/toolbox.php

Week 1-2 Report (May 31, 2017 - June 14, 2017)

Findings, updates and achievements from weeks 1 and 2 -

• Tests for the following functions written and commits made -

  • mxCreateNumericMatrix

  • mxDestroyArray

  • mxIsNumeric

• The tests for the following functions will be written at a later stage because -

  • mxIsLogicalScalarTrue - test is already present in mxIsLogicalScalar.

  • mxSet - Function not implemented yet.

  • mxSetProperty - Function not implemented yet

• A good help/documentation for the mexlib module has to be added to the Scilab Help System. Most of the issues/bugs related to mex are due to lack of proper documentation for it's usage. It is given here - Mex Bugs. For that the following has been proposed -

  • A main page - giving a brief explanation of the mex module
  • A link direction to the currently implemented mex functions
  • Examples and so on.
  • Note from my Mentor - " The main point is : avoid duplicate Matlab mex API description but rather provide a good description of what can be done easily. "

Commits made during this period - (Mostly for the adding the tests for the functions)

• CID #1360807

• CID #1098686

• cellfun commit modified.

• Test for mxIsNumeric

• Test for mxCreateNumericMatrix

• Test for mexCall

Week 3-4 Report (May 15, 2017 - June 26, 2017)

Findings, updates and achievements from weeks 3 and 4 -

• The following functions were improperly implemented in the existing (earlier) module. -

  • mxCreateNumericMatrix

  • mxCreateNumericArray

  • The matlab documentation says - "Call mxCreateNumericMatrix to create a 2-D mxArray in which all data elements have the numeric data type specified by classid. After creating the mxArray, mxCreateNumericMatrix initializes all its real data elements to 0. If ComplexFlag equals mxCOMPLEX in C (1 in Fortran), mxCreateNumericMatrix also initializes all its imaginary data elements to 0. mxCreateNumericMatrix allocates dynamic memory to store the created mxArray. When you finish using the mxArray, call mxDestroyArray to destroy it." So the array should be initialized with all data elements to 0. But the earlier implementation just returned an uninitialized array. This was fixed and the tests were modified accordingly.

• The following functions were implemented -

  • mxCreateUninitNumericArray

  • mxCreateUninitNumericMatrix

  • mxIsScalar

  • mxIsSparse

  • mxCreateSparse - still some changes have to be made here.

• Tests for the following functions(which were newly implemented) added -

  • mxCreateNumericArray

  • mxCreateNumericMatrix - for this and the above function the tests were modified due to modification of the implementation.

  • mxCreateUninitNumericArray

  • mxCreateUninitNumericMatrix

  • mxIsScalar

  • mxIsSparse

  • mxCreateSparse

• The implementation for mxCreateSparse is incomplete because still have to figure out how to handle/use the "nzmax" variable. Same is the issue with mxCreateSparseLogicalMatrix function.

Commits made during this period - (Modified functions and new functions implemented along with their tests)

• Fixed mxCreateNumericArray and mxCreateNumericMatrix and modified tests accordingly.

• Added mxCreateUninitNumericArray and mxCreateUninitNumericMatrix functions and corresponding tests.

• Adding mxIsScalar function and tests.

• Added mxCreateSparse Function and tests

• Added mxIsSparse Function and tests

Next plan of action

Will be working on the following in the upcoming weeks -

• Continue work on the functions which have not been implemented.
• Tests for the functions
• Help for the mex module
• DEMO for the mexlib module.

Week 5-6 Report (July 4, 2017 - July 17, 2017)

Week 7-8 Report (July 18, 2017 - July 23, 2017)

Findings, updates and achievements from weeks 3 and 4 -

• Made some commits for some minor bugs and fixes.
• The following functions were implemented -

  • mxArrayToUTF8String

  • mxCreateSparseLogicalMatrix

  • mxFree

  • mxSetDimensions - Added isSparse part of code for mxSetDimensions

• The following functions were reviewed (they were mostly the memory allocation functions) -

  • mxCalloc

  • mxMalloc

  • mxRealloc - still some changes have to be made here.

• Tests for the following functions(which were newly implemented) added -

  • mxArrayToUTF8String

  • mxCreateSparseLogicalMatrix

  • mxFree

  • mxSetDimensions

  • mxCalloc

  • mxMalloc

Note - Carried forward from phase 1 - The implementation for mxCreateSparse is incomplete because still have to figure out how to handle/use the "nzmax" variable. Same is the issue with mxCreateSparseLogicalMatrix function.

Commits made during weeks 5-8 (Modified functions and new functions implemented along with their tests)

• Added isSparse part of code for mxSetDimensions and tests

• Added mxArrayToUTF8String function and tests for the same

• Added mxCreateSparseLogicalMatrix in mexlib module and tests.

• Minor changes and fixes in mexlib module

• Added mxFree function and tests for mxFree

• Review mxCalloc function and add tests

• Review mxMalloc and add relevant tests

final work report ongoing

Week 9-10 Report (July 24, 2017 - August 6, 2017)

Note - *All the functions for which Scilab API is present have been implemented. The list of functions which have not been implemented or are not compatible with Scilab will be listed at the end. (Have to discuss this with my mentors).

Week 11-12 Report (August 7, 2017 - August 21, 2017)

Findings, updates and achievements from weeks 3 and 4 -

• Made some commits for some minor bugs and fixes.
• The following functions were implemented -

  • mxGetIr

  • mxGetJc

  • mxGetNzmax

  • mexErrMsgIdAndTxt

  • mexWarnMsgIdAndtTxt

• The following functions were handled with appropriate commits -

  • mxexMakeArrayPersistent

  • mexMakeMemoryPersistent

  • mxSetProperty

  • mxGetProperty

  • mexGet

  • mexSet

• Added a help page with documentation to the help system of Scilab with examples for the usage of Mex module in Scilab -
• Tests for the following functions(which were newly implemented) added -

  • mxGetIr

  • mxGetJc

  • mxGetNzmax

Note - Carried forward from phase 1 and phase 2 - The implementation for mxCreateSparse is incomplete because still have to figure out how to handle/use the "nzmax" variable. Same is the issue with mxCreateSparseLogicalMatrix function.

Commits made during weeks 9-12 (Modified functions and new functions implemented along with their tests)

• Adding mxGetJc and related tests

• Adding mxGetNzmax function

• Adding mxGetIr function and related test files

• Adding mexlib help

• mexlib: minor cleanup

• Added mexErrMsgIdAndTxt and mexWarnMsgIdAndTxt

• Add dummy functions mexMakeArrayPersistent and mexMakeMemoryPersistent

• Add comments for dummy functions mexGet, mexSet, mxGetProperty, mxSetProperty

Updates on August 25, 2017. Some points to note and comments after discussion with my mentor -

• mexMakeMemoryPersistent is a dummy function as in Scilab6 the memory is not tracked in any way. mexMakeArrayPersistent is the same.
• Regarding mexLock and mexUnlock on "context.cpp" file there is some code but it seems to be highly predef oriented not sure if it is possible then.
• For now mexErrMsgIdAndTxt() call mexErrMsgTxt() and mexWarnMsgIdandTxt() call mexWarnMsgTxt() and ignore the id. (see function definition)
• mexGet, mexSet, mxGetProperty, mxSetProperty - These functions require "objects" which look like Matlab class instances and in Scilab there are no classes.
• The following functions are yet to be implemented or handled-

  • mexCallMatlabWithTrap

  • mexEvalStringWithTrap

  • mexSetTrapFlag

  • mexIsLocked

  • mxIsFromGlobalWS

  • mxSetClassName

  • mxRemoveField

  • mxSetNzMax

  • mxSetIr

  • mxSetJc

Note - There maybe still somethings left to do, which are indicated by the "TODO" comments or other appropriate comments in the mexlib.cpp file.

My Commits

Link

CategoryHomepage