GSoC 2017 - Matlab Mex Files Wrapper
Contents
-
GSoC 2017 - Matlab Mex Files Wrapper
- Project Link
- Student and Mentors
- Timeline for GSoC 2017
- Overview
- Community Bonding
- Coding Period Begins May 30, 2017
- Week 1-2 Report (May 31, 2017 - June 14, 2017)
- Week 3-4 Report (May 15, 2017 - June 26, 2017)
- Next plan of action
- Week 5-6 Report (July 4, 2017 - July 17, 2017)
- Week 7-8 Report (July 18, 2017 - July 23, 2017)
- final work report ongoing
- Week 9-10 Report (July 24, 2017 - August 6, 2017)
- Week 11-12 Report (August 7, 2017 - August 21, 2017)
- My Commits
Project Link
Student and Mentors
Student Name -
- Siddhartha Gairola
Mentors -
- Clément David
- Dhruv Khattar
Timeline for GSoC 2017
This timeline is for after the students projects were announced on 4th of May, 2017.
- May 5 - 30, 2017 - Community Bonding
- May 30, 2017 - June 30, 2017 - Students Work on their Projects.
- June 26 - 30, 2017 - First Evaluations
- July 1 - 24, 2017 - Students Continue Coding
- July 24 - 28, 2017 - Second Evaluations
- July 28, 2017 - August 21, 2017 - Students Continue Coding
- August 21 - 29, 2017 - Students Submit Code and Evaluations
- August 29, 2017 - September 5, 2017 - Mentors Submit Final Evaluations
- September 6, 2017 - Results Announced
Overview
The main aim of this project is to provide a wrapper of Matlab MEX-Files headers on the current API Scilab. By using the MEX-Files wrapper the toolboxes in Matlab will be very easy to port to Scilab and also existing libraries using Matlab MEX could be directly used in Scilab. A module - mexlib module already exists in Scilab which is compatible with the Matlab-Mex API. Extending the current module and writing tests for each extension will be the major part of the project. Adding tests and a working demo for the already existing MEX functions will be included.
Community Bonding
The community bonding period was from May 5 - 30, 2017.
Tasks |
Description |
Status |
||
Discussing with mentors the workflow and get a final idea of how to approach the project. |
Here I was told by my mentors to go through the existing module and the tests. |
Done |
||
Boost my knowledge about the API’s and skills required for the project. |
I went through the Scilab API 6 help wiki and some of the examples to get a better understanding of the API. |
Done |
||
Get acquainted with the code base and exploring the Matlab Mex module examples and documentation to get a complete understanding of the module. |
Re-iteration of going through the code base and the existing module plus going through the mex examples distributed with MATLAB. |
Done |
||
Make final changes to any open bugs/coverity issues. |
Commit made for Bug #6701, Coverity fix : CID #1098686, CID #1360807. |
Pending |
(1.) Findings and updates from the community bonding period -
- There are a few functions in the existing module which do not have tests. These functions are as follows -
mxCreateNumericMatrix
mxCreateUninitNumericArray
mxCreateUninitNumericMatrix
mxDestroyArray
mxIsNumeric
mxIsLogicalScalarTrue - test is already present in mxIsLogicalScalar.
mxSet - Function not implemented yet.
mxSetProperty - Function not implemented yet.
(2.) There is no proper help/documentation for the usage of mex in Scilab. There is no list of functions which are present in the "mexlib" module in Scilab. So it is very difficult for users to use mex files without any proper help pages. The bugs related to this are as follows -
(3.) There are many functions which are still not present in the "mexlib" module in Scilab, that are there in the MEX API of MATLAB. (These have been mentioned in my proposal but am re-listing them here for reference)
* MEX Library API: * mexCallMATLABWithTrap() * mexEvalStringWithTrap() * mexGet() * mexSet() * mexPrintf(.) - size limit not defined currently * mexSetTrapFlag() * mexErrMsgIdAndTxt() * mexWarnMsgIdAndTxt() * mexIsLocked() * mexLock() * mexUnlock() * mexMakeArrayPersistent() * mexMakeMemoryPersistent() * Create or Delete Array: * mxCreateUninitNumericMatrix() * mxCreateUninitNumericArray() * mxCreateSparseLogicalMatrix() * mxCreateSparse() * mxCalloc() * mxMalloc() * mxRealloc() * mxFree() * Validate Data: * mxIsScalar() * mxIsClass() - does not handle class name and class id * mxIsSparse() * mxIsFromGlobalWS() * Convert Data Types * mxArrayToUTF8String() * Access Data * mxSetClassName() * mxSetDimensions() - mxIsSparse() is not handled. * mxGetProperty() * mxSetProperty() * mxRemoveField() * mxGetNzmax() * mxSetNzmax() * mxGetIr() * mxSetIr() * mxGetJc() * mxSetJc()
Coding Period Begins May 30, 2017
My mentor suggested that I should select an already existing toolbox for mex to focus my dev on. One can also refer the mex examples which are distributed with MATLAB, so that we can say that the Scilab mex is fully compatible with atleast all the MATLAB mex modules.
Here are some of the modules which I shortlisted, but I feel they are too "big" for including as a working demo with Scilab. (also they are dependent on external libraries)
https://kyamagu.github.io/mexopencv/, https://github.com/kyamagu/mexopencv
http://petercorke.com/wordpress/toolboxes/machine-vision-toolbox
Week 1-2 Report (May 31, 2017 - June 14, 2017)
Tasks |
Description |
Status |
||
Write tests for the already existing mexlib module. |
Worked on writing tests for the functions which did not have test coverage. (These functions are identified in the above section). |
Done |
||
Finding corner cases and boundary conditions for the functions. |
The tests for the functions have to be written with care. |
Done |
||
Resolve bugs related to the Mex module in the Bug Tracker - MEX Module Bugs Link |
Most of the issues/bugs related to mex are lack of proper documentation or help for the usage. It is given here - Mex Bugs |
Ongoing |
Findings, updates and achievements from weeks 1 and 2 -
- Tests for the following functions written and commits made -
mxCreateNumericMatrix
mxDestroyArray
mxIsNumeric
- The tests for the following functions will be written at a later stage because -
mxIsLogicalScalarTrue - test is already present in mxIsLogicalScalar.
mxSet - Function not implemented yet.
mxSetProperty - Function not implemented yet
A good help/documentation for the mexlib module has to be added to the Scilab Help System. Most of the issues/bugs related to mex are due to lack of proper documentation for it's usage. It is given here - Mex Bugs. For that the following has been proposed -
- A main page - giving a brief explanation of the mex module
- A link direction to the currently implemented mex functions
- Examples and so on.
- Note from my Mentor - " The main point is : avoid duplicate Matlab mex API description but rather provide a good description of what can be done easily. "
Commits made during this period - (Mostly for the adding the tests for the functions)
Week 3-4 Report (May 15, 2017 - June 26, 2017)
Tasks |
Description |
Status |
||
Get everything merged in the master branch and fix any kind of issues arising during implementation or clearing any backlog. |
Made final commits for some of coverity issues and bugs I was working on. (Although some still are pending review and further refactor.) |
Done |
||
Continue writing the tests for the already existing module and finish with all the tests for the mx* and mex* functions. |
All the tests for the existing module were written in the past weeks. Now we could add more tests for solid coverage. (Will add tests for the new functions as they are implemented.) |
Done |
||
Work on the unresolved bugs related to the Mex module in the Bug Tracker - MEX Module Bugs Link |
As mentioned earlier, most of the bugs in the bug tracker are due to lack of proper help. |
Ongoing |
||
Identify the areas which still use the old Scilab API. |
Did identify some regions which use the old API. Like the mxCreateNumeric functions. |
Done |
||
Upgrade the current implementation to use the new API for Scilab 6. |
Some work done. |
Ongoing |
||
Write documentation for the functions and usage of the mexlib module. This will be added to the Scilab help system. |
The layout was identified in the last couple of weeks. It still has to be worked upon. |
Ongoing |
Findings, updates and achievements from weeks 3 and 4 -
- The following functions were improperly implemented in the existing (earlier) module. -
mxCreateNumericMatrix
mxCreateNumericArray
The matlab documentation says - "Call mxCreateNumericMatrix to create a 2-D mxArray in which all data elements have the numeric data type specified by classid. After creating the mxArray, mxCreateNumericMatrix initializes all its real data elements to 0. If ComplexFlag equals mxCOMPLEX in C (1 in Fortran), mxCreateNumericMatrix also initializes all its imaginary data elements to 0. mxCreateNumericMatrix allocates dynamic memory to store the created mxArray. When you finish using the mxArray, call mxDestroyArray to destroy it." So the array should be initialized with all data elements to 0. But the earlier implementation just returned an uninitialized array. This was fixed and the tests were modified accordingly.
- The following functions were implemented -
mxCreateUninitNumericArray
mxCreateUninitNumericMatrix
mxIsScalar
mxIsSparse
mxCreateSparse - still some changes have to be made here.
- Tests for the following functions(which were newly implemented) added -
mxCreateNumericArray
mxCreateNumericMatrix - for this and the above function the tests were modified due to modification of the implementation.
mxCreateUninitNumericArray
mxCreateUninitNumericMatrix
mxIsScalar
mxIsSparse
mxCreateSparse
The implementation for mxCreateSparse is incomplete because still have to figure out how to handle/use the "nzmax" variable. Same is the issue with mxCreateSparseLogicalMatrix function.
Commits made during this period - (Modified functions and new functions implemented along with their tests)
Fixed mxCreateNumericArray and mxCreateNumericMatrix and modified tests accordingly.
Added mxCreateUninitNumericArray and mxCreateUninitNumericMatrix functions and corresponding tests.
Next plan of action
Will be working on the following in the upcoming weeks -
- Continue work on the functions which have not been implemented.
- Tests for the functions
- Help for the mex module
- DEMO for the mexlib module.
Week 5-6 Report (July 4, 2017 - July 17, 2017)
Tasks |
Description |
Status |
||
Implement the functions which are missing from the current module and have been listed in Part 1 of the Project Proposal. |
The functions left have to be implemented. |
Done (some left) |
||
Write unit tests for each function which has been implemented. |
The tests were written for the functions which were implemented. |
Done |
Week 7-8 Report (July 18, 2017 - July 23, 2017)
Tasks |
Description |
Status |
||
Get everything merged in the master branch and fix any kind of issues arising during implementation or clearing any backlog. |
The backlog was reviewed and merged. |
Done |
||
Continue with the implementation of the missing functions. |
The functions left have to be implemented. |
Done |
||
Write unit tests for each function which has been implemented. |
The tests were written for the functions which were implemented. |
Done |
||
Get the code ready for the Phase 2 evaluation and fix any kind of issues which might arise. |
Fix any errors and get your code merged. |
Ongoing |
Findings, updates and achievements from weeks 3 and 4 -
- Made some commits for some minor bugs and fixes.
- The following functions were implemented -
mxArrayToUTF8String
mxCreateSparseLogicalMatrix
mxFree
mxSetDimensions - Added isSparse part of code for mxSetDimensions
- The following functions were reviewed (they were mostly the memory allocation functions) -
mxCalloc
mxMalloc
mxRealloc - still some changes have to be made here.
- Tests for the following functions(which were newly implemented) added -
mxArrayToUTF8String
mxCreateSparseLogicalMatrix
mxFree
mxSetDimensions
mxCalloc
mxMalloc
Note - Carried forward from phase 1 - The implementation for mxCreateSparse is incomplete because still have to figure out how to handle/use the "nzmax" variable. Same is the issue with mxCreateSparseLogicalMatrix function.
Commits made during weeks 5-8 (Modified functions and new functions implemented along with their tests)
final work report ongoing
Week 9-10 Report (July 24, 2017 - August 6, 2017)
Tasks |
Description |
Status |
||
Get the code ready for the Phase 2 evaluation and fix any kind of issues which might arise. |
If everything works fine then jump to the next task. |
Done |
||
Get finished with the implementation of all the missing functions. |
The remaining functions which were left (for which Scilab API was present) were implemented. |
Done* |
||
Write unit tests for each function which has been implemented. |
The tests were written for the functions which were implemented. |
Done |
Note - *All the functions for which Scilab API is present have been implemented. The list of functions which have not been implemented or are not compatible with Scilab will be listed at the end. (Have to discuss this with my mentors).
Week 11-12 Report (August 7, 2017 - August 21, 2017)
Tasks |
Description |
Status |
||
Write documentation and demonstrate with the help of examples the use of the MEX API for existing MEX based libraries. Explaining in detail how each function works. |
Add pages to the Scilab help system for user reference |
Done |
||
Write a working demo for the Mex module for Scilab. |
This would be a working example for reference. |
Pending |
Findings, updates and achievements from weeks 3 and 4 -
- Made some commits for some minor bugs and fixes.
- The following functions were implemented -
mxGetIr
mxGetJc
mxGetNzmax
mexErrMsgIdAndTxt
mexWarnMsgIdAndtTxt
- The following functions were handled with appropriate commits -
mxexMakeArrayPersistent
mexMakeMemoryPersistent
mxSetProperty
mxGetProperty
mexGet
mexSet
- Added a help page with documentation to the help system of Scilab with examples for the usage of Mex module in Scilab -
- Tests for the following functions(which were newly implemented) added -
mxGetIr
mxGetJc
mxGetNzmax
Note - Carried forward from phase 1 and phase 2 - The implementation for mxCreateSparse is incomplete because still have to figure out how to handle/use the "nzmax" variable. Same is the issue with mxCreateSparseLogicalMatrix function.
Commits made during weeks 9-12 (Modified functions and new functions implemented along with their tests)
Add dummy functions mexMakeArrayPersistent and mexMakeMemoryPersistent
Add comments for dummy functions mexGet, mexSet, mxGetProperty, mxSetProperty
Updates on August 25, 2017. Some points to note and comments after discussion with my mentor -
- mexMakeMemoryPersistent is a dummy function as in Scilab6 the memory is not tracked in any way. mexMakeArrayPersistent is the same.
- Regarding mexLock and mexUnlock on "context.cpp" file there is some code but it seems to be highly predef oriented not sure if it is possible then.
- For now mexErrMsgIdAndTxt() call mexErrMsgTxt() and mexWarnMsgIdandTxt() call mexWarnMsgTxt() and ignore the id. (see function definition)
- mexGet, mexSet, mxGetProperty, mxSetProperty - These functions require "objects" which look like Matlab class instances and in Scilab there are no classes.
- The following functions are yet to be implemented or handled-
mexCallMatlabWithTrap
mexEvalStringWithTrap
mexSetTrapFlag
mexIsLocked
mxIsFromGlobalWS
mxSetClassName
mxRemoveField
mxSetNzMax
mxSetIr
mxSetJc
Note - There maybe still somethings left to do, which are indicated by the "TODO" comments or other appropriate comments in the mexlib.cpp file.