The aim of this project is to add Scilab interfaces to the software Dakota.
See attached SEP: SEP_ScilabDakotaInterface.pdf
Write the code for the script interface
Write the code for the compiled interface
Write the code for the linked interface
Make a toolbox Dakota for Scilab
Make examples of use
Adapt for CMake
June 6th 2011
During these weeks, I started the development by the implementation of the script interface. This interface is simpler than the others. It uses a shell script to manage the files used for the Dakota/Scilab communication. Next, it calls Scilab, which launchs the Scilab script. This script takes care to read the input file from Dakota, and launchs the required calculation as the objective function calculation. This function is another Scilab script and must be adapt according with the different examples. After the calculations, it writes the result in the output file for Dakota. When the shell script finishes an iteration, it deletes the files.
I had no big problems working on this interface implementation. The script is developped in shell langage. Nevertheless, I found more difficult to write the Scilab script code, including to read file and write on it. Some of Scilab functions are new for me, so I needed few times to adapt to the langage specifications.
The next step is to implement some examples of use, and test the interface. Then, it will be time to start the compiled interface. I am already thinking on it.
June 17th 2011
During the past two weeks, I finished the implementation of the script interface including two examples : Rosenbrock and PID. I started to implement the compiled interface.
- As noted in the introduction, two examples have been developped. First, the rosenbrock example is based, like its name suggests, on the Rosenbrock function. This is a typical function, used in mathematical optimization, to test optimization algorithms.
This function is defined in scilab by:
function f = rosenbrock(x,alpha)
f = alpha*( x(2) - x(1)2 )2 + ( 1 - x(1) )^2;
- The second example is a PID controller. It is a widely used mechanical regulator. P,I and D means proportional (error), integral (past error) and derivative (future error). By varying these three parameters, it is possible to minimize the calculated error to reach to desired value for the system.
This objective function is implemented in scilab by :
function y = f(x)
- context.w0 = w0; context.m = m; context.K0 = K0; context.P = x(1); context.I = x(2); Info = scicos_simulate(scs_m,list(),context,flag='nw'); y_error = mean(abs((block_output('values')(:,1) - block_output('values')(:,2)))); y_diff = mean(abs(diff(block_output('values')(:,2)))); y = 0.5*y_error + 0.5*1*y_diff; ...
This exemple was more complicated to implement than Rosenbrock example because it uses xcos, so I had teach how use this Scilab editor.
A script to vizualize the solution was implemented too. Below, we can see the graph of the solution:
After that, I started to implement a new interface. This one is faster than the script interface because Scilab is launched once, thanks to API call_scilab used by a server. This API contains three particularly interesting features in our case : StartScilab, SendScilabJob, TerminateScilab. This functions starts (StartScilab) and runs (SendScilabJob) all Scilab instructions, before close Scilab (TerminateScilab).
Below are some extracts from the code of the interface which enhances how use the different fonctions.
ep = StartScilab(NULL,NULL,NULL);
ep = StartScilab(getenv("SCI"),NULL,NULL);
ts = TerminateScilab(NULL);
There was no problem in utilization of this functions. Indeed, the API call_scilab is well documented. Nevertheless, I have a few to implement the server and the communication via named pipes. Set up this server is the next step. Moreover, It is expected to make another example of use using the Scilab module called Femtruss. Finally, I must adapt all the examples for the compiled interface.
July 1st 2011
In accordance with the schedule, I started the last two weeks to finish the compiled interface by setting up the server. After, I began to implement the linked interface, based on Dakota source code (DirectApplicationInterface.C).
Server of the compiled interface
The compiled interface used fork function to launch the server. This server is used to read the instructions from Dakota (transmitted through a file), and to launch the required computations via Scilab. Next, the server write the results of computations in the intended file. This communication between Dakota and the server is through the named pipe. The pre-defined files (the names of those files are precised in the Dakota configuration file) are transmitted to the server via the named pipes. The computation are launched according the input Dakota file via a implemented function. This function read all informations from Dakota, and use the Api call_scilab to send the computation in Scilab.
This interface is the fastest. It doesn't use the file communication. It uses a Dakota structure which contains all informations which are necessary to launch computation, and so to solve optimization problems. This structure contains reserved fields to stock the result of computation at each iteration. Indeed, Dakota structure is copied in Scilab via a mlist. To create it, I used the Api list. So this mlist is updated at each iteration and the communication between Dakota et Scilab is instantaneous. This communication means too that I will use specific functions to read/write in the mlist (the Dakota structure finally).
- Mlist creation :
_SciErr = createNamedMList(pvApiCtx, "Dakota", SCI_NUMBER_OF_FIELDS, &piMListAddr);
The next two weeks, I hope that I will finish the linked interface to focus my work on an added projet part : made a Scilab toolbox which permit to use Dakota optimization skills. This toolbox would be use another specific interface to call Dakota from Scilab.
July 14th 2011
The last two weeks, my work was to develop the linked interface and the toolbox Dakota for Scilab wisely called Scidakota.
In Scilab, the matrices are used in many cases. In this project, they are used to complete the Scilab mlist with Dakota informations. After, they are used to read the result of Scilab computation and to send the result to Dakota. Thereafter, Dakota can evaluate if another iteration is necessary.
* Matrix of Double creation
_SciErr = createMatrixOfDoubleInNamedList(pvApiCtx,"Dakota",piMListAddr,2,1,1,&dbl_tmp);
In this case, it is the second position of the mlist which is complete with the double dbl_tmp. To create a matrix of String or another, it is significantly the same.
* Read a matrix of double
_SciErr = readMatrixOfDoubleInNamedList(pvApiCtx,"Dakota_out", piMListAddr, 22, &piRows, &piCols, NULL);
- x_tmp = (double*) malloc (sizeof(double) * piRows * piCols);
_SciErr = readMatrixOfDoubleInNamedList(pvApiCtx, "Dakota_out", piMListAddr, 22, &piRows, &piCols, x_tmp);
- x_tmp = (double*) malloc (sizeof(double) * piRows * piCols);
printError(&_SciErr, 0); return (SCILAB_FAIL);
To read a matrix, it is a bit more complicated. Indeed, it is necessary to call the function readMatrixOfDoubleInNamedList twice. The first call permits to allocate the memory. The second call is used to store the double, in x_tmp in this case. It is almost the same with other types of matrices.
Now, this interface runs. Justly, it is used in the toolbox Dakota.
The Scilab toolbox expands the Gsoc project. It is very interessant for Scilab because it will permit to use Dakota optimization skills in Scilab without the Dakota software.
To implement this toolbox, it is necessary to use the linked interface. It runs the Scilab computation. Nevertheless, we must retrieve the result. After, the user can access to the result directly in Scilab and exploits them for another computations for example. To do it, I will use the specific Dakota functions to complete the result mlist.
To conclude this new weekly report, I will give the plan for the next two weeks. First of all, I will continue to develop the toolbox, and after I will start the Documentations for both all interfaces and toolbox Scidakota.
July 29th 2011
Here is one of the last report for this project. So to start I have made a small review of the project advancement. Next I have detailled the job I have done over the past two weeks, namely on the Scilab toolbox Scidakota.
I start the GSOC project by planning to implement 3 interfaces which link Dakota and Scilab. Thanks to them, Dakota can use Scilab to perform computations, and to solve optimization problems.
Firstly, I realized the script interface. This interface is based on a shell script used by Dakota like driver. After, the script launch Scilab computation. This interface was the simplest to implement, so it was a good one to get familiar me with the project. Nevertheless, It is a slow interface, due to all the Scilab calls. Morover, it uses a file communication and it is improvable.
Secondly, it is the compiled interface I have implemented. This one also uses a file communication, but Scilab is called only once. Indeed, the driver uses by Dakota is not a shell script here, but it is a C program which runs a server. Then it is charge to launch Scilab and to link Dakota and Scilab via file communication. Here, the compiled interface spend less time than the script interface.
Finally, I have implemented the linked interface. This interface is useful because it uses a Dakota data structure to link Dakota and Scilab. So this interface has the great advantage to run fast because the communication between Dakota and Scilab is instantaneous.
Of course, to use the different interface, it is necessary to implement some specific file to configure Dakota. This is in this file that the used interface is precize. To have some details about the syntax of configuration file, we can refer to the documentation of Dakota software.
To complete, the work on those differents interface, I adapt cmake file to enable compiled with cmake. Indeed, cmake is a open source build system. The advantage is that cmake is independent to compile. It doesn't depend on other programs or modules. So it was interessant to adapt the Dakota cmake file to be able to compile the Scilab interfaces with cmake. The compilation process is simpler then. This work consisted of add HAVE_SCILAB option in the CMakeLists.txt and add FindScilab.cmake file to configure Scilab modules. Now it remains to write the documentation files including latex files.
Like I said in the last report, the initial GSOC project was expanded. So the development of a Scilab toolbox was added. The advantage of this toolbox is to offer a portable solution for the Scilab users who are insteresting by the Dakota skills.
To run, this toolbox use the linked interface without the Scilab call to solve optimization problems. Indeed it was necessary to delete the Scilab call in the used interface because Scilab already runs when the interface is used by the toolbox. So if the interface try to call Scilab while Scilab already runs, it creates a bug. After this, the work consisted of create a mlist which host the result of the optimization. This mlist will return at the end, and the user can collect all the informations on the past optimization.
Here a quick example to get the function number :
printError(&_SciErr,0); return (SCILAB_FAIL);
Of course, some others functions have allowed to complete all the result mlist. All this functions are sorted in the developer Dakota documentation. Currently, there is just one doubt about the good performances of the toolbox, it is on the recover of the hessian matrix. To verify, I will try to develop a specific example. In the next two weeks, I will write the toolbox documentation.
To finish this report, I think I am in time to finish this project. The different interface and the toolbox runs. The final step is to write the documentation including a latex chapter for the interfaces and a xml file for the toolbox. For the versions of code, there is the link for the latest version of the Scidakota in bottom of this page (forge Scidakota). For the interfaces codes, I joined patches on Scidakota forge.
August 12th 2011
This was the last two weeks to work. To finish this program, I wrote differents documentations types including latex documentation. I try also to develop an example for the toolbox to test the hessian function implementation.
I am started my work in updated the xml file for the toolbox. This file describes the toolbox use, the parameters and an example of use. In paralell, I wrote a file for each interfaces in the Dakota code. This "readme" files precised the interface uses and presented several examples of use. Finally, I wrote a latex chapter to complete the Dakota latex documentation. This file contains approximately the same content that the readme files.
So, this program is finished but I have some little work to finish. For example, my toolbox hessian example don't run. I must have add my latex chapter to the Dakota documentation too. Otherwise, I finished all my works, so I think I have half a week to finish totally my Gsoc project. Nevertheless, I will contact my mentor to beneficit of some advices.
August 20th 2011
Here is the last report before the deadline. The report give some informations about documentation.
Full documentation from user perspective
I wrote a user documentation for Dakota interfaces and for Scidakota. For the Dakota interfaces, I wrote a latex chapter for the Dakota users. This chapter is called : Scilab and Dakota, section : Scilab interfaces. To build this documentation, we must compile Dakota with cmake and turn on the option called : ENABLE_DAKOTA_DOCS.
For the toolbox there is a user documentation too. It is a xml doc. So the Scidakota documentation can be include in the Scilab help system. To build it, we can use the builder to compile the toolbox and the doc with this commande : exec builder.sce (run this commande in Scilab). After, to use the toolbox, we can do the same with the loader : exec loader.sce.
For both interfaces and Scidakota, I wrote some readme too which don't need compilation. Those readme give informations like the presentation of the toolbox, interfaces or use details ...
The toolbox docs and code are available on the scilab forge. I will update the interfaces patch on the forge to report the last modifications on the interfaces, including documentations like the latex chapter. All interfaces and Scidakota runs on Scilab 5.2 and higher versions.
I precized in all readme : those first version is : 1.0 version. I haven't made a tarball but all codes are available on the forge (see links part). Nevertheless, I can do it if it's necessary.
There is no bugs to report now. Just an interrogation about the hessian result with the toolbox. I tried different implementations, but I'm not sure I found the right one. Nevertheless, all functions runs without bugs. It is the same for interfaces : no problems. To complete toolbox stuff, we can develop another examples.
Otherwise, all my work runs on Linux, and I try to port the most of my work on Windows. So I wrote a .bat file for the script interface and the linked interface may runs on Windows. So Scidakota may runs on Windows too. Unhappily the compiled interface use fork function. Currently, It runs just on Linux.
For the demos and examples, there are a lot examples which were implemented. For the interfaces, we can find all the examples in the specific directory in the Dakota trunk. We can find Rosenbrock example, PID example, FemTRUSS example. For the toolbox, there is only one demo now. We can find it in the demo directory of the toolbox. It is the Rosenbrock example. This example is appropriate to test the toolbox and interfaces performances in optimization.
I appreciated work on the toolbox and the interfaces, so I am ready to maintain my work in the future.
GSOC and Me
First of all, I want to thank particularly my mentor Yann Collette who was there to answer to all my questions. He was a great mentor, reactive and patient. I also thank the Scilab team that I met in the ScilabTec. I really appreciate the work atmosphere, and it makes me motivate to continue to contribute to Scilab in the future. The GSOC was a great opportunity for me and during this summer I have learn lot of stuff. The most hard for me was to traduct in computing code the mathematics knowledge, so start to code was really hard because I just learned the new mathematics notions. It is probably that which makes me feel that my subject was hard. Happily, my mentor took time to explain me the new things when I was a problem. When I took the pace, I was launched for the summer.
I liked the Scilab community, and I don't know what can be improve for next year. My mentor was really available and I appreciate write a report all two weeks. It was interessant for me and it permitted me to measure my work progress. Finally, I found that Scilab had a dynamic team and it was great to be in touch with them during GSOC program.