Creating an ATOMS toolbox

This page describes the usual method to create, package, and distribute a Scilab external module (ATOMS Toolbox).

In the first part, we show how to structure the module by creating directories, sub-directories and files at the right place. Then we customize the templates to create the builder(s), the loader(s), and the Scilab help files. In the last section, we present how to upload the module on the Scilab website.

The examples in this document are provided in the module skeleton provided with Scilab. This module is a template of module that is located in SCI/contrib/toolbox_skeleton (or on the website). To get this example on Windows, you must install "A toolbox skeleton (to extend Scilab)" component when we install Scilab.

You can see also these other pages:

General concepts

Toolboxes have their own terminology for a few concepts; here are the words that you will use used in the documentation, and the convention for the toolbox structure and files.

Concepts

Script

A Scilab script is a text file having the extension .sce (see example), used to store Scilab code, and which can be executed using the exec function.

Macro

A Scilab macro is a Scilab function written in Scilab code and stored in a file having the extension .sci (see macro).

Primitive

A Scilab primitive is a Scilab function whose implementation is done in a native function, in C, C++, even Fortran, or any other language compiled natively. A native function is called through a Scilab gateway.

Gateway

A Scilab gateway is a function, written in C or C++, that wraps a native function so that it can be called from Scilab as a primitive (by extension, the term gateway is sometimes used for all gateways provided in a toolbox). The native function can either be a symbol defined in an external library, or a function whose source is provided and compiled into a library at the beginning of the toolbox's compilation. A gateway is responsible for converting the data transmitted to and from the native function, and calling that native function. Once a gateway is declared in the builder script, built and loaded, the primitives are accessible directly from Scilab.

See also: How to create a gateway.

Builder

A builder file is a Scilab script used to build the module i.e. to create a binary version which Scilab can load from a source version. This file is named builder*.sce (Examples: builder.sce, buildhelp.sce, ...).

Loader

A loader is a Scilab script which loads a toolbox component (or the whole toolbox) in Scilab. The loader names have loader*.sce (Examples: loader.sce, loadhelp.sce, ...). This file is usually generated by the corresponding builder file; as a toolbox developer, you don't need to create one.

Toolbox start & quit

Every installed toolbox is started at Scilab startup, and quit when Scilab is quit. When a toolbox is started (respectively quit), a toolbox initialization (respectively finalization) script is executed.

Toolbox structure

By convention, toolboxes have the following structure. They could have others, but all toolboxes hosted on atoms.scilab.org MUST follow it.

The toolbox skeleton, can give you an idea of the structure of a toolbox; all files are not needed, though.

By convention, the root directory has the generic name of the module (here toolbox_skeleton). Its sub-directories should be:

Directory

Content

Required or optional ?

macros

macros (.sci), macros builder (buildmacros.sce)

required (your toolbox would be useless without any macros...)

src

source code files (.c, .cpp, .f,...), and source builder & loaders split (loader*.sce), in several sub-directories for each language (c, cpp, fortran)

only if the toolbox has such source code

sci_gateway

source code files of gateways (.c, ...) and gateway builder & loader (.sce)

only if the toolbox has src source code

jar

java packages, and help files (.jar)

not in the toolbox source: generated by the build

help

XML help files (.xml), split in several sub-directories for each language (en_US, fr_FR,...)

strongly suggested

etc

initialization (.start) and finalization (.quit) scripts

optional in 6.0 (required in 5.5)

tests

module test scripts (.tst)

strongly suggested

demos

some script examples to illustrate your module (.sce)

optional

includes

header files that you want to publish with your module (.h)

optional

and it should have the following files:

File

Description

Required or optional ?

readme.txt

installation guide, etc.

optional

DESCRIPTION

description of the toolbox; see the sample in toolbox skeleton for its syntax

Only needed when installing locally; replaced by the information present on http://atoms.scilab.org when publishing

builder.sce

the main builder file

currently required (may be optional in the final 6.0 release)

loader.sce

the main loader file

no need to create it, it generated at build time by builder.sce

license.txt

license file

optional (but license information should be present in the DESCRIPTION, or on atoms.scilab.org)

Build and load a module

Two scripts are required to create a module: a builder, that will be used to build (package) the toolbox; and a loader, that will be run when the toolbox is loaded in Scilab. Bother are Scilab scripts (sce files) and must be respectively "builder.sce" and "loader.sce"

The main builder is used to to build the module:

exec builder.sce

Once built, the module can be loaded in Scilab with the main loader script:

exec loader.sce

Main builder

The main builder script is the builder.sce file in the root directory of the toolbox. It relies on several sub-builders, each of one is responsible of build a component of the module.

Following is the code of a generic builder, which executes sub builders:

mode(-1);
lines(0);
try
 getversion('scilab');
catch
 error(gettext('Scilab 5.0 or more is required.'));
end;
// ====================================================================
if ~with_module('development_tools') then
  error(msprintf(gettext('%s module not installed.'),'development_tools'));
end
// ====================================================================
TOOLBOX_NAME = 'toolbox_skeleton';
TOOLBOX_TITLE = 'Toolbox Skeleton';
// ====================================================================
toolbox_dir = get_absolute_file_path('builder.sce');

tbx_builder_macros(toolbox_dir);
tbx_builder_src(toolbox_dir);
tbx_builder_gateway(toolbox_dir);
tbx_builder_help(toolbox_dir);
tbx_build_loader(TOOLBOX_NAME, toolbox_dir);

clear toolbox_dir TOOLBOX_NAME TOOLBOX_TITLE;
// ====================================================================

Start and quit script

Each toolbox has an initialization script (also known as Start script) and a finalization script (also known as Quit script). These scripts are in the etc directory, are named with the toolbox name, and have .start and .quit extension. For example the start and quit script for toolbox_skeleton are toolbox_skeleton.start and toolbox_skeleton.quit.

These script are automatically called by Scilab, when the related toolbox are started or quit.

The start script is responsible of loading the toolbox. For this it relies on several sub-loaders, each one of them loads a specific component of the toolbox.

For example of toolbox_skeleton, the start script toolbox_skeleton.start calls the gateways and the Java package loaders, like this:

...
exec(path_convert(root_tlbx + "/sci_gateway/loader_gateway.sce");
...
exec(path_convert(root_tlbx + "/src/java/loader.sce");

Main loader

The main loader script is the loader.sce file in the root directory of the toolbox. It is used to manually load the toolbox. As the loading is already implemented in the start script, the main loader script can call that start script to do the job.

For example of toolbox_skeleton, the main loader script code is:

exec(get_absolute_file_path("loader.sce")+"etc\"+"toolbox_skeleton.start");  

Macros

By convention the builder included in the macros directory is named buildmacros.sce.

In this example, we consider that the macros directory of the toolbox_skeleton module contains just only one macro, the function scilab_sum (see above script). This function returns s the sum of A + B (A and B are the inputs of the function).

Example of Scilab macro: scilab_sum.sci

function s = scilab_sum(A, B)
  s = A + B;
endfunction

The macros builder

The builder (see below script) creates a library variable from macros stored in the directory macros, and saves it in the file calledlib. The builder file contents is generic, it is executed in two steps:

The library generated is called: toolbox_skeletonlib i.e. yourtoolboxnamelib (yourtoolboxname must be 21-characters long max).

Contents of buildmacros.sce file:

tbx_build_macros(TOOLBOX_NAME, get_absolute_file_path('buildmacros.sce'));

clear tbx_build_macros;

The macros loader

The loader loads the library generated by the builder in the directory macros.

Primitives

Source code of a primitive native function is stored in the src directory, in the sub-directory corresponding to the language in which it is implemented ("c", "cpp", "fortran", ...). For each Scilab primitive a gateway function must be created, which the source code is stored in the sci_gateway directory. By convention gateway names begin with "sci_"

C Primitives

We consider a C function csum which returns the sum of two scalars. The name of the corresponding primitive in Scilab is c_sum and the associated gateway name is sci_csum. The both following scripts represent the C code of csum and sci_csum.

The primitive c_sum can be called in Scilab as follows:

--> Y = c_sum(A,B)

Source code of csum.c:

int csum(double *a, double *b, double *c)
{
        *c = *a + *b;
        return 0;
}

Source code of sci_csum.c:

#include "api_scilab.h"

/* ==================================================================== */
extern int csum(double *a,double *b,double *c);
/* ==================================================================== */
int sci_csum(char *fname)
{
    /* error management*/
    SciErr sciErr;
    int iRet = 0;

    /* Variables for first input argument*/
    int* piAddr1 = NULL;
    double dblVal1 = 0;

    /* Variables for second input argument*/
    int* piAddr2 = NULL;
    double dblVal2 = 0;

    /* Variable for return value */
    double dblRetVal = 0;

    /* Check that we have only 2 input parameters */
    CheckInputArgument(pvApiCtx, 2, 2);

    /* Check that we have only 1 output parameter */
    CheckOutputArgument(pvApiCtx, 1, 1);

    /* get first parameter and put in 'dblVal1' */
    sciErr = getVarAddressFromPosition(pvApiCtx, 1, &piAddr1);
    if(sciErr.iErr)
    {
        printError(&sciErr, 0);
        /* No return variable */ 
        AssignOutputVariable(pvApiCtx, 1) = 0;
        return 0;
    }
    
    iRet = getScalarDouble(pvApiCtx, piAddr1, &dblVal1);
    if(iRet)
    {
        /* No return variable */ 
        AssignOutputVariable(pvApiCtx, 1) = 0;
        return 0;
    }

    /* get second parameter and put in 'dblVal2' */
    sciErr = getVarAddressFromPosition(pvApiCtx, 2, &piAddr2);
    if(sciErr.iErr)
    {
        printError(&sciErr, 0);
        /* No return variable */ 
        AssignOutputVariable(pvApiCtx, 1) = 0;
        return 1;
    }
    
    iRet = getScalarDouble(pvApiCtx, piAddr2, &dblVal2);
    if(iRet)
    {
        /* No return variable */ 
        AssignOutputVariable(pvApiCtx, 1) = 0;
        return 1;
    }

    /* call csum subroutine */
    csum(&dblVal1, &dblVal2, &dblRetVal);

    /* create a variable on scilab's memory */
    iRet = createScalarDouble(pvApiCtx, nbInputArgument(pvApiCtx) + 1, dblRetVal);
    if(iRet)
    {
        /* If error, no return variable */ 
        AssignOutputVariable(pvApiCtx, 1) = 0;
        return 1;
    }

    /* assign new variable to return value*/
    AssignOutputVariable(pvApiCtx, 1) = nbInputArgument(pvApiCtx) + 1;
    return 0;
}

Fortran primitives

We consider an fortran subroutine fsum which returns the sum of two scalars. The name of the corresponding primitive in Scilab is fortran_sum and the associated interface program name is sci_fsum. The both following scripts represent the fortran code of fsum and sci_fsum.

The primitive fortran_sum in can be called in Scilab as follows:

--> Y = fortran_sum(A,B)

Source code of fsum.f:

c =================================
      subroutine fsum(a,b,c)
c =================================
      double precision a,b,c
                        c = a + b
      end
c =================================

Source code of sci_fsum.c (we call a fortran subroutine from a C gateway):

#include "stack-c.h"
/* ==================================================================== */
extern int F2C(fsum)(double *a,double *b,double *c);
/* ==================================================================== */
int sci_fsum(char *fname)
{
    /* error management*/
    SciErr sciErr;
    int iRet = 0;

    /* Variables for first input argument*/
    int* piAddr1 = NULL;
    double dblVal1 = 0;

    /* Variables for second input argument*/
    int* piAddr2 = NULL;
    double dblVal2 = 0;

    /* Variable for return value */
    double dblRetVal = 0;

    /* Check that we have only 2 input parameters */
    CheckInputArgument(pvApiCtx, 2, 2);

    /* Check that we have only 1 output parameter */
    CheckOutputArgument(pvApiCtx, 1, 1);

    /* get first parameter and put in 'dblVal1' */
    sciErr = getVarAddressFromPosition(pvApiCtx, 1, &piAddr1);
    if(sciErr.iErr)
    {
        printError(&sciErr, 0);
        /* No return variable */ 
        AssignOutputVariable(pvApiCtx, 1) = 0;
        return 0;
    }
    
    iRet = getScalarDouble(pvApiCtx, piAddr1, &dblVal1);
    if(iRet)
    {
        /* No return variable */ 
        AssignOutputVariable(pvApiCtx, 1) = 0;
        return 0;
    }

    /* get second parameter and put in 'dblVal2' */
    sciErr = getVarAddressFromPosition(pvApiCtx, 2, &piAddr2);
    if(sciErr.iErr)
    {
        printError(&sciErr, 0);
        /* No return variable */ 
        AssignOutputVariable(pvApiCtx, 1) = 0;
        return 1;
    }
    
    iRet = getScalarDouble(pvApiCtx, piAddr2, &dblVal2);
    if(iRet)
    {
        /* No return variable */ 
        AssignOutputVariable(pvApiCtx, 1) = 0;
        return 1;
    }

    /* call fortran fsum subroutine */
    F2C(fsum)(&dblVal1, &dblVal2, &dblRetVal);

    /* create a variable on scilab's memory */
    iRet = createScalarDouble(pvApiCtx, nbInputArgument(pvApiCtx) + 1, dblRetVal);
    if(iRet)
    {
        /* If error, no return variable */ 
        AssignOutputVariable(pvApiCtx, 1) = 0;
        return 1;
    }

    /* assign new variable to return value*/
    AssignOutputVariable(pvApiCtx, 1) = nbInputArgument(pvApiCtx) + 1;
    return 0;
}

Primitives builder

Now the src and the sci_gateway directories contain all necessary files to create the builder (see template below) for the primitive c_sum.

We need to write two builders:

Contents of builder_c.sce file:

tbx_build_src(['csum','csub'], ['csum.o','csub.o'], 'c', ..
              get_absolute_file_path('builder_c.sce'));

clear tbx_build_src;

Contents of buildsci_gateway.sce file:

tbx_build_gateway('skeleton_c', ['c_sum','sci_csum';'c_sub','sci_csub'], ['sci_csum.o','sci_csub.o'], ..
                  get_absolute_file_path('builder_gateway_c.sce'), ..
                  ['../../src/c/libcsum']);

clear tbx_build_gateway;

Java packages

As of Scilab version 5.5, toolbox functions can be also implemented in Java, to be precise in Java packages. The mechanism is different than the one used for macros or primitives: to build, load and use Java packages, the Java Scilab API is used (there is no need to write a gateway). But from the toolbox developer's view, the process of building and loading is roughly the same in Java.

Java sources

Let us recall that Java packages are organized in a hierarchical way, each package containing sub-packages. The root package is named with a naming pattern which is usually something like org.company.software.package. Each Java class is located in a directory that respects the package hierarchy.

The Java packages are stored in the src/java directory. For each Java package, the sources are stored in the specific sub-directory tree.

For example let's consider the Java package org.scilab.scilab.toolboxskeleton, containing a Java class Sum. The package source tree in src/java may look like as following:

src/java:  
   tooboxskeleton
      org  
         scilab 
            scilab 
               toolboxskeleton             
                  Sum.java     

The Sum class contains a static function sum which returns the sum of two doubles. The code is following:

package org.scilab.scilab.toolboxskeleton;
public class Sum {
    public static double sum(double a, double b) {
        return a + b;
    }
} 

Java builder

The source builder script builder_src.sce (in src directory) has to be modified to take in account Java sources.

The function used in this script is tbx_builder_src_lang and run the build of build sources for a list of languages. The Java language has to be added to the list of languages, as following:

   langage_src = ["fortran" "c" "java"];
   path_src = get_absolute_file_path("builder_src.sce");
   tbx_builder_src_lang(langage_src, path_src);     

The Java building script is the file builder_java.sce in src/java directory.

The needed command to compile Java sources and to build a JAR file is ilib_build_jar. The script code looks like following:

   package_name = "org.scilab.scilab.toolboxskeleton";
   jar_file_path = fullfile(jar_dir, package_name + ".jar");
   ilib_build_jar(jar_file_path, package_name, fullfile(src_java_dir, "src"));

Where:

Java loader

The generated Java loader script is loader.sce in the src/java directory. This script has to be called from the toolbox start script.

Help

The help files are JAR files stored in the jar directory. They are produced from the help source files, which are XML files, and a builder script builder_help.sce, all stored in the help directory.

Creation of XML files

Here is a template which shows you how to write the XML help files. You should just fill the different items for your functions and put them in the help directory.

See http://wiki.scilab.org/howto/scilab_documentation_kit for more information about documentation.

Example of Scilab help file, c_sum.xml:

<?xml version="1.0" encoding="UTF-8"?>
<refentry version="5.0-subset Scilab" xml:id="c_sum" xml:lang="en"
          xmlns="http://docbook.org/ns/docbook"
          xmlns:xlink="http://www.w3.org/1999/xlink"
          xmlns:svg="http://www.w3.org/2000/svg"
          xmlns:ns3="http://www.w3.org/1999/xhtml"
          xmlns:mml="http://www.w3.org/1998/Math/MathML"
          xmlns:db="http://docbook.org/ns/docbook">
  <info>
    <pubdate>$LastChangedDate: 2008-03-26 09:50:39 +0100 (mer., 26 mars 2008)$</pubdate>
  </info>

  <refnamediv>
    <refname>c_sum</refname>

    <refpurpose>sum from C</refpurpose>
  </refnamediv>

  <refsynopsisdiv>
    <title>Calling Sequence</title>

    <synopsis>a = c_sum(b,c)</synopsis>
  </refsynopsisdiv>

  <refsection>
    <title>Description</title>

    <para>Do a sum.</para>

    <para>Add here a paragraph of the function description </para>
  </refsection>

  <refsection>
    <title>Examples</title>

    <programlisting role="example">c_sum(3,4)</programlisting>
  </refsection>

  <refsection>
    <title>Authors</title>

    <simplelist type="vert">
      <member>YOUR NAME</member>
    </simplelist>
  </refsection>
</refentry>

The help builder

This file creates a file containing all your module documentation so that it can be loaded in Scilab help browser.

Contents of build_help.sce:

help_lang_dir = get_absolute_file_path('build_help.sce');

tbx_build_help(TOOLBOX_TITLE, help_lang_dir);

clear help_lang_dir;

Open the help

To see the help pages about toolbox_skeleton, launch the help browser with:

help

Uploading your toolbox

Once your toolbox can be compiled and loaded, you can upload it to ATOMS:

Scilab Enterprises can then take care of compiling your toolbox for multiple platforms and versions of Scilab. However, in order to be part included in the ATOMS packaging system, your toolbox must respect the format of the toolbox outlined above. See also the Guidelines on submitting a module below for a few tips.

Q&A

How can I have my module included to ATOMS?

See Uploading your toolbox section above.

Once it is done, we should process (generate binary versions) your module within a few days. If this does not happen, feel free to contact us on the mailing lists, or directly via email.

Note : if you are posting your sources, don't forget to set its class as "sources of a valid Scilab package".

When we have generated binaries for the toolbox, you will receive an automated email; and your module will be available for all users through ATOMS.

To see the new package in Scilab, update your package list by running:

atomsSystemUpdate()

For more details, please read atomsSystemUpdate

What are the technical expectations ?

See above for the requirements. As a reminder:

Note that the content of the DESCRIPTION file will NOT be used on the Atoms website: instead, you must enter all the needed information on the Atoms website itself. However, it is a good idea to enter some minimal information about your toolbox in it.

If you work depends on a third party native library, let us know and please provide as much as information as possible. If you don't want your module to be packaged under a certain OS, please let us know after posting it.

What kind of restrictions is there?

There is no restriction on the kind of modules which are accepted into ATOMS as long as they provide a new Scilab feature (whatever it is).

Since we need to build the module, sources must be available. However, there is no restriction on the license of a module (even if we prefer free licenses).

In any case, do not submit binary files, as they are often not working on different OS or configurations.

Will you modify my module?

Sometimes, we have to patch to make sure that the module builds and runs perfectly. We usually send back the patches for upstream incorporation. Those change are usually :

Consider those points before submitting a module, as they are quickly fixed by you and often make us lose a lot of time, which could be used improving ATOMS and fixing its bugs!

I have just a Scilab function/macro and I would like to have it into ATOMS

No problem, just submit it

Preparing for Scilab 6

See Gateways: from Scilab 5 to Scilab 6 for migrating your toolbox to Scilab 6. In particular, stack-c.h is not available anymore in Scilab 6; it should be replaced by api_scilab.h.

Guidelines on submitting a module

public: howto/Create a toolbox (last edited 2016-06-23 12:42:47 by francois.granade@scilab-enterprises.com)