Scilab Non-Regression and unitary tests for Scilab > 5.0

"Non-Regression testing" definition

Non-regression testing on Wikipedia

Non-Regression tests in Scilab

In Scilab, there are some non-regression tests, which are generally created after a bug report (Scilab's bug tracker), and some unitary tests. Each unitary test is specified to a function. Only non-regression tests will be explained here but it is the same creation method for the unitary ones.

Location

Non-regression and unitary tests of Scilab are included into Scilab source tree. They are available in modules/*/tests/nonreg_tests for non-regression and in modules/*/tests/unit_tests for unitary tests. Indeed, at least one module is concerned by the test, so this test must be included in this module directory.

Filenames

  bug_<bug number>.tst

for example :

The .tst file is a Scilab script and must use the dedicated assert functions. Note that for unitary tests, file name will be the function name, that is to say: <function_name>.tst.

Once the test is written, a reference file will be created in the same directory:

bug_<bug number>.dia.ref

Template

// =============================================================================
// Scilab ( http://www.scilab.org/ ) - This file is part of Scilab
// Copyright (C) Date - Name
//
//  This file is distributed under the same license as the Scilab package.
// =============================================================================
//
// <-- Non-regression test for bug <bug_number> -->
//
// <-- Bugzilla URL -->
// http://bugzilla.scilab.org/show_bug.cgi?id=<bug_number>
//
// <-- Short Description -->
// <bug_description>

// Declaration of some testing variables
assert_checktrue(TEST);

Creation of the reference file

When launching the test, Scilab will test if the current result matches the expected one (the reference). To create a reference file, call the function test_run the following way:

test_run("<module name>","bug_XXX","create_ref");

Note that if the test have some particularities as to use graphics or to be an interactive test, it is necessary to add some options in the test header (see test_run).

Examples

Non Regression for bug 13170 (cacsd module)

// =============================================================================
// Scilab ( http://www.scilab.org/ ) - This file is part of Scilab
// Copyright (C) 2014 - Scilab Enterprises - Paul Bignier
//
//  This file is distributed under the same license as the Scilab package.
// =============================================================================
//
// <-- Non-regression test for bug 13170 -->
//
// <-- Bugzilla URL -->
// http://bugzilla.scilab.org/show_bug.cgi?id=13170
//
// <-- Short Description -->
// plzr plotted wrong legends.

s = %s;
G1 = syslin("c", 1,   s^3+s^2+5*s+2);
G2 = syslin("c", s+3, s^3+s^2+5*s+2);

plzr(G1);
a = gce();
assert_checkequal(a.text, _("Poles"));

plzr(G2);
b = gce();
assert_checkequal(b.text, [_("Poles"); _("Zeros")]);

Non Regression for bug 13313 (graphics module)

// =============================================================================
// Scilab ( http://www.scilab.org/ ) - This file is part of Scilab
// Copyright (C) 2014 - Scilab Enterprises - Calixte DENIZET
//
//  This file is distributed under the same license as the Scilab package.
// =============================================================================

// <-- TEST WITH GRAPHIC -->

// <-- Non-regression test for bug 13313 -->
//
// <-- Bugzilla URL -->
// http://bugzilla.scilab.org/13313
//
// <-- Short Description -->
// Set orientation did not disabled auto_orientation

clf
plot(1:100)
e = gce();
c = e.children;
d = datatipCreate(c, [73 73]);

assert_checkequal(d.auto_orientation, "on");
assert_checkequal(d.orientation, 1);
d.orientation = 2;
assert_checkequal(d.orientation, 2);
assert_checkequal(d.auto_orientation, "off");

public: nonRegressionTests (last edited 2015-05-30 09:51:46 by atoms.scilab.org@farialima.net)