Bi-weekly Report - 29th July 2011
Work done in the past two weeks
The testing phase has been started. Its composed of three phases
- elementary
- accuracy
- performance.
Once fdlibm was integrated with Linux and was giving right results, I checked to see if it was performing well for the available tests in the source tree. I used the basic test_run command to run the tests for elementary_functions()
test_run(“elementary_functions”, [ ], “no_check_error_ouput”)
This resulted in 6 errors. The failed functions are asin, cumsum, cumprod, prod, sum and bug_2052. Except asin which threw the error message “failed:one or several tests”, all the others ended with “failed: premature end of the test script”. This wasn’t the case with v5.3, it had 100% result.
I went through a few function but couldn’t find any link to the fdlibm library except in case of asin which failed here.
A = rand(100,100); if or( (asin(A) - acos(-A) + %pi/2) > %eps) then pause, end
Since, the rest of the functions didn’t connect to fdlibm, I suspected it to be an issue with the v6 branch. When I compiled and ran the same test on the fresh branch, without any involvement of fdlibm, it threw the same errors. As a results of this, I’m not considering the errors a part of fdlibm testing.
Moving on to the inclusion of more accuracy test cases, I referred to the tests provided with the crlibm package. They provide test cases for almost all the functions we need. The trouble in the beginning was the format used in their test data files. They used hex values and I wasn’t aware of any means to convert them into decimal, but with Michael’s help it was easily done.
A few days ago, I sent Michael a rough draft of the script which uses the test data and checks for satisfactory results. I’ve updated it to be a part of the actual script format. I’ll include it in the singular values section(2). Based upon the kind of test cases - worst cases, part of libmcr packages etc. I’ve divided them into further subsections subsections. I’ve some doubts related to the rounding modes mentioned with each test case and the number of test cases to include(some have > 10k). Once Michael clears it up, its just a matter of including the test cases for all the functions, which shouldn’t take more than a day or two.
ToDo
I’m confident that I’ll finish this project with success. The accuracy test cases for all high-priority functions will be added by the beginning of next week. Michael suggested a rewrite of the test cases in section 5 by including tests for matrices and hyper-matrices. This should be done by next week.
This’ll be followed by performance testing. We hardly have any benchmark tests in the elementary_functions module. Currently, I'm not aware of any standard benchmarks for elementary functions. In the worst case, if couldn't find any appropriate one, we can go for comparing fdlibm's performance with the old functions. This is required because we need to decide if fdlibm stays as the default or as an optional library based upon its performance results.