Philip Hudson wrote:
[…]
> For my part, and accepting that I'm not a major contributor so what I
> want doesn't necessarily carry that much weight, I only feel
> comfortable contributing Java code that I know will continuously be
> tested. That is, I want the JUnit tests I *always* write for my Java
> code to be part of the same commit as that code. I want those tests
> integrated into a build which contains a JUnit step. That step runs
> all JUnit tests available.
I think we're all set up for those who wish to follow this sort of
discipline. Running the current test suite as assembled in 'abcl.asd'
takes about twenty minutes on a recent Intel Duo Core 2 processor not
constrained by RAM, so perhaps this is a reason that some people avoid
the suite.
I might as well given an informal explanation of the nature of the tests
in the Ant-based build system, as perhaps it is not widely understood
what my intentions were.
As a design principle, I have attempted to implement the invocation of
the Lisp based tests via the ASDF definition in 'abcl.asd' so the test
suite would be accessible to the Lisp-hosted build as well. Due to
either my lack of understanding of ASDF, or the early version of the
ASDF facility included with ABCL (ABCL has ASDF 1.3, where SBCL ships
with ASDF 1.130. ABCL cannot be upgraded to a later ADSF version without
at least implementing the long form of DEFINE-METHOD-COMBINATION), the
use of ASDF is currently very hackish, requiring a :FORCE t passed for
the invocations, and in reality relying on "stub code" for each test
suite under 'test/lisp'.
As Erik mentioned earlier, the target 'abcl.test' will do what you ask:
build ABCL, and run tests. The tests are subdivided into the Lisp
based tests invoked via the 'abcl.test.lisp' target and the JUnit based
tests invoked by the 'abcl.test.java' target.
'abcl.test.lisp' will first invoke the compiled version of the GCL ANSI
test suite if a sibling directory to the ABCL root named
'#p../ansi-tests exists. Then 'abcl.test.lisp' will invoke the internal
ABCL tests located in 'test/lisp/abcl'.
'abcl.test.java' will invoke the JUnit tests located under 'test/src'.
As each test suite is run, a log-file is created on the filesystem
containing the output of the test-suite run. The ANSI tests currently
create this file in the top-level ABCL, whereas the ABCL tests create
the log file under 'test/lisp/abcl'. This should be rationalized to one
or the other locations. The log files are named with the invocation
time including HHMM, so as long as you don't invoke the tests in the
same minute, you shouldn't clobber previous test results. The removal
of the log files should probably be added to the 'abcl.clean' target.
What is not working: 'test/lisp/cl-bench' is an incomplete wrapping of
the CL-BENCH suite. Unfortunately, I don't remember now what the
problem was (something about the invocation depending on a Bourne shell
script?)
To submit tests with your patches:
1. Lisp based tests
ABCL uses its own version of the venerable MIT Regression Test (RT)
package which in order to not conflict with the RT tests in the GCL ANSI
suite is defined in theABCL-REGRESSION-TEST package. Other than that, I
think it is pretty identical to the version shipped with the GCL ANSI
test suite.
To define a test, one uses the DEFTEST form which has the basic template
of:
(in-package #:abcl.test.lisp)
(deftest foo.1
TEST-FORM
RESULT-FORM)
When invoked, the FOO.1 test evaluates the TEST-FORM, and compares it to
the RESULT-FORM. If [the two are EQL ][1], the test passes.
[1]: Not sure about the EQL comparison: might be something slightly
different.
By convention tests are named 'FOO.N' where N is a positive integer, but
this is just a grouping mechanism for human parsing as far as I know.
To add a Lisp test, either add it to an existing file in
'test/lisp/abcl', like 'misc-tests.lisp', or create a new file and add
it to the RUN method in 'test/lisp/abcl/package.lisp'. One can use
ABCL-RT:DO-TEST to invoke individual tests, although sometimes RT gets
weird about which package one invokes the tests from. Changing the
invoking REPL to ABCL-TEST-LISP package seems to help from what I remember.
2. Java based tests
ABCL currently uses JUnit 4.5, for the reason that the sole tests ABCL
ever received (from Phillip Hudson) used this package. As far as I can
tell, JUnit with the use of Java 5 annotations is about everything you
really need in a lightweight Java test suite, but if someone had reasons
for using, say, 'Test-NG' we would definitely entertain employing that
to get contributed tests.
Adding a JUnit test consists of first including the code under
'test/src', probably in the 'test/src/org/armedbear/lisp' directory (and
associated package). Since this code doesn't get included in the
'abcl.jar', it is apparently pretty safe to use the same package as the
code you are testing. But I don't think this issue has been given much
thought, so perhaps tests should go in 'org.armedbear.lisp.test', to
avoid static initialization problems.
After the code is in place, one has to explicitly mention the source
file name in the 'abcl.test.java' target in 'build.xml'. It should be
pretty easy to automatically pick up everything under 'test/src', but
since we currently have only one test, I didn't bother doing this quite yet.
---
Hopefully, that information can help someone out who wishes to
contribute tests with ABCL patches. I haven't really had the time to
keep up with my planned ABCL work over the past half year, so if someone
wanted to come and re-think how we add tests to ABCL, I wouldn't be
offended. But at least I have now sort of documented the current state,
and some of the background thinking, to give someone a head start.
yers in the Bear,
Mark <
[hidden email]>
--
"A screaming comes across the sky. It has happened before, but there
is nothing to compare to it now."
------------------------------------------------------------------------------
Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
is a gathering of tech-side developers & brand creativity professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, &
iPhoneDevCamp as they present alongside digital heavyweights like Barbarian
Group, R/GA, & Big Spaceship.
http://p.sf.net/sfu/creativitycat-com
_______________________________________________
armedbear-j-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/armedbear-j-devel
