Developing tasks with JBench
JBench allows users to write their own code to be benchmarked (it
would be pretty useless otherwise!) by developing tasks. A
task is potentially extremely simple - sorting a list, for example.
All tasks in JBench implement the
BenchTask
interface, and can also have extra methods for configuration, which
JBench automatically finds. The documentation for BenchTask provides
almost everything you need to know, but the gist of it is that the
following happens:
- JBench reads the configuration
file specified on the command line.
This contains some general information, such as the number of tests
to run for each task, and task-specific information such as the
class implementing the task and any configuration parameters the
task needs.
- For each correctly configured task, JBench runs the number of
tests specified in
jbench.runs. Before each test is
run, its prepareTest is called so it can perform any
per-test initialisation (such as preparing test data to sort). The stopwatch
is then started, runTest is called, and the stopwatch
is stopped. checkResults is then called so that the
task can check that it has actually done what it should (eg that the
data which should have been sorted is now in the desired order).
After all the tests for a task are completed (and if no errors occur,
ie no exceptions are thrown), the results for the task will then be
displayed. If any test fails, the whole task is deemed to fail. Once
a task has been run, JBench releases its references so that any memory
used may be released. Note, however, that all the instances will be
created before any are run, so expensive initialisations should be
performed in prepareTest rather than in the task's
constructor. The garbage collector is encouraged to run between the
prepareTest and runTest methods in order
to minimise the chances of garbage collection interfering with the
test results.
JBench Manual