Top | ![]() |
![]() |
![]() |
![]() |
#define | IGT_EXIT_TIMEOUT |
#define | IGT_EXIT_SKIP |
#define | IGT_EXIT_SUCCESS |
#define | IGT_EXIT_INVALID |
#define | igt_main |
#define | igt_simple_main |
struct | igt_helper_process |
enum | igt_log_level |
This library implements the core of the i-g-t test support infrastructure. Main features are the subtest enumeration, cmdline option parsing helpers for subtest handling and various helpers to structure testcases with subtests and handle subtest test results.
Auxiliary code provides exit handlers, support for forked processes with test result propagation. Other generally useful functionality includes optional structure logging infrastructure and some support code for running reduced test set on in simulated hardware environments.
When writing tests with subtests it is extremely important that nothing interferes with the subtest enumeration. In i-g-t subtests are enumerated at runtime, which allows powerful testcase enumeration. But it makes subtest enumeration a bit more tricky since the test code needs to be careful to never run any code which might fail (like trying to do privileged operations or opening device driver nodes).
To allow this i-g-t provides igt_fixture code blocks for setup code outside
of subtests and automatically skips the subtest code blocks themselves. For
special cases igt_only_list_subtests()
is also provided.
i-g-t makes heavy use of C macros which serve as magic control blocks. They work fairly well and transparently but since C doesn't have full-blown closures there are caveats:
Asynchronous blocks which are used to spawn children internally use fork()
.
Which means that nonsensical control flow like jumping out of the control
block is possible, but it will badly confuse the i-g-t library code. And of
course all caveats of a real fork()
call apply, namely that file
descriptors are copied, but still point at the original file. This will
terminally upset the libdrm buffer manager if both parent and child keep on
using the same open instance of the drm device. Usually everything related
to interacting with the kernel driver must be reinitialized to avoid such
issues.
Code blocks with magic control flow are implemented with setjmp()
and
longjmp()
. This applies to igt_fixture and igt_subtest blocks and all the
three variants to finish test: igt_success()
, igt_skip()
and igt_fail()
.
Mostly this is of no concern, except when such a control block changes
stack variables defined in the same function as the control block resides.
Any store/load behaviour after a longjmp()
is ill-defined for these
variables. Avoid such code.
Quoting the man page for longjmp()
:
"The values of automatic variables are unspecified after a call to
longjmp()
if they meet all the following criteria:"
"they are local to the function that made the corresponding setjmp()
call;
"their values are changed between the calls to setjmp()
and longjmp()
; and
"they are not declared as volatile."
Kernel tests itself tend to have fairly complex logic already. It is therefore paramount that helper code, both in libraries and test-private functions, add as little boilerplate code to the main test logic as possible. But then dense code is hard to understand without constantly consulting the documentation and implementation of all the helper functions if it doesn't follow some clear patterns. Hence follow these established best practices:
Make extensive use of the implicit control flow afforded by igt_skip()
,
igt_fail and igt_success()
. When dealing with optional kernel features
combine igt_skip()
with igt_fail()
to skip when the kernel support isn't
available but fail when anything else goes awry. void should be the most
common return type in all your functions, except object constructors of
course.
The main test logic should have no explicit control flow for failure
conditions, but instead such assumptions should be written in a declarative
style. Use one of the many macros which encapsulate i-g-t's implicit
control flow. Pick the most suitable one to have as much debug output as
possible without polluting the code unnecessarily. For example
igt_assert_cmpint()
for comparing integers or do_ioctl()
for running ioctls
and checking their results. Feel free to add new ones to the library or
wrap up a set of checks into a private function to further condense your
test logic.
When adding a new feature test function which uses igt_skip()
internally,
use the <prefix>_require_<feature_name> naming scheme. When you
instead add a feature test function which returns a boolean, because your
main test logic must take different actions depending upon the feature's
availability, then instead use the <prefix>_has_<feature_name>.
As already mentioned eschew explicit error handling logic as much as possible. If your test absolutely has to handle the error of some function the customary naming pattern is to prefix those variants with __. Try to restrict explicit error handling to leaf functions. For the main test flow simply pass the expected error condition down into your helper code, which results in tidy and declarative test logic.
Make your library functions as simple to use as possible. Automatically
register cleanup handlers through igt_install_exit_handler()
. Reduce the
amount of setup boilerplate needed by using implicit singletons and lazy
structure initialization and similar design patterns.
Don't shy away from refactoring common code, even when there are just 2-3 users and even if it's not a net reduction in code. As long as it helps to remove boilerplate and makes the code more declarative the resulting clearer test flow is worth it. All i-g-t library code has been organically extracted from testcases in this fashion.
For general coding style issues please follow the kernel's rules laid out in CodingStyle.
i-g-t testcase are all executables which should be run as root on an otherwise completely idle system. The test status is reflected in the exitcode. IGT_EXIT_SUCCESS means "success", IGT_EXIT_SKIP "skip", IGT_EXIT_TIMEOUT that some operation "timed out". All other exit codes encode a failed test result, including any abnormal termination of the test (e.g. by SIGKILL).
On top of that tests may report unexpected results and minor issues to stderr. If stderr is non-empty the test result should be treated as "warn".
The test lists are generated at build time. Simple testcases are listed in tests/single-tests.txt and tests with subtests are listed in tests/multi-tests.txt. When running tests with subtest from a test runner it is recommend to run each subtest individually, since otherwise the return code will only reflect the overall result.
To do that obtain the lists of subtests with "--list-subtests", which can be run as non-root and doesn't require the i915 driver to be loaded (or any intel gpu to be present). Then individual subtests can be run with "--run-subtest". Usage help for tests with subtests can be obtained with the "--help" command line option.
#define IGT_TEST_DESCRIPTION(str) const char* __igt_test_description = str
Defines a description for a test. This is used as the output for the "--help-description" option and is also included in the generated documentation.
#define igt_fixture
Annotate global test fixture code
Testcase with subtests often need to set up a bunch of global state as the common test fixture. To avoid such code interferring with the subtest enumeration (e.g. when enumerating on systemes without an intel gpu) such blocks should be annotated with igt_fixture.
void igt_subtest_init (int argc
,char **argv
);
This initializes the for tests with subtests without the need for additional
cmdline options. It is just a simplified version of
igt_subtest_init_parse_opts()
.
If there's not a reason to the contrary it's less error prone to just use an
igt_main block instead of stitching the tests's main()
function together
manually.
int igt_subtest_init_parse_opts (int argc
,char **argv
,const char *extra_short_opts
,struct option *extra_long_opts
,const char *help_str
,igt_opt_handler_t extra_opt_handler
);
This function handles the subtest related cmdline options and allows an arbitrary set of additional options. This is useful for tests which have additional knobs to tune when run manually like the number of rounds execute or the size of the allocated buffer objects.
Tests without special needs should just use igt_subtest_init()
or use
igt_main directly instead of their own main()
function.
argc |
argc from the test's |
|
argv |
argv from the test's |
|
extra_short_opts |
getopt_long() compliant list with additional short options |
|
extra_long_opts |
getopt_long() compliant list with additional long options |
|
help_str |
help string for the additional options |
|
extra_opt_handler |
handler for the additional options |
#define igt_tokencat(x, y) __igt_tokencat2(x, y)
C preprocessor helper to concatenate two variables while properly expanding them.
#define igt_subtest(name)
This is a magic control flow block which denotes a subtest code block. Within that code block igt_skip|success will only bail out of the subtest. The _f variant accepts a printf format string, which is useful for constructing combinatorial tests.
This is a simpler version of igt_subtest_f()
#define igt_subtest_f(f...)
This is a magic control flow block which denotes a subtest code block. Within that code block igt_skip|success will only bail out of the subtest. The _f variant accepts a printf format string, which is useful for constructing combinatorial tests.
Like igt_subtest()
, but also accepts a printf format string instead of a
static string.
void igt_simple_init_parse_opts (int argc
,char **argv
,const char *extra_short_opts
,struct option *extra_long_opts
,const char *help_str
,igt_opt_handler_t extra_opt_handler
);
This initializes a simple test without any support for subtests and allows an arbitrary set of additional options.
argc |
argc from the test's |
|
argv |
argv from the test's |
|
extra_short_opts |
getopt_long() compliant list with additional short options |
|
extra_long_opts |
getopt_long() compliant list with additional long options |
|
help_str |
help string for the additional options |
|
extra_opt_handler |
handler for the additional options |
void igt_skip (const char *f
,...
);
Subtest aware test skipping. The format string is printed to stderr as the reason why the test skipped.
For tests with subtests this will either bail out of the current subtest or mark all subsequent subtests as SKIP (presuming some global setup code failed).
For normal tests without subtest it will directly exit.
void
igt_success (void
);
Complete a (subtest) as successful
This bails out of a subtests and marks it as successful. For global tests it it won't bail out of anything.
void
igt_fail (int exitcode
);
Fail a testcase. The exitcode is used as the exit code of the test process. It may not be 0 (which indicates success) or 77 (which indicates a skipped test).
For tests with subtests this will either bail out of the current subtest or mark all subsequent subtests as FAIL (presuming some global setup code failed).
For normal tests without subtest it will directly exit with the given exitcode.
void
igt_exit (void
);
exit() for both types (simple and with subtests) of i-g-t tests.
This will exit the test with the right exit code when subtests have been skipped. For normal tests it exits with a successful exit code, presuming everything has worked out. For subtests it also checks that at least one subtest has been run (save when only listing subtests.
It is an error to normally exit a test calling igt_exit()
- without it the
result reporting will be wrong. To avoid such issues it is highly recommended
to use igt_main or igt_simple_main instead of a hand-rolled main()
function.
#define igt_assert(expr)
Fails (sub-)test if the condition is not met.
Should be used everywhere where a test checks results.
#define igt_assert_f(expr, f...)
Fails (sub-)test if the condition is not met.
Should be used everywhere where a test checks results.
In addition to the plain igt_assert()
helper this allows to print additional
information to help debugging test failures.
#define igt_fail_on(expr) igt_assert(!(expr))
Fails (sub-)test if the condition is met.
Should be used everywhere where a test checks results.
#define igt_fail_on_f(expr, f...) igt_assert_f(!(expr), f)
Fails (sub-)test if the condition is met.
Should be used everywhere where a test checks results.
In addition to the plain igt_assert()
helper this allows to print additional
information to help debugging test failures.
#define igt_assert_cmpint(n1, cmp, ncmp, n2)
Fails (sub-)test if the condition is not met
Should be used everywhere where a test compares two integer values.
Like igt_assert()
, but displays the values being compared on failure instead
of simply printing the stringified expression.
#define igt_assert_eq(n1, n2) igt_assert_cmpint(n1, ==, !=, n2)
Fails (sub-)test if the two integers are not equal. Beware that for now this only works on integers.
Like igt_assert()
, but displays the values being compared on failure instead
of simply printing the stringified expression.
#define igt_assert_neq(n1, n2) igt_assert_cmpint(n1, !=, ==, n2)
Fails (sub-)test if the two integers are equal. Beware that for now this only works on integers.
Like igt_assert()
, but displays the values being compared on failure instead
of simply printing the stringified expression.
#define igt_assert_lte(n1, n2) igt_assert_cmpint(n1, <=, >, n2)
Fails (sub-)test if the second integers is greater than the first. Beware that for now this only works on integers.
Like igt_assert()
, but displays the values being compared on failure instead
of simply printing the stringified expression.
#define igt_require(expr)
Skip a (sub-)test if a condition is not met.
Should be used everywhere where a test checks results to decide about
skipping. This is useful to streamline the skip logic since it allows for a more flat
code control flow, similar to igt_assert()
#define igt_skip_on(expr)
Skip a (sub-)test if a condition is met.
Should be used everywhere where a test checks results to decide about
skipping. This is useful to streamline the skip logic since it allows for a more flat
code control flow, similar to igt_assert()
#define igt_require_f(expr, f...)
Skip a (sub-)test if a condition is not met.
Should be used everywhere where a test checks results to decide about
skipping. This is useful to streamline the skip logic since it allows for a more flat
code control flow, similar to igt_assert()
In addition to the plain igt_require()
helper this allows to print additional
information to help debugging test failures.
#define igt_skip_on_f(expr, f...)
Skip a (sub-)test if a condition is met.
Should be used everywhere where a test checks results to decide about
skipping. This is useful to streamline the skip logic since it allows for a more flat
code control flow, similar to igt_assert()
In addition to the plain igt_skip_on()
helper this allows to print additional
information to help debugging test failures.
#define igt_fork(child, num_children)
This is a magic control flow block which spawns parallel test threads with
fork()
.
The test children execute in parallel to the main test thread. Joining all test threads should be done with igt_waitchildren to ensure that the exit codes of all children are properly reflected in the test status.
Note that igt_skip()
will not be forwarded, feature tests need to be done
before spawning threads with igt_fork()
.
void
igt_waitchildren (void
);
Wait for all children forked with igt_fork.
The magic here is that exit codes from children will be correctly propagated to the main thread, including the relevant exit code if a child thread failed. Of course if multiple children failed with different exit codes the resulting exit code will be non-deterministic.
Note that igt_skip()
will not be forwarded, feature tests need to be done
before spawning threads with igt_fork()
.
#define igt_fork_helper(proc)
This is a magic control flow block which denotes an asynchronous helper
process block. The difference compared to igt_fork()
is that failures from
the child process will not be forwarded, making this construct more suitable
for background processes. Common use cases are regular interference of the
main test thread through e.g. sending signals or evicting objects through
debugfs. Through the explicit igt_helper_process they can also be controlled
in a more fine-grained way than test children spawned through igt_fork()
.
For tests with subtest helper process can be started outside of a igt_subtest block.
Calling igt_wait_helper()
joins a helper process and igt_stop_helper()
forcefully terminates it.
int
igt_wait_helper (struct igt_helper_process *proc
);
Joins a helper process. It is an error to call this on a helper process which hasn't been spawned yet.
void
igt_stop_helper (struct igt_helper_process *proc
);
Terminates a helper process. It is an error to call this on a helper process which hasn't been spawned yet.
void
(*igt_exit_handler_t) (int sig
);
Exit handler type used by igt_install_exit_handler()
. Note that exit handlers
can potentially be run from signal handling contexts, the sig
parameter can
be used to figure this out and act accordingly.
void
igt_install_exit_handler (igt_exit_handler_t fn
);
Set a handler that will be called either when the process calls exit()
or
returns from the main function, or one of the signals in
'handled_signals' is raised. MAX_EXIT_HANDLERS handlers can be installed,
each of which will be called only once, even if a subsequent signal is
raised. If the exit handlers are called due to a signal, the signal will be
re-raised with the original signal disposition after all handlers returned.
The handler will be passed the signal number if called due to a signal, or
0 otherwise. Exit handlers can also be used from test children spawned with
igt_fork()
, but not from within helper processes spawned with
igt_fork_helper()
. The list of exit handlers is reset when forking to
avoid issues with children cleanup up the parent's state too early.
void
igt_enable_exit_handler (void
);
Re-enable all exit handlers temporarily disabled with
igt_disable_exit_handler()
.
void
igt_disable_exit_handler (void
);
Temporarily disable all exit handlers. Useful for library code doing tricky things.
bool
igt_run_in_simulation (void
);
This function can be used to select a reduced test set when running in simulation environments. This i-g-t mode is selected by setting the INTEL_SIMULATION environment variable to 1.
#define SLOW_QUICK(slow,quick) (igt_run_in_simulation() ? (quick) : (slow))
Simple macro to select between two values (e.g. number of test rounds or test buffer size) depending upon whether i-g-t is run in simulation mode or not.
void
igt_skip_on_simulation (void
);
Skip tests when INTEL_SIMULATION environment variable is set. It uses
igt_skip()
internally and hence is fully subtest aware.
Note that in contrast to all other functions which use igt_skip()
internally
it is allowed to use this outside of an igt_fixture block in a test with
subtests. This is because in contrast to most other test requirements,
checking for simulation mode doesn't depend upon the present hardware and it
so makes a lot of sense to have this check in the outermost igt_main block.
void igt_log (const char *domain
,enum igt_log_level level
,const char *format
,...
);
This is the generic structured logging helper function. i-g-t testcase should output all normal message to stdout. Warning level message should be printed to stderr and the test runner should treat this as an intermediate result between SUCESS and FAILURE.
The log level can be set through the IGT_LOG_LEVEL environment variable with values "debug", "info", "warn" and "none". By default verbose debug message are disabled. "none" completely disables all output and is not recommended since crucial issues only reported at the IGT_LOG_WARN level are ignored.
void igt_vlog (const char *domain
,enum igt_log_level level
,const char *format
,va_list args
);
This is the generic logging helper function using an explicit varargs structure and hence useful to implement domain-specific logging functions.
If there is no need to wrap up a vararg list in the caller it is simpler to
just use igt_log()
.
#define igt_debug(f...) igt_log(IGT_LOG_DOMAIN, IGT_LOG_DEBUG, f)
Wrapper for igt_log()
for message at the IGT_LOG_DEBUG level.
#define igt_info(f...) igt_log(IGT_LOG_DOMAIN, IGT_LOG_INFO, f)
Wrapper for igt_log()
for message at the IGT_LOG_INFO level.
#define igt_warn(f...) igt_log(IGT_LOG_DOMAIN, IGT_LOG_WARN, f)
Wrapper for igt_log()
for message at the IGT_LOG_WARN level.
#define igt_warn_on(condition)
Print a IGT_LOG_WARN level message if a condition is not met.
Should be used everywhere where a test checks results to decide about
printing warnings. This is useful to streamline the test logic since it
allows for a more flat code control flow, similar to igt_assert()
#define igt_warn_on_f(condition, f...)
Skip a (sub-)test if a condition is not met.
Print a IGT_LOG_WARN level message if a condition is not met.
Should be used everywhere where a test checks results to decide about
printing warnings. This is useful to streamline the test logic since it
allows for a more flat code control flow, similar to igt_assert()
In addition to the plain igt_warn_on_f()
helper this allows to print
additional information (again as warnings) to help debugging test failures.
void
igt_set_timeout (unsigned int seconds
);
Fail a test and exit with IGT_EXIT_TIMEOUT status after the specified number of seconds have elapsed. If the current test has subtests and the timeout occurs outside a subtest, subsequent subtests will be skipped and marked as failed.
Any previous timer is cancelled and no timeout is scheduled if seconds
is
zero.
#define IGT_EXIT_INVALID 79
Exit status indicating an invalid option or subtest was specified
#define igt_main
This is a magic control flow block used instead of a main()
function for
tests with subtests. Open-coding the main()
function is only recommended if
the test needs to parse additional cmdline arguments of its own.
#define igt_simple_main
This is a magic control flow block used instead of a main()
function for
simple tests. Open-coding the main()
function is only recommended if
the test needs to parse additional cmdline arguments of its own.
struct igt_helper_process { bool running; bool use_SIGKILL; pid_t pid; int id; };