QTest Device Emulation Testing Framework

• Qtest Driver Framework
  • QGraph concepts
    • Nodes
    • Edges
    • Execution steps
    • Command line
    • Troubleshooting unavailable tests
  • Creating a new driver and its interface
  • Adding a new test
  • Qgraph API reference

QTest is a device emulation testing framework. It can be very useful to test device models; it could also control certain aspects of QEMU (such as virtual clock stepping), with a special purpose “qtest” protocol. Refer to QTest Protocol for more details of the protocol.

QTest cases can be executed with

make check-qtest

The QTest library is implemented by tests/qtest/libqtest.c and the API is defined in tests/qtest/libqtest.h.

Consider adding a new QTest case when you are introducing a new virtual hardware, or extending one if you are adding functionalities to an existing virtual device.

On top of libqtest, a higher level library, libqos, was created to encapsulate common tasks of device drivers, such as memory management and communicating with system buses or devices. Many virtual device tests use libqos instead of directly calling into libqtest. Libqos also offers the Qgraph API to increase each test coverage and automate QEMU command line arguments and devices setup. Refer to Qtest Driver Framework for Qgraph explanation and API.

Steps to add a new QTest case are:

1. Create a new source file for the test. (More than one file can be added as necessary.) For example, tests/qtest/foo-test.c.

2. Write the test code with the glib and libqtest/libqos API. See also existing tests and the library headers for reference.

3. Register the new test in tests/qtest/meson.build. Add the test executable name to an appropriate qtests_* variable. There is one variable per architecture, plus qtests_generic for tests that can be run for all architectures. For example:

   qtests_generic = [
     ...
     'foo-test',
     ...
   ]
   

4. If the test has more than one source file or needs to be linked with any dependency other than qemuutil and qos, list them in the qtests dictionary. For example a test that needs to use the QIO library will have an entry like:

   {
     ...
     'foo-test': [io],
     ...
   }
   

Debugging a QTest failure is slightly harder than the unit test because the tests look up QEMU program names in the environment variables, such as QTEST_QEMU_BINARY and QTEST_QEMU_IMG, and also because it is not easy to attach gdb to the QEMU process spawned from the test. But manual invoking and using gdb on the test is still simple to do: find out the actual command from the output of

make check-qtest V=1

which you can run manually.

QTest Protocol

Line based protocol, request/response based. Server can send async messages so clients should always handle many async messages before the response comes in.

Valid requests

Clock management:

The qtest client is completely in charge of the QEMU_CLOCK_VIRTUAL. qtest commands let you adjust the value of the clock (monotonically). All the commands return the current value of the clock in nanoseconds.

> clock_step
< OK VALUE

Advance the clock to the next deadline. Useful when waiting for asynchronous events.

> clock_step NS
< OK VALUE

Advance the clock by NS nanoseconds.

> clock_set NS
< OK VALUE

Advance the clock to NS nanoseconds (do nothing if it’s already past).

PIO and memory access:

> outb ADDR VALUE
< OK

> outw ADDR VALUE
< OK

> outl ADDR VALUE
< OK

> inb ADDR
< OK VALUE

> inw ADDR
< OK VALUE

> inl ADDR
< OK VALUE

> writeb ADDR VALUE
< OK

> writew ADDR VALUE
< OK

> writel ADDR VALUE
< OK

> writeq ADDR VALUE
< OK

> readb ADDR
< OK VALUE

> readw ADDR
< OK VALUE

> readl ADDR
< OK VALUE

> readq ADDR
< OK VALUE

> read ADDR SIZE
< OK DATA

> write ADDR SIZE DATA
< OK

> b64read ADDR SIZE
< OK B64_DATA

> b64write ADDR SIZE B64_DATA
< OK

> memset ADDR SIZE VALUE
< OK

ADDR, SIZE, VALUE are all integers parsed with strtoul() with a base of 0. For ‘memset’ a zero size is permitted and does nothing.

DATA is an arbitrarily long hex number prefixed with ‘0x’. If it’s smaller than the expected size, the value will be zero filled at the end of the data sequence.

B64_DATA is an arbitrarily long base64 encoded string. If the sizes do not match, the data will be truncated.

IRQ management:

> irq_intercept_in QOM-PATH
< OK

> irq_intercept_out QOM-PATH
< OK

Attach to the gpio-in (resp. gpio-out) pins exported by the device at QOM-PATH. When the pin is triggered, one of the following async messages will be printed to the qtest stream:

IRQ raise NUM
IRQ lower NUM

where NUM is an IRQ number. For the PC, interrupts can be intercepted simply with “irq_intercept_in ioapic” (note that IRQ0 comes out with NUM=0 even though it is remapped to GSI 2).

Setting interrupt level:

> set_irq_in QOM-PATH NAME NUM LEVEL
< OK

where NAME is the name of the irq/gpio list, NUM is an IRQ number and LEVEL is an signed integer IRQ level.

Forcibly set the given interrupt pin to the given level.

libqtest API reference

QTestState *qtest_initf(const char *fmt, ...)

Parameters

const char *fmt

QTestState *qtest_vinitf(const char *fmt, va_list ap)

Parameters

const char *fmt

QTestState *qtest_init(const char *extra_args)

Parameters

const char *extra_args

QTestState *qtest_init_with_env(const char *var, const char *extra_args)

Parameters

const char *var

QTestState *qtest_init_without_qmp_handshake(const char *extra_args)

Parameters

const char *extra_args

QTestState *qtest_init_with_serial(const char *extra_args, int *sock_fd)

Parameters

const char *extra_args

void qtest_wait_qemu(QTestState *s)

Parameters

QTestState *s

void qtest_kill_qemu(QTestState *s)

Parameters

QTestState *s

void qtest_quit(QTestState *s)

Parameters

QTestState *s

QDict *qtest_qmp_fds(QTestState *s, int *fds, size_t fds_num, const char *fmt, ...)

Parameters

QTestState *s

QDict *qtest_qmp(QTestState *s, const char *fmt, ...)

Parameters

QTestState *s

void qtest_qmp_send(QTestState *s, const char *fmt, ...)

Parameters

QTestState *s

void qtest_qmp_send_raw(QTestState *s, const char *fmt, ...)

Parameters

QTestState *s

int qtest_socket_server(const char *socket_path)

Parameters

const char *socket_path

QDict *qtest_vqmp_fds(QTestState *s, int *fds, size_t fds_num, const char *fmt, va_list ap)

Parameters

QTestState *s

QDict *qtest_vqmp(QTestState *s, const char *fmt, va_list ap)

Parameters

QTestState *s

void qtest_qmp_vsend_fds(QTestState *s, int *fds, size_t fds_num, const char *fmt, va_list ap)

Parameters

QTestState *s

void qtest_qmp_vsend(QTestState *s, const char *fmt, va_list ap)

Parameters

QTestState *s

QDict *qtest_qmp_receive_dict(QTestState *s)

Parameters

QTestState *s

QDict *qtest_qmp_receive(QTestState *s)

Parameters

QTestState *s

void qtest_qmp_set_event_callback(QTestState *s, QTestQMPEventCallback cb, void *opaque)

Parameters

QTestState *s

void qtest_qmp_eventwait(QTestState *s, const char *event)

Parameters

QTestState *s

QDict *qtest_qmp_eventwait_ref(QTestState *s, const char *event)

Parameters

QTestState *s

QDict *qtest_qmp_event_ref(QTestState *s, const char *event)

Parameters

QTestState *s

char *qtest_hmp(QTestState *s, const char *fmt, ...)

Parameters

QTestState *s

char *qtest_vhmp(QTestState *s, const char *fmt, va_list ap)

Parameters

QTestState *s

bool qtest_get_irq(QTestState *s, int num)

Parameters

QTestState *s

void qtest_irq_intercept_in(QTestState *s, const char *string)

Parameters

QTestState *s

void qtest_irq_intercept_out(QTestState *s, const char *string)

Parameters

QTestState *s

void qtest_irq_intercept_out_named(QTestState *s, const char *qom_path, const char *name)

Parameters

QTestState *s

void qtest_set_irq_in(QTestState *s, const char *string, const char *name, int irq, int level)

Parameters

QTestState *s

void qtest_outb(QTestState *s, uint16_t addr, uint8_t value)

Parameters

QTestState *s

void qtest_outw(QTestState *s, uint16_t addr, uint16_t value)

Parameters

QTestState *s

void qtest_outl(QTestState *s, uint16_t addr, uint32_t value)

Parameters

QTestState *s

uint8_t qtest_inb(QTestState *s, uint16_t addr)

Parameters

QTestState *s

uint16_t qtest_inw(QTestState *s, uint16_t addr)

Parameters

QTestState *s

uint32_t qtest_inl(QTestState *s, uint16_t addr)

Parameters

QTestState *s

void qtest_writeb(QTestState *s, uint64_t addr, uint8_t value)

Parameters

QTestState *s

void qtest_writew(QTestState *s, uint64_t addr, uint16_t value)

Parameters

QTestState *s

void qtest_writel(QTestState *s, uint64_t addr, uint32_t value)

Parameters

QTestState *s

void qtest_writeq(QTestState *s, uint64_t addr, uint64_t value)

Parameters

QTestState *s

uint8_t qtest_readb(QTestState *s, uint64_t addr)

Parameters

QTestState *s

uint16_t qtest_readw(QTestState *s, uint64_t addr)

Parameters

QTestState *s

uint32_t qtest_readl(QTestState *s, uint64_t addr)

Parameters

QTestState *s

uint64_t qtest_readq(QTestState *s, uint64_t addr)

Parameters

QTestState *s

void qtest_memread(QTestState *s, uint64_t addr, void *data, size_t size)

Parameters

QTestState *s

uint64_t qtest_rtas_call(QTestState *s, const char *name, uint32_t nargs, uint64_t args, uint32_t nret, uint64_t ret)

Parameters

QTestState *s

void qtest_bufread(QTestState *s, uint64_t addr, void *data, size_t size)

Parameters

QTestState *s

void qtest_memwrite(QTestState *s, uint64_t addr, const void *data, size_t size)

Parameters

QTestState *s

void qtest_bufwrite(QTestState *s, uint64_t addr, const void *data, size_t size)

Parameters

QTestState *s

void qtest_memset(QTestState *s, uint64_t addr, uint8_t patt, size_t size)

Parameters

QTestState *s

int64_t qtest_clock_step_next(QTestState *s)

Parameters

QTestState *s

int64_t qtest_clock_step(QTestState *s, int64_t step)

Parameters

QTestState *s

int64_t qtest_clock_set(QTestState *s, int64_t val)

Parameters

QTestState *s

bool qtest_big_endian(QTestState *s)

Parameters

QTestState *s

const char *qtest_get_arch(void)

Parameters

void

no arguments

Return

bool qtest_has_accel(const char *accel_name)

Parameters

const char *accel_name

void qtest_add_func(const char *str, void (*fn)(void))

Parameters

const char *str

void qtest_add_data_func(const char *str, const void *data, void (*fn)(const void*))

Parameters

const char *str

void qtest_add_data_func_full(const char *str, void *data, void (*fn)(const void*), GDestroyNotify data_free_func)

Parameters

const char *str

qtest_add

qtest_add (testpath, Fixture, tdata, fsetup, ftest, fteardown)

Parameters

testpath

void qtest_add_abrt_handler(GHookFunc fn, const void *data)

Parameters

GHookFunc fn

void qtest_remove_abrt_handler(void *data)

Parameters

void *data

QDict *qtest_vqmp_assert_success_ref(QTestState *qts, const char *fmt, va_list args)

Parameters

QTestState *qts

void qtest_vqmp_assert_success(QTestState *qts, const char *fmt, va_list args)

Parameters

QTestState *qts

QDict *qtest_vqmp_fds_assert_success_ref(QTestState *qts, int *fds, size_t nfds, const char *fmt, va_list args)

Parameters

QTestState *qts

void qtest_vqmp_fds_assert_success(QTestState *qts, int *fds, size_t nfds, const char *fmt, va_list args)

Parameters

QTestState *qts

QDict *qtest_qmp_assert_failure_ref(QTestState *qts, const char *fmt, ...)

Parameters

QTestState *qts

QDict *qtest_vqmp_assert_failure_ref(QTestState *qts, const char *fmt, va_list args)

Parameters

QTestState *qts

QDict *qtest_qmp_assert_success_ref(QTestState *qts, const char *fmt, ...)

Parameters

QTestState *qts

void qtest_qmp_assert_success(QTestState *qts, const char *fmt, ...)

Parameters

QTestState *qts

QDict *qtest_qmp_fds_assert_success_ref(QTestState *qts, int *fds, size_t nfds, const char *fmt, ...)

Parameters

QTestState *qts

void qtest_qmp_fds_assert_success(QTestState *qts, int *fds, size_t nfds, const char *fmt, ...)

Parameters

QTestState *qts

void qtest_cb_for_every_machine(void (*cb)(const char *machine), bool skip_old_versioned)

Parameters

void (*cb)(const char *machine)

char *qtest_resolve_machine_alias(const char *var, const char *alias)

Parameters

const char *var

bool qtest_has_machine(const char *machine)

Parameters

const char *machine

bool qtest_has_machine_with_env(const char *var, const char *machine)

Parameters

const char *var

bool qtest_has_device(const char *device)

Parameters

const char *device

void qtest_qmp_device_add_qdict(QTestState *qts, const char *drv, const QDict *arguments)

Parameters

QTestState *qts

void qtest_qmp_device_add(QTestState *qts, const char *driver, const char *id, const char *fmt, ...)

Parameters

QTestState *qts

void qtest_qmp_add_client(QTestState *qts, const char *protocol, int fd)

Parameters

QTestState *qts

void qtest_qmp_device_del_send(QTestState *qts, const char *id)

Parameters

QTestState *qts

void qtest_qmp_device_del(QTestState *qts, const char *id)

Parameters

QTestState *qts

bool qtest_probe_child(QTestState *s)

Parameters

QTestState *s

void qtest_set_expected_status(QTestState *s, int status)

Parameters

QTestState *s

void qtest_qom_set_bool(QTestState *s, const char *path, const char *property, bool value)

Parameters

QTestState *s

bool qtest_qom_get_bool(QTestState *s, const char *path, const char *property)

Parameters

QTestState *s

pid_t qtest_pid(QTestState *s)

Parameters

QTestState *s

bool have_qemu_img(void)

Parameters

void

no arguments

Return

bool mkimg(const char *file, const char *fmt, unsigned size_mb)

Parameters

const char *file