POSIX AIO vs libaio
- The POSIX AIO is a user-level implementation that performs normal blocking I/O in multiple threads. (-lrt)
- The kernel AIO (i.e. io_submit() et.al.) is kernel support for asynchronous I/O operations, where the io requests are actually queued up in the kernel. (-laio)
The current Linux POSIX AIO implementation is provided in user space by glibc.
This has a number of limitations, most notably that maintaining multiple threads to perform I/O operations is expensive and scales poorly. Work has been in progress for some time on a kernel state-machine-based implementation of asynchronous I/O.
libaio 是 io_submit() 等系统调用的封装. 一般只用在存储设备的读写操作上，如磁盘，闪存盘等。经常与 O_DIRECT 配合使用。
Currently, the AIO interface is best for O_DIRECT access to a raw block device like a disk, flash drive or storage array.
- AIO read and write on raw (and O_DIRECT on blockdev).
- AIO read and write on files opened with O_DIRECT on ext2, ext3, jfs, xfs.
What Does Not Work?
- AIO read and write on files opened without O_DIRECT (i.e. normal buffered filesystem AIO). On ext2, ext3, jfs, xfs and nfs, these do not return an explicit error, but quietly default to synchronous or rather non-AIO behaviour (i.e io_submit waits for I/O to complete in these cases). For most other filesystems, -EINVAL is reported.
- AIO fsync (not supported for any filesystem).
- AIO read and write on sockets (doesn’t return an explicit error, but quietly defaults to synchronous or rather non-AIO behavior).
- AIO read and write on pipes (reports -EINVAL).
- Not all devices (including TTY) support AIO (typically return -EINVAL).
- Kernel Asynchronous I/O (AIO) Support for Linux
- Linux Asynchronous I/O Explained
Direct I/O is a feature of the file system whereby file reads and writes go directly from the applications to the storage device, bypassing the operating system read and write caches. Direct I/O is used only by applications (such as databases) that manage their own caches.
An application invokes direct I/O by opening a file with the O_DIRECT flag. Alternatively, GFS can attach a direct I/O attribute to a file, in which case direct I/O is used regardless of how the file is opened.
When a file is opened with O_DIRECT, or when a GFS direct I/O attribute is attached to a file, all I/O operations must be done in block-size multiples of 512 bytes. The memory being read from or written to must also be 512-byte aligned.
To cause the O_DIRECT flag to be defined with recent glibc libraries, define _GNU_SOURCE at the beginning of a source file before any includes, or define it on the cc line when compiling.