...one of the most highly
regarded and expertly designed C++ library projects in the
world.
— Herb Sutter and Andrei
Alexandrescu, C++
Coding Standards
Boost.process V2 is an redesign of boost.process, based on previous design mistakes & improved system APIs.
The major changes are
In process v1 one can define partial settings in the constructor of the process, which has lead to a small DSL.
child c{exe="test", args+="--help", std_in < null(), env["FOO"] += "BAR"};
While this looks fancy at first, it really does not scale well with more parameters. For process v2, the interfaces is simple:
extern std::unordered_map<std::string, std::string> my_env; extern asio::io_context ctx; process proc(ctx, "./test", {"--help"}, process_io{nullptr, {}, {}}, process_environment(my_env));
Every initializer addresses one logical component (e.g. stdio) instead of multiple ones accumulating. Furthermore, every process has a path and arguments, instead of a confusing mixture of cmd-style and exe-args that can be randomly spread out.
Since process v1 came out, linux has moved along and added pidfd_open which
allows users to get a file descriptor for a process. This is much more
reliable since it is not as easy to miss as a SIGCHLD
.
FreeBSD has a similar feature with pdfork
which is also supported, while windows has provided HANDLE
s
for processes all along. Unless the OS doesn't support it, process v2 will
use file descriptors and handles to implement waiting for processes.
Process v1 aimed to make asio optional, but synchronous IO with subprocesses usually means one is begging for deadlocks. Since asio added pipes in boost 1.78, boost process V2 is fully asio based and uses it's pipes and file-handles for the subprocess.
Certain parts of boost.process were not as reliable as they should've been.
This concerns especially the wait_for
and wait_until
functions
on the process. The latter are easy to do on windows, but posix does not
provide an API for this. Thus the wait_for used signals or fork, which
was all but safe. Since process v2 is based on asio and thus supports cancellation,
a wait_for can not safely be implemented with an async_wait + timeout.
“UTF-8 or GTFO”--Vinnie Falco
Instead of using ascii-APIs on windows, process V2 just assumes UTF-8 everywhere.
Boost.process v2 supports separate compilation similar to other boost libraries.
It can be achieved by defining BOOST_PROCESS_V2_SEPARATE_COMPILATION
and including <process/v2/src.hpp>
in a single compile unit.
While not a problem on windows (since HANDLEs get manually enabled for inheritance), posix systems create a problem with inheriting file handles by default.
Process V2 will automatically close all non-whitelisted descriptors, without needing any option to enable it.
A process needs four things to be launched:
// process(asio::any_io_executor, filesystem::path, range<string> args, AdditionalInitializers...) asio::io_context ctx; process proc(ctx, "/usr/bin/cp", {"source.txt", "target.txt"});
The started process can then be awaited or terminated.
If the process handle goes out of scope, it will terminate the subprocess.
You can prevent this, by calling proc.detach()
; do however note that this can lead to
zombie processes.
A process that completed will deliver an exit-code, which can be obtained
by calling .exit_code
on the exited process and which is also returned from .wait()
.
process proc("/bin/ls", {}); assert(proc.wait() == 0);
The normal exit-code is what the subprocess returned from main
; posix will however add additional
information about the process. This is called the native_exit_code
.
The .running()
function can be used to detect if the
process is still active.
The parent process can signal the subprocess demanding certain actions.
.terminate
will cause the subprocess to exit immediately (SIGKILL
on posix). This is the only reliable & portable way to end a subprocess.
process proc("/bin/totally-not-a-virus", {}); proc.terminate();
.request_exit
will ask the subprocess to shutdown (SIGTERM
on posix), which the subprocess might ignore.
process proc("/bin/bash", {}); proc.request_exit(); proc.wait();
.interrupt
will send an SIGINT to the subprocess, which a subprocess might interpret
as a signal to shutdown.
Warning | |
---|---|
interrupt requires the initializer |
process proc("/usr/bin/addr2line", {}); proc.request_exit(); proc.wait();
Process v2 provides execute
and async_execute
functions
that can be used for managed executions.
assert(execute(process("/bin/ls", {}) == 0));
The async version supports cancellation and will forward cancellation types as follows:
- asio::cancellation_type::total -> interrupt - asio::cancellation_type::partial -> request_exit - asio::cancellation_type::terminal -> terminate
asio::io_context ctx; asio::steady_timer timeout{ctx, std::chrono::seconds(10)}; asio::cancellation_signal sig; async_execute(process("/usr/bin/g++", {"hello_world.cpp"}), asio::bind_cancellation_slot(sig.slot(), [&](error_code ec, int exit_code) { timeout.cancel(); // we're done earlier })); timeout.async_wait( [&](error_code ec) { if (ec) // we were cancelled, do nothing return ; sig.emit(asio::cancellation_type::partial); // request exit first, but terminate after another 10 sec timeout.expires_after(std::chrono::seconds(10)); timeout.async_wait( [&](error_code ec) { if (!ec) sig.emit(asio::cancellation_type::terminal); }); ); });
The process creation is done by a process_launcher. The constructor of process
will use the default_launcher,
which varies by system. There are additional launcher available on most systems.
Table 28.2. Launcher overview
Name |
Summary |
Default on |
Available on |
---|---|---|---|
|
Launcher using |
windows |
windows |
|
Launcher using |
windows |
|
|
Launcher using |
windows |
|
|
Launcher using |
windows |
|
|
Launcher using fork & an error pipe |
most of posix |
posix |
|
Launcher using fork without error pipe |
posix |
|
|
Launcher using pdfork with an error pipe |
FreeBSD |
FreeBSD |
|
Launcher using vfork |
posix |
A launcher is invoked through the call operator.
auto l = windows::as_user_launcher((HANDLE)0xDEADBEEF); asio::io_context ctx; boost::system::error_code ec; auto proc = l(ctx, ec, "C:\\User\\boost\\Downloads\\totally_not_a_virus.exe", {});
The launcher will call certain functions on the initializer if they're present, as documented below. The initializer are used to modify the process behaviour.
The default and pdfork launchers on linux open an internal pipe to communicate errors that occur after forking back to the parent process.
This can be prevented by using the fork_and_forget_launcher
.
Alternatively, the vfork_launcher
can report errors directly back to the parent process.
Thus some calls to the initializers occur after forking from the child process.
struct custom_initializer { // functions called from the parent process: // called before a call to fork. A returned error will cancel the launch. template<typename Launcher> error_code on_setup(Launcher & launcher, const filesystem::path &executable, const char * const * (&cmd_line)); // called for every initializer if an error occurred during setup or process creation template<typename Launcher> void on_error(Launcher & launcher, const filesystem::path &executable, const char * const * (&cmd_line), const error_code & ec); // called after successful process creation template<typename Launcher> void on_success(Launcher & launcher, const filesystem::path &executable, const char * const * (&cmd_line)); // called for every initializer if an error occurred when forking, in addition to on_error. template<typename Launcher> void on_fork_error(Launcher & launcher, const filesystem::path &executable, const char * const * (&cmd_line), const error_code & ec); // called before a call to execve. A returned error will cancel the launch. Called from the child process. template<typename Launcher> error_code on_exec_setup(Launcher & launcher, const filesystem::path &executable, const char * const * (&cmd_line)); // called after a failed call to execve from the child process. template<typename Launcher> void on_exec_error(Launcher & launcher, const filesystem::path &executable, const char * const * (&cmd_line)); };
The call sequence on success:
The call sequence when fork fails:
The call sequence when exec fails:
The launcher will close all non-whitelisted file descriptors after on_exec_setup
.
Windows launchers are pretty straight forward, they will call the following functions on the initializer if present.
struct custom_initializer { // called before a call to CreateProcess. A returned error will cancel the launch. template<typename Launcher> error_code on_setup(Launcher & launcher, const filesystem::path &executable, std::wstring &cmd_line); // called for every initializer if an error occurred during setup or process creation template<typename Launcher> void on_error(Launcher & launcher, const filesystem::path &executable, std::wstring &cmd_line, const error_code & ec); // called after successful process creation template<typename Launcher> void on_success(Launcher & launcher, const filesystem::path &executable, std::wstring &cmd_line); };
Note | |
---|---|
All the additional launchers for windows inherit |
The call sequence is as follows:
The easier initializer to use is process_start_dir
:
asio::io_context ctx; process ls(ctx, "/ls", {}, process_start_dir("/home")); ls.wait();
This will run ls
in the folder
/home
instead of the current folder.
Warning | |
---|---|
If your path is relative, it may fail on posix, because the directory is changed before a call to execve. |
When using io with a subprocess, all three standard streams (stdin, stdout, stderr) get set for the child-process. The default setting is to inherit the parent process.
This feature meant to be flexible, which is why there is little checking on the arguments assigned to one of those streams.
asio pipes can be used for io. When using in process_stdio they will get automatically connected and the other side will get assigned to the child process:
asio::io_context ctx; asio::readable_pipe rp; process proc(ctx, "/usr/bin/g++", {"--version"}, process_stdio{{ /* in to default */}, rp, { /* err to default */ }}); std::string output; system::error_code ec; rp.read(asio::dynamic_buffer(output), ec); assert(ec == asio::eof); proc.wait();
readable pipes can be assigned to out
an err
, while writable_pipes
can be assigned to in
.
FILE*
can also be used for either side; this allows the stdin
,
stderr
, stdout
macros to be used:
asio::io_context ctx; // forward both stderr & stdout to stdout of the parent process process proc(ctx, "/usr/bin/g++", {"--version"}, process_stdio{{ /* in to default */}, stdout, stdout}); proc.wait();
nullptr
may be used to set
a given stream to be opened on the null-device (/dev/null
on posix, NUL
on windows). This is used to ignore output or give only EOF as input.
asio::io_context ctx; // ignore stderr process proc(ctx, "/usr/bin/g++", {"--version"}, process_stdio{{ /* in to default */}, {}, nullptr}); proc.wait();
A native handle can be used as well, which means an int
on posix or a HANDLE
on
windows. Furthermore, any object that has a native_handle
returning that native handle type is valid, too.
asio::io_context ctx; // ignore stderr asio::ip::tcp::socket sock{ctx}; connect_my_socket(sock); process proc(ctx, "~/not-a-virus", {}, process_stdio{sock, sock, nullptr}); proc.wait();
Additionally, process v2 provides a popen
class. It starts a process and connects pipes for stdin and stdout, so
that the popen object can be used as a stream.
popen proc(executor, "/usr/bin/addr2line, {argv[0]}); asio::write(proc, asio::buffer("main\n")); std::string line; asio::read_until(proc, asio::dynamic_buffer(line), '\n');
The environment
namespace
provides all sorts of facilities to query and manipulate the environment
of the current process.
The api should be straight forward, but one oddity that needs to be pointed out is, that environment names are not case sensitive on windows. The key_traits class implements the proper traits depending on the current system.
Additionally, environment can be lists separated by :
or ;
; environment::value
and environment::value_view
can be used to iterate those.
Beyond that, the requirements on an environment are a low as possible; an environment is either a list of strings or a list of string-pairs. It is however recommended to use the environment types, as to have the right value comparisons.
To note is the find_executable
functions, which searches in an environment for an executable.
// search in the current environment auto exe = environment::find_executable("g++"); std::unordered_map<environment::key, environment::value> my_env = { {"SECRET", "THIS_IS_A_TEST"} {"PATH", {"/bin", "/usr/bin"} }; auto other_exe = environment::find_executable("g++", my_env);
The subprocess environment assignment follows the same constraints:
asio::io_context ctx; std::unordered_map<environment::key, environment::value> my_env = { {"SECRET", "THIS_IS_A_TEST"} {"PATH", {"/bin", "/usr/bin"} }; auto exe = find_executable("g++"), my_env); process proc(ctx, exe, {"main.cpp"}, process_environment(my_env)); process pro2(ctx, exe, {"test.cpp"}, process_environment(my_env));
namespace boost { namespace process { namespace v2 { template<typename Launcher, typename ... Init> struct bound_launcher; template<typename Launcher, typename ... Init> auto bind_launcher(Launcher && launcher, Init &&... init); template<typename ... Init> auto bind_default_launcher(Init &&... init); } } }
namespace boost { namespace process { namespace v2 { template<typename CharT, typename Traits = std::char_traits<CharT> > struct basic_cstring_ref; typedef basic_cstring_ref< char > cstring_ref; typedef basic_cstring_ref< wchar_t > wcstring_ref; typedef basic_cstring_ref< char16_t > u16cstring_ref; typedef basic_cstring_ref< char32_t > u32cstring_ref; template<typename charT, typename traits> std::basic_ostream< charT, traits > & operator<<(std::basic_ostream< charT, traits > & os, const basic_cstring_ref< charT, traits > & str); template<typename charT, typename traits> std::size_t hash_value(basic_string_view< charT, traits > s); } } }
namespace boost { namespace process { namespace v2 { typedef implementation_defined default_process_launcher; } } }
namespace boost { namespace process { namespace v2 { struct process_environment; namespace environment { struct current_view; struct key; struct key_value_pair; struct key_value_pair_view; struct key_view; struct value; struct value_iterator; struct value_view; typedef implementation_defined key_char_traits; typedef implementation_defined value_char_traits; typedef implementation_defined char_type; // The character type used by the environment. Eitherchar
orwchar_t
. typedef implementation_defined native_handle; // The native handle of an environment. Note that this can be an owning pointer and is generally not thread safe. constexpr char_type equality_sign; // The equal character in an environment string used to separate key and value. constexpr char_type delimiter; // The delimiter in environemtn lists. Commonly used by thePATH
variable. std::size_t hash_value(const key_view & value); std::size_t hash_value(const boost::process::v2::environment::value_view & value); std::size_t hash_value(const boost::process::v2::environment::key_value_pair_view & value); bool operator==(const value_view &, const value_view); bool operator!=(const value_view &, const value_view); bool operator<=(const value_view &, const value_view); bool operator<(const value_view &, const value_view); bool operator>(const value_view &, const value_view); bool operator>=(const value_view &, const value_view); bool operator==(const key_value_pair_view &, const key_value_pair_view); bool operator!=(const key_value_pair_view &, const key_value_pair_view); bool operator<=(const key_value_pair_view &, const key_value_pair_view); bool operator<(const key_value_pair_view &, const key_value_pair_view); bool operator>(const key_value_pair_view &, const key_value_pair_view); bool operator>=(const key_value_pair_view &, const key_value_pair_view); // Obtain a handle to the current environment. current_view current(); template<typename Environment = current_view> filesystem::path home(Environment && = current()); template<typename Environment = current_view> boost::process::v2::filesystem::path find_executable(boost::process::v2::filesystem::path, Environment && = current()); // Get an environment variable from the current process. value get(const key & k, error_code & ec); value get(const key & k); value get(basic_cstring_ref< char_type, key_char_traits< char_type > > k, error_code & ec); value get(basic_cstring_ref< char_type, key_char_traits< char_type > > k); value get(const char_type * c, error_code & ec); value get(const char_type * c); void set(const key &, value_view, error_code &); void set(const key & k, value_view vw); void set(basic_cstring_ref< char_type, key_char_traits< char_type > > k, value_view vw, error_code & ec); void set(basic_cstring_ref< char_type, key_char_traits< char_type > > k, value_view vw); void set(const char_type * k, value_view vw, error_code & ec); void set(const char_type * k, value_view vw); template<typename Char, typename = typename std::enable_if<!std::is_same<Char, char_type>::value>::type> void set(const key & k, const Char * vw, error_code & ec); template<typename Char, typename = typename std::enable_if<!std::is_same<Char, char_type>::value>::type> void set(const key & k, const Char * vw); template<typename Char, typename = typename std::enable_if<!std::is_same<Char, char_type>::value>::type> void set(basic_cstring_ref< char_type, key_char_traits< char_type > > k, const Char * vw, error_code & ec); template<typename Char, typename = typename std::enable_if<!std::is_same<Char, char_type>::value>::type> void set(basic_cstring_ref< char_type, key_char_traits< char_type > > k, const Char * vw); template<typename Char, typename = typename std::enable_if<!std::is_same<Char, char_type>::value>::type> void set(const char_type * k, const Char * vw, error_code & ec); template<typename Char, typename = typename std::enable_if<!std::is_same<Char, char_type>::value>::type> void set(const char_type * k, const Char * vw); void unset(const key &, error_code &); void unset(const key & k); void unset(basic_cstring_ref< char_type, key_char_traits< char_type > > k, error_code & ec); void unset(basic_cstring_ref< char_type, key_char_traits< char_type > > k); void unset(const char_type * c, error_code & ec); void unset(const char_type * c); } namespace posix { } } } }namespace std { template<> struct hash<boost::process::v2::environment::key>; template<> struct hash<boost::process::v2::environment::key_value_pair>; template<> struct hash<boost::process::v2::environment::key_value_pair_view>; template<> struct hash<boost::process::v2::environment::key_view>; template<> struct hash<boost::process::v2::environment::value>; template<> struct hash<boost::process::v2::environment::value_view>; template<> class tuple_element<0u, boost::process::v2::environment::key_value_pair>; template<> class tuple_element<0u, boost::process::v2::environment::key_value_pair_view>; template<> class tuple_element<1u, boost::process::v2::environment::key_value_pair>; template<> class tuple_element<1u, boost::process::v2::environment::key_value_pair_view>; template<> class tuple_size<boost::process::v2::environment::key_value_pair>; template<> class tuple_size<boost::process::v2::environment::key_value_pair_view>; template<std::size_t Idx> auto get(const boost::process::v2::environment::key_value_pair & kvp); template<std::size_t Idx> auto get(boost::process::v2::environment::key_value_pair_view kvp); }
namespace boost { namespace process { namespace v2 { namespace error { // Errors used for utf8 <-> UCS-2 conversions. enum utf8_conv_error { insufficient_buffer = = 1, invalid_character }; static const error_category & utf8_category; static const error_category & exit_code_category; // An error category that can be used to interpret exit codes of subprocesses. const error_category & get_utf8_category(); const error_category & get_exit_code_category(); } } } }
namespace boost { namespace process { namespace v2 { template<typename Executor> int execute(basic_process< Executor >); template<typename Executor> int execute(basic_process< Executor > proc, error_code & ec); template<typename Executor = boost::asio::any_io_executor, Token WaitHandler DEFAULT_TYPE> deduced async_execute(basic_process< Executor >, WaitHandler &&handler ); } } }
namespace boost { namespace process { namespace v2 { typedef implementation_defined native_exit_code_type; error_code check_exit_code(error_code &, native_exit_code_type, const error_category & = error::get_exit_code_category()); // Check if the native exit code indicates the process is still running. bool process_is_running(native_exit_code_type code); // Obtain the portable part of the exit code, i.e. what the subprocess has returned from main. int evaluate_exit_code(native_exit_code_type code); } } }
namespace boost { namespace process { namespace v2 { typedef implementation_defined pid_type; // Get the process id of the current process. pid_type current_pid(); // List all available pids. std::vector< pid_type > all_pids(boost::system::error_code & ec); // List all available pids. std::vector< pid_type > all_pids(); pid_type parent_pid(pid_type pid, boost::system::error_code & ec); pid_type parent_pid(pid_type pid); std::vector< pid_type > child_pids(pid_type pid, boost::system::error_code & ec); std::vector< pid_type > child_pids(pid_type pid); } } }
namespace boost { namespace process { namespace v2 { template<typename Executor = boost::asio::any_io_executor> struct basic_popen; typedef basic_popen<> popen; // A popen object with the default executor. } } }
namespace boost { namespace process { namespace v2 { template<typename Executor = boost::asio::any_io_executor> struct basic_process; typedef basic_process process; // Process with the default executor. } } }
namespace boost { namespace process { namespace v2 { template<typename Executor = boost::asio::any_io_executor> struct basic_process_handle; } } }
namespace boost { namespace process { namespace v2 { struct shell; static const error_category & shell_category; // Error category used by the shell parser. const error_category & get_shell_category(); } } }
namespace boost { namespace process { namespace v2 { struct process_start_dir; } } }
namespace boost { namespace process { namespace v2 { struct process_stdio; } } }