Boost C++ Libraries

In process v1 one can define partial settings in the constructor of the process, which has lead to a small DSL.

child c{exe="test", args+="--help", std_in < null(), env["FOO"] += "BAR"};

While this looks fancy at first, it really does not scale well with more parameters. For process v2, the interfaces is simple:

extern std::unordered_map<std::string, std::string> my_env;
extern asio::io_context ctx;
process proc(ctx, "./test", {"--help"}, process_io{nullptr, {}, {}}, process_environment(my_env));

Every initializer adresses one logical compoent (e.g. stdio) instead of multiple ones accumulating. Furthermore, every process has a path and arguments, instead of a confusing mixture of cmd-style and exe-args that can be randomly spread out.

Since process v1 came out, linux has moved along and added pidfd_open which allows users to get a file descriptor for a process. This is much more reliable since it is not as easy to miss as a SIGCHLD. FreeBSD has a similar feature with pdfork which is also supported, while windows has provided HANDLEs for processes all along. Unless the OS doesn't support it, process v2 will use file descriptors and handles to implement waiting for processes.

Process v1 aimed to make asio optional, but synchronous IO with subprocesses usually means one is begging for deadlocks. Since asio added pipes in boost 1.78, boost process V2 is fully asio based and uses it's pipes and file-handles for the subprocess.

Certain parts of boost.process were not as reliable as they should've been.

This concerns especially the wait_for and wait_until` functions on the process. The latter are easy to do on windows, but posix does not provide an API for this. Thus the wait_for used signals or fork, which was all but safe. Since process v2 is based on asio and thus supports cancellation, a wait_for can not safely be implemented with an async_wait + timeout.

“UTF-8 or GTFO”--Vinnie Falco

Instead of using ascii-APIs on windows, process V2 just assumes UTF-8 everywhere.

Boost.process v2 supports separate compilation similar to other boost libraries. It can be achieved by defining BOOST_PROCESS_V2_SEPARATE_COMPILATION and including <process/v2/src.hpp> in a single compile unit.

While not a problem on windows (since HANDLEs get manually enabled for inheritance), posix systems create a problem with inheriting file handles by default.

Process V2 will automatically close all non-whitelisted descriptors, without needing any option to enable it.

If the process handle goes out of scope, it will terminate the subprocess. You can prevent this, by calling proc.detach(); do however note that this can lead to zombie processes.

A process that completed will deliver an exit-code, which can be obtained by calling .exit_code on the exited process and which is also returned from .wait().

process proc("/bin/ls", {});
assert(proc.wait() == 0);

The normal exit-code is what the subprocess returned from main; posix will however add addtional information about the process. This is called the native_exit_code.

The .running() function can be used to detect if the process is still active.

The parent process can signal the subprocess demaning certain actions.

.terminate will cause the subprocess to exit immediately (SIGKILL on posix). This is the only reliable & portable way to end a subprocess.

process proc("/bin/totally-not-a-virus", {});
proc.terminate();

.request_exit will ask the subprocess to shutdown (SIGTERM on posix), which the subprocess might ignore.

process proc("/bin/bash", {});
proc.request_exit();
proc.wait();

.interrupt will send an SIGINT to the subprocess, which a subprocess might interpret as a signal to shutdown.

	Warning
	interrupt requires the initializer windows::create_new_process_group to be set

process proc("/usr/bin/addr2line", {});
proc.request_exit();
proc.wait();

Process v2 provides execute and async_execute functons that can be used for managed executions.

assert(execute(process("/bin/ls", {}) == 0));

The async version supports cancellation and will forward cancellation types as follows:

- asio::cancellation_type::total -> interrupt - asio::cancellation_type::partial -> request_exit - asio::cancellation_type::terminal -> terminate

asio::io_context ctx;
asio::steady_timer timout{ctx, std::chrono::seconds(10)};

asio::cancellation_signal sig;
async_execute(process("/usr/bin/g++", {"hello_world.cpp"}),
             asio::bind_cancellation_slot(sig.slot(),
                                          [&](error_code ec, int exit_code)
                                          {
                                              timeout.cancel(); // we're done earlier
                                          }));

timeout.async_wait(
    [&](error_code ec)
    {
        if (ec) // we were cancelled, do nothing
            return ;
        sig.emit(asio::cancellation_type::partial);
        // request exit first, but terminate after another 10 sec
        timeout.expires_after(std::chrono::seconds(10));
        timeout.async_wait(
            [&](error_code ec)
            {
                if (!ec)
                    sig.emit(asio::cancellation_type::terminal);
            });
        );
    });

The default and pdfork launchers on linux open an internal pipe to communicate errors that occur after forking back to the parent process.

This can be prevented by using the fork_and_forget_launcher. Alternatively, the vfork_launcher can report errors directly back to the parent process.

Thus some calls to the initializers occur after forking from the child process.

struct custom_initalizer
{
    // functions called from the parent process:


    // called before a call to fork. A returned error will cancel the launch.
    template<typename Launcher>
    error_code on_setup(Launcher & launcher, const filesystem::path &executable, const char * const * (&cmd_line));

    // called for every initializer if an error occured during setup or process creation
    template<typename Launcher>
    void on_error(Launcher & launcher, const filesystem::path &executable, const char * const * (&cmd_line),
                  const error_code & ec);

    // called after successful process creation
    template<typename Launcher>
    void on_success(Launcher & launcher, const filesystem::path &executable, const char * const * (&cmd_line));

    // called for every initializer if an error occured when forking, in addtion to on_error.
    template<typename Launcher>
    void on_fork_error(Launcher & launcher, const filesystem::path &executable, const char * const * (&cmd_line),
                       const error_code & ec);


    // called before a call to execve. A returned error will cancel the launch. Called from the child process.
    template<typename Launcher>
    error_code on_exec_setup(Launcher & launcher, const filesystem::path &executable, const char * const * (&cmd_line));


    // called after a failed call to execve from the child process.
    template<typename Launcher>
    void on_exec_error(Launcher & launcher, const filesystem::path &executable, const char * const * (&cmd_line));
};

The call sequence on success:

The call sequence when fork fails:

The call sequence when exec fails:

The launcher will close all non-whitelisted file descriptors after on_exec_setup.

Windows launchers are pretty streight forward, they will call the following functions on the initializer if present.

struct custom_initializer
{
    // called before a call to CreateProcess. A returned error will cancel the launch.
    template<typename Launcher>
    error_code on_setup(Launcher & launcher, const filesystem::path &executable, std::wstring &cmd_line);

    // called for every initializer if an error occured during setup or process creation
    template<typename Launcher>
    void on_error(Launcher & launcher, const filesystem::path &executable, std::wstring &cmd_line,
                  const error_code & ec);

    // called after successful process creation
    template<typename Launcher>
    void on_success(Launcher & launcher, const filesystem::path &executable, std::wstring &cmd_line);

};

	Note
	All the additional launchers for windows inherit default_launcher

The call sequence is as follows:

asio pipes can be used for io. When using in process_stdio they will get automatically connected and the other side will get assigned to the child process:

asio::io_context ctx;
asio::readable_pipe rp;

process proc(ctx, "/usr/bin/g++", {"--version"}, process_stdio{{ /* in to default */}, rp, { /* err to default */ }});
std::string output;

system::error_code ec;
rp.read(asio::dynamic_buffer(output), ec);
assert(ec == asio::eof);
proc.wait();

readable pipes can be assigned to out an err, while writable_pipes can be assigned to in`.

FILE* can also be used for either side; this allows the stdin, stderr, stdout macros to be used:

asio::io_context ctx;
// forward both stderr & stdout to stdout of the parent process
process proc(ctx, "/usr/bin/g++", {"--version"}, process_stdio{{ /* in to default */}, stdout, stdout});
proc.wait();

nullptr may be used to set a given stream to be opened on the null-device (/dev/null on posix, NUL on windows). This is used to ignore output or give only EOF as input.

asio::io_context ctx;
// ignore stderr
process proc(ctx, "/usr/bin/g++", {"--version"}, process_stdio{{ /* in to default */}, {}, nullptr});
proc.wait();

A native handle can be used as well, which means an int on posix or a HANDLE on windows. Furthermore, any object that has a native_handle returning that native handle type is valid, too.

asio::io_context ctx;
// ignore stderr
asio::ip::tcp::socket sock{ctx};
connect_my_socket(sock);
process proc(ctx, "~/not-a-virus", {}, process_stdio{sock, sock, nullptr});
proc.wait();

Additionally, process v2 provides a popen class. It starts a process and connects pipes for stdin and stdout, so that the popen object can be used as a stream.

popen proc(executor, "/usr/bin/addr2line, {argv[0]});
asio::write(proc, asio::buffer("main\n"));
std::string line;
asio::read_until(proc, asio::dynamic_buffer(line), '\n');

The subprocess environment assignment follows the same constraints:

asio::io_context ctx;
std::unordered_map<environment::key, environment::value> my_env =
    {
        {"SECRET", "THIS_IS_A_TEST"}
        {"PATH", {"/bin", "/usr/bin"}
    };
auto exe = find_executable("g++"), my_env);
process proc(ctx, exe, {"main.cpp"}, process_environment(my_env));
process pro2(ctx, exe, {"test.cpp"}, process_environment(my_env));

namespace boost {
  namespace process {
    namespace v2 {
      template<typename CharT, typename Traits = std::char_traits<CharT> > 
        struct basic_cstring_ref;

      typedef basic_cstring_ref< char > cstring_ref;
      typedef basic_cstring_ref< wchar_t > wcstring_ref;
      typedef basic_cstring_ref< char16_t > u16cstring_ref;
      typedef basic_cstring_ref< char32_t > u32cstring_ref;
      template<typename charT, typename traits> 
        std::basic_ostream< charT, traits > & 
        operator<<(std::basic_ostream< charT, traits > & os, 
                   const basic_cstring_ref< charT, traits > & str);
      template<typename charT, typename traits> 
        std::size_t hash_value(basic_string_view< charT, traits > s);
    }
  }
}

namespace boost {
  namespace process {
    namespace v2 {
      typedef implementation_defined default_process_launcher;
    }
  }
}

namespace boost {
  namespace process {
    namespace v2 {
      struct process_environment;
      namespace environment {
        struct current_view;
        struct key;
        struct key_value_pair;
        struct key_value_pair_view;
        struct key_view;
        struct value;
        struct value_iterator;
        struct value_view;
        typedef implementation_defined key_char_traits;
        typedef implementation_defined value_char_traits;
        typedef implementation_defined char_type;  // The character type used by the environment. Either char or wchar_t. 
        typedef implementation_defined native_handle;  // The native handle of an environment. Note that this can be an owning pointer and is generally not thread safe. 

        constexpr char_type equality_sign;        // The equal character in an environment string used to separate key and value. 
        constexpr char_type delimiter;        // The delimiter in environemtn lists. Commonly used by the PATH variable. 
        std::size_t hash_value(const key_view & value);
        std::size_t hash_value(const boost::process::v2::environment::value_view & value);
        std::size_t hash_value(const boost::process::v2::environment::key_value_pair_view & value);
        bool operator==(const value_view &, const value_view);
        bool operator!=(const value_view &, const value_view);
        bool operator<=(const value_view &, const value_view);
        bool operator<(const value_view &, const value_view);
        bool operator>(const value_view &, const value_view);
        bool operator>=(const value_view &, const value_view);
        bool operator==(const key_value_pair_view &, 
                        const key_value_pair_view);
        bool operator!=(const key_value_pair_view &, 
                        const key_value_pair_view);
        bool operator<=(const key_value_pair_view &, 
                        const key_value_pair_view);
        bool operator<(const key_value_pair_view &, const key_value_pair_view);
        bool operator>(const key_value_pair_view &, const key_value_pair_view);
        bool operator>=(const key_value_pair_view &, 
                        const key_value_pair_view);

        // Obtain a handle to the current environment. 
        current_view current();
        template<typename Environment = current_view> 
          filesystem::path home(Environment && = current());
        template<typename Environment = current_view> 
          boost::process::v2::filesystem::path 
          find_executable(boost::process::v2::filesystem::path, 
                          Environment && = current());
        // <purpose>Get an environment variable from the current process. </purpose>
        value get(const key & k, error_code & ec);
        value get(const key & k);
        value get(basic_cstring_ref< char_type, key_char_traits< char_type >> k, 
                  error_code & ec);
        value get(basic_cstring_ref< char_type, key_char_traits< char_type >> k);
        value get(const char_type * c, error_code & ec);
        value get(const char_type * c);
        // <purpose>Set an environment variable for the current process. </purpose>
        void set(const key & k, value_view vw, error_code & ec);
        void set(const key & k, value_view vw);
        void set(basic_cstring_ref< char_type, key_char_traits< char_type >> k, 
                 value_view vw, error_code & ec);
        void set(basic_cstring_ref< char_type, key_char_traits< char_type >> k, 
                 value_view vw);
        void set(const char_type * k, value_view vw, error_code & ec);
        void set(const char_type * k, value_view vw);
        template<typename Char, 
                 typename  = typename std::enable_if<!std::is_same<Char, char_type>::value>::type> 
          void set(const key & k, const Char * vw, error_code & ec);
        template<typename Char, 
                 typename  = typename std::enable_if<!std::is_same<Char, char_type>::value>::type> 
          void set(const key & k, const Char * vw);
        template<typename Char, 
                 typename  = typename std::enable_if<!std::is_same<Char, char_type>::value>::type> 
          void set(basic_cstring_ref< char_type, key_char_traits< char_type >> k, 
                   const Char * vw, error_code & ec);
        template<typename Char, 
                 typename  = typename std::enable_if<!std::is_same<Char, char_type>::value>::type> 
          void set(basic_cstring_ref< char_type, key_char_traits< char_type >> k, 
                   const Char * vw);
        template<typename Char, 
                 typename  = typename std::enable_if<!std::is_same<Char, char_type>::value>::type> 
          void set(const char_type * k, const Char * vw, error_code & ec);
        template<typename Char, 
                 typename  = typename std::enable_if<!std::is_same<Char, char_type>::value>::type> 
          void set(const char_type * k, const Char * vw);
        // <purpose>Remove an environment variable from the current process. </purpose>
        void unset(const key & k, error_code & ec);
        void unset(const key & k);
        void unset(basic_cstring_ref< char_type, key_char_traits< char_type >> k, 
                   error_code & ec);
        void unset(basic_cstring_ref< char_type, key_char_traits< char_type >> k);
        void unset(const char_type * c, error_code & ec);
        void unset(const char_type * c);
      }
      namespace posix {
      }
    }
  }
}namespace std {
  template<> struct hash<boost::process::v2::environment::key>;
  template<> struct hash<boost::process::v2::environment::key_value_pair>;
  template<> 
    struct hash<boost::process::v2::environment::key_value_pair_view>;
  template<> struct hash<boost::process::v2::environment::key_view>;
  template<> struct hash<boost::process::v2::environment::value>;
  template<> struct hash<boost::process::v2::environment::value_view>;

  template<> 
    class tuple_element<0u, boost::process::v2::environment::key_value_pair>;
  template<> 
    class tuple_element<0u, boost::process::v2::environment::key_value_pair_view>;
  template<> 
    class tuple_element<1u, boost::process::v2::environment::key_value_pair>;
  template<> 
    class tuple_element<1u, boost::process::v2::environment::key_value_pair_view>;
  template<> 
    class tuple_size<boost::process::v2::environment::key_value_pair>;
  template<> 
    class tuple_size<boost::process::v2::environment::key_value_pair_view>;
  template<std::size_t Idx> 
    auto get(const boost::process::v2::environment::key_value_pair & kvp);
  template<std::size_t Idx> 
    auto get(boost::process::v2::environment::key_value_pair_view kvp);
}

namespace boost {
  namespace process {
    namespace v2 {
      namespace error {

        // Errors used for utf8 <-> UCS-2 conversions. 
        enum utf8_conv_error { insufficient_buffer = = 1, invalid_character };

        static const error_category & utf8_category;
        static const error_category & exit_code_category;        // An error category that can be used to interpret exit codes of subprocesses. 
        const error_category & get_utf8_category();
        const error_category & get_exit_code_category();
      }
    }
  }
}

namespace boost {
  namespace process {
    namespace v2 {
      template<typename Executor> int execute(basic_process< Executor >);
      template<typename Executor> 
        int execute(basic_process< Executor >, error_code &);
      template<typename Executor = boost::asio::any_io_executor, 
               Token WaitHandler DEFAULT_TYPE> 
         async_execute(basic_process< Executor >, WaitHandler &&handler );
    }
  }
}

namespace boost {
  namespace process {
    namespace v2 {
      typedef implementation_defined native_exit_code_type;

      // Check if the native exit code indicates the process is still running. 
      bool process_is_running(native_exit_code_type code);

      // Obtain the portable part of the exit code, i.e. what the subprocess has returned from main. 
      int evaluate_exit_code(native_exit_code_type code);
    }
  }
}

namespace boost {
  namespace process {
    namespace v2 {
      typedef implementation_defined pid_type;

      // Get the process id of the current process. 
      pid_type current_pid();
    }
  }
}

namespace boost {
  namespace process {
    namespace v2 {
      template<typename Executor = boost::asio::any_io_executor> 
        struct basic_popen;

      typedef basic_popen<> popen;  // A popen object with the default executor. 
    }
  }
}

namespace boost {
  namespace process {
    namespace v2 {
      template<typename Executor = boost::asio::any_io_executor> 
        struct basic_process;

      typedef basic_process process;  // Process with the default executor. 
    }
  }
}

namespace boost {
  namespace process {
    namespace v2 {
      template<typename Executor = boost::asio::any_io_executor> 
        struct basic_process_handle;
    }
  }
}

BOOST_PROCESS_V2_SOURCE

namespace boost {
  namespace process {
    namespace v2 {
      struct process_start_dir;
    }
  }
}

namespace boost {
  namespace process {
    namespace v2 {
      struct process_stdio;
    }
  }
}