klemens-morgenstern / boost-process Goto Github PK
View Code? Open in Web Editor NEWBoost.Process is a library to manage system processes
Boost.Process is a library to manage system processes
I came to this question because of the complications using fork
in a multi-threaded application.
How well does Boost Process accomodate multi threaded programs?
My understanding is that (on POSIX) Boost Process employs classical [v]fork; exec[vpe]
patterns. These are fraught with limitations, and I'm very happy to use a library that encapsulate the pitfalls for me.
Some issues seem to suggest that thread safety has been a concern in designing some of the building blocks for Boost Process:
(from #2; issue #5 also mentions thread-safety (of boost::asio::streambuf
)).
In just how far are multi threaded applications really facilitated?
pthread_atfork
Under macOS the default shell environment path does not seem to be searched and used when launching a new process. Using the c++ standard library call std::system()
works as expected but when using bp::system
it yields the following error:
libc++abi.dylib: terminating with uncaught exception of type boost::process::process_error: execve failed: No such file or directory
The error is consistent on a clean install (virtual machine) running MacOS 10.12.3 with clang version = Apple LLVM version 8.0.0 (clang-800.0.42.1)
The error also persists on macOS 10.10.0 as well.
See example failing code below:
#include <iostream>
#include <boost/process.hpp>
using namespace boost::process;
int main() {
namespace bp = boost::process;
// This fails since 'ls' executable is not found
int result = bp::system("ls -al");
std::cout << "Result of operation is " << result << std::endl;
return 0;
}
This code on the contrary does work using the standard library:
#include <iostream>
#include <boost/process.hpp>
using namespace boost::process;
int main() {
namespace bp = boost::process;
// This works
int result = std::system("ls -al");
std::cout << "Result of operation is " << result << std::endl;
return 0;
}
This also works since we are using an absolute path to the 'ls' process
#include <iostream>
#include <boost/process.hpp>
using namespace boost::process;
int main() {
namespace bp = boost::process;
// This works
int result = std::system("/bin/ls -al");
std::cout << "Result of operation is " << result << std::endl;
return 0;
}
It is also possible to see that processes themselves do have access to the OS environment. We can define and set environment variables with no problem but unfortunately I have to define the absolute path to the 'env' binary here:
#include <iostream>
#include <boost/process.hpp>
using namespace boost::process;
int main() {
namespace bp = boost::process;
// Get a handle to the current environment
auto env = boost::this_process::environment();
// Add a variable to the current environment
env["VALUE_1"] = "foo";
// Copy it into a environment separate to the one of this process
bp::environment env_ = env;
// Display the newly set environment
int result = bp::system("/usr/bin/env", env_);
std::cout << "Result of operation is " << result << std::endl;
return 0;
}
This yields the following output on my macOS VM:
TERM_PROGRAM=Apple_Terminal
TERM=xterm-256color
SHELL=/bin/bash
TMPDIR=/var/folders/cs/lk7v9cjs2g58m_hs35n0dx4c0000gn/T/
Apple_PubSub_Socket_Render=/private/tmp/com.apple.launchd.ej5DhWmXxg/Render
TERM_PROGRAM_VERSION=388
TERM_SESSION_ID=DCDD1EFC-0B64-493A-8583-6A32EB91FEDF
USER=someuser
SSH_AUTH_SOCK=/private/tmp/com.apple.launchd.RpzJNLOc63/Listeners
__CF_USER_TEXT_ENCODING=0x1F5:0x0:0x0
PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
PWD=/Users/someuser/Development/cpp/bprocess
LANG=en_US.UTF-8
XPC_FLAGS=0x0
XPC_SERVICE_NAME=0
HOME=/Users/someuser
SHLVL=1
LOGNAME=someuser
_=./bprocess
OLDPWD=/Users/someuser/Development/cpp
VALUE_1=foo
Result of operation is 0
The way to determine if a named pipe does exist is not implemented right on windows.
Currently an internal counter increments, but I did not find any way to reliable check if a pipe is used.
That needs to be fixed, especially since several programs might use named pipes and may then clash, since those are system-wide.
Is C++11 required? I was seeing errors which suggested I needed to compiler set to c++11
Including the boost/process.hpp header results in the following error:
In file included from bprocess.cpp:2:
In file included from /usr/local/include/boost/process.hpp:24:
In file included from /usr/local/include/boost/process/async_system.hpp:22:
In file included from /usr/local/include/boost/process/child.hpp:22:
In file included from /usr/local/include/boost/process/detail/execute_impl.hpp:24:
/usr/local/include/boost/process/detail/posix/executor.hpp:46:17: error: assigning to 'char **' from incompatible type
'::boost::process::environment' (aka 'basic_environment<char>')
env._env_impl = env;
^ ~~~
/usr/local/include/boost/process/detail/posix/executor.hpp:53:24: error: no member named 'search_path' in namespace 'boost::process'
fn = boost::process::search_path(fn, val);
~~~~~~~~~~~~~~~~^
2 errors generated.
Here is the example failing code
#include <iostream>
#include <boost/process.hpp>
int main() {
return 0;
}
I wrote a few examples and checked on macOS 10.12.2 (clang-800.0.42.1) and Windows 10 (msvc 2017)
#include <boost/process.hpp>
#include <iostream>
#include <chrono>
#include <map>
namespace bp = boost::process;
int main()
{
std::map<int, bp::child> q;
bp::group g;
for (int i = 1; i <= 5; ++i)
{
#ifdef _WIN32
std::string exe = "powershell";
std::vector<std::string> args =
{
"-command",
"Start-Sleep",
"-s",
std::to_string(i),
";",
"echo",
std::to_string(i)
};
#else
std::string exe = "sleep";
std::vector<std::string> args =
{
std::to_string(i),
";",
"echo",
std::to_string(i)
};
#endif
q[i] = bp::child(
bp::exe = exe,
bp::args = args,
bp::shell,
g,
bp::std_out > stdout);
}
while (!q.empty())
{
g.wait();
for (auto it = q.begin(); it != q.end(); )
{
bp::child& c = it->second;
if (!c.running())
{
std::cout << "exit " << it->first << std::endl;
it = q.erase(it);
}
else
{
++it;
}
}
}
}
This code works on macOS, but on Windows hangs on the line g.wait();
I rewrite this code without use bp::group
#include <boost/process.hpp>
#include <iostream>
#include <chrono>
#include <map>
namespace bp = boost::process;
int main()
{
std::map<int, bp::child> q;
for (int i = 1; i <= 5; ++i)
{
#ifdef _WIN32
std::string exe = "powershell";
std::vector<std::string> args =
{
"-command",
"Start-Sleep",
"-s",
std::to_string(i),
";",
"echo",
std::to_string(i)
};
#else
std::string exe = "sleep";
std::vector<std::string> args =
{
std::to_string(i),
";",
"echo",
std::to_string(i)
};
#endif
q[i] = bp::child(
bp::exe = exe,
bp::args = args,
bp::shell,
bp::std_out > stdout);
}
while (!q.empty())
{
for (auto it = q.begin(); it != q.end(); )
{
bp::child& c = it->second;
std::error_code ec;
c.wait_for(std::chrono::milliseconds(100), ec);
if (!c.running(ec))
{
std::cout << "exit " << it->first << std::endl;
it = q.erase(it);
}
else
{
++it;
}
}
std::this_thread::sleep_for(std::chrono::milliseconds(100));
}
}
Now, i have a problem on macOS. I get following output:
exit 1
1
exit 2
exit 3
exit 4
2
exit 5
4
5
If I removec.wait_for(std::chrono::milliseconds(100), ec);
I will get the correct output:
1
exit 1
2
exit 2
3
exit 3
4
exit 4
5
exit 5
I think the problem is that _exit_status->store(exit_code)
in wait_for
it is called if boost::process::detail::api::wait_for
ended with an error
In the documentation tutorial page, error code section it mentions:
As with all other functions
What other functions? Standard library (clearly not all of those)? Others in boost process (all of those?)? How about "In a manner similar to XXX() and YYY(),", or just remove that phrase.
Hi,
When using boost::process::child in a posix environment and with a boost::asio::io_service, the retrieved exit status (child::exit_code()) will always be zero even if it is actually non-zero (in case of failure for example).
This can easily be verified with strace, looking for the wait* syscalls.
For example the following code should print "exit code: 1", but the result is "exit code: 0" :
asio::io_service ios;
std::future<std::string> data;
bp::child c("/bin/false",
bp::std_in.close(),
bp::std_out > bp::null,
bp::std_err > data,
ios);
ios.run();
// std::cout << data.get() << std::endl;
std::cout << "exit code: " << c.exit_code();
I think the problem in this case, is that WEXITSTATUS() is called twice on the exit status returned by the system. The first time in detail/posix/io_service_ref.hpp and the second time in detail/child_decl.hpp by eval_exit_status().
Hence, the value is always zero after the second time.
Here is a patch that seems to fix this issue :
diff -ru tmp/include/boost/process/detail/posix/io_service_ref.hpp boost-process-boost_review/include/boost/process/detail/posix/io_service_ref.hpp
--- tmp/include/boost/process/detail/posix/io_service_ref.hpp 2016-10-26 14:11:25.000000000 +0200
+++ boost-process-boost_review/include/boost/process/detail/posix/io_service_ref.hpp 2017-02-21 15:09:57.343266433 +0100
@@ -120,9 +120,9 @@
::wait(&status);
std::error_code ec(ec_in.value(), std::system_category());
- int val = WEXITSTATUS(status);
- exit_status->store(val);
+ exit_status->store(status);
+ int val = WEXITSTATUS(status);
for (auto & func : funcs)
func(val, ec);
}
-- Thank you for your work.
If the process creation fails both the process and asio libraries attempt to close the the async handles when they are destructed. The second attempts fail.
Here is the callstack:
EspFcService.exe!boost::asio::detail::win_iocp_handle_service::close_for_destruction(boost::asio::detail::win_iocp_handle_service::implementation_type & impl) Line 509 C++
EspFcService.exe!boost::asio::detail::win_iocp_handle_service::destroy(boost::asio::detail::win_iocp_handle_service::implementation_type & impl) Line 168 C++
EspFcService.exe!boost::asio::windows::stream_handle_service::destroy(boost::asio::detail::win_iocp_handle_service::implementation_type & impl) Line 109 C++
EspFcService.exe!boost::asio::basic_io_object<boost::asio::windows::stream_handle_service,1>::~basic_io_object<boost::asio::windows::stream_handle_service,1>() Line 197 C++
EspFcService.exe!boost::asio::windows::basic_handle<boost::asio::windows::stream_handle_service>::~basic_handle<boost::asio::windows::stream_handle_service>() Line 269 C++
[External Code]
EspFcService.exe!boost::process::detail::windows::async_pipe::~async_pipe() Line 85 C++
[External Code]
EspFcService.exe!boost::process::detail::windows::executor<char,boost::fusion::joint_view<boost::fusion::tuple<boost::process::detail::windows::exe_cmd_init<char>,boost::process::detail::windows::io_service_ref,boost::fusion::void_,boost::fusion::void_,boost::fusion::void_,boost::fusion::void_,boost::fusion::void_,boost::fusion::void_,boost::fusion::void_,boost::fusion::void_>,boost::fusion::filter_view<boost::fusion::tuple<std::basic_string<char,std::char_traits<char>,std::allocator<char> > & __ptr64,boost::process::detail::windows::async_pipe_out<2,-1> & __ptr64,boost::process::detail::windows::async_pipe_out<1,-1> & __ptr64,boost::process::detail::windows::async_pipe_in & __ptr64,boost::process::detail::arg_setter_<char,0> & __ptr64,boost::asio::io_service & __ptr64,boost::process::detail::on_error_<<lambda_3265e35fa715b3d8a520025e781189f9> > & __ptr64,boost::process::detail::windows::on_exit_ & __ptr64,boost::fusion::void_,boost::fusion::void_> const ,boost::process::detail::is_initializer<boost::mpl::arg<-1> > > > >::internal_error_handle(const std::error_code & ec, const char * msg, boost::mpl::bool_<0> __formal, boost::mpl::bool_<0> __formal) Line 125 C++
EspFcService.exe!boost::process::detail::windows::executor<char,boost::fusion::joint_view<boost::fusion::tuple<boost::process::detail::windows::exe_cmd_init<char>,boost::process::detail::windows::io_service_ref,boost::fusion::void_,boost::fusion::void_,boost::fusion::void_,boost::fusion::void_,boost::fusion::void_,boost::fusion::void_,boost::fusion::void_,boost::fusion::void_>,boost::fusion::filter_view<boost::fusion::tuple<std::basic_string<char,std::char_traits<char>,std::allocator<char> > & __ptr64,boost::process::detail::windows::async_pipe_out<2,-1> & __ptr64,boost::process::detail::windows::async_pipe_out<1,-1> & __ptr64,boost::process::detail::windows::async_pipe_in & __ptr64,boost::process::detail::arg_setter_<char,0> & __ptr64,boost::asio::io_service & __ptr64,boost::process::detail::on_error_<<lambda_3265e35fa715b3d8a520025e781189f9> > & __ptr64,boost::process::detail::windows::on_exit_ & __ptr64,boost::fusion::void_,boost::fusion::void_> const ,boost::process::detail::is_initializer<boost::mpl::arg<-1> > > > >::set_error(const std::error_code & ec, const char * msg) Line 227 C++
EspFcService.exe!boost::process::detail::windows::executor<char,boost::fusion::joint_view<boost::fusion::tuple<boost::process::detail::windows::exe_cmd_init<char>,boost::process::detail::windows::io_service_ref,boost::fusion::void_,boost::fusion::void_,boost::fusion::void_,boost::fusion::void_,boost::fusion::void_,boost::fusion::void_,boost::fusion::void_,boost::fusion::void_>,boost::fusion::filter_view<boost::fusion::tuple<std::basic_string<char,std::char_traits<char>,std::allocator<char> > & __ptr64,boost::process::detail::windows::async_pipe_out<2,-1> & __ptr64,boost::process::detail::windows::async_pipe_out<1,-1> & __ptr64,boost::process::detail::windows::async_pipe_in & __ptr64,boost::process::detail::arg_setter_<char,0> & __ptr64,boost::asio::io_service & __ptr64,boost::process::detail::on_error_<<lambda_3265e35fa715b3d8a520025e781189f9> > & __ptr64,boost::process::detail::windows::on_exit_ & __ptr64,boost::fusion::void_,boost::fusion::void_> const ,boost::process::detail::is_initializer<boost::mpl::arg<-1> > > > >::operator()() Line 213 C++
EspFcService.exe!boost::process::detail::basic_execute_impl<char,std::basic_string<char,std::char_traits<char>,std::allocator<char> >,boost::process::detail::windows::async_pipe_out<2,-1>,boost::process::detail::windows::async_pipe_out<1,-1>,boost::process::detail::windows::async_pipe_in,boost::process::detail::arg_setter_<char,0>,boost::asio::io_service & __ptr64,boost::process::detail::on_error_<<lambda_3265e35fa715b3d8a520025e781189f9> >,boost::process::detail::windows::on_exit_>(std::basic_string<char,std::char_traits<char>,std::allocator<char> > && <args_0>, boost::process::detail::windows::async_pipe_out<2,-1> && <args_1>, boost::process::detail::windows::async_pipe_out<1,-1> && <args_2>, boost::process::detail::windows::async_pipe_in && <args_3>, boost::process::detail::arg_setter_<char,0> && <args_4>, boost::asio::io_service & <args_5>, boost::process::detail::on_error_<<lambda_3265e35fa715b3d8a520025e781189f9> > && <args_6>, boost::process::detail::windows::on_exit_ && <args_7>) Line 267 C++
EspFcService.exe!boost::process::detail::execute_impl<std::basic_string<char,std::char_traits<char>,std::allocator<char> >,boost::process::detail::windows::async_pipe_out<2,-1>,boost::process::detail::windows::async_pipe_out<1,-1>,boost::process::detail::windows::async_pipe_in,boost::process::detail::arg_setter_<char,0>,boost::asio::io_service & __ptr64,boost::process::detail::on_error_<<lambda_3265e35fa715b3d8a520025e781189f9> >,boost::process::detail::windows::on_exit_>(std::basic_string<char,std::char_traits<char>,std::allocator<char> > && <args_0>, boost::process::detail::windows::async_pipe_out<2,-1> && <args_1>, boost::process::detail::windows::async_pipe_out<1,-1> && <args_2>, boost::process::detail::windows::async_pipe_in && <args_3>, boost::process::detail::arg_setter_<char,0> && <args_4>, boost::asio::io_service & <args_5>, boost::process::detail::on_error_<<lambda_3265e35fa715b3d8a520025e781189f9> > && <args_6>, boost::process::detail::windows::on_exit_ && <args_7>) Line 275 C++
EspFcService.exe!boost::process::child::child<std::basic_string<char,std::char_traits<char>,std::allocator<char> >,boost::process::detail::windows::async_pipe_out<2,-1>,boost::process::detail::windows::async_pipe_out<1,-1>,boost::process::detail::windows::async_pipe_in,boost::process::detail::arg_setter_<char,0>,boost::asio::io_service & __ptr64,boost::process::detail::on_error_<<lambda_3265e35fa715b3d8a520025e781189f9> >,boost::process::detail::windows::on_exit_>(std::basic_string<char,std::char_traits<char>,std::allocator<char> > && <args_0>, boost::process::detail::windows::async_pipe_out<2,-1> && <args_1>, boost::process::detail::windows::async_pipe_out<1,-1> && <args_2>, boost::process::detail::windows::async_pipe_in && <args_3>, boost::process::detail::arg_setter_<char,0> && <args_4>, boost::asio::io_service & <args_5>, boost::process::detail::on_error_<<lambda_3265e35fa715b3d8a520025e781189f9> > && <args_6>, boost::process::detail::windows::on_exit_ && <args_7>) Line 35 C++
\boost\process\detail\windows\async_pipe.hpp:
~async_pipe()
{
if (_sink .native() != ::boost::detail::winapi::INVALID_HANDLE_VALUE_)
::boost::detail::winapi::CloseHandle(_sink.native());
if (_source.native() != ::boost::detail::winapi::INVALID_HANDLE_VALUE_)
::boost::detail::winapi::CloseHandle(_source.native());
}
boost\asio\detail\impl\win_iocp_handle_service.ipp:
void win_iocp_handle_service::close_for_destruction(implementation_type& impl)
{
if (is_open(impl))
{
BOOST_ASIO_HANDLER_OPERATION(("handle", &impl, "close"));
::CloseHandle(impl.handle_);
impl.handle_ = INVALID_HANDLE_VALUE;
impl.safe_cancellation_thread_id_ = 0;
}
}
boost\asio\detail\win_iocp_handle_service.hpp:
bool is_open(const implementation_type& impl) const
{
return impl.handle_ != INVALID_HANDLE_VALUE;
}
Here is the error:
Exception thrown at 0x0000000076FABB47 (ntdll.dll) in EspFcService.exe: 0xC0000008: An invalid handle was specified.
If there is a handler for this exception, the program may be safely continued.
This error can be ignored and the program runs, but makes it difficult to use an attached debugger or the application verifier.
The async_pipe destructor calls ::CloseHandle first on the sink and source handles, then the asio base class destructor calls ::CloseHandle again. async_pipe could set the handles to INVALID_HANDLE_VALUE or it would probably be better to call some asio functions to do that instead of manipulating the handles directly.
Synopsis:
bool this_process::is_grouped();
struct group
{
group();
void wait();
void terminate();
void detach();
/**/ native_handle();
void assign(child &);
};
Putting a process into another group:
group g;
execute("gcc", g);
Problem: the this_process::group()
cannot be implemented in a proper way, since I cannot get the handle of the current process group on windows. BUT I can create a group and assign the current process to it, if it does NOT have a group assigned to it. The latter is also a windows constraint.
I.e. done like this:
group g;
auto c = execute("gcc");
g.assign(c);
That throws if this_process is already assigned to a group (windows constraint).
For Posix: this is done indirectly, a group will have a bool which says if it's initialized. If it is not initialized it will be after fork and write the gid into a self-pipe.
In the documentation, on the Tutorial Page:
This might lead to different results which are platform dependent.
What does this mean? Is this windows vs. linux or more obscure platforms?
The default depends on the system, but usually this will just write it to the output of the launching process.
What should be expected on which systems. It seems like system variances in these items should get their own page of explanation with links to these spots.
Design page:
On posix all those callbacks will be handles by this process, not the created one.
Is this different on windows? Other platforms?
Write a test only passing a command line, for linux as well as windows
lets have a pseudocode simulating usage of boost::process in wrapped highly threaded environment
void some_func() {
child c("tee", "-");
start_thread_in_background: do_wait_thread(c); // it triggers do_wait_thread asynchrnonously
do_some_expensive_thing();
c->terminate();
}
void do_wait_thread(child *c) {
c->wait(); // "waitpid(2) failed: No child processes" is raised here after terminate() is called
}
is it expected behavior?
hi,
I'm trying to compile stuff on Windows using msys2/mingw toolchain.
Boost 1.63 is provided from msys2 as a package mingw-w64-x86_64-boost
Q: is mingw/msys2 supported at all?
Q: why it fails to compile?
Scanning dependencies of target process
[ 16%] Building CXX object CMakeFiles/process.dir/src/processpriv.cpp.obj
In file included from C:/Users/pvanek/src/qore/module-process/src/boost/process/child.hpp:22:0,
from C:/Users/pvanek/src/qore/module-process/src/boost/process/async_system.hpp:22,
from C:/Users/pvanek/src/qore/module-process/src/boost/process.hpp:24,
from C:/Users/pvanek/src/qore/module-process/src/processpriv.h:5,
from C:/Users/pvanek/src/qore/module-process/src/processpriv.cpp:4:
C:/Users/pvanek/src/qore/module-process/src/boost/process/detail/execute_impl.hpp: In instantiation of 'struct boost::process::detail::make_builders_from_view<boost::fusion::vector_iterator<boost::fusion::vector<long long int&>, 0>, boost::fusion::vector_iterator<boost::fusion::vector<long long int&>, 1> >':
C:/Users/pvanek/src/qore/module-process/src/boost/process/detail/execute_impl.hpp:255:77: required from 'boost::process::child boost::process::detail::basic_execute_impl(Args&& ...) [with Char = char; Args = {long long int&}]'
C:/Users/pvanek/src/qore/module-process/src/boost/process/detail/execute_impl.hpp:275:45: required from 'boost::process::child boost::process::detail::execute_impl(Args&& ...) [with Args = {long long int&}]'
C:/Users/pvanek/src/qore/module-process/src/boost/process/child.hpp:35:51: required from 'boost::process::child::child(Args&& ...) [with Args = {long long int&}]'
C:/Users/pvanek/src/qore/module-process/src/processpriv.cpp:17:38: required from here
C:/Users/pvanek/src/qore/module-process/src/boost/process/detail/execute_impl.hpp:117:54: error: invalid use of incomplete type 'struct boost::process::detail::initializer_tag<long long int>'
typedef typename initializer_tag<res_type>::type tag;
^~~
In file included from C:/Users/pvanek/src/qore/module-process/src/boost/process/detail/traits/cmd_or_exe.hpp:15:0,
from C:/Users/pvanek/src/qore/module-process/src/boost/process/detail/basic_cmd.hpp:13,
from C:/Users/pvanek/src/qore/module-process/src/boost/process/args.hpp:33,
from C:/Users/pvanek/src/qore/module-process/src/boost/process.hpp:22,
from C:/Users/pvanek/src/qore/module-process/src/processpriv.h:5,
from C:/Users/pvanek/src/qore/module-process/src/processpriv.cpp:4:
C:/Users/pvanek/src/qore/module-process/src/boost/process/detail/traits/decl.hpp:33:8: note: declaration of 'struct boost::process::detail::initializer_tag<long long int>'
struct initializer_tag;// { typedef void type; };
^~~~~~~~~~~~~~~
In file included from C:/Users/pvanek/src/qore/module-process/src/boost/process/child.hpp:22:0,
from C:/Users/pvanek/src/qore/module-process/src/boost/process/async_system.hpp:22,
from C:/Users/pvanek/src/qore/module-process/src/boost/process.hpp:24,
from C:/Users/pvanek/src/qore/module-process/src/processpriv.h:5,
from C:/Users/pvanek/src/qore/module-process/src/processpriv.cpp:4:
C:/Users/pvanek/src/qore/module-process/src/boost/process/detail/execute_impl.hpp:118:53: error: invalid use of incomplete type 'struct boost::process::detail::initializer_tag<long long int>'
typedef typename initializer_builder<tag>::type builder_type;
^~~~~~~~~~~~
In file included from C:/Users/pvanek/src/qore/module-process/src/boost/process/detail/traits/cmd_or_exe.hpp:15:0,
from C:/Users/pvanek/src/qore/module-process/src/boost/process/detail/basic_cmd.hpp:13,
from C:/Users/pvanek/src/qore/module-process/src/boost/process/args.hpp:33,
from C:/Users/pvanek/src/qore/module-process/src/boost/process.hpp:22,
from C:/Users/pvanek/src/qore/module-process/src/processpriv.h:5,
from C:/Users/pvanek/src/qore/module-process/src/processpriv.cpp:4:
C:/Users/pvanek/src/qore/module-process/src/boost/process/detail/traits/decl.hpp:33:8: note: declaration of 'struct boost::process::detail::initializer_tag<long long int>'
struct initializer_tag;// { typedef void type; };
^~~~~~~~~~~~~~~
In file included from C:/Users/pvanek/src/qore/module-process/src/boost/process/child.hpp:22:0,
from C:/Users/pvanek/src/qore/module-process/src/boost/process/async_system.hpp:22,
from C:/Users/pvanek/src/qore/module-process/src/boost/process.hpp:24,
from C:/Users/pvanek/src/qore/module-process/src/processpriv.h:5,
from C:/Users/pvanek/src/qore/module-process/src/processpriv.cpp:4:
C:/Users/pvanek/src/qore/module-process/src/boost/process/detail/execute_impl.hpp:119:75: error: invalid use of incomplete type 'struct boost::process::detail::initializer_tag<long long int>'
typedef typename boost::fusion::result_of::has_key<set, builder_type> has_key;
^~~~~~~
In file included from C:/Users/pvanek/src/qore/module-process/src/boost/process/detail/traits/cmd_or_exe.hpp:15:0,
from C:/Users/pvanek/src/qore/module-process/src/boost/process/detail/basic_cmd.hpp:13,
from C:/Users/pvanek/src/qore/module-process/src/boost/process/args.hpp:33,
from C:/Users/pvanek/src/qore/module-process/src/boost/process.hpp:22,
from C:/Users/pvanek/src/qore/module-process/src/processpriv.h:5,
from C:/Users/pvanek/src/qore/module-process/src/processpriv.cpp:4:
C:/Users/pvanek/src/qore/module-process/src/boost/process/detail/traits/decl.hpp:33:8: note: declaration of 'struct boost::process::detail::initializer_tag<long long int>'
struct initializer_tag;// { typedef void type; };
^~~~~~~~~~~~~~~
In file included from C:/Users/pvanek/src/qore/module-process/src/boost/process/child.hpp:22:0,
from C:/Users/pvanek/src/qore/module-process/src/boost/process/async_system.hpp:22,
from C:/Users/pvanek/src/qore/module-process/src/boost/process.hpp:24,
from C:/Users/pvanek/src/qore/module-process/src/processpriv.h:5,
from C:/Users/pvanek/src/qore/module-process/src/processpriv.cpp:4:
C:/Users/pvanek/src/qore/module-process/src/boost/process/detail/execute_impl.hpp:128:21: error: invalid use of incomplete type 'struct boost::process::detail::initializer_tag<long long int>'
>::type type;
^~~~
In file included from C:/Users/pvanek/src/qore/module-process/src/boost/process/detail/traits/cmd_or_exe.hpp:15:0,
from C:/Users/pvanek/src/qore/module-process/src/boost/process/detail/basic_cmd.hpp:13,
from C:/Users/pvanek/src/qore/module-process/src/boost/process/args.hpp:33,
from C:/Users/pvanek/src/qore/module-process/src/boost/process.hpp:22,
from C:/Users/pvanek/src/qore/module-process/src/processpriv.h:5,
from C:/Users/pvanek/src/qore/module-process/src/processpriv.cpp:4:
C:/Users/pvanek/src/qore/module-process/src/boost/process/detail/traits/decl.hpp:33:8: note: declaration of 'struct boost::process::detail::initializer_tag<long long int>'
struct initializer_tag;// { typedef void type; };
^~~~~~~~~~~~~~~
The code below doesn't compile with the newest winapi modules and /WX
boost::process::child c("c:/windows/system32/foo.exe", "/r", dll); c.wait();
Visual Studio 2015
64-bit debug build
boost 1.62.0 static build, with static runtime
flags: /WX /W4
pp: _SCL_SECURE_NO_WARNINGS;_CRT_SECURE_NO_WARNINGS;
`
Severity Code Description Project File Line Suppression State
Warning C4244 'argument': conversion from 'int64' to 'int', possible loss of data fpreg D:\src\trunk\third_party\boost\include\boost\process\detail\windows\locale.hpp 49
Warning C4244 'argument': conversion from 'int64' to 'int', possible loss of data fpreg D:\src\trunk\third_party\boost\include\boost\process\detail\windows\locale.hpp 71
Warning C4800 'boost::detail::winapi::BOOL': forcing value to bool 'true' or 'false' (performance warning) fpreg D:\src\trunk\third_party\boost\include\boost\process\detail\windows\child_handle.hpp 85
Warning C4800 'boost::detail::winapi::BOOL': forcing value to bool 'true' or 'false' (performance warning) fpreg D:\src\trunk\third_party\boost\include\boost\process\detail\windows\child_handle.hpp 92
Warning C4800 'boost::detail::winapi::BOOL': forcing value to bool 'true' or 'false' (performance warning) fpreg D:\src\trunk\third_party\boost\include\boost\process\detail\windows\group_handle.hpp 146
Warning C4800 'boost::detail::winapi::BOOL': forcing value to bool 'true' or 'false' (performance warning) fpreg D:\src\trunk\third_party\boost\include\boost\process\detail\windows\group_handle.hpp 153
Warning C4800 'boost::detail::winapi::BOOL_': forcing value to bool 'true' or 'false' (performance warning) fpreg D:\src\trunk\third_party\boost\include\boost\process\detail\windows\group_handle.hpp 183
Warning C4099 'boost::process::detail::windows::group_handle': type name first seen using 'struct' now seen using 'class' fpreg D:\src\trunk\third_party\boost\include\boost\process\detail\windows\wait_group.hpp 16
Warning C4100 'state': unreferenced formal parameter fpreg D:\src\trunk\third_party\boost\include\boost\process\detail\windows\locale.hpp 38
Warning C4100 'state': unreferenced formal parameter fpreg D:\src\trunk\third_party\boost\include\boost\process\detail\windows\locale.hpp 60
d:\src\trunk\third_party\boost\include\boost\fusion\container\generation\detail\preprocessed\make_set10.hpp 35
Warning C4459 declaration of 'env' hides global declaration fpreg D:\src\trunk\third_party\boost\include\boost\process\environment.hpp 57
Warning C4458 declaration of 'ios' hides class member fpreg D:\src\trunk\third_party\boost\include\boost\process\async.hpp 59
Warning C4459 declaration of 'on_error' hides global declaration fpreg D:\src\trunk\third_party\boost\include\boost\process\detail\windows\executor.hpp 183
Warning C4459 declaration of 'on_error' hides global declaration fpreg D:\src\trunk\third_party\boost\include\boost\process\detail\windows\executor.hpp 215
Warning C4459 declaration of 'on_setup' hides global declaration fpreg D:\src\trunk\third_party\boost\include\boost\process\detail\windows\executor.hpp 178
Warning C4459 declaration of 'on_success' hides global declaration fpreg D:\src\trunk\third_party\boost\include\boost\process\detail\windows\executor.hpp 206
I'd really like to have the boost.process library, but since development was on hiatus for two years, I think a few small design changes would be nice - or rather big ones, as described below.
But, not everything I write here is completely tested. I'd really need some discussion about those and other ideas how such a library could look like, because I really would change the frontend.
Here's the reference of the current version.
@dlaugt @BorisSchaeling @whizmo @hotgloupi @nat-goodspeed @havoc-io @rconde01 @JeffGarland
Since we now have a variadic templates, I think the execute function can be extended and much more flexible. Please notice, that execute
shall return an undefined type, which depends on the parameters. It might be converted to child
, as discussed below.
#1 Defaults
First of all, I like to unify the signal handling, to provide the following:
void on_setup(Executor&) const
void on_error(Executor&, const std::error_code &ec) const
void on_success(Executor&) const
This would require the posix-executor to always use the self-pipe in order to provide the on_success signal. Therefore I think it would be also possible to add a initializer which removes this, in the following way:
using namespace boost::process::initializers;
auto p = execute("some_prog", posix::no_success_signal);
If then a success handler is provided, it would provide an error, i.e.:
using namespace boost::process::initializers;
auto p = execute("some_prog", posix::no_success_signal, on_success([](auto){}));
//yields an compiler error
The way it should work on posix is this:
A possible code in posix would look like this:
executor::operator()(...)
{
pipe error_pipe = create_pipe(FD_CLOEXEC); //closes the pipe on execve call
on_setup();
auto pid = fork();
if (pid == -1)
on_fork_error();
else if (pid == 0)
{
::close(selfe_pipe.source);
on_exec_setup();
::execve(exe, cmd_line, env);
on_exec_error();
::write(error_pipe.sink, &errno, sizeof(int));
::_exit(EXIT_FAILURE);
}
//ok, here we are back in the father-process
int code, cnt;
do
{
cnt = ::read(error_pipe.source, &code, sizeof(code));
int err = errno;
if (res == EDABF) //pipe is closed.
break;
//EAGAIN not yet forked, EINTR interrupted, i.e. try again
else if ((err != EAGAIN) && (err != EINTR))
else
throw error();
}
while(cnt == -1);
if (cnt == 4) //error.
on_error(code);
else if (cnt > 0) //some other strang error
on_error(...); //don't know yet
else
on_success();
}
Now, on default the father process is waiting. I think to provide a ignore_error
would be very usefule, since this would then ignore this.
The additional signal-handlers will be availble as part of the boost::process::posix
namespace.
I also would make the throw_on_error the default, because it just makes the most sense, since execute constructs a process-object. I.e. not being able to construct it should result in an error.
auto p1 = execute("not_runnable"); //throws
auto p2 = execute("not_runnable", ignore_error); //no throw, no handling;
std::error_code ec;
auto p3 = execute("not_runnable", set_on_error(ec)); //no throw, takes the error
auto p3 = execute("not_runnable", ec); //no throw, takes the error, shortcut.
I would introduce posix::no_success_signal
and ignore_error
because the posix implementation now waits for the child-process to start. A user might consider that unneeded and could hence wish to skip that.
#2. Simplified syntax
I think it would be possible to make the syntax much simpler, as in the following examples (corresponding to the current). All are equal.
auto p1 = execute(exe("cmd"), args("x", "y")); //shortened version of the current syntax
auto p2 = execute(exe="cmd", args={"x", "y"}); //alternate syntax
auto p3 = execute(exe="cmd", args="x", args+="y"); //other syntax
auto p4 = execute("cmd", "x", "y"); //lazy syntax
auto p5 = execute(cmd("cmd x y"));
auto p6 = execute(cmd="cmd x y");
auto p7 = execute("cmd x y");
The cmd and args get parsed differently, that is:
execute(cmd="cmd x y z");//yields "cmd x y z"
execute(cmd="cmd \"x y\" z");// yields "cmd \"x y\" z"
execute(exe="cmd", args={"x y", "z"}); //yields "cmd \"x y\" z"
The following signal handlers shall have syntax similar to the above.
And the platform specifics.
This is the part where I am the least confident, but those are all things I consider possible.
If a pipe shall be closed, it should be possible in the following way
execute("ls", std_out=null); //redirect to /dev/null
execute("ls", std_out=close); //close the pipe
//alternate syntax
execute("ls", std_out>null); //redirect to /dev/null
execute("ls", close(std_out)); //close the pipe
The sync I/O can be simplified in the following way (current example). I.e. the pipe can be constructed automatically.
file_descriptor_sink sink;
execute("my_prog", std_in=sink);//alternatively '|', because pipe.
stream<file_descriptor_sink> str;
stream<file_descriptor_source> str;
execute("my_prog", std_out=str);
If the output shall be redirectet into a file, this can be done via a boost::filesystem::path
:
execute("my_prog", std_out=path("./log.txt"));
It may also be following to allow a lazy redirection to files:
execute("my_prog", std_out>"./log.txt");
With this syntax it is of course also easy to pipe from one process to another (though the implementation is not yet clear, i might need to use a named pipe on windows)
process::pipe = create_pipe();
execute("prog1", std_out>pipe);
execute("prog2", std_in<pipe);
Removed, because that makes child struct unecessary complicated
Another possibility, that would definitly be possible would be to allow the process to adapt to a string. I don't know it this would actually be worth the effort.
auto p = execute("c++filt", std_in=self, std_out=self);
p << "_ZN9wikipedia7article8wikilinkC1ERKSs";
std::string dem;
p >> dem;
//dem == "wikipedia::article::wikilink::wikilink(std::string const&)"
This is only convenience, it allows short-handed I/O. One could also use the stderr instead of stdout, while declaring both as self will yield an error.
This might be the most difficult part. I imagine that we can do it in someway like this:
boost::asio::io_service io_service;
stringstream ss;//threadunsafe, but get's the point.
string buf;
execute("some_prog", std_in<ss, std_out>buffer(buf), std_err>cerr, io_service);
io_service.run();
Thereby the whole mitigate thing can be hidden and managed automatically. Addtionally, if no io_service is passed, it could be created by the execute command and run it in another thread.
The example from the current tutorial would look like this:
boost::asio::io_service io_service;
std::array<char, 4096> buffer;
auto p = execute("test.exe", std_out>process::buffer(buffer), io_service);
io_service.run();
I can determine that buffer must be read async,because it's no stream and can construct the async_pipe in place. Also I can call async_read
from the executor, because that does only start on io_service.run()
.
Additionally, the io_service could be skipped and constructed in the executor and launched in a new thread. Don't know if that'd be smart though.
But to be honest: I am currently not nearly familiar enough with the way io_service
works.
The same would apply to any std::ostream
or std::istream
, which is not stream<file_descriptor_sink>
, so the cerr
or ss
in my example should work this way. Now of course, binding std_err>cerr
is by far the stupidest way, because I could pipe that directly (std_err>stderr
) - at least if we provide a syntax for that (i.e. also redirecting std_err>stdout
).
This whole concept needs more experimentation!
In 0.5 the inherit_env
was empty for windows, I don't know if that actually works. Now I would use the following syntax:
execute("thingy"); //inherit env
execute("thingy", env+={"MY_VAR=Thingy"}); //inhertit and add
execute("things", env= {"MY_VAR=Thingy"}); //do not inherit
So the following would be awesome, though I don't know if necessary:
//inherit the environment and append "MyThing" to "MY_VAR"
execute("thingy", env["MY_VAR"]+="MyThingy"};
#3. Classes
When calling execute a process-object is returned, whichs type is undefined. It holds any data needed by the initialization. It has a set of functions, which are the following
If the process shall be stored in an undefined class, it can be converted to child
. The data is then moved onto the stack and stored in a pointer inside the child. The child class will have member-functions similar to the functions for process.
A child is movable, but not copyable.
#4. Launch Mode
Removed, is done via process groups #8
In order to provide a process in an RAII way I do think that different launch modes should be available. I do not know as for now if this can be switched at runtime or has to be defined at launch time.
If a childprocess runs attached it will be terminated on destructor-call.
If a process is detached nothing will happend on destruction
By default a process is launched in an attached way. This would cause this syntax to immediately terminate the application:
execute("prog");
But this behaviour could of course be specified by the following way:
execute("prog", wait); //blocking
execute("prog", detach); //not blocking.
#5 this_process
Similar to std::this_thread
I would add a this_process
namespace with the following synopsis.
namespace this_process
{
int get_id();
/*undefined*/ native_handle();
environment get_environment();
}
Hi, I'm trying to compile a simple hello world using g++4.8 against boost 1.55:
#include <boost/process.hpp>
namespace bp = boost::process;
int main()
{
std::string cmd = "echo 'Hello, world!'";
bp::child child(
cmd,
bp::shell,
bp::std_in.close(),
bp::std_err.close(),
bp::std_out.close()
);
child.wait();
return child.exit_code();
}
This is what I get:
In file included from /boost-1.55/include/boost/fusion/container/vector/vector10.hpp:65:0,
from /boost-1.55/include/boost/fusion/container/vector/vector_fwd.hpp:15,
from /boost-1.55/include/boost/fusion/mpl/detail/clear.hpp:11,
from /boost-1.55/include/boost/fusion/mpl/clear.hpp:13,
from /boost-1.55/include/boost/fusion/mpl.hpp:17,
from /boost-1.55/include/boost/fusion/view/detail/strictest_traversal.hpp:16,
from /boost-1.55/include/boost/fusion/view/transform_view/transform_view.hpp:21,
from /boost-1.55/include/boost/fusion/algorithm/transformation/transform.hpp:11,
from src/boost/process/detail/posix/io_service_ref.hpp:16,
from src/boost/process/async.hpp:30,
from src/boost/process.hpp:21,
from test.cpp:1:
/boost-1.55/include/boost/fusion/container/vector/detail/preprocessed/vector10.hpp: In instantiation of ‘boost::fusion::vector_data1<T0>::vector_data1(typename boost::fusion::detail::call_param<Car>::type) [with T0 = boost::process::detail::posix::exe_cmd_init; typename boost::fusion::detail::call_param<Car>::type = const boost::process::detail::posix::exe_cmd_init&]’:
/boost-1.55/include/boost/fusion/container/vector/detail/preprocessed/vector10.hpp:54:36: required from ‘static boost::fusion::vector_data1<T0> boost::fusion::vector_data1<T0>::init_from_sequence(const Sequence&) [with Sequence = boost::fusion::tuple<boost::process::detail::posix::exe_cmd_init, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_>; T0 = boost::process::detail::posix::exe_cmd_init]’
/boost-1.55/include/boost/fusion/container/vector/detail/preprocessed/vector10.hpp:128:58: required from ‘boost::fusion::vector1<T0>::vector1(const Sequence&, typename boost::disable_if<boost::is_convertible<Sequence, Car> >::type*) [with Sequence = boost::fusion::tuple<boost::process::detail::posix::exe_cmd_init, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_>; T0 = boost::process::detail::posix::exe_cmd_init; typename boost::disable_if<boost::is_convertible<Sequence, Car> >::type = void]’
/boost-1.55/include/boost/fusion/container/vector/detail/preprocessed/vvector10.hpp:43:50: required from ‘boost::fusion::vector<T0, T1, T2, T3, T4, T5, T6, T7, T8, T9>::vector(const Sequence&) [with Sequence = boost::fusion::tuple<boost::process::detail::posix::exe_cmd_init, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_>; T0 = boost::process::detail::posix::exe_cmd_init; T1 = boost::fusion::void_; T2 = boost::fusion::void_; T3 = boost::fusion::void_; T4 = boost::fusion::void_; T5 = boost::fusion::void_; T6 = boost::fusion::void_; T7 = boost::fusion::void_; T8 = boost::fusion::void_; T9 = boost::fusion::void_]’
/boost-1.55/include/boost/fusion/tuple/detail/preprocessed/tuple10.hpp:20:28: required from ‘boost::fusion::tuple<T0, T1, T2, T3, T4, T5, T6, T7, T8, T9>::tuple(const boost::fusion::tuple<T0, T1, T2, T3, T4, T5, T6, T7, T8, T9>&) [with T0 = boost::process::detail::posix::exe_cmd_init; T1 = boost::fusion::void_; T2 = boost::fusion::void_; T3 = boost::fusion::void_; T4 = boost::fusion::void_; T5 = boost::fusion::void_; T6 = boost::fusion::void_; T7 = boost::fusion::void_; T8 = boost::fusion::void_; T9 = boost::fusion::void_]’
src/boost/process/detail/execute_impl.hpp:205:75: required from ‘boost::process::child boost::process::detail::execute_impl(Args&& ...) [with Args = {std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, const boost::process::detail::shell_&, boost::process::detail::posix::close_in, boost::process::detail::posix::close_out<2, -1>, boost::process::detail::posix::close_out<1, -1>}]’
src/boost/process/child.hpp:34:79: required from ‘boost::process::child::child(Args&& ...) [with Args = {std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, const boost::process::detail::shell_&, boost::process::detail::posix::close_in, boost::process::detail::posix::close_out<2, -1>, boost::process::detail::posix::close_out<1, -1>}]’
test.cpp:15:5: required from here
/boost-1.55/include/boost/fusion/container/vector/detail/preprocessed/vector10.hpp:34:20: error: use of deleted function ‘boost::process::detail::posix::exe_cmd_init::exe_cmd_init(const boost::process::detail::posix::exe_cmd_init&)’
: m0(_0) {}
^
In file included from src/boost/process/detail/basic_cmd.hpp:19:0,
from src/boost/process/args.hpp:20,
from src/boost/process.hpp:20,
from test.cpp:1:
src/boost/process/detail/posix/basic_cmd.hpp:104:5: error: declared here
exe_cmd_init(const exe_cmd_init & ) = delete;
^
However, if I make exe_cmd_init
copyable, it compiles fine and seems to work
Hello,
I tried the following code, following the example in the documentation :
#include <iostream>
#include <boost/process.hpp>
#include <boost/asio.hpp>
int main( int argc, char * argv[] ) {
boost::asio::io_service ios;
boost::process::async_pipe ap( ios );
boost::process::child c( boost::process::search_path( "ls" ), "."
, boost::process::std_out > ap
);
std::vector<char> buf;
// Get content of pipe -- before wait
boost::asio::async_read( ap, boost::asio::buffer( buf )
, []( boost::system::error_code const &, std::size_t ) {}
);
ios.run();
// Print read data: NOTHING (might depend on scheduling)
std::cout << "result: ";
for ( auto c : buf ) std::cout << c;
std::cout << std::endl;
c.wait();
std::cout << "exit code: " << c.exit_code() << std::endl;
// Get content of pipe -- after wait
boost::asio::async_read( ap, boost::asio::buffer( buf )
, []( boost::system::error_code const &, std::size_t ) {}
);
ios.run();
// Print read data: NOTHING (should be output of ls)
std::cout << "result2: ";
for ( auto c : buf ) std::cout << c;
std::cout << std::endl;
}
Unfortunately, in neither tests (before and after the wait) the code gets the output of the sub-process.
The inclusion of boost/process.hpp in a vanilla cpp source file does not compile under macOS 10.10 or greater. I was able to reproduce this issue in a clean install of macOS 10.12 with the latest version of Xcode and the Apple command line tools. I am using clang-800.0.42.1 with the following code:
test_process_program.cpp:
`#include <boost/process.hpp>
int main() {
return 0;
}`
The following compiler command was entered whilst using a working install of Boost 1.63 with the process.hpp and process folders dropped into /usr/local/include/boost/ where the boost libraries have been installed under macOS 10.12.
g++ -std=c++11 -ansi -Wall -lboost_system test_process_program.cpp -o test_process_program
This results in the following error:
In file included from test_process_program.cpp:1: In file included from /usr/local/include/boost/process.hpp:22: In file included from /usr/local/include/boost/process/args.hpp:33: In file included from /usr/local/include/boost/process/detail/basic_cmd.hpp:10: In file included from /usr/local/include/boost/process/detail/config.hpp:24: /usr/local/include/boost/process/exception.hpp:22:30: error: using declaration cannot refer to a constructor using std::system_error::system_error; ~~~~~~~~~~~~~~~~~~~^ In file included from test_process_program.cpp:1: In file included from /usr/local/include/boost/process.hpp:22: In file included from /usr/local/include/boost/process/args.hpp:33: In file included from /usr/local/include/boost/process/detail/basic_cmd.hpp:10: /usr/local/include/boost/process/detail/config.hpp:50:41: error: expected function body after function declarator inline std::error_code get_last_error() noexcept ^ /usr/local/include/boost/process/detail/config.hpp:82:12: error: unknown type name 'constexpr' template<> constexpr char null_char<char> (){return '\0';} ^ /usr/local/include/boost/process/detail/config.hpp:82:22: error: expected unqualified-id template<> constexpr char null_char<char> (){return '\0';} ^ /usr/local/include/boost/process/detail/config.hpp:86:12: error: unknown type name 'constexpr' template<> constexpr char equal_sign<char> () {return '='; } ^ /usr/local/include/boost/process/detail/config.hpp:86:22: error: expected unqualified-id template<> constexpr char equal_sign<char> () {return '='; } ^ /usr/local/include/boost/process/detail/config.hpp:90:12: error: unknown type name 'constexpr' template<> constexpr char quote_sign<char> () {return '"'; } ^ /usr/local/include/boost/process/detail/config.hpp:90:22: error: expected unqualified-id template<> constexpr char quote_sign<char> () {return '"'; } ^ /usr/local/include/boost/process/detail/config.hpp:94:12: error: unknown type name 'constexpr' template<> constexpr char space_sign<char> () {return ' '; } ^ /usr/local/include/boost/process/detail/config.hpp:94:22: error: expected unqualified-id template<> constexpr char space_sign<char> () {return ' '; } ^ In file included from test_process_program.cpp:1: In file included from /usr/local/include/boost/process.hpp:22: In file included from /usr/local/include/boost/process/args.hpp:33: In file included from /usr/local/include/boost/process/detail/basic_cmd.hpp:12: /usr/local/include/boost/process/detail/handler_base.hpp:21:5: error: unknown type name 'constexpr' constexpr make_handler_t() {} ^ /usr/local/include/boost/process/detail/handler_base.hpp:21:15: error: constructor cannot have a return type constexpr make_handler_t() {} ^~~~~~~~~~~~~~ /usr/local/include/boost/process/detail/handler_base.hpp:23:5: error: unknown type name 'constexpr' constexpr Template<Handler> operator()(Handler handler) const {return Template<Handler>(handler);} ^ /usr/local/include/boost/process/detail/handler_base.hpp:23:15: error: member 'Template' declared as a template constexpr Template<Handler> operator()(Handler handler) const {return Template<Handler>(handler);} ^ /usr/local/include/boost/process/detail/handler_base.hpp:23:23: error: expected ';' at end of declaration list constexpr Template<Handler> operator()(Handler handler) const {return Template<Handler>(handler);} ^ In file included from test_process_program.cpp:1: In file included from /usr/local/include/boost/process.hpp:22: In file included from /usr/local/include/boost/process/args.hpp:33: In file included from /usr/local/include/boost/process/detail/basic_cmd.hpp:13: In file included from /usr/local/include/boost/process/detail/traits/cmd_or_exe.hpp:15: /usr/local/include/boost/process/detail/traits/decl.hpp:58:5: error: unknown type name 'constexpr' constexpr static bool value = is_initializer<First>::value || !std::is_void<typename initializer_tag<First>::type... ^ /usr/local/include/boost/process/detail/traits/decl.hpp:58:15: error: expected member name or ';' after declaration specifiers constexpr static bool value = is_initializer<First>::value || !std::is_void<typename initializer_tag<First>::type... ~~~~~~~~~ ^ /usr/local/include/boost/process/detail/traits/decl.hpp:59:42: error: use of undeclared identifier 'value' typedef std::integral_constant<bool, value> type; ^ /usr/local/include/boost/process/detail/traits/decl.hpp:65:5: error: unknown type name 'constexpr' constexpr static bool my_value = is_initializer<First>::value || !std::is_void<typename initializer_tag<First>::t... ^ fatal error: too many errors emitted, stopping now [-ferror-limit=] 20 errors generated.
Any particular reason for the inconsistency?
hi again. It's me, master or unusal usage of boost::process ;)
What is a purpose of bp::start_dir? It seems to me it's like "to setup an initial working dir" for child process.
But in this case it's somehow broken (?). See:
#include <boost/process.hpp>
#include <iostream>
namespace bp = boost::process;
int main(int argc, char** argv)
{
bp::child c( bp::search_path("test.sh")
, bp::start_dir = "/tmp"
);
c.wait();
int result = c.exit_code();
return result;
}
then test.sh is (execute flag set on):
#!/bin/bash
echo $PWD
and run:
pvanek@linux-j415:~/oss/process/test-impl> export PATH=$PATH:.
pvanek@linux-j415:~/oss/process/test-impl> ./test-impl
terminate called after throwing an instance of 'boost::process::process_error'
what(): execve failed: No such file or directory
Aborted (core dumped)
coredump is fine, I just don't catch an exception. All is working if I omit start_dir option.
But I'd expect something different: test.sh should be run, but its starting dir would be /tmp...
It doesn't seem like the library supports OSX. Is there a plan for this?
hi,
I'm getting errors since commit 70dad97 in unchanged code.
Is it wanted situation (API breakage) so I should change my code? Or is it a regression?
pvanek@spongebob:~/src/qore/module-process/build> make
Scanning dependencies of target process
[ 16%] Building CXX object CMakeFiles/process.dir/src/processpriv.cpp.o
In file included from /export/home/pvanek/src/qore/module-process/src/boost/process.hpp:28:0,
from /export/home/pvanek/src/qore/module-process/src/processpriv.h:5,
from /export/home/pvanek/src/qore/module-process/src/processpriv.cpp:4:
/export/home/pvanek/src/qore/module-process/src/boost/process/env.hpp: In instantiation of ‘constexpr boost::fusion::vector_detail::store<<anonymous>, T>::store() [with long unsigned int <anonymous> = 1ul; T = boost::process::detail::env_builder<char>]’:
/usr/include/boost/fusion/container/vector/vector.hpp:196:13: required from ‘constexpr boost::fusion::vector_detail::vector_data<boost::fusion::detail::index_sequence<Left ...>, T ...>::vector_data() [with long unsigned int ...I = {0ul, 1ul}; T = {boost::process::detail::exe_builder<char>, boost::process::detail::env_builder<char>}]’
/usr/include/boost/fusion/container/vector/vector.hpp:288:9: required from ‘constexpr boost::fusion::vector<T>::vector() [with T = {boost::process::detail::exe_builder<char>, boost::process::detail::env_builder<char>}]’
/usr/include/boost/fusion/container/set/set.hpp:102:20: required from ‘constexpr boost::fusion::set<T>::set() [with T = {boost::process::detail::exe_builder<char>, boost::process::detail::env_builder<char>}]’
/export/home/pvanek/src/qore/module-process/src/boost/process/detail/execute_impl.hpp:257:15: required from ‘boost::process::child boost::process::detail::basic_execute_impl(Args&& ...) [with Char = char; Args = {boost::process::detail::exe_setter_<char>, boost::process::detail::arg_setter_<char, false>, boost::process::detail::env_init<char>, boost::process::detail::posix::start_dir_init<char>, QoreProcessHandler, boost::process::detail::posix::pipe_out<1, -1>, boost::process::detail::posix::pipe_out<2, -1>, boost::process::detail::posix::pipe_in}]’
/export/home/pvanek/src/qore/module-process/src/boost/process/detail/execute_impl.hpp:278:13: required from ‘boost::process::child boost::process::detail::execute_impl(Args&& ...) [with Args = {boost::process::detail::exe_setter_<char>, boost::process::detail::arg_setter_<char, false>, boost::process::detail::env_init<char>, boost::process::detail::posix::start_dir_init<char>, QoreProcessHandler, boost::process::detail::posix::pipe_out<1, -1>, boost::process::detail::posix::pipe_out<2, -1>, boost::process::detail::posix::pipe_in}]’
/export/home/pvanek/src/qore/module-process/src/boost/process/child.hpp:35:79: required from ‘boost::process::child::child(Args&& ...) [with Args = {boost::process::detail::exe_setter_<char>, boost::process::detail::arg_setter_<char, false>, boost::process::detail::env_init<char>, boost::process::detail::posix::start_dir_init<char>, QoreProcessHandler, boost::process::detail::posix::pipe_out<1, -1>, boost::process::detail::posix::pipe_out<2, -1>, boost::process::detail::posix::pipe_in}]’
/export/home/pvanek/src/qore/module-process/src/processpriv.cpp:73:34: required from here
/export/home/pvanek/src/qore/module-process/src/boost/process/env.hpp:302:8: error: conversion from ‘boost::process::basic_native_environment<char>’ to non-scalar type ‘boost::process::basic_environment<char>’ requested
struct env_builder
^
In file included from /usr/include/boost/fusion/container/vector/vector10.hpp:25:0,
from /usr/include/boost/fusion/view/transform_view/transform_view.hpp:22,
from /usr/include/boost/fusion/algorithm/transformation/transform.hpp:11,
from /export/home/pvanek/src/qore/module-process/src/boost/process/detail/posix/io_service_ref.hpp:16,
from /export/home/pvanek/src/qore/module-process/src/boost/process/async.hpp:42,
from /export/home/pvanek/src/qore/module-process/src/boost/process.hpp:23,
from /export/home/pvanek/src/qore/module-process/src/processpriv.h:5,
from /export/home/pvanek/src/qore/module-process/src/processpriv.cpp:4:
/usr/include/boost/fusion/container/vector/vector.hpp:131:24: note: synthesized method ‘constexpr boost::process::detail::env_builder<char>::env_builder()’ first required here
: elem() // value-initialized explicitly
^
CMakeFiles/process.dir/build.make:76: recipe for target 'CMakeFiles/process.dir/src/processpriv.cpp.o' failed
make[2]: *** [CMakeFiles/process.dir/src/processpriv.cpp.o] Error 1
CMakeFiles/Makefile2:131: recipe for target 'CMakeFiles/process.dir/all' failed
make[1]: *** [CMakeFiles/process.dir/all] Error 2
Makefile:149: recipe for target 'all' failed
make: *** [all] Error 2
I just cloned from the branch and tried the compile it. As of 1.61 it files like error_codes.hpp, jobs.hpp are missing.
The documentation says boost 1.62 isn't released yet but it has been released. Is the separate WinAPI still required or can this be used with the default 1.62 release?
Hi, I'm trying to use the develop branch against boost 1.55 using g++4.8 in c++11 mode.
I'm trying to compile a simple hello world:
#include <boost/process.hpp>
namespace bp = boost::process;
int main()
{
std::string cmd = "echo 'Hello, world!'";
bp::child child(
cmd,
bp::shell,
bp::std_in.close(),
bp::std_err > bp::null,
bp::std_out > bp::null
);
child.wait();
return child.exit_code();
}
But I get those errors:
$ g++ -std=c++11 test.cpp -I src -I /boost-1.55/include
In file included from src/boost/process/io.hpp:22:0,
from src/boost/process.hpp:30,
from test.cpp:1:
src/boost/process/detail/posix/null_out.hpp: In instantiation of ‘boost::process::detail::posix::null_out<p1, p2> boost::process::detail::std_out_<p1, p2>::operator>(const boost::process::detail::null_t&) const [with int p1 = 2; int p2 = -1]’:
test.cpp:13:27: required from here
src/boost/process/detail/posix/null_out.hpp:21:8: error: converting to ‘boost::process::detail::posix::file_descriptor’ from initializer list would use explicit constructor ‘boost::process::detail::posix::file_descriptor::file_descriptor(const char*, boost::process::detail::posix::file_descriptor::mode_t)’
struct null_out : handler_base_ext
^
In file included from src/boost/process.hpp:30:0,
from test.cpp:1:
src/boost/process/io.hpp:136:87: note: synthesized method ‘constexpr boost::process::detail::posix::null_out<2, -1>::null_out()’ first required here
api::null_out<p1,p2> operator>(const null_t &) const {return api::null_out<p1,p2>();}
^
In file included from src/boost/process/io.hpp:22:0,
from src/boost/process.hpp:30,
from test.cpp:1:
src/boost/process/detail/posix/null_out.hpp: In instantiation of ‘boost::process::detail::posix::null_out<p1, p2> boost::process::detail::std_out_<p1, p2>::operator>(const boost::process::detail::null_t&) const [with int p1 = 1; int p2 = -1]’:
test.cpp:14:27: required from here
src/boost/process/detail/posix/null_out.hpp:21:8: error: converting to ‘boost::process::detail::posix::file_descriptor’ from initializer list would use explicit constructor ‘boost::process::detail::posix::file_descriptor::file_descriptor(const char*, boost::process::detail::posix::file_descriptor::mode_t)’
struct null_out : handler_base_ext
^
In file included from src/boost/process.hpp:30:0,
from test.cpp:1:
src/boost/process/io.hpp:136:87: note: synthesized method ‘constexpr boost::process::detail::posix::null_out<1, -1>::null_out()’ first required here
api::null_out<p1,p2> operator>(const null_t &) const {return api::null_out<p1,p2>();}
Just a question. Does this REALLY have to use some of the latest developments in 1.63? Sure would be nice if we could stay boost 1.62 compatible. I would like to just use boost-all-dev package from Ubuntu.
Implement an ignore_error test, especially for linux
Not sure if this qualifies as an issue or feature request, but you are apparently an expert here. I'd ask it anyway.
Under Windows, I execute cmd.exe(the shell), and them inside it, I could run built-in command(e.g. set), programs and batch scripts in a row and mix them all in the same shell process.
Can I do this with boost process?
I am asking for this because the scripts and programs I am running in the same shell window relies on previous one changes to the system environment. If I run then one by one with, say, std::system(), a new process is created for each run, and the scripts/programs but the very first one would fail due to wrong/missing environment variables.
I can not change the scripts/programs, except for wrapping them in a certain way.
I read the documentation, it looks I could create a console process and use WriteConsoleInput() to append the scripts/programs, but it seems to be overkill. Does boost.process offer a simpler solution? The key is to share the same environment for my immediate usage.
It specifies that the posix items are in the boost::process::windows namespace, instead of the boost::process::posix namespace as the code suggests.
The warning note includes a link that is only relevant to the windows specific functionality. It should be a link to posix specific documentation for those items.
The Synchronous I/O example in the tutorial pipes the output of nm to c++filt. However, it is not specified that the output of c++filt should be made available in bp::ipstream is.
I've tried to build and run example/async_io.cpp
. I use a Hello World application as the child process, which terminates after displaying Hello World
. The example app (async_io.cpp
) hangs for ever.
I'm testing on Ubuntu 14.04 with g++ 4.8.5 and I'm using this library together with Boost 1.61.0.
I can see some of the headers in Boost.Process (for example, boost/process/cmd.hpp, boost/process/env.hpp) contain Windows line endings. Incorrect line endings have been released in Boost 1.64 packages for POSIX systems.
Please, convert line endings to POSIX. You can configure automatic conversion of line endings locally on Windows machines as described here: https://help.github.com/articles/dealing-with-line-endings/. Note that there is .gitattributes
at Boost root that provides file types for most of the content in Boost, so you don't have to do it yourself.
hi,
maybe it's expected behavior, but at least for me it's quite confusing. The child::exit_code() returns 0 after i call child::terminate().
Maybe it would be better to use something like eg. bp::status::terminated (maybe -1?), or maybe there can be somethign like bool child::terminated() I mean to have something to get info if the process was terminated from inside the program...
#include <boost/process.hpp>
namespace bp = boost::process;
int main(int argc, char** argv)
{
const char* command = "tee";
// arguments: simulate a situation when we don't know how
// many args will be used in real life
std::vector<std::string> a;
a.push_back("-");
boost::filesystem::path p = bp::search_path(command);
bp::environment e = boost::this_process::environment();
bp::child c(bp::exe = p.string(),
bp::args = a,
bp::env = e, // TODO/FIXME: later from options
bp::start_dir = "." // TODO/FIXME: later from options
);
c.wait_for(std::chrono::seconds(1));
c.terminate();
return c.exit_code();
}
the output is:
pvanek@spongebob:~/work/process> ./process
pvanek@spongebob:~/work/process> echo $?
0
hi,
is there some special reason for it?
http://klemens-morgenstern.github.io/process/boost/process/extend/windows_executor.html
vs.
http://klemens-morgenstern.github.io/process/boost/process/extend/posix_executor.html
I'm not sure if this is a bug or if I'm just doing something wrong...
Minimal example here: https://gist.github.com/mika-fischer/e308f93aa987a4d1d64b0024c7df316d
Basically, I'm spawning a child process with a pipe to the childs stdin and want to close this pipe, so that the child reading from stdin gets EOF. But this never happens...
docs say:
* The path must be a set of directories. Directories must be
* separated by colons on POSIX and by semicolons on Windows.
* If path is empty, the environment variable PATH is used.
but the default argument is of type:
const std::vector<boost::filesystem::path> path = ::boost::this_process::path()
When I post a path object with "foo:bar" for example (on linux) it is not split as docs say. I have to write my own "splitter" for it to fit std::vectorboost::filesystem::path. Maybe the docs string is from a legacy boost::process version...
Fair enough, but please update the docs (or provide another helper function to parse a string to get path components) ;)
I've managed to create a small example that reproduce the problem:
#include <boost/process.hpp>
#include <boost/asio.hpp>
#include <iostream>
namespace bp = boost::process;
int main()
{
// We will start 10 concurrent processes
std::vector<
std::pair<
std::shared_ptr<bp::child>,
std::future<std::string>
>
> processes;
processes.resize(10);
// Let's setup a simple timer
boost::asio::io_service io_service;
boost::posix_time::milliseconds interval(1000);
boost::asio::deadline_timer timer(io_service, interval);
std::function<void(boost::system::error_code const& ec)> update_cb;
update_cb = [&](boost::system::error_code const& ec) -> void {
if (ec) return;
for (auto& p: processes)
{
// The process can be launched
if (p.first == nullptr)
{
p.first = std::make_shared<bp::child>(
"/bin/sh", "-c", "echo \"hello, world!\"",
bp::std_in.close(),
bp::std_err.close(),
bp::std_out > p.second,
io_service
);
}
else if (!p.first->running()) // The process has finished
{
std::string out = p.second.get();
std::cout << "got result: '" << out << "'\n";
assert(!out.empty()); // <-- should never abort, but does...
p.first.reset();
}
}
timer.expires_at(timer.expires_at() + interval);
timer.async_wait(update_cb);
};
timer.async_wait(update_cb);
io_service.run();
return 0;
}
On my platform (centos 6), this is what I got:
$ ./a.out
got result: 'hello, world!
'
got result: 'hello, world!
'
got result: 'hello, world!
'
got result: 'hello, world!
'
got result: 'hello, world!
'
got result: 'hello, world!
'
got result: 'hello, world!
'
got result: ''
a.out: test.cpp:44: main()::__lambda24: Assertion `!out.empty()' failed.
Aborted (core dumped)
Did I do something wrong ?
Replace "asnyc.hpp" with "async.hpp" in the following line in "io.hpp":
\note `boost/process/asnyc.hpp` must also be included for this to work
Hi,
I have an issue with environment variables differing only by case in Windows.
If I understand correctly, the following code should spawn a child with a modified environment:
boost::process::environment env = boost::this_process::environment();
env["PATH"] += some_path;
boost::process::child child("program", env);
However, when the current process has an environment variable named "Path"
or "path"
, the child process is created with two different environment variables: the pre-existing one unmodified, and the new one. This causes many problems, for example with DLL search paths (it looks like Windows arbitrarily chooses one of the two variables and ignores the other one).
It is my understanding that Windows handles environment variables ignoring differences in capitalization, at least when using the standard functions (getenv, _putenv, GetEnvironmentVariable, SetEnvironmentVariable); however, CreateProcess doesn't perform case normalization in the passed environment block, so the burden is on the caller.
I noticed that boost::this_process::path() does a case-insensitive search, and I think boost::process::environment should do the same when matching variable names in Windows.
This issue was closed once with #21, however, I have the same problem now... g++5.4 / boost 1.56 / system;filesystem Boost 1.53.0
Is it possible to attach to an exiting process for monitoring purposes and perhaps even to send a terminate signal?
Thanks for working on this, BTW. It is a great library to have. Even if it cannot be everything to everybody.
I'm trying to use boost::process with some sample playground stuff. Unfortunately I'm not able to inject or even re-use current process env. I'm pretty sure I'm overlooking something... I just picked code fragments from docs/tutorials.
I'm using linux, bp commit 9db6449
#include <boost/process.hpp>
#include <iostream>
int main(int argc, char** argv)
{
namespace bp = boost::process; //we will assume this for all further examples
bp::environment e = boost::this_process::environment();
std::cout << "ENV size: " << e.size() << std::endl;
std::cout << "ENV PATH: " << e.at("PATH").to_string() << std::endl;
try {
int r = bp::system("g++ -v", e);
std::cout << "system: " << r << std::endl;
return r;
}
catch (const std::exception& e)
{
std::cout<< "exception:" << e.what() << std::endl;
return 1;
}
}
result/stdout is:
ENV size: 96
ENV PATH: /usr/lib64/qt5/bin:/usr/bin:/home/pvanek/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/games:/home/pvanek/src/drei/sepl-tools/scripts:/srv/qorus/bin
exception:execve failed: No such file or directory
on Posix error_code contains value = 0, category = "system", but message differs
Linux: "Success"
OS X: "Undefined error: 0"
is it expected behavior (getting these strings from system)?
Been waiting for a long time. Any chance it will be in boost-6.4?
The callback version for async_stdin doesn't seem to work, so I'd rather implement it in another way, which would look like this:
bp::async_writer aw;
execute("ding", std_in < aw );
aw.write("Dings");
May also work with boost::asio::streambuf, don't know if this is thread-safe though.
Is there some way to ensure that the child is terminated when the parent dies?
I have a process that shouldn't be left running after my main program dies. My first solution has been to wrap the child in a class that calls terminate in the destructor but this is insufficient as they aren't called on abnormal exit.
Using the code example (intro.cpp) the app never terminates. No problem with the asynchronous version though. It reads all the child's output and the child terminates, but the programs sill waits for read.
Call stack:
[External Code]
ProcessTest3.exe!boost::detail::winapi::ReadFile(void * hFile, void * lpBuffer, unsigned long nNumberOfBytesToWrite, unsigned long * lpNumberOfBytesWritten, boost::detail::winapi::_OVERLAPPED * lpOverlapped) Line 424 C++
ProcessTest3.exe!boost::process::detail::windows::basic_pipe<char,std::char_traits >::read(char * data, int count) Line 83 C++
ProcessTest3.exe!boost::process::basic_pipebuf<char,std::char_traits >::underflow() Line 186 C++
[External Code]
ProcessTest3.exe!main() Line 91 C++
[External Code]
std::string process = "...";
std::string work_folder = "...";
bp::ipstream pipe_stream;
bp::child child(process,
bp::start_dir(work_folder),
bp::std_out > pipe_stream,
bp::std_err > pipe_stream,
bp::on_exit([&pipe_stream](int exit, const std::error_code& ec)
{
std::cout << "exit: " << ec.message() << std::endl;
}));
std::cout << child.id() << std::endl;
for (std::string line; pipe_stream && std::getline(pipe_stream, line) && !line.empty(); line.clear())
std::cout << line << std::endl;
std::error_code ec;
child.wait(ec);
exit_code = child.exit_code();
Windows 10, boost-1.63.0 or boost-1.63.0-b1.
Implement the steal functions as ref-qualified member-functions
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.