boost.python c++ multithreading
boost.python c++ multithreading
我正在编写一个包括C 模块的Python程序(.so
,使用boost.python
)。
我正在启动几个运行C 功能的Python线程。
这就是C 代码的样子:
#include <boost/python.hpp>
using namespace boost;
void f(){
// long calculation
// call python function
// long calculation
}
BOOST_PYTHON_MODULE(test)
{
python::def("f", &f);
}
和Python代码:
from test import f
t1 = threading.Thread(target=f)
t1.setDaemon(True)
t1.start()
print "Still running!"
我遇到一个问题:"仍在运行!"消息未显示,我发现C 线程持有GIL。
在我从Python代码运行C 代码的情况下,处理GIL的最佳方法是什么?
谢谢!gal
我经常发现,使用RAII风格的类来管理全局解释器锁(GIL)提供了一个优雅的异常安全解决方案。
例如,使用以下with_gil
类,当创建with_gil
对象时,调用线程将获取GIL。当with_gil
对象破坏时,它会恢复GIL状态。
/// @brief Guard that will acquire the GIL upon construction, and
/// restore its state upon destruction.
class with_gil
{
public:
with_gil() { state_ = PyGILState_Ensure(); }
~with_gil() { PyGILState_Release(state_); }
with_gil(const with_gil&) = delete;
with_gil& operator=(const with_gil&) = delete;
private:
PyGILState_STATE state_;
};
和互补的without_gil
类也相反:
/// @brief Guard that will unlock the GIL upon construction, and
/// restore its staet upon destruction.
class without_gil
{
public:
without_gil() { state_ = PyEval_SaveThread(); }
~without_gil() { PyEval_RestoreThread(state_); }
without_gil(const without_gil&) = delete;
without_gil& operator=(const without_gil&) = delete;
private:
PyThreadState* state_;
};
它们在函数中的用法可能如下:
void f()
{
without_gil no_gil; // release gil
// long calculation
...
{
with_gil gil; // acquire gil
// call python function
...
} // restore gil (release)
// long calculation
...
} // restore gil (acquire)
也可以使用更高级别的方便类来提供std::lock_guard
之类的经验。GIL获取和释放,保存和恢复语义与普通静音略有不同。因此,gil_guard
接口不同:
-
gil_guard.acquire()
将获得GIL -
gil_guard.release()
将发布GIL -
gil_guard_restore()
将还原以前的状态
/// @brief Guard that provides higher-level GIL controls.
class gil_guard
{
public:
struct no_acquire_t {} // tag type used for gil acquire strategy
static no_acquire;
gil_guard() { acquire(); }
gil_guard(no_acquire_t) { release(); }
~gil_guard() { while (!stack_.empty()) { restore(); } }
void acquire() { stack_.emplace(new with_gil); }
void release() { stack_.emplace(new without_gil); }
void restore() { stack_.pop(); }
static bool owns_gil()
{
// For Python 3.4+, one can use `PyGILState_Check()`.
return _PyThreadState_Current == PyGILState_GetThisThreadState();
}
gil_guard(const gil_guard&) = delete;
gil_guard& operator=(const gil_guard&) = delete;
private:
// Use std::shared_ptr<void> for type erasure.
std::stack<std::shared_ptr<void>> stack_;
};
的用法将是:
void f()
{
gil_guard gil(gil_guard::no_acquire); // release gil
// long calculation
...
gil.acquire(); // acquire gil
// call python function
...
gil.restore(); // restore gil (release)
// long calculation
...
} // restore gil (acquire)
这是一个完整的示例,展示了这些辅助类别的GIL管理:
#include <cassert>
#include <iostream> // std::cout, std::endl
#include <memory> // std::shared_ptr
#include <thread> // std::this_thread
#include <stack> // std::stack
#include <boost/python.hpp>
/// @brief Guard that will acquire the GIL upon construction, and
/// restore its state upon destruction.
class with_gil
{
public:
with_gil() { state_ = PyGILState_Ensure(); }
~with_gil() { PyGILState_Release(state_); }
with_gil(const with_gil&) = delete;
with_gil& operator=(const with_gil&) = delete;
private:
PyGILState_STATE state_;
};
/// @brief Guard that will unlock the GIL upon construction, and
/// restore its staet upon destruction.
class without_gil
{
public:
without_gil() { state_ = PyEval_SaveThread(); }
~without_gil() { PyEval_RestoreThread(state_); }
without_gil(const without_gil&) = delete;
without_gil& operator=(const without_gil&) = delete;
private:
PyThreadState* state_;
};
/// @brief Guard that provides higher-level GIL controls.
class gil_guard
{
public:
struct no_acquire_t {} // tag type used for gil acquire strategy
static no_acquire;
gil_guard() { acquire(); }
gil_guard(no_acquire_t) { release(); }
~gil_guard() { while (!stack_.empty()) { restore(); } }
void acquire() { stack_.emplace(new with_gil); }
void release() { stack_.emplace(new without_gil); }
void restore() { stack_.pop(); }
static bool owns_gil()
{
// For Python 3.4+, one can use `PyGILState_Check()`.
return _PyThreadState_Current == PyGILState_GetThisThreadState();
}
gil_guard(const gil_guard&) = delete;
gil_guard& operator=(const gil_guard&) = delete;
private:
// Use std::shared_ptr<void> for type erasure.
std::stack<std::shared_ptr<void>> stack_;
};
void f()
{
std::cout << "in f()" << std::endl;
// long calculation
gil_guard gil(gil_guard::no_acquire);
assert(!gil.owns_gil());
std::this_thread::sleep_for(std::chrono::milliseconds(500));
std::cout << "calculating without gil..." << std::endl;
// call python function
gil.acquire();
assert(gil.owns_gil());
namespace python = boost::python;
python::object print =
python::import("__main__").attr("__builtins__").attr("print");
print(python::str("calling a python function"));
gil.restore();
// long calculation
assert(!gil.owns_gil());
std::cout << "calculating without gil..." << std::endl;
}
BOOST_PYTHON_MODULE(example)
{
// Force the GIL to be created and initialized. The current caller will
// own the GIL.
PyEval_InitThreads();
namespace python = boost::python;
python::def("f", +[] {
// For exposition, assert caller owns GIL before and after
// invoking function `f()`.
assert(gil_guard::owns_gil());
f();
assert(gil_guard::owns_gil());
});
}
交互式用法:
>>> import threading
>>> import example
>>> t1 = threading.Thread(target=example.f)
>>> t1.start(); print "Still running"
in f()
Still running
calculating without gil...
calling a python function
calculating without gil...
>>> t1.join()
相关文章:
- Multithreading unordered_map
- boost.python c++ multithreading
- Multithreading Visual Studio C++
- C++ multithreading
- Linux gnu++11,在运行时获取"Enable multithreading to use std::thread: Operation not permitted"
- Nginx + fastcgi multithreading
- Multithreading, OpenMP,C
- Error C3867 Multithreading C++
- gSOAP Multithreading
- QuantLib Multithreading/Concurrecy
- MFC multithreading: AfxBeginThread vs Boost.Thread?
- openMP multithreading c++
- c/c++ epoll multithreading
- OpenCV C++ Multithreading