对象清理序列和atexit在Python模块中调用的函数

Sequence of object cleanup and functions called by atexit in Python module

本文关键字:模块 Python 调用 函数 atexit 对象      更新时间:2023-10-16

我正在使用boost Python将遗留的C++库与Python集成。遗留库有一些全局初始化,然后其中的类使用应用程序范围的数据。我需要确保在销毁所有封装的对象后调用遗留库的关闭函数,并认为这可以通过使用atexit注册关闭函数来实现。然而,我发现在atexit调用shutdown函数后,被包装的对象正在被清理,导致遗留库中出现多个segfault!

在退出之前,我可以通过在包装的对象上调用del来实现所需的行为,但希望将删除留给Python。我已经签出了对象文档中的红色警告框__del__,我想知道我的理想世界是否遥不可及。

在python模块中封装遗留代码时,有什么建议可以确保在清理完所有对象后调用关闭方法?

一些重要的平台细节:

  • Python 2.7.2
  • Visual Studio 2013
  • 64位构建

最小代码:

#include <iostream>
#include <boost/python.hpp>
using namespace std;
namespace legacy
{
    void initialize() { cout << "legacy::initialize" << endl; }
    void shutdown() { cout << "legacy::shutdown" << endl; }
    class Test
    {
    public:
        Test();
        virtual ~Test();
    };
    Test::Test() { }
    Test::~Test() { cout << "legacy::Test::~Test" << endl; }
}
BOOST_PYTHON_MODULE(legacy)
{
    using namespace boost::python;
    legacy::initialize();
    class_<legacy::Test>("Test");
    def("_finalize", &legacy::shutdown);
    object atexit = object(handle<>(PyImport_ImportModule("atexit")));
    object finalize = scope().attr("_finalize");
    atexit.attr("register")(finalize);
}

编译后,可以使用python运行,并显示以下输入和输出:

>>>导入旧版
旧式::初始化
>>>测试=遗留。测试()
>>>^Z
旧式::关闭
遗留::测试::~测试

简而言之,创建一个保护类型,在其构造函数和析构函数中初始化和关闭遗留库,然后通过每个公开对象中的智能指针管理保护。


有一些微妙的细节可能会使销毁过程变得困难:

  • Py_Finalize()中对象和模块中对象的破坏顺序是随机的
  • 没有模块的定稿。特别是,动态加载的扩展模块不会被卸载
  • 遗留API只有在所有使用它的对象都被销毁后才应该关闭。然而,对象本身可能不知道彼此

为了实现这一点,Boost.Python对象需要协调何时初始化和关闭遗留API。这些对象还需要对使用遗留API的遗留对象拥有所有权。使用单一责任原则,可以将责任划分为几个类别。

可以使用资源获取即初始化(RAII)习惯用法来初始化和关闭遗留AP。例如,对于以下legacy_api_guard,当构造legacy_api_guard对象时,它将初始化传统的API。当legacy_api_guard对象被破坏时,它将关闭传统的API。

/// @brief Guard that will initialize or shutdown the legacy API.
struct legacy_api_guard
{
  legacy_api_guard()  { legacy::initialize(); }
  ~legacy_api_guard() { legacy::shutdown();   }
};

由于多个对象需要共享何时初始化和关闭遗留API的管理,因此可以使用智能指针(如std::shared_ptr)来负责管理保护。以下示例延迟初始化并关闭遗留API:

/// @brief Global shared guard for the legacy API.
std::weak_ptr<legacy_api_guard> legacy_api_guard_;
/// @brief Get (or create) guard for legacy API.
std::shared_ptr<legacy_api_guard> get_api_guard()
{
  auto shared = legacy_api_guard_.lock();
  if (!shared)
  {
    shared = std::make_shared<legacy_api_guard>();
    legacy_api_guard_ = shared;
  }
  return shared;
}

最后,将嵌入Boost.Python对象的实际类型需要在创建遗留对象的实例之前获得遗留API保护的句柄。此外,在销毁时,应在销毁遗留对象后释放遗留API保护。实现这一点的一种非侵入性方法是在向Boost.Python公开遗留类型时使用提供自定义HeldType。在公开类型时,需要抑制默认的Boost.Pathon生成的初始值设定项,因为将使用自定义工厂函数来提供对对象创建的控制:

/// @brief legacy_object_holder is a smart pointer that will hold
///        legacy types and help guarantee the legacy API is initialized
///        while these objects are alive.  This smart pointer will remain
///        transparent to the legacy library and the user-facing Python.
template <typename T>
class legacy_object_holder
{
public:
  typedef T element_type;
  template <typename... Args>
  legacy_object_holder(Args&&... args)
    : legacy_guard_(::get_api_guard()),
      ptr_(std::make_shared<T>(std::forward<Args>(args)...))
  {}
  legacy_object_holder(legacy_object_holder& rhs) = default;
  element_type* get() const { return ptr_.get(); }
private:
  // Order of declaration is critical here.  The guard should be
  // allocated first, then the element.  This allows for the
  // element to be destroyed first, followed by the guard.
  std::shared_ptr<legacy_api_guard> legacy_guard_;
  std::shared_ptr<element_type> ptr_;
};
/// @brief Helper function used to extract the pointed to object from
///        an object_holder.  Boost.Python will use this through ADL.
template <typename T>
T* get_pointer(const legacy_object_holder<T>& holder)
{
  return holder.get();
}
/// Auxiliary function to make exposing legacy objects easier.
template <typename T, typename ...Args>
legacy_object_holder<T>* make_legacy_object(Args&&... args)
{
  return new legacy_object_holder<T>(std::forward<Args>(args)...);
}
BOOST_PYTHON_MODULE(example)
{
  namespace python = boost::python;
  python::class_<
      legacy::Test, legacy_object_holder<legacy::Test>, 
      boost::noncopyable>("Test", python::no_init)
    .def("__init__", python::make_constructor(
      &make_legacy_object<legacy::Test>))
    ;
}

以下是一个完整的示例,演示使用自定义HeldType以非侵入性的方式惰性地保护具有共享管理的资源:

#include <iostream> // std::cout, std::endl
#include <memory> // std::shared_ptr, std::weak_ptr
#include <boost/python.hpp>
/// @brief legacy namespace that cannot be changed.
namespace legacy {
void initialize() { std::cout << "legacy::initialize()" << std::endl; }
void shutdown()   { std::cout << "legacy::shutdown()" << std::endl;   }
class Test
{
public:
  Test()          { std::cout << "legacy::Test::Test()" << std::endl;  }
  virtual ~Test() { std::cout << "legacy::Test::~Test()" << std::endl; }
};
void use_test(Test&) {}
} // namespace legacy
namespace {
/// @brief Guard that will initialize or shutdown the legacy API.
struct legacy_api_guard
{
  legacy_api_guard()  { legacy::initialize(); }
  ~legacy_api_guard() { legacy::shutdown();   }
};
/// @brief Global shared guard for the legacy API.
std::weak_ptr<legacy_api_guard> legacy_api_guard_;
/// @brief Get (or create) guard for legacy API.
std::shared_ptr<legacy_api_guard> get_api_guard()
{
  auto shared = legacy_api_guard_.lock();
  if (!shared)
  {
    shared = std::make_shared<legacy_api_guard>();
    legacy_api_guard_ = shared;
  }
  return shared;
}
} // namespace 
/// @brief legacy_object_holder is a smart pointer that will hold
///        legacy types and help guarantee the legacy API is initialized
///        while these objects are alive.  This smart pointer will remain
///        transparent to the legacy library and the user-facing Python.
template <typename T>
class legacy_object_holder
{
public:
  typedef T element_type;
  template <typename... Args>
  legacy_object_holder(Args&&... args)
    : legacy_guard_(::get_api_guard()),
      ptr_(std::make_shared<T>(std::forward<Args>(args)...))
  {}
  legacy_object_holder(legacy_object_holder& rhs) = default;
  element_type* get() const { return ptr_.get(); }
private:
  // Order of declaration is critical here.  The guard should be
  // allocated first, then the element.  This allows for the
  // element to be destroyed first, followed by the guard.
  std::shared_ptr<legacy_api_guard> legacy_guard_;
  std::shared_ptr<element_type> ptr_;
};
/// @brief Helper function used to extract the pointed to object from
///        an object_holder.  Boost.Python will use this through ADL.
template <typename T>
T* get_pointer(const legacy_object_holder<T>& holder)
{
  return holder.get();
}
/// Auxiliary function to make exposing legacy objects easier.
template <typename T, typename ...Args>
legacy_object_holder<T>* make_legacy_object(Args&&... args)
{
  return new legacy_object_holder<T>(std::forward<Args>(args)...);
}
// Wrap the legacy::use_test function, passing the managed object.
void legacy_use_test_wrap(legacy_object_holder<legacy::Test>& holder)
{
  return legacy::use_test(*holder.get());
}
BOOST_PYTHON_MODULE(example)
{
  namespace python = boost::python;
  python::class_<
      legacy::Test, legacy_object_holder<legacy::Test>, 
      boost::noncopyable>("Test", python::no_init)
    .def("__init__", python::make_constructor(
      &make_legacy_object<legacy::Test>))
    ;
  python::def("use_test", &legacy_use_test_wrap);
}

交互式使用:

>>> import example
>>> test1 = example.Test()
legacy::initialize()
legacy::Test::Test()
>>> test2 = example.Test()
legacy::Test::Test()
>>> test1 = None
legacy::Test::~Test()
>>> example.use_test(test2)
>>> exit()
legacy::Test::~Test()
legacy::shutdown()

请注意,基本的总体方法也适用于非懒惰解决方案,在该解决方案中,遗留API在导入模块时初始化。需要使用shared_ptr而不是weak_ptr,并使用atexit.register():注册清理函数

/// @brief Global shared guard for the legacy API.
std::shared_ptr<legacy_api_guard> legacy_api_guard_;
/// @brief Get (or create) guard for legacy API.
std::shared_ptr<legacy_api_guard> get_api_guard()
{
  if (!legacy_api_guard_)
  {
    legacy_api_guard_ = std::make_shared<legacy_api_guard>();
  }
  return legacy_api_guard_;
}
void release_guard()
{
  legacy_api_guard_.reset();
}
...
BOOST_PYTHON_MODULE(example)
{
  // Boost.Python may throw an exception, so try/catch around
  // it to initialize and shutdown legacy API on failure.
  namespace python = boost::python;
  try
  {
    ::get_api_guard(); // Initialize.  
    ...
    // Register a cleanup function to run at exit.
    python::import("atexit").attr("register")(
      python::make_function(&::release_guard)
    );
  }
  // If an exception is thrown, perform cleanup and re-throw.
  catch (const python::error_already_set&)
  {
    ::release_guard();  
    throw;
  }
}

请参阅此处进行演示。