Python Coverage for C++ PyImport

Python Coverage for C++ PyImport

本文关键字:PyImport C++ for Coverage Python      更新时间:2023-10-16

情况:
我正在尝试获取当前项目中所有python代码的覆盖率报告。我在很大程度上利用了 Coverage.py,取得了巨大的成功。 目前,我正在利用 sitecustomize.py 过程像这样使用它。对于从命令行启动的所有内容,它的效果都很棒。

问题:
我无法通过 PyImport_Import(( 类型语句从C++运行 python 模块来实际跟踪和输出覆盖率数据。

例:

[测试.cpp]

#include <stdio.h>
#include <iostream>
#include <Python.h>
int main()
{
Py_Initialize();
PyObject* sysPath = PySys_GetObject("path");
PyList_Append(sysPath, PyString_FromString("."));
// Load the module
PyObject *pName = PyString_FromString("test_mod");
PyObject *pModule = PyImport_Import(pName);
if (pModule != NULL) {
std::cout << "Python module foundn";
// Load all module level attributes as a dictionary
PyObject *pDict = PyModule_GetDict(pModule);
PyObject *pFunc = PyObject_GetAttrString(pModule, "getInteger");
if(pFunc)
{
if(PyCallable_Check(pFunc))
{
PyObject *pValue = PyObject_CallObject(pFunc, NULL);
std::cout << PyLong_AsLong(pValue) << std::endl;
}
else
{
printf("ERROR: function getInteger()n");
}
}
else
{
printf("ERROR: pFunc is NULLn");
}
}
else
std::cout << "Python Module not foundn";
return 0;
}

[test_mod.py]

#!/bin/python
def getInteger():
print('Python function getInteger() called')
c = 100*50/30
return c
print('Randomness')

输出:
如果我手动运行test_mod.py它会按预期输出。但是,如果我运行编译的测试.cpp二进制文件,它不会输出任何覆盖率数据。我知道 sitecustomize.py 仍然受到打击,因为我添加了一些调试以确保我不会发疯。我还可以在覆盖率调试日志中看到它确实想要跟踪模块。

[.log]

New process: executable: /usr/bin/python
New process: cmd: ???
New process: parent pid: 69073
-- config ----------------------------------------------------
_include: None
_omit: None
attempted_config_files: /tmp/.coveragerc
branch: True
concurrency: thread
multiprocessing
config_files: /tmp/.coveragerc
cover_pylib: False
data_file: /tmp/python_data/.coverage
debug: process
trace
sys
config
callers
dataop
dataio
disable_warnings: -none-
exclude_list: #s*(pragma|PRAGMA)[:s]?s*(no|NO)s*(cover|COVER)
extra_css: None
fail_under: 0.0
html_dir: htmlcov
html_title: Coverage report
ignore_errors: False
note: None
New Section 1 Page 2note: None
parallel: True
partial_always_list: while (True|1|False|0):
if (True|1|False|0):
partial_list: #s*(pragma|PRAGMA)[:s]?s*(no|NO)s*(branch|BRANCH)
paths: {'source': ['/tmp/python_source', '/opt/test']}
plugin_options: {}
plugins: -none-
precision: 0
report_include: None
report_omit: None
run_include: None
run_omit: None
show_missing: False
skip_covered: False
source: /opt/test/
timid: False
xml_output: coverage.xml
xml_package_depth: 99
-- sys -------------------------------------------------------
version: 4.5.4
coverage: /usr/lib64/python2.7/site-packages/coverage/__init__.pyc
cover_paths: /usr/lib64/python2.7/site-packages/coverage
pylib_paths: /usr/lib64/python2.7
tracer: PyTracer
plugins.file_tracers: -none-
plugins.configurers: -none-
config_files: /tmp/.coveragerc
configs_read: /tmp/.coveragerc
data_path: /tmp/python_data/.coverage
python: 2.7.5 (default, Jun 11 2019, 14:33:56) [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]
platform: Linux-3.10.0-1062.el7.x86_64-x86_64-with-redhat-7.7-Maipo
implementation: CPython
executable: /usr/bin/python
cwd: /opt/test
path: /usr/lib64/python27.zip
/usr/lib64/python2.7
/usr/lib64/python2.7/plat-linux2
/usr/lib64/python2.7/lib-tk
/usr/lib64/python2.7/lib-old
/usr/lib64/python2.7/lib-dynload
/usr/lib64/python2.7/site-packages
environment: COVERAGE_DEBUG = process,trace,sys,config,callers,dataop,dataio
COVERAGE_DEBUG_FILE = /tmp/cov.log
COVERAGE_PROCESS_START = /tmp/.coveragerc
command_line: ???
source_match: /opt/test
source_pkgs_match: -none-
include_match: -none-
omit_match: -none-
cover_match: -none-
pylib_match: -none-
-- end -------------------------------------------------------
<module> : /usr/lib64/python2.7/site.py @556
New Section 1 Page 3<module> : /usr/lib64/python2.7/site.py @556
main : /usr/lib64/python2.7/site.py @539
addsitepackages : /usr/lib64/python2.7/site.py @317
addsitedir : /usr/lib64/python2.7/site.py @190
addpackage : /usr/lib64/python2.7/site.py @152
<module> : <string> @1
process_startup : /usr/lib64/python2.7/site-packages/coverage/control.py @1289
start : /usr/lib64/python2.7/site-packages/coverage/control.py @690
_init : /usr/lib64/python2.7/site-packages/coverage/control.py @362
_write_startup_debug : /usr/lib64/python2.7/site-packages/coverage/control.py @382
write_formatted_info : /usr/lib64/python2.7/site-packages/coverage/debug.py @120
Not tracing '/usr/lib64/python2.7/threading.py': falls outside the --source trees
<module> : /usr/lib64/python2.7/site.py @556
main : /usr/lib64/python2.7/site.py @539
addsitepackages : /usr/lib64/python2.7/site.py @317
addsitedir : /usr/lib64/python2.7/site.py @190
addpackage : /usr/lib64/python2.7/site.py @152
<module> : <string> @1
process_startup : /usr/lib64/python2.7/site-packages/coverage/control.py @1289
start : /usr/lib64/python2.7/site-packages/coverage/control.py @701
start : /usr/lib64/python2.7/site-packages/coverage/collector.py @318
settrace : /usr/lib64/python2.7/threading.py @99
_trace : /usr/lib64/python2.7/site-packages/coverage/pytracer.py @111
_should_trace : /usr/lib64/python2.7/site-packages/coverage/control.py @593

[... Not tracing a bunch of common python code ...]

Tracing './test_mod.py'
<module> : ./test_mod.py @3
_trace : /usr/lib64/python2.7/site-packages/coverage/pytracer.py @111
_should_trace : /usr/lib64/python2.7/site-packages/coverage/control.py @593

我使用您的代码重现了该问题,而您只是忘记调用 Py_Finalize((。因此,在收集数据时,永远不会生成报告。

它适用于以下代码段:

#include <stdio.h>
#include <iostream>
#include <Python.h>
int main()
{
Py_Initialize();
PyEval_InitThreads();
PyObject* sysPath = PySys_GetObject("path");
PyList_Append(sysPath, PyString_FromString("."));
// Load the module
PyObject *pName = PyString_FromString("test_mod");
PyObject *pModule = PyImport_Import(pName);
if (pModule != NULL) {
std::cout << "Python module foundn";
// Load all module level attributes as a dictionary
PyObject *pDict = PyModule_GetDict(pModule);
PyObject *pFunc = PyObject_GetAttrString(pModule, "getInteger");
if(pFunc)
{
if(PyCallable_Check(pFunc))
{
PyObject *pValue = PyObject_CallObject(pFunc, NULL);
std::cout << PyLong_AsLong(pValue) << std::endl;
}
else
{
printf("ERROR: function getInteger()n");
}
}
else
{
printf("ERROR: pFunc is NULLn");
}
}
else
std::cout << "Python Module not foundn";
Py_Finalize();
return 0;

PyObject *PySys_GetObject(char *name( 返回借用的引用。引用计数不是应该增加吗?怎么样:

// ...
PyObject* sysPath = PySys_GetObject("path");
Py_INCREF(sysPath);
PyList_Append(sysPath, PyString_FromString("."));
Py_DECREF(sysPath);
// sysPath = NULL;
// ...

我自己才刚刚开始使用 Python-C API,但我的理解是导入模块实际上并没有将它们添加到您的主模块中。 您需要单独执行此操作。 我不确定这是否有助于解决您的问题,但我有效的方法(减去错误检查(如下:

// Initialize main module
PyObject* mainModule = PyImport_AddModule("__main__");;
//  Initialize module to be added
PyObject* moduleNamePyObject= PyUnicode_DecodeFSDefault("moduleName");
PyImport_Import(moduleNamePyObject);
// Add module to main module
PyObject_SetAttrString(mainModulePtr, "moduleName", modulePyObject);

通常,在导入模块时,Python 会尝试在导入模块(包含 import 语句的模块(旁边查找模块文件。然后,Python 尝试 "sys.path" 中的目录。通常不考虑当前工作目录。在我们的例子中,导入是通过 API 执行的,因此没有导入模块的目录 Python 可以搜索"test_mod.py"。该插件也不在"sys.path"上。使 Python 能够找到插件的一种方法是通过 API 执行等效的"sys.path.append('.'(",将当前工作目录添加到模块搜索路径中。

Py_Initialize();
PyObject* sysPath = PySys_GetObject((char*)"path");
PyObject* programName = PyString_FromString(<DIRECTORY>.c_str());
PyList_Append(sysPath, programName);
Py_DECREF(programName);

如果你使用的是python3,

PyString_FromString更改为PyUnicode_FromString.

来源:

https://realmike.org/blog/2012/07/08/embedding-python-tutorial-part-1/

Python 嵌入:PyImport_Import不是来自当前目录