将数据从 OpenCV C++传递到 NodeJS/JS |电子

Pass data from OpenCV C++ to NodeJS/JS | Electron

本文关键字:NodeJS JS 电子 数据 OpenCV C++      更新时间:2023-10-16

我正在尝试使用以下堆栈为视频处理应用程序做一个POC,并将处理后的媒体流从c ++应用程序传递到电子前端GUI。

Electron  
|
Nodejs
|
C++ Application

C++应用程序将读取IP/网络摄像头(仅使用OpenCV获取数据(并处理输入流(不使用OpenCV(。我正在尝试找出一种方法,以良好的fps将该流从C++发送到Electron GUI(NodeJS/JS(。现在我使用 node-gyp 编译了我的 C++ 应用程序并将其安装为节点包。

另外,我不想过多地更改我的C++应用程序(例如将OpenCV作为节点包包含在内(,因为稍后我将单独使用该C++应用程序与另一个应用程序集成。

挑战:

我们希望在单独的工作线程中执行繁重的提升代码,同时在执行期间将结果(流数据块(发送回主线程。

NAN(Native Abstractions for Node.js(已经提供了一种使用(AsyncProgressWorker(执行此操作的方法。

但是,我们无法知道在执行期间是否实际调用了 HandleProgressCallback 以发回我们的结果。当我们的运行时只是为了快速,因此永远不会执行回调时,就会发生这种情况。

建议的解决方案:

我们只是将我们的流输出收集在一个堆栈(StackCollect(中。我们尝试立即清除此堆栈并将流结果发送回主线程(如果可能( - (StackDrain(。如果我们没有时间立即清除堆栈,我们会在执行运行结束时耗尽(剩余的((HandleOKCallback(。

实现示例:

演示.cpp(我们的C++节点/电子插件(:

#include <nan.h>
#include <node.h>
#include <v8.h>
#include <iostream>
#include <string>
#include <vector>
#include <mutex>
#include <chrono>
#include <thread>
class vSync_File : public Nan::AsyncProgressWorker {
public:
~vSync_File();
vSync_File(Nan::Callback * result, Nan::Callback * chunk);
void Execute(const Nan::AsyncProgressWorker::ExecutionProgress& chunk);
void HandleOKCallback();
void HandleProgressCallback(const char *tout, size_t tout_size);
//needed for stream data collection
void StackCollect(std::string & str_chunk, const Nan::AsyncProgressWorker::ExecutionProgress& tchunk);
//drain stack
void StackDrain();
private:
Nan::Callback * chunk;
//stores stream data - use other data types for different output
std::vector<std::string> stack;
//mutex
std::mutex m;
};
vSync_File::vSync_File(Nan::Callback * result, Nan::Callback * chunk)
: Nan::AsyncProgressWorker(result), chunk(chunk) {}
vSync_File::~vSync_File() {
delete chunk;
}
void vSync_File::StackCollect(std::string & str_chunk, const Nan::AsyncProgressWorker::ExecutionProgress& tchunk) { 
std::lock_guard<std::mutex> guardme(m);
stack.push_back(str_chunk);
//attempt drain
std::string dummy = "NA";
tchunk.Send(dummy.c_str(), dummy.length());
}
//Dump out stream data
void vSync_File::StackDrain() {
std::lock_guard<std::mutex> guardme(m);
for (uint i = 0; i < stack.size(); i++) {
std::string th_chunk = stack[i];
v8::Local<v8::String> chk = Nan::New<v8::String>(th_chunk).ToLocalChecked();
v8::Local<v8::Value> argv[] = { chk };
chunk->Call(1, argv, this->async_resource);
}
stack.clear();
}
//Our main job in a nice worker thread
void vSync_File::Execute(const Nan::AsyncProgressWorker::ExecutionProgress& tchunk) {
//simulate some stream output
for (unsigned int i = 0; i < 20; i++) {
std::string out_chunk;
out_chunk = "Simulated stream data " + std::to_string(i);   
std::this_thread::sleep_for(std::chrono::milliseconds(300)); //so our HandleProgressCallback is invoked, otherwise we are too fast in our example here
this->StackCollect(out_chunk, tchunk);
}   
}
//Back at the main thread - if we have time stream back the output 
void vSync_File::HandleProgressCallback(const char *tout, size_t tout_size) {
Nan::HandleScope scope;
this->StackDrain();
}
//Back at the main thread - we are done
void vSync_File::HandleOKCallback () {
this->StackDrain(); //drain leftovers from stream stack
v8::Local<v8::String> result_mess = Nan::New<v8::String>("done reading").ToLocalChecked();
v8::Local<v8::Value> argv[] = { result_mess };
callback->Call(1, argv, this->async_resource);
}

NAN_METHOD(get_stream_data) {
Nan::Callback *result = new Nan::Callback(info[0].As<v8::Function>());
Nan::Callback *chunk = new Nan::Callback(info[1].As<v8::Function>());
AsyncQueueWorker(new vSync_File(result, chunk));
}

NAN_MODULE_INIT(Init) {
//we want stream data
Nan::Set(target, Nan::New<v8::String>("get_stream_data").ToLocalChecked(),
Nan::GetFunction(Nan::New<v8::FunctionTemplate>(get_stream_data)).ToLocalChecked());
}
NODE_MODULE(stream_c_electron, Init)

索引.js(电子实现示例(:

const stream_c_electron = require('./build/linux_x64/stream_c_electron.node');
stream_c_electron.get_stream_data(function(res) {
//we are done
console.log(res);
}, function(chk) {
console.log("a line streamed");
console.log(chk);
});

package.json:

{
"name": "stream_c_electron",
"version": "1.0.0",
"description": "stream from c++ node addon demo",
"main": "index.js",
"scripts": {
"start": "electron .",
"build_this": "HOME=~/.electron-gyp node-gyp rebuild --target=2.0.8 --arch=x64 --dist-url=https://atom.io/download/electron",
"test": "echo "Error: no test specified" && exit 1"
},
"author": "11AND2",
"license": "MIT",
"dependencies": {
"nan": "2.11.0"
},
"devDependencies": {
"electron": "2.0.8"
}
}

binding.gyp:

{
"targets": [
{
"target_name": "stream_c_electron",
"sources": [  "c_src/demo.cpp"  ],
"conditions": [
[
'OS=="linux"',
{
"cflags": ["-Wall", "-std=c++11"],
'product_dir' : 'linux_x64',
"include_dirs": [
"<!(node -e "require('nan')")"
]
}
]
]
}
]
}

您必须使用emscripten 将c++的东西编译为静态库,并通过import MyLib from "./MyLib";require将其加载并使用node --experimental-modules --napi-modules main.mjs运行。基本上这个想法是 V8 引擎能够读取您的本机代码。与纯JavaScript代码相比,它也非常快。

当您知道该怎么做时,这实际上很容易。请查看此示例代码。它基本上使用本地c++ libpng库进行javascript。唯一棘手的事情实际上是将c++javascript接口。

https://github.com/skanti/png-decoder-javascript/tree/devel