如何在MPI_send中发送集合对象

How to send a set object in MPI_Send

本文关键字:集合 对象 send MPI      更新时间:2023-10-16

我搜索发送一个set对象,发现最接近的是vector(它不同,不适用于set)。

如何在MPI_send中发送集合对象?(不使用boost库)任何人都可以举一个简单的例子?

无论您必须在MPI中将复杂的数据结构写入文件还是通过网络写入,问题都是一样的;您必须将数据提取到"Plain Old data"(POD)中,保存它,然后输出它,同样也能够将保存的数据解压缩到相同的结构中。通常,这被称为序列化。

对于任何给定的结构,你总是可以编写自己的例程来实现这一点,但在C++中,Boost中有一个名为Boost序列化库的框架可以实现这一目标;这个例子有点重,但它适用于大多数(所有?)STL容器,并且有一些钩子可以添加对自己类的支持。

使用Boost的主要技巧是,Boost库(以及所有示例)使将数据写入文件变得非常容易,但在这里,您希望将数据保存在内存中,并通过网络发送/接收数据;这意味着要跳过更多的环节来确保序列化进入一个可以访问的数组。SO的回答在这方面非常有帮助。

因此,一个完整的工作示例如下:

#include <mpi.h>
#include <set>
#include <string>
#include <boost/archive/binary_iarchive.hpp>
#include <boost/archive/binary_oarchive.hpp>
#include <boost/serialization/set.hpp>
#include <boost/iostreams/stream_buffer.hpp>
#include <boost/iostreams/stream.hpp>
#include <boost/iostreams/device/back_inserter.hpp>
int main(int argc,char** argv) {
    int size, rank;
    MPI_Init(&argc, &argv);
    MPI_Comm_size(MPI_COMM_WORLD, &size);
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);
    if (size < 2) {
        if (rank == 0)
            std::cerr << "Require at least 2 tasks" << std::endl;
        MPI_Abort(MPI_COMM_WORLD, 1);
    }
   const int lentag=0;
   const int datatag=1;
   if (rank == 0) {
        int nums[] = {1,4,9,16};
        std::set<int> send_set(nums, nums+4);
        std::cout << "Rank " << rank << " sending set: ";
        for (std::set<int>::iterator i=send_set.begin(); i!=send_set.end(); i++)
            std::cout << *i << " ";
        std::cout << std::endl;
        // We're going to serialize into a std::string of bytes, and then send this
        std::string serial_str;
        boost::iostreams::back_insert_device<std::string> inserter(serial_str);
        boost::iostreams::stream<boost::iostreams::back_insert_device<std::string> > s(inserter);
        boost::archive::binary_oarchive send_ar(s);
        send_ar << send_set;
        s.flush();
        int len = serial_str.size();
        // Send length, then data
        MPI_Send( &len, 1, MPI_INT, 1, lentag, MPI_COMM_WORLD );
        MPI_Send( (void *)serial_str.data(), len, MPI_BYTE, 1, datatag, MPI_COMM_WORLD );
    } else if (rank == 1) {
        int len;
        MPI_Recv( &len, 1, MPI_INT, 0, lentag, MPI_COMM_WORLD, MPI_STATUS_IGNORE);
        char data[len+1];
        MPI_Recv( data, len, MPI_BYTE, 0, datatag, MPI_COMM_WORLD, MPI_STATUS_IGNORE);
        data[len] = '';
        boost::iostreams::basic_array_source<char> device(data, len);
        boost::iostreams::stream<boost::iostreams::basic_array_source<char> > s(device);
        boost::archive::binary_iarchive recv_ar(s);
        std::set<int> recv_set;
        recv_ar >> recv_set;
        std::cout << "Rank " << rank << " got set: ";
        for (std::set<int>::iterator i=recv_set.begin(); i!=recv_set.end(); i++)
            std::cout << *i << " ";
        std::cout << std::endl;
    }
    MPI_Finalize();
    return 0;
}

运行给予:

$ mpic++ mpi-set.cxx -o mpiset -lboost_serialization
$ mpirun -np 2 ./mpiset
Rank 0 sending set: 1 4 9 16
Rank 1 got set: 1 4 9 16

如果你真的不想使用Boost,因为你实际上无法直接看到设置的数据结构,除了将数据提取到数组或向量中,并以这种方式发送数据外,别无选择:

#include <mpi.h>
#include <set>
#include <vector>
int main(int argc,char** argv) {
    int size, rank;
    MPI_Init(&argc, &argv);
    MPI_Comm_size(MPI_COMM_WORLD, &size);
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);
    if (size < 2) {
        if (rank == 0)
            std::cerr << "Require at least 2 tasks" << std::endl;
        MPI_Abort(MPI_COMM_WORLD, 1);
    }
   const int lentag=0;
   const int datatag=1;
   if (rank == 0) {
        int nums[] = {1,4,9,16};
        std::set<int> send_set(nums, nums+4);
        std::cout << "Rank " << rank << " sending set: ";
        for (std::set<int>::iterator i=send_set.begin(); i!=send_set.end(); i++)
            std::cout << *i << " ";
        std::cout << std::endl;
        // Send length, then data
        int len = send_set.size();
        MPI_Send( &len, 1, MPI_INT, 1, lentag, MPI_COMM_WORLD );
        std::vector<int> send_vec;
        for (std::set<int>::iterator i=send_set.begin(); i!=send_set.end(); i++) 
            send_vec.push_back(*i);
        MPI_Send( send_vec.data(), len, MPI_INT, 1, datatag, MPI_COMM_WORLD );
    } else if (rank == 1) {
        int len;
        MPI_Recv( &len, 1, MPI_INT, 0, lentag, MPI_COMM_WORLD, MPI_STATUS_IGNORE);
        int recv_data[len];
        MPI_Recv( recv_data, len, MPI_INT, 0, datatag, MPI_COMM_WORLD, MPI_STATUS_IGNORE);
        std::set<int> recv_set;
        for (int i=0; i<len; i++) 
            recv_set.insert(recv_data[i]);
        std::cout << "Rank " << rank << " got set: ";
        for (std::set<int>::iterator i=recv_set.begin(); i!=recv_set.end(); i++)
            std::cout << *i << " ";
        std::cout << std::endl;
    }
    MPI_Finalize();
    return 0;
}

运行给出:

$ mpicxx -o mpisetvector mpi-set-vector.cxx 
$ mpirun -np 2 mpisetvector
Rank 0 sending set: 1 4 9 16 
Rank 1 got set: 1 4 9 16 

但实际上,如果你也要对其他类型的对象进行此操作,Boost是最好的选择。