来自UDP插座的定期延迟峰值由周期性sendto()/recvFrom()延迟引起的Linux RT抢先系统的C

Periodic latency spikes from UDP socket caused by periodic sendto()/recvfrom() delay, C++ for Linux RT-PREEMPT system

本文关键字:延迟 recvFrom RT 系统 Linux 抢先 sendto 插座 UDP 周期性 来自      更新时间:2023-10-16

我已经设置了两个Raspberry PI,用于使用UDP插座,一个用作客户端,一个用作服务器。内核已用RT抢先(4.9.43-RT30 )修补。客户端充当服务器的回声,以允许计算往返延迟(RTL)。目前,在服务器端使用2个线程的发送频率:一个用于将消息发送给客户端,一个用于从客户端接收消息。这些线程设置为使用循环计划的时间表为95。

服务器构造了一个消息,其中包含发送消息的时间以及开始发送消息以来的过去时间。此消息将从服务器发送到客户端,然后立即返回服务器。从客户端接收到消息后,服务器计算往返延迟,然后将其存储在.txt文件中,用于使用Python绘制。

问题是,在分析图表时,我注意到RTL中有周期性的尖峰。图像的顶部图:rtl延迟和sendto() recvfrom()次。在传说中,我使用了RTT而不是RTL。这些尖峰与服务器端sendto()和recvFrom()调用中显示的尖峰直接相关。关于如何删除这些峰值的任何建议,因为我的应用程序非常依赖一致性?

我尝试并注意到的事情:

  1. 发送消息的大小没有效果。我尝试了较大的消息(1024个字节)和较小的消息(0字节),并且周期性延迟不会改变。这向我表明,这不是一个缓冲问题,因为没有任何填补?
  2. 发送消息的频率确实起着很大的作用,如果频率加倍,则延迟峰值的发生频率是两倍。然后,这表明有些东西正在填充,而在缩小sendto()/recvfrom()函数时会经历延迟?
  3. 用setSockop()更改缓冲区大小没有效果。
  4. 我已经尝试了其他许多设置(MSG_DONTWAIT等)。

我绝不是插座/C 编程/Linux专家,因此,当我没有想法时,给出的任何建议都将不胜感激。以下是用于创建套接字的代码,并启动用于发送和接收消息的服务器线程。下面是用于从服务器发送消息的代码,如果您需要其余的,请告诉我,但现在我担心的是sendto()函数引起的延迟。如果您需要其他任何需要,请告诉我。谢谢。

    thread_priority = priority;  
    recv_buff = recv_buff_len;
    std::cout << del << " Second start-up delay..." << std::endl;
    sleep(del);
    std::cout << "Delay complete..." << std::endl;
    master = socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP);
    // master socket creation
    if(master == 0){// Try to create the UDP socket
        perror("Could not create the socket: ");
        exit(EXIT_FAILURE);
    }    
    std::cout << "Master Socket Created..." << std::endl;
    std::cout << "Adjusting send and receive buffers..." << std::endl;
    setBuff();
    // Server address and port creation
    serv.sin_family = AF_INET;// Address family
    serv.sin_addr.s_addr = INADDR_ANY;// Server IP address, INADDR_ANY will                 
    work on the server side only
    serv.sin_port = htons(portNum);
    server_len = sizeof(serv);
    // Binding of master socket to specified address and port
    if (bind(master, (struct sockaddr *) &serv, sizeof (serv)) < 0) {
    //Attempt to bind master socket to address
        perror("Could not bind socket...");
        exit(EXIT_FAILURE);
    }
    // Show what address and port is being used
    char IP[INET_ADDRSTRLEN];                                 
    inet_ntop(AF_INET, &(serv.sin_addr), IP, INET_ADDRSTRLEN);// INADDR_ANY         
    allows all network interfaces so it will always show 0.0.0.0
    std::cout << "Listening on port: " << htons(serv.sin_port) << ", and         
    address: " << IP << "..." << std::endl;  
    // Options specific to the server RPi
    if(server){
        std::cout << "Run Time: " << duration << " seconds." << std::endl;
        client.sin_family = AF_INET;// Address family
        inet_pton(AF_INET, clientIP.c_str(), &(client.sin_addr));
        client.sin_port = htons(portNum);
        client_len = sizeof(client);
        serv_send = std::thread(&SocketServer::serverSend, this);  
        serv_send.detach();// The server send thread just runs continuously
        serv_receive = std::thread(&SocketServer::serverReceive, this);
        serv_receive.join();        
    }else{// Specific to client RPi
        SocketServer::clientReceiveSend();
    } 

和发送消息的代码:

    // Setup the priority of this thread
    param.sched_priority = thread_priority;
    int result = sched_setscheduler(getpid(), SCHED_RR, &param);
    if(result){
        perror ("The following error occurred while setting serverSend() priority");
    }
    int ched = sched_getscheduler(getpid());
    printf("serverSend() priority result %i : Scheduler priority id %i n", result, ched);
    std::ofstream Out;
    std::ofstream Out1;
    Out.open(file_name);
    Out << duration << std::endl; 
    Out << frequency << std::endl;
    Out << thread_priority << std::endl;
    Out.close(); 
    Out1.open("Server Side Send.txt");
    packets_sent = 0;
    Tbegin = std::chrono::high_resolution_clock::now();    
    // Send messages for a specified time period at a specified frequency
    while(!stop){ 
        // Setup the message to be sent
        Tstart = std::chrono::high_resolution_clock::now();
        TDEL = std::chrono::duration_cast< std::chrono::duration<double>>(Tstart - Tbegin); // Total time passed before sending message
        memcpy(&message[0], &Tstart, sizeof(Tstart));// Send the time the message was sent with the message
        memcpy(&message[8], &TDEL, sizeof(TDEL));// Send the time that had passed since Tstart
        // Send the message to the client
        T1 = std::chrono::high_resolution_clock::now();
        sendto(master, &message, 16, MSG_DONTWAIT, (struct sockaddr *)&client, client_len);  
        T2 = std::chrono::high_resolution_clock::now();
        T3 = std::chrono::duration_cast< std::chrono::duration<double>>(T2-T1);
        Out1 << T3.count() << std::endl;
        packets_sent++;
        // Pause so that the required message send frequency is met
        while(true){
            Tend = std::chrono::high_resolution_clock::now();
            Tdel = std::chrono::duration_cast< std::chrono::duration<double>>(Tend - Tstart);
            if(Tdel.count() > 1/frequency){
                break;
            }            
        }
        TDEL = std::chrono::duration_cast< std::chrono::duration<double>>(Tend - Tbegin);

        // Check to see if the program has run as long as required
        if(TDEL.count() > duration){
            stop = true;
            break;
        }        
    } 
    std::cout << "Exiting serverSend() thread..." << std::endl;   
    // Save extra results to the end of the last file    
    Out.open(file_name, std::ios_base::app);
    Out << packets_sent << "tt " << packets_returned << std::endl;    
    Out.close();   
    Out1.close();
    std::cout << "^C to exit..." << std::endl;

我已经解决了问题。它不是ARP表,因为即使在ARP功能上禁用了周期性的尖峰。由于禁用ARP功能,与一系列延迟尖峰相比,延迟中只有一个尖峰。

事实证明,我正在使用的线程是一个问题,因为CPU上只能一次处理一个线程。发送信息的一个线程受到接收信息的第二个线程的影响。我将主题优先级更改为很多(发送优先级高于接收,接收到高于发送和发送等于接收的优先级)无济于事。现在,我购买了一个有4个内核的Raspberry Pi,并且将发送线程设置为在Core 2上运行,而接收线在Core 3上运行,从而阻止了螺纹相互干扰。这不仅删除了延迟峰值,而且还减少了我的设置的平均延迟。