我可以报告开放mp任务的进度吗?
Can I report progress for openmp tasks?
想象一个经典的OMP任务:
- 对 [0.0, 1.0] 范围内的大双精度向量求和
住在科里鲁
using namespace std;
int main() {
vector<double> v;
// generate some data
generate_n(back_inserter(v), 1ul << 18,
bind(uniform_real_distribution<double>(0,1.0), default_random_engine { random_device {}() }));
long double sum = 0;
{
#pragma omp parallel for reduction(+:sum)
for(size_t i = 0; i < v.size(); i++)
{
sum += v[i];
}
}
std::cout << "Done: sum = " << sum << "n";
}
我很难想出如何报告进度的想法。毕竟,OMP 正在为我处理团队线程之间的所有协调,我没有全局状态。
我可能会使用常规std::thread
并从那里观察一些共享变量,但是没有一种更"omp-ish"的方法来实现这一点吗?
在没有本机原子支持的处理器上(即使有它们),使用 #pragma omp atomic
,正如此处的其他答案所表明的那样,可能会减慢程序速度。
进度指示器的想法是让用户了解何时完成某事。如果您的目标正负总运行时间的一小部分,则用户不会太烦恼。也就是说,用户希望事情更快地完成,但代价是更准确地知道事情何时完成。
出于这个原因,我通常只跟踪单个线程的进度,并使用它来估计总进度。这对于每个线程具有类似工作负载的情况来说很好。由于您使用的是#pragma omp parallel for
,因此您可能正在处理一系列没有相互依赖关系的类似元素,因此我的假设可能对您的用例有效。
我已将此逻辑包装在类ProgressBar
中,我通常将其包含在头文件中,以及它的辅助类Timer
。该类使用 ANSI 控制信号来保持外观美观。
输出如下所示:
[====== ] (12% - 22.0s - 4 threads)
让编译器通过声明 -DNOPROGRESS
编译标志来消除进度栏的所有开销也很容易。
代码和示例用法如下:
#include <iostream>
#include <chrono>
#include <thread>
#include <iomanip>
#include <stdexcept>
#ifdef _OPENMP
///Multi-threading - yay!
#include <omp.h>
#else
///Macros used to disguise the fact that we do not have multithreading enabled.
#define omp_get_thread_num() 0
#define omp_get_num_threads() 1
#endif
///@brief Used to time how intervals in code.
///
///Such as how long it takes a given function to run, or how long I/O has taken.
class Timer{
private:
typedef std::chrono::high_resolution_clock clock;
typedef std::chrono::duration<double, std::ratio<1> > second;
std::chrono::time_point<clock> start_time; ///< Last time the timer was started
double accumulated_time; ///< Accumulated running time since creation
bool running; ///< True when the timer is running
public:
Timer(){
accumulated_time = 0;
running = false;
}
///Start the timer. Throws an exception if timer was already running.
void start(){
if(running)
throw std::runtime_error("Timer was already started!");
running=true;
start_time = clock::now();
}
///Stop the timer. Throws an exception if timer was already stopped.
///Calling this adds to the timer's accumulated time.
///@return The accumulated time in seconds.
double stop(){
if(!running)
throw std::runtime_error("Timer was already stopped!");
accumulated_time += lap();
running = false;
return accumulated_time;
}
///Returns the timer's accumulated time. Throws an exception if the timer is
///running.
double accumulated(){
if(running)
throw std::runtime_error("Timer is still running!");
return accumulated_time;
}
///Returns the time between when the timer was started and the current
///moment. Throws an exception if the timer is not running.
double lap(){
if(!running)
throw std::runtime_error("Timer was not started!");
return std::chrono::duration_cast<second> (clock::now() - start_time).count();
}
///Stops the timer and resets its accumulated time. No exceptions are thrown
///ever.
void reset(){
accumulated_time = 0;
running = false;
}
};
///@brief Manages a console-based progress bar to keep the user entertained.
///
///Defining the global `NOPROGRESS` will
///disable all progress operations, potentially speeding up a program. The look
///of the progress bar is shown in ProgressBar.hpp.
class ProgressBar{
private:
uint32_t total_work; ///< Total work to be accomplished
uint32_t next_update; ///< Next point to update the visible progress bar
uint32_t call_diff; ///< Interval between updates in work units
uint32_t work_done;
uint16_t old_percent; ///< Old percentage value (aka: should we update the progress bar) TODO: Maybe that we do not need this
Timer timer; ///< Used for generating ETA
///Clear current line on console so a new progress bar can be written
void clearConsoleLine() const {
std::cerr<<"r 33[2K"<<std::flush;
}
public:
///@brief Start/reset the progress bar.
///@param total_work The amount of work to be completed, usually specified in cells.
void start(uint32_t total_work){
timer = Timer();
timer.start();
this->total_work = total_work;
next_update = 0;
call_diff = total_work/200;
old_percent = 0;
work_done = 0;
clearConsoleLine();
}
///@brief Update the visible progress bar, but only if enough work has been done.
///
///Define the global `NOPROGRESS` flag to prevent this from having an
///effect. Doing so may speed up the program's execution.
void update(uint32_t work_done0){
//Provide simple way of optimizing out progress updates
#ifdef NOPROGRESS
return;
#endif
//Quick return if this isn't the main thread
if(omp_get_thread_num()!=0)
return;
//Update the amount of work done
work_done = work_done0;
//Quick return if insufficient progress has occurred
if(work_done<next_update)
return;
//Update the next time at which we'll do the expensive update stuff
next_update += call_diff;
//Use a uint16_t because using a uint8_t will cause the result to print as a
//character instead of a number
uint16_t percent = (uint8_t)(work_done*omp_get_num_threads()*100/total_work);
//Handle overflows
if(percent>100)
percent=100;
//In the case that there has been no update (which should never be the case,
//actually), skip the expensive screen print
if(percent==old_percent)
return;
//Update old_percent accordingly
old_percent=percent;
//Print an update string which looks like this:
// [================================================ ] (96% - 1.0s - 4 threads)
std::cerr<<"r 33[2K["
<<std::string(percent/2, '=')<<std::string(50-percent/2, ' ')
<<"] ("
<<percent<<"% - "
<<std::fixed<<std::setprecision(1)<<timer.lap()/percent*(100-percent)
<<"s - "
<<omp_get_num_threads()<< " threads)"<<std::flush;
}
///Increment by one the work done and update the progress bar
ProgressBar& operator++(){
//Quick return if this isn't the main thread
if(omp_get_thread_num()!=0)
return *this;
work_done++;
update(work_done);
return *this;
}
///Stop the progress bar. Throws an exception if it wasn't started.
///@return The number of seconds the progress bar was running.
double stop(){
clearConsoleLine();
timer.stop();
return timer.accumulated();
}
///@return Return the time the progress bar ran for.
double time_it_took(){
return timer.accumulated();
}
uint32_t cellsProcessed() const {
return work_done;
}
};
int main(){
ProgressBar pg;
pg.start(100);
//You should use 'default(none)' by default: be specific about what you're
//sharing
#pragma omp parallel for default(none) schedule(static) shared(pg)
for(int i=0;i<100;i++){
pg.update(i);
std::this_thread::sleep_for(std::chrono::seconds(1));
}
}
只需让团队中的每个线程跟踪本地进度并以原子方式更新全局计数器即可。您仍然可以让另一个线程观察它,或者,如下面的示例所示,您可以在 OMP 关键部分中执行终端输出。
这里的关键是调整不会导致高度频繁更新的步长,因为这样锁定关键区域(以及在较小程度上原子负载/存储)会降低性能。
住在科里鲁
#include <omp.h>
#include <vector>
#include <random>
#include <algorithm>
#include <iterator>
#include <functional>
#include <iostream>
#include <iomanip>
using namespace std;
int main() {
vector<double> v;
// generate some data
generate_n(back_inserter(v), 1ul << 18, bind(uniform_real_distribution<double>(0,1.0), default_random_engine { random_device {}() }));
auto step_size = 100ul;
auto total_steps = v.size() / step_size + 1;
size_t steps_completed = 0;
long double sum = 0;
#pragma omp parallel
{
size_t local_count = 0;
#pragma omp for reduction(+:sum)
for(size_t i = 0; i < v.size(); i++)
{
sum += v[i];
if (local_count++ % step_size == step_size-1)
{
#pragma omp atomic
++steps_completed;
if (steps_completed % 100 == 1)
{
#pragma omp critical
std::cout << "Progress: " << steps_completed << " of " << total_steps << " (" << std::fixed << std::setprecision(1) << (100.0*steps_completed/total_steps) << "%)n";
}
}
}
}
std::cout << "Done: sum = " << sum << "n";
}
最后,打印结果。输出:
Progress: 1 of 2622 (0.0%)
Progress: 191 of 2622 (7.3%)
Progress: 214 of 2622 (8.2%)
Progress: 301 of 2622 (11.5%)
Progress: 401 of 2622 (15.3%)
Progress: 501 of 2622 (19.1%)
Progress: 601 of 2622 (22.9%)
Progress: 701 of 2622 (26.7%)
Progress: 804 of 2622 (30.7%)
Progress: 901 of 2622 (34.4%)
Progress: 1003 of 2622 (38.3%)
Progress: 1101 of 2622 (42.0%)
Progress: 1201 of 2622 (45.8%)
Progress: 1301 of 2622 (49.6%)
Progress: 1402 of 2622 (53.5%)
Progress: 1501 of 2622 (57.2%)
Progress: 1601 of 2622 (61.1%)
Progress: 1701 of 2622 (64.9%)
Progress: 1801 of 2622 (68.7%)
Progress: 1901 of 2622 (72.5%)
Progress: 2001 of 2622 (76.3%)
Progress: 2101 of 2622 (80.1%)
Progress: 2203 of 2622 (84.0%)
Progress: 2301 of 2622 (87.8%)
Progress: 2402 of 2622 (91.6%)
Progress: 2501 of 2622 (95.4%)
Progress: 2601 of 2622 (99.2%)
Done: sum = 130943.8
我下面的代码与 sehe 代码类似,但存在一些差异,这让我能够处理由于精确相等而报告的跳过点,涉及除以模。此外,全局计数器收集所有线程的实际循环执行,但它可能不精确 - 对于此特定问题,这是可以接受的。我只使用主线程进行报告。
const size_t size = ...
const size_t step_size = size / 100;
const size_t nThreads = ...
const size_t local_count_max = step_size / nThreads;
size_t count = 0;
#pragma omp parallel num_threads(nThreads)
{
size_t reported_count = 0;
size_t local_count = 0;
#pragma omp for
for (size_t i = 0; i < size; ++i)
{
<... do some useful work ...>
// -------------------------- update local and global progress counters
if (local_count >= local_count_max)
{
#pragma omp atomic
count += local_count_max;
local_count = 0;
}
else
{
++local_count;
}
// ------------------------------ report progress (in master thread only)
#pragma omp master
if (count - reported_count >= step_size)
{
<... report the progress ...>
reported_count = count;
}
}
}
- Android NDK传感器向事件队列报告奇怪的间隔
- 如何在 C 中正确使用 libiconv 使其不会报告"Arg list too long"?
- 使用std::source_location报告错误的最佳实践
- xmake总是报告:错误:无法获取cxx的程序,为什么
- 当用户超过按钮点击限制时报告
- 使用调试/崩溃报告将应用程序部署到客户端
- 为什么cudaMemGetInfo报告设备内存总量的变化
- 有没有任务栏API可以立即应用注册表更改
- 如何创建线程序列以按照启动顺序执行任务?
- 为什么瓦尔格林德在不释放恶意内存后没有报告任何问题?
- C++一个线程如何正确通信其任务已完成?
- 在线程之间拆分任务总是值得的吗?
- 递归求和任务的错误答案
- 内存清理程序报告全局对象构造中未初始化值的使用
- "main"函数堆栈中的对象在第一个任务运行时被覆盖 (FreeRTOS)
- C++ 任务流库入门
- QDataStream 读取和写入的字节数比 QFile::length() 报告要多
- 我可以报告开放mp任务的进度吗?
- 多线程任务总体进度报告的设计模式
- 堆大小(由 umdh 报告)和(私有)工作集(由任务管理器报告)之间的差异的原因是什么