非常快速地阅读 CSV 文件

要找到行尾，您必须通读该行中的所有列以查找行尾。这是不可避免的。不过，您不必处理那些不需要的字段。

从这个链接答案的选项二中汲取灵感，我得到类似的东西

//discard first line without looking at it. 
if (retailFile.ignore(std::numeric_limits<std::streamsize>::max(), 'n')
{ // ALWAYS test IO transactions to make sure they worked, even something as 
// trivial as ignoring the input. 
std::string line;
while (std::getline(retailFile, line))
{ // read the whole line
// wrap the line in a stream for easy parsing
std::istringstream stream (line);
if (std::getline(retailFile, invoiceNo, ';') && 
std::getline(retailFile, stockCode, ';') &&
std::getline(retailFile, desc, ';'))
{ // successfully read all three required columns
// Do not use anything you read until after you know it is good. Not 
// checking leads to bugs and malware.
// strongly consider doing something with the variables here. The next loop 
// iteration will write over them
i++;
}
else
{
// failed to find all three columns. You should look into why and 
// handle accordingly.
}
}
}
else
{
// failed to ignore the line. You should look into why and handle accordingly.
}

您可能不会发现太多的实际速度差异。从磁盘读取文件通常比对文件执行任何操作更耗时，除非您在读取文件后对文件的数据执行大量操作。有潜在的更快方法来拆分行，但同样，差异可能首先隐藏在读取文件的成本中。

问题是：什么是快？

在下面的演示中，我创建了包含 591.000 行的 I 文件。大小为 74MB。

然后我为std::ifstream设置一个更大的输入缓冲区，读取所有行，解析它们，并将前 3 个条目复制到生成的向量中。其余的我确实忽略了。

为了避免结果被优化，我显示了 50 行输出。

VS2019，C++17，发布模式，所有优化开启。

结果：~2.7 秒用于读取和解析机器上的所有行。(我必须承认，我通过 PCIe 在 RAID 0 中有 4 个固态硬盘(

#include <iostream>
#include <fstream>
#include <vector>
#include <string>
#include <regex>
#include <array>
#include <chrono>
#include <iterator>
int main() {
// Put whatever filename you want
static const std::string fileName{ "r:\big.txt" };
// Start Time measurement
auto start = std::chrono::system_clock::now();
#if 0
// Write file with 591000 lines
if (std::ofstream ofs(fileName); ofs) {
for (size_t i = 0U; i < 591000U; ++i) {
ofs << "invoiceNo_" << i << ";"
<< "stockCode_" << i << ";"
<< "description_" << i << ";"
<< "Field_4_" << i << ";"
<< "Field_5_" << i << ";"
<< "Field_6_" << i << ";"
<< "Field_7_" << i << ";"
<< "Field_8_" << i << "n";
}
}
#endif
auto end = std::chrono::system_clock::now();
auto elapsed = std::chrono::duration_cast<std::chrono::milliseconds>(end - start);
// How long did it take?
std::cout << "Time for writing the file:       " << elapsed.count() << " msn";

// We are just interested in 3 fields
constexpr size_t NumberOfNeededFields = 3U;
// We expect 591000 lines, give a little bit more
constexpr size_t NumberOfExpectedFilesInFile = 600000U;
// We will create a bigger input buffer for our stream
constexpr size_t ifStreamBufferSize = 100000U;
static char buffer[ifStreamBufferSize];
// The delimtzer for our csv
static const std::regex delimiter{ ";" };
// Main working variables
using Fields3 = std::array<std::string, NumberOfNeededFields>;
static Fields3 fields3;
static std::vector<Fields3> fields{};
// Reserve space to avoid reallocation
fields.reserve(NumberOfExpectedFilesInFile);
// Start timer
start = std::chrono::system_clock::now();
// Open file and check, if it is open
if (std::ifstream ifs(fileName); ifs) {
// Set bigger file buffer
ifs.rdbuf()->pubsetbuf(buffer, ifStreamBufferSize);
// Read all lines
for (std::string line{}; std::getline(ifs, line); ) {
// Parse string
std::copy_n(std::sregex_token_iterator(line.begin(), line.end(), delimiter, -1), NumberOfNeededFields, fields3.begin());
// Store resulting 3 fields
fields.push_back(std::move(fields3));
}
}
end = std::chrono::system_clock::now();
elapsed = std::chrono::duration_cast<std::chrono::milliseconds>(end - start);
std::cout << "Time for parsing the file:       " << elapsed.count() << " msn";
// Show some result 
for (size_t i = 0; i < fields.size(); i += (fields.size()/50)) {
std::copy_n(fields[i].begin(), NumberOfNeededFields, std::ostream_iterator<std::string>(std::cout, " "));
std::cout << "n";
}
return 0;
}