如何使用ifstream (c++)只读一些以前知道的行

How to read only some previously know lines using ifstream (C++)

本文关键字:ifstream 何使用 c++ 只读      更新时间:2023-10-16

通过对文件的预处理,我发现了一些需要进一步处理的行,知道我想要读取这些行。有比使用ifstream::getline(...)逐行读取行更快的解决方案吗?

例如,我知道我只想要产品4的行(0-4-8-12-16-…)或特殊行号存储在向量中…

现在我要这样做:

string line;
int counter = 0;
while( getline(ifstr,line) ){
   if(counter%4 =0){
      // some code working with line
   }
}

但我想要这样(如果更快)

while(getline(ifstr,line)){ 
  // some code working with line
  while(++counter%4 !=0){ // or checking on index vector
     skipline(ifstr)         
  }
}

让我再次提到,我有一些行索引(排序,但不是这个规则),但为了简单起见,我使用这个示例product4。

编辑:我想在开始时跳到行,例如我知道我需要从行号2000读取,如何快速跳过1999行?谢谢所有的

因为@caps说这让他觉得标准库中没有任何东西可以帮助完成这类任务,所以我觉得有必要演示一下:)

Live On Coliru

template <typename It, typename Out, typename Filter = std::vector<int> >
Out retrieve_lines(It begin, It const end, Filter lines, Out out, char const* delim = "\n") {
    if (lines.empty())
        return out;
    // make sure input is orderly
    assert(std::is_sorted(lines.begin(), lines.end()));
    assert(lines.front() >= 0);
    std::regex re(delim);
    std::regex_token_iterator<It> line(begin, end, re, -1), eof;
    // make lines into incremental offsets
    std::adjacent_difference(lines.begin(), lines.end(), lines.begin());
    // iterate advancing by each offset requested
    auto advanced = [&line, eof](size_t n) { while (line!=eof && n--) ++line; return line; };
    for (auto offset = lines.begin(); offset != lines.end() && advanced(*offset) != eof; ++offset) {
        *out++ = *line;
    }
    return out;
}

这显然更通用。权衡(目前)是标记化迭代器需要随机访问迭代器。我发现这是一个很好的权衡,因为对文件的"随机访问"实际上无论如何都要求内存映射文件

实时演示1:从字符串到vector<string>

Live On Coliru

int main() {
    std::vector<std::string> output_lines;
    std::string is(" a b c d enf g hijklmnopnqrstuvwnxyz");
    retrieve_lines(is.begin(), is.end(), {0,3,999}, back_inserter(output_lines));
    // for debug purposes
    for (auto& line : output_lines)
        std::cout << line << "n";
}

打印

 a b c d e
xyz

现场演示2:从文件到cout

Live On Coliru

#include <boost/iostreams/device/mapped_file.hpp>
int main() {
    boost::iostreams::mapped_file_source is("/etc/dictionaries-common/words");
    retrieve_lines(is.begin(), is.end(), {13,784, 9996}, std::ostream_iterator<std::string>(std::cout, "n"));
}

打印。

ASL's
Apennines
Mercer's

使用boost::iostreams::mapped_file_source可以很容易地替换为直接的::mmap,但我发现它在演示示例中更难看。

std::fstream::streampos实例对应于文件的行开头存储到std::vector中,然后您可以使用该向量的索引访问特定的行。下面是一个可能的实现,

class file_reader {
public:
    // load file streampos offsets during construction
    explicit file_reader(const std::string& filename) 
        : fs(filename) { cache_file_streampos(); }
    std::size_t lines() const noexcept { return line_streampos_vec.size(); }
    // get a std::string representation of specific line in file
    std::string read_line(std::size_t n) {
        if (n >= line_streampos_vec.size() - 1)
            throw std::out_of_range("out of bounds");
        navigate_to_line(n);
        std::string rtn_str;
        std::getline(fs, rtn_str);
        return rtn_str;
    }
private:
    std::fstream fs;
    std::vector<std::fstream::streampos> line_streampos_vec;
    const std::size_t max_line_length = // some sensible value
    // store file streampos instances in vector
    void cache_file_streampos() {
        std::string s;
        s.reserve(max_line_length);
        while (std::getline(fs, s)) 
            line_streampos_vec.push_back(fs.tellg());
    }
    // go to specific line in file stream
    void navigate_to_line(std::size_t n) {
        fs.clear();
        fs.seekg(line_streampos_vec[n]);
    }
};

然后你可以通过

读取文件的特定行
file_reader fr("filename.ext");
for (int i = 0; i < fr.lines(); ++i) {
    if (!(i % 4))
        std::string line_contents = fr.read_line(i); // then do something with the string 
}

ArchbishopOfBanterbury的回答很好,我同意他的观点,当你做预处理时,你会得到更干净的代码和更好的效率,只要存储每行开头的字符位置。

但是,假设这是不可能的(也许预处理是由其他API处理的,或者是来自用户输入的),有一个解决方案应该做最少的工作,只读取指定的行。

基本问题是,给定一个行长度可变的文件,您无法知道每行的开始和结束位置,因为一行被定义为以'n'结尾的字符序列。因此,您必须解析每个字符以检查它是否为'n',如果是,则前进行计数器并读入行,如果行计数器匹配所需的输入之一。

auto retrieve_lines(std::ifstream& file_to_read, std::vector<int> line_numbers_to_read) -> std::vector<std::string>
{
    auto begin = std::istreambuf_iterator<char>(file_to_read);
    auto end = std::istreambuf_iterator<char>();
    auto current_line = 0;
    auto next_line_num = std::begin(line_numbers_to_read);
    auto output_lines = std::vector<std::string>();
    output_lines.reserve(line_numbers_to_read.size());  //this may be a silly "optimization," since all the strings are still separate unreserved buffers
    //we can bail if we've reached the end of the lines we want to read, even if there are lines remaining in the stream
    //we *must* bail if we've reached the end of the stream, even if there are supposedly lines left to read; that input must have been incorrect
    while(begin != end && next_line_num != std::end(line_numbers_to_read))
    {
        if(current_line == *next_line_num)
        {
            auto matching_line = std::string();
            if(*begin != 'n')
            {
                //potential optimization: reserve matching_line to something that you expect will fit most/all of your input lines
                while(begin != end && *begin != 'n')
                {
                    matching_line.push_back(*begin++);
                }
            }
            output_lines.emplace_back(matching_line);
            ++next_line_num;
        }
        else 
        {
            //skip this "line" by finding the next 'n'
            while(begin != end && *begin != 'n')
            {
                ++begin;
            }
        }
        //either code path in the previous if/else leaves us staring at the 'n' at the end of a line,
        //which is not the right state for the next iteration of the loop.
        //So skip this 'n' to get to the beginning of the next line
        if (begin != end && *begin == 'n')
        {
            ++begin;
        }
        ++current_line;
    }
    return output_lines;
}

这是Coliru上的现场直播,以及我测试它的输入。正如您所看到的,它正确地处理空行以及正确地处理被告知抓取比文件中更多的行。