干净的方式，使可移植的端正确的文件读取/编写代码在c++
Clean way to make portable endian-correct file-reading / writing code in C++
我想写一些c++代码，可以以正确的方式从文件中读取和写入。更确切地说，我希望能够读取特定类型的文件，其端序可以很容易地检测到(其幻数是否反转)。
但是我该如何正确地读取文件呢?我读了下面这篇文章，它给了我一个有用的想法:
http://www.gamedev.net/page/resources/_/technical/game-programming/writing-endian-independent-code-in-c-r2091这里的想法是创建一个类，其中有一些函数指针指向所需的端进正确的read()函数。但在我的经验中，函数指针很慢，特别是当你必须如此频繁地调用它们时，就像在这种情况下。另一种选择是使用
if (file_was_detected_big_endian) { read_bigendian(); } else { read_littleendian(); }
为每一个read_x_bit_int()函数，但这似乎也效率低下。
我正在使用Boost，所以我有它所有的辉煌来帮助我。特别地，有一个端序子库:
http://www.boost.org/doc/libs/develop/libs/endian/doc/buffers.html虽然我不确定如何干净地使用这段代码来做我想要的。我想有一些代码，在那里我可以读说16个字节直接进入一个struct的指针，代表文件的一部分，同时自动纠正端序。我当然可以自己写这段代码，但我有一种感觉，一个坚实的解决方案必须已经存在。
我认为我所有的代码都将被手动填充，以防止对齐问题。
谢谢!
因此，dasblinkenlight提出的虚拟函数方法可能就足够了——特别是因为I/O可能是主要的时间消耗者。但是，如果确实发现read函数占用了大量cpu时间，则可以通过模板化文件读取器来摆脱虚拟函数调度。
下面是一些伪代码来演示这一点:基本上，创建两个读取器类，每个端序一个:
class LittleReader {
public:
LittleReader(std::istream& is) : m_is(is) {}
char read_char() {//read byte from m_is}
int read_int32() {//read 32-bit int and convert;}
float read_float()....
private:
std::istream& m_is;
};
class BigReader {
public:
BigReader(std::istream& is): m_is(is){}
char read_char(){...}
int read_int32(){..}
float read_float(){...}
private:
std::istream& m_is;
}
将读取逻辑的主要部分(除了幻数位)分离到一个函数模板中，该模板接受上述类之一的实例作为参数:
template <class Reader>
void read_endian(Reader &rdr){
field1 = rdr.read_int32();
field2 = rdr.read_float();
// process rest of data file
...
}
本质上，编译器将创建read_endian函数的两个实现——每个端倒数一个。由于没有动态分派，编译器还可以内联所有对read_int32、read_float等的调用。
最后，在主阅读器函数中，查看幻数以确定要实例化哪种阅读器:
void read_file(std::istream& is){
int magic(read_magic_no(is));
if (magic == MAGIC_BIG_ENDIAN)
read_endian(BigReader(is));
else
read_endian(LittleReader(is));
}
该技术为您提供了灵活性，而不会产生任何虚拟调度开销，代价是增加(二进制)代码大小。当你有非常紧凑的循环并且你需要压缩每一点性能时，它会非常有用。
解决这个问题有两种方法:
- 用不区分大小写的方式写文件，
- 添加标记，并以字节序感知的方式读取文件
第一种方法需要更多的书写工作，而第二种方法使书写"无开销"。
这两种方法都可以在没有函数指针的情况下实现:由于虚函数*的存在，对函数指针的需求在c++中大大减少了。
实现这两种方法是相似的:您需要创建一个抽象基类来序列化基本数据类型，创建该类的一个实例来读取正确的端序，并调用其虚拟成员函数来进行读写:
struct PrimitiveSerializer {
virtual void serializeInt(ostream& out, const int val) = 0;
virtual void serializeChar(ostream& out, const char val) = 0;
virtual void serializeString(ostream& out, const std::string& val) = 0;
...
virtual int deserializeInt(istream& in) = 0;
virtual char deserializeChar(istream& in) = 0;
virtual std::string deserializeString(istream& in) = 0;
};
struct BigEndianSerializer : public PrimitiveSerializer {
...
};
struct LittleEndianSerializer : public PrimitiveSerializer {
...
};
根据不同的方法，决定使用哪个子类是不同的。如果您使用第一种方法(即编写与端序无关的文件)，那么您将实例化与系统的端序匹配的序列化器。如果采用第二种方法，您将从文件中读取幻数，并选择与文件的端序匹配的子类。
另外，第一种方法可以使用hton/ntoh函数来实现。
*函数指针本身并不"慢"，尽管它们使编写低效率的代码更容易。
我已经编写了一个小的。h和。cpp，现在可以处理(可能)所有的端序问题。虽然我已经为自己的应用程序调整了这些函数，但它们可能会对其他人有所帮助。
endian_bis.h:
/**
* endian_bis.h - endian-gnostic binary input stream functions
* Copyright (C) 2015
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
* 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
*/
#pragma once
#include <cstdint>
#include <istream>
class BinaryInputStream {
public:
inline int8_t read_int8(std::istream &in) { char buf[1]; in.read(buf, 1); return read_int8(buf, 0); }
inline int16_t read_int16(std::istream &in) { char buf[2]; in.read(buf, 2); return read_int16(buf, 0); }
inline int32_t read_int32(std::istream &in) { char buf[4]; in.read(buf, 4); return read_int32(buf, 0); }
inline int64_t read_int64(std::istream &in) { char buf[8]; in.read(buf, 8); return read_int64(buf, 0); }
inline uint8_t read_uint8(std::istream &in) { char buf[1]; in.read(buf, 1); return read_uint8(buf, 0); }
inline uint16_t read_uint16(std::istream &in) { char buf[2]; in.read(buf, 2); return read_uint16(buf, 0); }
inline uint32_t read_uint32(std::istream &in) { char buf[4]; in.read(buf, 4); return read_uint32(buf, 0); }
inline uint64_t read_uint64(std::istream &in) { char buf[8]; in.read(buf, 8); return read_uint64(buf, 0); }
inline float read_float(std::istream &in) { char buf[4]; in.read(buf, 4); return read_float(buf, 0); }
inline double read_double(std::istream &in) { char buf[8]; in.read(buf, 8); return read_double(buf, 0); }
inline int8_t read_int8(char buf[], int off) { return (int8_t)buf[off]; }
inline uint8_t read_uint8(char buf[], int off) { return (uint8_t)buf[off]; }
virtual int16_t read_int16(char buf[], int off) = 0;
virtual int32_t read_int32(char buf[], int off) = 0;
virtual int64_t read_int64(char buf[], int off) = 0;
virtual uint16_t read_uint16(char buf[], int off) = 0;
virtual uint32_t read_uint32(char buf[], int off) = 0;
virtual uint64_t read_uint64(char buf[], int off) = 0;
virtual float read_float(char buf[], int off) = 0;
virtual double read_double(char buf[], int off) = 0;
static BinaryInputStream *endianCorrectStream(int streamIsBigEndian);
static BinaryInputStream *endianCorrectStream(std::istream &in,
uint32_t expectedBigEndianMagic,
uint32_t expectedLittleEndianMagic);
};
endian_bis.cpp:
/**
* endian_bis.cpp - endian-gnostic binary input stream functions
* Copyright (C) 2015 Jonah Schreiber (jonah.schreiber@gmail.com)
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
* 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
*/
#include "endian_bis.h"
#include <cstring>
/*
* Delegated functions
*/
static inline int16_t read_be_int16(char buf[], int off) {
return (int16_t)(((buf[off] & 0xff) << 8) |
((buf[off+1] & 0xff)));
}
static inline int32_t read_be_int32(char buf[], int off) {
return (int32_t)(((buf[off] & 0xff) << 24) |
((buf[off+1] & 0xff) << 16) |
((buf[off+2] & 0xff) << 8) |
((buf[off+3] & 0xff)));
}
template<int> static inline int64_t read_be_int64(char buf[], int off); // template indicates default word size (size_t)
template<> inline int64_t read_be_int64<4>(char buf[], int off) {
return (((int64_t)(((buf[off] & 0xff) << 24) |
((buf[off+1] & 0xff) << 16) |
((buf[off+2] & 0xff) << 8) |
((buf[off+3] & 0xff)))
) << 32) | (
(int64_t)(((buf[off+4] & 0xff) << 24) |
((buf[off+5] & 0xff) << 16) |
((buf[off+6] & 0xff) << 8) |
((buf[off+7] & 0xff))));
}
static inline uint16_t read_be_uint16(char buf[], int off) {
return (uint16_t)(((buf[off] & 0xff) << 8) |
((buf[off+1] & 0xff)));
}
static inline uint32_t read_be_uint32(char buf[], int off) {
return (uint32_t)(((buf[off] & 0xff) << 24) |
((buf[off+1] & 0xff) << 16) |
((buf[off+2] & 0xff) << 8) |
((buf[off+3] & 0xff)));
}
template<int> static inline uint64_t read_be_uint64(char buf[], int off); // template indicates default word size (size_t)
template<> inline uint64_t read_be_uint64<4>(char buf[], int off) {
return (((uint64_t)(((buf[off] & 0xff) << 24) |
((buf[off+1] & 0xff) << 16) |
((buf[off+2] & 0xff) << 8) |
((buf[off+3] & 0xff)))
) << 32) | (
(uint64_t)(((buf[off+4] & 0xff) << 24) |
((buf[off+5] & 0xff) << 16) |
((buf[off+6] & 0xff) << 8) |
((buf[off+7] & 0xff))));
}
inline static int16_t read_le_int16(char buf[], int off) {
return (int16_t)(((buf[off+1] & 0xff) << 8) |
((buf[off] & 0xff)));
}
inline static int32_t read_le_int32(char buf[], int off) {
return (int32_t)(((buf[off+3] & 0xff) << 24) |
((buf[off+2] & 0xff) << 16) |
((buf[off+1] & 0xff) << 8) |
((buf[off] & 0xff)));
}
template<int> static inline int64_t read_le_int64(char buf[], int off); // template indicates default word size (size_t)
template<> inline int64_t read_le_int64<4>(char buf[], int off) {
return (((int64_t)(((buf[off+7] & 0xff) << 24) |
((buf[off+6] & 0xff) << 16) |
((buf[off+5] & 0xff) << 8) |
((buf[off+4] & 0xff)))
) << 32) | (
(int64_t)(((buf[off+3] & 0xff) << 24) |
((buf[off+2] & 0xff) << 16) |
((buf[off+1] & 0xff) << 8) |
((buf[off] & 0xff))));
}
inline static uint16_t read_le_uint16(char buf[], int off) {
return (uint16_t)(((buf[off+1] & 0xff) << 8) |
((buf[off] & 0xff)));
}
inline static uint32_t read_le_uint32(char buf[], int off) {
return (uint32_t)(((buf[off+3] & 0xff) << 24) |
((buf[off+2] & 0xff) << 16) |
((buf[off+1] & 0xff) << 8) |
((buf[off] & 0xff)));
}
template<int> static inline uint64_t read_le_uint64(char buf[], int off); // template indicates default word size (size_t)
template<> inline uint64_t read_le_uint64<4>(char buf[], int off) {
return (((uint64_t)(((buf[off+7] & 0xff) << 24) |
((buf[off+6] & 0xff) << 16) |
((buf[off+5] & 0xff)<< 8) |
((buf[off+4] & 0xff)))
) << 32) | (
(uint64_t)(((buf[off+3] & 0xff) << 24) |
((buf[off+2] & 0xff) << 16) |
((buf[off+1] & 0xff) << 8) |
((buf[off] & 0xff))));
}
/* WARNING: UNTESTED FOR 64 BIT ARCHITECTURES; FILL IN 3 MORE METHODS LIKE THIS TO TEST
THE CORRECT FUNCTION WILL BE SELECTED AUTOMATICALLY AT COMPILE TIME
template<> inline uint64_t read_uint64_branch<8>(char buf[], int off) {
return (int64_t)((buf[off] << 56) |
(buf[off+1] << 48) |
(buf[off+2] << 40) |
(buf[off+3] << 32) |
(buf[off+4] << 24) |
(buf[off+5] << 16) |
(buf[off+6] << 8) |
(buf[off+7]));
}*/
inline static float read_matching_float(char buf[], int off) {
float f;
memcpy(&f, &buf[off], 4);
return f;
}
inline static float read_mismatched_float(char buf[], int off) {
float f;
char buf2[4] = {buf[3], buf[2], buf[1], buf[0]};
memcpy(&f, buf2, 4);
return f;
}
inline static double read_matching_double(char buf[], int off) {
double d;
memcpy(&d, &buf[off], 8);
return d;
}
inline static double read_mismatched_double(char buf[], int off) {
double d;
char buf2[8] = {buf[7], buf[6], buf[5], buf[4], buf[3], buf[2], buf[1], buf[0]};
memcpy(&d, buf2, 4);
return d;
}
/*
* Types (singleton instantiations)
*/
/*
* Big-endian stream, Big-endian runtime
*/
static class : public BinaryInputStream {
public:
int16_t read_int16(char buf[], int off) { return read_be_int16(buf, off); }
int32_t read_int32(char buf[], int off) { return read_be_int32(buf, off); }
int64_t read_int64(char buf[], int off) { return read_be_int64<sizeof(size_t)>(buf, off); }
uint16_t read_uint16(char buf[], int off) { return read_be_uint16(buf, off); }
uint32_t read_uint32(char buf[], int off) { return read_be_uint32(buf, off); }
uint64_t read_uint64(char buf[], int off) { return read_be_uint64<sizeof(size_t)>(buf, off); }
float read_float(char buf[], int off) { return read_matching_float(buf, off); }
double read_double(char buf[], int off) { return read_matching_double(buf, off); }
} beStreamBeRuntime;
/*
* Big-endian stream, Little-endian runtime
*/
static class : public BinaryInputStream {
public:
int16_t read_int16(char buf[], int off) { return read_be_int16(buf, off); }
int32_t read_int32(char buf[], int off) { return read_be_int32(buf, off); }
int64_t read_int64(char buf[], int off) { return read_be_int64<sizeof(size_t)>(buf, off); }
uint16_t read_uint16(char buf[], int off) { return read_be_uint16(buf, off); }
uint32_t read_uint32(char buf[], int off) { return read_be_uint32(buf, off); }
uint64_t read_uint64(char buf[], int off) { return read_be_uint64<sizeof(size_t)>(buf, off); }
float read_float(char buf[], int off) { return read_mismatched_float(buf, off); }
double read_double(char buf[], int off) { return read_mismatched_double(buf, off); }
} beStreamLeRuntime;
/*
* Little-endian stream, Big-endian runtime
*/
static class : public BinaryInputStream {
public:
int16_t read_int16(char buf[], int off) { return read_le_int16(buf, off); }
int32_t read_int32(char buf[], int off) { return read_le_int32(buf, off); }
int64_t read_int64(char buf[], int off) { return read_le_int64<sizeof(size_t)>(buf, off); }
uint16_t read_uint16(char buf[], int off) { return read_le_uint16(buf, off); }
uint32_t read_uint32(char buf[], int off) { return read_le_uint32(buf, off); }
uint64_t read_uint64(char buf[], int off) { return read_le_uint64<sizeof(size_t)>(buf, off); }
float read_float(char buf[], int off) { return read_mismatched_float(buf, off); }
double read_double(char buf[], int off) { return read_mismatched_double(buf, off); }
} leStreamBeRuntime;
/*
* Little-endian stream, Little-endian runtime
*/
static class : public BinaryInputStream {
public:
int16_t read_int16(char buf[], int off) { return read_le_int16(buf, off); }
int32_t read_int32(char buf[], int off) { return read_le_int32(buf, off); }
int64_t read_int64(char buf[], int off) { return read_le_int64<sizeof(size_t)>(buf, off); }
uint16_t read_uint16(char buf[], int off) { return read_le_uint16(buf, off); }
uint32_t read_uint32(char buf[], int off) { return read_le_uint32(buf, off); }
uint64_t read_uint64(char buf[], int off) { return read_le_uint64<sizeof(size_t)>(buf, off); }
float read_float(char buf[], int off) { return read_matching_float(buf, off); }
double read_double(char buf[], int off) { return read_matching_double(buf, off); }
} leStreamLeRuntime;
/*
* "Factory" singleton methods (plus helper)
*/
static inline int isRuntimeBigEndian() {
union { int32_t i; int8_t c[4]; } bint = {0x01020304};
return bint.c[0] == 1;
}
BinaryInputStream *BinaryInputStream::endianCorrectStream(int streamIsBigEndian) {
if (streamIsBigEndian) {
if (isRuntimeBigEndian()) {
return &beStreamBeRuntime;
} else {
return &beStreamLeRuntime;
}
} else {
if (isRuntimeBigEndian()) {
return &leStreamBeRuntime;
} else {
return &leStreamLeRuntime;
}
}
}
BinaryInputStream *BinaryInputStream::endianCorrectStream(std::istream &in,
uint32_t expectedBigEndianMagic,
uint32_t expectedLittleEndianMagic) {
uint32_t magic = ((BinaryInputStream*)&beStreamBeRuntime)->read_uint32(in);
if (magic == expectedBigEndianMagic) {
if (isRuntimeBigEndian()) {
return &beStreamBeRuntime;
} else {
return &beStreamLeRuntime;
}
} else if (magic == expectedLittleEndianMagic) {
if (isRuntimeBigEndian()) {
return &leStreamBeRuntime;
} else {
return &leStreamLeRuntime;
}
} else {
return 0; /* not expected magic number */
}
}
建议使用:
BinaryInputStream *bis = BinaryInputStream::endianCorrectStream(in, 0x01020304, 0x04030201);
if (bis == 0) {
cerr << "error: infile is not an Acme EarthQUAKEZ file" << endl;
return 1;
}
in.ignore(4);
int32_t number = bis->read_int32(in);
...
相关文章：
- 读取最后一行代码算法 - c++ 时出现问题
- 我编写了以下代码来读取C++矩阵，然后打印其行和列.我收到此错误
- 如何使用指针直接从托管代码中的非托管代码中读取矢量数据
- 在 c++ 中逐行读取文件(将代码从 Python 转换为 C++）
- 使用 Visual Studio 代码读取 C++ 中的输入流时出错
- 如何只读取和剥离特定的代码行
- boost::asio-async_read_some示例代码没有读取套接字中的所有数据
- 从C++代码中直接读取成员属性
- 驱动程序，以支持从 C++ 代码读取或写入 HIVE
- 读取代码中的文件版本值
- 如何让这个C++代码从用户那里读取五个整数而不是一个整数?
- C++ - 使用代码块从文件中读取 IP 地址
- 为什么 ifstream 文件中的换行符 - 当被此代码读取时 - 占用 2 个字节
- 我在编写从文本文件中读取数据并按升序打印的代码时遇到问题
- 为什么此代码无法正确读取用户输入
- 结构无法正确读取C++代码::块 13.12
- 无法读取代码页不匹配的文件版本信息
- 读取代码行的数量
- 使用引用和取消引用运算符读取代码行时遇到问题
- 从.txt [c++]读取代码
