通过boost :: asio发送原始数据:: asio

Question

我试图通过boost :: asio发送原始数据，因为boost :: serialization对于我的需求来说太慢了。继各种实例和提升文档，我有一个客户端：

SimulationClient：

void SimulationClient::sendData(std::vector<WaveformDefinition>waveformPackets) { 
     socket.async_send_to(boost::asio::buffer(waveformPackets), 
       receiver_endpoint, 
       boost::bind(&ClientEnvironmentEngine::sendComplete, this, 
         boost::asio::placeholders::error, 
         boost::asio::placeholders::bytes_transferred)); 
} 

我试图坦纳桑斯伯里的解决方案下面，但无法得到它的工作。不过，我使用具有成功：

class WaveformReceiver { 
    WaveformDefinition *buffer; 

    WaveformReceiver(){ 
     buffer = new WaveformDefinition[MAX_WAVEFORMS]; 
     startReceive(); 
    } 

    void startReceive() { 
     socket_.async_receive_from(boost::asio::null_buffers(), remote_endpoint_, 
       boost::bind(&WaveformReceiver::handleReceive, this, 
       boost::asio::placeholders::error, 
       boost::asio::placeholders::bytes_transferred)); 
    } 

    void handleReceive(const boost::system::error_code& error, 
     std::size_t size/*bytes_transferred*/) 
    { 
      if (!error) 
      { 
       int available = socket_.available(); 
       int numWaveforms = available/sizeof(WaveformDefinition_c); 
       socket_.receive(boost::asio::buffer(buffer, available)); 

       //copy buffer into another buffer so we can re-use the original buffer for the next read 
       WaveformDefinition_c* tempBuffer = new WaveformDefinition_c[numWaveforms]; 
       std::memcpy (tempBuffer, buffer, available); 

       //schedule a thread to handle the array of waveforms that we copied 
       threadPool.schedule(boost::bind(handleWaveforms, tempBuffer, numWaveforms)); 
       //start listening for more waveforms 
       startReceive(); 
      } 
    } 
} 

坦纳，或别人，你能告诉我，如果我在做什么也应该工作，或者如果我刚开幸运，它目前正在？

Answer 1

问题的根本部分是关于序列化和反序列化集合。

在不控制服务器和客户端的编译器和体系结构的情况下，发送原始结构通常是不安全的，因为字节表示在系统之间可能会有所不同。虽然编译器和体系结构在这种特定情况下是相同的，但#pragma pack(1)是无关紧要的，因为WAVEFORM_DATA_STRUCT没有作为原始内存写入套接字。相反，为收集write操作提供了多个内存缓冲区。

boost::array<boost::asio::mutable_buffer,2> buffer = {{ 
    boost::asio::buffer(&waveformPacket->numWaveforms, ...), // &numWaveforms 
    boost::asio::buffer(waveformPacket->waveforms)   // &waveforms[0] 
}}; 

有各种工具可以帮助序列化数据结构，例如Protocol Buffers。

---

下面的代码将演示序列化网络通信的数据结构的基础知识。为了简化代码和解释，我选择了专注于序列化和反序列化，而不是从套接字读写。位于本节下面的另一个示例将显示更多的原始方法，它假定使用相同的编译器和体系结构。

与碱性foo类型开始：

struct foo 
{ 
    char a; 
    char b; 
    boost::uint16_t c; 
}; 

可以确定该数据可以被打包成4个字节总数。下面是一种可能的导线reprensetation：

0  8  16  24  32 
|--------+--------+--------+--------| 
| a | b |  c  | 
'--------+--------+--------+--------' 

随着所确定的线表示，两个功能可以用于序列化（保存）一个foo对象到缓冲器，而另一个可被用于从一个反序列化（负载）foo缓冲。由于foo.c大于一个字节，所以这些函数也需要考虑endianness。我选择在Boost.Asio细节名称空间中使用endian字节交换函数来实现某些平台中立性。

/// @brief Serialize foo into a network-byte-order buffer. 
void serialize(const foo& foo, unsigned char* buffer) 
{ 
    buffer[0] = foo.a; 
    buffer[1] = foo.b; 

    // Handle endianness. 
    using ::boost::asio::detail::socket_ops::host_to_network_short; 
    boost::uint16_t c = host_to_network_short(foo.c); 
    std::memcpy(&buffer[2], &c, sizeof c); 
} 

/// @brief Deserialize foo from a network-byte-order buffer. 
void deserialize(foo& foo, const unsigned char* buffer) 
{ 
    foo.a = buffer[0]; 
    foo.b = buffer[1]; 

    // Handle endianness. 
    using ::boost::asio::detail::socket_ops::network_to_host_short; 
    boost::uint16_t c; 
    std::memcpy(&c, &buffer[2], sizeof c); 
    foo.c = network_to_host_short(c); 
} 

与序列化和反序列为foo完成后，下一步就是要处理foo对象的集合。在编写代码之前，需要确定电线表示。在这种情况下，我决定在一个具有32位计数字段的foo元素序列前缀。

0  8  16  24  32 
|--------+--------+--------+--------| 
|  count of foo elements [n] | 
|--------+--------+--------+--------| 
|   serialized foo [0]  | 
|--------+--------+--------+--------| 
|   serialized foo [1]  | 
|--------+--------+--------+--------| 
|    ...    | 
|--------+--------+--------+--------| 
|   serialized foo [n-1]  | 
'--------+--------+--------+--------' 

再次，可以引入两个辅助功能，序列化和反序列化foo对象的集合，并且还需要占计数字段的字节顺序。

/// @brief Serialize a collection of foos into a network-byte-order buffer. 
template <typename Foos> 
std::vector<unsigned char> serialize(const Foos& foos) 
{ 
    boost::uint32_t count = foos.size(); 

    // Allocate a buffer large enough to store: 
    // - Count of foo elements. 
    // - Each serialized foo object. 
    std::vector<unsigned char> buffer(
     sizeof count +   // count 
     foo_packed_size * count); // serialize foo objects 

    // Handle endianness for size. 
    using ::boost::asio::detail::socket_ops::host_to_network_long; 
    count = host_to_network_long(count); 

    // Pack size into buffer. 
    unsigned char* current = &buffer[0]; 
    std::memcpy(current, &count, sizeof count); 
    current += sizeof count; // Adjust position. 

    // Pack each foo into the buffer. 
    BOOST_FOREACH(const foo& foo, foos) 
    { 
    serialize(foo, current); 
    current += foo_packed_size; // Adjust position. 
    } 

    return buffer; 
}; 

/// @brief Deserialize a buffer into a collection of foo objects. 
std::vector<foo> deserialize(const std::vector<unsigned char>& buffer) 
{ 
    const unsigned char* current = &buffer[0]; 

    // Extract the count of elements from the buffer. 
    boost::uint32_t count; 
    std::memcpy(&count, current, sizeof count); 
    current += sizeof count; 

    // Handle endianness. 
    using ::boost::asio::detail::socket_ops::network_to_host_long; 
    count = network_to_host_long(count); 

    // With the count extracted, create the appropriate sized collection. 
    std::vector<foo> foos(count); 

    // Deserialize each foo from the buffer. 
    BOOST_FOREACH(foo& foo, foos) 
    { 
    deserialize(foo, current); 
    current += foo_packed_size; 
    } 

    return foos; 
}; 

下面是完整的示例代码：

#include <iostream> 
#include <vector> 
#include <boost/asio.hpp> 
#include <boost/asio/detail/socket_ops.hpp> // endian functions 
#include <boost/cstdint.hpp> 
#include <boost/foreach.hpp> 
#include <boost/tuple/tuple.hpp>   // boost::tie 
#include <boost/tuple/tuple_comparison.hpp> // operator== for boost::tuple 

/// @brief Mockup type. 
struct foo 
{ 
    char a; 
    char b; 
    boost::uint16_t c; 
}; 

/// @brief Equality check for foo objects. 
bool operator==(const foo& lhs, const foo& rhs) 
{ 
    return boost::tie(lhs.a, lhs.b, lhs.c) == 
     boost::tie(rhs.a, rhs.b, rhs.c); 
} 

/// @brief Calculated byte packed size for foo. 
/// 
/// @note char + char + uint16 = 1 + 1 + 2 = 4 
static const std::size_t foo_packed_size = 4; 

/// @brief Serialize foo into a network-byte-order buffer. 
/// 
/// @detail Data is packed as follows: 
/// 
/// 0  8  16  24  32 
/// |--------+--------+--------+--------| 
/// | a | b |  c  | 
/// '--------+--------+--------+--------' 
void serialize(const foo& foo, unsigned char* buffer) 
{ 
    buffer[0] = foo.a; 
    buffer[1] = foo.b; 

    // Handle endianness. 
    using ::boost::asio::detail::socket_ops::host_to_network_short; 
    boost::uint16_t c = host_to_network_short(foo.c); 
    std::memcpy(&buffer[2], &c, sizeof c); 
} 

/// @brief Deserialize foo from a network-byte-order buffer. 
void deserialize(foo& foo, const unsigned char* buffer) 
{ 
    foo.a = buffer[0]; 
    foo.b = buffer[1]; 

    // Handle endianness. 
    using ::boost::asio::detail::socket_ops::network_to_host_short; 
    boost::uint16_t c; 
    std::memcpy(&c, &buffer[2], sizeof c); 
    foo.c = network_to_host_short(c); 
} 

/// @brief Serialize a collection of foos into a network-byte-order buffer. 
/// 
/// @detail Data is packed as follows: 
/// 
/// 0  8  16  24  32 
/// |--------+--------+--------+--------| 
/// |  count of foo elements [n] | 
/// |--------+--------+--------+--------| 
/// |   serialized foo [0]  | 
/// |--------+--------+--------+--------| 
/// |   serialized foo [1]  | 
/// |--------+--------+--------+--------| 
/// |    ...    | 
/// |--------+--------+--------+--------| 
/// |   serialized foo [n-1]  | 
/// '--------+--------+--------+--------' 
template <typename Foos> 
std::vector<unsigned char> serialize(const Foos& foos) 
{ 
    boost::uint32_t count = foos.size(); 

    // Allocate a buffer large enough to store: 
    // - Count of foo elements. 
    // - Each serialized foo object. 
    std::vector<unsigned char> buffer(
     sizeof count +   // count 
     foo_packed_size * count); // serialize foo objects 

    // Handle endianness for size. 
    using ::boost::asio::detail::socket_ops::host_to_network_long; 
    count = host_to_network_long(count); 

    // Pack size into buffer. 
    unsigned char* current = &buffer[0]; 
    std::memcpy(current, &count, sizeof count); 
    current += sizeof count; // Adjust position. 

    // Pack each foo into the buffer. 
    BOOST_FOREACH(const foo& foo, foos) 
    { 
    serialize(foo, current); 
    current += foo_packed_size; // Adjust position. 
    } 

    return buffer; 
}; 

/// @brief Deserialize a buffer into a collection of foo objects. 
std::vector<foo> deserialize(const std::vector<unsigned char>& buffer) 
{ 
    const unsigned char* current = &buffer[0]; 

    // Extract the count of elements from the buffer. 
    boost::uint32_t count; 
    std::memcpy(&count, current, sizeof count); 
    current += sizeof count; 

    // Handle endianness. 
    using ::boost::asio::detail::socket_ops::network_to_host_long; 
    count = network_to_host_long(count); 

    // With the count extracted, create the appropriate sized collection. 
    std::vector<foo> foos(count); 

    // Deserialize each foo from the buffer. 
    BOOST_FOREACH(foo& foo, foos) 
    { 
    deserialize(foo, current); 
    current += foo_packed_size; 
    } 

    return foos; 
}; 

int main() 
{ 
    // Create a collection of foo objects with pre populated data. 
    std::vector<foo> foos_expected(5); 
    char a = 'a', 
     b = 'A'; 
    boost::uint16_t c = 100; 

    // Populate each element. 
    BOOST_FOREACH(foo& foo, foos_expected) 
    { 
    foo.a = a++; 
    foo.b = b++; 
    foo.c = c++; 
    } 

    // Serialize the collection into a buffer. 
    std::vector<unsigned char> buffer = serialize(foos_expected); 

    // Deserialize the buffer back into a collection. 
    std::vector<foo> foos_actual = deserialize(buffer); 

    // Compare the two. 
    std::cout << (foos_expected == foos_actual) << std::endl; // expect 1 

    // Negative test. 
    foos_expected[0].c = 0; 
    std::cout << (foos_expected == foos_actual) << std::endl; // expect 0 
} 

其产生1和0了预期的效果。

---

如果使用相同的编译器和体系结构，那么它可能有可能从原始缓冲液作为foo对象数组重新解释foo对象的连续序列，以及与复制构造填充std::vector<foo>。例如：

// Create and populate a contiguous sequence of foo objects. 
std::vector<foo> foo1; 
populate(foo1); 

// Get a handle to the contiguous memory block. 
const char* buffer = reinterpret_cast<const char*>(&foo1[0]); 

// Populate a new vector via iterator constructor. 
const foo* begin = reinterpret_cast<const foo*>(buffer); 
std::vector<foo> foos2(begin, begin + foos1.size()); 

最后，foo1应等于foo2。 foo2中的foo对象将由位于foo1所有的内存中的重新解释的foo对象复制构建。

#include <iostream> 
#include <vector> 
#include <boost/cstdint.hpp> 
#include <boost/foreach.hpp> 
#include <boost/tuple/tuple.hpp>   // boost::tie 
#include <boost/tuple/tuple_comparison.hpp> // operator== for boost::tuple 

/// @brief Mockup type. 
struct foo 
{ 
    char a; 
    char b; 
    boost::uint16_t c; 
}; 

/// @brief Equality check for foo objects. 
bool operator==(const foo& lhs, const foo& rhs) 
{ 
    return boost::tie(lhs.a, lhs.b, lhs.c) == 
     boost::tie(rhs.a, rhs.b, rhs.c); 
} 

int main() 
{ 
    // Create a collection of foo objects with pre populated data. 
    std::vector<foo> foos_expected(5); 
    char a = 'a', 
     b = 'A'; 
    boost::uint16_t c = 100; 

    // Populate each element. 
    BOOST_FOREACH(foo& foo, foos_expected) 
    { 
    foo.a = a++; 
    foo.b = b++; 
    foo.c = c++; 
    } 

    // Treat the collection as a raw buffer. 
    const char* buffer = 
     reinterpret_cast<const char*>(&foos_expected[0]); 

    // Populate a new vector. 
    const foo* begin = reinterpret_cast<const foo*>(buffer); 
    std::vector<foo> foos_actual(begin, begin + foos_expected.size()); 

    // Compare the two. 
    std::cout << (foos_expected == foos_actual) << std::endl; 

    // Negative test. 
    foos_expected[0].c = 0; 
    std::cout << (foos_expected == foos_actual) << std::endl; 
} 

如同其它方法，这产生的1和0预期的结果。

Answer 2

首先，它不安全使用pragma pack(1)。包装可能与不同的编译器/拱不同。此外，您将在协议更改中遇到问题。我建议改用google protobuf。

二。您正在发送std::vector，但此向量的实际数据不在结构内WAVEFORM_DATA_STRUCT（向量将其数据保存在堆中）。所以，你发送矢量和它的指向堆到另一台机器，这个指针definetly无效。你需要以某种方式序列化你的向量。

P.S.与boost :: asio没有任何关系，但这个问题是关于正确的序列化/反序列化。