Using Clumsy simulate loss packet 10%，the data is not the same with sent

[zekyang]

Using Clumsy simulate loss packet 10%，the data is not the same with sent, but the total length is equal。Is the msquic changed the data?

[nibanks]

I do not understand your question. Can you please elaborate?

[zekyang]

Thank you for your reply。
Well, My English is not so well, I will try to explain more detailly. Using msquic to send video file.

server peer :
send stream：

int QuicServer::SendStream(const uint8_t* data, std::size_t len, bool last /*= false*/)
{
    QUIC_STATUS status;
    QUIC_BUFFER* buffer = (QUIC_BUFFER*)malloc(sizeof(QUIC_BUFFER) + len);
    if (nullptr == buffer)
    {
        return -1;
    }

    buffer->Length = len;
    buffer->Buffer = (uint8_t*)buffer + sizeof(QUIC_BUFFER);
    memcpy(buffer->Buffer, data, len);

    if (QUIC_FAILED(status = _msQuic->StreamSend(_stream, buffer, 1, last ? QUIC_SEND_FLAG_FIN : QUIC_SEND_FLAG_NONE, buffer)))
    {
        free(buffer);
        return -1;
    }

    return 0;
}

And free buffer in the QUIC_STREAM_EVENT_SEND_COMPLETE event:
free(Event->SEND_COMPLETE.ClientContext);

2、client peer write received data into file:

    case QUIC_STREAM_EVENT_RECEIVE:
    {
        printf("data received len:%d\n", Event->RECEIVE.TotalBufferLength);

        if (g_file)
        {
            for (uint32_t i = 0; i < Event->RECEIVE.BufferCount; i++)
            {
                const QUIC_BUFFER& buffer = Event->RECEIVE.Buffers[i];
                if (buffer.Length > 0)
                {
                    fwrite(buffer.Buffer, 1, buffer.Length, g_file);
                    fflush(g_file);
                }
            }
        }

       break;
   }

using clumsy simulate packet loss。

when simulating 20% loss packet， the data what received is not equal with sent。

when not set loss packet, the data is same。

[nibanks]

Ok, so you are essentially saying that the length you put in StreamSend is not the length you get back in the QUIC_STREAM_EVENT_RECEIVE event on the other side. This is to be expected. Just like a TCP connection, QUIC must segment any application data to fit into packets. A single send therefore may be broken up into many different packets. Likewise, multiple small sends may be coalesced into a single packet. On the receive side, the QUIC stack reassembles the data, and delivers it to the app as it is available. This generally results in different sizes and patterns, but the data is always in order.

[nibanks]

Which version/commit of MsQuic are you testing? We did have a recent bug in main that caused some corruption, but it has since been fixed. So, if you're trying main could you please try the latest and report if you are still seeing the corruption?

[zekyang]

I used MsQuic v2.3.5

[nibanks]

In your picture above, where you show the debugger view of two sections of memory, and you state length is the same but data is different. Which data is different? From what I see (just by glancing), everything is the same up to the length of the first section of memory. The highlighted (in red) memory in the section is just trailer data that you should ignore. It's not included in the length.

Am I missing something? It would be helpful to see a dump of the length and bytes you call send with, and the length and bytes of each individual receive event, if I am misinterpreting you. Thanks!

[zekyang]

The highlighted in red memory is the section where the data is the diff. This is my call send stream

void test_send_data(QuicServer* quic_server)
{
    FILE* fp = fopen("./test.h264", "rb");
    if(nullptr == fp)
    {
        printf("open file failed!\n");
        return;
    }

    do
    {
        std::size_t len = 2000;
        uint8_t* data = (uint8_t*)malloc(len);
        ret = fread(data, 1, len, fp);
        if(ret <= 0)
        {
            fseek(fp, 0, SEEK_SET);
            break;
        }

        quic_server->SendStream(data, len);

        std::this_thread::sleep_for(std::chrono::milliseconds(5));
        printf("SendStream size:%ld\n", len);

        free(data);
    } while (true);
}

dump the length and bytes of each individual receive event maybe a lot of files, and can not compare with the file sent.

[nibanks]

The send code looks fine.

The highlighted in red memory is the section where the data is the diff.

Can you explain more? How is it different? From what I can see, it is not in the same length of bytes as the send. Is that correct?

[zekyang]

No, the length of bytes is equal，see the picture, the file test_src.h264 is the sent (total length: 70706638 bytes). the file test_t.h264 is the received file (total length is also : 70706638 bytes).

[nibanks]

Can you share your receive handling code please?

[zekyang]

This is my receive handling code, I directly write the received data into file.

    case QUIC_STREAM_EVENT_RECEIVE:
    {
        printf("data received len:%d\n", Event->RECEIVE.TotalBufferLength);

        if (g_file)
        {
            for (uint32_t i = 0; i < Event->RECEIVE.BufferCount; i++)
            {
                const QUIC_BUFFER& buffer = Event->RECEIVE.Buffers[i];
                if (buffer.Length > 0)
                {
                    fwrite(buffer.Buffer, 1, buffer.Length, g_file);
                    fflush(g_file);
                }
            }
        }

       break;
   }

In QUIC_CONNECTION_EVENT_PEER_STREAM_STARTED event:

    case QUIC_CONNECTION_EVENT_PEER_STREAM_STARTED:
	g_file = fopen("test_t.h264", "wb");
        break;

and in QUIC_STREAM_EVENT_PEER_SEND_SHUTDOWN event:

    case QUIC_STREAM_EVENT_PEER_SEND_SHUTDOWN:
            fclose(g_file);
            break;

[nibanks]

Would you be willing to share your full code (at least code that reproduces the issue) somewhere? I tried to make a test case that selectively drops every packet possible, and I couldn't reproduce this problem on main at least.