Binary Data Not Supported in WebSocket

[Shivam-002]

It seems that binary data (such as audio or video blobs) cannot be transmitted from the client to the server using WebSockets.

Upon debugging, I found that the client is stringifying the data before sending it through the websocket.

this.ws.send(JSON.stringify([event, data]));

This leads to binary data being lost, and the data becomes an empty object {}.

File affected: packages/jstack-shared/src/event-emitter.ts.

Moreover, the framework using Zod validation and allowing only string, which fails to parse ArrayBuffer or any binary data. Here's an example from the server side where data is being parsed as a string:

ws.onmessage = (event) => {
  const data = z2.string().parse(event.data);
  const eventSchema = z2.tuple([z2.string(), z2.unknown()]);
  const parsedData = JSON.parse(data);
  const parseResult = eventSchema.safeParse(parsedData);
  if (parseResult.success) {
    const [eventName, eventData] = parseResult.data;
    this.emitter.handleEvent(eventName, eventData);
  } else {
    logger.warn("Unable to parse event:", event.data);
  }
};

As I'm not an expert, I might be missing something, but from what I can see, it doesn't seem to support binary data. If there is a way to achieve this functionality, I would appreciate any guidance, or it may be worth considering adding support for binary data in future releases.

[joschan21]

Hey! As a workaround for transmitting the blob directly, you could upload to a third-party storage (e.g., S3, R2) from the client, and then transmit the URL after upload. Could you elaborate on the use case a bit?

Thanks for creating the issue!

[Shivam-002]

Yes, I took that path. Using R2 and sending acknowledgement to server.

Usecase
I am building a AI-powered interview platform. So need to send users speech to the server.