Is there any way to send or receive images, audio and video using Photon Pun2 or Photon Chat? - unity3d

Question

I am using both packages Photon Pun2 and Photon Chat in my application. But i can't find any way to send or receive images, audio, or video via private message.

Answer 1

Yes there is ... However, it is quite limited.
Exactly how big are the things you have to send?
Chat
In Photon Chat Message payload is limited to something between 400.000 and 500.000 bytes. I didn't test it more in detail but if your message size hits a certain limit you are immediately disconnected without a proper feadback for the reason ^^
See Photon Chat Intro
Example
public class MyChatListner : IChatClientListener
{
public event Action<byte[]> OnReceived;
private ChatClient cient;
public void Initialize()
{
client = new ChatClient(this);
client.ChatRegion = ...;
client.Connect(....);
}
// Sicne you already work with the chat you should know but anyway
// This has to be called continuously (who knows in what intervals)
public void Heartbeat()
{
client.Service();
}
public void SendData(string recipient, byte[] data)
{
client.SendPrivateMessage(recipient, data);
}
public void OnPrivateMessage(string sender, object message, string channelName)
{
OnReceived?.Invoke((byte[])message);
}
// also will have to implement the other methods from IChatClientListener ...
}
PUN2
In Pun2 I don't know if such a limit exists but it will definitely delay everything else until the file is fully received. Here you can directly send and receive byte[] via PhotonNetwork.RaiseEvent and OnEvent (via the interface IOnEventCallback)
Example
// doesn't even have to be monobehaviour necessarily
public class FileTransmitter : IOnEventCallback
{
private const byte eventID = 42;
public event Action<byte[]> OnReceived;
public void Enable()
{
PhotonNetwork.AddCallbackTarget(this);
}
public void Disable()
{
PhotonNetwork.RemoveCallbackTarget(this);
}
public void SendData(byte[] data, SendOptions sendoptions, RaiseEventOptions raiseEventOptions = null)
{
PhotonNetwork.RaiseEvent(eventID, data, raiseEventOptions, sendoptions);
}
public void OnEvent(EventData photonEvent)
{
if(photonEvent.Code != eventID) return;
var data = (byte[]) photonEvent.CustomData;
OnReceived?.Invoke(data);
}
}
Alternatively you can of course also use the RPC directly and use e.g.
public class FileTransfer : MonoBehaviourPun
{
[PunRPC]
void ChatMessage(string fileName, byte[] content)
{
// e.g.
File.WriteAllBytes(Path.Combine(Application.persistentDataPath, fileName), content);
}
public void SendFile(string fileName, byte[] content, PhotonPlayer targetPlayer)
{
photonView.RPC(nameof(ChatMessage), targetPlayer, fileName, content);
}
}
Either way personally what I did was using only one single byte[] and encoded all required information into it. Photon will do this anyway with all parameters but if you already know exactly what data you send and how to deserialize it it is way more efficient to do it yourself e.g. in a Thread/Task. Then on the receiver side I deserialized these into the individual information again.

Answer 2

It's possible and working just fine (even though you should watch for the limits as #derHugo said). My implementation uses RPC to send an array of bytes using Texture2D.EncodeToPNG and then decoding it on the clients upon receiving it.
To convert the byte array back into the image, you can use Texture2D.LoadImage method.
The approach is similar for sending other multimedia types, just the encoding/decoding methods are different.

Answer 3

PhotonStream allows you to stream any byte[] data though the network.
In theory, any data type can be converted in to byte[], and decode them on the receiver side.
Below example script is just an example with minimal setup.
reference from: https://frozenmist.com/docs/apis/fmetp-stream/pun2-example/
It's also possible to live stream video, audio and remote commands with popular third party plugins like FMETP STREAM, which has native fast encoder for capturing your Game View and Desktop View in real time, even compatible with mobile and VR devices.
using Photon.Pun;
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
// ref: https://frozenmist.com/docs/apis/fmetp-stream/pun2-example/
public class FMStreamPUN : Photon.Pun.MonoBehaviourPun, IPunObservable {
private Queue<byte[]> appendQueueSendData = new Queue<byte[]>();
public int appendQueueSendDataCount { get { return appendQueueSendData.Count; } }
public UnityEventByteArray OnDataByteReadyEvent = new UnityEventByteArray();
public void OnPhotonSerializeView(PhotonStream stream, PhotonMessageInfo info) {
if (stream.IsWriting) {
//Send the meta data of the byte[] queue length
stream.SendNext(appendQueueSendDataCount);
//Sending the queued byte[]
while(appendQueueSendDataCount > 0) {
byte[] sentData = appendQueueSendData.Dequeue();
stream.SendNext(sentData);
}
}
if (stream.IsReading) {
if (!photonView.IsMine) {
//Get the queue length
int streamCount = (int)stream.ReceiveNext();
for (int i = 0; i < streamCount; i++) {
//reading stream one by one
byte[] receivedData = (byte[])stream.ReceiveNext();
OnDataByteReadyEvent.Invoke(receivedData);
}
}
}
}
public void Action_SendData(byte[] inputData) {
//inputData(byte[]) is the encoded byte[] from your encoder
//doesn't require any stream, when there is only one player in the room
if(PhotonNetwork.CurrentRoom.PlayerCount > 1) appendQueueSendData.Enqueue(inputData);
}
}