Agora SDK not working in Windows Build. VideoSurface.cs always gets tmpi = -1 in Update - unity3d

I am trying to implement Screen broadcast with Unity using the Agora Video Chat SDK for Unity. I used this source, which doesn't work initially. But after modifying the code as below, I am able to receive my own stream through the server, inside Unity editor (2019.1.2f1).
//Adding inside Start
mRtcEngine.OnJoinChannelSuccess = Joined;
}
private void Joined(string channelName, uint uid, int elapsed)
{
var videoSource = FindObjectOfType<VideoSurface>();
videoSource.SetForUser(uid);
videoSource.SetEnable(true);
}
But nothing happens in the Windows build. I checked the VideoSurface.cs file. I am continuously getting tmpi = -1 inside Update. What could be the reason?
PS. I check all firewall permissions for the build. Also, the user is able to join the channel. It's just the stream that is not being received. Help appreciated.

You shouldn't need to modify the code like that. And also, in the code above you register the callback for the local user. If you want to show remote user's video, you should register the callback for OnUserJoined().
Have you seen the tutorial about the Screensharing? https://www.agora.io/en/blog/how-to-broadcast-your-screen-with-unity3d-and-agora/
Please try that. If you are still confused, you may take a look at this github repo. It has different contents to share, but the concept and Agora API usage are pretty much the same.

Related

How to stop tokbox screen sharing in SWIFT

I am working on an IOS application(SWIFT) in which i have used tokbox for screensharing, i am able to share the screen but not able to stop screensharing.
This is the code I have used for screensharing.
publisher?.videoType = .screen
publisher?.audioFallbackEnabled = false
let cap = ScreenCapturer(withView:view)
publisher?.videoCapture = cap
session?.publish(publisher, error: &error)
Can anyone guide to stop screensharing in swift.
To stop screen sharing will need to stop the publisher from streaming. To do that you can call:
[OTSession unpublish:error:]
More info is available on the Video API guides
For your case, where you are adding screen sharing to an existing call, you will need to create an additional publisher for the screen sharing rather than editing the existing one. To use the existing publisher it will require the publisher to be reinitialised to switch between publishing a camera feed vs a screen which will stop publishing audio too.
In addition to creating a new publisher, you need to create a new subscriber for the other user, you can do that in the subscriberDidConnect delegate function on the OTSubscriberDelegate.
Additionally, you will need to handle the destruction of both the new publisher and subscriber. This will be done in the delegate functions are you using already on the OTSessionDelegate and OTPublisherDelegate.
I have created a demo app which demonstrates this behaviour.

How to stop ScreenSharing manually with tokbox sdk in Swift [duplicate]

I am working on an IOS application(SWIFT) in which i have used tokbox for screensharing, i am able to share the screen but not able to stop screensharing.
This is the code I have used for screensharing.
publisher?.videoType = .screen
publisher?.audioFallbackEnabled = false
let cap = ScreenCapturer(withView:view)
publisher?.videoCapture = cap
session?.publish(publisher, error: &error)
Can anyone guide to stop screensharing in swift.
To stop screen sharing will need to stop the publisher from streaming. To do that you can call:
[OTSession unpublish:error:]
More info is available on the Video API guides
For your case, where you are adding screen sharing to an existing call, you will need to create an additional publisher for the screen sharing rather than editing the existing one. To use the existing publisher it will require the publisher to be reinitialised to switch between publishing a camera feed vs a screen which will stop publishing audio too.
In addition to creating a new publisher, you need to create a new subscriber for the other user, you can do that in the subscriberDidConnect delegate function on the OTSubscriberDelegate.
Additionally, you will need to handle the destruction of both the new publisher and subscriber. This will be done in the delegate functions are you using already on the OTSessionDelegate and OTPublisherDelegate.
I have created a demo app which demonstrates this behaviour.

Voice message capture no longer working regardless of Secure Context / HTTPS on mobile

I am building an appointment scheduling web app (calendar like) that allows users to leave voice messages (explanations) at certain time points that others can retrieve remotely at their convenience. It works well on PC but I can't get it to work on mobiles anymore. Apparently new security context standards are in place (WebRTC?) and a Secure Context must be provided, but though I am using HTTPS (SSL Site/Domain), I can't get it to work in any of the major browsers. How else can I provide such a Secure Context?
The code I attach is quite popular and used to work well (it still does on laptops!). What can I do to provide the right context on mobile-phones (IOS, Android, Tablets)? Thank you for any help.
navigator.mediaDevices.getUserMedia(constraints).then(function(stream) {
//console.log("getUserMedia() success, stream created, initializing Speakit_recorder.js ...");
audioContext = new AudioContext();
//update the format
/* assign to gumStream for later use */
gumStream = stream;
/* use the stream */
input = audioContext.createMediaStreamSource(stream);
/* Create the Recorder object, configure to record mono sound(1channel). Recording 2channels will double the file size */
rec = new Recorder(input,{numChannels:2})
//start the recording process
rec.record();
//console.log("Recording started");
}).catch(function(err) {
//do whatever is necesssary if getUserMedia() fails
});
This might be due to the autoplay changes that landed in Chrome 71. See https://developers.google.com/web/updates/2017/09/autoplay-policy-changes as well as https://bugs.chromium.org/p/chromium/issues/detail?id=835767 for details.
You might want to log audioContext.state is 'running'. If not, you need to call audioContext.resume() on a user action (such as a button click)

Show Twitch.tv stream within a unity3D application

I would like to make an Unity3D Application in which one can watch a current Twitch.tv live stream.
I am not sure if this is possible, for example, with the twitch api (https://github.com/justintv/twitch-api)
I know about the video textures in Unity3D and I know how to use the default twitch api basics but I do not have an idea how to integrate a running twitch stream into my application.
Can someone please give me a hint if this is possible?
Thanks very much and best regards
Meph
Ultimately the stream coming from twitch is MPEG-4 (H264/M3U). So if you can render that in Unity, then you can render twitch streams.
There's a few steps to get the right URLs requested from twitch, and that can change over time. You'll have to inspect a current twitch page while a stream is playing to see how the javascript builds the request (url and headers).
In the end, the javascript will build an access token then use it to request a file called index-live.m3u8 from one of the twitch edge servers. That file contains a list of files names for the last few seconds of the live stream (and some other meta data). It looks something like this
https://video-edge-c61b44.lax01.hls.ttvnw.net/v0/[some-long-access-token]/index-live.m3u8
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:5
#ID3-EQUIV-TDTG:2017-10-27T16:53:27
#EXT-X-MEDIA-SEQUENCE:8788
#EXT-X-TWITCH-ELAPSED-SECS:17576.000
#EXT-X-TWITCH-TOTAL-SECS:17589.870
#EXTINF:2.000,
index-0000008788-Y6OH.ts
#EXTINF:2.000,
index-0000008789-l0QY.ts
#EXTINF:2.000,
index-0000008790-gCUV.ts
#EXTINF:2.000,
index-0000008791-1ngg.ts
#EXTINF:2.000,
index-0000008792-wpQL.ts
#EXTINF:2.000,
index-0000008793-koO4.ts
You then swap out index-live.m3u8 with the name of a file in the list and request it to get that clip. Something like:
https://video-edge-c61b44.lax01.hls.ttvnw.net/v0/[the-same-long-access-token]/index-0000008793-koO4.ts
It will be an MPEG-4 stream about 1 second long. The list is about 6 files long, so if you request them all you can get about a 6 second buffer.
Every second or two, index-live.m3u8 is updated and older files roll off as new ones are added to the bottom. You need to re-request this file every few seconds as your buffer runs out, and request the new clips in it to keep your playback going.
using Ted Bigham's answer you can get the m3u8 stream of the video. You can then feed this stream into a plugin like AVPro video from Renderheads to play it directly in unity on any shape or model you want, with a wide variety of customisation and settings.
However this is not a free plugin, and there may be alternatives. I am in no way affiliated to them, but have used this plug-in in the past with good results.
p.s i originally wanted to post this as just a comment, and believe Ted Bigham answered the grunt of your question. But am not allowed due to reputation.
#Ted Bigham is there anyway to do this just using the default unity video player?
Everything you've said so far has worked great! With just chrome, I'm able to open the .m3u8 and download each 1 second .ts file. With just any video player, I can view each video file just fine.
When I try to pass the url through Video Player though, it says "Cannot read file." I even tried replacing .ts with .mp4 since that is a supported format. This works when I test it outside of Unity, but gives me the same error.
Here's my code.
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.Networking;
using UnityEngine.Video;
public class TwitchStreamer : MonoBehaviour
{
public string m3u8 = "https://video-weaver.atl01.hls.ttvnw.net/v1/playlist/Cq4EuEYkh5IqurC4gy3nHqGhPVtUwH_" +
"5QcQbCatC5Fhit9qbivMQ2rdh2MS_m_2OuLd3VS2mF0eTrCKrz8YTuDO19mcbIBMJL3BUMz4jnNuU_t-e53V51TOtaN3vcCk9n3Qr" +
"dP0WyLREnVR0n_30d4PlUjjW4_si5Wr2XuePQ0dPtxP6jnsenKnX56YTLohtCN2-FdfSvHQMdfd0aw68FA4h9wawHoIM9-U6YRmPa" +
"fDsfdCiZr_iToSR6lZi81VoYPVjt7Ygf7xKwhjlNwrvA5SnsAnWQGIOVt4UjDkNLw-hmNMAr7RT0iiDghKXZY1VI6Tuc-umB1VXYE" +
"7BH5hHbDfHgB3_IYNb0fjoudtSuaZxISyWoazPrw3AibEO7k1-quhdcjarBTGpIi_dlPEp-yZlQOy98_OZY_tqjk8ZWTBIaAAYEG_" +
"miwqsgH4d6eIfkh3ehyMvPQH1C5dVG9tQcSWPUYU6D6hWhxvJhEr-UC0_BYWIVzX7z_Uf74FJGIEqSQc0d6igiowdMM_lyD8ZV9BE" +
"7wqQs3RegMPqux-AOfF-_Q7Ki2MBv9u7D9ZRXMH_cm20bTx5-ShEDRnWMApSfXK-9bAGNXUcw8YlBbHYeSN5VxEZMC2oGjcivBsGs" +
"RPMTQ_yNBSM1S6GxFRIR4nqA-mbdXg3rXMW3V6MNybBb1lrrQeEqF1tdYE0rfxe3Ki5WWkxeKmSjMGbMl1tHCwMaReTYkQnX5Qhjl" +
"HXXtKLtEIEEhB3cXW3oF05-E_q87s68JQIGgyIEKPiQlTsANR9zRc.m3u8";
public float updateClip = 1f, updateFile = 6f;
public VideoPlayer vp;
Queue<string> urls;
void OnEnable()
{
StartCoroutine(UpdateFile());
//InvokeRepeating("UpdateFile", 0f, updateFile);
InvokeRepeating("UpdateClip", updateClip, updateClip);
urls = new Queue<string>();
vp.prepareCompleted += Vp_prepareCompleted;
vp.errorReceived += Vp_errorReceived;
}
private void Vp_errorReceived(VideoPlayer source, string message)
{
Debug.Log("4567: Playback preparation failed. " + message, source);
}
private void Vp_prepareCompleted(VideoPlayer source)
{
Debug.Log("4567: Playback preparation complete.");
source.Play();
}
void OnDisable()
{
CancelInvoke();
StopAllCoroutines();
vp.prepareCompleted -= Vp_prepareCompleted;
vp.errorReceived -= Vp_errorReceived;
}
void UpdateClip()
{
//vp.Stop();
if (urls.Count > 0)
{
vp.url = urls.Dequeue();
vp.Prepare();
Debug.Log("4567: Prepare playback for " + vp.url);
}
}
IEnumerator UpdateFile()
{
UnityWebRequest www = UnityWebRequest.Get(m3u8);
yield return www.SendWebRequest();
if (www.isNetworkError || www.isHttpError)
{
Debug.Log(www.error);
}
else
{
// Show results as text
//Debug.Log(www.downloadHandler.text);
string[] raw = www.downloadHandler.text.Split('\n');
foreach (string line in raw)
{
if (line.Contains("http"))
{
string cnvrt = line.Replace(".ts", ".mp4");
urls.Enqueue(cnvrt);
Debug.Log(line + " has been added to stream queue.");
}
}
// Or retrieve results as binary data
//byte[] results = www.downloadHandler.data;
}
yield return new WaitForSeconds(updateFile);
}
}
I believe what this person wants to know is whether or not the Twitch API actually retrieves a live video feed and passes the data through to the application. The short answer is, it doesn't. The Twitch API doesn't allow you to take a live video feed, only get the information about it.
You can get strings telling you the name of the channel, the game and links to take you to the channel. You can also get their channel background or previews as image files. What you cannot do is get video files or stream video data. The best you can do is get the URLs and link to them.
The only method I can think of to get a Live Stream in any game is for it to be the player's. Essentially you would code in a method by which they can stream their game within the game itself, taking the stream data and copying it into some variable or class somewhere, before it is uploaded to Twitch, so you can use the data at the same time it is being broadcast. But if you want to retrieve some random streamer's gameplay and display it on a computer screen in your game, I'm afraid you can't do that. Not in Unity, anyway.
You can embed a twitch stream into a website
There are web browser plugins for Unity(see)
So if you combine these ideas, you can show a twitch stream in unity by running an in-app browser that will run a custom web page which embed the stream that you want. I don't know what kind of performance you'll get out of this but that sounds simple enough to do. It probably is not as simple as directly showing the stream on a texture but it's at least simpler than learning C, network programming, video processing and etc like Thomas suggested.
This is a tricky thing to do. You would need an API key from Twitch, a program that can establish not only a connection with your chosen API, but also a data stream. I would suggest starting with learning basic network programming in a less specialized language like C, and work from there. After you understand the basics, please come back and ask more specific questions. I can't do that much with this question in the state that it's in.
As with everything else, this is definitely possible, just maybe not with C#. For the sake of not destroying your game's performance, I would suggest writing a separate program that feeds the data into the game without the game directly grabbing data.
Once you gain an understanding of network programming, I would hop over to the twitch dev chat on the API, found here

Soundcloud record flash plugin don't work with current version of chrome under macOS

this record plugin stops to work for us with current version of chrome under macOS
Chrome: Version 23.0.1271.97
MacOS: 10.8.2
http://connect.soundcloud.com/examples/recording.html
how to reproduce:
click record
allow using your microphone
-> recording don't start
(In fact sometimes it works (20% of cases for me))
and also sometimes there is an error in console
PepperFlashPlayer.plugin: 0x2A052 is not valid resource ID.
Please help us, we use it for production and a lot of users can not record sound
regards, Dmitry
My investigation show:
sometimes the "SampleDataEvent.SAMPLE_DATA" event not triggered (or not assigned) in flash.
To solve this issue I do:
Added to event handler "StatusEvent.STATUS" (or call manually if microphone unmuted):
tti = setInterval(applySampleData, 100);
Added function:
protected function applySampleData() : void {
microphone.removeEventListener(SampleDataEvent.SAMPLE_DATA, recordSampleDataHandler);
microphone.addEventListener(SampleDataEvent.SAMPLE_DATA, recordSampleDataHandler);
}
And in function "recordSampleDataHandler" (it's SampleDataEvent.SAMPLE_DATA listener):
if(tti) { clearInterval(tti); tti = 0; }
Try this.