How to offset note scheduling for interactive recording of notes via user controls - web-audio-api

In the code below I have a note scheduler that increments a variable named current16thNote up to 16 and then looping back around to 1. The ultimate goal of the application is to allow the user to click a drum pad and push the current16thNote value to an array. On each iteration of current16thNote a loop is run on the track arrays looking for the current current16thNote value, if it finds it the sound will play.
//_________________________________________________________General variable declarations
var isPlaying = false,
tempo = 120.0, // tempo (in beats per minute)
current16thNote = 1,
futureTickTime = 0.0,
timerID = 0,
noteLength = 0.05; // length of "beep" (in seconds)
//_________________________________________________________END General variable declarations
//_________________________________________________________Load sounds
var kick = audioFileLoader("sounds/kick.mp3"),
snare = audioFileLoader("sounds/snare.mp3"),
hihat = audioFileLoader("sounds/hihat.mp3"),
shaker = audioFileLoader("sounds/shaker.mp3");
//_________________________________________________________END Load sounds
//_________________________________________________________Track arrays
var track1 = [],
track2 = [5, 13],
track3 = [],
track4 = [1, 3, 5, 7, 9, 11, 13, 15];
//_________________________________________________________END Track arrays
//_________________________________________________________Future tick
function futureTick() {
var secondsPerBeat = 60.0 / tempo;
futureTickTime += 0.25 * secondsPerBeat;
current16thNote += 1;
if (current16thNote > 16) {
current16thNote = 1
}
}
//_________________________________________________________END Future tick
function checkIfRecordedAndPlay(trackArr, sound, beatDivisionNumber, time) {
for (var i = 0; i < trackArr.length; i += 1) {
if (beatDivisionNumber === trackArr[i]) {
sound.play(time);
}
}
}
//__________________________________________________________Schedule note
function scheduleNote(beatDivisionNumber, time) {
var osc = audioContext.createOscillator(); //____Metronome
if (beatDivisionNumber === 1) {
osc.frequency.value = 800;
} else {
osc.frequency.value = 400;
}
osc.connect(audioContext.destination);
osc.start(time);
osc.stop(time + noteLength);//___________________END Metronome
checkIfRecordedAndPlay(track1, kick, beatDivisionNumber, time);
checkIfRecordedAndPlay(track2, snare, beatDivisionNumber, time);
checkIfRecordedAndPlay(track3, hihat, beatDivisionNumber, time);
checkIfRecordedAndPlay(track4, shaker, beatDivisionNumber, time);
}
//_________________________________________________________END schedule note
//_________________________________________________________Scheduler
function scheduler() {
while (futureTickTime < audioContext.currentTime + 0.1) {
scheduleNote(current16thNote, futureTickTime);
futureTick();
}
timerID = window.requestAnimationFrame(scheduler);
}
//_________________________________________________________END Scheduler
The Problem
In addition to the previous code I have some user interface controls as shown in the following image.
When a user mousedowns on a “drum pad” I want to do two things. The first is to hear the sound immediately , and the second is to push the current16thNote value to the respective array.
If I use the following code to do this a few problems emerge.
$("#kick").on("mousedown", function() {
kick.play(audioContext.currentTime)
track1.push(current16thNote)
})
The first problem is that the sound plays twice. This is because when the sound is pushed to the array it is immediately recognized by the next iteration of the note scheduler and immediately plays. I fixed this by creating a delay with setInterval to offset the push to the array.
$("#kick").on("mousedown", function() {
kick.play(audioContext.currentTime)
window.setTimeout(function() {
track1.push(note)
}, 500)
})
The second problem is musical.
When a user clicks a drum pad the value that the user anticipates the drum will be recorded at is one 16th value earlier. In other words if you listen to the metronome and click on the kick drum pad with the intent of landing right on the 1/1 down beat this won't happen. Instead, when the metronome loops back around it will have been “recorded” at one 16th increment later.
This can be remedied by writing code that intentionally offsets the value that is pushed to the array by -1 .
I wrote a helper function named pushNote to do this.
$("#kick").on("mousedown", function() {
var note = current16thNote;
kick.play(audioContext.currentTime)
window.setTimeout(function() {
pushNote(track1, note)
}, 500)
})
//________________________________________________Helper
function pushNote(trackArr, note) {
if (note - 1 === 0) {
trackArr.push(16)
} else {
trackArr.push(note - 1)
}
}
//________________________________________________END Helper
My question is really a basic one. Is there a way to solve this problem without creating these odd “offsets” ?
I suspect there is a way to set/write/place the current16thNote increment without having to create offsets to other parts of the program. But I'm hazy on what it could be.
In the world of professional audio recording there isn't a tick per 16th division value , you usually have 480 ticks per quarter note. I want to begin exploring writing my apps using this larger value but I want to resolve this "offset" issue before I go down that rabbit hole.

Related

Check microphone for silence

While recording user voice, i want to know when he/she stopped talking to end the recording and send the audio file to google speech recognition API.
I found this thread here and tried to use it's solution but i am always getting the same value from the average of spectrum data which is 5.004574E-08:
Unity - Microphone check if silent
This is the code i am using for getting the GetSpectrumData value:
public void StartRecordingSpeech()
{
//If there is a microphone
if (micConnected)
{
if (!Microphone.IsRecording(null))
{
goAudioSource.clip = Microphone.Start(null, true, 10, 44100); //Currently set for a 10 second clip max
goAudioSource.Play();
StartCoroutine(StartRecordingSpeechCo());
}
}
else
{
Debug.LogError("No microphone is available");
}
}
IEnumerator StartRecordingSpeechCo()
{
while (Microphone.IsRecording(null))
{
float[] clipSampleData = new float[128];
goAudioSource.GetSpectrumData(clipSampleData, 0, FFTWindow.Rectangular);
Debug.Log(clipSampleData.Average());
yield return null;
}
}
PS: I am able to record the users voice, save it and get the right response from the voice recognition api.
The following method is what worked for me. it detect the volume of the microphone, turn it into decibels. It does not need to play the recorded audio or anything. (credit goes to this old thread in the unity answers: https://forum.unity.com/threads/check-current-microphone-input-volume.133501/).
public float LevelMax()
{
float levelMax = 0;
float[] waveData = new float[_sampleWindow];
int micPosition = Microphone.GetPosition(null) - (_sampleWindow + 1); // null means the first microphone
if (micPosition < 0) return 0;
goAudioSource.clip.GetData(waveData, micPosition);
// Getting a peak on the last 128 samples
for (int i = 0; i < _sampleWindow; i++)
{
float wavePeak = waveData[i] * waveData[i];
if (levelMax < wavePeak)
{
levelMax = wavePeak;
}
}
float db = 20 * Mathf.Log10(Mathf.Abs(levelMax));
return db;
}
In my case, if the value is bigger then -40 then the user is talking!if its 0 or bigger then there is a loud noise, other then that, its silence!
If you are interested in a volume then GetSpectrumData is actually not really what you want. This is used for frequency analysis and returns - as the name says - a frequency spectrum so how laud is which frequency in a given frequency range.
What you rather want to use is GetOutputData which afaik returns an array with amplitudes from -1 to 1. So you have to square all values, get the average and take the square root of this result (source)
float[] clipSampleData = new float[128];
goAudioSource.GetOutputData(clipSampleData, 0);
Debug.Log(Mathf.Sqrt(clipSampleData.Select(f => f*f).Average()));

Knowing when I can skip to any point in an audio file without buffering / delay in playback

I'm loading an MP3 on my webpage using audio = new Audio(). But I'd like to know that when setting audio.currentTime, the audio can skip to any point in the file -near the end or wherever -without any delay in playback. Ie I want to know when the MP3 has downloaded in its entirety.
Can I use the Audio object/element for this, or must I use an AudioContext as shown here?
Every AudioElement is exposing its buffered data as a TimeRanges object. TimeRanges is an object which tells you how many continuous parts aka ranges are already buffered. It does also have getters which return the respective start and end of each range in seconds.
In case your AudioElement is named audio the following code snippet will log the buffered time ranges at a given point in time.
const numberOfRanges = audio.buffered.length;
for (let i = 0; i < numberOfRanges; i += 1) {
console.log(
audio.buffered.start(i),
audio.buffered.end(i)
);
}
If you want to detect the point in time at which all data is buffered you could use a check similar to this one:
const isBufferedCompletely = (audio.buffered.length === 1
&& audio.buffered.start(0) === 0
&& audio.buffered.end(0) === audio.duration);
I used the Gist referenced in the comments below to construct an example. The following snippet will periodically check if the file is already buffered. It will log a message to the console once that is the case. I tested it on Chrome (v74) and Firefox (v66) on OS X. Please note that the file can't be played at the same time as the script will set the currentTime of the Audio Element.
const audio = new Audio('http://www.obamadownloads.com/mp3s/charleston-eulogy-speech.mp3');
audio.preload = 'auto';
function detectBuffered(duration) {
// Stick with the duration once it is known because it might get updated
// when reaching the end of the file.
if (duration === undefined && !isNaN(audio.duration)) {
duration = audio.duration;
}
const isBufferedCompletely = (audio.buffered.length === 1
&& audio.buffered.start(0) === 0
&& audio.buffered.end(0) === duration);
if (isBufferedCompletely) {
const seconds = Math.round(duration);
console.log('The complete file is buffered.');
console.log(`It is about ${ seconds } seconds long.`);
} else {
// Move the playhead of the audio element to get the browser to load
// the complete file.
if (audio.buffered.length > 0) {
audio.currentTime = Math.max(0, audio.buffered.end(0) - 1);
}
setTimeout(detectBuffered, 100, duration);
}
}
detectBuffered();

Digging technique counter

I'm making and game which has digging as it's feature so I need timer which will count exact float (in seconds) and then destroy gameObject. This is what I tried now but it's freezing unity:
function Update()
{
if (Input.GetMouseButtonDown(0))
{
digTime = 1.5; // in secounds
}
while (!Input.GetMouseButtonUp(0)) // why is this infinite loop?
{
digtime -= Time.deltaTime;
if (digtime <= 0)
{
Destroy(hit.collider.gameObject);
}
}
Here is a basic example how you can check if player has clicked for a certain time period.
#pragma strict
// This can be set in the editor
var DiggingTime = 1.5;
// Time when last digging started
private var diggingStarted = 0.0f;
function Update () {
// On every update were the button is not pressed reset the timer
if (!Input.GetMouseButton(0))
{
diggingStarted = Time.timeSinceLevelLoad;
}
// Check if the DiggingTime has passed from last setting of the timer
if (diggingStarted + DiggingTime < Time.timeSinceLevelLoad)
{
// Do the digging things here
Debug.Log("Digging time passed");
// Reset the timer
diggingStarted = Time.timeSinceLevelLoad;
}
}
It is firing every DiggingTime of seconds even the player is holding the button down. If you want that the player needs to release the button and press again one solution is to add Boolean telling if the timer is on or not. It can be set true on GetMouseButtonDown and false on GetMouseButtonUp.
Update function is called every frame. If you add a while loop inside this function waiting for the mouseButtonUp, you'll freeze Unity for sure.
You don't need the while loop. Just check GetMouseButtonUp without while loop.
EDIT
This is the Update function:
void Update ()
{
if ( Input.GetMouseButtonDown( 0 ) )
{
digTime = 1.5f;
}
else if ( Input.GetMouseButton( 0 ) )
{
if ( digTime <= 0 )
{
Destroy( hit.collider.gameObject );
}
else
{
digTime -= Time.deltaTime;
}
}
}
Minor controls should be added to avoid destroying gameObject several times, but this is the idea to proceed

When pausing game(using a bool and time.timescale) it doesn't pause the score counter

this will be my first question so go on easy on me please ;)
I'm building my first game in Unity using my limited knowledge, tutorials and troubleshooting on google but i can't seem to fix this issue
i have a script that counts score(with a GUIText) and another one to pause the game(using timescale) and you probably guessed it already but when i pause the game it doesn't pause the score.
Script for Pausing:
var isPaused : boolean = false;
function Update()
{
if(Input.GetKeyDown("p"))
{
Pause();
}
}
function Pause()
{
if (isPaused == true)
{
Time.timeScale = 1;
isPaused = false;
}
else
{
Time.timeScale = 0;
isPaused = true;
}
}
Script for Score:
var Counter : int = 100000;
var Substractor : int = 0;
var Score : int = Counter - Substractor;
function Update (
) {
Score--;
guiText.text = "Score: "+Score;
}
the score script is attached to a gameobject with a gui text and the script for pausing the game is attached to the player
another issue is that when i'm moving(using the arrow keys) then press pause and then unpause the player moves faster and unpredictable for a splitsecond but this is only when i press pause while pressing the arrow buttons when i only press pause there is no issue, this is a small bugg that i'l try to fix myself, just thought i'd add it here for those that have an easy answer to this issue aswell
I only know this problem from other game frameworks and I've got no experiance with unity. But it should work the same way for unity as it does in other frameworks. You have two options:
Update is called after every rendering frame! So your timescale doesn'T influence how often update is called. Instead you have to find some dt or deltaT (don't know how it is called in unity and how to get it, but I mean the time, since unities the last call to update, please tell me in the comments, so I can edit this answer).
Then don't calculate your score like:
Score--;
but use something like:
Score -= 1 * dt
or
Score -= 1 * dt * Time.timeScale
(depending how exactly unity works)
alternatively you can sourround your update-block with some if-statement:
in Score:
function Update (
) {
if (! isPaused()) { // not sure how this function / property is called in unity
Score--;
guiText.text = "Score: "+Score;
}
}

Controlling robot makes a shaking movement

Im trying to control a robot by sending positions with 100hz. It's making a shaking movement when sending so much positions. When I send 1 position that is like 50 mm from his start position it moves smoothly. When I use my sensor to steer,(so it send every position from 0 to 50mm) it is shaking. I'm probably sending like X0-X1-X2-X1-X2-X3-X4-X5-X4-X5 and this is the reason why it might shake. How can I solve this making the robot move smoothly when I use my mouse to steer it?
Robot is asking 125hz
IR sensor is sending 100hz
Otherwise does the 25hz makes the diffrent?
Here is my code.
while(true)
// If sensor 1 is recording IR light.
if (listen1.newdata = true)
{
coX1 = (int) listen1.get1X(); //
coY1 = (int) listen1.get1Y();
newdata = true;
} else {
coX1 = 450;
coY1 = 300;
}
if (listen2.newdata = true)
{
coX2 = (int) listen2.get1X();
coY2 = (int) listen2.get1Y();
newdata = true;
} else {
coY2 = 150;
}
// If the sensor gets further then the workspace, it will automaticly correct it to these
// coordinates.
if (newdata = true)
{
if (coX1< 200 || coX1> 680)
{
coX1 = 450;
}
if (coY1<200 || coY1> 680)
{
coY1 = 300;
}
if (coY2<80 || coY2> 300)
{
coY2 = 150;
}
}
// This is the actually command send to a robot.
Gcode = String.format( "movej(p[0.%d,-0.%d, 0.%d, -0.5121, -3.08, 0.0005])"+ "\n", coX1, coY1, coY2);
//sends message to server
send(Gcode, out);
System.out.println(Gcode);
newdata = false;
}
}
private static void send(String movel, PrintWriter out) {
try {
out.println(movel); /*Writes to server*/
// System.out.println("Writing: "+ movel);
// Thread.sleep(250);
}
catch(Exception e) {
System.out.print("Error Connecting to Server\n");
}
}
}
# Edit
I discovered on wich way I can do this. It is via min and max. So basicly what I think I have to do is:
* put every individual coordinate in a array( 12 coordinates)
* Get the min and max out of this array
* Output the average of the min and max
Without knowing more about your robot characteristics and how you could control it, here are some general considerations:
To have a smooth motion of your robot, you should control it in speed with a well designed PID controller algorithm.
If you can only control it in position, the best you can do is monitoring the position & waiting for it to be "near enough" from the targetted position before sending the next position.
If you want a more detailed answer, please give more information on the command you send to the robot (movej), I suspect that you can do much more than just sending [x,y] coordinates.