I try to call rest API from Bitfinex by using bitfinex library (https://www.npmjs.com/package/bitfinex). The documentation warned this error when running more than 1 process at the same time with the same API keys. However, I believe that I only run a process at once and even I generate new API keys, the error still happens. I'm thinking about overriding the nonce or extending it but I don't know how to do this. I haved tried some suggestions on the Internet, but still nothing worked.
Below is my code for reference:
import Bitfinex = require('bitfinex');
import pollingtoevent = require('polling-to-event');
import { Logger, LoggerFactory } from '../../common';
import { AppDataServices } from '../../data';
export class BitfinexPoller {
private static readonly LOGGER: Logger = LoggerFactory.getLogger();
private bitfinex: any = undefined;
private emitter: any = undefined;
private public_key: string = '<my-public-key>';
private secret_key: string = '<my-secret-key>';
private nonce: any = new Date().getTime();
constructor(private appServices: AppDataServices) {
BitfinexPoller.LOGGER.info('Bitfinex poller init');
this.bitfinex = new Bitfinex(this.public_key, this.secret_key, this.nonce);
// Lend book service
this.emitter = pollingtoevent((done: any) => {
this.bitfinex.lendbook('USD', (err: any, res: any, orderId: any) => {
done(err, res);
});
}, { interval: 30000, eventName: 'bitfinex-lending' });
this.emitter.on('bitfinex-lending', (data: any) => {
BitfinexPoller.LOGGER.info(data);
if (data.bids !== undefined) {
for (const row of data.bids) {
appServices.lendbookService.insert(BitfinexPoller.lendingData('BID', row));
}
}
if (data.asks !== undefined) {
for (const row of data.asks) {
appServices.lendbookService.insert(BitfinexPoller.lendingData('ASK', row));
}
}
});
// Order book service
this.emitter = pollingtoevent((done: any) => {
this.bitfinex.orderbook('btcusd', (err: any, res: any, orderId: any) => {
done(err, res);
});
}, { interval: 30000, eventName: 'bitfinex-order' });
this.emitter.on('bitfinex-order', (data: any) => {
BitfinexPoller.LOGGER.info(data);
if (data.bids !== undefined) {
for (const row of data.bids) {
appServices.lendbookService.insert(BitfinexPoller.orderData('BID', row));
}
}
if (data.asks !== undefined) {
for (const row of data.asks) {
appServices.lendbookService.insert(BitfinexPoller.orderData('ASK', row));
}
}
});
}
}
It's most likely you're firing off two authenticated calls in quick succession. The order that you send requests is not always the same as the order that they're processed, so the second request with the higher nonce is being processed first causing the first request to fail.
You can create and use multiple API keys for different requests or have a pool that you cycle through so you don't use the same api key multiple times in quick succession.
My solution with nodejs has been to defer the calls via promises.
Something like this:
function sleep(millis) {
return new Promise(resolve => setTimeout(resolve, millis));
}
sleep(500).then(() => {
//do work
});
sleep(600).then(() => {
//do work
});
Obviously if you are looking at extremely low latency and high speed this isn't for you.
Related
In response to a click handler, i currently perform a:
axios.get("http://whatever.com/itemId").then(response => {
console.log(response);
});
theoretically, a user might click several times extremely quickly in succession. however, I need to process responses in the same order as the requests are made, and these responses are not always ordered.
Can anyone give any pointers?
Came up with the following, which seems to work well. Maybe such a thing should be part of the library itself. Hope others find it useful.
import axios from 'axios';
class OrderedAxios {
constructor() {
this.promises = []
}
get(url) {
var outsideResolve;
var outsideReject;
let promise = new Promise((resolve, reject) => {
outsideResolve = resolve;
outsideReject = reject;
});
promise.resolve = outsideResolve;
promise.reject = outsideReject;
this.promises.push(promise);
axios.get(url)
.then(response => {
promise.response = response;
this.processPromises();
})
.catch(error => {
promise.error = error;
this.processPromises();
})
return promise;
}
processPromises() {
let continueProcessing = false;
do {
continueProcessing = false;
if (this.promises.length) {
let promise = this.promises[0];
if (promise.response || promise.error) {
this.promises.shift();
if (promise.response) {
promise.resolve(promise.response);
} else {
promise.reject(promise.error);
}
continueProcessing = true;
}
}
} while (continueProcessing);
}
}
export default OrderedAxios;
Im struggling with getting axios interceptors to work.
When my token expires, i need it to refresh the access token and retry the original request once the token is refreshed.
I have this part working.
The problem is if i have concurrent api calls it will only retry the first request when the token was first invalid.
Here is my interceptor code:
export default function execute() {
let isRefreshing = false
// Request
axios.interceptors.request.use(
config => {
var token = Storage.getAccessToken() //localStorage.getItem("token");
if (token) {
console.log('Bearer ' + token)
config.headers['Authorization'] = 'Bearer ' + token
}
return config
},
error => {
return Promise.reject(error)
}
)
// Response
axios.interceptors.response.use(
response => {
return response
},
error => {
const originalRequest = error.config
// token expired
if (error.response.status === 401) {
console.log('401 Error need to reresh')
originalRequest._retry = true
let tokenModel = {
accessToken: Storage.getAccessToken(),
client: 'Web',
refreshToken: Storage.getRefreshToken()
}
//Storage.destroyTokens();
var refreshPath = Actions.REFRESH
if (!isRefreshing) {
isRefreshing = true
return store
.dispatch(refreshPath, { tokenModel })
.then(response => {
isRefreshing = false
console.log(response)
return axios(originalRequest)
})
.catch(error => {
isRefreshing = false
console.log(error)
// Logout
})
} else {
console.log('XXXXX')
console.log('SOME PROBLEM HERE') // <------------------
console.log('XXXXX')
}
} else {
store.commit(Mutations.SET_ERROR, error.response.data.error)
}
return Promise.reject(error)
}
)
}
I'm not sure what i need in the else block highlighted above.
EDIT:
When I do
return axios(originalRequest)
in the else block it works, however im not happy with the behaviours. It basically retries all the requests again and again until the token is refreshed.
I would rather it just retried once after the token had been refreshed
Any ideas
Thanks
You can just have additional interceptor which can refresh token and execute your pending requests.
In this, countDownLatch class can help.
Here is sample Interceptor code,
class AutoRefreshTokenRequestInterceptorSample() : Interceptor {
companion object {
var countDownLatch = CountDownLatch(0)
var previousAuthToken = ""
const val SKIP_AUTH_TOKEN = "SkipAccessTokenHeader"
const val AUTHORIZATION_HEADER = "AUTHORIZATION_HEADER_KEY"
}
#Throws(IOException::class)
override fun intercept(chain: Interceptor.Chain): Response? {
val request = chain.request()
if (shouldExecuteRequest(request)) {
// Execute Request
val response = chain.proceed(request)
if (!response.isSuccessful) {
// Failed Case
val errorBody = response.peekBody(java.lang.Long.MAX_VALUE).string()
val error = parseErrorModel(errorBody)
// Gives Signal to HOLD the Request Queue
countDownLatch = CountDownLatch(1)
handleError(error!!)
// After updating token values, execute same request with updated values.
val updatedRequest = getUpdatedRequest(request)
// Gives Signal to RELEASE Request Queue
countDownLatch.countDown()
//Execute updated request
return chain.proceed(updatedRequest)
} else {
// success case
return response
}
}
// Change updated token values in pending request objects and execute them!
// If Auth header exists, and skip header not found then hold the request
if (shouldHoldRequest(request)) {
try {
// Make this request to WAIT till countdown latch has been set to zero.
countDownLatch.await()
} catch (e: Exception) {
e.printStackTrace()
}
// Once token is Updated, then update values in request model.
if (previousAuthToken.isNotEmpty() && previousAuthToken != "newAccessToken") {
val updatedRequest = getUpdatedRequest(request)
return chain.proceed(updatedRequest)
}
}
return chain.proceed(request)
}
private fun handleError(error: ErrorDto) {
// update your token as per your error code logic
//Here it will make new API call to update tokens and store it in your local preference.
}
/***
* returns Request object with updated token values.
*/
private fun getUpdatedRequest(request: Request): Request {
var updateAuthReqBuilder: Request.Builder = request.newBuilder()
var url = request.url().toString()
if (url.contains(previousAuthToken.trim()) && previousAuthToken.trim().isNotEmpty()) {
url = url.replace(previousAuthToken, "newAccessToken")
}
updateAuthReqBuilder = updateAuthReqBuilder.url(url)
// change headers if needed
return updateAuthReqBuilder.build()
}
private fun shouldExecuteRequest(request: Request) =
shouldHoldRequest(request) && isSharedHoldSignalDisabled()
/**
* If count down latch has any value then it is reported by previous request's error signal to hold the whole pending chain.
*/
private fun isSharedHoldSignalDisabled() = countDownLatch.count == 0L
private fun shouldHoldRequest(request: Request) = !hasSkipFlag(request) && hasAuthorizationValues(request)
private fun hasAuthorizationValues(request: Request) = isHeaderExist(request, AUTHORIZATION_HEADER)
private fun hasSkipFlag(request: Request) = isHeaderExist(request, SKIP_AUTH_TOKEN)
private fun isHeaderExist(request: Request, headerName: String): Boolean {
return request.header(headerName) != null
}
private fun parseErrorModel(errorBody: String): Error? {
val parser = JsonParser()
// Change this logic according to your requirement.
val jsonObject = parser.parse(errorBody).asJsonObject
if (jsonObject.has("Error") && jsonObject.get("Error") != null) {
val errorJsonObj = jsonObject.get("Error").asJsonObject
return decodeErrorModel(errorJsonObj)
}
return null
}
private fun decodeErrorModel(jsonObject: JsonObject): Error {
val error = Error()
// decode your error object here
return error
}
}
This is how I do:
let isRefreshing = false;
let failedQueue = [];
const processQueue = (error, token = null) => {
failedQueue.forEach(prom => {
if (error) {
prom.reject(error);
} else {
prom.resolve(token);
}
});
failedQueue = [];
};
axios.interceptors.response.use(
response => response,
error => {
const originalRequest = error.config;
if (error.response.status === 400) {
// If response is 400, logout
store.dispatch(logout());
}
// If 401 and I'm not processing a queue
if (error.response.status === 401 && !originalRequest._retry) {
if (isRefreshing) {
// If I'm refreshing the token I send request to a queue
return new Promise((resolve, reject) => {
failedQueue.push({ resolve, reject });
})
.then(() => {
originalRequest.headers.Authorization = getAuth();
return axios(originalRequest);
})
.catch(err => err);
}
// If header of the request has changed, it means I've refreshed the token
if (originalRequest.headers.Authorization !== getAuth()) {
originalRequest.headers.Authorization = getAuth();
return Promise.resolve(axios(originalRequest));
}
originalRequest._retry = true; // mark request a retry
isRefreshing = true; // set the refreshing var to true
// If none of the above, refresh the token and process the queue
return new Promise((resolve, reject) => {
// console.log('REFRESH');
refreshAccessToken() // The method that refreshes my token
.then(({ data }) => {
updateToken(data); // The method that sets my token to localstorage/Redux/whatever
processQueue(null, data.token); // Resolve queued
resolve(axios(originalRequest)); // Resolve current
})
.catch(err => {
processQueue(err, null);
reject(err);
})
.then(() => {
isRefreshing = false;
});
});
}
return Promise.reject(error);
},
);
I don't know what is the schema of your token (after decrypted) but one of the attributes which is a good practice to keep is the exp "expiration_date".
Said so, having the expiration date you can know when you should refresh your token.
Without understanding your architecture is hard to inform the right solution. But let's say you are doing everything manually, usually onIdle/onActive is when we check if the user session is still ok, so at this time you could use the token info to know if you should refresh its value.
It is important to understand this process because the token should be refreshed only if the user is constantly active and it is about to expire (like 2min before).
Please refer to angular version of the code for which i was facing the same problem and after changing many approaches this was my final code which is working at its best.
Re Initaite the last failed request after refresh token is provided
I'm building an observable data service based on the following article: https://coryrylan.com/blog/angular-2-observable-data-services
In the article he used an array as an example, here I will use the user object since I'm developing the user service.
Here's what I got:
import { Injectable } from '#angular/core';
import { Http, Response } from '#angular/http';
import { Events, SqlStorage, Storage } from 'ionic-angular';
import { Subject } from 'rxjs/Subject';
export interface DataStore {
user: Object
}
#Injectable()
export class UserService {
private baseUrl: string;
private storage: Storage;
private _user$: Subject<Object>;
private dataStore: DataStore;
constructor(
private http: Http
) {
this.baseUrl = 'http://localhost:3000';
this.storage = new Storage(SqlStorage);
this._user$ = <Subject<Object>>new Subject();
this.dataStore = {
user: { name: '' }
};
}
set user$(user: Object) {
this.storage.set('user', JSON.stringify(user));
this.dataStore.user = user;
this._user$.next(this.dataStore.user);
}
get user$() {
return this._user$.asObservable();
}
loadUser() {
return this.storage.get('user').then(
((user: string): Object => {
this.dataStore.user = JSON.parse(user);
this._user$.next(this.dataStore.user);
return this.dataStore.user;
})
);
}
login(accessToken: string) {
return this.http
.post('http://localhost:3000/login', { access_token: accessToken })
.retry(2)
.map((res: Response): any => res.json());
}
logout(): void {
this.storage.remove('user');
}
}
To authenticate I call the login() function and set the user data if everything ok.
this.userService.login(this.data.accessToken)
.subscribe(
(user: Object) => {
this.userService.user$ = user;
this.nav.setRoot(EventListComponent);
},
(error: Object) => console.log(error)
);
I feel it is better set the user data inside the service. I could do the following:
login(accessToken: string) {
return this.http
.post('http://localhost:3000/login', {
access_token: accessToken
})
.retry(2)
.map((res: Response): any => res.json())
.subscribe(
(user: Object) => {
this.userService.user$ = user;
this.nav.setRoot(EventListComponent);
},
(error: Object) => console.log(error)
);
}
But I won't be able to subscribe to the login() function in the component since it's already subscribed. How could I redirect the user if everything ok or show an alert if anything goes wrong in the component but setting the user inside the service?
In the main component I load the user data and set the rootPage:
this.userService.loadUser().then(
(user: Object) => this.rootPage = EventListComponent,
(error: Object) => this.rootPage = LoginComponent
);
I thought that calling the loadUser() function at this time I would not have to call it again, but I have to call it in all components that I need the user data:
this.user = this.userService.user$;
this.userService.loadUser();
I don't think the service is the way it should, what could I improve? Is there any better way to achieve what I want? Any example or idea?
Someone please help me not kill my Server.. Here's my MVC controller action:
(Don't worry about names, I'm mid-refactoring)
public async Task<ActionResult> AllByLead(int leadId)
{
try
{
var lead = await _researchService.GetLeadWithContacts(leadId);
var contactViewModels = Mapper.Map<Lead, List<ContactViewModel>>(lead);
contactViewModels.Each(contact => PopulateContactOptions(contact));
var listViewModel = new ContactListViewModel {Results = contactViewModels};
return PartialView(listViewModel);
}
catch
{
return Json(string.Format(Resources.BasicErrorMessageFormat, "Error retrieving Lead Contacts"),
JsonRequestBehavior.AllowGet);
}
}
Service:
public async Task<Lead> GetLeadWithContacts(int leadId)
{
return await _repository.GetLeadWithContacts(leadId).ConfigureAwait(false);
}
Repo:
public async Task<Lead> GetLeadWithContacts(int leadId)
{
var leadEntity = await _context.Leads
.Where(lead => lead.LeadID == leadId)
//.Include(lead => lead.LeadContactMaps.Select(map => map.Contact.Addresses))
//.Include(lead => lead.LeadContactMaps.Select(map => map.Contact.PhoneNumbers))
//.Include(lead => lead.Organizations.Select(business => business.Addresses))
//.Include(lead => lead.Organizations.Select(business => business.PhoneNumbers))
.FirstOrDefaultAsync();
return leadEntity;
}
EDIT
DbContext Module
internal class DbContextModule : Module
{
protected override void Load(ContainerBuilder builder)
{
builder.Register(ctx => new CRTechEntities()).InstancePerLifetimeScope();
}
}
JS Ajax Call:
function populateContactList() {
var leadId = $("#LeadId").val();
$.ajax({
url: url + "/Contact/AllByLead/",
type: "GET",
data: { 'leadId': leadId },
success: function(data) {
$("#contactContainer").html(data);
},
error: function(data) {
}
});
}
Bonus points on if you can school me on my includes, they may very well be terrible.. It's pretty slow just grabbing the lead right now. Hence the async change. I'd like to see if it will be easier on the system with more users. (I can do my own profiling/tests on whether explicit loading will be better here, just saying..)
Anyway, I hit this, the server is completely borked when the await FirstOrDefaultAsync() gets hit.
EDIT 2: I've updated the controller action to show exactly what I'm doing here. I only included the code that was being hit originally.
Um, are you returning anything in your controller? That would cause it to hang.
Try
public async Task<JsonResult> AllByLead(int leadId)
{
var lead = await _researchService.GetLeadWithContacts(leadId);
return Json(lead);
}
I have successfully created one route in ember-cli-mirage, but am having trouble loading the related data.
The API should be returning JSON API compliant data.
I'm not really sure if there are any good methods or not for debugging mirage's request interception. Here is my config.js
export default function() {
this.urlPrefix = 'https://myserver/';
this.namespace = 'api/v1';
this.get('/machines', function(db, request) {
return {
data: db.machines.map(attrs => (
{
type: 'machines',
id: attrs.id,
attributes: attrs
}
))
};
});
this.get('/machines/:id', function(db, request){
let id = request.params.id;
debugger;
return {
data: {
type: 'machines',
id: id,
attributes: db.machines.find(id),
relationships:{
"service-orders": db["service-orders"].where({machineId: id})
}
}
};
});
this.get('/machines/:machine_id/service-orders', function(db, request){
debugger; // this never gets caught
});
}
Most of this is working fine (I think). I can create machines and service orders in the factory and see the db object being updated. However, where my application would normally make a call to the api for service-orders: //myserver/machines/:machine_id/service-orders, the request is not caught and nothing goes out to the API
EDIT:
This is the route that my Ember app is using for /machines/:machine_id/service-orders:
export default Ember.Route.extend(MachineFunctionalRouteMixin, {
model: function() {
var machine = this.modelFor('machines.show');
var serviceOrders = machine.get('serviceOrders');
return serviceOrders;
},
setupController: function(controller, model) {
this._super(controller, model);
}
});
And the model for machines/show:
export default Ember.Route.extend({
model: function(params) {
var machine = this.store.find('machine', params.machine_id);
return machine;
},
setupController: function(controller, model) {
this._super(controller, model);
var machinesController = this.controllerFor('machines');
machinesController.set('attrs.currentMachine', model);
}
});
Intuitively, I would think that machine.get('serviceOrders'); would make a call to the API that would be intercepted and handled by Mirage. Which does not seem to be the case