sending request through proxy. request library works, axios does not - axios

I am trying to update some old code to get rid of the request package since it is no longer maintained. I attempted to replace a proxy request with axios, but it doesn't work (I just get a timeout). Am I missing an axios config somewhere? The example using the request package works fine.
FAILS
export function sendAxiosApiRequest(enableProxy, proxyIndex,url,filepath?:string):object {
//https://support.zyte.com/support/discussions/topics/22000014602
let ca='READ IN FILE HERE'
let getOptions = {
url: url,
httpsAgent: tunnel.httpsOverHttp({
ca: ca,
proxy: {
host: 'http://MY_API_KEY:#proxy.crawlera.com',
port: '8011',
},
}),
proxy: false, //disable auto config, bc we set it manually
} as any;
console.log({getOptions})
return new Promise(resolve => {
try {
axios.get(getOptions,(err,response,html)=>{
if(err){
console.log(err);
resolve(false);
}
else {
try{
const output = JSON.parse(html);
resolve(output);
}catch(e){
console.log({html})
throw `ERROR parsing html: `+JSON.stringify(e)
}
}
})
}
catch (e) {
console.log(`Err parsing result from sendApiRequest:`,e);
resolve(false);
}
})
}
WORKS
export function sendRequestApiRequest(enableProxy, proxyIndex,url,filepath?:string):object {
let ca='READ IN FILE HERE'
let getOptions = {
url: url,
jar: true,
followAllRedirects: false,
} as any;
//console.log({filepath})
getOptions.proxy= 'http://MY_API_KEY:#proxy.crawlera.com'
getOptions.ca=ca
getOptions.requestCert =true
getOptions.rejectUnauthorized= true
return new Promise(resolve => {
try {
request.get(getOptions,(err,response,html)=>{
if(err){
console.log(err);
resolve(false);
}
else {
const output = JSON.parse(html);
resolve(output);
}
})
}
catch (e) {
console.log(e);
resolve(false);
}
})
}

Please have a look at the axios docs.
The method signature for the get-requests is axios.get(url[, config]) but your first parameter is actually an object. You might want to use axios({}) and update your getOptions with the missing method key:
let getOptions = {
url: url,
method: 'get', // this was missing!
httpsAgent: tunnel.httpsOverHttp({
ca: ca,
proxy: {
host: 'MY_API_KEY:#proxy.crawlera.com', // http is not needed, but it was http but you use httpsAgent?!
port: 8011,
},
}),
proxy: false,
} as any;

Related

Cookie does not appear to be sent via fetch or hapi server is unable to receive cookie

So I have a simple backend server created with Hapi API and the frontend I'm using fetch. These are on different ports so I have CORs enabled and all the sweet stuff. I'm currently trying to set a refresh token in the browser using a http only cookie. As far as I can verify, the http only cookie is being set in the browser when login function is completed. I'm currently trying to send the http only cookie back to the server so I can set up the refresh token route and I can't seem to send or even verify that http token is sent back to the server.
Here's the server setting.
"use strict";
require("dotenv").config();
const Hapi = require("#hapi/hapi");
const Jwt = require("#hapi/jwt");
const routes = require("./routes/routes");
exports.init = async () => {
const server = Hapi.server({
port: 3000,
host: "localhost",
routes: {
cors: {
origin: ["*"],
credentials: true,
},
},
});
require("./models");
await server.register(Jwt);
server.auth.strategy("jwt", "jwt", {
keys: { key: process.env.SECRET_KEY, algorithms: ["HS256"] },
verify: { aud: false, iss: false, sub: false, exp: true },
validate: false,
});
server.state("refresh", {
ttl: 1000 * 60 * 60 * 24,
isSecure: true,
isHttpOnly: true,
encoding: "base64json",
clearInvalid: true,
strictHeader: true,
isSameSite: "None",
});
server.route(routes);
return server;
};
process.on("unhandledRejection", (err) => {
console.log(err);
process.exit(1);
});
Here's the login request and returns the http only cookie. This part works, the http cookie is returned and set.
const validateUserAndReturnToken = async (req, h) => {
const user = await User.findOne({
$or: [{ email: req.payload.username }, { username: req.payload.username }],
});
if (user) {
const match = await bcrypt.compare(req.payload.password, user.passwordHash);
if (match) {
const token = await createToken(match);
const refreshToken = await createRefreshToken(match);
h.state("refresh", refreshToken);
return { id_token: token, user: formatUser(user) };
} else {
throw boom.notAcceptable("Username and password did not match.");
}
} else {
throw boom.notAcceptable("Username or email was not found.");
}
};
Here's the fetch request I'm using to test sending a http cookie only back. I have credential: include so I don't know what is problem?
import type { DateInfo } from "#/stores/application";
const api = "http://localhost:3000/report";
let token = localStorage.getItem("user-token");
const headers = new Headers();
headers.append("Authorization", `Bearer ${token}`);
headers.append("Content-Type", "application/json");
export const getJobReport = async (dateFilter: DateInfo) => {
let response = await fetch(
`${api}/${dateFilter.startDate}/${dateFilter.endDate}`,
{
method: "GET",
headers,
credentials: "include",
}
);
return await response.json();
};
I have checked the application tab as well as the network request so I know set cookie is being sent and set on the browser. The problem is I can't seem to get the cookie back from the browser when fetch request is sent back to the server.
Here's the code I'm using to just check the existence of the cookie. According to Hapi Doc , req.state[cookie-name] which in this case is 'refresh' should have the cookie value. Refresh is returning undefined so I went up one level and check for req.state and gets an empty object {}.
route
{
method: "GET",
path: "/report/{startDate}/{endDate}",
options: {
auth: "jwt",
state: {
parse: true,
failAction: "error",
},
validate: {
params: Joi.object({
startDate: Joi.string(),
endDate: Joi.string(),
}),
},
},
handler: handlers.report.getJobApplicationReport,
},
handler
const getJobApplicationReport = async (req, h) => {
console.log("TEST", req.state);
const start = new Date(req.params.startDate);
const end = new Date(req.params.endDate);
try {
const applications = await Application.find({
dateApplied: { $gte: start, $lt: end },
});
// 'Applied', 'In Process', 'Rejected', 'Received Offer'
const total = applications.length;
let rejectedCount = 0;
let inProcessCount = 0;
applications.forEach((app) => {
if (app.status === "Rejected") {
rejectedCount++;
}
if (app.status === "In Process") {
inProcessCount++;
}
});
return {
total,
rejectedCount,
inProcessCount,
};
} catch (error) {
console.log(error);
throw boom.badRequest(error);
}
};
I've looked through all the Hapi documentation, fetch documentation and stackoverflow question/answers but can't seem to find a solution. I can't verify whether it's the fetch request that's not sending the http only cookie or the server setting that's not parsing it. Any help to determine the issue or solution would be greatly appreciated.
I've looked through all the Hapi documentation, fetch documentation and stackoverflow question/answers but can't seem to find a solution. I can't verify whether it's the fetch request that's not sending the http only cookie or the server setting that's not parsing it. Any help to determine the issue or solution would be greatly appreciated.

How to configure API URL running on localhost on port 8080 in nuxt js?

I have the following vue file. My REST API base URL is http://localhost:8080/api/. When I access http://localhost:8080/api/dfc/system/docbases directly, I get the response as shown.
["gr_swy","SubWayX_DEMO"]
But I want to get the response through nuxt js which is running on http://localhost:3000/restapi/. I tried to follow all the articles, but not able to figure out where I'm doing wrong.
<template>
<div class="container">
{{docbases}}
</div>
</template>
<script>
import axios from "axios";
#import axios from "../../.nuxt/axios"; (tried both)
export default {
methods: {
// asyncData({ req, params }) {
// return axios.get("http://localhost:8080/api/dfc/system/docbases")
// .then(res => {
// return { docbases: res.data };
// }).catch((e) => {
// error({ statusCode: 404, message: 'Not found' })
// })
// },
async asyncData ({ params }) {
const { data } = await axios.get('http://localhost:8080/api/dfc/system/docbases');
return { docbases: data }
}
},
head: {
title: "D2Rest"
}
};
</script>
My nuxt.config.js is like this: I tried changeOrigin with true and false both. Can you please help me what extra things I need to configure?
axios: {
proxy: true,
},
env: {
baseUrl: process.env.BASE_URL || 'http://localhost:3000'
},
proxy: {
'/api/': {
target: 'http://localhost:8080/',
pathRewrite: { "^/api": "" },
changeOrigin: false,
prependPath: false
}
},
Based on your configuration, I'm assuming you're using the Nuxt Axios module...
The problem seems to be that you're importing Axios unnecessarily, thus bypassing your axios configuration in nuxt.config.js. The Nuxt Axios module docs describe its usage in components:
export default {
async asyncData({ $axios }) {
const ip = await $axios.$get('http://icanhazip.com')
return { ip }
}
}
Note the destructured parameter $axios. Use that parameter instead of importing your own instance of axios (i.e., don't do import axios from 'axios'), which is not the same as the one configured by Nuxt. No other imports are needed for $axios.
Proxy URL
Another problem is that your explicitly requesting the proxy address in the URL, but that should be excluded:
// const { data } = await $axios.get('http://localhost:8080/api/dfc/system/docbases'); // DON'T DO THIS
const { data } = await $axios.get('/api/dfc/system/docbases');
Sorry, I didn't enable Cross Origin in my java code. I have enabled now and it's resolved.
#CrossOrigin(origins = "http://localhost:3000")

How can I write conditional interceptor in Axios

I am able to add an interceptor for the Axios pipeline. Also, I need the loader to be conditional based. The situation is some requests can run in the background and don't need a loader to be blocking the UI. In such cases, I will be able to let the Axios know by sending an extra parameter saying isBackground call. How can I achieve this?
axios.interceptors.request.use((config) => {
this.isLoading = true; // Or trigger start loader
return config
}, (error) => {
this.isLoading = false // Or trigger stoploader
return Promise.reject(error)
})
axios.interceptors.response.use((response) => {
this.isLoading = false // Or trigger stoploader
return response
}, function(error) {
this.isLoading = false // Or trigger stoploader
return Promise.reject(error)
})
Just use your own custom property isBackground on the config like this:
axios.interceptors.request.use((config) => {
console.log(config.isBackground)
return config
}, (error) => {
console.log(error.config.isBackground)
return Promise.reject(error)
})
axios.interceptors.response.use((response) => {
console.log(response.config.isBackground)
return response
}, function(error) {
console.log(error.config.isBackground)
return Promise.reject(error)
})
const config = {
isBackground: true
}
axios.get('https://httpbin.org/get', config)
.then(response => {
console.log(response)
})
.catch(error => {
console.log(error)
})
Note that there is a bug in current release 0.19.0 waiting to be fixed, which breaks this functionality. Works ok in version 0.18...
Fiddle

Implementing redirect in Redux middleware

Let's say I have following action:
export function signIn(data) {
return {
type: USER_SIGN_IN,
promise: api.post('/sign_in', data)
}
}
and following middleware:
export default function promiseMiddleware() {
return next => action => {
const { promise, type, ...rest } = action
if (!promise) {
return next(action)
}
const SUCCESS = `${type}_SUCCESS`
const REQUEST = `${type}_REQUEST`
const ERROR = `${type}_ERROR`
next({ type: REQUEST, ...rest })
return promise
.then(res => {
next({ response: res.data, type: SUCCESS, ...rest })
return true
})
.catch(error => {
...
})
}
}
This code is loosely based on https://github.com/reactGo/reactGo/
But what if in then callback after calling next I want to make a redirect to another path?
I did following. I passed redirect url through action:
export function signIn(data) {
return {
type: USER_SIGN_IN,
promise: api.post('/sign_in', data),
redirect: '/'
}
}
and added another call of next method with push from react-router-redux.
import { push } from 'react-router-redux'
export default function promiseMiddleware() {
return next => action => {
const { promise, type, redirect, ...rest } = action
...
return promise
.then(res => {
next({ response: res.data, type: SUCCESS, ...rest })
next(push(redirect))
return true
})
.catch(error => {
...
})
}
}
It seems like it works, but I'm not sure if this is a good idea or if there are some pitfalls of multiple next calls and I shouldn't do like this?
Maybe there are some better approaches for implementing such redirects?

Multiple functions in restify function to elasticsearch client

I'm building a REST API using node and restify that communicaties with an elasticsearch database. Now when I delete an object, I want this to do a kind of cascading delete to some other objects. I know this is not really what to use elasticsearch for but bear with me.
So here is my code:
function deleteHostname(req, res, next) {
res.setHeader('Access-Control-Allow-Origin', '*');
var endpoints = [];
client.search({
index: 'test',
type: 'something',
body: {
from: 0, size: 100,
query: {
match: {
hostname: 'www.test.com'
}
}
}
}).then(function (error, resp) {
if(error) {
res.send(error);
}
endpoints = resp.hits.hits;
for (index = 0, len = endpoints.length; index < len; ++index) {
client.delete({
index: 'test',
type: 'something',
id: endpoints[index]._id
}, function (error, response) {
if(error) {
res.send(error);
}
});
}
res.send(endpoints);
return next();
});
}
So basically I just want to search for any objects with hostname www.test.com ( I just hard coded this to test it ). Then I want to delete all objects I found. It follows the error path and sends me this:
{
"took":1,
"timed_out":false,
"_shards":{
"total":5,
"successful":5,
"failed":0
},
"hits":{
"total":1,
"max_score":2.098612,
"hits":[
{
"_index":"test",
"_type":"something",
"_id":"123456",
"_score":2.098612,
"_source":{
"duration":107182,
"date":"2016-05-04 00:54:43",
"isExceptional":true,
"hostname":"www.test.com",
"eta":613,
"hasWarnings":false,
"grade":"A+",
"ipAddress":"ipip",
"progress":100,
"delegation":2,
"statusMessage":"Ready"
}
}
]
}
}
So in my opinion this doesn't look like an error? So why am I getting it back as an error? If I remove:
if(error) {
res.send(error);
}
From my code, I won't get any response.
You need to change your code like this (see the changes denoted by -> to the left):
if(error) {
1-> return res.send(error);
}
endpoints = resp.hits.hits;
for (index = 0, len = endpoints.length; index < len; ++index) {
2-> (function(id){
client.delete({
index: 'test',
type: 'something',
3-> id: id
}, function (error, response) {
if(error) {
4-> next(error);
}
});
5-> })(endpoints[index._id]);
}
6-> //res.send(endpoints);
I'm now explaining each change:
If you don't return you'll send the error and then you'll continue with processing the hits
(3/5) Since client.delete is an asynchronous function, you need to call it in an anonymous function
In case of error you need to call next(error) not res.send
You cannot send the response at this point since your for loop might not be terminated yet. Instead of a for loop, you should use the excellent async library instead (see an example of using asynch.each below)
Async example:
var async = require('async');
...
if(error) {
return res.send(error);
}
endpoints = resp.hits.hits;
async.each(endpoints,
function(endpoint, callback) {
client.delete({
index: 'test',
type: 'something',
id: endpoint._id
}, callback);
},
// this is called when all deletes are done
function(err){
if (err) {
next(err);
} else {
res.send(endpoints);
next();
}
}
);
Another solution for you to achieve exactly what you want is to use the delete by query plugin. That feature allows you to do all the above in a single query.
If you are still on ES 1.x, delete-by-query is still part of the core and you can simply call the deleteByQuery function of the Javascript client.
If you are on ES 2.x, delete-by-query is now a plugin, so yo need to install it and then also require the deleteByQuery extension library for the Javascript client
function deleteHostname(req, res, next) {
res.setHeader('Access-Control-Allow-Origin', '*');
client.deleteByQuery({
index: 'test',
type: 'something',
body: {
query: {
match: { hostname: 'www.test.com' }
}
}
}, function (error, response) {
if (error) {
next(error);
} else {
res.send(endpoints);
next();
}
});
}