I am trying to get CloudWatchLogs filterLogEvents by sending parameters.
some times it work and sends me the exact logs in that time, but for the different lambda function it returns empty events with next token.
parameters = {
'logGroupName' : metricFilter.logGroupName,
'filterPattern' : metricFilter.filterPattern ? metricFilter.filterPattern : "",
'startTime' : timestamp - offset,
'endTime' : timestamp
};
i am retrying by sending parameters with next token but its still getting empty events. Does anyone has idea about it?
{
"events": [],
"searchedLogStreams": [],
"nextToken": "long text"
}
You might have been previously testing with a query from Cloudwatch Log Insights, then via the StartQuery API and noted that you can not currently paginate over the results GetQueryResults.
Looking to overcome the limitation of 10000 records then you might have tried the FilterLogEvents, that supports pagination. If that is the case, please note that the parameters startTime and endTime from FilterLogEvents are specified in miliseconds while the parameters startTime and endTime from StartQuery must be specified in seconds.
So in order to use the same code used to calculate startTime and endTime from startQuery you need to multiply the timestamp value * 1000.
I had the same problem when I tried to capture the aws clould watch logs using aws sdk.
I finally solved this issue when I used logStreamNamePrefix, which in my case is the uuid of the device.
So I created the function below and when I run it the first time I do it without the nextToken and the subsequent times I take the token in the request and call it again using the token to continue the search.
loadCloudWatchLogs(nextToken?: string) {
// Set the region
AWS.config.update({
region: 'sa-east-1',
credentials: {
accessKeyId: environment.awsAccessKeyId,
secretAccessKey: environment.awsSecretAccessKey,
},
});
// Create the CloudWatchLogs service object
const cloudwatchlogs = new AWS.CloudWatchLogs({ apiVersion: '2014-03-28' });
// Defines the params attributes pattern.
let params: AWSCloudWatchParams;
// Check if token was provided to permorm a next query to AWSCloudWatchLogs.
if (nextToken) {
params = {
logGroupName: 'group-name' /* required */,
startTime: this.startTime,
endTime: this.endTime,
logStreamNamePrefix: this.device,
filterPattern: `{ ($.device="${this.device}") }`,
nextToken,
};
} else {
params = {
logGroupName: 'group-name' /* required */,
startTime: this.startTime,
endTime: this.endTime,
logStreamNamePrefix: this.device,
filterPattern: `{ ($.device="${this.device}") }`,
};
}
// Execute a filter for logs.
cloudwatchlogs.filterLogEvents(params, (err, data) => {
if (err) {
console.log(err, err.stack);
} else {
if (data.searchedLogStreams.length > 0) {
// Chegk if exists more logs for query.
const loadMoreLogs = !data.searchedLogStreams[
data.searchedLogStreams.length - 1
].searchedCompletely;
// Update the token for the next query.
this.nextToken = data.nextToken;
data.events.forEach(log => {
// log proccessing...
});
}
}
});
}
This function I did in an Angular project, but it will work even with javascript vanila making some adjustments.
Related
I have the following code which returns the internal ID of a sales order by looking it up from a support case record.
So the order of events is:
A support case is received via email
The free text message body field contains a reference to a sales order transaction number. This is identified by the use of the number convention of 'SO1547878'
A workflow is triggered on case creation from the email case creation feature. The sales order number is extracted and stored in a custom field.
The internal ID of the record is looked up and written to the console (log debug) using the workflow action script below:
*#NApiVersion 2.x
*#NScriptType WorkflowActionScript
* #param {Object} context
define(["N/search", "N/record"], function (search, record) {
function onAction(context) {
var recordObj = context.newRecord;
var oc_number = recordObj.getValue({ fieldId: "custevent_case_creation" });
var s = search
.create({
type: "salesorder",
filters: [
search.createFilter({
name: "tranid",
operator: search.Operator.IS,
values: [oc_number],
}),
],
columns: ["internalid"],
})
.run()
.getRange({
start: 0,
end: 1,
});
log.debug("result set", s);
return s[0];
}
return {
onAction: onAction,
};
});
I am trying to return the resulting internal ID as a parameter so I can create a link to the record on the case record.
I'm getting stuck trying to work out how I would do this?
Is there a way to store the value on the case record, of the internal ID, that is looked up? (i.e.the one currently on the debug logs)?
I am very new to JS and Suitescript so am not sure at what point in this process, this value would need to be stored in the support case record.
At the moment. the workflow action script (which is the part of the workflow the above script relates to) is set to trigger after submit
Thanks
Edit: Thanks to Bknights, I have a solution that works.
The workflow:
The new revised script is as follows:
*#NApiVersion 2.x
*#NScriptType WorkflowActionScript
* #param {Object} context
*/
define(["N/search", "N/record"], function (search, record) {
function onAction(context) {
var recordObj = context.newRecord;
var oc_number = recordObj.getValue({ fieldId: "custevent_case_creation" });
var s = search
.create({
type: "salesorder",
filters: [
search.createFilter({
name: "tranid",
operator: search.Operator.IS,
values: [oc_number],
}),
],
columns: ["internalid"],
})
.run()
.getRange({
start: 0,
end: 1,
});
log.debug("result set", s[0].id);
return s[0].id;
}
return {
onAction: onAction,
};
});
On the script record for the workflow action script, set the type of return you expect. In this case, it would be a sales order record:
This would allow you to use a list/record field to store the value from the 'search message' workflow action created by the script
the result
Edit 2: A variation of this
/**
*#NApiVersion 2.x
*#NScriptType WorkflowActionScript
* #param {Object} context
*/
define(["N/search", "N/record"], function (search, record) {
function onAction(context) {
try {
var recordObj = context.newRecord;
var oc_number = recordObj.getValue({
fieldId: "custevent_case_creation",
});
var s = search
.create({
type: "salesorder",
filters: [
search.createFilter({
name: "tranid",
operator: search.Operator.IS,
values: [oc_number],
}),
],
columns: ["internalid","department"],
})
.run()
.getRange({
start: 0,
end: 1,
});
log.debug("result set", s[0]);
recordObj.setValue({fieldId:'custevent_case_sales_order', value:s[0].id});
// return s[0]
} catch (error) {
log.debug(
error.name,
"recordObjId: " +
recordObj.id +
", oc_number:" +
oc_number +
", message: " +
error.message
);
}
}
return {
onAction: onAction,
};
});
Depending on what you want to do with the order link you can do a couple of things.
If you want to reference the Sales Order record from the Support Case record you'd want to add a custom List/Record field to support cases that references transactions. (ex custevent_case_order)
Then move this script to a beforeSubmit UserEvent script and instead of returning extend it like:
recordObj.setValue({fieldId:'custevent_case_order', value:s[0].id});
For performance you'll probably want to test whether you are in a create/update event and that the custom order field is not yet filled in.
If this is part of a larger workflow you may still want to look up the Sales Order in the user event script and then start you workflow when that field has been populated.
If you want to keep the workflow intact your current code could return s[0].id to a workflow or workflow action custom field and then apply it to the case with a Set Field Value action.
Good day everyone, I am working with watermelondb and I have the code below, but I don't know how to actually use it. I am new in watermelondb and I don't know how to pass data as props to the pullChanges and pushChanges objects. How do I pass necessary data like changes and lastPulledAt from the database into the sync function when I call it. And I need more explanation on the migrationsEnabledAtVersion: 1 too. Thanks in advance for your gracious answers.
import { synchronize } from '#nozbe/watermelondb/sync'
async function mySync() {
await synchronize({
database,
pullChanges: async ({ lastPulledAt, schemaVersion, migration }) => {
const urlParams = `last_pulled_at=${lastPulledAt}&schema_version=${schemaVersion}&migration=${encodeURIComponent(JSON.stringify(migration))}`
const response = await fetch(`https://my.backend/sync?${urlParams}`)
if (!response.ok) {
throw new Error(await response.text())
}
const { changes, timestamp } = await response.json()
return { changes, timestamp }
},
pushChanges: async ({ changes, lastPulledAt }) => {
const response = await fetch(`https://my.backend/sync?last_pulled_at=${lastPulledAt}`, {
method: 'POST',
body: JSON.stringify(changes)
})
if (!response.ok) {
throw new Error(await response.text())
}
},
migrationsEnabledAtVersion: 1,
})
}
Watermelondb's documentation is terrible and its link to typescript even worse.
I spent almost a week to get 100% synchronization with a simple table, now I'm having the same problems to solve the synchronization with associations.
Well, the object you need to return in pullChanges is of the following form:
return {
changes: {
//person is the name of the table in the models
person: {
created: [
{
// in created you need to send null in the id, if you don't send the id it doesn't work
id: null,
// other fields of your schema, not model
}
],
updated: [
{
// the fields of your schema, not model
}
],
deleted: [
// is a string[] consisting of the watermelondb id of the records that were deleted in the remote database
],
}
},
timestamp: new Date().getTime() / 1000
}
In my case, the remote database is not a watermelondb, it's a mySQL, and I don't have an endpoint in my API that returns everything in the watermelon format. For each table I do a search with deletedAt, updatedAt or createdAt > lastPulledAt and do the necessary filtering and preparations so that the data from the remote database is in the schema format of the local database.
In pushChanges I do the reverse data preparation process by calling the appropriate creation, update or deletion endpoints for each of the tables.
It's costly and annoying to do, but in the end it works fine, the biggest problem is watermelon's documentation which is terrible.
I am trying to make an assistant app and was using the cloud firestore service of firebase to send the response back to my app using webhook as fulfilment. I have used 'session' parameter in request JSON according to this documentation and sending fulfilmentText as response to the user. But whenever user launches the app, a new session is created which I don't want. I simply want, just a single entry for each user in my database so how to achieve that using dialogflow.
In Alexa Skill, we have deviceId as parameter by which we can uniquely identify the user irrespective of the session id but is there any parameter in the dialogflow request JSON. If not, then how to achieve this task without it.
The request JSON I am getting from Dialogflow has a userID in it, so can I use the userId or should I go with userStorage provided the userStorage parameter is not available in the request JSON.
request.body.originalDetectIntentRequest { source: 'google', version: '2', payload: { surface: { capabilities: [Object] },
inputs: [ [Object] ],
user:
{ locale: 'en-US',
userId: 'ABwppHG5OfRf2qquWWjI-Uy-MwfiE1DQlCCeoDrGhG8b0fHVg7GsPmaKehtxAcP-_ycf_9IQVtUISgfKhZzawL7spA' },
conversation:
{ conversationId: '1528790005269',
type: 'ACTIVE',
conversationToken: '["generate-number-followup"]' },
availableSurfaces: [ [Object] ] } }
EDIT : Thank You #Prisoner for the answer but I am unable to send the random ID generated in the response and set in in the payload. Below is the code where I am generating the uuid and storing it in firestore. What I am doing wrong in the below code due to which new uuid is generated for returning user and therefore response is shown as No document found in the database. I suppose I am not sending uuid appropriately. Please help.
exports.webhook = functions.https.onRequest((request, response) => {
console.log("request.body.queryResult.parameters", request.body.queryResult.parameters);
console.log("request.body.originalDetectIntentRequest.payload", request.body.originalDetectIntentRequest.payload);
let userStorage = request.body.originalDetectIntentRequest.payload.user.userStorage || {};
let userId;
console.log("userStorage", userStorage);
if (userId in userStorage) {
userId = userStorage.userId;
} else {
var uuid = require('uuid/v4');
userId = uuid();
userStorage.userId = userId
}
console.log("userID", userId);
switch (request.body.queryResult.action) {
case 'FeedbackAction': {
let params = request.body.queryResult.parameters;
firestore.collection('users').doc(userId).set(params)
.then(() => {
response.send({
'fulfillmentText' : `Thank You for visiting our ${params.resortLocation} hotel branch and giving us ${params.rating} and your comment as ${params.comments}.` ,
'payload': {
'google': {
'userStorage': userStorage
}
}
});
return console.log("resort location", params.resortLocation);
})
.catch((e => {
console.log('error: ', e);
response.send({
'fulfillmentText' : `something went wrong when writing to database`,
'payload': {
'google': {
'userStorage': userStorage
}
}
});
}))
break;
}
case 'countFeedbacks':{
var docRef = firestore.collection('users').doc(userId);
docRef.get().then(doc => {
if (doc.exists) {
// console.log("Document data:", doc.data());
var dat = doc.data();
response.send({
'fulfillmentText' : `You have given feedback for ${dat.resortLocation} and rating as ${dat.rating}`,
'payload': {
'google': {
'userStorage': userStorage
}
}
});
} else {
// doc.data() will be undefined in this case
console.log("No such document!");
response.send({
'fulfillmentText' : `No feedback found in our database`,
'payload': {
'google': {
'userStorage': userStorage
}
}
});
}
return console.log("userStorage_then_wala", userStorage);
}).catch((e => {
console.log("Error getting document:", error);
response.send({
'fulfillmentText' : `something went wrong while reading from the database`,
'payload': {
'google': {
'userStorage': userStorage
}
}
})
}));
break;
}
You have a couple of options, depending on your exact needs.
Simple: userStorage
Google provides a userStorage object which is persisted across conversations when it can identify a user. This lets you store your own identifier when you need to track when a user returns.
The easiest way to do this is to check the userStorage object for the identifier when your webhook is called. If it doesn't exist, create one using something like a v4 UUID and save it in the userStorage object.
If you are using the actions-on-google library, the code might look something like this:
let userId;
// if a value for userID exists un user storage, it's a returning user so we can
// just read the value and use it. If a value for userId does not exist in user storage,
// it's a new user, so we need to generate a new ID and save it in user storage.
if (userId in conv.user.storage) {
userId = conv.user.storage.userId;
} else {
// Uses the "uuid" package. You can get this with "npm install --save uuid"
var uuid = require('uuid/v4');
userId = uuid();
conv.user.storage.userId = userId
}
If you are using the dialogflow library, you can use the above, but you'll need this line first:
let conv = agent.conv();
If you're using the multivocal library, it does all of the above for you and will provide a UserID in the environment under the path User/Id.
If you're handling the JSON directly, and you are using the Dialogflow v2 protocol, you can get the userStorage object by examining originalDetectIntentRequest.payload.user.userStorage in the JSON request object. You'll set the payload.google.userStorage object in the JSON response. The code is similar to the above and might look something like this:
let userStorage = body.originalDetectIntentRequest.payload.user.userStorage || {};
let userId;
// if a value for userID exists un user storage, it's a returning user so we can
// just read the value and use it. If a value for userId does not exist in user storage,
// it's a new user, so we need to generate a new ID and save it in user storage.
if (userId in userStorage) {
userId = userStorage.userId;
} else {
// Uses the "uuid" package. You can get this with "npm install --save uuid"
var uuid = require('uuid/v4');
userId = uuid();
userStorage.userId = userId
}
// ... Do stuff with the userID
// Make sure you include the userStorage as part of the response
var responseBody = {
payload: {
google: {
userStorage: JSON.stringify(userStorage),
// ...
}
}
};
Note the first line of the code - if userStorage doesn't exist, use an empty object. It won't exist until you send a response that includes storing something in it for the first time, which will happen in the last few lines of this code.
Advanced: Account Linking
You can request users to sign in to your Action using Google Sign In. This can be done just using voice for the simplest of cases and would only interrupt the flow the first time.
After this, your Action is given a JWT which contains their Google ID which you can use as their identifier.
If you're using the actions-on-google library, you can get the ID from the decoded JWT with a line such as:
const userId = conv.user.profile.payload.sub;
In the multivocal library, the ID from the decoded JWT is available in the environment under the path User/Profile/sub
Deprecated: Anonymous User ID
You'll see some answers here on StackOverflow that reference an Anonymous User ID. Google has deprecated this identifier, which was not always a reliable way to verify returning users, and will be removing it 1 Jun 2019.
This code is currently still being sent, but this will be removed starting 1 Jun 2019.
I've been having this weird issue with an application I'm building. Essentially a function is invoked I want to read in a user's current game statistics -Wins, losses, draws etc - I do this using a service which creates an observable and consumes data from my rest api. On first call of this method the data read in is the most current up to date version but after this point I update the document for the user in the database and then when I execute the function again it reads in the original document before the update. However when I check the database the document has in face been updated.
Here is my provider function for consuming the data.
getUser(id) {
if (this.data) {
return Promise.resolve(this.data);
}
return new Promise(resolve => {
this.http.get('https://pitchlife-hearts.herokuapp.com/api/users/' + id)
.map(res => res.json())
.subscribe(data => {
this.data = data;
resolve(this.data);
});
});
}
Here is the call I make in my function.
play(challenger, opponent) {
this.userService.getUser(_id).then((data) => {
this.challenger_account = {
_id: data._id,
points: data.maroon_points,
wins: data.wins,
draws: data.draws,
losses: data.losses
};
Here is my update call.
this.userService.updateUser(this.challenger_account);
Here is my api endpoint call as well although this does work every time I update the data.
app.post('/api/users/update', function (req, res) {
// Update a user
var options = {};
User.update({_id : req.body._id }, {
maroon_points: req.body.points,
wins: req.body.wins,
draws: req.body.draws,
losses: req.body.losses
}, options,
function (err, user) {
if (err)
res.send(err);
res.json(user);
});
});
Any help with this would be hugely appreciated as this is driving me crazy.
When are you updating the this.data property that the getUser(id) { ... } method uses?
Because the first time the getUser(id) {...} method is executed, this.data is null and because of that the http request is made. But after that, the value of this.data is always returned, but if you don't update it manually, it'll be always the first value it was set to.
I am in the process of changing the schema for one of my MongoDB collections. (I had been storing dates as strings, and now my application stores them as ISODates; I need to go back and change all of the old records to use ISODates as well.) I think I know how to do this using an update, but since this operation will affect tens of thousands of records I'm hesitant to issue an operation that I'm not 100% sure will work. Is there any way to do a "dry run" of an update that will show me, for a small number of records, the original record and how it would be changed?
Edit: I ended up using the approach of adding a new field to each record, and then (after verifying that the data was right) renaming that field to match the original. It looked like this:
db.events.find({timestamp: {$type: 2}})
.forEach( function (e) {
e.newTimestamp = new ISODate(e.timestamp);
db.events.save(e);
} )
db.events.update({},
{$rename: {'newTimestamp': 'timestamp'}},
{multi: true})
By the way, that method for converting the string times to ISODates was what ended up working. (I got the idea from this SO answer.)
My advice would be to add the ISODate as a new field. Once confirmed that all looks good you could then unset the the string date.
Create a test environment with your database structure. Copy a handful of records to it. Problem solved. Not the solution you were looking for, I'm sure. But, I believe, this is the exact circumstances that a 'test environment' should be used for.
Select ID of particular records that you would like to monitor. place in the update {_id:{$in:[<your monitored id>]}}
Another option which depends of the amount of overhead it will cause you -
You can consider writing a script, that performs the find operation, add printouts or run in debug while the save operation is commented out. Once you've gained confidence you can apply the save operation.
var changesLog = [];
var errorsLog = [];
events.find({timestamp: {$type: 2}}, function (err, events) {
if (err) {
debugger;
throw err;
} else {
for (var i = 0; i < events.length; i++) {
console.log('events' + i +"/"+(candidates.length-1));
var currentEvent = events[i];
var shouldUpdateCandidateData = false;
currentEvent.timestamp = new ISODate(currentEvent.timestamp);
var change = currentEvent._id;
changesLog.push(change);
// // ** Dry Run **
// currentEvent.save(function (err) {
// if (err) {
// debugger;
// errorsLog.push(currentEvent._id + ", " + currentEvent.timeStamp + ', ' + err);
// throw err;
// }
// });
}
console.log('Done');
console.log('Changes:');
console.log(changesLog);
console.log('Errors:');
console.log(errorsLog);
return;
}
});
db.collection.find({"_manager": { $exists: true, $ne: null }}).forEach(
function(doc){
doc['_managers']=[doc._manager]; // String --> List
delete doc['_manager']; // Remove "_managers" key-value pair
printjson(doc); // Debug by output the doc result
//db.teams.save(doc); // Save all the changes into doc data
}
)
In my case the collection contain _manager and I would like to change it to _managers list. I have tested it in my local working as expected.
In the several latest versions of MongoDB (at least starting with 4.2), you could do that using a transaction.
const { MongoClient } = require('mongodb')
async function main({ dryRun }) {
const client = new MongoClient('mongodb://127.0.0.1:27017', {
maxPoolSize: 1
})
const pool = await client.connect()
const db = pool.db('someDB')
const session = pool.startSession()
session.startTransaction()
try {
const filter = { id: 'some-id' }
const update = { $rename: { 'newTimestamp': 'timestamp' } }
// This is the important bit
const options = { session: session }
await db.collection('someCollection').updateMany(
filter,
update,
options // using session
)
const afterUpdate = db.collection('someCollection')
.find(
filter,
options // using session
)
.toArray()
console.debug('updated documents', afterUpdate)
if (dryRun) {
// This will roll back any changes made within the session
await session.abortTransaction()
} else {
await session.commitTransaction()
}
} finally {
await session.endSession()
await pool.close()
}
}
const _ = main({ dryRun: true })