How to simplify the search query parameter? - algolia

The problem
I have a movie database with the indexName: 'movies'.
Let's say my query is John then the domain is domain.tld/?movies[query]=John.
I want to simplify the search query parameter to domain.tld/?keywords=John. How can I do that?
What I already know
After reading through the docs I know that I have to modify the createURL and the parseURL somehow:
createURL({ qsModule, location, routeState }) {
const { origin, pathname, hash } = location;
const indexState = routeState['movies'] || {};
const queryString = qsModule.stringify(routeState);
if (!indexState.query) {
return `${origin}${pathname}${hash}`;
}
return `${origin}${pathname}?${queryString}${hash}`;
},
...
parseURL({ qsModule, location }) {
return qsModule.parse(location.search.slice(1));
},

After some try and error here is a solution:
createURL({ qsModule, location, routeState }) {
const { origin, pathname, hash } = location;
const indexState = routeState['movies'] || {}; // routeState[indexName]
//const queryString = qsModule.stringify(routeState); // default -> movies[query]
const queryString = 'keywords=' + encodeURIComponent(indexState.query); // NEW
if (!indexState.query) {
return `${origin}${pathname}${hash}`;
}
return `${origin}${pathname}?${queryString}${hash}`;
},
...
parseURL({ qsModule, location }) {
//return qsModule.parse(location.search.slice(1)); // default: e.g. movies%5Bquery%5D=john
const query = location.search.match(/=(.*)/g) || []; // NEW
const queryString = 'movies%5Bquery%5D' + query[0]; // NEW
return qsModule.parse(queryString); // NEW
},

Related

how to update a collection if you already called it MongoDB Mongoos

Ok so I have a problem in which I use a collection to gather some ratings data and work with it, by the time I finish the rating update process, I have new ratings that I would like to update the collection with. However I can't call update because I get the error "Cannot overwrite model once compiled." I understand that I already called once the model to work with the data and that's why I get the error. is there any way I can update the collection? Or I will just have to workaround by creating a new collection with the latest rating, and then matching the latest ratings collection with the one I use to work with the data.
This is my code
let calculateRating = async () => {
const getData = await matchesCollection().find().lean();
const playerCollection = await playersCollection();
const getDataPlayer = await playerCollection.find().lean();
let gamesCounting = [];
getDataPlayer.forEach((player) => {
player.makePlayer = ranking.makePlayer(1500);
});
for (let i = 0; i < getData.length; i++) {
const resultA = getDataPlayer.findIndex(({ userId }, index) => {
if (userId === getData[i].userA) {
return index;
}
});
const resultB = getDataPlayer.findIndex(
({ userId }) => userId === getData[i].userB
);
const winner = getData[i].winner;
if (getDataPlayer[resultA] === undefined) {
continue;
} else if (getDataPlayer[resultB] === undefined) {
continue;
}
gamesCounting.push([
getDataPlayer[resultA].makePlayer,
getDataPlayer[resultB].makePlayer,
winner,
]);
}
ranking.updateRatings(gamesCounting);
let ratingsUpdate = [];
getDataPlayer.forEach((item) => {
let newRating = item.makePlayer.getRating();
let newDeviation = item.makePlayer.getRd();
let newVolatility = item.makePlayer.getVol();
item.rating = newRating;
item.rd = newDeviation;
item.vol = newVolatility;
ratingsUpdate.push(item);
});
};
I try the work around with creating the new collection

How do I generate SEO URL's using Instantsearch

I'm using Algolia's Instantsearch.js with Typesense adapter, which works well for my search page, but if I go to the search page with a query or refinement in the URL, it doesn't filter. For example, if I do https://example.com/buy-list/buying/Sports (sports being the category), it doesn't automatically filter.
Here is the relevant snippet. I've narrowed it down to somewhere in the router because if I set it router: true, it works, it's just not pretty URL's.
const search = instantsearch({
searchClient,
indexName: INSTANT_SEARCH_INDEX_NAME,
routing: {
router: instantSearchRouter({
windowTitle({category, query}) {
const queryTitle = query ? `Results for "${query}"` : 'Buy List | DA Card World';
if (category) {
return `${category} – ${queryTitle}`;
}
return queryTitle;
},
createURL({qsModule, routeState, location}) {
const urlParts = location.href.match(/^(.*?)\/buying\/buy-list/);
const baseUrl = `${urlParts ? urlParts[1] : ''}/`;
const categoryPath = routeState.category
? `${getCategorySlug(routeState.category)}/`
: '';
const queryParameters = {};
if (routeState.query) {
queryParameters.query = encodeURIComponent(routeState.query);
}
if (routeState.page !== 1) {
queryParameters.page = routeState.page;
}
if (routeState.types) {
queryParameters.types = routeState.types.map(encodeURIComponent);
}
if (routeState.years) {
queryParameters.years = routeState.years.map(encodeURIComponent);
}
if (routeState.series) {
queryParameters.series = routeState.series.map(encodeURIComponent);
}
const queryString = qsModule.stringify(queryParameters, {
addQueryPrefix: true,
arrayFormat: 'repeat'
});
return `${baseUrl}buying/buy-list/${categoryPath}${queryString}`;
},
parseURL({qsModule, location}) {
const pathnameMatches = location.pathname.match(/\/buying\/buy-list\/(.*?)\/?$/);
const category = getCategoryName(
(pathnameMatches && pathnameMatches[1]) || ''
);
const {query = '', page, types = [], years = [], series = []} = qsModule.parse(
location.search.slice(1)
);
// `qs` does not return an array when there's a single value.
const allTypes = Array.isArray(types) ? types : [types].filter(Boolean);
const allYears = Array.isArray(years) ? years : [years].filter(Boolean);
const allSeries = Array.isArray(series) ? series : [series].filter(Boolean);
return {
query: decodeURIComponent(query),
page,
types: allTypes.map(decodeURIComponent),
years: allYears.map(decodeURIComponent),
series: allSeries.map(decodeURIComponent),
category
};
}
}),
stateMapping: {
stateToRoute(uiState) {
const indexUiState = uiState[INSTANT_SEARCH_INDEX_NAME] || {};
return {
query: indexUiState.query,
page: indexUiState.page,
types: indexUiState.refinementList && indexUiState.refinementList.package,
years: indexUiState.refinementList && indexUiState.refinementList.year,
series: indexUiState.refinementList && indexUiState.refinementList.series,
category: indexUiState.hierarchicalMenu && indexUiState.hierarchicalMenu[INSTANT_SEARCH_HIERARCHICAL_ATTRIBUTE]
&& indexUiState.hierarchicalMenu[INSTANT_SEARCH_HIERARCHICAL_ATTRIBUTE].join('>')
};
},
routeToState(routeState) {
return {
instant_search: {
query: routeState.query,
page: routeState.page,
hierarchicalMenu: {
"category": routeState.category && routeState.category.split('>')
},
refinementList: {
packages: routeState.types,
years: routeState.years,
series: routeState.series
}
}
};
}
}
},
searchFunction: function (helper) {
helper.search();
const title = document.querySelector('title');
const header = document.querySelector('#results_title');
let titleText = 'Buy Lists';
if (INSTANT_SEARCH_HIERARCHICAL_ATTRIBUTE in helper.state.hierarchicalFacetsRefinements) {
titleText = helper.state.hierarchicalFacetsRefinements[INSTANT_SEARCH_HIERARCHICAL_ATTRIBUTE] + ' Buy List';
}
title.text = titleText + ' | DA Card World';
header.innerText = titleText;
}
});

Batches with BulkWriter in google firestore

Does anyone know why this does not work, what am I doing wrong here. It get stuck after the console.log "after read stream"
I am trying to read a bunch of files, convert it to json and upload with bulkwriter to firestore.
After each 400 document I am calling close to write them to firestore and then I am creating a new bulkwriter
I also tried awaiting bulkWriter.create(eventDoc, {}) but it does not work. It also get stuck and there is no error. Why is this ? the create method returns a promise.
Why can't it be awaited ?
https://googleapis.dev/nodejs/firestore/latest/BulkWriter.html#create
The idea is to process 1 file at the time and it can contains tens of thousands of rows which needs to be uploaded to firestore
I am calling this method in for...of loop and awaiting the processBatch method
Any help highly appreciated
async processBatch(document: string, file: string): Promise<void> {
const db = admin.firestore();
console.log('start: ', document);
let bulkWriter;
const writeBatchLimit = 400;
let documentsInBatch = 0;
let totalInDocument = 0;
const eventsCollectionRef = db.collection('events');
const eventDoc = eventsCollectionRef.doc(document);
return new Promise((resolve, reject) => {
console.log('promise');
bulkWriter = db.bulkWriter();
const csvStream = fs.createReadStream(file);
console.log('after read stream');
bulkWriter.create(eventDoc, {})
.then(result => {
console.log('Successfully: ', result);
csvStream.pipe(csvParser())
.on('data', row => {
console.log('row');
bulkWriter.create(eventDoc.collection('event').doc(), row);
documentsInBatch++;
if (documentsInBatch > writeBatchLimit) {
bulkWriter.close();
totalInDocument = + documentsInBatch;
documentsInBatch = 0;
bulkWriter = db.bulkWriter();
}
})
.on('end', () => {
console.log('file: ', file + ', totalInDocument: ', totalInDocument);
resolve();
});
})
.catch(err => {
console.log('Failed: ', err);
reject();
});
});
}
This seems to work:
async processBatch(document: string, file: string): Promise<void> {
const db = admin.firestore();
console.log(`start: ${document}`);
let bulkWriter;
const writeBatchLimit = 400;
let documentsInBatch = 0;
let numOfBatches = 0;
let totalInDocument = 0;
const eventsCollectionRef = db.collection('events');
const eventDoc = eventsCollectionRef.doc(document);
bulkWriter = db.bulkWriter();
const csvStream = fs.createReadStream(file);
bulkWriter.create(eventDoc, {});
csvStream.pipe(csvParser())
.on('data', row => {
bulkWriter.create(eventDoc.collection('event').doc(), row);
documentsInBatch++;
if (documentsInBatch > writeBatchLimit) {
numOfBatches++;
totalInDocument += documentsInBatch;
documentsInBatch = 0;
bulkWriter.close();
console.log(`Committing batch ${numOfBatches}, cumulative: ${totalInDocument}`);
bulkWriter = db.bulkWriter();
}
})
.on('end', () => {
console.log(`file: ${file}, totalInDocument: ${totalInDocument}`);
});
}

Chrome Developer Tools - Performance profiling

In the below image (from chrome performance profiling tab for a API call), what is resource loading which costs 719 ms ?
If I visit the network tab, for the same API call, I see only 10.05 seconds.
What is resource loading mean here ? Is there any specific activity the browser does after receiving the data ?
As #wOxxOM stated, buildNetworkRequestDetails is being called from Source Code
From the statement by #Sanju singh :
that statement doesn't tell anything, why it is taking that much time to make the resource available?
I think its necessary to break down exactly what is happening..
Summary:
Activity Browser and Network Activity are using different algorithms for calculating completion. Network Activity is calculating the response times from the request and Activity Browser is calculating the response time + time it took to add it into the WebInspector tracer.
Looking at
/**
* #param {!TimelineModel.TimelineModel.NetworkRequest} request
* #param {!TimelineModel.TimelineModel.TimelineModelImpl} model
* #param {!Components.Linkifier.Linkifier} linkifier
* #return {!Promise<!DocumentFragment>}
*/
static async buildNetworkRequestDetails(request, model, linkifier) {
const target = model.targetByEvent(request.children[0]);
const contentHelper = new TimelineDetailsContentHelper(target, linkifier);
const category = TimelineUIUtils.networkRequestCategory(request);
const color = TimelineUIUtils.networkCategoryColor(category);
contentHelper.addSection(ls`Network request`, color);
if (request.url) {
contentHelper.appendElementRow(ls`URL`, Components.Linkifier.Linkifier.linkifyURL(request.url));
}
// The time from queueing the request until resource processing is finished.
const fullDuration = request.endTime - (request.getStartTime() || -Infinity);
if (isFinite(fullDuration)) {
let textRow = Number.millisToString(fullDuration, true);
// The time from queueing the request until the download is finished. This
// corresponds to the total time reported for the request in the network tab.
const networkDuration = request.finishTime - request.getStartTime();
// The time it takes to make the resource available to the renderer process.
const processingDuration = request.endTime - request.finishTime;
if (isFinite(networkDuration) && isFinite(processingDuration)) {
const networkDurationStr = Number.millisToString(networkDuration, true);
const processingDurationStr = Number.millisToString(processingDuration, true);
const cacheOrNetworkLabel = request.cached() ? ls`load from cache` : ls`network transfer`;
textRow += ls` (${networkDurationStr} ${cacheOrNetworkLabel} + ${processingDurationStr} resource loading)`;
}
contentHelper.appendTextRow(ls`Duration`, textRow);
}
if (request.requestMethod) {
contentHelper.appendTextRow(ls`Request Method`, request.requestMethod);
}
if (typeof request.priority === 'string') {
const priority = PerfUI.NetworkPriorities.uiLabelForNetworkPriority(
/** #type {!Protocol.Network.ResourcePriority} */ (request.priority));
contentHelper.appendTextRow(ls`Priority`, priority);
}
if (request.mimeType) {
contentHelper.appendTextRow(ls`Mime Type`, request.mimeType);
}
let lengthText = '';
if (request.memoryCached()) {
lengthText += ls` (from memory cache)`;
} else if (request.cached()) {
lengthText += ls` (from cache)`;
} else if (request.timing && request.timing.pushStart) {
lengthText += ls` (from push)`;
}
if (request.fromServiceWorker) {
lengthText += ls` (from service worker)`;
}
if (request.encodedDataLength || !lengthText) {
lengthText = `${Number.bytesToString(request.encodedDataLength)}${lengthText}`;
}
contentHelper.appendTextRow(ls`Encoded Data`, lengthText);
if (request.decodedBodyLength) {
contentHelper.appendTextRow(ls`Decoded Body`, Number.bytesToString(request.decodedBodyLength));
}
const title = ls`Initiator`;
const sendRequest = request.children[0];
const topFrame = TimelineModel.TimelineModel.TimelineData.forEvent(sendRequest).topFrame();
if (topFrame) {
const link = linkifier.maybeLinkifyConsoleCallFrame(target, topFrame, {tabStop: true});
if (link) {
contentHelper.appendElementRow(title, link);
}
} else {
const initiator = TimelineModel.TimelineModel.TimelineData.forEvent(sendRequest).initiator();
if (initiator) {
const initiatorURL = TimelineModel.TimelineModel.TimelineData.forEvent(initiator).url;
if (initiatorURL) {
const link = linkifier.maybeLinkifyScriptLocation(target, null, initiatorURL, 0, {tabStop: true});
if (link) {
contentHelper.appendElementRow(title, link);
}
}
}
}
if (!request.previewElement && request.url && target) {
request.previewElement = await Components.ImagePreview.ImagePreview.build(
target, request.url, false,
{imageAltText: Components.ImagePreview.ImagePreview.defaultAltTextForImageURL(request.url)});
}
if (request.previewElement) {
contentHelper.appendElementRow(ls`Preview`, request.previewElement);
}
return contentHelper.fragment;
}
We can easily see that the request parameter is of type
`TimelineModel.TimelineModel.NetworkRequest`
NetWorkRequest has the following code:
_didStopRecordingTraceEvents: function()
{
var metadataEvents = this._processMetadataEvents();
this._injectCpuProfileEvents(metadataEvents);
this._tracingModel.tracingComplete();
this._resetProcessingState();
var startTime = 0;
for (var i = 0, length = metadataEvents.page.length; i < length; i++) {
var metaEvent = metadataEvents.page[i];
var process = metaEvent.thread.process();
var endTime = i + 1 < length ? metadataEvents.page[i + 1].startTime : Infinity;
this._currentPage = metaEvent.args["data"] && metaEvent.args["data"]["page"];
for (var thread of process.sortedThreads()) {
if (thread.name() === WebInspector.TimelineModel.WorkerThreadName && !metadataEvents.workers.some(function(e) { return e.args["data"]["workerThreadId"] === thread.id(); }))
continue;
this._processThreadEvents(startTime, endTime, metaEvent.thread, thread);
}
startTime = endTime;
}
this._inspectedTargetEvents.sort(WebInspector.TracingModel.Event.compareStartTime);
this._cpuProfiles = null;
this._buildTimelineRecords();
this._buildGPUTasks();
this._insertFirstPaintEvent();
this._resetProcessingState();
this.dispatchEventToListeners(WebInspector.TimelineModel.Events.RecordingStopped);
},
We can see that endTime is being calculated from:
metaEvent.thread.process()
We can see that metaEvent.page is being set by:
_processMetadataEvents: function()
{
var metadataEvents = this._tracingModel.devToolsMetadataEvents();
var pageDevToolsMetadataEvents = [];
var workersDevToolsMetadataEvents = [];
for (var event of metadataEvents) {
if (event.name === WebInspector.TimelineModel.DevToolsMetadataEvent.TracingStartedInPage)
pageDevToolsMetadataEvents.push(event);
else if (event.name === WebInspector.TimelineModel.DevToolsMetadataEvent.TracingSessionIdForWorker)
workersDevToolsMetadataEvents.push(event);
}
if (!pageDevToolsMetadataEvents.length) {
// The trace is probably coming not from DevTools. Make a mock Metadata event.
var pageMetaEvent = this._loadedFromFile ? this._makeMockPageMetadataEvent() : null;
if (!pageMetaEvent) {
console.error(WebInspector.TimelineModel.DevToolsMetadataEvent.TracingStartedInPage + " event not found.");
return {page: [], workers: []};
}
pageDevToolsMetadataEvents.push(pageMetaEvent);
}
var sessionId = pageDevToolsMetadataEvents[0].args["sessionId"] || pageDevToolsMetadataEvents[0].args["data"]["sessionId"];
this._sessionId = sessionId;
var mismatchingIds = new Set();
/**
* #param {!WebInspector.TracingModel.Event} event
* #return {boolean}
*/
function checkSessionId(event)
{
var args = event.args;
// FIXME: put sessionId into args["data"] for TracingStartedInPage event.
if (args["data"])
args = args["data"];
var id = args["sessionId"];
if (id === sessionId)
return true;
mismatchingIds.add(id);
return false;
}
var result = {
page: pageDevToolsMetadataEvents.filter(checkSessionId).sort(WebInspector.TracingModel.Event.compareStartTime),
workers: workersDevToolsMetadataEvents.filter(checkSessionId).sort(WebInspector.TracingModel.Event.compareStartTime)
};
if (mismatchingIds.size)
WebInspector.console.error("Timeline recording was started in more than one page simultaneously. Session id mismatch: " + this._sessionId + " and " + mismatchingIds.valuesArray() + ".");
return result;
}

Draft.js. How can I update block atomic:image src later on for example article save?

I am having trouble updating my image blocks in editorState in draft.js.
I want to change atomic:image src on button save.
So the src is for example now blob:http://localhost:3000/7661d307-871b-4039-b7dd-6efc2701b623
but I would like to update to src to for example /uploads-from-my-server/test.png
onSave(e) {
e.preventDefault();
const { editorState } = this.state;
const contentState = editorState.getCurrentContent();
editorState.getCurrentContent().getBlockMap().map((block) => {
const type = block.getType();
if (type === 'atomic:image') {
const rangeToReplace = new SelectionState({
anchorKey: block.getKey(),
focusKey: block.getKey(),
});
Modifier.replaceText(contentState, rangeToReplace, '/uploads-from-my-server/test.png');
const newContentState = editorState.getCurrentContent();
this.setState({ editorState: newContentState });
}
return true;
});
I know I can access src string with block.getData().get('src') but I cant set though
Thank you for your awesome editor
I was struggling with a similar problem, I ended up converting the content state to raw array using convertToRaw and then updating it manually and use convertFromRaw and set the new state :
import {EditorState, ContentState, convertToRaw, convertFromRaw /*, ...*/} from 'draft-js';
// ...
onSave(e) {
e.preventDefault();
const { editorState } = this.state;
const contentState = editorState.getCurrentContent();
let rawContent = convertToRaw(contentState);
for(let i = 0; i < rawContent.blocks.length; i++) {
let b = rawContent.blocks[i];
if(b['type'] !== "unstyled" && b.entityRanges.length === 1) {
const entityKey = b['entityRanges'][0]['key'];
const entityMap = rawContent['entityMap'][entityKey];
if(entityMap["type"] === "image") {
rawContent['entityMap'][entityKey]['data']['src'] = '/uploads-from-my-server/test.png';
}
}
}
const newContentState = convertFromRaw(rawContent);
const newEditorState = EditorState.push(this.state.editorState, newContentState, 'update-contentState');
this.setState({editorState: newEditorState});
}
Note: This is not a fully working example, it's just a starting point. Hope it helps :)