const client = new MongoClient(uri);
await client.connect();
await client
.db('mydb1')
.collection('foo');
const session = client.startSession();
const transactionOptions = {
readPreference: 'primary',
readConcern: { level: 'local' },
writeConcern: { w: 'majority' }
};
try {
await session.withTransaction(async () => {
const coll1 = client.db('mydb1').collection('foo');
await coll1.updateOne({user_id: 12344, paid: false }, { $set: { paid: true } } { session });
// long running computation after this line.
// what if another query deletes the document inserted above
// before this transaction completes.
await calls_third_party_payment_vendor_api_to_process_payment();
}, transactionOptions);
} finally {
await session.endSession();
await client.close();
}
What if the update document inside the transaction is simultaneously updated from an outside query before the transaction is committed?
What you have described is a transaction/operation conflict and
the operation blocks behind the transaction commit and infinitely retries with backoff logic until MaxTimeMS is reached
I wrote a report on MongoDB transactions, containing some examples in NodeJS too. if you are interested on the subject, I recommend you the paragraphs WiredTiger Cache and What happens when we create a multi-document transaction?
I am using axios to stream data from an API, something like this
const response = await axios.get(url,
{
responseType: `stream`,
}
);
const stream = response.data;
stream.on(`data`, (data) => {
...
});
stream.on(`error`, (err) => {
...
});
From time-to-time I get an error
{
"code": "ECONNRESET"
}
Does this mean it's the API that is throwing this error as opposed to my application that caused it?
Thank you,
i'm implemented transaction in the post method. it was work fine. But now I have to update unit test case for that method. I tried to mock startSession() and startTransaction() to check toHaveBeenCalled.But while running test case i got like MongooseError: Connection 0 was disconnected when calling startSession``. I am new to that so i don't know how to mock that?.
Method:
static post = (funcCall: Promise<Document>) => async (response: Response, fields?: string[]) => {
const session = await startSession();
session.startTransaction();
try {
const dbResponse = await funcCall; // model.save(request.body)
// commit the changes if everything was successful
await session.commitTransaction();
success(pick(dbResponse, fields ? fields : ['_id']), 201)(response);
} catch (error) {
// this will rollback any changes made in the database
await session.abortTransaction();
throwError(error);
} finally {
// ending the session
session.endSession();
}
};
My Test case:
it('should perform post when valid parameters is passed.', async () => {
// Preparing
const mockSaveReturn = {
_id: objectID,
test_name: 'sample',
};
jest.spyOn(mongoose, 'startSession')
const spySave = jest.spyOn(modelPrototype.prototype, 'save').mockReturnValueOnce(mockSaveReturn);
const document = new modelPrototype(mockSaveReturn);
// Executing
await post(document.save())(response as Response);
expect(response.send).toHaveBeenCalledWith({ _id: '54759eb3c090d83494e2d804' });
expect(spySave).toHaveBeenCalled();
// Cleaning
spySave.mockClear();
});
scenario:
im setting autocommit false,
producing 12 messages
consuming them.. (say from offset 100)
shutting down consumer
start a new consumer
in that stage, I expect to the 2nd consumer to read all messages again starting offset 100 (because no committing was done)
but when producing new messages I see that the 2nd consumer starts from the new offset (113) i.e the commit still occurs somehow..
what do I get wrong?
that is my consumer code
const { Kafka } = require('kafkajs');
const kafka = new Kafka({
clientId: 'my-app',
brokers: ['192.168.14.10:9095']
});
const admin = kafka.admin();
const consumer = kafka.consumer({ groupId: 'test-group' });
const run = async () => {
// admin
await admin.connect();
// Consuming
await consumer.connect();
await consumer.subscribe({ topic: 'topic-test2'});
await consumer.run({
autoCommit: false,
eachMessage: async ({ topic, partition, message }) => {
console.log({
partition,
offset: message.offset,
value: message.value.toString()
});
}
}
});
};
run().catch(console.error);
I am new to React hooks/Context API. I have read the React hook/context docs, and I am still having trouble with the following:
My attempts to update global state by multiple consumer components
currently causes frequent overwriting of context state due to
rerendering (e.g., activity or details state is sometimes
null/undefined). This probably is why...
... I am getting 400 (bad request) and/or 500 (server) errors on random refreshes of the page (~30% of the time content loads as
expected, ~70% errors are thrown. I believe this is happening
because we have various context states that are being called
asynchronously).
I am not sure how to implement Axios Cancellation, given that our useEffect hooks are calling dispatch functions (e.g.,
getActivities()) in different files. The examples I've seen
involve fetching data within the component (rather than in context).
I am seeking assistance for #1 specifically. I would love guidance on how to accurately fetch data and store in context as global state, and then provide that context to child components, allowing them to consume/update context state without unnecessary rerendering.
Tried to only provide relevant code snippets below:
ActivityState.js -- should fetch activity data
...
const ActivityState = props => {
const initialState = {
activities: [],
isLoading: false,
isError: false
};
const HEADERS = {
'Content-Type': 'application/json',
'user_id': 1
}
const [state, dispatch] = useReducer(ActivityReducer, initialState);
const userContext = useContext(UserContext);
const getActivities = async () => {
const { loggedIn } = contactContext;
let didCancel = false; // attempts to start implementing axios cancellation
try {
const res = await axios.get(url);
dispatch({ type: GET_ACTIVITIES, payload: res.data.data.activities });
} catch (err) {
if (!didCancel) {
dispatch({ type: 'FETCH_FAILURE' });
}
}
}
const updateActivity = (path, data) => { //update context state
dispatch({ type: UPDATE_ACTIVITY, payload: { path: path, data: data } });
};
const saveActivity = () => { //send new activity data to the backend
const postData = {
actions: [{"293939": []}],
activities: state.activities
};
try {
const res = axios.post(url,{ data: postData }, { headers: HEADERS });
} catch (err) {
console.log(err);
}
}
return (
<ActivityContext.Provider
value={{
activities: state.activities,
data: state.data,
backup_data: state.backup_data,
getActivities,
updateActivity,
saveActivity,
}}
>
{props.children}
</ActivityContext.Provider>
);
};
export default ActivityState;
ActivityReducer.js -- switch statements to be dispatched by ActivityState.js
...
export default (state, action) => {
switch (action.type) {
case GET_ACTIVITIES:
return {
...state,
activities: action.payload,
isLoading: true
};
case FETCH_FAILURE:
return {
...state,
isLoading: false,
isError: true
};
case UPDATE_ACTIVITY:
const { payload: { path }, payload } = action;
const data = state;
if (!data.activities)
return { data };
const index = data.activities.findIndex(e => e.socium_tracking_number == path.id);
if(index === -1)
return { data };
_.set(data, `activities[${index}].${path.field}`, payload.data);
return {
data,
};
...
DetailsState.js -- dispatch functions to fetch details
const DetailsState = props => {
const initialState = {
details: null,
};
const [state, dispatch] = useReducer(DetailsReducer, initialState);
const getDetails = async () => {
try {
const res = await axios.get(url);
dispatch({ type: GET_DETAILS, payload: res.data.data[0].details});
}catch(err) {
console.log(err)
}
};
return (
<DetailsContext.Provider
value={{ details: state.details, getDetails }}
>
{ props.children }
</DetailsContext.Provider>
);
}
export default SchemaState;
DetailsReducer.js -- switch statement
export default (state, action) => {
switch (action.type) {
case GET_DETAILS:
return {
...state,
details: action.payload,
};
default:
return state;
}
};
ActivityTable.js -- component that consumes Activity Info
...
const ActivityTable = ({ activity }) => {
const activityContext = useContext(ActivityContext);
const { activities, filtered, getActivities } = activityContext;
const [order, setOrder] = React.useState('asc');
const [orderBy, setOrderBy] = React.useState(activities.wait_time);
// Get activity data on mount
useEffect(() => {
async function fetchData() {
await getActivities()
}
fetchData();
}, []);
...
CreateActivity.js -- component that consumes Activity and Details data
...
const CreateActivity = props => {
const activityContext = useContext(ActivityContext);
const { activities, filtered, getActivities, addActivity } = activityContext;
const detailsContext = useContext(DetailsContext);
const { details, getDetails } = detailsContext;
// Get activity and details data on mount
useEffect(() => {
async function fetchData() {
await getActivities();
await getSchema();
}
fetchData();
}, []);
...
I really tried to get smarter on these issues before approaching the SO community, so that my question(s) was more defined. But this is what I have. Happy to provide any info that I missed or clarify confusion. Thank you for your time