Asyncio consumer put element in queue as reference. They duplicates in loop - queue

Question

I have implemented async paginated api requester with timeout, queue and multiple tokens (workers).
I have 3 workers (requesters), that consuming elements from queue. If result from request telling me that it is page 1 and total pages more than 1, my code put already used arguments to queue with another page param (from 2 to "total page"):
if result['page'] < result['total_pages'] and result['page'] == 1:
for page in range(2, result['total_pages']+1):
request_kwargs['params']['page'] = page
queue.put_nowait(request_kwargs)
Everything works good until adding new arguments to the queue:
On the first for loop queue._queue (in debug) looks like:
deque([{'method': 'get', 'url': 'some_url', 'params': {'count': 100, 'page': 2}}])
But on the second for loop something goes wrong and adding new args overwrite previous and adding the same:
deque([{'method': 'get', 'url': 'some_url', 'params': {'count': 100, 'page': 3}},{'method': 'get', 'url': 'some_url', 'params': {'count': 100, 'page': 3}}])
...
On the 6th loop i have 6 similar elements in queue, with similar {'page':6}
Where am i wrong? Why does putting new request_kwargs overwrites existing ones?
Adding new elements works fine when i am just adding 'page' instead of dict with page in it.
That works fine:
if result['page'] < result['total_pages'] and result['page'] == 1:
for page in range(2, result['total_pages']+1):
queue.put_nowait(page)
I think that adding new element to queue put reference to that element, not an instance. A have tried to add additional var before putting it into queue and deleting var after putting. Nothing changes.

Answer 1

From the context that You are providing, You are actually putting the same object over and over again:
queue.put_nowait(request_kwargs)
You are not creating a separate request_kwargs variable in each loop pass. It is the same, mutable object every time the for loop iterates.
And moreover, You are performing explicit mutations of that variable on each loop pass:
request_kwargs['params']['page'] = page
This way, You are:
changing all previous queue items, updating the page - cause each query item is actually the same object
pushing the same queue item again
I think that such change should resolve Your issues:
import copy
# ...
if result['page'] < result['total_pages'] and result['page'] == 1:
for page in range(2, result['total_pages']+1):
request_kwargs = copy.deepcopy(request_kwargs)
request_kwargs['params']['page'] = page
queue.put_nowait(request_kwargs)
But using deepcopy always bring questions if my code is actually properly designed. And here it may not be.