weird object returned by vector_indexing_suite - boost-python

Question

I have a
std::vector<const T*>
that I return from a c++ function:
getallTs()
I have exposed the T class with:
class_<T,T*>
and the vector like so:
class_<std::vector<const T*> >("TsList")
.def(vector_indexing_suite<std::vector<const T*>,true>())
;
What does the NoProxy argument mean?
I expose the function like so:
def("getallTs", getallTs,
return_value_policy<return_by_value>{});
I observe a weird behaviour.
When I call from python
tlist = getallTs()
I get a TsList object.
len(tlist)
works.
tlist[<anycorrectindex>].<someattribute>
also works.
However, if I just
print(tlist[0])
and
print(tlist[100])
python prints
object T at <address>
This address is the same for all the Ts in tlist.
Also, I cannot iterate over Tlist with a python for loop.
for t in tlist:
doesn't work.
Any ideas what is wrong with the way I am exposing the vector and the function to python?
I understand the python objects that each wrap a c++ T hold a raw pointer to T.
These T instances exist throughout the process in a global table.
The c++ function retunrns a vector of pointers to those instances.
What does indexing_suite do with those?
Thanks,

Answer 1

When accessing elements by index, the indexing suite defaults to providing a proxy to the element, as a means to provide reference semantics for mutable types that Python users will often expect with collections:
val = c[i]
c[i].m() # Mutates state, equivalent to `val.m()`
assert(val == c[i]) # Have same state.
val.m()
assert(val == c[i]) # Have same state.
In the above example, val is a proxy object that is aware of the container element. When NoProxy is true, one gets value semantics when indexing, resulting in a copy on each index access.
val = c[i] # val is a copy.
c[i].m() # Modify a copy of c[i].
assert(val == c[i]) # These have the same state because c[i] returns a new copy.
val.m()
assert(val != c[i]) # These do not have the same state.
When proxies are not used, the mutations to the elements will only persists when invoked on a reference to the element, such as during iteration:
for val in c:
val.m() # modification observed in c[#]
When invoking print(c[i]), a temporary proxy object is created and passed to print, and the lifetime of the proxy object ends upon returning from print(). Hence, the memory and identification used by the temporary proxy object may be re-used. This can result in elements appearing to have the same identification:
id0 = id(c[0]) # id of the temporary proxy
id1 = id(c[1]) # id of another temporary proxy
assert(id0 ?? id1) # Non-deterministic if these will be the same.
assert(c[0] is not c[1]) # Guaranteed to not be the same.
On the other hand, during the lifetime of a proxy, other proxies to the same element will have identical identification, and proxies to different elements will have different identification:
c0 = c[0] # proxy to element 0.
c0_2 = c[0] # another proxy to element 0.
c1 = c[1] # proxy to element 1
assert(c0 is c0_2)
assert(c0 is c[0])
assert(c0 is not c1)
In the situation where T has been exposed as being held by T*, iteration over std::vector<const T*> will fail in Python if there is no to-Python conversion for const T* to a Python object. Exposing class T as being held by T* registers automatic to-Python and from-Python conversions for T*, not const T*. When iterating over the collection in Python, references to elements are returned, resulting in a Python object failing to be constructed from a const T*. On the other hand, when accessing elements via index, the resulting Python object is either a proxy or a copy, which can use the existing converters. To resolve this, consider either:
having std::vector<>'s element type be the same as T's held type
explicitly registering a const T* to-Python converter