C++ Move vs Copy a local stl container to a list inside a loop - copy

In the lower fictitious code example (C++17) I ask myself whether I should optimize here with std::move or better not. The last emplace_back seems to be a suitable candidate, since "idsLocal" is certainly no longer needed afterwards. With the first emplace_back I wonder if that would make sense here. If I move "idsLocal" there and call the clear() afterwards, I avoid the copy, but I am feared that "idsLocal" would potentially have to reallocate multiple times again in the next iterations (memory is moved?!), which it probably wouldn't have done otherwise because of the reserve-call. So would I have an advantage here (and why) if I use std::move everywhere in the emplace_back-calls? My time measurements were very fluctuating and did not really give a clear result for me.
For example, let's take the following input data for divideIntoIdsList:
=> "limit" is 1024 and we have 5120 different id's
Naively thought, the following could come out:
Without move in the FIRST emplace_back:
-> 1 malloc(s) for the reserve at idsLocal
-> 5 malloc(s) for copying idsLocal 5 times in idsList
With move in the FIRST emplace_back:
-> 1 malloc(s) for the reserve at idsLocal
-> 0 malloc(s) for moving idsLocal 5 times in idsList (?)
-> 40 malloc(s) for dynamic resizes in idsLocal beginning
after the first move (4 x 10 push_back's per limit) (?)
Assuming that allocations are expensive, would the
move variant be even slower?
Code-Example:
#include <list>
#include <numeric>
#include <vector>
template < class T >
auto divideIntoIdsList( size_t limit, const T &ids, std::list< T > &idsList )
{
if ( ids.empty() )
return;
T idsLocal;
idsLocal.reserve( limit );
size_t iIdCount=0;
for( typename T::const_iterator itIds = ids.begin();
itIds != ids.end(); ++itIds )
{
idsLocal.push_back( *itIds );
if( ++iIdCount == limit )
{
idsList.emplace_back( idsLocal ); //std::move(idsLocal) here?!
idsLocal.clear();
iIdCount=0;
}
}
if( iIdCount > 0 )
idsList.emplace_back( idsLocal ); //std::move(idsLocal) here seems useful
}
int main()
{
using ids_t = std::vector< size_t >;
ids_t ids( 5120 );
std::iota( std::begin( ids ), std::end( ids ), 0 );
std::list< ids_t > idslist;
divideIntoIdsList( 1024, ids, idslist );
return 0;
}

Related

Unaligned access performance on Intel x86 vs AMD x86 CPUs

I have implemented a simple linear probing hash map with an array of structs memory layout. The struct holds the key, the value, and a flag indicating whether the entry is valid. By default, this struct gets padded by the compiler, as key and value are 64-bit integers, but the entry only takes up 8 bools. Hence, I have also tried packing the struct at the cost of unaligned access. I was hoping to get better performance from the packed/unaligned version due to higher memory density (we do not waste bandwidth on transferring padding bytes).
When benchmarking this hash map on an Intel Xeon Gold 5220S CPU (single-threaded, gcc 11.2, -O3 and -march=native), I see no performance difference between the padded version and the unaligned version. However, on an AMD EPYC 7742 CPU (same setup), I find a performance difference between unaligned and padded. Here is a graph depicting the results for hash map load factors 25 % and 50 %, for different successful query rates on the x axis (0,25,50,75,100): As you can see, on Intel, the grey and blue (circle and square) lines almost overlap, the benefit of struct packing is marginal. On AMD, however, the line representing unaligned/packed structs is consistently higher, i.e., we have more throughput.
In order to investigate this, I tried to built a smaller microbenchmark. In this microbenchmark, we perform a similar benchmark, but without the hash map find logic (i.e., we just pick random indices in the array and advance a little there). Please find the benchmark here:
#include <atomic>
#include <chrono>
#include <cstdint>
#include <iostream>
#include <random>
#include <vector>
void ClobberMemory() { std::atomic_signal_fence(std::memory_order_acq_rel); }
template <typename T>
void doNotOptimize(T const& val) {
asm volatile("" : : "r,m"(val) : "memory");
}
struct PaddedStruct {
uint64_t key;
uint64_t value;
bool is_valid;
PaddedStruct() { reset(); }
void reset() {
key = uint64_t{};
value = uint64_t{};
is_valid = 0;
}
};
struct PackedStruct {
uint64_t key;
uint64_t value;
uint8_t is_valid;
PackedStruct() { reset(); }
void reset() {
key = uint64_t{};
value = uint64_t{};
is_valid = 0;
}
} __attribute__((__packed__));
int main() {
const uint64_t size = 134217728;
uint16_t repetitions = 0;
uint16_t advancement = 0;
std::cin >> repetitions;
std::cout << "Got " << repetitions << std::endl;
std::cin >> advancement;
std::cout << "Got " << advancement << std::endl;
std::cout << "Initializing." << std::endl;
std::vector<PaddedStruct> padded(size);
std::vector<PackedStruct> unaligned(size);
std::vector<uint64_t> queries(size);
// Initialize the structs with random values + prefault
std::random_device rd;
std::mt19937 gen{rd()};
std::uniform_int_distribution<uint64_t> dist{0, 0xDEADBEEF};
std::uniform_int_distribution<uint64_t> dist2{0, size - advancement - 1};
for (uint64_t i = 0; i < padded.size(); ++i) {
padded[i].key = dist(gen);
padded[i].value = dist(gen);
padded[i].is_valid = 1;
}
for (uint64_t i = 0; i < unaligned.size(); ++i) {
unaligned[i].key = padded[i].key;
unaligned[i].value = padded[i].value;
unaligned[i].is_valid = 1;
}
for (uint64_t i = 0; i < unaligned.size(); ++i) {
queries[i] = dist2(gen);
}
std::cout << "Running benchmark." << std::endl;
ClobberMemory();
auto start_padded = std::chrono::high_resolution_clock::now();
PaddedStruct* padded_ptr = nullptr;
uint64_t sum = 0;
for (uint16_t j = 0; j < repetitions; j++) {
for (const uint64_t& query : queries) {
for (uint16_t i = 0; i < advancement; i++) {
padded_ptr = &padded[query + i];
if (padded_ptr->is_valid) [[likely]] {
sum += padded_ptr->value;
}
}
doNotOptimize(sum);
}
}
ClobberMemory();
auto end_padded = std::chrono::high_resolution_clock::now();
uint64_t padded_runtime = static_cast<uint64_t>(std::chrono::duration_cast<std::chrono::milliseconds>(end_padded - start_padded).count());
std::cout << "Padded Runtime (ms): " << padded_runtime << " (sum = " << sum << ")" << std::endl; // print sum to avoid that it gets optimized out
ClobberMemory();
auto start_unaligned = std::chrono::high_resolution_clock::now();
uint64_t sum2 = 0;
PackedStruct* packed_ptr = nullptr;
for (uint16_t j = 0; j < repetitions; j++) {
for (const uint64_t& query : queries) {
for (uint16_t i = 0; i < advancement; i++) {
packed_ptr = &unaligned[query + i];
if (packed_ptr->is_valid) [[likely]] {
sum2 += packed_ptr->value;
}
}
doNotOptimize(sum2);
}
}
ClobberMemory();
auto end_unaligned = std::chrono::high_resolution_clock::now();
uint64_t unaligned_runtime = static_cast<uint64_t>(std::chrono::duration_cast<std::chrono::milliseconds>(end_unaligned - start_unaligned).count());
std::cout << "Unaligned Runtime (ms): " << unaligned_runtime << " (sum = " << sum2 << ")" << std::endl;
}
When running the benchmark, I pick repetitions = 3 and advancement = 5, i.e., after compiling and running it, you have to enter 3 (and press newline) and then enter 5 and press enter/newline. I updated the source code to (a) avoid loop unrolling by the compiler because repetition/advancement were hardcoded and (b) switch to pointers into that vector as it more closely resembles what the hash map is doing.
On the Intel CPU, I get:
Padded Runtime (ms): 13204
Unaligned Runtime (ms): 12185
On the AMD CPU, I get:
Padded Runtime (ms): 28432
Unaligned Runtime (ms): 22926
So while in this microbenchmark, Intel still benefits a little from the unaligned access, for the AMD CPU, both the absolute and relative improvement is higher. I cannot explain this. In general, from what I've learned from relevant SO threads, unaligned access for a single member is just as expensive as aligned access, as long as it stays within a single cache line (1). Also in (1), a reference to (2) is given, which claims that the cache fetch width can differ from the cache line size. However, except for Linus Torvalds mail, I could not find any other documentation of cache fetch widths in processors and especially not for my concrete two CPUs to figure out if that might somehow have to do with this.
Does anybody have an idea why the AMD CPU benefits much more from the struct packing? If it is about reduced memory bandwidth consumption, I should be able to see the effects on both CPUs. And if the bandwidth usage is similar, I do not understand what might be causing the differences here.
Thank you so much.
(1) Relevant SO thread: How can I accurately benchmark unaligned access speed on x86_64?
(2) https://www.realworldtech.com/forum/?threadid=168200&curpostid=168779
The L1 Data Cache fetch width on the Intel Xeon Gold 5220S (and all the other Skylake/CascadeLake Xeon processors) is up to 64 naturally-aligned Bytes per cycle per load.
The core can execute two loads per cycle for any combination of size and alignment that does not cross a cacheline boundary. I have not tested all the combinations on the SKX/CLX processors, but on Haswell/Broadwell, throughput was reduced to one load per cycle whenever a load crossed a cacheline boundary, and I would assume that SKX/CLX are similar. This can be viewed as necessary feature rather than a "penalty" -- a line-splitting load might need to use both ports to load a pair of adjacent lines, then combine the requested portions of the lines into a payload for the target register.
Loads that cross page boundaries have a larger performance penalty, but to measure it you have to be very careful to understand and control the locations of the page table entries for the two pages: DTLB, STLB, in the caches, or in main memory. My recollection is that the most common case is pretty fast -- partly because the "Next Page Prefetcher" is pretty good at pre-loading the PTE entry for the next page into the TLB before a sequence of loads gets to the end of the first page. The only case that is painfully slow is for stores that straddle a page boundary, and the Intel compiler works very hard to avoid this case.
I have not looked at the sample code in detail, but if I were performing this analysis, I would be careful to pin the processor frequency, measure the instruction and cycle counts, and compute the average number of instructions and cycles per update. (I usually set the core frequency to the nominal (TSC) frequency just to make the numbers easier to work with.) For the naturally-aligned cases, it should be pretty easy to look at the assembly code and estimate what the cycle counts should be. If the measurements are similar to observations for that case, then you can begin looking at the overhead of unaligned accesses in reference to a more reliable understanding of the baseline.
Hardware performance counters can be valuable for this case as well, particularly the DTLB_LOAD_MISSES events and the L1D.REPLACEMENT event. It only takes a few high-latency TLB miss or L1D miss events to skew the averages.
The number of cache-line accesses when using 24-byte data structures may be the same as when using 17-byte data structure.
Please see this blog post: https://lemire.me/blog/2022/06/06/data-structure-size-and-cache-line-accesses/

stm32f429, spi dr register not write data

code_1
code_2
register on debug
logic analyzer
void SPI_SendData(SPI_RegDef_t *pSPIx,uint8_t *pTxBuffer , uint32_t Len)
{
while(Len > 0)
{
// 1 . wait until TXE is set ,
while(SPI_GetFlagStatus(pSPIx, SPI_TXE_FLAG) == FLAG_RESET);
// 2. check the DFF bit in CR1
if( (pSPIx->CR1 & (1 << SPI_CR1_DFF) ) )
{
// 16 BIT DFF
pSPIx->DR = *((uint16_t*)pTxBuffer); // dereferencing(typecasting );
Len--;
Len--;
(uint16_t*)pTxBuffer++; // typecasting this pointer to uint16 type and incrementing by 2.
/* The buffer is a uint8_t pointer type. When using the 16-bit data frame,
* we pick up 16-bits of data, increment pointer by 2 bytes,*/
}else
{
// 8 BIT DFF
pSPIx->DR = *pTxBuffer;
Len--;
pTxBuffer++;
/*
*(( uint8_t*)&hspi->Instance->DR) = (*pData);
pData += sizeof(uint8_t);
hspi->TxXferCount--;
*/
}
}
}
i see, MOSI always send 255 on logic analyzer but wrong data.
(uint16_t*)pTxBuffer++; increments the pointer by 1 byte, not two that you say you hope it will in the comment.
If you want to do it by converting to halfword pointer and incrementing, then you need to do something like:
pTxBuffer = (uint8_t*)(((uint16_t*)pTxBuffer) + 1);
But that is just a silly way of saying pTxBuffer += 2;
Really it doesn't make sense to have the if inside the loop, because the value of the DFF bit doesn't change unless you write to it, and you don't. I suggest you write one loop over words and one loop over bytes and have the if at the top level.

Promela system with unranged values

int rq_begin = 0, rq_end = 0;
int av_begin = 0, av_end = 0;
#define MAX_DUR 10
#define RQ_DUR 5
proctype Writer() {
do
:: (av_end < rq_end) -> av_end++;
if
:: (av_end - av_begin) > MAX_DUR -> av_begin = av_end - MAX_DUR;
:: else -> skip;
fi
printf("available span: [%d,%d]\n", av_begin, av_end);
od
}
proctype Reader() {
do
:: d_step {
rq_begin++;
rq_end = rq_begin + RQ_DUR;
}
printf("requested span: [%d,%d]\n", rq_begin, rq_end);
(rq_begin >= av_begin && rq_end <= av_end);
printf("got requested span\n");
od
}
init {
run Writer();
run Reader();
}
This system (only an example) should model a reader/writer queue where the reader requests a certain span of frames [rq_begin,rq_end], and the writer should then make at least this span available. [av_begin,av_end] is the span of available frames.
The 4 values are absolute frame indices, rq_begin gets incremented infinitley as the reader reads the next span of frames.
The system cannot be directly verified because the values are unranges (generating infinitely many states). Does Promela/Spin (or a similar software) has support to verify a system like this, and automatically transform it such that it becomes finite?
For example if all the 4 values were incremented by the same amount, the situation would still be the same. Or it could be reformulated into a model which instead has variables for the differences of these values, for example av_end - rq_end.
I'm using Promela/Spin to verify a more complex queuing system which uses absolute frame indices like this.

Expression result unused

I got some codes and I'm trying to fix some compiling bugs:
StkFrames& PRCRev :: tick( StkFrames& frames, unsigned int channel )
{
#if defined(_STK_DEBUG_)
if ( channel >= frames.channels() - 1 ) {
errorString_ << "PRCRev::tick(): channel and StkFrames arguments are incompatible!";
handleError( StkError::FUNCTION_ARGUMENT );
}
#endif
StkFloat *samples = &frames[channel];
unsigned int hop = frames.channels();
for ( unsigned int i=0; i<frames.frames(); i++, samples += hop ) {
*samples = tick( *samples );
*samples++; <<<<<<<<<--------- Expression result unused.
*samples = lastFrame_[1];
}
return frames;
}
I don't understand what the codes is trying to do. The codes are huge and I fixed quite a few. But googling didn't work for this.
Any ideas?
First, you do an increment (the line which actually gives you warning).
*samples++;
And then you assign to that variable something else, which makes previous action unused.
*samples = lastFrame_[1];
I recommend you to read this code inside 'for' loop more carefully. It doesn't look very logical.

Why is the call to array_view::synchronize() so slow?

i've started experimenting with C++ AMP. I've created a simple test app just to see what it can do, however the results are quite surprising to me. Consider the following code:
#include <amp.h>
#include "Timer.h"
using namespace concurrency;
int main( int argc, char* argv[] )
{
uint32_t u32Threads = 16;
uint32_t u32DataRank = u32Threads * 256;
uint32_t u32DataSize = (u32DataRank * u32DataRank) / u32Threads;
uint32_t* pu32Data = new (std::nothrow) uint32_t[ u32DataRank * u32DataRank ];
for ( uint32_t i = 0; i < u32DataRank * u32DataRank; i++ )
{
pu32Data[i] = 1;
}
uint32_t* pu32Sum = new (std::nothrow) uint32_t[ u32Threads ];
Timer tmr;
tmr.Start();
array< uint32_t, 1 > source( u32DataRank * u32DataRank, pu32Data );
array_view< uint32_t, 1 > sum( u32Threads, pu32Sum );
printf( "Array<> deep copy time: %.6f\n", tmr.Stop() );
tmr.Start();
parallel_for_each(
sum.extent,
[=, &source](index<1> idx) restrict(amp)
{
uint32_t u32Sum = 0;
uint32_t u32Start = idx[0] * u32DataSize;
uint32_t u32End = (idx[0] * u32DataSize) + u32DataSize;
for ( uint32_t i = u32Start; i < u32End; i++ )
{
u32Sum += source[i];
}
sum[idx] = u32Sum;
}
);
double dDuration = tmr.Stop();
printf( "gpu computation time: %.6f\n", dDuration );
tmr.Start();
sum.synchronize();
dDuration = tmr.Stop();
printf( "synchronize time: %.6f\n", dDuration );
printf( "first and second row sum = %u, %u\n", pu32Sum[0], pu32Sum[1] );
tmr.Start();
for ( uint32_t idx = 0; idx < u32Threads; idx++ )
{
uint32_t u32Sum = 0;
for ( uint32_t i = 0; i < u32DataSize; i++ )
{
u32Sum += pu32Data[(idx * u32DataSize) + i];
}
pu32Sum[idx] = u32Sum;
}
dDuration = tmr.Stop();
printf( "cpu computation time: %.6f\n", dDuration );
printf( "first and second row sum = %u, %u\n", pu32Sum[0], pu32Sum[1] );
delete [] pu32Sum;
delete [] pu32Data;
return 0;
}
Note that Timer is a simple timing class using QueryPerformanceCounter. Anyway, the output of the code is the following:
Array<> deep copy time: 0.089784
gpu computation time: 0.000449
synchronize time: 8.671081
first and second row sum = 1048576, 1048576
cpu computation time: 0.006647
first and second row sum = 1048576, 1048576
Why is the call to synchronize() taking so long? Is there a way how to get around this? Other than that the performance of the computation performance is amazing, however the synchronize() overhead makes it unusable for me.
It is also possible that i am doing something terribly wrong, if so, please tell me. Thanks in advance.
Function synchronize() is probably taking so long because it is waiting for the actual kernel to complete its work.
From parallel_for_each from amp.h:
Please note that the parallel_for_each executes as if synchronous to the calling code, but in reality, it is asynchronous. I.e. once the parallel_for_each call is made and the kernel has been passed to the runtime, the [code after the parallel_for_each] continues to execute immediately by the CPU thread, while in parallel the kernel is executed by the GPU threads.
So, measuring the time spent in parallel_for_each is not particularly meaningful.
EDIT: The way the algorithm is written, it won't benefit much from GPU acceleration. The read of source[i] is non-coalesced, and so it will be almost 16x slower than a coalesced read. It is possible to coalesce the read by using shared memory, but it is not quite trivial. I'd recommend reading up on GPU programming.
If you just want a simple example that demonstrates the utility of C++ AMP, try matrix multiplication.
Of course, the performance you'll observe also greatly depends on the model of you GPU hardware.
In addition to Igor's response on your specific algorithm, please note that there are multiple incorrect aspects of the way you are measuring C++ AMP performance in general (no runtime initialization exclusion, no discarding of initial JIT, no warmup of data, and the already pointed out assumption of p_f_e being synchronous), so please follow our guidelines here:
http://blogs.msdn.com/b/nativeconcurrency/archive/2011/12/28/how-to-measure-the-performance-of-c-amp-algorithms.aspx