Category: Shyft Blog

Toledo Ohio Finance Franchise Investment Guide

franchise opportunities toledo ohio

Unlock Toledo’s finance franchise potential with stable business models, diverse revenue streams, and strategic advantages in this affordable Midwest market for financial entrepreneurs.

Toledo Energy Savings: Commercial LED Retrofit Incentive Guide A while ago, we discussed a small number of things that could make your C or C++ programs significantly faster. For those who are interested, here’s a more in-depth look at the nuts and bolts of modern CPUs from a programmer’s perspective, focusing on the aspects that we can take advantage of in our code to make it faster. The first consideration is cache. Cache is a limited amount of very fast memory built into the CPU. If data is in cache, it’s very fast to read; if it’s not in cache, it takes much longer as it must be loaded from main memory. Even memory is much faster than disk, which is why virtual memory is so slow when it’s swapping to disk. For a general overview of how memory caching works, see the Wikipedia article on CPU cache. But from a programming perspective, here are the most important takeaways: A cache miss occurs when the CPU is looking for data and can’t find it in the fastest level of cache. It’s then forced to go to the next level of cache, and if it’s not there either, it must go to RAM. The miss itself and the round trip to fetch data from RAM can stall the CPU for a long time. That’s why good cache behavior can have an extremely large impact on performance. Bad cache behavior can cause slowdowns of more than an order of magnitude. One thing we can do to avoid cache misses is ensure good spatial locality. Memory is loaded into cache in 64-byte chunks called cache lines. If your CPU reads one byte, the next 63 bytes will also be loaded into cache in anticipation that they’ll also be needed soon, since that’s often the case. So an array of bytes, int8_t[64], fits in exactly one cache line. An array of int32_t[16] also fits in one cache line. More on how to use this later. Another thing we can do is ensure good temporal locality. Once data is loaded into cache, it stays there until the CPU needs the space for something else. When that happens, it evicts the least recently used data. This means data that is frequently accessed is likely to remain in cache, since it was recently used. If your code accesses a wide range of memory addresses one after the other, other data will be evicted from cache, and it’ll have to be reloaded from RAM the next time it’s needed. More on this later, too. A cache line for a given memory address might be mapped to one specific location in cache, called direct mapping; be mappable to a small set of specific locations, called set associative; or be mappable to any location, called fully associative. Modern Intel CPUs generally have small L1 caches (closest to the processor, fastest) that are 8-way set associative (a given cache line can go in 8 different locations), larger L2 caches that are 4-way or 8-way set associative, and large L3 caches that are 8-way or 16-way set associative and shared among multiple cores. Most modern caches have a least recently used (LRU) eviction policy, which means that it preferentially evicts cache lines that haven’t been accessed in a while. Now, about spatial locality: iterating through an array of integers, *i.e.* int32_t arr[1000], will automatically have good spatial locality because the CPU will load a cache line full of integers at a time. If we’re doing a simple read of the array, each cache line will provide 16 integers, since it’s 64 bytes, or 16 * 4 bytes. But for the following code: “`c int32_t sum = 0; for (int32_t i = 0; i < 1000; i += 16) { sum += arr[i]; } ``` the performance is actually the same as iterating through each element of the array, because the CPU is still loading a full cache line at a time. That's because cache loads and evictions operate on entire cache lines; individual elements of a cache line can't be loaded or evicted separately. But what if your array isn't of a simple type, but rather a struct or class? For example, if your array is of the struct ```c struct Person { char name[64]; int32_t age; }; ``` and you want to compute the average age: ```c int32_t age_sum = 0; for (int32_t i = 0; i < num_people; i++) { age_sum += people[i].age; } double avg_age = (double)age_sum / num_people; ``` Now you're paying a steep cost. The problem is that each Person is 68 bytes, so you're having to load more than an entire cache line (or most of one) just to get the 4 bytes containing the person's age, and then you're just throwing the rest of it away. So we only get about one value per cache line, and to add insult to injury, most of what's loaded in the cache isn't even used. This is a classic example of poor spatial locality. If the struct is used in many different parts of the code, it might be hard to change. But the data can often be rearranged to be more efficient: ```c struct People { char name[num_people][64]; int32_t age[num_people]; } ``` Or if we can't reorganize the struct, we could extract the relevant data: ```c int32_t ages[num_people]; for (int32_t i = 0; i < num_people; i++) { ages[i] = people[i].age; } ``` Then computing the average is the same as before, except now our accesses have great spatial locality because we're accessing a contiguous array of int32_t. This is called an array of structs vs. struct of arrays issue. C++'s standard library, for example, uses an array of structs for std::vector and an array of pointers to structs for std::list. This is part of why vectors are more performant. Temporal locality is simpler: if you use the same data multiple times, it's best to do the operations close together to take advantage of the data already being in cache, rather than interleaving it with a lot of operations on other data that could cause the original data to be evicted. For example, it's much better to do an entire first pass through an array and then do an entire second pass, rather than to do part of the first pass, then part of the second pass, then go back to the first pass, etc. Prefetching is a CPU feature that allows it to load data from memory into cache before it's actually needed. Modern CPUs have several types of prefetching, some of which we can take advantage of and others that are internal to the CPU. One of the most basic forms of prefetching is next-line prefetching, which loads the next cache line into cache when the current one is accessed. This works great for linear traversals of arrays, but not as well for non-linear access patterns. Another form of prefetching is stride-based prefetching, where the CPU detects that the program is accessing memory in a pattern and prefetches data according to that pattern. For example, if a program is accessing every 10th element of an array, the CPU will detect this pattern and prefetch elements that are multiples of 10 ahead of the current position. Software prefetching is a technique where we explicitly tell the CPU to prefetch data with specific instructions. In C++, this can be done using intrinsics: ```cpp #include <immintrin.h> // Prefetch the cache line containing address `addr` to the L1 cache _mm_prefetch((const char*)addr, _MM_HINT_T0); // Prefetch the cache line containing address `addr` to the L2 cache _mm_prefetch((const char*)addr, _MM_HINT_T1); // Prefetch the cache line containing address `addr` to the L1 and L2 caches // and mark it for early eviction _mm_prefetch((const char*)addr, _MM_HINT_T2); // Prefetch the cache line containing address `addr` to the L1, L2, and L3 caches // and mark it for late eviction _mm_prefetch((const char*)addr, _MM_HINT_NTA); “` But in most cases, the automatic hardware prefetcher in modern CPUs is smart enough to detect access patterns. Software prefetching is only needed in special cases where the access pattern is known to the programmer but might not be predictable by the CPU. For example, if you’re using a linked list or a tree and you know which nodes will be accessed in advance (e.g., you’ve calculated a path), you can prefetch those nodes. A third optimization concerns branch prediction. Modern CPUs execute many instructions in parallel, in a pipeline. The simplest form of pipelining is called instruction-level parallelism (ILP). For an extreme example, if we have an array of integers and we want to add 1 to each element: “`c for (int32_t i = 0; i < n; i++) { arr[i] += 1; } ``` The additions are all independent, and the processor can execute many of them at once, limited by how many execution units it has. But what happens when we have a conditional? ```c for (int32_t i = 0; i < n; i++) { if (arr[i] > threshold) { arr[i] += 1; } } “` Now the processor needs to evaluate the condition before it knows whether to execute the addition. The condition results in a branch (a decision to execute one set of instructions or another), and the CPU can’t know for sure which way it will go until the condition is evaluated. To keep the pipeline full, it guesses, or “predicts,” which way the branch will go and executes those instructions speculatively. If the prediction is correct, execution continues normally; if it’s wrong, the CPU has to throw away all of the speculative work and go back to the branch, which is a costly operation called a branch misprediction. Modern CPUs have sophisticated branch predictors that learn from the program’s behavior. The gist of what we need to know as programmers is that if a branch goes the same way most of the time (e.g., the condition is almost always true or almost always false), the predictor will pick up on that pattern, and the branch will be well-predicted. But if the branch direction is unpredictable, e.g., it’s true for 50% of the cases in a seemingly random pattern, the predictor will mispredict more frequently, and the performance will suffer. If a branch is unpredictable, we can sometimes eliminate it by using branchless techniques like predication. For example, we can rewrite the above code as: “`c for (int32_t i = 0; i < n; i++) { arr[i] += (arr[i] > threshold); } “` In C, a comparison like arr[i] > threshold evaluates to 1 if true and 0 if false. So this adds 1 to arr[i] if the condition is true, and adds 0 (effectively doing nothing) if the condition is false. This eliminates the branch, and the CPU can execute the code without any branch mispredictions. SIMD (Single Instruction, Multiple Data) is a form of data-level parallelism where a single instruction operates on multiple data elements at once. Modern CPUs have SIMD instructions (SSE, AVX, AVX2, AVX-512, etc. on x86 architectures; NEON on ARM architectures) that can operate on 128, 256, or 512 bits of data at once. That means we can process multiple integers or floating-point values in a single instruction. For example, with 256-bit AVX2 registers, we can process 8 int32_t values at once. We can rewrite our earlier example as: “`cpp #include <immintrin.h> void add_one(int32_t* arr, int32_t n) { __m256i one = _mm256_set1_epi32(1); // Process 8 elements at a time for (int32_t i = 0; i < (n - n % 8); i += 8) { __m256i vec = _mm256_loadu_si256((__m256i*)&arr[i]); vec = _mm256_add_epi32(vec, one); _mm256_storeu_si256((__m256i*)&arr[i], vec); } // Handle remaining elements for (int32_t i = (n - n % 8); i < n; i++) { arr[i] += 1; } } ``` Now we're processing 8 elements at once, which can significantly improve performance, especially for large arrays. Auto-vectorization is a feature of modern compilers that automatically generates SIMD instructions from scalar code when possible. If our original simple loop was compiled with a modern compiler with optimization enabled, it might generate SIMD instructions similar to the hand-vectorized code we wrote. However, not all code is easy for compilers to auto-vectorize. Loops with complex control flow, irregular memory access patterns, or dependencies between iterations can be difficult or impossible to auto-vectorize. In these cases, we might need to write vectorized code by hand using intrinsics or assembly. For our example with the conditional, we can vectorize it as well, but it's more complex: ```cpp #include <immintrin.h> void conditional_add_one(int32_t* arr, int32_t n, int32_t threshold) { __m256i one = _mm256_set1_epi32(1); __m256i threshold_vec = _mm256_set1_epi32(threshold); // Process 8 elements at a time for (int32_t i = 0; i < (n - n % 8); i += 8) { __m256i vec = _mm256_loadu_si256((__m256i*)&arr[i]); // Compare each element with threshold __m256i mask = _mm256_cmpgt_epi32(vec, threshold_vec); // Add 1 only to elements that are greater than threshold __m256i to_add = _mm256_and_si256(one, mask); vec = _mm256_add_epi32(vec, to_add); _mm256_storeu_si256((__m256i*)&arr[i], vec); } // Handle remaining elements for (int32_t i = (n - n % 8); i < n; i++) { if (arr[i] > threshold) { arr[i] += 1; } } } “` This is complex, and a good optimizing compiler would do it for us, but it illustrates another form of branchless programming using SIMD. In addition to regular SIMD operations, modern CPUs also have fused operations that can perform multiple operations in a single instruction. For example, FMA (Fused Multiply-Add) instructions can perform a multiplication and an addition in a single instruction, with only one rounding operation. The latency of an instruction is the time it takes for the result to be available after the instruction starts. The throughput is the rate at which instructions can be issued. For example, an instruction with a latency of 4 cycles and a throughput of 1 instruction per cycle means that it takes 4 cycles for the result to be available, but a new instruction can be started every cycle. In the case of FMA, the operation a = b *

commercial led lighting retrofit incentive programs toledo ohio

Unlock substantial rebates and energy savings with Toledo’s LED lighting retrofit programs, cutting operational costs while meeting sustainability goals through utility incentives and financing options.