SoA implementation for float and AVX2 includind the polyexp fix
This commit is contained in:
Linus Baumberger
committed by
[Quy Anh] «Elliot» Nguyen
[Quy Anh] «Elliot» Nguyen
parent
1d585ffba7
commit
f7bea275b3
@@ -0,0 +1,74 @@
|
||||
# Optimization: SoA (float version)
|
||||
|
||||
- **Author**: Linus
|
||||
- **Predecessor**: variant_float_inlining.cpp
|
||||
- **File(s)**: variants/variant_float_inlining_SoA.cpp
|
||||
- **Status**: TBD
|
||||
|
||||
## What this changes
|
||||
|
||||
Same SoA layout switch as the double `variant_inlining_7_SoA`, but starting
|
||||
from the float baseline instead of the double one. `Pedestrian` struct is
|
||||
replaced by file-scope `vector<float>` arrays (`pos_x`, `pos_y`, `sb_e_x`,
|
||||
`sb_e_y`, `sb_sq`, `e_x`, `e_y`, `w_x`, `w_y`, `curr_vel_x`, `curr_vel_y`,
|
||||
`desired_v0`, `v_max`, `destination_x`, `destination_y`, `color`). Board
|
||||
obstacles stay AoS.
|
||||
|
||||
Mostly a setup step for the float AVX2 variant. Same reasoning as the
|
||||
double SoA: gets the j-side loads contiguous so the upcoming 8-wide AVX2
|
||||
loop has clean aligned chunks.
|
||||
|
||||
The j loop also keeps the `r_ab_norm > EPS ? : EPS` and `two_b_sq > EPS_SQ
|
||||
? : EPS_SQ` clamps from the float baseline. Float precision around close
|
||||
encounters can collapse those to zero where double would not, and the clamp
|
||||
keeps the division finite.
|
||||
|
||||
## Code (relevant snippet)
|
||||
|
||||
|
||||
See `variants/variant_float_inlining_SoA.cpp`.
|
||||
|
||||
## Benchmark
|
||||
|
||||
Conditions: min of N runs (TBD), N_ITER=1000, -O3 -march=native -flto.
|
||||
Diff is vs `variant_float_inlining`.
|
||||
|
||||
### x86
|
||||
|
||||
**Machine**: TBD
|
||||
|
||||
**Frequency**: TBD
|
||||
|
||||
| N | W(n) [Flops] | T(n) [Cycles] | Performance [Flops/Cycle] | Diff vs Predecessor |
|
||||
| ---- | ------------ | ------------- | ------------------------- | ------------------- |
|
||||
| 50 | | | | |
|
||||
| 100 | | | | |
|
||||
| 200 | | | | |
|
||||
| 500 | | | | |
|
||||
|
||||
### ARM (Apple)
|
||||
|
||||
**Machine**: TBD
|
||||
|
||||
**Frequency**: TBD
|
||||
|
||||
| N | W(n) [Flops] | T(n) [Cycles] | Performance [Flops/Cycle] | Diff vs Predecessor |
|
||||
| ---- | ------------ | ------------- | ------------------------- | ------------------- |
|
||||
| 50 | | | | |
|
||||
| 100 | | | | |
|
||||
| 200 | | | | |
|
||||
| 500 | | | | |
|
||||
|
||||
## Analysis
|
||||
|
||||
Layout change only, so the expectation is roughly no win on its own. The
|
||||
float scalar loop already auto-vectorises in pieces with `-O3
|
||||
-march=native`, so SoA doesnt buy much over what the AoS version was
|
||||
already doing. The real payoff lands in the AVX2 child.
|
||||
|
||||
Verify passes against `baseline_float_n60_i500.csv` (from
|
||||
`variant_float_baseline`) at the 1cm tolerance.
|
||||
|
||||
## Takeaway
|
||||
|
||||
Setup step for the AVX2 variant.
|
||||
@@ -0,0 +1,98 @@
|
||||
# Optimization: AVX2 vectorization of the ped-ped loop (float, 8-wide)
|
||||
|
||||
- **Author**: Linus
|
||||
- **Predecessor**: variant_float_inlining_SoA.cpp
|
||||
- **File(s)**: variants/variant_float_inlining_SoA_AVX2.cpp
|
||||
- **Status**: TBD
|
||||
|
||||
## What this changes
|
||||
|
||||
Float port of the double `variant_inlining_7_SoA_AVX2`. Inner ped-ped j
|
||||
loop runs on `__m256` (8 floats) instead of the double version's `__m256d`
|
||||
(4 doubles). 8 ped-ped interactions per j step. SoA layout from the
|
||||
predecessor makes the j-side loads contiguous.
|
||||
|
||||
Same masking trick as the double variant: build an `active` mask from
|
||||
`j_idx != i` AND `j_idx < n_p`, blend a safe `1.0` into the denominator
|
||||
where `active` is zero so the i==j and padding lanes don't NaN out, then
|
||||
`vandps` the per-lane `f_x`/`f_y` against `active` before accumulating
|
||||
into `acc_x`/`acc_y`.
|
||||
|
||||
n_p is padded to a multiple of 8 (`n_p_pad = (n_p + 7) & ~7`) so the j
|
||||
loop has no scalar tail. Padding positions are set to `1e9f` and `sb_e`,
|
||||
`sb_sq` to `0` so any unmasked contribution from those lanes would still
|
||||
be ~zero through `expf(very negative) ≈ 0`.
|
||||
|
||||
EPS clamps from the float scalar predecessor are kept and lifted to SIMD
|
||||
form (`_mm256_max_ps(r_ab_n, v_eps)` etc.). Same correctness reason as
|
||||
the scalar version: float precision around close encounters can collapse
|
||||
norms to zero.
|
||||
|
||||
The `expf` calls are still scalar inside the j loop: 8 of them per j
|
||||
step, manually unrolled into a scratch `tmp_arg[8]`/`tmp_val[8]` array.
|
||||
gcc refused to unroll the loop on its own (it serialised all 8 calls on
|
||||
`xmm0`), the manual unroll at least lets them issue independently. Kill
|
||||
these next with the polyexp variant.
|
||||
|
||||
## Code (relevant snippet)
|
||||
|
||||
```cpp
|
||||
for (int j = 0; j < n_p_pad; j += 8) {
|
||||
__m256 j_idx = _mm256_add_ps(_mm256_set1_ps((float)j), lane_off);
|
||||
__m256 active = _mm256_and_ps(
|
||||
_mm256_cmp_ps(j_idx, v_i, _CMP_NEQ_OQ),
|
||||
_mm256_cmp_ps(j_idx, v_np, _CMP_LT_OQ));
|
||||
|
||||
__m256 pjx = _mm256_loadu_ps(&pos_x[j]);
|
||||
__m256 pjy = _mm256_loadu_ps(&pos_y[j]);
|
||||
// ... sb_e_x, sb_e_y, sb_sq
|
||||
|
||||
// body: r_ab, r_ab2, two_b, scalar expf x 8, smc, sm_r_ab, sm_r_ab2,
|
||||
// f_x, f_y, w_val branch, AND with active, fmadd into acc_x/acc_y.
|
||||
}
|
||||
```
|
||||
|
||||
See `variants/variant_float_inlining_SoA_AVX2.cpp` for the full inner
|
||||
loop.
|
||||
|
||||
## Benchmark
|
||||
|
||||
Conditions: min of N runs (TBD), N_ITER=1000, -O3 -march=native -flto.
|
||||
Diff is vs `variant_float_inlining_SoA`.
|
||||
|
||||
### x86
|
||||
|
||||
**Machine**: TBD
|
||||
|
||||
**Frequency**: TBD
|
||||
|
||||
| N | W(n) [Flops] | T(n) [Cycles] | Performance [Flops/Cycle] | Diff vs Predecessor |
|
||||
| ---- | ------------ | ------------- | ------------------------- | ------------------- |
|
||||
| 50 | | | | |
|
||||
| 100 | | | | |
|
||||
| 200 | | | | |
|
||||
| 500 | | | | |
|
||||
|
||||
### ARM (Apple)
|
||||
|
||||
Not applicable, this variant uses AVX2 intrinsics.
|
||||
|
||||
## Analysis
|
||||
|
||||
Same shape of win as the double AVX2 variant, just at 8-wide. Scalar
|
||||
`expf` is still the dominant cost, polyexp kills it next.
|
||||
|
||||
One catch: this variant only hits its real performance if FTZ is
|
||||
enabled in `main-float.cpp` (that gets added on the polyexp MR). Without
|
||||
it the active-lane chain produces subnormal intermediates (the `smc =
|
||||
numer/denom` step is the one that lands there for typical mid-range pair
|
||||
distances) and every following FMA stalls 100+ cycles in microcode. With
|
||||
FTZ on it runs as expected, without it benchmarks look 5-10x slow.
|
||||
|
||||
Verify passes against `baseline_float_n60_i500.csv` at 1cm tolerance.
|
||||
Local max drift ~3mm.
|
||||
|
||||
## Takeaway
|
||||
|
||||
Polyexp is the next step. Note:
|
||||
benchmarks are only meaningful with FTZ/DAZ on in main-float.cpp.
|
||||
@@ -0,0 +1,75 @@
|
||||
# Optimization: SIMD polynomial exp (float, 8-wide) + FTZ/DAZ
|
||||
|
||||
- **Author**: Linus
|
||||
- **Predecessor**: variant_float_inlining_SoA_AVX2.cpp
|
||||
- **File(s)**: variants/variant_float_inlining_SoA_AVX2_polyexp.cpp, main-float.cpp (FTZ/DAZ enable)
|
||||
- **Status**: TBD, positive locally (approx twice as fast)
|
||||
|
||||
## What this changes
|
||||
|
||||
The 8 scalar `expf` calls in the j-loop become a single SIMD polynomial
|
||||
approximation `exp256_ps` that operates on `__m256`. Range-reduce
|
||||
`x = n*ln2 + r` with `n = round(x/ln2)`, evaluate a degree-5 Horner
|
||||
polynomial for `exp(r)` on `r in [-ln2/2, ln2/2]`, then scale by `2^n`
|
||||
via the Schraudolph bit-trick on the IEEE 754 exponent slot (shift the
|
||||
biased exponent by 23). Degree-5 is enough for float, the polynomial is
|
||||
good to ~1e-5 absolute on the reduced range, well under the 1cm verify
|
||||
tolerance.
|
||||
|
||||
|
||||
`main-float.cpp` also gets two MXCSR writes at the top of `main()`:
|
||||
|
||||
_MM_SET_FLUSH_ZERO_MODE(_MM_FLUSH_ZERO_ON);
|
||||
_MM_SET_DENORMALS_ZERO_MODE(_MM_DENORMALS_ZERO_ON);
|
||||
|
||||
This was the part that actually mattered, see Analysis.
|
||||
|
||||
## Code (relevant snippet)
|
||||
|
||||
|
||||
See `variants/variant_float_inlining_SoA_AVX2_polyexp.cpp` for the full
|
||||
j-loop and `main-float.cpp` for the FTZ/DAZ setup.
|
||||
|
||||
## Benchmark
|
||||
|
||||
Conditions: min of N runs (TBD), N_ITER=1000, -O3 -march=native -flto.
|
||||
Diff is vs `variant_float_inlining_SoA_AVX2`.
|
||||
|
||||
### x86
|
||||
|
||||
**Machine**: TBD
|
||||
|
||||
**Frequency**: TBD
|
||||
|
||||
| N | W(n) [Flops] | T(n) [Cycles] | Performance [Flops/Cycle] | Diff vs Predecessor |
|
||||
| ---- | ------------ | ------------- | ------------------------- | ------------------- |
|
||||
| 50 | | | | |
|
||||
| 100 | | | | |
|
||||
| 200 | | | | |
|
||||
| 500 | | | | |
|
||||
|
||||
### ARM (Apple)
|
||||
|
||||
Not applicable, AVX2-only. A NEON port would need its own vectorised
|
||||
exp.
|
||||
|
||||
## Analysis
|
||||
|
||||
roughly twice as fast
|
||||
|
||||
|
||||
## Takeaway
|
||||
|
||||
Polyexp helps as expected. The real fix is FTZ/DAZ in `main-float.cpp`,
|
||||
without it everything downstream in the float chain is silently
|
||||
subnormal-stalled
|
||||
|
||||
## References
|
||||
|
||||
- Schraudolph, N. N. (1999). "A Fast, Compact Approximation of the
|
||||
Exponential Function." Neural Computation 11(4), 853-862.
|
||||
- IEEE 754 double-precision format. Wikipedia.
|
||||
https://en.wikipedia.org/wiki/Double-precision_floating-point_format
|
||||
- Horner's method. Wikipedia.
|
||||
https://en.wikipedia.org/wiki/Horner%27s_method
|
||||
|
||||
Reference in New Issue
Block a user