admin/263-0007-00L-Advanced-Systems-Lab
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

SoA implementation for float and AVX2 includind the polyexp fix

Browse Source
This commit is contained in:
Linus Baumberger
2026-05-25 12:07:46 +02:00
committed by [Quy Anh] «Elliot» Nguyen
parent 1d585ffba7
commit f7bea275b3
9 changed files with 31574 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Makefile
+10 -3
View File
@@ -108,13 +108,20 @@ BENCH_ITER ?= 1000
# Verification parameters # Verification parameters
ifneq ($(filter float_%,$(VARIANT)),) ifneq ($(filter float_%,$(VARIANT)),)
REFERENCE_CSV ?= reference/baseline_float_n60_i4000.csv # Float verify uses a short horizon (500 iters). The simulation is chaotic - a 1-ULP perturbation
# amplifies by ~exp(t/200) - so floats can't hold bit-equality across variants over 4000 iters
# no matter what we do (different layouts -> different FMA placements -> 1-ULP drift -> meters by t=4000).
# At 500 iters the amplifier is still small: AVX2 variants drift max ~4mm vs the scalar reference,
# well under 1cm, and real bugs still show as O(m) errors. 4000 iters drifts to meters even when
# the variant is correct, so that horizon is useless for verify.
REFERENCE_N_ITER ?= 500
REFERENCE_CSV ?= reference/baseline_float_n60_i500.csv
else else
REFERENCE_N_ITER ?= 4000
REFERENCE_CSV ?= reference/baseline_n60_i4000.csv REFERENCE_CSV ?= reference/baseline_n60_i4000.csv
endif endif
REFERENCE_N_P ?= 60 REFERENCE_N_P ?= 60
REFERENCE_N_ITER ?= 4000
VERIFY_CSV := verify_$(VARIANT).csv VERIFY_CSV := verify_$(VARIANT).csv
.PHONY: all main run bench run-bench bench-suite video asm test clean clean-all verify .PHONY: all main run bench run-bench bench-suite video asm test clean clean-all verify
@@ -194,4 +201,4 @@ clean:
# WARNING: this deletes all *.csv and *.mp4 files in this directory. # WARNING: this deletes all *.csv and *.mp4 files in this directory.
# Move any output files you want to keep to another directory before running it. # Move any output files you want to keep to another directory before running it.
clean-all: clean clean-all: clean
rm -f *.csv *.mp4 rm -f *.csv *.mp4
docs/optimization_float_inlining_SoA.md
+74
View File
@@ -0,0 +1,74 @@
# Optimization: SoA (float version)
- **Author**: Linus
- **Predecessor**: variant_float_inlining.cpp
- **File(s)**: variants/variant_float_inlining_SoA.cpp
- **Status**: TBD
## What this changes
Same SoA layout switch as the double `variant_inlining_7_SoA`, but starting
from the float baseline instead of the double one. `Pedestrian` struct is
replaced by file-scope `vector<float>` arrays (`pos_x`, `pos_y`, `sb_e_x`,
`sb_e_y`, `sb_sq`, `e_x`, `e_y`, `w_x`, `w_y`, `curr_vel_x`, `curr_vel_y`,
`desired_v0`, `v_max`, `destination_x`, `destination_y`, `color`). Board
obstacles stay AoS.
Mostly a setup step for the float AVX2 variant. Same reasoning as the
double SoA: gets the j-side loads contiguous so the upcoming 8-wide AVX2
loop has clean aligned chunks.
The j loop also keeps the `r_ab_norm > EPS ? : EPS` and `two_b_sq > EPS_SQ
? : EPS_SQ` clamps from the float baseline. Float precision around close
encounters can collapse those to zero where double would not, and the clamp
keeps the division finite.
## Code (relevant snippet)
See `variants/variant_float_inlining_SoA.cpp`.
## Benchmark
Conditions: min of N runs (TBD), N_ITER=1000, -O3 -march=native -flto.
Diff is vs `variant_float_inlining`.
### x86
**Machine**: TBD
**Frequency**: TBD
| N | W(n) [Flops] | T(n) [Cycles] | Performance [Flops/Cycle] | Diff vs Predecessor |
| ---- | ------------ | ------------- | ------------------------- | ------------------- |
| 50 | | | | |
| 100 | | | | |
| 200 | | | | |
| 500 | | | | |
### ARM (Apple)
**Machine**: TBD
**Frequency**: TBD
| N | W(n) [Flops] | T(n) [Cycles] | Performance [Flops/Cycle] | Diff vs Predecessor |
| ---- | ------------ | ------------- | ------------------------- | ------------------- |
| 50 | | | | |
| 100 | | | | |
| 200 | | | | |
| 500 | | | | |
## Analysis
Layout change only, so the expectation is roughly no win on its own. The
float scalar loop already auto-vectorises in pieces with `-O3
-march=native`, so SoA doesnt buy much over what the AoS version was
already doing. The real payoff lands in the AVX2 child.
Verify passes against `baseline_float_n60_i500.csv` (from
`variant_float_baseline`) at the 1cm tolerance.
## Takeaway
Setup step for the AVX2 variant.
docs/optimization_float_inlining_SoA_AVX2.md
+98
View File
@@ -0,0 +1,98 @@
# Optimization: AVX2 vectorization of the ped-ped loop (float, 8-wide)
- **Author**: Linus
- **Predecessor**: variant_float_inlining_SoA.cpp
- **File(s)**: variants/variant_float_inlining_SoA_AVX2.cpp
- **Status**: TBD
## What this changes
Float port of the double `variant_inlining_7_SoA_AVX2`. Inner ped-ped j
loop runs on `__m256` (8 floats) instead of the double version's `__m256d`
(4 doubles). 8 ped-ped interactions per j step. SoA layout from the
predecessor makes the j-side loads contiguous.
Same masking trick as the double variant: build an `active` mask from
`j_idx != i` AND `j_idx < n_p`, blend a safe `1.0` into the denominator
where `active` is zero so the i==j and padding lanes don't NaN out, then
`vandps` the per-lane `f_x`/`f_y` against `active` before accumulating
into `acc_x`/`acc_y`.
n_p is padded to a multiple of 8 (`n_p_pad = (n_p + 7) & ~7`) so the j
loop has no scalar tail. Padding positions are set to `1e9f` and `sb_e`,
`sb_sq` to `0` so any unmasked contribution from those lanes would still
be ~zero through `expf(very negative) ≈ 0`.
EPS clamps from the float scalar predecessor are kept and lifted to SIMD
form (`_mm256_max_ps(r_ab_n, v_eps)` etc.). Same correctness reason as
the scalar version: float precision around close encounters can collapse
norms to zero.
The `expf` calls are still scalar inside the j loop: 8 of them per j
step, manually unrolled into a scratch `tmp_arg[8]`/`tmp_val[8]` array.
gcc refused to unroll the loop on its own (it serialised all 8 calls on
`xmm0`), the manual unroll at least lets them issue independently. Kill
these next with the polyexp variant.
## Code (relevant snippet)
```cpp
for (int j = 0; j < n_p_pad; j += 8) {
__m256 j_idx = _mm256_add_ps(_mm256_set1_ps((float)j), lane_off);
__m256 active = _mm256_and_ps(
_mm256_cmp_ps(j_idx, v_i, _CMP_NEQ_OQ),
_mm256_cmp_ps(j_idx, v_np, _CMP_LT_OQ));
__m256 pjx = _mm256_loadu_ps(&pos_x[j]);
__m256 pjy = _mm256_loadu_ps(&pos_y[j]);
// ... sb_e_x, sb_e_y, sb_sq
// body: r_ab, r_ab2, two_b, scalar expf x 8, smc, sm_r_ab, sm_r_ab2,
// f_x, f_y, w_val branch, AND with active, fmadd into acc_x/acc_y.
}
```
See `variants/variant_float_inlining_SoA_AVX2.cpp` for the full inner
loop.
## Benchmark
Conditions: min of N runs (TBD), N_ITER=1000, -O3 -march=native -flto.
Diff is vs `variant_float_inlining_SoA`.
### x86
**Machine**: TBD
**Frequency**: TBD
| N | W(n) [Flops] | T(n) [Cycles] | Performance [Flops/Cycle] | Diff vs Predecessor |
| ---- | ------------ | ------------- | ------------------------- | ------------------- |
| 50 | | | | |
| 100 | | | | |
| 200 | | | | |
| 500 | | | | |
### ARM (Apple)
Not applicable, this variant uses AVX2 intrinsics.
## Analysis
Same shape of win as the double AVX2 variant, just at 8-wide. Scalar
`expf` is still the dominant cost, polyexp kills it next.
One catch: this variant only hits its real performance if FTZ is
enabled in `main-float.cpp` (that gets added on the polyexp MR). Without
it the active-lane chain produces subnormal intermediates (the `smc =
numer/denom` step is the one that lands there for typical mid-range pair
distances) and every following FMA stalls 100+ cycles in microcode. With
FTZ on it runs as expected, without it benchmarks look 5-10x slow.
Verify passes against `baseline_float_n60_i500.csv` at 1cm tolerance.
Local max drift ~3mm.
## Takeaway
Polyexp is the next step. Note:
benchmarks are only meaningful with FTZ/DAZ on in main-float.cpp.
docs/optimization_float_inlining_SoA_AVX2_polyexp.md
+75
View File
@@ -0,0 +1,75 @@
# Optimization: SIMD polynomial exp (float, 8-wide) + FTZ/DAZ
- **Author**: Linus
- **Predecessor**: variant_float_inlining_SoA_AVX2.cpp
- **File(s)**: variants/variant_float_inlining_SoA_AVX2_polyexp.cpp, main-float.cpp (FTZ/DAZ enable)
- **Status**: TBD, positive locally (approx twice as fast)
## What this changes
The 8 scalar `expf` calls in the j-loop become a single SIMD polynomial
approximation `exp256_ps` that operates on `__m256`. Range-reduce
`x = n*ln2 + r` with `n = round(x/ln2)`, evaluate a degree-5 Horner
polynomial for `exp(r)` on `r in [-ln2/2, ln2/2]`, then scale by `2^n`
via the Schraudolph bit-trick on the IEEE 754 exponent slot (shift the
biased exponent by 23). Degree-5 is enough for float, the polynomial is
good to ~1e-5 absolute on the reduced range, well under the 1cm verify
tolerance.
`main-float.cpp` also gets two MXCSR writes at the top of `main()`:
_MM_SET_FLUSH_ZERO_MODE(_MM_FLUSH_ZERO_ON);
_MM_SET_DENORMALS_ZERO_MODE(_MM_DENORMALS_ZERO_ON);
This was the part that actually mattered, see Analysis.
## Code (relevant snippet)
See `variants/variant_float_inlining_SoA_AVX2_polyexp.cpp` for the full
j-loop and `main-float.cpp` for the FTZ/DAZ setup.
## Benchmark
Conditions: min of N runs (TBD), N_ITER=1000, -O3 -march=native -flto.
Diff is vs `variant_float_inlining_SoA_AVX2`.
### x86
**Machine**: TBD
**Frequency**: TBD
| N | W(n) [Flops] | T(n) [Cycles] | Performance [Flops/Cycle] | Diff vs Predecessor |
| ---- | ------------ | ------------- | ------------------------- | ------------------- |
| 50 | | | | |
| 100 | | | | |
| 200 | | | | |
| 500 | | | | |
### ARM (Apple)
Not applicable, AVX2-only. A NEON port would need its own vectorised
exp.
## Analysis
roughly twice as fast
## Takeaway
Polyexp helps as expected. The real fix is FTZ/DAZ in `main-float.cpp`,
without it everything downstream in the float chain is silently
subnormal-stalled
## References
- Schraudolph, N. N. (1999). "A Fast, Compact Approximation of the
Exponential Function." Neural Computation 11(4), 853-862.
- IEEE 754 double-precision format. Wikipedia.
https://en.wikipedia.org/wiki/Double-precision_floating-point_format
- Horner's method. Wikipedia.
https://en.wikipedia.org/wiki/Horner%27s_method
main-float.cpp
+7
View File
@@ -76,6 +76,13 @@ long long flush_cache() {
} }
int main(int argc, const char *argv[]) { int main(int argc, const char *argv[]) {
#ifdef __x86_64__
// flush denormals to zero via MXCSR. one-time, per thread.
// any subnormal in the j-loop chain stalls fma for 100+ cycles, this kills that.
_MM_SET_FLUSH_ZERO_MODE(_MM_FLUSH_ZERO_ON);
_MM_SET_DENORMALS_ZERO_MODE(_MM_DENORMALS_ZERO_ON);
#endif
// Check that the correct number of arguments has been passed. // Check that the correct number of arguments has been passed.
// //
// Expected usage: // Expected usage:
reference/baseline_float_n60_i500.csv
+30061
View File
File diff suppressed because it is too large Load Diff
variants/variant_float_inlining_SoA.cpp
+349
View File
@@ -0,0 +1,349 @@
// # Changes: SoA layout for pedestrians (file-scope arrays instead of struct fields)
// # Predecessor: variant_float_inlining.cpp
#include "../headers/board_reduced_float.hpp"
#include "../headers/constants_float.hpp"
#include "../headers/pedestrian_reduced_float.hpp"
#include <algorithm>
#include <cmath>
#include <fstream>
#include <iostream>
#include <math.h>
#include <random>
#include <tuple>
#include <vector>
#define SEED 42
#define BOARD_WIDTH 10.0f
#define BOARD_LENGTH 50.0f
#define WALL_EPS 1e-3f
#define BOARD_WIDTH_MINUS_WALL_EPS 9.999f
#ifndef RANDOM_SEED
#define RANDOM_SEED 0
#endif
#ifndef BENCHMARK
#define BENCHMARK 0
#endif
using namespace std;
// we wanna use arrays so we can implement SoA, one array per pedestrian field
vector<float> pos_x, pos_y;
vector<float> sb_sq;
vector<float> sb_e_x, sb_e_y;
vector<float> e_x, e_y;
vector<float> curr_vel_x, curr_vel_y;
vector<float> w_x, w_y;
vector<float> desired_v0;
vector<float> v_max;
vector<float> destination_x, destination_y;
vector<int> color;
Pedestrian *init_pedestrians(int n_p) {
#if RANDOM_SEED
random_device rd;
mt19937 gen(rd());
#else
mt19937 gen(SEED);
#endif
normal_distribution<float> velocity_gen(v0_mean, v0_std_dev);
uniform_real_distribution<float> y_gen(0.3, BOARD_WIDTH - 0.3);
uniform_real_distribution<float> x_gen_left(2.0, BOARD_LENGTH / 2.0);
uniform_real_distribution<float> x_gen_right(BOARD_LENGTH / 2.0, BOARD_LENGTH - 2.0);
// resize all arrays for this run
pos_x.resize(n_p);
pos_y.resize(n_p);
sb_sq.resize(n_p);
sb_e_x.resize(n_p);
sb_e_y.resize(n_p);
e_x.resize(n_p);
e_y.resize(n_p);
curr_vel_x.resize(n_p);
curr_vel_y.resize(n_p);
w_x.resize(n_p);
w_y.resize(n_p);
desired_v0.resize(n_p);
v_max.resize(n_p);
destination_x.resize(n_p);
destination_y.resize(n_p);
color.resize(n_p);
int n_p_half = n_p / 2;
for (int i = 0; i < n_p_half; i++) {
float x = x_gen_left(gen);
float y = y_gen(gen);
float dest_x = BOARD_LENGTH + 30.0; // renamed to avoid shadowing destination_x[]
float dest_y = y;
float init_e_x = dest_x - x;
float init_e_y = dest_y - y;
float init_e_norm = sqrt(init_e_x * init_e_x + init_e_y * init_e_y);
float init_e_unit_x = init_e_x / init_e_norm;
float init_e_unit_y = init_e_y / init_e_norm;
float d_v0;
do {
d_v0 = velocity_gen(gen);
} while (d_v0 <= 0.1);
float v_m = 1.3 * d_v0;
color[i] = 0;
pos_x[i] = x;
pos_y[i] = y;
e_x[i] = init_e_unit_x;
e_y[i] = init_e_unit_y;
v_max[i] = v_m;
desired_v0[i] = d_v0;
destination_x[i] = dest_x;
destination_y[i] = dest_y;
// Fields the Pedestrian constructor would have zero-initialized (c_v_x=c_v_y=0):
w_x[i] = 0.0f;
w_y[i] = 0.0f;
curr_vel_x[i] = 0.0f;
curr_vel_y[i] = 0.0f;
sb_sq[i] = 0.0f;
sb_e_x[i] = 0.0f;
sb_e_y[i] = 0.0f;
}
for (int i = n_p_half; i < n_p; i++) {
float x = x_gen_right(gen);
float y = y_gen(gen);
float dest_x = -30;
float dest_y = y;
float init_e_x = dest_x - x;
float init_e_y = dest_y - y;
float init_e_norm = sqrt(init_e_x * init_e_x + init_e_y * init_e_y);
float init_e_unit_x = init_e_x / init_e_norm;
float init_e_unit_y = init_e_y / init_e_norm;
float d_v0;
do {
d_v0 = velocity_gen(gen);
} while (d_v0 <= 0.1);
float v_m = 1.3 * d_v0;
color[i] = 1;
pos_x[i] = x;
pos_y[i] = y;
e_x[i] = init_e_unit_x;
e_y[i] = init_e_unit_y;
v_max[i] = v_m;
desired_v0[i] = d_v0;
destination_x[i] = dest_x;
destination_y[i] = dest_y;
w_x[i] = 0.0f;
w_y[i] = 0.0f;
curr_vel_x[i] = 0.0f;
curr_vel_y[i] = 0.0f;
sb_sq[i] = 0.0f;
sb_e_x[i] = 0.0f;
sb_e_y[i] = 0.0f;
}
return nullptr;
}
Board init_board_without_obstacles() { return Board(BOARD_LENGTH, BOARD_WIDTH, 0, nullptr); }
int run_simulation(Pedestrian *pedestrians, Board &board, int n_p, int n_iter, const char *dump_path) {
(void)pedestrians;
#if BENCHMARK
(void)dump_path;
#endif
#if !BENCHMARK
const int dump_interval = 1;
vector<tuple<int, int, float, float, float, float, int>> samples;
if (dump_path) {
samples.reserve(static_cast<size_t>(n_iter / dump_interval + 1) * n_p);
for (int i = 0; i < n_p; i++) {
samples.emplace_back(0, i, pos_x[i], pos_y[i], e_x[i], e_y[i], color[i]);
}
}
#endif
for (int t = 0; t < n_iter; t++) {
for (int i = 0; i < n_p; i++) {
float desired_velocity_x = e_x[i] * desired_v0[i];
float desired_velocity_y = e_y[i] * desired_v0[i];
float f_alpha_dt_x = (desired_velocity_x - curr_vel_x[i]) * dt_times_tau_alpha_inv;
float f_alpha_dt_y = (desired_velocity_y - curr_vel_y[i]) * dt_times_tau_alpha_inv;
for (int j = 0; j < n_p; j++) {
if (i == j)
continue;
float r_ab_x = pos_x[i] - pos_x[j];
float r_ab_y = pos_y[i] - pos_y[j];
float r_ab_norm = sqrt(r_ab_x * r_ab_x + r_ab_y * r_ab_y);
r_ab_norm = r_ab_norm > EPS ? r_ab_norm : EPS;
float r_ab2_x = r_ab_x - sb_e_x[j];
float r_ab2_y = r_ab_y - sb_e_y[j];
float r_ab2_norm = sqrt(r_ab2_x * r_ab2_x + r_ab2_y * r_ab2_y);
r_ab2_norm = r_ab2_norm > EPS ? r_ab2_norm : EPS;
float r_ab_norm_sum = r_ab_norm + r_ab2_norm;
float two_b_sq = r_ab_norm_sum * r_ab_norm_sum - sb_sq[j];
two_b_sq = two_b_sq > EPS_SQ ? two_b_sq : EPS_SQ;
float two_b = sqrt(two_b_sq);
float scalar_mult_common = (r_ab_norm_sum / (two_b * r_ab_norm * r_ab2_norm)) * (NEG_V0_AB_div_TWO_SIGMA * exp(two_b * NEG_TWO_SIGMA_INV));
float scalar_mult_r_ab = scalar_mult_common * r_ab2_norm;
float scalar_mult_r_ab2 = scalar_mult_common * r_ab_norm;
float f_alpha_beta_neg_x = r_ab_x * scalar_mult_r_ab + r_ab2_x * scalar_mult_r_ab2;
float f_alpha_beta_neg_y = r_ab_y * scalar_mult_r_ab + r_ab2_y * scalar_mult_r_ab2;
float w_val;
float f_norm_sq = f_alpha_beta_neg_x * f_alpha_beta_neg_x + f_alpha_beta_neg_y * f_alpha_beta_neg_y;
float rhs = f_norm_sq * cos_phi_sq;
float e_dot_f = e_x[i] * f_alpha_beta_neg_x + e_y[i] * f_alpha_beta_neg_y;
float e_dot_f_sq = e_dot_f * e_dot_f;
if (e_dot_f >= 0.0 || e_dot_f_sq <= rhs)
w_val = dt_neg;
else
w_val = c_neg_times_dt;
f_alpha_dt_x = f_alpha_dt_x + f_alpha_beta_neg_x * w_val;
f_alpha_dt_y = f_alpha_dt_y + f_alpha_beta_neg_y * w_val;
}
for (int k = 0; k < board.n_obstacles; k++) {
float v1_x = pos_x[i] - board.obstacles[k].start_x;
float v1_y = pos_y[i] - board.obstacles[k].start_y;
float v2_x = board.obstacles[k].obstacle_v_x;
float v2_y = board.obstacles[k].obstacle_v_y;
float projection = (v1_x * v2_x + v1_y * v2_y) * board.obstacles[k].length_sq_inv;
if (projection < 0.0)
projection = 0.0;
else if (projection > 1.0)
projection = 1.0;
float r_aB_x = pos_x[i] - (board.obstacles[k].start_x + board.obstacles[k].obstacle_v_x * projection);
float r_aB_y = pos_y[i] - (board.obstacles[k].start_y + board.obstacles[k].obstacle_v_y * projection);
float r_aB_norm = sqrt(r_aB_x * r_aB_x + r_aB_y * r_aB_y);
r_aB_norm = r_aB_norm > EPS ? r_aB_norm : EPS;
float scalar_mult = (NEG_U0_AB_div_R * exp(r_aB_norm * NEG_R_INV)) / r_aB_norm;
float f_alpha_B_neg_x = r_aB_x * scalar_mult;
float f_alpha_B_neg_y = r_aB_y * scalar_mult;
float w_val;
float e_dot_f = (f_alpha_B_neg_y == 0) ? 0 : (f_alpha_B_neg_y < 0 ? -e_y[i] : e_y[i]);
if (e_dot_f >= cos_phi)
w_val = dt_neg;
else
w_val = c_neg_times_dt;
float f_alpha_B_dt_x = f_alpha_B_neg_x * w_val;
float f_alpha_B_dt_y = f_alpha_B_neg_y * w_val;
f_alpha_dt_x += f_alpha_B_dt_x;
f_alpha_dt_y += f_alpha_B_dt_y;
}
w_x[i] += f_alpha_dt_x;
w_y[i] += f_alpha_dt_y;
}
for (int i = 0; i < n_p; i++) {
float w_norm = sqrt(w_x[i] * w_x[i] + w_y[i] * w_y[i]);
float w_norm2 = w_norm > EPS ? w_norm : EPS;
float correction_factor = min(1.0f, v_max[i] / w_norm2);
curr_vel_x[i] = w_x[i] * correction_factor;
curr_vel_y[i] = w_y[i] * correction_factor;
pos_x[i] = pos_x[i] + curr_vel_x[i] * dt;
pos_y[i] = pos_y[i] + curr_vel_y[i] * dt;
if (color[i] == 0 && pos_x[i] > (BOARD_LENGTH - 1.5)) {
pos_x[i] = 1.0;
destination_y[i] = pos_y[i];
w_x[i] = desired_v0[i];
w_y[i] = 0.0;
curr_vel_x[i] = w_x[i];
curr_vel_y[i] = w_y[i];
}
if (color[i] == 1 && pos_x[i] < 1.5) {
pos_x[i] = BOARD_LENGTH - 1.0;
destination_y[i] = pos_y[i];
w_x[i] = -desired_v0[i];
w_y[i] = 0.0;
curr_vel_x[i] = w_x[i];
curr_vel_y[i] = w_y[i];
}
pos_y[i] = clamp(pos_y[i], WALL_EPS, BOARD_WIDTH_MINUS_WALL_EPS);
float ex = destination_x[i] - pos_x[i];
float ey = destination_y[i] - pos_y[i];
float e_norm = sqrt(ex * ex + ey * ey);
float e_norm2 = e_norm > EPS ? e_norm : EPS;
float e_norm_inv = 1.0 / e_norm2;
e_x[i] = ex * e_norm_inv;
e_y[i] = ey * e_norm_inv;
float velocity_norm_sq = curr_vel_x[i] * curr_vel_x[i] + curr_vel_y[i] * curr_vel_y[i];
float velocity_norm = sqrt(velocity_norm_sq);
sb_sq[i] = velocity_norm_sq * delta_t_sq;
sb_e_x[i] = e_x[i] * (velocity_norm * delta_t);
sb_e_y[i] = e_y[i] * (velocity_norm * delta_t);
#if !BENCHMARK
if (dump_path && (t % dump_interval == 0)) {
samples.emplace_back(t + 1, i, pos_x[i], pos_y[i], e_x[i], e_y[i], color[i]);
}
#endif
}
}
#if !BENCHMARK
if (dump_path) {
ofstream out(dump_path);
out << "time,id,x,y,p.e.x,p.e.y,color\n";
for (auto &[t, id, x, y, e_x, e_y, c] : samples) {
out << t << ',' << id << ',' << x << ',' << y << ',' << e_x << ',' << e_y << ',' << c << '\n';
}
}
#endif
return 0;
}
variants/variant_float_inlining_SoA_AVX2.cpp
+443
View File
@@ -0,0 +1,443 @@
// # Changes: AVX2 vectorization of the ped-ped inner loop (8 floats per __m256, exp still scalar)
// # Predecessor: variant_float_inlining_SoA.cpp
#include "../headers/board_reduced_float.hpp"
#include "../headers/constants_float.hpp"
#include "../headers/pedestrian_reduced_float.hpp"
#include <immintrin.h> // avx2
#include <algorithm>
#include <cmath>
#include <fstream>
#include <iostream>
#include <math.h>
#include <random>
#include <tuple>
#include <vector>
#define SEED 42
#define BOARD_WIDTH 10.0f
#define BOARD_LENGTH 50.0f
#define WALL_EPS 1e-3f
#define BOARD_WIDTH_MINUS_WALL_EPS 9.999f
#ifndef RANDOM_SEED
#define RANDOM_SEED 0
#endif
#ifndef BENCHMARK
#define BENCHMARK 0
#endif
using namespace std;
// same SoA arrays as the predecessor, but we pad to multiple of 8 so the avx loop is clean
vector<float> pos_x, pos_y;
vector<float> sb_sq;
vector<float> sb_e_x, sb_e_y;
vector<float> e_x, e_y;
vector<float> curr_vel_x, curr_vel_y;
vector<float> w_x, w_y;
vector<float> desired_v0;
vector<float> v_max;
vector<float> destination_x, destination_y;
vector<int> color;
Pedestrian *init_pedestrians(int n_p) {
#if RANDOM_SEED
random_device rd;
mt19937 gen(rd());
#else
mt19937 gen(SEED);
#endif
normal_distribution<float> velocity_gen(v0_mean, v0_std_dev);
uniform_real_distribution<float> y_gen(0.3, BOARD_WIDTH - 0.3);
uniform_real_distribution<float> x_gen_left(2.0, BOARD_LENGTH / 2.0);
uniform_real_distribution<float> x_gen_right(BOARD_LENGTH / 2.0, BOARD_LENGTH - 2.0);
const int n_p_pad = (n_p + 7) & ~7; // round up to a multiple of 8 for avx float lanes
pos_x.resize(n_p_pad);
pos_y.resize(n_p_pad);
sb_sq.resize(n_p_pad);
sb_e_x.resize(n_p_pad);
sb_e_y.resize(n_p_pad);
e_x.resize(n_p_pad);
e_y.resize(n_p_pad);
curr_vel_x.resize(n_p_pad);
curr_vel_y.resize(n_p_pad);
w_x.resize(n_p_pad);
w_y.resize(n_p_pad);
desired_v0.resize(n_p_pad);
v_max.resize(n_p_pad);
destination_x.resize(n_p_pad);
destination_y.resize(n_p_pad);
color.resize(n_p_pad);
int n_p_half = n_p / 2;
for (int i = 0; i < n_p_half; i++) {
float x = x_gen_left(gen);
float y = y_gen(gen);
float dest_x = BOARD_LENGTH + 30.0; // renamed to avoid shadowing destination_x[]
float dest_y = y;
float init_e_x = dest_x - x;
float init_e_y = dest_y - y;
float init_e_norm = sqrt(init_e_x * init_e_x + init_e_y * init_e_y);
float init_e_unit_x = init_e_x / init_e_norm;
float init_e_unit_y = init_e_y / init_e_norm;
float d_v0;
do {
d_v0 = velocity_gen(gen);
} while (d_v0 <= 0.1);
float v_m = 1.3 * d_v0;
color[i] = 0;
pos_x[i] = x;
pos_y[i] = y;
e_x[i] = init_e_unit_x;
e_y[i] = init_e_unit_y;
v_max[i] = v_m;
desired_v0[i] = d_v0;
destination_x[i] = dest_x;
destination_y[i] = dest_y;
w_x[i] = 0.0f;
w_y[i] = 0.0f;
curr_vel_x[i] = 0.0f;
curr_vel_y[i] = 0.0f;
sb_sq[i] = 0.0f;
sb_e_x[i] = 0.0f;
sb_e_y[i] = 0.0f;
}
for (int i = n_p_half; i < n_p; i++) {
float x = x_gen_right(gen);
float y = y_gen(gen);
float dest_x = -30;
float dest_y = y;
float init_e_x = dest_x - x;
float init_e_y = dest_y - y;
float init_e_norm = sqrt(init_e_x * init_e_x + init_e_y * init_e_y);
float init_e_unit_x = init_e_x / init_e_norm;
float init_e_unit_y = init_e_y / init_e_norm;
float d_v0;
do {
d_v0 = velocity_gen(gen);
} while (d_v0 <= 0.1);
float v_m = 1.3 * d_v0;
color[i] = 1;
pos_x[i] = x;
pos_y[i] = y;
e_x[i] = init_e_unit_x;
e_y[i] = init_e_unit_y;
v_max[i] = v_m;
desired_v0[i] = d_v0;
destination_x[i] = dest_x;
destination_y[i] = dest_y;
w_x[i] = 0.0f;
w_y[i] = 0.0f;
curr_vel_x[i] = 0.0f;
curr_vel_y[i] = 0.0f;
sb_sq[i] = 0.0f;
sb_e_x[i] = 0.0f;
sb_e_y[i] = 0.0f;
}
// padding lanes: park them far away, zero everything else.
// exp(very negative) is roughly 0 and the active mask zeroes their contribution anyway.
for (int j = n_p; j < n_p_pad; j++) {
pos_x[j] = 1e9f;
pos_y[j] = 1e9f;
sb_sq[j] = 0.0f;
sb_e_x[j] = 0.0f;
sb_e_y[j] = 0.0f;
}
return nullptr;
}
Board init_board_without_obstacles() { return Board(BOARD_LENGTH, BOARD_WIDTH, 0, nullptr); }
int run_simulation(Pedestrian *pedestrians, Board &board, int n_p, int n_iter, const char *dump_path) {
(void)pedestrians;
#if BENCHMARK
(void)dump_path;
#endif
#if !BENCHMARK
const int dump_interval = 1;
vector<tuple<int, int, float, float, float, float, int>> samples;
if (dump_path) {
samples.reserve(static_cast<size_t>(n_iter / dump_interval + 1) * n_p);
for (int i = 0; i < n_p; i++) {
samples.emplace_back(0, i, pos_x[i], pos_y[i], e_x[i], e_y[i], color[i]);
}
}
#endif
const int n_p_pad = (n_p + 7) & ~7;
// {0,1,2,3,4,5,6,7} so we can build per-iter j indices as (j + lane_off)
alignas(32) const float lane_offset_arr[8] = {0.0f, 1.0f, 2.0f, 3.0f, 4.0f, 5.0f, 6.0f, 7.0f};
const __m256 lane_off = _mm256_load_ps(lane_offset_arr);
// constants broadcast to all 8 lanes once
const __m256 v_neg_v0_div_2s = _mm256_set1_ps(NEG_V0_AB_div_TWO_SIGMA);
const __m256 v_neg_2s_inv = _mm256_set1_ps(NEG_TWO_SIGMA_INV);
const __m256 v_cos_phi_sq = _mm256_set1_ps(cos_phi_sq);
const __m256 v_dt_neg = _mm256_set1_ps(dt_neg);
const __m256 v_c_neg_dt = _mm256_set1_ps(c_neg_times_dt);
const __m256 v_zero = _mm256_setzero_ps();
const __m256 v_one = _mm256_set1_ps(1.0f);
const __m256 v_eps = _mm256_set1_ps(EPS);
const __m256 v_eps_sq = _mm256_set1_ps(EPS_SQ);
const __m256 v_np = _mm256_set1_ps((float)n_p);
for (int t = 0; t < n_iter; t++) {
for (int i = 0; i < n_p; i++) {
float desired_velocity_x = e_x[i] * desired_v0[i];
float desired_velocity_y = e_y[i] * desired_v0[i];
float f_alpha_dt_x = (desired_velocity_x - curr_vel_x[i]) * dt_times_tau_alpha_inv;
float f_alpha_dt_y = (desired_velocity_y - curr_vel_y[i]) * dt_times_tau_alpha_inv;
// i is fixed for the inner loops, broadcast its fields once
const __m256 pix = _mm256_set1_ps(pos_x[i]);
const __m256 piy = _mm256_set1_ps(pos_y[i]);
const __m256 eix = _mm256_set1_ps(e_x[i]);
const __m256 eiy = _mm256_set1_ps(e_y[i]);
const __m256 v_i = _mm256_set1_ps((float)i);
__m256 acc_x = _mm256_setzero_ps();
__m256 acc_y = _mm256_setzero_ps();
for (int j = 0; j < n_p_pad; j += 8) {
// 8 lanes at once, lane k maps to pedestrian j+k
__m256 j_idx = _mm256_add_ps(_mm256_set1_ps((float)j), lane_off);
// active = (j+k != i) AND (j+k < n_p), zero in lanes we wanna skip
__m256 m_neq_i = _mm256_cmp_ps(j_idx, v_i, _CMP_NEQ_OQ);
__m256 m_in_range = _mm256_cmp_ps(j_idx, v_np, _CMP_LT_OQ);
__m256 active = _mm256_and_ps(m_neq_i, m_in_range);
// j-side loads, SoA is contiguous. loadu because std::vector only guarantees 16B align
__m256 pjx = _mm256_loadu_ps(&pos_x[j]);
__m256 pjy = _mm256_loadu_ps(&pos_y[j]);
__m256 sbe_jx = _mm256_loadu_ps(&sb_e_x[j]);
__m256 sbe_jy = _mm256_loadu_ps(&sb_e_y[j]);
__m256 sb_jsq = _mm256_loadu_ps(&sb_sq[j]);
// r_ab = pos_i - pos_j ; |r_ab| with the EPS clamp the scalar version does
__m256 r_ab_x = _mm256_sub_ps(pix, pjx);
__m256 r_ab_y = _mm256_sub_ps(piy, pjy);
__m256 r_ab_n2 = _mm256_fmadd_ps(r_ab_x, r_ab_x, _mm256_mul_ps(r_ab_y, r_ab_y));
__m256 r_ab_n = _mm256_sqrt_ps(r_ab_n2);
r_ab_n = _mm256_max_ps(r_ab_n, v_eps);
// r_ab2 = r_ab - sb_e_j ; |r_ab2| also clamped
__m256 r_ab2_x = _mm256_sub_ps(r_ab_x, sbe_jx);
__m256 r_ab2_y = _mm256_sub_ps(r_ab_y, sbe_jy);
__m256 r_ab2_n2 = _mm256_fmadd_ps(r_ab2_x, r_ab2_x, _mm256_mul_ps(r_ab2_y, r_ab2_y));
__m256 r_ab2_n = _mm256_sqrt_ps(r_ab2_n2);
r_ab2_n = _mm256_max_ps(r_ab2_n, v_eps);
// 2b = sqrt((|r_ab| + |r_ab2|)^2 - sb_sq_j) ; clamp the inside with EPS_SQ
__m256 r_sum = _mm256_add_ps(r_ab_n, r_ab2_n);
__m256 two_b_sq = _mm256_fmsub_ps(r_sum, r_sum, sb_jsq);
two_b_sq = _mm256_max_ps(two_b_sq, v_eps_sq);
__m256 two_b = _mm256_sqrt_ps(two_b_sq);
// exp scalar fallback. manually unrolled, otherwise gcc keeps it as a loop with
// one expf call and serialises all 8 (the double avx2 with 4 iters does get unrolled, the 8 here doesn't).
__m256 e_arg = _mm256_mul_ps(two_b, v_neg_2s_inv);
alignas(32) float tmp_arg[8], tmp_val[8];
_mm256_store_ps(tmp_arg, e_arg);
tmp_val[0] = expf(tmp_arg[0]);
tmp_val[1] = expf(tmp_arg[1]);
tmp_val[2] = expf(tmp_arg[2]);
tmp_val[3] = expf(tmp_arg[3]);
tmp_val[4] = expf(tmp_arg[4]);
tmp_val[5] = expf(tmp_arg[5]);
tmp_val[6] = expf(tmp_arg[6]);
tmp_val[7] = expf(tmp_arg[7]);
__m256 e_val = _mm256_load_ps(tmp_val);
// smc = (r_sum * NEG_V0_AB_div_TWO_SIGMA * e_val) / (two_b * r_ab_n * r_ab2_n)
// i==j and padding lanes collapse the denom, so we swap in 1.0 there to keep smc finite.
// the AND with active below zeroes those lanes anyway.
__m256 numer = _mm256_mul_ps(_mm256_mul_ps(r_sum, v_neg_v0_div_2s), e_val);
__m256 denom = _mm256_mul_ps(_mm256_mul_ps(two_b, r_ab_n), r_ab2_n);
__m256 denom_safe = _mm256_blendv_ps(v_one, denom, active);
__m256 smc = _mm256_div_ps(numer, denom_safe);
__m256 sm_r_ab = _mm256_mul_ps(smc, r_ab2_n);
__m256 sm_r_ab2 = _mm256_mul_ps(smc, r_ab_n);
// scalar: f_x = r_ab_x * sm_r_ab + r_ab2_x * sm_r_ab2
__m256 f_x = _mm256_fmadd_ps(r_ab2_x, sm_r_ab2, _mm256_mul_ps(r_ab_x, sm_r_ab));
__m256 f_y = _mm256_fmadd_ps(r_ab2_y, sm_r_ab2, _mm256_mul_ps(r_ab_y, sm_r_ab));
// float w_val: default dt_neg, switch to c_neg_times_dt when
//(e_dot_f < 0 AND e_dot_f_sq > f_norm_sq * cos_phi_sq).
// this is the DeMorgan of the scalar's if (e_dot_f >= 0 || e_dot_f_sq <= rhs)
__m256 f_norm_sq = _mm256_fmadd_ps(f_x, f_x, _mm256_mul_ps(f_y, f_y));
__m256 rhs = _mm256_mul_ps(f_norm_sq, v_cos_phi_sq);
__m256 e_dot_f = _mm256_fmadd_ps(eix, f_x, _mm256_mul_ps(eiy, f_y));
__m256 e_dot_f_sq = _mm256_mul_ps(e_dot_f, e_dot_f);
__m256 c1 = _mm256_cmp_ps(e_dot_f, v_zero, _CMP_LT_OQ);
__m256 c2 = _mm256_cmp_ps(e_dot_f_sq, rhs, _CMP_GT_OQ);
__m256 cond = _mm256_and_ps(c1, c2);
__m256 w_val = _mm256_blendv_ps(v_dt_neg, v_c_neg_dt, cond);
// mask out i==j lane and padding before accumulating.
// AND with all-zeros = 0, AND with all-ones passes through, cheaper than blendv here.
__m256 f_x_safe = _mm256_and_ps(f_x, active);
__m256 f_y_safe = _mm256_and_ps(f_y, active);
acc_x = _mm256_fmadd_ps(f_x_safe, w_val, acc_x);
acc_y = _mm256_fmadd_ps(f_y_safe, w_val, acc_y);
}
// horizontal sum of acc_x and acc_y at the same time
__m256 hsum = _mm256_hadd_ps(acc_x, acc_y);
__m128 hsum_lo = _mm256_castps256_ps128(hsum);
__m128 hsum_hi = _mm256_extractf128_ps(hsum, 1);
__m128 sum4 = _mm_add_ps(hsum_lo, hsum_hi);
sum4 = _mm_hadd_ps(sum4, sum4);
alignas(16) float sum_out[4];
_mm_store_ps(sum_out, sum4);
f_alpha_dt_x += sum_out[0];
f_alpha_dt_y += sum_out[1];
// obstacle loop kept scalar, board.n_obstacles is 0 in this setup anyway
for (int k = 0; k < board.n_obstacles; k++) {
float v1_x = pos_x[i] - board.obstacles[k].start_x;
float v1_y = pos_y[i] - board.obstacles[k].start_y;
float v2_x = board.obstacles[k].obstacle_v_x;
float v2_y = board.obstacles[k].obstacle_v_y;
float projection = (v1_x * v2_x + v1_y * v2_y) * board.obstacles[k].length_sq_inv;
if (projection < 0.0)
projection = 0.0;
else if (projection > 1.0)
projection = 1.0;
float r_aB_x = pos_x[i] - (board.obstacles[k].start_x + board.obstacles[k].obstacle_v_x * projection);
float r_aB_y = pos_y[i] - (board.obstacles[k].start_y + board.obstacles[k].obstacle_v_y * projection);
float r_aB_norm = sqrt(r_aB_x * r_aB_x + r_aB_y * r_aB_y);
r_aB_norm = r_aB_norm > EPS ? r_aB_norm : EPS;
float scalar_mult = (NEG_U0_AB_div_R * exp(r_aB_norm * NEG_R_INV)) / r_aB_norm;
float f_alpha_B_neg_x = r_aB_x * scalar_mult;
float f_alpha_B_neg_y = r_aB_y * scalar_mult;
float w_val;
float e_dot_f = (f_alpha_B_neg_y == 0) ? 0 : (f_alpha_B_neg_y < 0 ? -e_y[i] : e_y[i]);
if (e_dot_f >= cos_phi)
w_val = dt_neg;
else
w_val = c_neg_times_dt;
float f_alpha_B_dt_x = f_alpha_B_neg_x * w_val;
float f_alpha_B_dt_y = f_alpha_B_neg_y * w_val;
f_alpha_dt_x += f_alpha_B_dt_x;
f_alpha_dt_y += f_alpha_B_dt_y;
}
w_x[i] += f_alpha_dt_x;
w_y[i] += f_alpha_dt_y;
}
// position update loop stays scalar, same as the double AVX2 variant does
for (int i = 0; i < n_p; i++) {
float w_norm = sqrt(w_x[i] * w_x[i] + w_y[i] * w_y[i]);
float w_norm2 = w_norm > EPS ? w_norm : EPS;
float correction_factor = min(1.0f, v_max[i] / w_norm2);
curr_vel_x[i] = w_x[i] * correction_factor;
curr_vel_y[i] = w_y[i] * correction_factor;
pos_x[i] = pos_x[i] + curr_vel_x[i] * dt;
pos_y[i] = pos_y[i] + curr_vel_y[i] * dt;
if (color[i] == 0 && pos_x[i] > (BOARD_LENGTH - 1.5)) {
pos_x[i] = 1.0;
destination_y[i] = pos_y[i];
w_x[i] = desired_v0[i];
w_y[i] = 0.0;
curr_vel_x[i] = w_x[i];
curr_vel_y[i] = w_y[i];
}
if (color[i] == 1 && pos_x[i] < 1.5) {
pos_x[i] = BOARD_LENGTH - 1.0;
destination_y[i] = pos_y[i];
w_x[i] = -desired_v0[i];
w_y[i] = 0.0;
curr_vel_x[i] = w_x[i];
curr_vel_y[i] = w_y[i];
}
pos_y[i] = clamp(pos_y[i], WALL_EPS, BOARD_WIDTH_MINUS_WALL_EPS);
float ex = destination_x[i] - pos_x[i];
float ey = destination_y[i] - pos_y[i];
float e_norm = sqrt(ex * ex + ey * ey);
float e_norm2 = e_norm > EPS ? e_norm : EPS;
float e_norm_inv = 1.0 / e_norm2;
e_x[i] = ex * e_norm_inv;
e_y[i] = ey * e_norm_inv;
float velocity_norm_sq = curr_vel_x[i] * curr_vel_x[i] + curr_vel_y[i] * curr_vel_y[i];
float velocity_norm = sqrt(velocity_norm_sq);
sb_sq[i] = velocity_norm_sq * delta_t_sq;
sb_e_x[i] = e_x[i] * (velocity_norm * delta_t);
sb_e_y[i] = e_y[i] * (velocity_norm * delta_t);
#if !BENCHMARK
if (dump_path && (t % dump_interval == 0)) {
samples.emplace_back(t + 1, i, pos_x[i], pos_y[i], e_x[i], e_y[i], color[i]);
}
#endif
}
}
#if !BENCHMARK
if (dump_path) {
ofstream out(dump_path);
out << "time,id,x,y,p.e.x,p.e.y,color\n";
for (auto &[t, id, x, y, e_x, e_y, c] : samples) {
out << t << ',' << id << ',' << x << ',' << y << ',' << e_x << ',' << e_y << ',' << c << '\n';
}
}
#endif
return 0;
}
variants/variant_float_inlining_SoA_AVX2_polyexp.cpp
+457
View File
@@ -0,0 +1,457 @@
// # Changes: simd polynomial expf, replaces the 8 scalar expf calls in the j loop
// # Predecessor: variant_float_inlining_SoA_AVX2.cpp
// the for-loop of 8 scalar expf calls was serialised by gcc (it didnt unroll), big perf hit.
// replace with one inlined simd polynomial approx, no libm call at all.
#include "../headers/board_reduced_float.hpp"
#include "../headers/constants_float.hpp"
#include "../headers/pedestrian_reduced_float.hpp"
#include <immintrin.h> // avx2
#include <algorithm>
#include <cmath>
#include <fstream>
#include <iostream>
#include <math.h>
#include <random>
#include <tuple>
#include <vector>
#define SEED 42
#define BOARD_WIDTH 10.0f
#define BOARD_LENGTH 50.0f
#define WALL_EPS 1e-3f
#define BOARD_WIDTH_MINUS_WALL_EPS 9.999f
#ifndef RANDOM_SEED
#define RANDOM_SEED 0
#endif
#ifndef BENCHMARK
#define BENCHMARK 0
#endif
using namespace std;
vector<float> pos_x, pos_y;
vector<float> sb_sq;
vector<float> sb_e_x, sb_e_y;
vector<float> e_x, e_y;
vector<float> curr_vel_x, curr_vel_y;
vector<float> w_x, w_y;
vector<float> desired_v0;
vector<float> v_max;
vector<float> destination_x, destination_y;
vector<int> color;
Pedestrian *init_pedestrians(int n_p) {
#if RANDOM_SEED
random_device rd;
mt19937 gen(rd());
#else
mt19937 gen(SEED);
#endif
normal_distribution<float> velocity_gen(v0_mean, v0_std_dev);
uniform_real_distribution<float> y_gen(0.3, BOARD_WIDTH - 0.3);
uniform_real_distribution<float> x_gen_left(2.0, BOARD_LENGTH / 2.0);
uniform_real_distribution<float> x_gen_right(BOARD_LENGTH / 2.0, BOARD_LENGTH - 2.0);
const int n_p_pad = (n_p + 7) & ~7;
pos_x.resize(n_p_pad);
pos_y.resize(n_p_pad);
sb_sq.resize(n_p_pad);
sb_e_x.resize(n_p_pad);
sb_e_y.resize(n_p_pad);
e_x.resize(n_p_pad);
e_y.resize(n_p_pad);
curr_vel_x.resize(n_p_pad);
curr_vel_y.resize(n_p_pad);
w_x.resize(n_p_pad);
w_y.resize(n_p_pad);
desired_v0.resize(n_p_pad);
v_max.resize(n_p_pad);
destination_x.resize(n_p_pad);
destination_y.resize(n_p_pad);
color.resize(n_p_pad);
int n_p_half = n_p / 2;
for (int i = 0; i < n_p_half; i++) {
float x = x_gen_left(gen);
float y = y_gen(gen);
float dest_x = BOARD_LENGTH + 30.0;
float dest_y = y;
float init_e_x = dest_x - x;
float init_e_y = dest_y - y;
float init_e_norm = sqrt(init_e_x * init_e_x + init_e_y * init_e_y);
float init_e_unit_x = init_e_x / init_e_norm;
float init_e_unit_y = init_e_y / init_e_norm;
float d_v0;
do {
d_v0 = velocity_gen(gen);
} while (d_v0 <= 0.1);
float v_m = 1.3 * d_v0;
color[i] = 0;
pos_x[i] = x;
pos_y[i] = y;
e_x[i] = init_e_unit_x;
e_y[i] = init_e_unit_y;
v_max[i] = v_m;
desired_v0[i] = d_v0;
destination_x[i] = dest_x;
destination_y[i] = dest_y;
w_x[i] = 0.0f;
w_y[i] = 0.0f;
curr_vel_x[i] = 0.0f;
curr_vel_y[i] = 0.0f;
sb_sq[i] = 0.0f;
sb_e_x[i] = 0.0f;
sb_e_y[i] = 0.0f;
}
for (int i = n_p_half; i < n_p; i++) {
float x = x_gen_right(gen);
float y = y_gen(gen);
float dest_x = -30;
float dest_y = y;
float init_e_x = dest_x - x;
float init_e_y = dest_y - y;
float init_e_norm = sqrt(init_e_x * init_e_x + init_e_y * init_e_y);
float init_e_unit_x = init_e_x / init_e_norm;
float init_e_unit_y = init_e_y / init_e_norm;
float d_v0;
do {
d_v0 = velocity_gen(gen);
} while (d_v0 <= 0.1);
float v_m = 1.3 * d_v0;
color[i] = 1;
pos_x[i] = x;
pos_y[i] = y;
e_x[i] = init_e_unit_x;
e_y[i] = init_e_unit_y;
v_max[i] = v_m;
desired_v0[i] = d_v0;
destination_x[i] = dest_x;
destination_y[i] = dest_y;
w_x[i] = 0.0f;
w_y[i] = 0.0f;
curr_vel_x[i] = 0.0f;
curr_vel_y[i] = 0.0f;
sb_sq[i] = 0.0f;
sb_e_x[i] = 0.0f;
sb_e_y[i] = 0.0f;
}
for (int j = n_p; j < n_p_pad; j++) {
pos_x[j] = 1e9f;
pos_y[j] = 1e9f;
sb_sq[j] = 0.0f;
sb_e_x[j] = 0.0f;
sb_e_y[j] = 0.0f;
}
return nullptr;
}
Board init_board_without_obstacles() { return Board(BOARD_LENGTH, BOARD_WIDTH, 0, nullptr); }
// vectorized expf for a __m256 (8 lanes), no libm call.
// same trick as the double polyexp: exp(x) = 2^n * exp(r) where n = round(x / ln2), r = x - n*ln2.
// after range reduce r is in [-ln2/2, ln2/2] ~= [-0.347, 0.347].
// degree-5 taylor is enough for float (worst-case abs error ~1e-5, way under 1cm verify tolerance).
// 2^n is free via the schraudolph bit trick on the ieee 754 exponent slot.
// float layout: sign(1) | exp(8, bias 127) | mantissa(23), so we shift the biased exponent by 23.
static inline __m256 exp256_ps(__m256 x) {
// n = round(x * log2(e))
const __m256 log2e = _mm256_set1_ps(1.4426950408889634f);
__m256 nf = _mm256_round_ps(_mm256_mul_ps(x, log2e),
_MM_FROUND_TO_NEAREST_INT | _MM_FROUND_NO_EXC);
// r = x - n * ln(2)
const __m256 ln2 = _mm256_set1_ps(0.6931471805599453f);
__m256 r = _mm256_fnmadd_ps(nf, ln2, x);
// poly(r) ~ exp(r), degree-5 horner from highest coeff downwards
const __m256 c5 = _mm256_set1_ps(1.0f / 120.0f);
const __m256 c4 = _mm256_set1_ps(1.0f / 24.0f);
const __m256 c3 = _mm256_set1_ps(1.0f / 6.0f);
const __m256 c2 = _mm256_set1_ps(0.5f);
const __m256 c1 = _mm256_set1_ps(1.0f);
const __m256 c0 = _mm256_set1_ps(1.0f);
__m256 p = c5;
p = _mm256_fmadd_ps(p, r, c4);
p = _mm256_fmadd_ps(p, r, c3);
p = _mm256_fmadd_ps(p, r, c2);
p = _mm256_fmadd_ps(p, r, c1);
p = _mm256_fmadd_ps(p, r, c0);
// 2^n via bit trick: float bias 127, exponent slot at bit 23.
// padding lanes (sb_e=0, pos=1e9) overflow this, but the active mask zeroes them out
// before accumulation so the garbage doesnt matter.
__m256i ni = _mm256_cvtps_epi32(nf);
__m256i biased = _mm256_add_epi32(ni, _mm256_set1_epi32(127));
__m256i scaled = _mm256_slli_epi32(biased, 23);
__m256 pow2n = _mm256_castsi256_ps(scaled);
return _mm256_mul_ps(p, pow2n);
}
int run_simulation(Pedestrian *pedestrians, Board &board, int n_p, int n_iter, const char *dump_path) {
(void)pedestrians;
#if BENCHMARK
(void)dump_path;
#endif
#if !BENCHMARK
const int dump_interval = 1;
vector<tuple<int, int, float, float, float, float, int>> samples;
if (dump_path) {
samples.reserve(static_cast<size_t>(n_iter / dump_interval + 1) * n_p);
for (int i = 0; i < n_p; i++) {
samples.emplace_back(0, i, pos_x[i], pos_y[i], e_x[i], e_y[i], color[i]);
}
}
#endif
const int n_p_pad = (n_p + 7) & ~7;
alignas(32) const float lane_offset_arr[8] = {0.0f, 1.0f, 2.0f, 3.0f, 4.0f, 5.0f, 6.0f, 7.0f};
const __m256 lane_off = _mm256_load_ps(lane_offset_arr);
const __m256 v_neg_v0_div_2s = _mm256_set1_ps(NEG_V0_AB_div_TWO_SIGMA);
const __m256 v_neg_2s_inv = _mm256_set1_ps(NEG_TWO_SIGMA_INV);
const __m256 v_cos_phi_sq = _mm256_set1_ps(cos_phi_sq);
const __m256 v_dt_neg = _mm256_set1_ps(dt_neg);
const __m256 v_c_neg_dt = _mm256_set1_ps(c_neg_times_dt);
const __m256 v_zero = _mm256_setzero_ps();
const __m256 v_one = _mm256_set1_ps(1.0f);
const __m256 v_eps = _mm256_set1_ps(EPS);
const __m256 v_eps_sq = _mm256_set1_ps(EPS_SQ);
const __m256 v_np = _mm256_set1_ps((float)n_p);
const __m256 v_exp_min = _mm256_set1_ps(-87.0f); // clamp for polyexp, below this exp(x) ~ 0 anyway
for (int t = 0; t < n_iter; t++) {
for (int i = 0; i < n_p; i++) {
float desired_velocity_x = e_x[i] * desired_v0[i];
float desired_velocity_y = e_y[i] * desired_v0[i];
float f_alpha_dt_x = (desired_velocity_x - curr_vel_x[i]) * dt_times_tau_alpha_inv;
float f_alpha_dt_y = (desired_velocity_y - curr_vel_y[i]) * dt_times_tau_alpha_inv;
const __m256 pix = _mm256_set1_ps(pos_x[i]);
const __m256 piy = _mm256_set1_ps(pos_y[i]);
const __m256 eix = _mm256_set1_ps(e_x[i]);
const __m256 eiy = _mm256_set1_ps(e_y[i]);
const __m256 v_i = _mm256_set1_ps((float)i);
__m256 acc_x = _mm256_setzero_ps();
__m256 acc_y = _mm256_setzero_ps();
for (int j = 0; j < n_p_pad; j += 8) {
__m256 j_idx = _mm256_add_ps(_mm256_set1_ps((float)j), lane_off);
__m256 m_neq_i = _mm256_cmp_ps(j_idx, v_i, _CMP_NEQ_OQ);
__m256 m_in_range = _mm256_cmp_ps(j_idx, v_np, _CMP_LT_OQ);
__m256 active = _mm256_and_ps(m_neq_i, m_in_range);
__m256 pjx = _mm256_loadu_ps(&pos_x[j]);
__m256 pjy = _mm256_loadu_ps(&pos_y[j]);
__m256 sbe_jx = _mm256_loadu_ps(&sb_e_x[j]);
__m256 sbe_jy = _mm256_loadu_ps(&sb_e_y[j]);
__m256 sb_jsq = _mm256_loadu_ps(&sb_sq[j]);
__m256 r_ab_x = _mm256_sub_ps(pix, pjx);
__m256 r_ab_y = _mm256_sub_ps(piy, pjy);
__m256 r_ab_n2 = _mm256_fmadd_ps(r_ab_x, r_ab_x, _mm256_mul_ps(r_ab_y, r_ab_y));
__m256 r_ab_n = _mm256_sqrt_ps(r_ab_n2);
r_ab_n = _mm256_max_ps(r_ab_n, v_eps);
__m256 r_ab2_x = _mm256_sub_ps(r_ab_x, sbe_jx);
__m256 r_ab2_y = _mm256_sub_ps(r_ab_y, sbe_jy);
__m256 r_ab2_n2 = _mm256_fmadd_ps(r_ab2_x, r_ab2_x, _mm256_mul_ps(r_ab2_y, r_ab2_y));
__m256 r_ab2_n = _mm256_sqrt_ps(r_ab2_n2);
r_ab2_n = _mm256_max_ps(r_ab2_n, v_eps);
__m256 r_sum = _mm256_add_ps(r_ab_n, r_ab2_n);
__m256 two_b_sq = _mm256_fmsub_ps(r_sum, r_sum, sb_jsq);
two_b_sq = _mm256_max_ps(two_b_sq, v_eps_sq);
__m256 two_b = _mm256_sqrt_ps(two_b_sq);
// simd polynomial expf, no libm call. replaces the 8-iter scalar loop that gcc refused to unroll.
// clamp e_arg >= -87 first: exp(-87) ~ 1.6e-38 is already basically 0, and the schraudolph 2^n
// bit trick blows up if (nf + 127) goes negative (which happens for padding lanes where e_arg ~ -4.7e9).
__m256 e_arg = _mm256_mul_ps(two_b, v_neg_2s_inv);
e_arg = _mm256_max_ps(e_arg, v_exp_min);
__m256 e_val = exp256_ps(e_arg);
__m256 numer = _mm256_mul_ps(_mm256_mul_ps(r_sum, v_neg_v0_div_2s), e_val);
__m256 denom = _mm256_mul_ps(_mm256_mul_ps(two_b, r_ab_n), r_ab2_n);
__m256 denom_safe = _mm256_blendv_ps(v_one, denom, active);
__m256 smc = _mm256_div_ps(numer, denom_safe);
__m256 sm_r_ab = _mm256_mul_ps(smc, r_ab2_n);
__m256 sm_r_ab2 = _mm256_mul_ps(smc, r_ab_n);
__m256 f_x = _mm256_fmadd_ps(r_ab2_x, sm_r_ab2, _mm256_mul_ps(r_ab_x, sm_r_ab));
__m256 f_y = _mm256_fmadd_ps(r_ab2_y, sm_r_ab2, _mm256_mul_ps(r_ab_y, sm_r_ab));
__m256 f_norm_sq = _mm256_fmadd_ps(f_x, f_x, _mm256_mul_ps(f_y, f_y));
__m256 rhs = _mm256_mul_ps(f_norm_sq, v_cos_phi_sq);
__m256 e_dot_f = _mm256_fmadd_ps(eix, f_x, _mm256_mul_ps(eiy, f_y));
__m256 e_dot_f_sq = _mm256_mul_ps(e_dot_f, e_dot_f);
__m256 c1 = _mm256_cmp_ps(e_dot_f, v_zero, _CMP_LT_OQ);
__m256 c2 = _mm256_cmp_ps(e_dot_f_sq, rhs, _CMP_GT_OQ);
__m256 cond = _mm256_and_ps(c1, c2);
__m256 w_val = _mm256_blendv_ps(v_dt_neg, v_c_neg_dt, cond);
__m256 f_x_safe = _mm256_and_ps(f_x, active);
__m256 f_y_safe = _mm256_and_ps(f_y, active);
acc_x = _mm256_fmadd_ps(f_x_safe, w_val, acc_x);
acc_y = _mm256_fmadd_ps(f_y_safe, w_val, acc_y);
}
__m256 hsum = _mm256_hadd_ps(acc_x, acc_y);
__m128 hsum_lo = _mm256_castps256_ps128(hsum);
__m128 hsum_hi = _mm256_extractf128_ps(hsum, 1);
__m128 sum4 = _mm_add_ps(hsum_lo, hsum_hi);
sum4 = _mm_hadd_ps(sum4, sum4);
alignas(16) float sum_out[4];
_mm_store_ps(sum_out, sum4);
f_alpha_dt_x += sum_out[0];
f_alpha_dt_y += sum_out[1];
for (int k = 0; k < board.n_obstacles; k++) {
float v1_x = pos_x[i] - board.obstacles[k].start_x;
float v1_y = pos_y[i] - board.obstacles[k].start_y;
float v2_x = board.obstacles[k].obstacle_v_x;
float v2_y = board.obstacles[k].obstacle_v_y;
float projection = (v1_x * v2_x + v1_y * v2_y) * board.obstacles[k].length_sq_inv;
if (projection < 0.0)
projection = 0.0;
else if (projection > 1.0)
projection = 1.0;
float r_aB_x = pos_x[i] - (board.obstacles[k].start_x + board.obstacles[k].obstacle_v_x * projection);
float r_aB_y = pos_y[i] - (board.obstacles[k].start_y + board.obstacles[k].obstacle_v_y * projection);
float r_aB_norm = sqrt(r_aB_x * r_aB_x + r_aB_y * r_aB_y);
r_aB_norm = r_aB_norm > EPS ? r_aB_norm : EPS;
float scalar_mult = (NEG_U0_AB_div_R * exp(r_aB_norm * NEG_R_INV)) / r_aB_norm;
float f_alpha_B_neg_x = r_aB_x * scalar_mult;
float f_alpha_B_neg_y = r_aB_y * scalar_mult;
float w_val;
float e_dot_f = (f_alpha_B_neg_y == 0) ? 0 : (f_alpha_B_neg_y < 0 ? -e_y[i] : e_y[i]);
if (e_dot_f >= cos_phi)
w_val = dt_neg;
else
w_val = c_neg_times_dt;
float f_alpha_B_dt_x = f_alpha_B_neg_x * w_val;
float f_alpha_B_dt_y = f_alpha_B_neg_y * w_val;
f_alpha_dt_x += f_alpha_B_dt_x;
f_alpha_dt_y += f_alpha_B_dt_y;
}
w_x[i] += f_alpha_dt_x;
w_y[i] += f_alpha_dt_y;
}
for (int i = 0; i < n_p; i++) {
float w_norm = sqrt(w_x[i] * w_x[i] + w_y[i] * w_y[i]);
float w_norm2 = w_norm > EPS ? w_norm : EPS;
float correction_factor = min(1.0f, v_max[i] / w_norm2);
curr_vel_x[i] = w_x[i] * correction_factor;
curr_vel_y[i] = w_y[i] * correction_factor;
pos_x[i] = pos_x[i] + curr_vel_x[i] * dt;
pos_y[i] = pos_y[i] + curr_vel_y[i] * dt;
if (color[i] == 0 && pos_x[i] > (BOARD_LENGTH - 1.5)) {
pos_x[i] = 1.0;
destination_y[i] = pos_y[i];
w_x[i] = desired_v0[i];
w_y[i] = 0.0;
curr_vel_x[i] = w_x[i];
curr_vel_y[i] = w_y[i];
}
if (color[i] == 1 && pos_x[i] < 1.5) {
pos_x[i] = BOARD_LENGTH - 1.0;
destination_y[i] = pos_y[i];
w_x[i] = -desired_v0[i];
w_y[i] = 0.0;
curr_vel_x[i] = w_x[i];
curr_vel_y[i] = w_y[i];
}
pos_y[i] = clamp(pos_y[i], WALL_EPS, BOARD_WIDTH_MINUS_WALL_EPS);
float ex = destination_x[i] - pos_x[i];
float ey = destination_y[i] - pos_y[i];
float e_norm = sqrt(ex * ex + ey * ey);
float e_norm2 = e_norm > EPS ? e_norm : EPS;
float e_norm_inv = 1.0 / e_norm2;
e_x[i] = ex * e_norm_inv;
e_y[i] = ey * e_norm_inv;
float velocity_norm_sq = curr_vel_x[i] * curr_vel_x[i] + curr_vel_y[i] * curr_vel_y[i];
float velocity_norm = sqrt(velocity_norm_sq);
sb_sq[i] = velocity_norm_sq * delta_t_sq;
sb_e_x[i] = e_x[i] * (velocity_norm * delta_t);
sb_e_y[i] = e_y[i] * (velocity_norm * delta_t);
#if !BENCHMARK
if (dump_path && (t % dump_interval == 0)) {
samples.emplace_back(t + 1, i, pos_x[i], pos_y[i], e_x[i], e_y[i], color[i]);
}
#endif
}
}
#if !BENCHMARK
if (dump_path) {
ofstream out(dump_path);
out << "time,id,x,y,p.e.x,p.e.y,color\n";
for (auto &[t, id, x, y, e_x, e_y, c] : samples) {
out << t << ',' << id << ',' << x << ',' << y << ',' << e_x << ',' << e_y << ',' << c << '\n';
}
}
#endif
return 0;
}