使用Armadillo和OpenBLAS进行多线程时性能不一致

Question

使用Armadillo我写了一个矩阵向量乘法和一个线性系统求解。犰狳是从源代码编译并使用OpenBLAS，也从源代码编译。不幸的是，我得到了单线程和多线程运行的不一致结果。矩阵向量乘法在单线程上运行得更快，而线程系统求解在多线程时运行得更快。我希望如果有人能够给我一些关于我做错了什么的指示。

见下文：

• 源码
• 编译&运行bash脚本
• 结果
• 系统信息

matmul_armadillo.cpp

#include <armadillo> 

using namespace arma; 

int main(int argc, char *argv[]) 
{ 
    const int n = atoi(argv[1]); 

    mat A = randu<mat>(n, n); 
    vec x = randu<vec>(n); 

    A*x; 

    return 0; 
} 

solve_armadillo.cpp

#include <armadillo> 

using namespace arma; 

int main(int argc, char *argv[]) 
{ 
    const int n = atoi(argv[1]); 

    mat A = randu<mat>(n, n); 
    vec b = randu<vec>(n); 
    vec x; 

    x = solve(A, b); 

    return 0; 
} 

benchmark.sh

#!/bin/bash 

g++ matmul_armadillo.cpp -o matmul_armadillo -O3 -march=native -std=c++11 -larmadillo 
g++ solve_armadillo.cpp -o solve_armadillo -O3 -march=native -std=c++11 -larmadillo 

N=7500 

export OPENBLAS_NUM_THREADS=1 
echo 'Running matmul_armadillo on' $OPENBLAS_NUM_THREADS 'threads' 
time ./matmul_armadillo $N 
echo '' 
echo 'Running solve_armadillo on' $OPENBLAS_NUM_THREADS 'threads' 
time ./solve_armadillo $N 
echo '' 

export OPENBLAS_NUM_THREADS=2 
echo 'Running matmul_armadillo on' $OPENBLAS_NUM_THREADS 'threads' 
time ./matmul_armadillo $N 
echo '' 
echo 'Running solve_armadillo on' $OPENBLAS_NUM_THREADS 'threads' 
time ./solve_armadillo $N 
echo '' 

export OPENBLAS_NUM_THREADS=3 
echo 'Running matmul_armadillo on' $OPENBLAS_NUM_THREADS 'threads' 
time ./matmul_armadillo $N 
echo '' 
echo 'Running solve_armadillo on' $OPENBLAS_NUM_THREADS 'threads' 
time ./solve_armadillo $N 
echo '' 

export OPENBLAS_NUM_THREADS=4 
echo 'Running matmul_armadillo on' $OPENBLAS_NUM_THREADS 'threads' 
time ./matmul_armadillo $N 
echo '' 
echo 'Running solve_armadillo on' $OPENBLAS_NUM_THREADS 'threads' 
time ./solve_armadillo $N 
echo '' 

export OPENBLAS_NUM_THREADS=5 
echo 'Running matmul_armadillo on' $OPENBLAS_NUM_THREADS 'threads' 
time ./matmul_armadillo $N 
echo '' 
echo 'Running solve_armadillo on' $OPENBLAS_NUM_THREADS 'threads' 
time ./solve_armadillo $N 
echo '' 

export OPENBLAS_NUM_THREADS=6 
echo 'Running matmul_armadillo on' $OPENBLAS_NUM_THREADS 'threads' 
time ./matmul_armadillo $N 
echo '' 
echo 'Running solve_armadillo on' $OPENBLAS_NUM_THREADS 'threads' 
time ./solve_armadillo $N 
echo '' 

export OPENBLAS_NUM_THREADS=7 
echo 'Running matmul_armadillo on' $OPENBLAS_NUM_THREADS 'threads' 
time ./matmul_armadillo $N 
echo '' 
echo 'Running solve_armadillo on' $OPENBLAS_NUM_THREADS 'threads' 
time ./solve_armadillo $N 
echo '' 

export OPENBLAS_NUM_THREADS=8 
echo 'Running matmul_armadillo on' $OPENBLAS_NUM_THREADS 'threads' 
time ./matmul_armadillo $N 
echo '' 
echo 'Running solve_armadillo on' $OPENBLAS_NUM_THREADS 'threads' 
time ./solve_armadillo $N 

结果

$ ./benchmark.sh 
Running matmul_armadillo on 1 threads 

real 0m0.943s 
user 0m0.628s 
sys  0m0.159s 

Running solve_armadillo on 1 threads 

real 0m13.910s 
user 0m13.553s 
sys  0m0.300s 

Running matmul_armadillo on 2 threads 

real 0m1.528s 
user 0m1.361s 
sys  0m0.402s 

Running solve_armadillo on 2 threads 

real 0m15.815s 
user 0m29.097s 
sys  0m1.083s 

Running matmul_armadillo on 3 threads 

real 0m1.534s 
user 0m1.480s 
sys  0m0.533s 

Running solve_armadillo on 3 threads 

real 0m11.729s 
user 0m31.022s 
sys  0m1.290s 

Running matmul_armadillo on 4 threads 

real 0m1.543s 
user 0m1.619s 
sys  0m0.674s 

Running solve_armadillo on 4 threads 

real 0m10.013s 
user 0m34.055s 
sys  0m1.696s 

Running matmul_armadillo on 5 threads 

real 0m1.545s 
user 0m1.620s 
sys  0m0.664s 

Running solve_armadillo on 5 threads 

real 0m9.945s 
user 0m33.803s 
sys  0m1.669s 

Running matmul_armadillo on 6 threads 

real 0m1.543s 
user 0m1.607s 
sys  0m0.684s 

Running solve_armadillo on 6 threads 

real 0m10.069s 
user 0m34.283s 
sys  0m1.699s 

Running matmul_armadillo on 7 threads 

real 0m1.542s 
user 0m1.622s 
sys  0m0.661s 

Running solve_armadillo on 7 threads 

real 0m10.041s 
user 0m34.154s 
sys  0m1.704s 

Running matmul_armadillo on 8 threads 

real 0m1.546s 
user 0m1.576s 
sys  0m0.712s 

Running solve_armadillo on 8 threads 

real 0m10.123s 
user 0m34.492s 
sys  0m1.697s 

系统信息

• 的openSUSE 13.1 64位
• 犰狳4.100.2（从源代码编译）
• OpenBLAS 0.2.8（从源代码编译）

Answer 1

我怀疑

A*x; 

可能因为你已经被优化掉不对结果做任何事情。 Armadillo乘法运算的延迟评估模板魔术很容易导致计算的Lapack例程永远不会被调用。所以如果你启用了线程，你只能测量设置的开销。因此，您的程序在禁用线程的情况下执行得更快。

随着

x = solve(A, b); 

它是不同的，因为这会导致相当直接到相应的LAPACK呼叫，这可能不能被优化掉，因为编译器不能排除副作用，你居然把结果赋给变量。 solve称此类大型矩阵具有多处理优势。

为了解决您的基准你应该做两两件事：

• 。利用计算的结果，从做太多
• 重复计算多次停止优化，以获得更好的统计数据，并减少的初始设置的影响成本

下面是一个未经测试的例子：

#include <iostream> 
#include <armadillo> 

using namespace arma; 

int main(int argc, char *argv[]) 
{ 
    const int n = atoi(argv[1]); 

    mat A = randu<mat>(n, n); 
    vec x = randu<vec>(n); 

    for (int i = 0; i < 100; ++i) { 
     x = A*x; 
    } 
    x.print(std::cout); 

    return 0; 
} 

可能不需要呼叫print。