我试图实现softmax函数(Softmax的雅可比矩阵)的导数矩阵。在Python中从头开始计算雅可比矩阵
我数学知道使用SoftMax(XI)的衍生物相对于X 1是:
其中红色增量是Kronecker符号。
到目前为止,我所实行的是:
def softmax_grad(s):
# input s is softmax value of the original input x. Its shape is (1,n)
# e.i. s = np.array([0.3,0.7]), x = np.array([0,1])
# make the matrix whose size is n^2.
jacobian_m = np.diag(s)
for i in range(len(jacobian_m)):
for j in range(len(jacobian_m)):
if i == j:
jacobian_m[i][j] = s[i] * (1-s[i])
else:
jacobian_m[i][j] = -s[i]*s[j]
return jacobian_m
当我测试:
In [95]: x
Out[95]: array([1, 2])
In [96]: softmax(x)
Out[96]: array([ 0.26894142, 0.73105858])
In [97]: softmax_grad(softmax(x))
Out[97]:
array([[ 0.19661193, -0.19661193],
[-0.19661193, 0.19661193]])
你们如何实现雅可比?我想知道是否有更好的方法来做到这一点。任何参考网站/教程也将不胜感激。