矩阵特殊运算

【注】对于相同的特殊运算不同书籍可能使用的符号不同,本文采用与 Wikipedia 相同的符号表示。

1. 直和

1.1 定义

m×mm \times m 矩阵 A\boldsymbol{A}n×nn \times n 矩阵 B\boldsymbol{B}直和记作 AB \boldsymbol{A} \oplus \boldsymbol{B} ,它是一个 (m+n)×(m+n)(m+n) \times (m+n) 的矩阵,定义为

AB=[A0m×n0n×mB]\begin{array}{c} \boldsymbol{A} \oplus \boldsymbol{B} = \left[ \begin{matrix} \boldsymbol{A} & \boldsymbol{0}_{m \times n} \\ \boldsymbol{0}_{n \times m} & \boldsymbol{B} \end{matrix} \right] \end{array}

1.2 性质

  • cc 为常数,则 c(AB)=cAcB c(\boldsymbol{A} \oplus \boldsymbol{B}) = c\boldsymbol{A} \oplus c\boldsymbol{B}

  • 直和通常不满足交换性质,AB=BA \boldsymbol{A} \oplus \boldsymbol{B} = \boldsymbol{B} \oplus \boldsymbol{A} 当且仅当 A=B \boldsymbol{A} = \boldsymbol{B}

  • A,B\boldsymbol{A},\boldsymbol{B}m×mm \times m 矩阵,且 C,D\boldsymbol{C},\boldsymbol{D}n×nn \times n 矩阵,则

(A±B)(C±D)=(AC)±(BD)(AC)(BD)=ABCD\begin{array}{c} (\boldsymbol{A} \pm \boldsymbol{B}) \oplus (\boldsymbol{C} \pm \boldsymbol{D}) = (\boldsymbol{A} \oplus \boldsymbol{C}) \pm (\boldsymbol{B} \oplus \boldsymbol{D}) \\ (\boldsymbol{A} \oplus \boldsymbol{C})(\boldsymbol{B} \oplus \boldsymbol{D}) = \boldsymbol{AB} \oplus \boldsymbol{CD} \end{array}

  • A,B,C\boldsymbol{A},\boldsymbol{B},\boldsymbol{C} 分别是 m×m,n×n,p×pm \times m, n \times n, p \times p 矩阵,则

A(BC)=(AB)C=ABC\begin{array}{c} \boldsymbol{A} \oplus (\boldsymbol{B} \oplus \boldsymbol{C}) = (\boldsymbol{A} \oplus \boldsymbol{B}) \oplus \boldsymbol{C} = \boldsymbol{A} \oplus \boldsymbol{B} \oplus \boldsymbol{C} \end{array}

  • Am×mA_{m \times m}Bn×nB_{n \times n} 均为正交矩阵,则 AB \boldsymbol{A} \oplus \boldsymbol{B} (m+n)×(m+n)(m+n) \times (m+n) 正交矩阵。

  • 矩阵直和的复共轭、转置、复共轭转置与逆矩阵(若 A,B\boldsymbol{A}, \boldsymbol{B} 可逆)的关系:

(AB)=AB(AB)T=ATBT(AB)H=AHBH(AB)1=A1B1\begin{array}{c} (\boldsymbol{A} \oplus \boldsymbol{B})^* = \boldsymbol{A}^* \oplus \boldsymbol{B}^* \\ (\boldsymbol{A} \oplus \boldsymbol{B})^T = \boldsymbol{A}^T \oplus \boldsymbol{B}^T \\ (\boldsymbol{A} \oplus \boldsymbol{B})^H = \boldsymbol{A}^H \oplus \boldsymbol{B}^H \\ (\boldsymbol{A} \oplus \boldsymbol{B})^{-1} = \boldsymbol{A}^{-1} \oplus \boldsymbol{B}^{-1} \end{array}

2. Hadamard 积

2.1 定义

m×nm \times n 矩阵 A=[aij]\boldsymbol{A} = [a_{ij}]m×nm \times n 矩阵 B=[bij]\boldsymbol{B} = [b_{ij}]Hadamard 积(也称为 Schur 积对应元素乘积)记作 AB \boldsymbol{A} \odot \boldsymbol{B} (或 AB\boldsymbol{A} \circ \boldsymbol{B}),它仍然是一个 m×nm \times n 矩阵,其元素定义为两个矩阵对应元素的乘积:

(AB)ij=aijbij\begin{array}{c} (\boldsymbol{A} \odot \boldsymbol{B})_{ij} = a_{ij}b_{ij} \end{array}

即 Hadamard 积是一映射 Rm×n×Rm×nRm×n \mathbb{R}^{m \times n} \times \mathbb{R}^{m \times n} \mapsto \mathbb{R}^{m \times n}

2.2 性质

2.2.1 基本性质

  • AB=BAA(BC)=(AB)CA(B±C)=AB±AC \boldsymbol{A} \odot \boldsymbol{B} = \boldsymbol{B} \odot \boldsymbol{A} \\ \boldsymbol{A} \odot (\boldsymbol{B} \odot \boldsymbol{C}) = (\boldsymbol{A} \odot \boldsymbol{B}) \odot \boldsymbol{C} \\ \boldsymbol{A} \odot (\boldsymbol{B} \pm \boldsymbol{C}) = \boldsymbol{A} \odot \boldsymbol{B} \pm \boldsymbol{A} \odot \boldsymbol{C}

  • A,B\boldsymbol{A},\boldsymbol{B} 均为 m×nm \times n 矩阵,则

(AB)T=ATBT(AB)H=AHBH(AB)=AB\begin{array}{c} (\boldsymbol{A} \odot \boldsymbol{B})^T = \boldsymbol{A}^T \odot \boldsymbol{B}^T \\ (\boldsymbol{A} \odot \boldsymbol{B})^H = \boldsymbol{A}^H \odot \boldsymbol{B}^H \\ (\boldsymbol{A} \odot \boldsymbol{B})^* = \boldsymbol{A}^* \odot \boldsymbol{B}^* \end{array}

  • A0=0A=0 \boldsymbol{A} \odot \boldsymbol{0} = \boldsymbol{0} \odot \boldsymbol{A} = \boldsymbol{0}

  • cc 为常数,则 c(AB)=(cA)B c(\boldsymbol{A} \odot \boldsymbol{B}) = (c\boldsymbol{A}) \odot \boldsymbol{B}

  • 正定(半正定)矩阵 A,B\boldsymbol{A},\boldsymbol{B} 的 Hadamard 积 AB\boldsymbol{A} \odot \boldsymbol{B} 也是正定(半正定)的。

2.2.2 其他性质

  • A,C\boldsymbol{A},\boldsymbol{C}m×mm \times m 矩阵,B,D\boldsymbol{B}, \boldsymbol{D}n×nn \times n 矩阵,则

(AB)(CD)=(AC)(BD)\begin{array}{c} (\boldsymbol{A} \oplus \boldsymbol{B}) \odot (\boldsymbol{C} \oplus \boldsymbol{D}) = (\boldsymbol{A} \odot \boldsymbol{C}) \oplus (\boldsymbol{B} \odot \boldsymbol{D}) \end{array}

  • A,B,C,D\boldsymbol{A}, \boldsymbol{B}, \boldsymbol{C}, \boldsymbol{D} 均为 m×nm \times n 矩阵,则

(A+B)(C+D)=AC+AD+BC+BD\begin{array}{c} (\boldsymbol{A} + \boldsymbol{B}) \odot (\boldsymbol{C} + \boldsymbol{D}) = \boldsymbol{A} \odot \boldsymbol{C} + \boldsymbol{A} \odot \boldsymbol{D} + \boldsymbol{B} \odot \boldsymbol{C} + \boldsymbol{B} \odot \boldsymbol{D} \end{array}

  • A,B,C\boldsymbol{A}, \boldsymbol{B}, \boldsymbol{C} 均为 m×nm \times n 矩阵,则

tr(AT(BC))=tr((ATBT)C)\begin{array}{c} \mathrm{tr}(\boldsymbol{A}^T(\boldsymbol{B} \odot \boldsymbol{C})) = tr((\boldsymbol{A}^T \odot \boldsymbol{B}^T) \boldsymbol{C} ) \end{array}

2.2.3 不等式性质

  • Oppenheim 不等式:A,B\boldsymbol{A}, \boldsymbol{B}n×nn \times n 半正定矩阵,则

ABa11annB\begin{array}{c} |\boldsymbol{A} \odot \boldsymbol{B}| \geq a_{11} \cdots a_{nn} |\boldsymbol{B}| \end{array}

  • ABAB|\boldsymbol{A} \odot \boldsymbol{B}| \geq |\boldsymbol{AB}|

  • 特征值不等式:A,B\boldsymbol{A}, \boldsymbol{B}n×nn \times n 半正定矩阵,λ1,,λn\lambda_1, \cdots, \lambda_nAB\boldsymbol{A} \odot \boldsymbol{B} 的特征值,λ1^,,λn^\hat{\lambda_1}, \cdots, \hat{\lambda_n} 是矩阵乘积 AB\boldsymbol{AB} 的特征值,则

i=knλii=knλi^,  k=1,,n\begin{array}{c} \prod_{i=k}^n \lambda_i \geq \prod_{i=k}^n \hat{\lambda_i}, \ \ k = 1, \cdots, n \end{array}

  • 秩不等式:A,B\boldsymbol{A},\boldsymbol{B}n×nn \times n 矩阵,则

rank(AB)rank(A)rank(B)\begin{array}{c} \mathrm{rank}(\boldsymbol{A} \odot \boldsymbol{B}) \leq \mathrm{rank}(\boldsymbol{A}) \mathrm{rank}(\boldsymbol{B}) \end{array}

3. Kronecker 积(直积/张量积)

3.1 定义

3.1.1 Kronecker 积

两个矩阵的 Kronecker 积(也称为直积/张量积)分为右 Kronecker 积和左 Kronecker 积。

  • 右 Kronecker 积:m×nm \times n 矩阵 A=[a1,,an] \boldsymbol{A} = [\boldsymbol{a}_1, \cdots, \boldsymbol{a}_n] p×qp \times q 矩阵 B\boldsymbol{B} 的右 Kronecker 积记作 AB \boldsymbol{A} \otimes \boldsymbol{B} ,它是一个 mp×nqmp \times nq 矩阵,定义为:

AB=[a1B,,anB]=[aijB]i=1,j=1m,n=[a11Ba12Ba1nBa21Ba22Ba2nBam1Bam2BamnB]\begin{array}{c} \boldsymbol{A} \otimes \boldsymbol{B} = [\boldsymbol{a}_1 \boldsymbol{B}, \cdots, \boldsymbol{a}_n \boldsymbol{B}] = [a_{ij}\boldsymbol{B}]_{i=1,j=1}^{m,n} = \left[ \begin{matrix} a_{11} \boldsymbol{B} & a_{12} \boldsymbol{B} & \cdots & a_{1n} \boldsymbol{B} \\ a_{21} \boldsymbol{B} & a_{22} \boldsymbol{B} & \cdots & a_{2n} \boldsymbol{B} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} \boldsymbol{B} & a_{m2} \boldsymbol{B} & \cdots & a_{mn} \boldsymbol{B} \end{matrix} \right] \end{array}

  • 左 Kronecker 积:m×nm \times n 矩阵 A \boldsymbol{A} p×qp \times q 矩阵 B==[b1,,bq] \boldsymbol{B} = = [\boldsymbol{b}_1, \cdots, \boldsymbol{b}_q] 的左 Kronecker 积记作 [AB]left [\boldsymbol{A} \otimes \boldsymbol{B}]_\mathrm{left} ,它是一个 mp×nqmp \times nq 矩阵,定义为:

[AB]left=[Ab1,,Abq]=[bijB]i=1,j=1p,q=[Ab11Ab12Ab1qAb21Ab22Ab2qAbp1Abp2Abpq]\begin{array}{c} [\boldsymbol{A} \otimes \boldsymbol{B}]_\mathrm{left} = [\boldsymbol{A} \boldsymbol{b}_1, \cdots, \boldsymbol{A} \boldsymbol{b}_q] = [b_{ij}\boldsymbol{B}]_{i=1,j=1}^{p,q} = \left[ \begin{matrix} \boldsymbol{A} b_{11} & \boldsymbol{A} b_{12} & \cdots & \boldsymbol{A} b_{1q} \\ \boldsymbol{A} b_{21} & \boldsymbol{A} b_{22} & \cdots & \boldsymbol{A} b_{2q} \\ \vdots & \vdots & \ddots & \vdots \\ \boldsymbol{A} b_{p1} & \boldsymbol{A} b_{p2} & \cdots & \boldsymbol{A} b_{pq} \end{matrix} \right] \end{array}

可以看出,Kronecker 积是一映射:Rm×n×Rm×nRmp×nq \mathbb{R}^{m \times n} \times \mathbb{R}^{m \times n} \mapsto \mathbb{R}^{mp \times nq} ,且左右 Kronecker 积满足:

[AB]left=BA\begin{array}{c} [\boldsymbol{A} \otimes \boldsymbol{B}]_{\mathrm{left}} = \boldsymbol{B} \otimes \boldsymbol{A} \end{array}

一般常用右 Kronecker 积。

3.1.2 广义 Kronecker 积

给定 NNm×rm \times r 矩阵 Ai,i=1,,N\boldsymbol{A}_i, i=1,\cdots,N 组成矩阵组 {A}N\{A\}_N 。该矩阵组与 N×lN \times l 矩阵 BB 的 Kronecker 积称为广义 Kronecker 积,定义为:

{A}NB=[A1b1ANbN]\begin{array}{c} \{\boldsymbol{A}\}_N \otimes \boldsymbol{B} = \left[ \begin{matrix} \boldsymbol{A}_1 \otimes \boldsymbol{b}_1 \\ \vdots \\ \boldsymbol{A}_N \otimes \boldsymbol{b}_N \end{matrix} \right] \end{array}

其中,bi\boldsymbol{b}_i 是矩阵 B\boldsymbol{B} 的第 ii 个行向量。

3.2 性质

3.2.1 基本性质

  • 不满足交换律:对矩阵 Am×n\boldsymbol{A}_{m \times n}Bp×q\boldsymbol{B}_{p \times q},一般有 ABBA \boldsymbol{A} \otimes \boldsymbol{B} \ne \boldsymbol{B} \otimes \boldsymbol{A}

  • A0=0A=0 \boldsymbol{A} \otimes \boldsymbol{0} = \boldsymbol{0} \otimes \boldsymbol{A} = \boldsymbol{0}

  • a,ba,b 为常数,则 aAbB=ab(AB) a\boldsymbol{A} \otimes b\boldsymbol{B} = ab(\boldsymbol{A} \otimes \boldsymbol{B})

  • ImIn=Imn \boldsymbol{I}_m \otimes \boldsymbol{I}_n = \boldsymbol{I}_{mn}

  • 对于矩阵 Am×n,Bn×k,Cl×p,Dp×q\boldsymbol{A}_{m \times n}, \boldsymbol{B}_{n \times k}, \boldsymbol{C}_{l \times p}, \boldsymbol{D}_{p \times q},有

(AB)(CD)=(AC)(BD)\begin{array}{c} (\boldsymbol{AB}) \otimes (\boldsymbol{CD}) = (\boldsymbol{A} \otimes \boldsymbol{C})(\boldsymbol{B} \otimes \boldsymbol{D}) \end{array}

  • 对于矩阵 Am×n,Bp×q,Cp×q\boldsymbol{A}_{m \times n}, \boldsymbol{B}_{p \times q}, \boldsymbol{C}_{p \times q},有

A(B±C)=AB±AC(B±C)A=BA±CA\begin{array}{c} \boldsymbol{A} \otimes (\boldsymbol{B} \pm \boldsymbol{C}) = \boldsymbol{A} \otimes \boldsymbol{B} \pm \boldsymbol{A} \otimes \boldsymbol{C} \\ (\boldsymbol{B} \pm \boldsymbol{C}) \otimes \boldsymbol{A} = \boldsymbol{B} \otimes \boldsymbol{A} \pm \boldsymbol{C} \otimes \boldsymbol{A} \end{array}

  • 对于矩阵 Am×n,Bm×n,Cp×q,Dp×q\boldsymbol{A}_{m \times n}, \boldsymbol{B}_{m \times n}, \boldsymbol{C}_{p \times q}, \boldsymbol{D}_{p \times q},有

(A+B)(C+D)=AC+AD+BC+BD\begin{array}{c} (\boldsymbol{A} + \boldsymbol{B}) \otimes (\boldsymbol{C} + \boldsymbol{D}) = \boldsymbol{A} \otimes \boldsymbol{C} + \boldsymbol{A} \otimes \boldsymbol{D} + \boldsymbol{B} \otimes \boldsymbol{C} + \boldsymbol{B} \otimes \boldsymbol{D} \end{array}

  • 对于矩阵 Am×n,Bp×q,Ck×l\boldsymbol{A}_{m \times n}, \boldsymbol{B}_{p \times q}, \boldsymbol{C}_{k \times l},有

(AB)C=A(BC)=ABC\begin{array}{c} (\boldsymbol{A} \otimes \boldsymbol{B}) \otimes \boldsymbol{C} = \boldsymbol{A} \otimes (\boldsymbol{B} \otimes \boldsymbol{C}) = \boldsymbol{A} \otimes \boldsymbol{B} \otimes \boldsymbol{C} \end{array}

  • Kronecker 积的转置与复共轭转置

(AB)T=ATBT(AB)H=AHBH\begin{array}{c} (\boldsymbol{A} \otimes \boldsymbol{B})^T = \boldsymbol{A}^T \otimes \boldsymbol{B}^T \\ (\boldsymbol{A} \otimes \boldsymbol{B})^H = \boldsymbol{A}^H \otimes \boldsymbol{B}^H \end{array}

3.2.2 其他性质

  • Kronecker 积的秩

rank(AB)=rank(A)rank(B)\begin{array}{c} \mathrm{rank}(\boldsymbol{A} \otimes \boldsymbol{B}) = \mathrm{rank}(\boldsymbol{A}) \mathrm{rank}(\boldsymbol{B}) \end{array}

  • Kronecker 积的行列式

det(An×nBm×m)=(detA)m(detB)n\begin{array}{c} \mathrm{det}(\boldsymbol{A}_{n \times n} \otimes \boldsymbol{B}_{m \times m}) = (\mathrm{det} \boldsymbol{A})^m (\mathrm{det} \boldsymbol{B})^n \end{array}

  • Kronecker 积的迹

tr(AB)=tr(A)tr(B)\begin{array}{c} \mathrm{tr}(\boldsymbol{A} \otimes \boldsymbol{B}) = \mathrm{tr}(\boldsymbol{A}) \mathrm{tr}(\boldsymbol{B}) \end{array}

4. Khatri-Rao 积

4.1 定义

两个具有相同列数的矩阵 GRp×n \boldsymbol{G} \in \mathbb{R}^{p \times n} FRq×n \boldsymbol{F} \in \mathbb{R}^{q \times n} 的 Khatri-Rao 积记为 FG \boldsymbol{F} * \boldsymbol{G} ,它是一个 pq×npq \times n 矩阵,定义为:

FG=[f1g1,,fngn]Rpq×n\begin{array}{c} \boldsymbol{F} * \boldsymbol{G} = [\boldsymbol{f}_1 \otimes \boldsymbol{g}_1 , \cdots, \boldsymbol{f}_n \otimes \boldsymbol{g}_n] \in \mathbb{R}^{pq \times n} \end{array}

它由两个矩阵对应列向量的 Kronecker 积排列而成(fi\boldsymbol{f}_igi\boldsymbol{g}_i 分别表示 F\boldsymbol{F}G\boldsymbol{G} 的列向量),因此 Khatri-Rao 积又称为对应列 Kronecker 积

4.2 性质

4.2.1 基本性质

  • 分配律:(A+B)D=AD+BD (\boldsymbol{A} + \boldsymbol{B}) * \boldsymbol{D} = \boldsymbol{A} * \boldsymbol{D} + \boldsymbol{B} * \boldsymbol{D}

  • 结合律:ABC=(AB)C=A(BC) \boldsymbol{A} * \boldsymbol{B} * \boldsymbol{C} = (\boldsymbol{A} * \boldsymbol{B}) * \boldsymbol{C} = \boldsymbol{A} * (\boldsymbol{B} * \boldsymbol{C})

  • 交换律:AB=Knn(BA) \boldsymbol{A} * \boldsymbol{B} = \boldsymbol{K}_{nn} (\boldsymbol{B} * \boldsymbol{A})

4.2.2 Khatri-Rao 积与 Hadamard 积的关系

  • (AB)(CD)=(AC)(BD) (\boldsymbol{A} * \boldsymbol{B}) \odot (\boldsymbol{C} * \boldsymbol{D}) = (\boldsymbol{A} \odot \boldsymbol{C}) * (\boldsymbol{B} \odot \boldsymbol{D})

  • (AB)T(AB)=(ATA)(BTB) (\boldsymbol{A} * \boldsymbol{B})^T(\boldsymbol{A} * \boldsymbol{B}) = (\boldsymbol{A}^T \boldsymbol{A}) \odot (\boldsymbol{B}^T \boldsymbol{B})

  • (AB)=[(ATA)(BTB)](AB)T (\boldsymbol{A} * \boldsymbol{B})^{\dagger} = [(\boldsymbol{A}^T \boldsymbol{A}) \odot (\boldsymbol{B}^T \boldsymbol{B})]^{\dagger} (\boldsymbol{A} * \boldsymbol{B})^T

4.2.3 Khatri-Rao 积与 Kronecker 积的关系

  • (AB)(FG)=AFBG (\boldsymbol{A} \otimes \boldsymbol{B})(\boldsymbol{F} * \boldsymbol{G}) = \boldsymbol{AF} * \boldsymbol{BG}

5. Face-Splitting 积

5.1 定义

两个具有相同行数的矩阵 GRn×p \boldsymbol{G} \in \mathbb{R}^{n \times p} FRn×q \boldsymbol{F} \in \mathbb{R}^{n \times q} 的 Face-Splitting 积记为 FG \boldsymbol{F} * \boldsymbol{G} ,它是一个 n×pqn \times pq 矩阵,定义为:

FG=[f1g1fngn]Rn×pq\begin{array}{c} \boldsymbol{F} \bullet \boldsymbol{G} = \left[ \begin{matrix} \boldsymbol{f}_1 \otimes \boldsymbol{g}_1 \\ \vdots \\ \boldsymbol{f}_n \otimes \boldsymbol{g}_n \end{matrix} \right] \in \mathbb{R}^{n \times pq} \end{array}

它由两个矩阵对应行向量的 Kronecker 积排列而成(fi\boldsymbol{f}_igi\boldsymbol{g}_i 分别表示 F\boldsymbol{F}G\boldsymbol{G} 的行向量),因此 Face-Splitting 积又称为对应行 Kronecker 积

5.2 性质

  • (AB)T=ATBT(\boldsymbol{A} \bullet \boldsymbol{B})^T = \boldsymbol{A}^T * \boldsymbol{B}^T

其余性质和 Khatri-Rao 类似。

附录

  • 《矩阵分析与应用(第二版)》by 张贤达