講到iterative method，得從最簡單的方法開始─Jacobi method
假設有一方程組
\[Ax=b\]
這邊我們舉例
\[\begin{bmatrix}{A}_{11}&{A}_{12}&{A}_{13}\\{A}_{21}&{A}_{22}&{A}_{23}\\{A}_{31}&{A}_{32}&{A}_{33}\end{bmatrix}\begin{bmatrix}{x}_{1}\\{x}_{2}\\{x}_{3}\end{bmatrix}=\begin{bmatrix}{b}_{1}\\{b}_{2}\\{b}_{3}\end{bmatrix}\]
展開上式得到
\({A_{11}}{x_1}+{A_{12}}{x_2}+{A_{13}}{x_3}=b_{1}\)
\({A_{21}}{x_1}+{A_{22}}{x_2}+{A_{23}}{x_3}=b_{2}\)
\({A_{31}}{x_1}+{A_{32}}{x_2}+{A_{33}}{x_3}=b_{3}\)
依序在等號左邊整理出\(x_1, x_2, x_3\)
\({x_{1}}=\frac{1}{{A_{11}}}\left(b_{1}-{A_{12}}{x_{2}}-{A_{13}}{x_{3}}\right)\)
\({x_{2}}=\frac{1}{{A_{22}}}\left(b_{2}-{A_{21}}{x_{1}}-{A_{23}}{x_{3}}\right)\)
\({x_{3}}=\frac{1}{{A_{33}}}\left(b_{3}-{A_{31}}{x_{1}}-{A_{32}}{x_{2}}\right)\)
這邊我們需要一個initial guess \(x^{(0)}\)，令等號右邊的\(x^{(n)}\)為已知，則可求出等號左邊新的\(x^{(n+1)}\)，並且在某些情況下迭代會收斂
\({x_{1}^{(n+1)}}=\frac{1}{{A_{11}}}\left(b_{1}-{A_{12}}{x_{2}^{(n)}}-{A_{13}}{x_{3}^{(n)}}\right)\)
\({x_{2}^{(n+1)}}=\frac{1}{{A_{22}}}\left(b_{2}-{A_{21}}{x_{1}^{(n)}}-{A_{23}}{x_{3}^{(n)}}\right)\)
\({x_{3}^{(n+1)}}=\frac{1}{{A_{33}}}\left(b_{3}-{A_{31}}{x_{1}^{(n)}}-{A_{32}}{x_{2}^{(n)}}\right)\)
到此就完成整個Jacobi method，將其整理成matrix form
\[x^{(n+1)}=D^{-1}\left(b-Rx^{(n)}\right)\]
其中D為diagonal matrix，其inverse為主對角線元素之倒數
\[D=\begin{bmatrix} { A }_{ 11 } & 0 & 0 \\ 0 & { A }_{ 22 } & 0 \\ 0 & 0 & { A }_{ 33 } \end{bmatrix},{D}^{-1}=\begin{bmatrix}\frac{1}{{A}_{11}}&0&0\\0&\frac{1}{{A}_{22}}&0\\0&0&\frac{1}{{A}_{33}}\end{bmatrix}\]
\[R=A-D\]
眼尖的讀者一定看出這式子可以展開做預乘加速，將式子展開
\[x^{(n+1)}=D^{-1}b-D^{-1}Rx^{(n)}\]
令\(C=D^{-1}b\)與\(T=-D^{-1}R\)，預先做乘法存起來
可以將迭代式子改為下式，計算時僅須一次矩陣乘法，雖然Jacobi方法實做上幾乎很少被使用…
\[x^{(n+1)}=Tx^{(n)}+C\]