Generalized inverse of a symmetric matrix
I have always found the common definition of the generalized inverse of a matrix quite unsatisfactory, because it is usually defined by a mere property, $A A^{} A = A$, which does not really give intuition on when such a matrix exists or on how it can be constructed, etc… But recently, I came across a much more satisfactory definition for the case of symmetric (or more general, normal) matrices.
As is well known, any symmetric matrix $A$ is diagonalizable,
where $D$ is a diagonal matrix with the eigenvalues of $A$ on its diagonal, and $Q$ is an orthogonal matrix with eigenvectors of $A$ as its columns (which magically form an orthogonal set , just kidding, absolutely no magic involved).
The Definition
Assume that $A$ is a real symmetric matrix of size $n\times n$ and has rank $k \leq n$. Denoting the $k$ nonzero eigenvalues of $A$ by $\lambda_1, \dots, \lambda_k$ and the corresponding $k$ columns of $Q$ by $q_1, \dots, q_k$, we have that
We define the generalized inverse of $A$ by
Why this definition makes sense

The common definition/property of generalized inverse still holds:
where we used the fact that $q_i^T q_j = 0$ unless $i = j$ (i.e., orthogonality of $Q$).

By a similar calculation, if $A$ is invertible, then $k = n$ and it holds that

If $A$ is invertible, then $A^{1}$ has eigenvalues $\frac{1}{\lambda_i}$ and eigenvectors $q_i$ (because $A^{1}q_i = \frac{1}{\lambda_i} A^{1} \lambda_i q_i = \frac{1}{\lambda_i} A^{1} A q_i = \frac{1}{\lambda_i} q_i$ for all $i = 1,\dots,n$).
Thus, Definition ($\ref{TheDefinition}$) is simply the diagonalization of $A^{1}$ if $A$ is invertible.

Since $q_1, \dots, q_k$ form an orthonormal basis for the range of A, it follows that the matrix
is the projection operator onto the range of $A$.
But what if A is not symmetric?
Well, then $A$ is not diagonalizable (in general), but instead we can use the singular value decomposition
and define
Easy.
References
Definition $(\ref{TheDefinition})$ is mentioned in passing on page 87 in
 Morris L. Eaton, Multivariate Statistics: A Vector Space Approach. Beachwood, Ohio, USA: Institute of Mathematical Statistics, 2007.