A note on the numerical solutions of kernel-based learning problems

In the last decade, kernel-based learning approaches typically employed for classification and regression have shown outstanding performance also in dynamic system identification. The typical way to compute the solution of this learning problem subsumes the inversion of the kernel matrix. However, due to limited machine precision, this might not be possible in many practical applications. In this work, we analyze the above problem and show that the typical estimate is just one of the possible infinite solutions that can be leveraged, considering both the supervised and the semi-supervised settings. We show under which conditions the infinite solutions are equivalent and, if it is not the case, we provide a bound on the mismatch between two generic solutions. Then, we propose two specific solutions that are particularly suited to boost sparsity or performance.