The INVESTIGATION ON LEAST-MEAN-SQUARE ALGORITHM

Abstract

The Non-Negative Least-Mean-Square (NNLMS) algorithm and its variants have been proposed for
online estimation under non-negativity constraints. The transient behavior of the NNLMS, Normalized
NNLMS, Exponential NNLMS and Sign-Sign NNLMS algorithms have been studied in the literature. In
this letter, we derive closed-form expressions for the steady-state excess mean-square error (EMSE) for
the four algorithms. Simulation results illustrate the accuracy of the theoretical results. This work
complements the understanding of the behavior of these algorithms. This algorithm builds on a fixedpoint iteration strategy driven by the Karush–Kuhn–Tucker conditions. It was shown to provide low
variance estimates, but it however suffers from unbalanced convergence rates of these estimates. In this
paper, we address this problem by introducing a variant of the NNLMS algorithm. We provide a
theoretical analysis of its behavior in terms of transient learning curve, steady-state and tracking
performance. We also introduce an extension of the algorithm for online sparse system identification.
Monte-Carlo simulations are conducted to illustrate the performance of the algorithm and to validate the
theoretical results.