This a lovely discussion you have made. Glad to read it. It is very useful for every optimizer.