This paper extends kernel weighted GMM estimators recently proposed by the author in the context of homoskedastic processes to a class of models with conditionally heteroskedastic innovations. GMM estimation of such models was previously studied by Kuersteiner (1997, 1999a/b) in the context of ARMA processes and Guo and Phillips (1997) in the context of ARCH processes. Optimal implementation of the GMM estimator requires to include more and more instruments as the sample size grows. The use of kernel weighted moment conditions is a natural way to handle the infinite dimensionality of the instrument space. In addition a higher order asymptotic theory is provided to choose the optimal number of instruments in a finite sample context. The higher order analysis reveals that the GMM implementation proposed in Kuersteiner (1997) does not suffer from the usual bias problems of standard 2SLS procedures.