斯坦福大学机器学习所有问题及答案合集.pdf

CS 229 机器学习  (问题及答案) 斯坦福大学 目录 (1) 作业1（Supervised Learning）            1 (2) 作业1解答（Supervised Learning）          5 (3) 作业2（Kernels, SVMs, and Theory）         15 (4) 作业2解答（Kernels, SVMs, and Theory）       19 (5) 作业3（Learning Theory and Unsupervised Learning）                      27 (6) 作业3解答（Learning Theory and Unsupervised Learning）                      31 (7) 作业4（Unsupervised Learning and Reinforcement Learning） 39 (8) 作业4解答（Unsupervised Learning and Reinforcement Learning）                      44 (9) Problem Set #1: Supervised Learning          56 (10) Problem Set #1 Answer               62 (11) Problem Set #2: Problem Set #2: Naive Bayes, SVMs, and Theory                       78 (12) Problem Set #2 Answer               85 CS229 Problem Set #11 CS 229, Public Course Problem Set #1:Supervised Learning 1. Newtons method for computing least squares In this problem, we will prove that if we use Newtons method solve the least squares optimization problem, then we only need one iteration to converge to . (a) Find the Hessian of the cost function J() = 1 2 Pm i=1( Tx(i) y(i)2. (b) Show that the fi rst iteration of Newtons method gives us = (XTX)1XT y, the solution to our least squares problem. 2. Locally-weighted logistic regression In this problem you will implement a locally-weighted version of logistic regression, where we weight diff erent training examples diff erently according to the query point. The locally- weighted logistic regression problem is to maximize () = 2 T m X i=1 w(i) h y(i)logh(x(i) (1 y(i)log(1 h(x(i) i . The 2 T here is what is known as a regularization parameter, which will be discussed in a future lecture, but which we include here because it is needed for Newtons method to perform well on this task. For the entirety of this problem you can use the value = 0.0001. Using this defi nition, the gradient of () is given by () = XTz where z Rm is defi ned by zi= w(i)(y(i) h(x(i) and the Hessian is given by H = XTDX I where D Rmmis a diagonal matrix with Dii= w(i)h(x(i)(1 h(x(i) For the sake of this problem you can just use the ab