A First Course in Machine Learning - Linear Model: A Least Squares Approach

几个引导性问题

1. 什么是线性模型，要说清楚谁与谁是线性。（谁是自变量，谁是因变量，谁是参数） 模型关于参数w是线性的, 模型关于data是线性的
2. 什么样的线性模型是好的（怎么去求参数）
   loss function ： least square； absolute loss
3. 矩阵求法简化计算
   
4. 利用矩阵的迹（trace）求偏导数和梯度
   迹的性质 (核心思想： $tr(a)=a$ 其中,a为常数)
   若 Loss Function是一个一维的标量，那么就可以利用上述思想直接通过求Loss Function的trace对参数的偏导，从而求得原Loss Function对参数的偏导。以下是trace的一些运算规则

 概念解释

=> Leave-One-Out Cross Validation(LOOCV)

An extreme case of K-fold cross-validation is where K=N, the  number of observation in our dataset: each data observation is held out in turn and used to test a model trained on the other N-1 objects.

Fold1, Fold2, ... Fold k, 轮流对当做测试集, 若 |Fold i|=1则 为 LOOCV

LOOCV一般可以达到用大量测试据测试出的效果是一样的, 所以比较常用.

K-fold cross validation 主要是要给出一个公平的测试结果,从而比较不同model之间的好坏.

=> Regularisation

As we don't want our model to become too complex, it makes sense to try and keep sum of parameter squares low.  So, rather than just minimising the average squared loss function, we could minimise a regularised loss function made by adding together our previous and a term penalising over-complexity.

=> $\lambda$

If is too small, our function is likely to be too complex, Too large, and we will not capture any useful trends in the data.