北大暑期课程《回归分析》(Linear Regression Analysis)讲义1.doc

Class 1, Page 9Class 1: Expectations, variances, and basics of estimationBasics of matrix (1) I. Organizational Matters(1) Course requirements:1) Exercises: There will be seven (7) exercises, the last of which is optional. Each exercise will be graded on a scale of 0-10. In addition to the graded exercise, an answer handout will be given to you in lab sections. 2) Examination: There will be one in-class, open-book examination. (2) Computer software: StataII. Teaching Strategies(1) Emphasis on conceptual understanding. Yes, we will deal with mathematical formulas, actually a lot of mathematical formulas. But, I do not want you to memorize them. What I hope you will do, is to understand the logic behind the mathematical formulas. (2) Emphasis on hands-on research experience.Yes, we will use computers for most of our work. But I do not want you to become a computer programmer. Many people think they know statistics once they know how to run a statistical package. This is wrong. Doing statistics is more than running computer programs. What I will emphasize is to use computer programs to your advantage in research settings. Computer programs are like automobiles. The best automobile is useless unless someone drives it. You will be the driver of statistical computer programs. (3) Emphasis on student-instructor communication.I happen to believe in students judgment about their own education. Even though I will be ultimately responsible if the class should not go well, I hope that you will feel part of the class and contribute to the quality of the course. If you have questions, do not hesitate to ask in class. If you have suggestions, please come forward with them. The class is as much yours as mine. Now let us get to the real business.III(1). Expectation and VarianceRandom Variable: A random variable is a variable whose numerical value is determined by the outcome of a random trial.Two properties: random and variable.A random variable assigns numeric values to uncertain outcomes. In a common language, give a number. For example, income can be a random variable. There are many ways to do it. You can use the actual dollar amounts. In this case, you have a continuous random variable. Or you can use levels of income, such as high, median, and low. In this case, you have an ordinal random variable 1=high, 2=median, 3=low. Or if you are interested in the issue of poverty, you can have a dichotomous variable: 1=in poverty, 0=not in poverty.In sum, the mapping of numeric values to outcomes of events in this way is the essence of a random variable.Probability Distribution: The probability distribution for a discrete random variable X associates with each of the distinct outcomes xi (i = 1, 2,., k) a probability P(X = xi).Cumulative Probability Distribution: The cumulative probability distribution for a discrete random variable X provides the cumulative probabilities P(X x) for all values x.Expected Value of Random Variable: The expected value of a discrete random variable X is denoted by EX and defined:EX = P(xi)where: P(xi) denotes P(X = xi). The notation E (read “expectation of”) is called the expectation operator.In common language, expectation is the mean. But the difference is that expectation is a concept for the entire population that you never observe. It is the result of the infinite number of repetitions. For example, if you toss a coin, the proportion of tails should be .5 in the limit. Or the expectation is .5. Most of the times you do not get the exact .5, but a number close to it. Conditional ExpectationIt is the mean of a variable conditional on the value of another random variable. Note the notation: E(Y|X).In 1996, per-capita average wages in three Chinese cities were (in RMB):Shanghai:3,778Wuhan:1,709Xian: 1,155Variance of Random Variable: The variance of a discrete random variable X is denoted by VX and defined:VX = (xi - EX)2 P(xi)where: P(xi) denotes P(X = xi). The notation V (read “variance of”) is called the variance operator.Since the variance of a random variable X is a weighted average of the squared deviations, (X - EX)2 , it may be defined equivalently as an expected value: VX = E(X - EX)2. An algebraically identical expression is: VX = EX2 - (EX)2.Standard Deviation of Random Variable: The positive square root of the variance of X is called the standard deviation of X and is denoted by sX: s X =The notation s (read “standard deviation of”) is called the standard deviation operator.Standardized Random Variables: If X is a random variable with expected value EX and standard deviation sX, then:Y=is known as the standardized form of random variable X. Covariance: The covariance of two discrete random variables X and Y is denoted by CovX,Y and defined:CovX, Y = where: P(xi, yj) denotes )The notation of Cov , (read “covariance of”) is called the covariance operator.When X and Y are independent, Cov X, Y = 0.Cov X, Y = E(X - EX)(Y - EY); Cov X, Y = EXY - EXEY(Variance is a special case of covariance.)Coefficient of Correlation: The coefficient of correlation of two random variables X and Y is denoted by rX,Y (Greek rho) and defined:where: sX is the standard deviation of X; sY is the standard deviation of Y; Cov is the covariance of X and Y.Sum and Difference of Two Random Variables: If X and Y are two random variables, then the expected value and the variance of X + Y are as follows:Expected Value: EX+Y = EX + EY; Variance: VX+Y = VX + VY+ 2 Cov (X,Y).If X and Y are two random variables, then the expected value and the variance of X - Y are as follows:Expected Value: EX - Y = EX - EY; Variance: VX - Y = VX + VY - 2 Cov (X,Y).Sum of More Than Two Independent Random Variables: If T = X1 + X2 + . + Xs is the sum of s independent random variables, then the expected value and the variance of T are as follows:Expected Value: ; Variance: III(2). Properties of Expectations and Covariances:(1) Properties of Expectations under Simple Algebraic Operations This says that a linear transformation is retained after taking an expectation. is called rescaling: is the location parameter, is the scale parameter. Special cases are: For a constant: For a different scale: , e.g., transforming the scale of dollars into the scale of cents.(2) Properties of Variances under Simple Algebraic OperationsThis says two things: (1) Adding a constant to a variable does not change the variance of the variable; reason: the definition of variance controls for the mean of the variable graphics. (2) Multiplying a constant to a variable changes the variance of the variable by a factor of the constant squared; this is to easy prove, and I will leave it to you. This is the reason why we often use standard deviation instead of variance is of the same scale as x. (3) Properties of Covariance under Simple Algebraic OperationsCov(a + bX, c + dY) = bd Cov(X,Y). Again, only scale matters, location does not. (4) Properties of Correlation under Simple Algebraic OperationsI will leave this as part of your first exercise: That is, neither scale nor location affects correlation. IV: Basics of matrix.1. DefinitionsA. MatricesToday, I would like to introduce the basics of matrix algebra. A matrix is a rectangular array of elements arranged in rows and columns:Index: row index, column index. Dimension: number of rows x number of columns (n x m)Elements: are denoted in small letters with subscripts. An example is the spreadsheet that records the grades for your home work in the following way: Name1st2nd.6thA710.9B6 5.8.Z8 9.8This is a matrix. Notation: I will use Capital Letters for Matrices.B. VectorsVectors are special cases of matrices: If the dimension of a matrix is n x 1, it is a column vector:If the dimension is 1 x m, it is a row vector:y = | . |Notation: small underlined letters for column vectors (in lecture notes)C. TransposeThe transpose of a matrix is another matrix with positions of rows and columns being exchanged symmetrically.For example: if It is easy to see that a row vector and a column vector are transposes of each other. 2. Matrix Addition and SubtractionAdditions and subtraction of two matrices are possible only when the matrices have the same dimension. In this case, addition or subtraction of matrices forms another matrix whose elements consist of the sum, or difference, of the corresponding elements of the two matrices. Examples: 3. Matrix Multiplication A. Multiplication of a scalar and a matrixMultiplying a scalar to a matrix is equivalent to multiplying the scalar to each of the elements of the matrix.B. Multiplication of a Matrix by a Matrix (Inner Product)The inner product of matrix X(a x b) and matrix Y(c x d) exists if b is equal to c. The inner product is a new matrix with the dimension (a x d). The element of the new matrix Z is:c k=1Note that XY and YX are very different. Very often, only one of the inner products (XY and YX) exists. Example: BA does not exist. AB has the dimension 2x1Other examples:If , , what is the dimension of AB? (3x3)If , , what is the dimension of BA? (5x5)If , , what is the dimension of AB? (1x1, scalar)If , , what is the dimension of BA? (nonexistent)4. Special MatricesA. Square MatrixB. Symmetric MatrixA special case of square matrix. For , . All i, j.A = AC. Diagonal MatrixA special case of