The midterm will cover everything in the Course Notes through Chapter 4.4.
My general advice for studying is to (i) study the notes that you have take in class, (ii) study the Course Notes, and (iii) for topics where you still have any doubts, follow the cross-references from the Course Notes to the textbook for an additional reference.
R questions are fair game for the exam though the exam will focus relatively more on the more recent topics in probability (and a little on statistics/properties of estimators). This means that you should expect relatively more questions from the probability/statistics portion of the class as well as relatively more difficult questions from those topics.
Finally, I have provided some extra questions (these also are roughly a practice test) here. My recommendation is to study some before you try to answer these questions. We will cover (some of) these questions in class on Monday.