In Reinforcement Learning, one often needs to evaluate a given policy using rewards observed by following another policy. This is called off-policy evaluation in Learning Theory parlance. The traditional methods for off-policy evaluation involve importance sampling, which comes with certain drawbacks. We shall look at these drawbacks and how linear regression may be used instead to overcome the same.
