Leveraging Compositional Structure for Reinforcement Learning and Sequential Decision Making Problems