Towards Understanding the Role of Knowledge in Improving Transformer-based Language Models