Louise is absolutely smitten with her new love, and the pair seemed closer than ever as they spent the day in London