Step One: First things first, try cleaning your headlights with soap! In some cases (but unfortunately, not my own) you can remove headlight grime with just soap and water. Allow ample drying time, and wait to see if the soap did the trick.
Step Two: If the soap didn’t quite clean things up for you, it’s time to move on to more of an intense method! Before continuing, make sure you line the area around your headlight with masking tape. This will protect the area around your headlights from scuffing.