Contact-Based Perception And Planning For Robotic Manipulation In Novel Environments