If you're interested in local religious buildings head to White Town— home to Arulmigu Manakula Vinayagar Temple.
You can find out about the history of Puducherry with a visit to Pondicherry Lighthouse. While you're in the area, stroll along the beaches.