Light a gas grill to medium-high heat. (Mix olive oil and brown sugar)
Cut the peaches in half, remove the pit, and brush each cut side with olive oil and brown sugar mix.
Place the peaches cut-side down on the grill for 4 to 5 minutes. Cook undisturbed until grill marks appear, 4 to 5 minutes.
Turn peaches over, and cook until tender, another 4 to 5 minutes.
Remove from the grill. Drizzle the caramel across the tops, then the whipped cream, or ice cream. And, add some sprinkles to the top for some fun color.