Preheat oven to 400 degrees. Thinly slice plums. Melt butter in small pan and add sugar, plums, and orange juice. Cook and stir until sugar melts and juice begins to thicken, about 10 minutes. Add rosemary and pinch of salt. Remove from heat and let cool 5 minutes. Slice rind of one side of Brie and place cut side up on an oven-safe plate or platter with enough rim to catch extra juices. Arrange plums and juices on top of Brie and bake for 10 - 15 minutes. Serve with bread, crackers, or toast.