It's largely aesthetics, but there are a few minor points... If you have a plant that prefers to be on the dry side, such as a cactus or other succulent, using a porous/unglazed ceramic pot is potentially worthwhile. This is because the porous wall of an unglazed ceramic or clay pot allows excessive water to escape, not only through the drainage hole, but through the actual wall of the pot too. If you have a plant that prefers to stay moist, plastic or glazed containers would be a better choice for the opposite reasons.
In either case the difference is small and plastic potting is usually going to be cheaper.