The two terms are not interchangeable. This soapbox is intended to help you learn to tell the difference if you can't already.
The primary difference is in architecture. Castles were built as forts; they have narrow windows, drawbridges, murder holes, etc. Dover Castle is an excellent example. Palaces were built as places of residence so they tend to be much more sprawling and lacking in drawbridges and barbicans.
Another difference is in location. Palaces were not built in hostile territory since they were intended to be residences whereas castles were used as outposts from which a lord could exercise control of the surrounding territory.
Castles and palaces could either be built in a city- like the one in Canterbury- or in more remote locations. What person doesn't like to have a place (or several) to retreat to when they need a break from things?
Hopefully this rather short soapbox has been useful!