Xcode could store tens of gigabytes in ~/Library/Developer folder.
Most of those cached files & symbols are not reclaimed over time and could consume a large amount of your storage, which is especially important if you have relatively small SSD drive.