Do sellers have to pay customs on items being returned to them?

Let's say I'm selling from Canada to the US. The tax de minimus in Canada is only $20. If a customer returns an item valued at over $20, do I the seller, have to pay taxes and duties to import it back to Canada? I've been having trouble finding the answer online, as most results only show a refund for taxes paid as a customer returning an item. Any insight would be appreciated!