I don't think this is accurate. It's more that the textiles are produced in Asia and transported in containers.
Due to the high shipping costs, they err on the side of filling up the containers to cover the fixed cost. After selling the clothes, there might be enough clothes left over to fill shipping containers to return the clothes, but they will be clothes from different brands and manufacturers.
It would require extraordinary coordination on both the origin and destination country to return the clothes to the manufacturer where they could add the left over clothes to the next batch that is being shipped out to a different country.
Due to the high shipping costs, they err on the side of filling up the containers to cover the fixed cost. After selling the clothes, there might be enough clothes left over to fill shipping containers to return the clothes, but they will be clothes from different brands and manufacturers.
It would require extraordinary coordination on both the origin and destination country to return the clothes to the manufacturer where they could add the left over clothes to the next batch that is being shipped out to a different country.