The Herschel Virgo Cluster Survey. III. A constraint on dust grain lifetime in early-type galaxies