I had the same problems when using a blended silicone type lubricant or diluted hair conditioner. The issue seems to occur when it dries out, which happens if you don’t go very often.
the best thing to do is to remove the elastic and bung and give it a really good soak in hot/warm water containing detergent with plenty of plain water rinses to remove the residues of the lube. re-elasticate as normal but don’t use any lube except water.
the other thing it could be if you don’t use lube is that you have picked up a lot of dirt inside the top kit that is preventing the elastic from running smoothly, in which case the best thing to do is to wash it out as above.