It's not about the material, it's about build quality.
The Angenieux zooms with policarbonate barrels were very well built, and have proven durable, but at the time they didn't enjoy the commercial success they deserved because photographers expected high quality lenses to be all-metal.
I'm testing turn of the century "silver" FA zooms. The construction is awful, but the average optical quality is superior to better built early FA and F counterparts.
The problem lies in durability and easy/possible repair procedures.
If a lens group is glued or set in a plastic casing, the only option is to find a spare.
BTW, I never found a metal zoom out of whack, but with no signs of damage. TWO expensive f/2.8 zooms looking absolutely pristine had bent focusing/zooming "tubes", causing vignetting and other problems.
The day the spares will be unavailable those very good and highly valuable lenses will be usable only as doorstops
Sigma still has spares for film era objectives. If I'm not mistaken some Pentax spares for slightly newer lenses are nowhere to be found.
If a partly plastic lens has brass-on-aluminium retaining rings, and cams/tubes/helicoids made of durable materials, that would be a high-quality objective that would last forever.
Plastic is not the devil.
Unfortunately a wobbly, plasticky barrel is often the sign of a very poor construction. A few of my wobbly zooms are surprisingly good... but I made sure to buy like new examples.
Some of my vintage lenses with great all-metal build are very worn and have horribly bent filter threads, but the optical performance is so good to make me think they perform as when they were new. A plastic lens would never survive a crash that causes that kind of damage.
When I mention durability I'm also referring to electro-mechanical problems. Some FA lenses are failing. My FA Macro 100mm, that is NOT cheaply built, had a problem with the limiter switch (AF problems). It has gone under surgery to repair the switch. I don't believe a cheap poorly made zoom would be as easy to repair, because containing costs has a price, if something fails.
Most of my favourite Pentax were purchased second-hand in the late seventies or early eighties. All of them work as the first day. Three of my very expensive zooms had to be repaired (one of them because it became a fungus nest while the other metal lenses stored in the same place where unaffected). Yes, plastic welcomes fungi more than metal. Personal experience.