"Bright daylight" lets us directly apply the
Sunny 16 rule (if you're not familiar with this, go learn it).
At ISO 100 and f16 your shutter speed should be (roughly) 1/100.
So at f11 your shutter speed should be (roughly) 1/50.
That means either your polarizer was cutting no light at all, or it wasn't really as bright outside as you thought it was. The latter is more likely, as most people not familiar with doing eyeball-metering aren't likely to notice the difference between a full Sunny 16 broad daylight level of light and one or two stops less than that.
Sounds to me like everything was working just fine. The problem is that you're using a light-reducing filter on a low ISO setting with the aperture shut way down and expecting high shutter speeds. In other words, it ain't the camera, it's the operator. If you expect/need higher shutter speeds then either:
1. Bump up your ISO
2. Open up your aperture
3. Remove the filter
4. Some combination of the above