I've worn a fanny pack for years cross country skiing. Lots of other guys wear them skiing too.
The only issue I see is the security thing. If you keep your eyes open while wearing one with a pistol in it, you should be OK. But, be vigilant.
I wear one hunting too.
Heck, I have a Duluth green canvas shell bag that I carry stuff in, on certain occasions, that looks like a purse. Of course, I'm 6' 3" tall, and weigh 220 pounds and am 73 years old, so no one says chit to me about carrying it.
Most importantly, I don't have a problem carrying it. If I can carry shotgun shells in it to the range, I can carry other stuff in it on an airplane trip with no bad feelings. Besides, all the women like and I get LOTS of contacts that way..LOL
Martyn