Would it be weird to wear soccer pants when I don't play soccer?

No, some of my friends who don't play soccer wear Samba's (which are indoor soccer shoes) and if ur really worried just tell people that you are borrowing them from somebody :-)