All fabrics will pill to some degree. The better the weave and the overall quality of the fibers, the more resistant this will be.
It's a normal part of the wear of such fabrics. The device alluded to above should work well. There are other products as well including stones. Care for such fabrics is also essential. Hand washing as needed with proper detergents and laying flat to dry vs. dry cleaning.
SG_67 and Flanderian are correct. If you own my book look in the Clothes Care Chapter for this:
Pilling is when the fabric surface gets little gnarls. Fabric pills happen when the tiny fibers in our clothes gradually loosen due to wear and washing. It's more prevalent on oxford cloth shirts. When the fibers come undone, one loose stand can snag another, which entangles. In time a pill forms.
Don't pull them off your clothes. You can remove them with an electric razor or better use an electric razor made for the purpose. There are also stones that erase pills on sweaters. Dry Cleaners can also remove pills.