Does the FM in a street address both have to be capitaized?

No, only the first letter of FM (Farm Market) should be capitalized in a street address. For example, 123 Main Street FM 1234.