Unfortunately, New York City is rather polluted for fishing as source of food.
For public transportation wise, the nearest hygienic places is the pier at Fort Totten area North East of Queens (Q13, Q16), the jetti at Cross Island Parkway and 28th Ave East of Queens (Q13), Sea Cliff Beach along Prospect Ave. in Glenwood Landing (N21) - There's a "hidden" beach with wide ocean view accessible by staircases (if on foot) by turning left on Tilley Pl. on Prospect Avenue.
I've yet to fish there, but scouting in the last area last evening seem very promising. Clear waters (Makes me wonder about Montauk's ocean again...perhaps someday when I have the time, although as clear as it seems it doesn't look as good as the Bahamas on google map, of course). Quite a few horseshoe crabs I must add. Just need to note the spots prohibited for fishing as it is mainly a place for Marina and bathing (swimming).
Oyster Bay which is reachable by train, hence not "free" as in metrocard unlimited, is the closest public transportation reacheable fishing site I have caught fish before. Mainly snappers and porgies (both using porgy hooks with clams), but I have seen some caught sand shark, stripper and fluke. This was about 3 years ago.