Socrates Sculpture Park. You have to walk around the wall, but this is allowed! That said, you're only allowed to go canoeing when the Parks Department and the LIC Community Boathouse arrange one of these neat canoeing outings, which they plan to start doing once a month. So, keep your kayak under you bed for now, but get out that half-busted beach chair you take to Jones Beach once a year.
To get to Hallet's Cove, walk West on 31st Avenue until you fall into the East River. Then backtrack until you're on sand.