The blocks around One World Trade Center feed a huge number of office workers on weekdays and a huge number of visitors every day, which means two things. There's plenty of food, and a lot of it is built for speed rather than for sitting down. If you've just come out of the 9/11 Memorial or down from the Observatory, you're probably tired, possibly with kids, and you want to know where to walk without a third opinion.
So here's the honest layout. Directly at the tower's feet you have Eataly inside 4 World Trade Center and the food counters in the Oculus. Cross West Street and you're at Brookfield Place, which has the two best food halls downtown. Walk a bit further east or south and you get actual restaurants with actual dinner service, on Stone Street and at the Seaport. We're at 17 John Street, about a ten-minute walk east of the plaza, and this is the order we send guests in.
One practical note: weekday lunch between noon and 1:30 is the crush. If you can shift an hour either way, everything below gets easier. And weekends downtown are quieter than people expect, so a Saturday lunch at Brookfield Place is a pleasant, half-empty affair. Hours move around, especially on Sundays, so check before you make the walk.