ML requested that I post some of my favorite places for fabric, paper, books, etc. in NYC. So I began searching for the stores that I have frequented... and it turns out I can't remember their names. I really would suggest, however, just going to the Garment District and wandering around. Many of the places I just ran into that way. Be sure to look up though! In this district there are many shops that are upstairs too. I will find the name of the place that I bought my wedding fabric though, since one would never find that from the street. Another caveat: Not all the stores are open on Saturday since this is a business district and not really touristy.
Here's a good link describing fabric, ribbon, and notions places in the Garment District.
While you are there be sure to go to The Burger Joint
118 W 57th St
New York, NY 10019
Cross Street: Between 6th Avenue and 7th Avenue
Directions: F at 57th St
Around a curtained corner in the posh Le Parker Meridien, a neon burger sign points the way. People pack into this crowded hole-in-the-wall, where names are etched into walls next to kid's drawings and movie posters. Even though it's decorated like a regular's hangout, no one sticks around--the few coveted booths are hovered over shamelessly.
As for book stores, I would suggest The Strand bookstore. They claim to have 18 miles of used books in their store. It's a great place to find deals, and is well organized.
Does the former New Yorker have anything to add to this?