First of all, I'm a developer. I might see attributes and attribute sets differently but I'll give it a try.
The short version:
attribute = one specification of a product
attribute set = all the specifications (attributes) of a product. This basically defines a product.
Now the longer version:
I saw that is a common practice to associate (at least verbally) an attribute set to a category. This is because it's in our nature to put everything in a group.
An attribute set represents the TYPE of the product. This article explains it better.
How far should you go with the attributes?
I don't know if there is a correct answer for this.
As an e-commerce website user I would like to have every single detail of a product in a separate attribute instead of having most of them in a single long text. This makes it easier to filter/search products by some attributes, to have a clear view while comparing them.
As the developer that has to make your website run faster and not crash on mass updates or high traffic I wouldn't want to have too many attributes and attribute sets (more entries in db = slower queries)
As the guy that has update the products (if there is no automated product import/sync) I would rather copy/paste some text from a document into a single field and click save.
Based on these 3 points of view here is what I would do for my personal e-commerce website (I have't decided yet on what I'm selling): Each product type will have a separate attribute set. Attribute sets will share some attributes (name, description...). Each user sensitive information about a product will be a separate attribute. Information that are not that important will all be put in a single attribute (called 'Additional information').
This is my opinion. I don't say is the best but it's the only one I've got.