And in my experience, the vigorous application of this logic begins to feel like the dark side of hungarian notation.

I can reverse engineer your code but I can’t reverse engineer your thoughts.

 If you are too lazy to update your comments when you update your code thats one thing. Just don’t encourage others to do it.

I rather see documentation as a tool to write better code. And now I’m not talking about writing comments inside methods, but commenting projects, classes/interfaces and properties/methods: XML style comments.

I try to explain my point of view here:
http://blog.gauffin.com/2010/06/my-name-is-more-docu-more/

This of course is in the perfect world of TDD, I do find myself commenting code when I need to fix something quickly and don’t have time to write or update a unit test, not ideal. I started off writing comments in pseudo code style comments as a way to organize my thoughts, you should use your Unit Test Class to do this in TDD style and not in the code. The only reasonable explanation for commenting a piece of code is when it needs work; i.e. better design, and to reflect that in the comment so that another developer knows when they see it that it needs better design, keeping in mind unit tests already exists for it.

“Everything is “API”, even if used by the same person/team. The software I usually work on is usually too complex to keep ALL the nitty-gritty details in my head. There is a great value in being able to FORGET the code that I’m not currently working on. Having high-level comments all over the place serves as a “shortcut” for understanding the code I now no longer need to read.”

Its ok not knowing and/or remembering the details of complex software. But if you open any given code file in such software, and you can’t easily/quickly understand what it is, it almost surely isn’t good code. You say having the high-level comments all over the place serves as a ‘shortcut’, but its usually there for the inability to use a better design. ps. I know sometimes we just have to deal with such code, but that doesn’t mean the situation wasn’t created by a different problem, ignoring that perpetuates the issue.

While I agree that commenting at the level of statements is usually superfluous (i.e. the code can usually express itself AT THAT PARTICULAR LEVEL OF ABSTRACTION), properly documenting all the other levels has great value for long-term maintainability of the product, which is where some forms of comments have their rightful place.

The rules of thumb for good comments are:

1. The comment should be AT HIGHER LEVEL OF ABSTRACTION than can be expressed directly in code.
2. The comment is usually written BEFORE code.

Here are some examples:

– Commenting the API is absolute must – the client will usually not have the access to the source code and not all of the pre/post-requisites for making a particular API call can be expressed through the language / communication technology chosen to implement the API. Typically, the best the API implementation code can do is throw an exception / error code, but by that time the high-level context for understanding the root cause of the problem may be lost.
– Everything is “API”, even if used by the same person/team. The software I usually work on is usually too complex to keep ALL the nitty-gritty details in my head. There is a great value in being able to FORGET the code that I’m not currently working on. Having high-level comments all over the place serves as a “shortcut” for understanding the code I now no longer need to read.
– Commenting intention – as code evolves, having a proper documentation on what is ESSENTIAL, versus what is INCIDENTAL lets me stick with essentials and break less clients as code evolves. In fact, well written comment will change LESS OFTEN than the implementation code.
– Commenting higher-level structures. Often, my data structure is a complex graph or tree of nodes, and needs to have a specific “shape” or certain things need to happen in certain sequence to have a proper behaviour. While assertions/contracts help, higher-level documentation is usually a must.
– Commenting dependencies – a product can depend on many pieces of code or even entire applications not written by me or my team. We are often forced to do certain things in a certain way simply by not existing in a vacuum. With proper documentation, it is easier to understand what changes need to be made in our code when the “environmental forces” change.
– Commenting on a choice of algorithm or data structure – there is often a situation when there are multiple choices of algorithms and/or data structures that can be used to solve a particular problem. Neither choice is “incorrect” – they all produce correct end result – by there are time/space/scalability tradeoffs to consider and the “correct” solution is the one that fits with the expected USAGE PATTERN best. Commenting what is the expected usage pattern and what are the performance tradeoffs chosen as a consequence can help tremendously in understanding performance bottlenecks when usage pattern changes.
– Comments can simply be LONGER than programming language names. Just the other day, I was tempted to make a method: “RemoveHandlesOfNotImplicitlyUnloadedModelsFromSession()”; at the end I settled for simple (but imprecise) “CleanupSession()” with appropriate comment.

Let me disagree with some of the specific points you make:

– “They do not change with the code.” – Nor the other way around. I often find myself editing the comment simply to “debug” my thinking and change the code only AFTER the comment looks right. If you code first and comment later, then I can see your point.
– “A wrong comment is worse than horribly complex non-commented code.” – True, but a comment rarely exists in vacuum. If the rest of you product is documented properly, the bad comment will be obviously inconsistent with the rest of the documentation and easy to spot. Actually, achieving “comment consistency” is useful technique for “paradigm debugging” in your head.
– “A comment almost always expresses an absolute thing, where a well named variable or method can express an abstract concept.” – Actually, my experience is the opposite. Good comments are always at the higher level of abstraction than the code.
– “Comments don’t show up in stack traces.” – No, they are one click away.
– “Reading comments is optional.” – Well, the basic OOP principle of encapsulation tells us that reading implementation code is optional as well. Without comments, what you are left with is reading names/signatures of classes/methods/parameters and that alone is often simply not expressive enough.

Method names also need to be descriptive – it’s all about the name.

Comments do have their place – but most comments can be left inside Source control systems and spec documents. That said, I like what Shazbot said:

If you really need comments then they should be put within the comment blocks that show on the intelisense within your VS IDE.

That’s why I added a rule into my refactoring tool to say any code where the comments are 1/3 the size of the code, needs to be refactored immediately.

While I agree that commenting at the level of statements is usually superfluous (i.e. the code can usually express itself AT THAT PARTICULAR LEVEL OF ABSTRACTION), properly documenting all the other levels has great value for long-term maintainability of the product, which is where some forms of comments have their rightful place.

The rules of thumb for good comments are:

1. The comment should be AT HIGHER LEVEL OF ABSTRACTION than can be expressed directly in code.
2. The comment is usually written BEFORE code.

Here are some examples:

– Commenting the API is absolute must – the client will usually not have the access to the source code and not all of the pre/post-requisites for making a particular API call can be expressed through the language / communication technology chosen to implement the API. Typically, the best the API implementation code can do is throw an exception / error code, but by that time the high-level context for understanding the root cause of the problem may be lost.
– Everything is “API”, even if used by the same person/team. The software I usually work on is usually too complex to keep ALL the nitty-gritty details in my head. There is a great value in being able to FORGET the code that I’m not currently working on. Having high-level comments all over the place serves as a “shortcut” for understanding the code I now no longer need to read.
– Commenting intention – as code evolves, having a proper documentation on what is ESSENTIAL, versus what is INCIDENTAL lets me stick with essentials and break less clients as code evolves. In fact, well written comment will change LESS OFTEN than the implementation code.
– Commenting higher-level structures. Often, my data structure is a complex graph or tree of nodes, and needs to have a specific “shape” or certain things need to happen in certain sequence to have a proper behaviour. While assertions/contracts help, higher-level documentation is usually a must.
– Commenting dependencies – a product can depend on many pieces of code or even entire applications not written by me or my team. We are often forced to do certain things in a certain way simply by not existing in a vacuum. With proper documentation, it is easier to understand what changes need to be made in our code when the “environmental forces” change.
– Commenting on a choice of algorithm or data structure – there is often a situation when there are multiple choices of algorithms and/or data structures that can be used to solve a particular problem. Neither choice is “incorrect” – they all produce correct end result – by there are time/space/scalability tradeoffs to consider and the “correct” solution is the one that fits with the expected USAGE PATTERN best. Commenting what is the expected usage pattern and what are the performance tradeoffs chosen as a consequence can help tremendously in understanding performance bottlenecks when usage pattern changes.
– Comments can simply be LONGER than programming language names. Just the other day, I was tempted to make a method: “RemoveHandlesOfNotImplicitlyUnloadedModelsFromSession()”; at the end I settled for simple (but imprecise) “CleanupSession()” with appropriate comment.

Let me disagree with some of the specific points you make:

– “They do not change with the code.” – Nor the other way around. I often find myself editing the comment simply to “debug” my thinking and change the code only AFTER the comment looks right. If you code first and comment later, then I can see your point.
– “A wrong comment is worse than horribly complex non-commented code.” – True, but a comment rarely exists in vacuum. If the rest of you product is documented properly, the bad comment will be obviously inconsistent with the rest of the documentation and easy to spot. Actually, achieving “comment consistency” is useful technique for “paradigm debugging” in your head.
– “A comment almost always expresses an absolute thing, where a well named variable or method can express an abstract concept.” – Actually, my experience is the opposite. Good comments are always at the higher level of abstraction than the code.
– “Comments don’t show up in stack traces.” – No, they are one click away.
– “Reading comments is optional.” – Well, the basic OOP principle of encapsulation tells us that reading implementation code is optional as well. Without comments, what you are left with is reading names/signatures of classes/methods/parameters and that alone is often simply not expressive enough.