Any developer has encountered bugs that make banging your head on the desk seem like relief. It's part of software development. It's complicated work which can be thrown off by the simplest of hidden mistakes. The question is; how do we reduce the frequency of bugs that stump us for long periods of time and increase cases where we quickly find the solution to the problem? There's many techniques, including testing, avoiding problematic patterns and coding only when awake and alert, avoiding late night or early morning coding when errors are more likely to happen. There's a ton of Best Practices for improving the reliability and debuggability of our code. But I'm going to talk about a classic, and one of my favorites here; logging.
The concept of programs writing logs of what they are doing is probably the oldest in computing. Even a Babbage computing machine will output the results of its calculation, which is itself a simple form of logging. But something I see a lot in modern software is anemic logging, or worse yet, useless logs that say what happened but no details about it that would let you determine what went wrong. Modern debugging tools can make a lot of traditional debug logging techniques obsolete when it comes to development, but what about when your code is in production, being used and you are unsure of what user input caused the error that you're trying to fix? This is where logging is indispensable and why you should err on the side of giving yourself more information than you might need rather than less. It's easy to grep for what you need and ignore what you don't, and it's impossible to generate logs that don't exist.
We live in a time where drive space is a commodity, and quite dispensable. When it comes to text logs, the space consumed is paltry compared to the value of a developer's time spent trying to chase down an error with no information. If your traffic volume is so high that storing the logs is still a consideration, you can always have them rotate aggressively. Having some logs is far better than having none.
