How do you decide what's meaningful? How do you decide where to put the log output? How do you decide what each level means? Can you come up with organizational logging standards, just like you have Java or C# coding standards?
@ThomasOwens The only right way to do it is AOP. It's like, the definition of a cross cutting concern
unfortunately, what I do instead is have a private static final Logger in any class where I want to log stuff, and I have an eclipse shortcut that generates the logger object for me.
@durron597 Meh, it's just a call to a logger. You still have to put calls to that somewhere in your code. You can inject the logger if you like, but it's not like you can stick an attribute above your method calls and it just automagically works.
@ThomasOwens I've been privileged enough to do it multiple times now in different applications and really think I've got quite a good approach now. The most important part of logging is the part everyone gets wrong or ignores: CONTEXT. Every log record should have a context identifier that can be used to trace it to an entire context of sequential actions - maybe the context starts and ends with a login, when I worked on a phone system it started and ended with the call
Yes this can be done on a per-thread basis.
You need to use Filters and MDC (Mapped Diagnostic Context). This solution will only affect the logging that occurs on the thread that is executing the @Scheduled method.
Overview of steps
In your @Scheduled method, add an entry to the MDC of that t...
@JimmyHoffa How would you go about doing that? In most places where I worked, logging didn't get put in until after you needed to know a particular thing. And then you fine-tuned it as you find out what you actually need from it.
@JimmyHoffa I have a class. My class makes widgets, that is it's job. If my class also logs information about it's widget making activities, now it has two jobs. SRP is violated.
@RobertHarvey the important part is coming up with a concept for contexts that makes sense in each place, and then figuring out how to create/remove them and when. In a SOA system I'd make all my services check their requests for a special context header, if it's not there, they create a new context, if it is, they continue with the given one
Wouldn't you have to record your actions somewhere for awhile, and then link to those actions if something bad happens? I know there are things like stack traces, but shit.
thread storage is usually a safe bet - I'm a fan of putting a simple dictionary in thread storage and an interface for accessing items in your thread storage as a dictionary
my logging looks a lot like this:
public void doStuff(myClass myObject){
log.log("public void doStuff(myClass myObject)");
myObject.doSomething();
log.log("myObject.doSomething()");
@RobertHarvey That's how I've seen it on projects, too. But I think that it may be better to consider logging and other debugging / testing aids at requirements time. That includes profiling and timing, logging, and anything else.
nope, you write individual actions atomically as you go - so the failure would be obvious: Context 1 Logging in Context 1 Authorizing Context 1 Requesting Google image Context 1 NullReferenceException blablabla
the other piece is I always put a data dictionary on logging messages so it'll say: Context 1 Logging in Data { User: "foo", IP: "1.1.1.1", Content-Type: "application/json" }
It's not to make it human readable. My user interface classes can handle converting from an object to a human-readable format of the appropriate type. The toString exists for logging and debugging purposes only.
And then, I just log the object and bam, I get a log message with its current state. Plus, you can autogenerate toString methods that do that insane easy in any IDE I've ever used.
In this shop, you say "Let's sit down and discuss how we can have meaningful logging," and the response is "Fsck it, let's just get some logging and make it pretty later."
@ThomasOwens no! The key is the data dictionary is queryable. It needs consistency for this, so it has to be a literal list of key, value columns in some queryable structure (mongo document, SQL columns, something) so you can find all events with PasswordLength = 129 to find out if that's the cause of the exception you had
real-time monitoring software can look for keys and values, but to make it parse a string... no
You're logging an object. I don't pass a string in, I pass the object.
So if the adapter is writing text, it invoked toString by default. If you're logging to a database (or some other data source), it actually has the object and can do things with it.
@RobertHarvey right. so you can go find all the scenarios which occurred in a certain way to know if/when something broke and why
find all the deposits where type is Visa are broke, or all of them with Visa and amount > $500 result in a rejection and now when you get all these complaints of intermittent deposit rejections, you have an answer
I'm trying to think if that would help us. If a web page breaks, that usually means that you get back either bad data or no data. Mostly, our problem is we have so little visibility that we don't even know that happened. We just know a scrape didn't take place overnight. We don't even know that really. All we know is that some customers have bad indicators.
We have no visibility at all.
So even getting rudimentary logging in place would be a big step up for us.
And since we're fighting so many fires, it's difficult to get that done.
It's not a complex structure, but these 2 things make finding out about your system so much easier... context, and an arbitrary structured data dictionary. You can find out what the most common usages are of your system at a high or low level. You can inspect timestamps to find out precisely which steps in the system take the longest, and in what scenarios they're fast based on the data attached to those steps (AMEX deposits end up taking 3 times longer than any others!)
It's great for you as a dev to diagnose and write live monitoring software, and it's great for the company to identify system behaviours and usages to make business and product development decisions off of
from a code standpoint it's just class Event { public int|guid|whatever ContextId {get;} public (string or enum) Description { get; } public Dictionary<string, string> Artifacts { get; } }
create a few simple methods to make it easy to create the object, you can put a .ToString() on it to make nice lines for the logfiles, otherwise the structured object goes to a repo which generates the timestamp, pulls the context or creates one if there isn't one... etc
again, most important of all is that your logging has that context so you can follow a sequence of events...
I wonder if this is a good argument for lightweight databases in applications. I don't know what .NET has, but Java has things like Derby.
Depending on how you structure it, I think you can just as easily send a Derby DB back from a deployment environment for debugging. Of course, if you're running a service and already have a database of some kind, just use that.
How much proliferation of logging data do you get? Do you run this all the time, or only when you need to trace out a problem? Do you keep, say, two weeks of history?
Does teh databasez fill up?
I assume that, instead of injecting a Logger, you now inject a ContextualLogger with the context pre-populated?
@RobertHarvey those are pieces of data on the event; you can have it if you want. Or you could just make them an event type so that value goes in the description field instead of dangling off the event. It's all very loose and up to you how you use the structure, as you work with the structure you'll come up with what data belongs where to serve the purposes you need. I leave this stuff running 100% of the time, and if database is going to fill; run a job at whatever frequency is necessary
delete or archive the data off somewhere else depending on the available disk constraints
Yes this can be done on a per-thread basis.
You need to use Filters and MDC (Mapped Diagnostic Context). This solution will only affect the logging that occurs on the thread that is executing the @Scheduled method.
Overview of steps
In your @Scheduled method, add an entry to the MDC of that t...
The Whiteboard
The Whiteboard