interface Action<E> { public void execute(E target); }
private Action<LevelContainerArray> forEachLevel(Action<LevelContainer> thingToDo) {
  return new Action<LevelContainerArray>() {
    public void execute(LevelContainerArray target) {
      for(LevelContainer l : target) thingToDo(l);
    }
  }
}

private cancelEntriesAndExits = forEachLevel(LevelContainer::cancelEntryAndExit);
private handleEntry = forEachLevel(LevelContainer::quote);
private cancelEntries = forEachLevel(LevelContainer::cancel);

can even do all that in the static constructor

handleEntry.Execute(levels) etc - you can even pass handleEntry, cancelEntries, and cancelEntriesAndExits around polymorphically because they're all the same type (Action<LevelContainerArray>)

don't actually know the appropriate type name for what you would do that for loop on in java, but I'm pretty certain that should work. If not there's other general approaches to it like if the :: method access on a type isn't available

new Action<LevelContainer>() { public void execute(LevelContainer target) { target.cancel(); } } for instance, but then... well it's all just as gnarly as the rest of it

welcome to Java @durron597 - and why the industry is full of people bitching about it these days; it takes that much messy expansive boilerplate just to do some simple things because Java lacks abstractions...

@psr leanpub.com/purescript/read#leanpub-auto-hello-purescript

@JimmyHoffa Usually causes more trouble than it solves. What happens if customers gain access to toggle features? Minimally, it's a business problem. Worst case it's a legal or regulatory problem. Not to mention test coverage for different feature sets will explode.

T-SQL is a terrible, terrible language.

They have abused it here beyond all hope of redemption.

@ThomasOwens I guess I misunderstand the feature. I've put in toggles for functionality tons in software I've written; turn on logging, turn off logging, toggle X authentication, Y authentication, toggle caching, toggles for all sorts of things...

@RobertHarvey haha nonsense! It's a great language! Language abuse is not the languages fault, that's just victim shaming. Blame the abuser.

nobody knows SQL for a good goddamn these days... I can see it now, 5-10 years from now quicksort will be out and it'll be all "The interviewer asked me to whiteboard a select statement! Honestly, who's even done that? That's what libraries are for..." and I'm going to be the grumpy prig complaining about how no one takes the time to learn their data structures bases anymore

the T part of T-SQL is kinda horrible, but it was a horrible idea from the start.

@Telastyn ? how so? pl/sql and t-sql are just six of one, half dozen the other to me, when you get beyond actual DB features themselves like table variables and table values parameters

bolting an imperative language atop of a nice lean declarative query language?

@Telastyn have you looked at purescript there? Pretty neat. The quick walk-through book I found there online actually explains a bunch of Haskell concepts quicker and more easily than most all haskell stuff I've read... on the other hand maybe I just understand it all at a glance and it's not actually explained any better

I have not. Is it yet another better Javascript that still compiles down to javascript?

@Telastyn performance favors imperative, and RDBMS need to perform... there's a reason you immediately pick up STRef in haskell as soon as you actually need to do something performant and eager.

meh.

@Telastyn something like that, but just scroll quickly scanning the examples here and try to spot something that isn't almost identical to Haskell... leanpub.com/purescript/read#leanpub-auto-hello-purescript

semi-afk, video games.

word. I think I'll be popping up Metro Last Light momentarily..

I especially love that the introduction to Applicatives is with Applicative validation, which is frankly the best damn validation API ever...

@Duga hah he actually quoted the guidelines. Now that's a suggestion I can get behind

fuckin aye @gnat you're fast.

I also like that dude called mods "monitors" from now on, @ThomasOwens you are a a monitor; would you like to be LED or LCD?

@JimmyHoffa LSD?

@gnat considering who it is, perhaps...

in the sky with ♦s

stackexchange said i will get reputation point for participating here, so i am here i dont know how can i will be benefited here ,any kind of help is appreciated.

@user143252 why do you need reputation? What kind of help are you looking for?

there's no rep to be had in joining chat, just knowledge.

granted, one helps you get the other, but that's neither here nor there.

i joined stackexhcnage as i start learning java , is there any chat room specific for java .

hmmm

good question

@user143252 yes

really?

I know there's one for C, but those guys are dicks

@naumaan chat.stackoverflow.com/rooms/139/java

thank you sir .

@Ampt Happy coffee day!

wow the chat room search is awful

i have one more question , so what kind of chat room is this , i mean if it is not for java ,so what you discus here .

@user143252 just about anything related to be a programmer

from coffee to monads

@user143252 silly animated gifs, apparently

are you people are students or profesioanl programmer

those only happen after a few beers usually though

the topics of this room in the top right corner

I think 99% of us are professionals

we may have some academics in now and again though.

If you ever want to chat, we're here :)

@user143252 how old are you?

thanks , this is first time some answering me , most of time stack exchange put my question hold.

We're pretty relaxed here

also, what is your first language?

feel free to pop in and drop your questions - no need to ask

i am graduated in Electronics engineer ..... 21 yrs old. I am from india

usually we'll answer them if they're interesting

My first language is Hindi

:22805931 or just the claim...

What university did you graduate from? I've got a few friends/coworkers over here on visas

or just the claim... ... i dont understand .. pardon

nevermind, I have no clue which ones they went to

@user143252 we usually have multiple conversations going at once, just have to try and keep up, sorry :)

@user143252 I think one way to get more people to give you attention on the main site is to be more careful with grammar and spelling

actually his first and only question here was fairly well received

3 up / 1 down

I asked your age and your first language because of your grammar and spelling errors, no offense

sorry, the answer had 5 up

@durron597 OFFENSE TAKEN!

OVERRULED

So , Do you know java .. can you tell me what where and how to start ...

how and where to start?

I like the head first books personally

i have downloaded lynda java course and watche 22 videos

but the language barrier may pose problems

yes ... ma'm

The best way to start is to just to try use it

How good is your English? is it tough to watch videos?

you may want to look for resources in Hindi then

then when you have specific problems, search for them, if you can't find them, you can ask them on stack overflow

It's hard enough learning to think like a computer in your native tongue, I can't imagine doing it in hard mode

when i go to java chat room ..... "You must have 20 reputation on Stack Overflow to talk here. See the faq."

i can watch , understand , and little speak english .... thanks to hollywood movies

@user143252 It looks like you have 20 rep

try again?

@user143252 I recommend that you start trying to improve your English before you start trying to learn Java.

Most of the educational resources are in English

@durron597 you sure?

there are a lot of programmers in India

I have no idea how to even find those resources, but I'm sure they exist

yes .. india is IT hub...

@Ampt All the active ones on Stack Overflow have very good english.

all the keywords in Java are in english. all the oracle documentation is in english

@user143252 where in India are you?

..... i am not complete idiot or child.... :)

We are both Sir lol

thats why i edited

of ccourse I'm a sock and he's a building brick, so who really knows

I am not saying these things to embarrass or insult you, I am saying these things because Stack Overflow has very, very little patience for even grammatical mistakes, let alone spelling and punctuation mistakes

This was posted by a Community Manager TODAY

> Edit. Whether you like it or not, a tremendous amount of knee-jerk moderation is triggered by plain old bad writing. If you care about a question, taking a few minutes to make it read as though it was written by an educated adult instead of drawn on a wall in crayon can make a huge difference in its future prospects.

i learn c language in B Tech course, have basic training in c++ , i have made my app .. using construct 2 engine .... , i know asselmy language very well

sir , what is this ?

@user143252 If you ask a question on Stack Overflow and you write "asselmy" instead of "assembly" your question will be closed and down voted. It is very simple.

hmm.... ok sir , i understand .... thats the reason for worry .. i will keep this in mind ........ agian .. thnaks

sed s/agian/again/ and sed s/thnaks/thanks/

I'm sure I sound harsh but I really am trying to help you.

as i am typing .... its happening due to speed......... i apologize.... ... one more thing .. is that sir , as my native language is not english so i dont know choose the correct word to express my feeling ........

@Ampt that conversation in the Java room is like trying to explain to a 3 year old why they can't have ice cream right now

It's just doomed to failure from the outset.

i am still barred to enter in java room.... thanks a lot both sir .. for helping me ... i will be later to ask more question ..

Good day :)

Have a good day!

@durron597 I've had one too many drinks and have been nerd sniped

send help

@user143252 I recommend three things: spend more time being careful / looking things up and so forth. 2. practice!

@durron597 is the third thing finishing your lists?

3. when you do post on Stack Overflow again... DO NOT explain all this stuff about not being a native english speaker. Just do your best to state the problem in as few words as possible, but with all the information necessary to reproduce your problem.

yes sir :)

@AshleyNunn HAPPY COFFEE DAY!

ok serisusly I need to go to bed

goodnight.

@Ampt drink a coffee so you can spell seriously better

@Ampt yup

@Ampt what's your poison?

@durron597 a small city near Delhi

@user143252 Do you speak Hindi or Punjabi (or both)?

I have to go to sleep. Good night

sad to see an upvote on this. Kinda says, "keep doing this buddy. Next time your code breaks, dump your question again both at SO and Programmers, it's okay"

@JimmyHoffa It sounds like you understood right. Logging should be handled by a logging configuration and logging levels, I guess that's a form of a toggle to determine if a message at a given output level should be written out to some output or not. But toggles that deal with functionality are rather dangerous, IMO, to business or legal requirements.

@ThomasOwens product features as provided by a deployment to a paying customer, yes; however functionality of a piece of software should be configurable; but functionality and features aren't the same things

@JimmyHoffa I don't see the difference.

Between functionality and feature, that is.

@ThomasOwens authentication is a feature, the way it authenticates is functionality; what if this deployment will be databaseless? A switch to turn off the local DB auth and turn the LDAP auth on would be handy... authentication is still an available feature, the functionality was just changed

@JimmyHoffa Would you consider the method of encryption used to be functionality? And you would allow for configuration of the method of encryption used?

@ThomasOwens yeah, that kind of thing

I always put stuff like that into config settings

not to turn off features for clients, but to allow the configuration to change with circumstance

@JimmyHoffa I'd disagree. There are some encryption methods that can't be given to certain people. Therefore, it just can't be in the binary that they have access to.

@ThomasOwens ok well you're talking about packaged saleable software, which like 1/20th of people work on

that's a totally different scenario

Well, yeah.

If you're running a service, I'd be more in favor of configurable.

I always think in terms of SAAS because frankly, boxed software sucks comparably. For consumers and developers.

(Did I mention that? I know I was thinking it before.)

Even our software that runs as a service is deployed. We have field service to install and run it, but it's on customer-owned hardware and other people have access to the binaries.

Other people being people who aren't employees of my company.

yeah I've worked on that stuff before. Again, I wouldn't sell different config settings at tier prices, but they have configuration options... Obviously not ones that would be dangerous like there would be no configuration settings allowing a user to completely desecuritize their system, but settings are still good so you can resell the same product to lots of different folk without having to maintain lots of custom client specific versions

Some configuration is required. But it shouldn't be configuration that turns things on and off. It should just be inputs.

the idea that making some piece of your software available but turned off is bad because users could hack up the code and meddle with it is silly - I can take any .NET assembly (probably JAR too) and use readily available tools to replace entire methods/classes with ones of my own inside the assembly because they're both intermediate languages and tools are available that let you do that.

You can't protect yourself from users doing messed up shit with your software

Including "turned off" things in a compiled codebase that outside entities have access to is just a bad idea. If they were motivated enough, they could work around these configurations and gain access to this "disabled" code. The only way to do it is as compile time - either compile time configuration or by selecting appropriate libraries.

ICK NO NO NO

@JimmyHoffa That's very true. You should take appropriate precautions though. Plus, there's an IP issue.

compile time configuration == EVIL

@JimmyHoffa It's safer than runtime configuration.

NO

absolutely not

wrong

far less safe

FAR less safe

What do you think compile time configuration is? Because it's one accepted practice for building a product line.

trust me when I tell you, using your build to split up compiled functionality is a spectacular way to end up releasing software that was never tested because switches were one way for devs, another way for QA, and because of mistakes which are rampant in build and release processes, a whole different way for production

Build switches swapping code in/out are the best way to find QA complaining and no devs capable of reproducing an issue

build switches should not effect the code being built, only the way in which it's built (optimizations, PDBs, obfuscation, etc)

That seems to be a CM issue. I don't prefer compile-time configuration. I would prefer component level replacement.

you should have the same code available from dev, QA, and for clients, or else you are going to have serious problems

@ThomasOwens this is definitely a better approach using plugin style stuff. But now you're back to - it really doesn't matter for non-packaged software

I'm just wondering how much software really isn't non-packaged. What the breakdown actually is.

and seriously trying to protect yourself from clients meddling with the code in your application when they get it... it's just not going to be a fruitful endeavour

@JimmyHoffa It's a requirement, though.

there's lots of packaged software, but the internet is so much bigger...

@ThomasOwens no, that's nonsense.

you can call it a requirement for people you get to make requirements for, I'll be glad I'm not one- it really is not a good concern to have.

happy coffee day, I just fresh ground some beans over at the grocery store in coarse for my french press since you mentioned it; let's see how much better "fresh ground" is, as everyone seems to agree it's far better...

@JimmyHoffa or ensure the client can't toggle build switches and ensure QA tests with the switches the client gets

@JimmyHoffa militaryaerospace.com/articles/2010/04/…

One of the requirements that has been driven down is that build binaries do not include any code that does not get executed during the execution of the software.

I hate configuration. Makes code coupled, fragile and overengineery

Component replacement is harder, but it's more elegant and more testable.

@ThomasOwens I wouldn't quote requirements from such a bureaucratic industry as being ones that are good ;P

@JimmyHoffa I think it's generally good to keep in mind, though. If you have a closed-source project, you want to minimize what things someone can see about what you're capable of doing.

also, the way that requirement reads; no code that doesn't get executed ?? That's an absolutely horrible requirement. They're basically saying your software must all be written in one procedure

@JimmyHoffa I'd have to look at the exact wording, but it explicitly deals with runtime configuration not being acceptable for controlling what features are available to specific users.

You can use runtime configuration for things like setting up default values and so on. But not for toggling or changing functionality.

@ThomasOwens nah, closed or open, you shouldn't clutter your concerns with crap like that- writing clean well architected, modular, maintainable code is a far better way to ensure you can provide features to your users, and if you don't want to release all features to everyone; use plugins.

@JimmyHoffa I agree with that. How do you state that you drive people to use plugins? By limiting their options from things that aren't plugins.

You can't say "you shall use plugins". Because that would exclude other possibly good and acceptable methods.

@ThomasOwens I still think you just tell them to plug into the wall jack, or steal their neighbors wifi, whatever, just use your login credentials and a good authorization system on the server will handle the available features. SAAS > packaged software. For packaged software a plugin feature will almost be a requirement just because it's the best way to make your application act as a fault tolerant host

even if you don't use it for the feature enable/disable functionality, packaged software should pretty much always be done with a plugin architecture to safely and easily handle extension for your developers over the course of time

makes deployment far easier, it's the closest you can get to the nice clean boundaries SAAS software gets by having multiple services communicating over IPC without actually using multiple processes or an arbitrary local IPC mechanism (desktop software should avoid IPC wherever possible, puts too much requirements on the configuration of the users machine)

Totally agree. I think the big difference in decision making is how you make your software available - service or packaged.

So you really have different criteria for evaluating your options for both.

On the subject of plugins...

How do you guys feel about storing things like SQL statements, XPath, etc. in database records?

edgy

As text?

Yes, as text. And then using them in code contexts.

@RobertHarvey .... it's called a stored procedure...

Ah, no.

Well, for SQL statements, why not used stored procedures?

Because it downs the server.

O_O

what?

^

It doesn't down the server if you have to put a new one in?

That was literally my face and what I said outloud.

Stored procedures?

No. You can add and drop them in MySQL and Postgres, anyway.

Well then why the fuck are they storing them in tables then?

They are being chosen dynamically, through an Entity Framework call. Does that make a difference?

haha no oh gods your conclusion would be so much nicer

That I don't know.

    [MethodImpl(MethodImplOptions.NoInlining)]
    public static DbRawSqlQuery<TElement> SqlQuery<TElement>(this IVerificationEntities entities, string statementName = null, params object[] parameters)
    {
        statementName = statementName ?? new StackTrace().GetFrame(1).GetMethod().ToString();
        var statement = GetSqlStatement(entities, statementName);
        return entities.Database.SqlQuery<TElement>(statement, parameters);
    }

private static string GetSqlStatement(IVerificationEntities entities, string statementName)
{
    return entities.Database.SqlQuery<string>(Statement,
          new SqlParameter("statementName", statementName)
        , new SqlParameter("status", Statuses.Active)
        , new SqlParameter("statementTypeKey", CodeResourceTypes.SQLStatement)).FirstOrDefault();
}

"They store SQL in the DB... I guess sprocs must down the server" no, the real conclusion is "I guess, they are absolutely mad"

I prefer your first conclusion, though it's wrong

There are posts about using stored procedures in EF.

EF works just fine with sprocs

even manages their versioning

Check out the code above. The second statement grabs a bit of SQL from a specific table the database, given a name from another column.

oh gods that's just nonsense... utter and pure nonsense

That doesn't seem like what you want to be doing at all.

@RobertHarvey does all the SQL have no parameters?

No, it has parameters. See the params object in the first bit?

also the reflection...

somebody literally knew nothing about databases, was given that project and after meddling for a bit decided "Wow! Databases are cool! I can think up all sorts of neat things to do with these!" without having any actual mentorship or experience in how you should use one. Between that and the C# in sql tables, he's using the database like a 3 year old would use a lamborghini: It's so fast! I wonder how it handles in the grocery store? I can turn on all of the knobs in the center! Look!

@Ampt telegraph.co.uk/news/worldnews/asia/bangladesh/11411766/…

@RobertHarvey ...but how does it know what params are necessary? A sproc would verify it, for all you know that code is handing in too many params some times, and other times missing necessary params...

please for the love of god tell me you're actually using commands to execute the statements with parameters when it gets to actual execution

entities.Database.SqlQuery is an EF call.

It's a bit of Microsoft magic that takes the output of whatever SQL you provide it, and maps it to a new object of the type you specify in the generic parameter.

I want to talk about the XPath, though.

I'm torn about that one.

Each page to be scraped is stored in a "service call" record. There are related records containing the names of the DOM elements to be retrieved, and the XPath needed to retrieve them.

When he gets it all working, he writes a script containing UPDATE queries that push the needed records into the production database.

That seems convoluted. But you do need to store the XPath query strings somewhere. A database probably provides decent lookup functionality to find the right one.

It seems to me like, if you're going to do that, you write a utility with to build up the records and write the script for you.

@RobertHarvey here's what you need to do: put a logging line in the DbRawSql method: "CREATE PROCEDURE [" + statementName + "] @p" + string.Join(", @p", params.Select((param, idx) => idx.ToString() + " " + GetDbTypeFromType(param))) + " AS BEGIN " + statement + "END"

@ThomasOwens None of the XPath is reused. There's one per field per page.

anything wrong with hardcoding them?

then let the software run for a few days, grab all that from the log and execute it in the SQL server

@ratchetfreak That was kind of my line of thinking.

let that run for a while and you'll create stored procedures from all of those

You could query the table directly to get the sprocs made but then you'd have to parse them for parameters... heck you may have to do that anyway

@RobertHarvey Wouldn't that mean rebuilding the software and restarting it every so often?

@ThomasOwens Yes, but that happens anyway, in practice.

@JimmyHoffa suggested that we just host the WCF service in IIS, so we can change out DLL's on the fly.

Oh. If you're already doing that, then maybe it would be better to put them into something. Either hard coding or in a resource file that's read.

The problem that I have with this is the sheer amount of indirection. It makes it very difficult to work on a system like this. And the Guids. OMG, all the guids.

And the Interfaces. Sooo many interfaces.

Anyway, I'm done ranting.

heh wow... I hope they're giving you free reign to do your work however you need

@JimmyHoffa Don't even get me started on logging.

haha ahh good times...

really though, I'd just sit down for an hour and write up the query that selected everything from that table in a fashion which concatenated strings to spit out the necessary SQL for creating a stored procedure from every one

then when a call came in to execute a named SQL statement, you just execute the sproc with the given parameters

@RobertHarvey Please, have a seat right here and tell us about logging.

@ThomasOwens I hope today is coffee day for that one.

@ThomasOwens coarse ground in french press = coffee-- :/

@ThomasOwens It's just half-assed, that's all. So much noise, and it doesn't write anywhere. There's a SignalR endpoint that goes to one guy's computer, but logging never occurs unless he's got that little program turned on.

stick with regular grounds, much less watered down

This will give you an idea of what I have to deal with. They have unit tests with test cases containing three Guids. Where do the Guids come from? Turns out the guy had a hotkey script that generated a SQL statement that, when executed, produces the three Guids.

One of the things that I've always wanted to think about is the right approach to logging.

There are so many ways to do it so very wrong.

Getting meaningful information out would be a good start.

@ThomasOwens The only right way to do it is AOP. It's like, the definition of a cross cutting concern

unfortunately, what I do instead is have a private static final Logger in any class where I want to log stuff, and I have an eclipse shortcut that generates the logger object for me.

@durron597 Meh, it's just a call to a logger. You still have to put calls to that somewhere in your code. You can inject the logger if you like, but it's not like you can stick an attribute above your method calls and it just automagically works.

mostly because SLF4J allows me to behave badly with static bindings. Yes, I'm blaming the language, @JimmyHoffa

@ThomasOwens I've been privileged enough to do it multiple times now in different applications and really think I've got quite a good approach now. The most important part of logging is the part everyone gets wrong or ignores: CONTEXT. Every log record should have a context identifier that can be used to trace it to an entire context of sequential actions - maybe the context starts and ends with a login, when I worked on a phone system it started and ended with the call

@JimmyHoffa My poison was a stout called Ivan the Terrible

@Ampt haha, russian imperial stouts can definitely do that...

Yes. Yes they can. Happy Coffee Day!

I don't prefer those, but Old Rasputin really is quite nice

@durron597 that's only trace logging...

@JimmyHoffa How would you go about doing that? In most places where I worked, logging didn't get put in until after you needed to know a particular thing. And then you fine-tuned it as you find out what you actually need from it.

@JimmyHoffa I have a class. My class makes widgets, that is it's job. If my class also logs information about it's widget making activities, now it has two jobs. SRP is violated.

@RobertHarvey the important part is coming up with a concept for contexts that makes sense in each place, and then figuring out how to create/remove them and when. In a SOA system I'd make all my services check their requests for a special context header, if it's not there, they create a new context, if it is, they continue with the given one

Wouldn't you have to record your actions somewhere for awhile, and then link to those actions if something bad happens? I know there are things like stack traces, but shit.

thread storage is usually a safe bet - I'm a fan of putting a simple dictionary in thread storage and an interface for accessing items in your thread storage as a dictionary

my logging looks a lot like this:

public void doStuff(myClass myObject){
    log.log("public void doStuff(myClass myObject)");
    myObject.doSomething();
    log.log("myObject.doSomething()");

That's trace logging.

Whatever logging we do, it needs to be meaningful from a business context.

@RobertHarvey That's how I've seen it on projects, too. But I think that it may be better to consider logging and other debugging / testing aids at requirements time. That includes profiling and timing, logging, and anything else.

"This website didn't scrape because such and such is missing."

the key is then you can go back and find all your exceptions in the logs, and track back up the context to see how the scenario was reached

@JimmyHoffa Ah, that looks a little bit like a transaction context.

Someone must have written about this?

@JimmyHoffa That's one reason why I almost always implement a toString method on my objects.

Right.

It's not to make it human readable. My user interface classes can handle converting from an object to a human-readable format of the appropriate type. The toString exists for logging and debugging purposes only.

nods

And then, I just log the object and bam, I get a log message with its current state. Plus, you can autogenerate toString methods that do that insane easy in any IDE I've ever used.

In this shop, you say "Let's sit down and discuss how we can have meaningful logging," and the response is "Fsck it, let's just get some logging and make it pretty later."

@ThomasOwens no! The key is the data dictionary is queryable. It needs consistency for this, so it has to be a literal list of key, value columns in some queryable structure (mongo document, SQL columns, something) so you can find all events with PasswordLength = 129 to find out if that's the cause of the exception you had

real-time monitoring software can look for keys and values, but to make it parse a string... no

@JimmyHoffa That's what a toString looks like for me.

What are you querying for? Are you just searching error messages for some string?

Surely it's a bit more structured than that.

@ThomasOwens strings aren't queryable...

But what is the key in the Data Dictionary?

you need a list, of key value pairs. A structured list - of a structure with distinct keys and values

@RobertHarvey typically strings, can be enums, doesn't matter - in storage it'll be string key string value, but you can query on those key strings

@JimmyHoffa Class[variable=value, variable=value arrayVariable={val1, val2, val3}]

It's extremely well structured.

@ThomasOwens ...that is not queryable.

that's a string

If you put data into a log file, you have strings.

If you log to a database, it will be queryable. But you would write an appropriate log adapter.

@ThomasOwens log files are fine, but you need a queryable structure. Log files can't be queried

OK, but... What are the keys? What's in them?

@ThomasOwens pah! let the adapter turn structured data into a string, don't make it try string parsing

@JimmyHoffa Well, yeah.

You're logging an object. I don't pass a string in, I pass the object.

So if the adapter is writing text, it invoked toString by default. If you're logging to a database (or some other data source), it actually has the object and can do things with it.

Well, you can serialize an object for logging purposes.

If you really think you need its state.

What are the goddamn keys?

@RobertHarvey depends, like if you're trying to execute a deposit, you might have:

So a JSON serialization of your object, then? More or less?

then you could go in and query it later to find all contexts where isGift was true, and see how many completed the deposit during their context

@RobertHarvey no, key, value, in a list

Alright. But why go to all that trouble?

Is it so you can track the values by time/step in your context?

@RobertHarvey I just said why. It's not a lot of trouble anyway. A table in mongodb, or sql, or otherwise with a key and value column?

You said because it's queryable.

@RobertHarvey exactly. Because that's how you find out what the steps are that cause various scenarios.

And it's queryable because you only want to look at values for one particular key?

@RobertHarvey right. so you can go find all the scenarios which occurred in a certain way to know if/when something broke and why

find all the deposits where type is Visa are broke, or all of them with Visa and amount > $500 result in a rejection and now when you get all these complaints of intermittent deposit rejections, you have an answer

I'm trying to think if that would help us. If a web page breaks, that usually means that you get back either bad data or no data. Mostly, our problem is we have so little visibility that we don't even know that happened. We just know a scrape didn't take place overnight. We don't even know that really. All we know is that some customers have bad indicators.

We have no visibility at all.

So even getting rudimentary logging in place would be a big step up for us.

And since we're fighting so many fires, it's difficult to get that done.

It's not a complex structure, but these 2 things make finding out about your system so much easier... context, and an arbitrary structured data dictionary. You can find out what the most common usages are of your system at a high or low level. You can inspect timestamps to find out precisely which steps in the system take the longest, and in what scenarios they're fast based on the data attached to those steps (AMEX deposits end up taking 3 times longer than any others!)

So it's a single table in a database, right? What are the columns?

It's great for you as a dev to diagnose and write live monitoring software, and it's great for the company to identify system behaviours and usages to make business and product development decisions off of

That sounds promissing @JimmyHoffa

You need more than just Key/Value don't you? You need context.

@RobertHarvey I actually do 3:

Now we're getting somewhere.

It's a simple structure, and very easy to use, but robust in how you can query and work with it

And you just hit it with Log() methods?

from a code standpoint it's just class Event { public int|guid|whatever ContextId {get;} public (string or enum) Description { get; } public Dictionary<string, string> Artifacts { get; } }

create a few simple methods to make it easy to create the object, you can put a .ToString() on it to make nice lines for the logfiles, otherwise the structured object goes to a repo which generates the timestamp, pulls the context or creates one if there isn't one... etc

again, most important of all is that your logging has that context so you can follow a sequence of events...

This is a really good starting point. Thanks.

I wonder if this is a good argument for lightweight databases in applications. I don't know what .NET has, but Java has things like Derby.

Depending on how you structure it, I think you can just as easily send a Derby DB back from a deployment environment for debugging. Of course, if you're running a service and already have a database of some kind, just use that.

How much proliferation of logging data do you get? Do you run this all the time, or only when you need to trace out a problem? Do you keep, say, two weeks of history?

Does teh databasez fill up?

I assume that, instead of injecting a Logger, you now inject a ContextualLogger with the context pre-populated?

That's a interesting notion

And do you have a severity field, like CRITICAL, ERROR, WARN, INFO, VERBOSE?

@RobertHarvey those are pieces of data on the event; you can have it if you want. Or you could just make them an event type so that value goes in the description field instead of dangling off the event. It's all very loose and up to you how you use the structure, as you work with the structure you'll come up with what data belongs where to serve the purposes you need. I leave this stuff running 100% of the time, and if database is going to fill; run a job at whatever frequency is necessary

delete or archive the data off somewhere else depending on the available disk constraints

Got it.

And how do you set the context?

I like to broadcast this stuff too, so you can have subscribers on a pub-sub MQ like activeMQ monitoring these things and processing particular types of events when extra processing makes sense. For instance I wrote one MQ listener that watched for exceptions, counted them over time and if it got 5 within a minute, would send an e-mail to notify that there's something up - where the number per time period were configurable fields

In the phone system I worked on, one listener took this data and inserted certain pieces into a legacy database that needed to be maintained with current data for legacy applications. This way the standard event publishing flowing out of my application was all I had to write, and the legacy concerns were totally separate, off in another process

@RobertHarvey depends on the context - in the phone system I created one when I got a call, and stuffed it into the same place all of the call session data was held. Most applications have some sense of session, if not you can stick it on your thread with [ThreadStatic] public static int ContextId or require it to be passed into constructors

@JimmyHoffa Control flow via exceptions. The purists would spit on your grave.

I found a third company that offers acceptable education benefits. It's just a lot more restrictive than my current company. They do offer other educational/development perks, like paying for membership in professional organizations. More companies need to offer tuition reimbursement (or, even better, up-front-payment), professional organization membership, and short course or conference costs as a benefit.