Maybe you can define a monad that can help you structure your solution: monads offer associative chaining of functions.

A few questions. (1) Are you applying a fixed operation f to each new image, or what does the function to be applied to each image depend on? (2) By operation Oi, do you mean the application of such a function f to image i? (3) When an operation Oi depends on the result of a previous operation O(i-n), how far in the past is this previous operation? (4) In general, can an operation depend on the result of one previous operation only, or on more than one (list of previous results)?

@Giorgio, I thought about this solution and did a bunch of reading ("state monad", in particular, got my attention), but I must admit I'm having trouble making the leap from tutorial to implementation. At the very least your comment is reassuring insofar as I'm on the right track -- If you have any specific ideas, I'd be very interested in hearing them.

@Giorgio, to answer your questions: (1) Yes, I'm defining a fixed operation to each frame. This having been said, it may be relevant to modify the state of the function object based on the contents of a frame, such that subsequent frames will be handled differently. (2-3) My Oi notation is actually quite ambiguous -- I mean that "an operation", O, is preceded and succeeded by other operations. Thus, Oi-n is the operation that happens n operations before Oi. In general, operations can depend on the results of several previous ones (see edits).

So, if I understand correctly, Oi indicate different operations on the same frame. So you never have an operation on frame j depend on a previous operation performed on an earlier frame, am I correct?

@Giorgio, that's essentially correct. More exactly, I never have an operation on input j that depends on the result of an operation on a previous input. I say 'input' because windowed operations are applied to groups of frames, but that's orthogonal to your point. We can indeed treat operations as being independent across inputs.

Excellent idea re continuing in chat :) I hope I'm not being too unclear as my FP vocabulary is a bit muddled. In any case, operations are independent from input-to-input. That input can either be a single frame or a group of frames.

Hi

hi!

And thanks, by the way, for your help!

I agree with amon's answer that you have a plain map. You can consider this map as a functor mapping functions on images to functions on streams of images.

In the case of composite operations on a single image, you basically have lists of operations. And a list of operations can be considered as a single operation.

Right, I agree so far

I am not sure I understand what the open questions are regarding to this.

But the actual act of chaining a function requires some checking and (possibly) reacting to certain conditions, e.g. if a prerequisite is missing I might want to add the missing function to the list

Now amon cautioned against doing this

but I'm not sure I understand why, actually. That's probably a good place to start

ok

I am reading your question again, so I do not ask the same questions again and again

What do you mean exactly with A -> B? A function applied to an image A producing image B?

No problem :) From amon's answer, I gather that he's assuming I may want to validate the chain of functions while it's being applied to input. That assumption is false -- this is something I'd want to validate before I even begin receiving data

I mean fn2(fn1(Image)) where fn1 returns a modified image

Or, if it makes more sense in a bash-y form, I mean: echo $my_image | fn1 | fn2

Yes, exactly that

And how do you determine this? You apply some predication (in turn, a function) to image?

So let me back up a bit, for clarity

predicate, not predication

sorry

ok

When the server receives a request, it'll come in the form of: "This person wants to run Analysis1 and Analysis9". The server knows that Analysis1 is equivalent to fn2(fn1(i)) and that Analysis9 is equivalent to fn3(fn1(i)).

So rather than repeat fn1, I'd like to compose a meta-analysis dynamically, so that the system would compose the following: fn3(fn2(fn1(i)))

Does this make sense?

The idea is to (a) avoid recomputing fn1 and (b) make sure that all dependencies are resolved before receiving data (and applying the functor)

And of course, the underlying assumption is that all of these functions return an image, thus making them composable in this manner. It seems like a good way to track dependencies for the composition step (again, not the application/fmap step) is with a writer monad that carries information about which functions are already in the chain.

I do not understand why you would do fn3(fn2(fn1(i))), this is not the same as fn3(fn1(i))

Or is it?

Ah okay, I see the confusion!

fn1, fn2 and fn3 each return the same type of instance as its input

But the input is mutable

So each of these functions will update a dictionary attribute

and in this example, fn3 and fn2 both depend on the thing that fn1 computes

Do you only change the dictionary or also the image data?

No

Ah hmm... actually that just threw a wrench in my gears. That may need to happen at some point...

Now that I think about it...

What need to happen? Changing the image data?

Yes. I haven't had to do it yet, but that's a requirement that may change

Sorry if this is so nebulous... I'm a bit out of my depth here. :/

and at each point you produce new copies of the data

Okay

By storing fn1(i) you would not need to compute it twice

So you could memoize the intermediate results and use them later, if you need them.

You could use strings as "fn1", "fn1-fn2" as keys.

mmhmm okay. Yeah the callable classes involved all have a .name attribute, so that makes things very easy

So, for the current image, if you want to compute fn8(fn5(fn4(fn1(i)))), you can look in your dictionary for a prefix that has already been computed

Ah, I see! You're suggesting I store small chains of functions as keys!

I like that.

Oh, is this what amon was referring to when he wrote about the compose pattern?

and so on

And what if I have fn1-fn4-fn5-fn8 and all I need is fn1-fn4 ?

maybe I should store intermediate results somewhere too.

This is starting to sound a lot like a tree

Or a set of trees to be exact. Do you see what I mean?

Or, what do you mean by tree?

I just mean that "fn1-fn4-fn5-fn8" and "fn1-fn4-fn6" could be represented as a tree that share fn1 as a root node and diverge at fn4.

So in this example, I'd start at fn1, go to the fn4 leaf

and then append the result of fn6 to that

but maybe I'm making this more complicated than it needs to be!

Alright in any case, I think this is a very good starting point for me. I don't want to take up too much of your time :)

No, I think it is a good idea. Only I would turn the tree upside down, like a syntax tree.

So I would have fn1 as a leaf with parent fn4, and so on.

Why is that?

It seems more intuitive to me because it is more common to represent expressions in this way.

My reasoning was that if I didn't find fn1 in my dict, then I knew the whole chain needed to be computed anyway, so I could abort the search.

Think about the tree for (2 + 3) * 5

2 3 and 5 are the leaves

right

+ is the parent of 2 and 3

* is the parent of + and 5

in your case, i is the leaf

ah ok, I see!

and the functions are parent nodes of whatever they are applied to

Okay cool! This is quite funny actually... I sort of had an intuition that this could be represented as an AST and I ended up reading "build your own lisp" in search of ideas :)

You want to share child nodes

so it is not going to be a tree

but it won't have cycles

because you have no recursion

so it is a directed acyclic graph

DAG

hmm you lost ma again. Woudn't it be a tree given that fn3 might branch from fn1 in your DAG illustration?

Actually, if you build a dictionary along the way, you are building this DAG from bottom to top

Oh, you are right, it is a tree because you only have unary functions!

Ah okay cool!

Alright well now I think it's time to punch out some code!

Using a dictionary you just build the tree from the bottom up

Right, makes perfect sense

each time you do not find your function in the tree, you add a new branch

Thanks so much for your help. You've just lifted a huge burden :)

I hope it works out, it is a cool problem anyway.

Yeah, it's certainly been stimulating!

And yeah I think it'll work out quite nicely

esp with amon's suggestions as well

I'm going to grab dinner and think about it for a bit longer, and then put hands to keyboard

good luck

bye

Thanks! Have a good one!