« first day    last day (16 days later) » 

4:47 AM
I'll try this in an hour.
@MikeL. Oh boi, I was burned yesterday! Thanks a lot.
MINUS C where C is CDF(x_min), right?
Also, the C will be accounted for in the inversion, won't it?
 
 
2 hours later…
7:01 AM
@Toxyd yes, except you drop the C when calculating C
@Toxyd It will, you'll just subtract it somewhere
 
7:20 AM
@MikeL. So it won't matter, we can actually ignore it.
 
@Toxyd Well it will, since it shifts the whole thing
 
um... ok.
 
the inversion will then be x=((y+C) (b+1)/a)^(1/(b+1))
you'd end up in the wrong interval if you ignored that
 
@MikeL. Also, the C is based in the same CDF or the previous CDF? Since up to the cut point we use the other function
 
@Toxyd you needn't care about the previous function once you've normalised everything
 
7:26 AM
Humm...
Wasn't normalization only necessary up to the point where I find out the intervals for each CDF?
So now after that point I won't be using the previous normalization any longer. I should be using the necessary normalization for each CDF in it's own. Right?
 
@Toxyd by normalisation I mean in this case scaling the CDF so that it has values 0..1
 
yep yep
@MikeL. It shocks me that the values of the PDF are normalized for the PDF accounting for the three piecewise functions (either input or output), and when acumulating for the CDFs and it's inverse, we can ignore the other functions... feels like it wouldn't add up correctly :P.
humm... the constant has to be substracted? weren't cosntants on intgrals added??
also, we are talking all the time about x^(a) but it's x^(-a)
@MikeL. The inverse of it's integral has only imaginary solutions under some cases :S
 
7:53 AM
@Toxyd this is not the same constant as pops up in the indefinite integral; that disappears when you apply the newton-leibniz formula
@Toxyd that shouldn't matter
 
8:11 AM
Hum... So once we have the cut points, the x ranging from 0..1 to feed to the chosen chosen inverse CDF... the only thing that matters is CDF^-1(x). In order to make it range from 0..1, we first have to modify the CDF so between it's limits it only spits out 0..1, then invert that modified CDF.
And those inverses of normalized CDFs are the only functions that will be actually used to generate the nice distribution we are looking for.
Looks that I'm clear now
@MikeL. Thankx for all
I'll now proceed to implement it. ( Fail and come back xD )
 
8:32 AM
@MikeL. The C (on the CDF) for the first CDF, should be the value of the same CDF for 0.0 ?
As opposed to the greater CDFs, in which C is the value of the previous CDF for their lower limit. Am I on the right track?
 
8:55 AM
@Toxyd If you have that piecewise PDF, then technically you have two constants there
the first is from the Netwon-Leibniz formula and is the value of the CDF integral at x_min
that's subtracted
the second is the high value of the preceding CDF, that's added
but you only care about that if you really want to have everything in a single function
 
9:07 AM
@MikeL. Oh, yes, I'm referring to the 'added' C, not the one from the indefinite integration.
So, for the 'lowest' CDF, it will be 0? Or CDFlow(0)?
 
 
2 hours later…
10:55 AM
@Oxy 0 - CDF (x_min)
 
Oxy
11:21 AM
@MikeL. Let's say the last CDF ( CDF3(x) ), has it's x_min = 0.6 and it's x_max = 1. I don't need to have it all in one function.
The inverse CDF3 is -( 10 / (13 · x^1.3) )
The CDF3(x) was without considering the correction factor_ CDF3(x) = (x^(1-2.3)) / (1-2.3)
a correction factor (multiplying the result) of 0.1966 brings the CDF3 into 0..1 range
the inverse CDF needs to return 0.5 to the input 0, and 150 to the input 1, and the range in between shouldn't ramp up much until the end.
considering that I have a random number between 0 and 1, and only with what I said since your last intervention, what should I do exactly?
The normalization to make the CDF inputs 0.5 .. 150 fit into the 0..1 range should be done in the CDF, right? And carry them onto the inverse.
And that should be it. The inverse of the corrected CDF should be able to be fed any value from 0 to 1 and should spit out values from 0.5 to 150
That's what the inverse is all about.
Humm..
 
Oxy
11:56 AM
I think I'm more lost on every step >:(
Every time I think I have it, I run some numbers on wolfram and I get either nonsense, or imaginary roots! xD
I think I'll redo it all from the beginning with a super simple function (non piecewise), see it working, then try to apply the piecewiseness.
Do you know any simple function whose CDF does already fit into the 0..1 range? I'll practice from there. I'll implement it all, following your previous instructions. If I succeed, then I'll try with a single of my PDFs, then if I succced I'll try with the piecewiseness :)
You should charge me for all these!
 
12:38 PM
@Oxy Yes to this
@Oxy go with e^x in some range
that has the advantage of being easy to integrate and invert
 
 
5 hours later…
Oxy
5:50 PM
@MikeL. This was quite the key...
for the last cdf ( CDF3(x) =-10/(13 x^(13/10)) ), once fitted so it's limits (0.5 and 150) return 0 to 1; ended up looking like this: y =1.00144-10/(13 (x*1.63305)^(13/10))
which is a simple multiplication to the input and addition to the result
Looks clean enough
And also escalates super quicly on the beggining, which, when inverted, should give me a nice quick escalation on the end
 
Oxy
6:08 PM
I'm using wolfram to speed up the process (doing it by hand or with a sci calc... would take ages. Soo easy to just input 'invert me f(x), and then tweak the results...)
I have to acknowdeledge that without knowing exactly what you are doing, wolfram can confuse you a lot
It approximates without telling, it transforms to exact forms when you didn't ask, and so forth
@MikeL. As you said you make a living from this, I imagine you spend some time transforming equations, as I've shown how I up here... what do you use for this??
Pure instinct?
Or do you have a more reliable tool to just mess a little to find out the behaviour of funtions and it's tweaks before getting into fine details?
Wolfram gets a little... don't know how to say it. Microsofty? Deciding what's important for you and not letting you touch anything... though it's the only alternative I've found to manually do every bolt and nut
It's not a problem for me to invert or integrate (though I think you thought that it was) functions like the ones in hand.
@MikeL. the problem was that I did not get it until this last intervention of yours.
but on some of the functions I have ahead... well, I don't think I'll be able to integrate manually half of them, much least explore their behaviour with wolfram (it refuses to even plot them, or solve, invert, or integrate them in any way). While solving and inverting wouldn't be a problem, exploring them fast enough or integrating them would pose a problem without a tool a little more reliable than wolfram. Tha'ts why I'm asking
 
7:02 PM
@Oxy Well, actually having something know, nice, clean and integrable to process and invert is so vanishingly rare an occurrence that it's completely feasible to just do it by hand.
Usually we deal with distributions that are at best known up to a multiplicative constant, but usually non-integrable, unknown and have too many dimensions to count. Then we just manipulate expressions on top of these, and being able to do a simple CDF inversion is almost a cause for celebration.
As in, being able to do that for one of the actually interesting problems gets you a journal paper more often than not.
 
Oxy
Awsome!
I used wolfram to explore, I did it all by hand at the end because wolfie likes to 'simplify' even things that can be expressed exactly by a simple division...
So I assume you use some nasty tools to explore on those complex equations
 
@Oxy It's not even equations, usually.
 
Oxy
Or do you end up implementing them, so you control everything?
Oh yeah... I imagine... that real world applications are rarely expressed as neat equations
 
The most common example in graphics is light transport where you basically need to run a simulation to evaluate a PDF at a point, and that's what you have to go on.
 
Oxy
7:22 PM
but having discrete points (ie.from a simulation) isn't easier to treat them like discrete distributions??
 
@Oxy No, that wouldn't get you the right results (since the space is really continuous) plus you don't even get to evaluate a whole lot of those points.
 
Oxy
I suppose not, or that you need the actual continuous functions at the end, of course, if it was that easy you wouldn't be working on it as a job :P
Well... I hope you well in these endeavours :D
I'm starting to get a grasp of the basics.
to get the result of the 3rd CDF, I 'moved' the function vertically to fit the highest point, then 'expanded' the function horizontally to fit the lowest point
I found that working on two independent axis made easy for them to be ok at the same time
Thoug... I get the feeling that I should have 'shrunked' it vertically, instead of expanding it horizontally
Is it right?
How do you manage to make both ends fit, ehn modifying only one axis, if when you modify it somehow, the other fit gets lost?
That's my only doubt now. Though the function has the adequate behaviour, so I'm not deeply concerned.
 
8:11 PM
I'm not really clear on what your problem is from that description
 
Oxy
9:03 PM
I'll try to come up on a solution alone, it's a problem specific to the case :)
Thanx for all! Really :D
 
@Oxy In general, and this is high school stuff that it maybe just hasn't occurred to you to employ, you'd use multiplication to transform a function so as to shrink or expand it both horizontally and vertically
given f(x)=y, f(x*a) shrinks vertically by a factor of a without affecting the vertical part at all
f(x)*a expands vertically, again without affecting the horizontal part at all
 

« first day    last day (16 days later) »