The h Bar

12:00 AM

In signal processing, cross-correlation is a measure of similarity of two waveforms as a function of a time-lag applied to one of them. This is also known as a sliding dot product or sliding inner-product. It is commonly used for searching a long signal for a shorter, known feature. It has applications in pattern recognition, single particle analysis, electron tomographic averaging, cryptanalysis, and neurophysiology. For continuous functions f and g, the cross-correlation is defined as: : (f \star g)(t)\ \stackrel{\mathrm{def}}{=} \int_{-\infty}^{\infty} f^*(\tau)\ g(\tau+t)\,d\tau, whe...

That seems like what I need to do, but I don't know how to actually implement it... how wide of a time window is needed for the Y_{t+\tau}? And how on earth do I load all that data at once without it taking forever?

And is there a better or other way to see if shear strain does cause temperature increase, potentially delayed in time

dmckee

Hmmm...not one of the one I know.

Maybe start dumb: just plot the temperature over some time range starting at the time the strain is applied and let the Mk I eyeball help you figure out what signal you are looking for?

If you can see anything then you know where to start?

Or is there too much data for that to be practical?

tpg2114

My data is 870x870 points and something like 1500 snapshots

I originally just plotted T vs gamma as a scatterplot, but all the high temperature points are at low strain

Which makes me think there's a time delay

dmckee

Ah. Well, you can just sample the snapshots for the first pass. Say look at 15 of them. Or even just 5-8 depending on how long each one takes.

Is your strain a single pulse? A step function? Periodic?

tpg2114

So it's a block of material flying at 50m/s that hits a wall

So the strain is caused by the pressure wave

user54412

so strain is dynamically computed along with T as a function of time?

tpg2114

12:06 AM

Yup

dmckee

I think I'm out of my depth here.

tpg2114

(me too)

user54412

well, if it is lagging but local, you don't need all 870x870 pixels to test the hypothesis, right?

dmckee

Perhaps there should be a time delay. Energy has to spread into multiple degrees of freedom.

tpg2114

True, I could identify a small region that shows both strain and temperature

Which would greatly reduce the dataset

dmckee

12:08 AM

Good call, Chris.

user54412

perhaps look at shear(t), T(t) averaged over a 10x10 region for all 1500 snapshots

hwlau

@tpg2114 So, is the data point you have are temperature T(x,y,t) and strain S(x,y,t), and you want to find the correlation?

user54412

I don't know the details of the sim, but I'd worry about trying to do this on a single pixel at a time - seems noisy

tpg2114

@hwlau Yes, but there is a lag in the data

@ChrisWhite That's also possible

Physics Meta

0

Q: Why is the following question "on-hold" worthy, and how could I improve it?

Link to the question: Learning roadmap for picking up enough mathematical know-how in order to model "shape", "form" and "material properties"? Alternatively, where could I go in order to have such a question answered?

user54412

12:10 AM

besides, heat will diffuse (presumably), so there will be pixel-to-pixel correlations in temperature I should think

hwlau

@tpg2114 Is there spatial correlation then?

tpg2114

@hwlau I could check for that too, but I don't think there's much of a physical reason why there would be one

hwlau

@tpg2114 Diffusion of heat

tpg2114

Heat diffusion is not included in the model

At this point

hwlau

@tpg2114 For reducing data point for calculating time correlation, you can run two exactly the simulation in parallel separated by the time lag dt. Then there is no need to store all snapshot and spatial points.

tpg2114

12:20 AM

@hwlau The sims are done and took many hours each and the snapshots are already stored so this is all post-processing

hwlau

@tpg2114 i see. That trick is for the case taking many many time sample points.

tpg2114

Now I'm digging through the web to try and figure out how to do it in Python

So I can give it 3D arrays (x,y,t) and it knows to only vary the 3rd dimension holding the first two fixed... I may have to write my own code

Kyle Kanos

1:39 AM

@DavidZ I wasn't trying to justify it's existence here, just merely pointing out that because there were some numerics questions posted here, some people might think it okay to post more. I still think marking it as a duplicate is a good idea, then probably an historical lock on the others (maybe with a warning that questions like these belong on Comp Sci?)

tpg2114

1:59 AM

I got a code running and data coming out but I don't really know what I'm looking at... hate when that happens

Kyle Kanos

2:46 AM

@tpg2114 Hmm, perhaps you could squint?

tpg2114

@KyleKanos I'm really really baffled by this output

hwlau

@tpg2114 How is the figure looki like?

tpg2114

@hwlau Let me find somewhere to post it, hang on

Okay, so I'll explain what it is cause it's not designed for other people to look at

Kyle Kanos

@tpg2114 I was just going to ask...

tpg2114

The x axis is the index in the array -- so I have 200 time series

Each one is equally spaced, 1e-9 seconds apart

The black line is \frac{d T}{d t} and doesn't have an axis -- I don't care what the values are

The solid blue line is the abs(shear strain) and is valued on the right axis

The dashed blue line is the result from scipy.signal.correlate

And is valued on the left axis

So what I don't understand: 1) Why is the correlation value negative when they look pretty positively correlated to me? 2) Why is the result from the correlation function 400 time steps long? 3) How do I find the lead/lag between the signals? Wikipedia says the argmin or argmax of the result will tell me that, but I don't know how

Kyle Kanos

2:54 AM

Hmm. Going back to your thinking it was lagging, it does appear that shear strain is lagging behind the temperature change

Or is it the other way....give me a moment.

tpg2114

Yeah, I'm confused too :)

Kyle Kanos

Wait, yes. Seems I had that backwards. First the shear spikes, then a moment later the dT/dt spikes

Wait.

I'm confused now about how it's being plotted temporally.

Is x=0 really t=0 or is it t=t_{end}?

tpg2114

The whole thing confuses the heck out of me

It is t = 0, at least for the solid lines

For the dashed correlation line, I have no idea what the time index corresponds to

Kyle Kanos

Okay...then it seems that the temperature spike comes before the shear strain spike

hwlau

If it is convolution that the indexx ~200 should be t=0

@tpg2114 and how can you get 200 time series? Cross correlation is supposed to be the value between two time series only

tpg2114

2:57 AM

@hwlau Sorry, I meant to say I have 200 time snapshots

hwlau

do you mean 200 time point at a particular location?

tpg2114

Yeah

hwlau

Then it really seems they have correlation from your figure

Kyle Kanos

Yeah, I don't do any correlations, so I'm of little help

Chi-by-eye is good enough for me

hwlau

@tpg2114 1) convolution of +ve and -ve, give you -ve? 3) seems temperature (black line) lead shear strain (blue line)

tpg2114

3:02 AM

@hwlau 1) Both are positive functions but I get a negative correlation value

3) I agree. But I need to find the time between the two signals, the offset that makes them line up

Which wikipedia says the argmax or argmin will tell me

But I don't understand how

Cross-correlation

In signal processing, cross-correlation is a measure of similarity of two waveforms as a function of a time-lag applied to one of them. This is also known as a sliding dot product or sliding inner-product. It is commonly used for searching a long signal for a shorter, known feature. It has applications in pattern recognition, single particle analysis, electron tomographic averaging, cryptanalysis, and neurophysiology. For continuous functions f and g, the cross-correlation is defined as: : (f \star g)(t)\ \stackrel{\mathrm{def}}{=} \int_{-\infty}^{\infty} f^*(\tau)\ g(\tau+t)\,d\tau, whe...

Because I don't know how the result is indexed in time

Kyle Kanos

Crazy idea: Take Fourier transform of data, then perform correlation?

2nd Crazy Idea: Call up a statistician at your Uni and give them your data and ask them to do this stuff for you.

tpg2114

Slides due for my boss in the morning to justify me continuing past this month...

Kyle Kanos

Hmm

Nevermind, crazy ideas usually aren't worth much effort unless you've got time to kill.

hwlau

@KyleKanos That is excellent idea

tpg2114

So I can buy the premise that halfway through the resulting series is tau = 0

Kyle Kanos

3:08 AM

Can't you present it as is (sans the odd correlation values) and say, "It seems that Temperature change occurs before shear strain"??

hwlau

@tpg2114 It is very likely that it is from t=-199 to t=199, total 401 points

tpg2114

I could, I may have to

hwlau

just check that yourself

tpg2114

It's 399 points actually

hwlau

sor it is from t=-198 to t=198

because of the zero index of the array

tpg2114

3:10 AM

Yeah

Kyle Kanos

Science often progresses by promises of something more much later down the line, so hopefully it'll work in your favor this time

hwlau

sorry, looking at the figure, I guess I am wrong somehow

the time lag should be around t~10

which function are you using?

tpg2114

How are you determining that?

hwlau

The peak to peak location

tpg2114

Oh, yeah

And that's pretty close to what I get now:

hwlau

3:15 AM

to have strongest correlation, the shape of two peaks should be almost overlapped

that is the definition

tpg2114

-2.7e-08

Each time index is 1e-9

So it's about 27 between the peaks

hwlau

ya, differe by 2.7 times, same order of magnitude

tpg2114

Yup. So for x < 200, is the time negative or positive?

On the result function

hwlau

how do you calculate it?

tpg2114

That's what I'm asking -- the correlation has 399 points in it. If time = 0 at point 200

Then is point 100 negative time or positive time

hwlau

3:21 AM

you use the scipy function?

tpg2114

Yeah, and I call it with scipy.signal.correlate(gradT, shear)

hwlau

so you use the "full" mode

have you ever tried the "same" mode

tpg2114

I did, it gives me an answer but I don't know how to get the lead/lag time from it

That's where I'm stuck now

user54412

hmm

hwlau

@tpg2114 So, if you use the "same" mode, do you see a peak/trough at index 25 (t=25e-9)?

There are always the problem of the discrete correlation

they are not the same as the continuous case

and you need some kind of padding of zero before and after it, so the problem.

tpg2114

3:32 AM

I'll have to run again with same and see what comes out

user54412

#!/usr/bin/python

import numpy as np
import scipy.signal as signal
import matplotlib.pyplot as plt

t = np.linspace(-10.0, 10.0, 100)
tt = np.linspace(-20.0, 20.0, 199)
signal_1 = np.exp(-t**2)
signal_2 = np.exp(-(t-5.0)**2)
correlation = signal.correlate(signal_1, signal_2)

plt.plot(t, signal_1, 'r--', t, signal_2, 'b:', tt, correlation, 'k-')
plt.savefig('correlation.png')

user54412

I don't know anything about scipy.signal.correlate(), but it seems to output the correlation on double the time interval

user54412

which makes sense - if both signals run from -10 to +10, you can ask about offsets anywhere from -20 to +20

hwlau

I think you know how it works

Transcript for