Signal = [FRAME, FRAME, FRAME, FRAME, FRAME,...]. What you want to avoid is Signal = [FRAME, FRAME, SKIP, FRAME, FRAME, FRAME, SKIP, SKIP, FRAME, ....]. With irregular sampling...
7:13 AM
@TanMath I don't know what sort of a camera you have but the FPS on web cameras is not fixed. It is more UP TO X FPS, rather than X FPS throughout. And you can see this if you can observe jitter in the FPS. If the FPS was constant, you would get something like:
@TanMath ...the FFT results would be distorted. And I think that to an extent they are. Usually, if you have this kind of jitter, what you do is interpolate the data and resample them in regular intervals. For example, say that you have a list of data like
(Timestamp, FrameAverageFaceValue]. You first create an interpolating model (that can be spline, polynomial, etc) and then, you "ask" this model to give you values at specific points in time. If a given point in
@TanMath time falls exactly on your frame, you will get its value, but if it falls between two known values you will get the interpolated value. This will take away the irregularities of the sampling inserted by the camera's FPS jitter.
@TanMath I think that what you see in your diagrams is the ringing of your bandpass filter as it is "excited" by those spikes in the input. It may be that your camera is not sensitive enough in the IR region that is required for PPG. IN general, I would say that this is a problem with the SNR. To get a feel about how good your system tracks a frequency in its input, try to create a pulsating GIF at a known frequency and check if that is the frequency that your system recognises.
7:52 AM
@A_A I just realized I think maybe the paper link in the question may be broken but here is a fixed link: osapublishing.org/oe/abstract.cfm?URI=oe-18-10-10762
It shouldn't be under a paywall but if so, ask me, and I can send a pdf version to you, though I think there's one online already...
@A_A I send this because the paper mentions a 15 fps video. How do they achieve a constant FPS with a webcam? Because they don't mention anything of resampling (or downsampling, assuming that's the same thing?) or interpolation. Is it possible to set a FPS for the camera and maybe I should look into such an option? I will definitely look into the interpolation approach as I have seen some approaches with that in video heart rate measurement.
@A_A However, what I am most confused is why the background video is showing a peak in the FFT? I wouldn't think the unevenly sampled video would affect that.
@A_A Another question is that are there methods for determining the power spectrum for unevenly sampled data? Maybe I could also use that? Because I would feel the interpolated approach is not that accurate because it is estimating the intermediate values...
@A_A Finally, I am not sure why you introduce IR. While it is true IR is required traditional PPG, that is because it is measuring SpO2 and is dependent on the absorption difference of the oxygenated hemoglobin and deoxygenated hemoglobin. There should be enough change in the RGB color channels to determine the pulse. Hopefully, there should be enough information in the paper that explains how it works.
2 hours later…
9:43 AM
@TanMath I have access to the paper. I do not see any major conflicts. Web cams have limited dynamic range. From my experience, I would find it difficult to accept (without some experimentation at least :) ) that the only signal that the web cam uses to infer heart rate comes from the visible spectrum. The change in face "redness" (if you like) because of the heart rate signal compared to everything else that is happening in a scene is miniscule. MAYBE, you can observe the "pulsating face" ...
@TanMath under very controlled conditions. But, with an off the shelf webcam that is only limited to the visible spectrum under general lighting conditions, it sounds like a lot to ask from the webcam sensor. Therefore, I think that to an extent, the extended sensitivity of the CMOS sensor to the infrared spectrum is taken advantage of as well. If you take a webcam and you try to shoot your gas stove or a flame, you will see what I mean. The image saturates all over the place with a lot of....
9:58 AM
@TanMath ...glow, coming from the infrared spectrum. The most typical test is holding an IR remote in front of the camera and watching the IR LED flash prominently.
I am not saying that the authors have not done something properly. I am saying that maybe it has not been described accurately.
The easiest way to control for this is with an infrared cut-off filter (https://en.wikipedia.org/wiki/Infrared_cut-off_filter). Once you have your system up and running, try to check BPM with and without the filter. Mind you, the filter would diminish your signal (worsen the SNR) so, you might have to…
I am not saying that the authors have not done something properly. I am saying that maybe it has not been described accurately.
The easiest way to control for this is with an infrared cut-off filter (https://en.wikipedia.org/wiki/Infrared_cut-off_filter). Once you have your system up and running, try to check BPM with and without the filter. Mind you, the filter would diminish your signal (worsen the SNR) so, you might have to…
> How do they achieve a constant FPS with a webcam?
I don't know. I do not own a Mac book. Maybe it can do 15 FPS stable.
I don't know. I do not own a Mac book. Maybe it can do 15 FPS stable.
> Is it possible to set a FPS for the camera and maybe I should look into such an option?
Of course. The webcam control software does that for you. Also, openCV should have system calls by which you can invoke the camera settings dialog (in Windows) or, calls by which you can setup the camera parameters. All of these are packaged in system calls in Windows and similar settings in Linux too.
Of course. The webcam control software does that for you. Also, openCV should have system calls by which you can invoke the camera settings dialog (in Windows) or, calls by which you can setup the camera parameters. All of these are packaged in system calls in Windows and similar settings in Linux too.
No, not the same thing. If the web cam jitter is more than 1 FPS, I would take it into account. I am not familiar with openCV entirely so I have to ask you: How do you do the frame-by-frame processing? Do you have 1 thread for image acquisition and 1 thread for processing? If NOT, then you are probably skipping frames beyond your control or the FPS. This is because you have to ensure that your image processing finishes within 1/15 seconds. And, colour-splitting, spatial averaging, ICA, ...
...bandpass filtering, FFTing and whatever else follows that until you get a BPM reading might be taking more than 1/15 seconds
@TanMath Therefore, you are probably dropping frames because you don't get to process another frame until your function has finished. EXCEPT if openCV does buffering (with a long buffer) and it guarantees to you that it will process all of the frames until you stop capturing. That buffer can't be too long tho
@TanMath Because your bandpass is ringing. (0.6+1.5)/2~1.05 Hz. That's your center frequency and you have a peak on that frequency. That's probably coming from the excitation of the filter from those spikes you have in the input.
@TanMath For relatively small jitter around the clock frequency (as you would have right here) you can use interpolation as indicated previously. If you are in a MATLAB-like environment, there would be something like interp1 available that you can use.
@TanMath Yes, I think that testing the whole pipeline with a known reference source will be best. I will provide an answer to your post anyway but there are certain things that are unknowns that I would rather clarify before I start writing that response
@TanMath Regarding a chat, I see what you mean but I am going to be traveling soon and will not be available until next Wednesday. In any case, I think it would be best if we keep communicating via this channel for the moment. I can see it is slightly inconvenient but probably best given the circumstances. Are you around Singapore, Malaysia timezone?
9 hours later…
7:29 PM
@A_A I do disagree with your explanation of how IR is important... While IR will have a much stronger heart rate signal, the RGB video is enough to measure the heart rate... I think even CCD cameras measuring only light in certain wavelengths within the visible spectrum have been used to measure the heart rate. The algorithm is looking for minute changes in the RGB signal that correspond to blood flowing into the face with each heartbeat...
... And this is separated from the rest of the RGB signal with the blind source separation methods like PCA and ICA... However, I guess those details don't really matter to the problem I having right now though...
@A_A I will look into that... I don't think it skips a frame, because I remember doing experiments with openCV where the computation was really long and because of that it got delayed with the webcam, so I think it processes every single frame... I am not sure if maybe the timestamps might be messed up though...
1 hour later…
« first day next day → last day (60 days later) »
Transcript for
Apr12
Apr '1813
Apr14
Discussion between A_A and TanMath
Imported from a comment discussion on dsp.stackexchange.com/qu...
