Time gating using sckit-rf (Time.py) correct use of FFT

[Roger Appleby]

Hi,

I have been using sckit-rf to successfully time gate data from a quasi-optic system so thank you for that.

I noticed today on line 266 of Time.py:

  265       #IFFT the gate, so we have it's frequency response, aka kernel
  266      kernel=fft.fftshift(fft.ifft(fft.ifftshift(gate, axes=0), axis=0))

 267       kernel =abs(kernel).flatten() # take mag and flatten
 268       kernel=kernel/sum(kernel) # normalize kernel

That an inverse Fast Fourier transform is being used to transform from the time domian to the frequency domain.  I would have expected this to be a Fast Fourier Transfom in this case rather than the inverse. Was there a reason for using the inverse or is this an error?

It probably makes little difference to the results especially as only the magnitude is used and the kernel is normalised on the following line.

Regards

Roger Appleby

[Julien Hillairet]

Hello,

Yes you are probably right. Did you try if this is equivalent?

By the way, it has been proposed to simplify the kernel calculation in https://github.com/scikit-rf/scikit-rf/pull/221, but never been tested to my knowledge.

Julien

--
You received this message because you are subscribed to the Google Groups "scikit-rf" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scikit-rf+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/scikit-rf/be684931-9d8c-4f9c-aa91-0458eac3d9c9n%40googlegroups.com.

[Roger Appleby]

Hi Julien,

I did test this and it makes no difference to the result. I guess it probably would have been picked up before if the results were wrong.

I also looked at 221. I note that ifftshift is used correctly in the latest version but I had also spotted it was wrong in an earlier version.  I had previously checked that the centre and stop start were calculated properly  and in my case they are but I am using start as negative and stop as positive and so the minus sign worked just fine.

I was unsure if this should be recorded as an issue on GitHub?

Happy to help straighten this out if needed.

Regards

Roger Appleby

[Julien Hillairet]

I also looked at 221. I note that ifftshift is used correctly in the latest version but I had also spotted it was wrong in an earlier version.  I had previously checked that the centre and stop start were calculated properly  and in my case they are but I am using start as negative and stop as positive and so the minus sign worked just fine.

Do you mean you start from negative time and stop at positive time? If yes, probably this kind of usage was not expected and would require appropriate treatment. So yes, if so, please submit a github issue explaining your use case and what you would expect.

Julien

 

[Roger Appleby]

Hi Julien,

I will add this as an issue. One way to overcome teh ambiguity of teh start and stop would be to use a Graphical interface showing the start, stop and centre markers. I think this kind of usage would be expected when operating in bandpass mode.

This was in fact what I did but in my case I was calling Scikit-rf from Matlab and also had the graphical interface showing the markers  in MATLAB. 

Regards

Roger

[appleb...@gmail.com]

Hi Julien,

I am still continuing to work on the  time gating of S parameters and uncertainty. I recently had cause to look at the Sckit-rf repository and although I am not very familiar with the workings of

Github I could see that there is a version of the tome gating software based on the FTT rather than the convolution. Whilst these are mathematically equivalent from my work I know that you need to window the frequency domain data to avoid edge effects before transforming to the time domain. This is well known in signal processing but can be awkward to get your head around.

 

Below I have plotted some data from a mismatched line time gated with the current convolution method and also with the FFT method on Github.

I am happy to help progress this if needed but I am not very familiar with Github.

 

Regards

 

Roger

--
You received this message because you are subscribed to a topic in the Google Groups "scikit-rf" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/scikit-rf/IoH8hNV58sI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to scikit-rf+...@googlegroups.com.

[Julien Hillairet]

Dear Roger,

Recently, Vincent has modified and improved the time gating code following your previous remarks. The new version is not yet released and is only on GitHub. The changes can be seen here:

https://github.com/scikit-rf/scikit-rf/pull/735/files

Did you check with this version under development? If not, do you have a minimal code to check how the version under development behaves?

Julien

You received this message because you are subscribed to the Google Groups "scikit-rf" group.

To unsubscribe from this group and stop receiving emails from it, send an email to scikit-rf+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/scikit-rf/00e101d8cd2d%24d85151a0%2488f3f4e0%24%40gmail.com.

[appleb...@gmail.com]

Dear Julien,

It was a fluke that I picked up on this.

 

I do have some code that I was using to develop the uncertainty work that I am doing for ESA.  NPL in the UK measured a mismatched line and provided the data in an excel spreadsheet.  I read it in and was time gating it in sckit-rf just to show them that it worked as in the PNA. PTB national metrology lab in Germany have a method for propagating uncertainty without differentiation and it was this that we are trying to evaluate.

 

The figure I provided shows time gated data using the current convolution method and the new FFT method given by your link.  I know you have to window the frequency domain data to avoid the roll off at the edge if you do this in the time domain.  This is something I have played around with in the past.  Of course you could make it a requirements of the user to window the data before time gating. Not so neat but possibly acceptable.

 

Let me know if I can help further.

 

Regards

 

Roger

To view this discussion on the web visit https://groups.google.com/d/msgid/scikit-rf/CA%2B9rY%3D5NyZFG514RrcLbMiN8DVJfr4UWrYLSWEiyNQKi9XZ_rA%40mail.gmail.com.

[vinc0110]

Dear Roger,

As Julien has already mentioned, the time-domain gating function included in the latest version of scikit-rf (v0.23.1) has an issue. I re-implemented the function and changed the algorithm from frequency-domain (FD) convolution to time-domain (TD) multiplication of the window function. It will be included in the next package release. Both methods yield identical results, but TD multiplication is faster. See this post for a summary or follow the entire discussion from the beginning of this issue:

https://github.com/scikit-rf/scikit-rf/issues/734#issuecomment-1191929466

In any case, the old FD implementation was given incorrect results due to boundary effects in the convolution. I suspect this is also causing the difference in your example when you compare the (hopefully correct) TD implementation with the incorrect FD version.

I don't think additional windowing of the frequency domain data is required before the FFT, but please tell us more if you have information on that. As far as I know, only time-domain windowing is required before the DFT/FFT to avoid spectral leakage, but this is not the case for the inverse:

https://dsp.stackexchange.com/questions/70813/windowing-function-for-inverse-fourier-transform

It would be great if you could provide data with known results for verification of our implementation. Maybe also comparisons to other implementations (MATLAB or others).

Regards, Vincent

[appleb...@gmail.com]

Dear Vincent,

Thanks for getting back to me so promptly. Are you going to the European Microwave conference in Milan next week if so perhaps we can meet up and discuss this in person.

 

I have done a lot of testing on the Convolution method both with simulated and measured data and for what we are using it for it gives very good results.  We do use it with Beta = 10 following a sensitivity analysis on all the gating parameters. Our use may be unusual in that we are measuring in a quasi-optic system and using time gating to remove system artefacts e.g. standing waves etc. I do not see these boundary effects in our work and I have only ever used the default reflect method.

 

I will read through the whole discussion in slower time to properly absorb the decision making and understand everyone’s position.

 

I have included the results for the mismatched line taken with the PNA for your convenience below this email. This is obviously not quasi-optic data but something we are working on with NPL and PTB. Metrology labs in UK and Germany for propagating uncertainty.

 

A few years ago I did some work comparing the FFT and convolution methods to support the uncertainty work  and followed the methodology in the attached presentation. I still have the test harness written in python and I can probably share it with you. I would need to seek permission from the European Space Agency. In this case I was using simulated data for silica with added noise. The silica is 15mm thick with permittivity (3.803 -0.001) and in the figure below you see the convolution and FFT overlapping but I could only get this result if I windowed in the frequency domain otherwise the FFT will roll off at the edges. The no noise data is a convolution.

I

  

I can work with you to resolve this but let me review the discussion and then I can consider what I can do.   What is your timescale?

 

Mismatched line data

 

 

 

b = 3.50 mm, a = 1.52 mm and c = 2.31 mm. This produces a 50 ohm characteristic impedance at both inputs to the line, and a central section where the characteristic impedance is 25 ohms.  

 

Fig 4. S₁₁ time-domain response

Fig 8. Frequency-domain response of the mismatch at l₁ (in the absence of the mismatch at l₂)

 

Figure 8 should be compared with the plot I previously provided.

 

Regards

 

Roger

To view this discussion on the web visit https://groups.google.com/d/msgid/scikit-rf/acc1e1b9-4fc8-46e7-9f7b-d839c74ce6b5n%40googlegroups.com.

[Max Kellermeier]

Dear Roger,

 

I’m following your discussions already for a while, and I’m highly interested in your work. I just wanted to let you know that I’m also doing measuring on a quasi-optic system and experienced the roll off at the band edges (using the FFT time gate). Do you have a reference and maybe an example on the uncertainty propagation? So far, I was not really able to do a proper uncertainty analysis on the time gating part of my measurements. I have only included uncertainties from sample characterization as they were pretty dominant so far. But they can definitely not explain the “oscillations” in frequency domain which are caused by the time gate (as far as I understood).

 

If you are interested I can share an example dataset. For instance of fused silica like in your simulated case. By the way I’m working in the range from 220 GHz to 330 GHz, basically only limited by the availability of the hardware.

 

Best,

Max Kellermeier

To view this discussion on the web visit https://groups.google.com/d/msgid/scikit-rf/006d01d8ce6b%245cf65660%2416e30320%24%40gmail.com.

 

[vinc0110]

Thanks for the example data. I just created an issue on GitHub with a replica of your example for further discussion: https://github.com/scikit-rf/scikit-rf/issues/757

I think it's easier over there, but of course we can also proceed here in the mailing list of that's preferred.

Unfortunately, I won't be at the EuMW as I'm not working in academia anymore. Too bad!

[appleb...@gmail.com]

Dear Max,

I am currently writing a paper on the propagation of uncertainty through the time gating process. The paper is being prepared with colleagues at PTB and NPL as I mentioned earlier.  One of the data sets we are using is the mismatched line. We will also present data for silica.

 

I would be happy to look at some of your  data and see if I can help in your understanding. In our case we started time gating calibrated data to remove instrument artefacts. This was an essential step in cleaning it up before extracting permittivity. If you can send me your calibrated data for silica as a touchstone file I can easily put it through our process and explain what I have done.  This will not include propagation of uncertainty as this is still work in progress.

 

Could I ask where you are doing your research?

 

Regards

 

Roger

To view this discussion on the web visit https://groups.google.com/d/msgid/scikit-rf/AS4PR07MB8534E622F286FDBD3188BD009A4E9%40AS4PR07MB8534.eurprd07.prod.outlook.com.

[appleb...@gmail.com]

Hi Vince,

I understand the difficulty in attending conferences and I have to limit myself to one a year!  Where are you based just in case I am passing that way.

To view this discussion on the web visit https://groups.google.com/d/msgid/scikit-rf/3a994886-6e5b-4d82-918d-2217d930e5e0n%40googlegroups.com.

[vinc0110]

I'm currently living in Bremen, Germany. It's not too far from PTB in Braunschweig in case you ever visit your research colleagues ;)

[max.kel...@hotmail.de]

Hi Roger,

Sorry for the late reply! I'm pretty busy at the moment, but it will be a little bit more relaxed in about two weeks.

I don't know if the calibrated data alone makes sense. The calibration itself is in that sense incomplete that it lacks the third calibration standard. This leads to a mismatch of the apparent "reflections" in the time domain transformation. The time gating ensures that only the main peak is included which is properly normalized by the calibration measurements. However, for a proper smooth permittivity result the internal reflections in the measured slab have to be present in the data.

I guess it could be better if I could use the TRL calibration. But the setup is not prepared for that so far, and also the quasi-optical beam propagation would be affected by the "line" standard.

I will prepare the data, and send it to you tomorrow! Attached you can already find the measured permittivity (the uncertainty band is somewhat unclear here). The motivation in my work is to highlight variations in the permittivity between various grades of fused silica, (and also from batch to batch as shown at NPL by Naftaly and Gregory). In simulations concerning our (original) field of research people often just assume a simple value based on literature. The variations can lead to significant deviations in experiments.

I'm doing my research at DESY Hamburg (which is actually not to far from Bremen, in case you would like to a have small lab tour at DESY).

Best,

Max

---

Von: scik...@googlegroups.com <scik...@googlegroups.com> im Auftrag von appleb...@gmail.com <appleb...@gmail.com>
Gesendet: Donnerstag, 22. September 2022 20:07
An: scik...@googlegroups.com <scik...@googlegroups.com>
Betreff: RE: Time gating using sckit-rf (Time.py) convolution and FFT

 

To view this discussion on the web visit https://groups.google.com/d/msgid/scikit-rf/00d501d8ceae%241e782010%245b686030%24%40gmail.com.