working with MT .edi files

[Janelle S]

Hi, 

I am new to the SimPEG codes but excited to be using them. I am primarily interested in using them for the MT applications. I have had no problems working through the MT jupyter notebooks for synthetic modelling.  I was hoping someone can point me in the right direction for information on how to run 1D inversions with real data from .edi files. I am more than happy to download the codes and work with them in my local python install if that is what is required. 

Thanks 

Janelle

[Lindsey Heagy]

Hi Janelle, 

Sorry for the delayed response! Gudni, who is the point person on MT in SimPEG is away at the moment, so when he returns we can hopefully get you some more examples. In the mean-time, there are two examples of synthetic inversions here: http://docs.simpeg.xyz/content/examples/index.html#natural-source-electromagnetics (These can hopefully be a start for getting familiar with the code). 

There are also some utilities that are currently a bit buried for working with edi files. You can use

from SimPEG.EM import NSEM

NSEM.Utils.EDIimporter

The code is here: https://github.com/simpeg/simpeg/blob/8cdb72c2b82bbb5d53bd1c64fff88e89a582cc80/SimPEG/EM/NSEM/Utils/ediFilesUtils.py. 

Feel free to try this out and post an issue on github (https://github.com/simpeg/simpeg/issues) for generating an example and we can work to get one generated - I am sure this will be of use for many in the MT community! 

All the best,

Lindsey 

[Gudni Rosenkjar]

Hi Janelle,

Sorry for the late response. I have been away doing field work and I am catching up on things.

In the attachment is a script that I have used to run 1D inversion of data imported from a directory of EDI files. It has some comments in it, explaining a little what is happening but it isn't super verbose. It runs for the EDI files I have worked on my computer and should work on the SimPEG master branch or the PyPi installed version (though I haven't tested that). Try it and see if it works for you. If it doesn't, don't hesitate to ask me questions and I will try to help you to get it working with your EDI files.

On that note, I would be very supportive if you would like to try to make this into an Example/Tutorial that could be integrated into SimPEG distro. I haven't worked with EDI files that are publicly available and running this for public data (like USarray data or similar) is still far down my priority list at the moment. Is this something you would be keen on? This would be appreciated work and welcomed by the community. 

Thank you.

Gudni

[Janelle S]

Hi Gudni/Lindesy,

I managed to get this code working with the PyPi distribution of simpeg. The last thing I am struggling with is checking the data fits. The code provided does not output the observed and calculated response for the inversion. I've have tried to add this to the code using NSEM.Utils.dataUtils.plotMT1DModelData but haven't had a lot of luck. Idealy i would like to view these fits graphically. 

on a related note, is there information about the data fit on the inversion terminal output? There are several terms I can't find defined anywhere that may help me:

f      |proj(x-g)-x|  LS

I am also unclear about the meaning of these outputs:

1 : |fc-fOld| = 0.0000e+00 <= tolF*(1+|f0|) = 2.8343e+01
1 : |xc-x_last| = 3.6971e-01 <= tolX*(1+|x0|) = 2.4026e+00
0 : |proj(x-g)-x|    = 1.7960e+01 <= tolG          = 1.0000e-01
0 : |proj(x-g)-x|    = 1.7960e+01 <= 1e3*eps       = 1.0000e-02

I have attached my modified inversion file. Any help is greatly appreciated.

Cheers,

Janelle

[Janelle S]

Update: I managed to work out how to get the observed and forward data out of the inversion. It would still be nice if someone could point me towards documentation (or explain) what the inversion outputs mentioned in the previous post are.

Cheers,

Janelle

[Gudni Rosenkjar]

Hi Janelle,

Sorry for the belated reply.

I am glad that you were able to figure how to access the observed and predicted data. The current state of the reporting and logging in the code needs updating, to make things more transparent and customizable. 

As for the current inversion outputs, there is very limited information about what those are. 

f      |proj(x-g)-x|  LS

In the line above, f is the objective function (phi), x is the model, g is the gradient and LS is the number line searches performed at each iteration of the inversion. This information can be handy to look at during the inversion (as well as the phi_d and phi_m) to make sure that the objective is decreasing.

1 : |fc-fOld| = 0.0000e+00 <= tolF*(1+|f0|) = 2.8343e+01
1 : |xc-x_last| = 3.6971e-01 <= tolX*(1+|x0|) = 2.4026e+00
0 : |proj(x-g)-x|    = 1.7960e+01 <= tolG          = 1.0000e-01
0 : |proj(x-g)-x|    = 1.7960e+01 <= 1e3*eps       = 1.0000e-02

Similarly, fc and fOld are the current and previous objective function evaluations(respectively), xc and x_last are current and previous models, x0 is the initial model and |proj(x-g) -x| is a global estimate of how much change will take place in the model. This has to do with the stopping criteria of the inversion and depends on the type of inversion. I personally don't find this information all that informative or helpful to detemain the reason for the termination of the inversion.  

When I run inversion, I do look at the progress of the objective functions ( phi_m, phi_d and f) at each iteration to make sure that they are converging. After completion, I plot the curves based on the data in the .npz files that are generated. I will try to put some better example together in the next couple of weeks on how I do this. 

I hope this was helpful (though late). If you have more question, post them and I will try to be better on top of them this time around.

Have a good day.

Gudni

[ibriah soniya]

Hi, i'm soniya and i use this script for importing my edi file data. But i got an error in script "EDIfiles.importFiles()" and when i print (ediFileList) it just appear [ ] and nothing more. Could you help me to solve this problem?

I got this kind of error:

"UnboundLocalError: local variable 'outTemp' referenced before assignment"
Thank you