Re: [fityk-users] How do I batch process 8000 .txt files?

[William Paxton]

Hi there,

I'll take a stab at this. I had to do a similar script a while ago. It was written in lua which can be interpreted by fityk.

for f=16,30 do

    filename = string.format("\\CSV\\SMP000%0d.csv", f)

    F:execute("@+ < '" .. filename .. ":0:1::'")

end

F:execute("@*: X = (6198/sin(1.521954959*pi/180)) / (160.0783+24.1616 * X)")

F:execute("@*: A = false")

F:execute("@*: A = a or (2.72279 < x and x < 2.79322)")

F:execute("plot [1.1:5] [] @*")

F:execute("%bg1 = Spline(2.72322,20.1, 2.79321,17.7)")

F:execute("@*: Y = y - %bg1(x)")

F:execute("@*: guess Gaussian")

F:execute("@*: fit")

F:execute("@*: info peaks_err >> 'data.peaks'")

Obviously yours wont be exactly the same because your file names will be different as well as your fitting process. Make sure you understand how to the process works with one spectra and then copy the commands out of the console.

Hope this helps.

On Tue, Mar 26, 2013 at 10:39 AM, <grandr...@gmail.com> wrote:

Hello All, I'm processing some diffraction data (see attached) which is a plot of angular position (0-180°) in column 1 and intensity in column 2.  I have over 8000 of these files.

What I need fityk to do for me;

1. Open file filename_000573.txt and plot it.
2. Guess a gaussian curve and fit it.
3.  Write the center of the peak (ie 91.9° for the attached file) to some new text file/spreadsheet
4.  Close the file, increment the number by one (ie open filename_000574.txt) and repeat the process.
5.  Stop when a given file name is reached.

At the end I want to have one single file, with all the peak center locations in it.  If I can have the filename as well as some sort of two column setup, that would be preferable but not necessary as long as they're written in order.

At that point, I'll turn it into a three column text file (adding x and y positions), which I will plot in gnuplot.

I am using the GUI version of fityk 0.9.8 on windows.  Alternatively if this can be done with fit2D, gnuplot or some other free software that's fine.

I really appreciate any help you can offer, computer programing is not my area of expertise.

--
--
http://groups.google.com/group/fityk-users
---
You received this message because you are subscribed to the Google Groups "fityk-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fityk-users...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

William A. Paxton
will....@gmail.com

732-421-7343

[William Paxton]

Okay, I have a better script for you. It is written in lua and can be opened in fityk or cfityk.

-- beginning parameters. start is the left bound of the peak, and finish is the right bound of the peak. 

start = 1.71

finish = 1.95

report = string.format("report.peaks")

psf = string.format("Gaussian")

-- start code. broken in to sections because you cannot do a for loop with leading zeros (to the best of my knowledge at least)

for f=1,9 do

    filename = string.format("file-000%0d.xy", f)

    F:execute("@+ < '" .. filename .. ":1:2::'")

F:execute("@*: A = false")

F:execute("@*: A = a or (".. start .." < x and x < ".. finish ..")")

F:execute("@*: guess ".. psf .. "")

F:execute("@*: fit")

F:execute("@*: info peaks_err >> '" .. report .. "'")

F:execute("delete @0")

F:execute("delete %*")

end

for f=10,99 do

    filename = string.format("file-00%0d.xy", f)

    F:execute("@+ < '" .. filename .. ":1:2::'")

F:execute("@*: A = false")

F:execute("@*: A = a or (".. start .." < x and x < ".. finish ..")")

F:execute("@*: guess ".. psf .. "")

F:execute("@*: fit")

F:execute("@*: info peaks_err >> '" .. report .. "'")

F:execute("delete @0")

F:execute("delete %*")

end

for f=100,999 do

    filename = string.format("file-0%0d.xy", f)

    F:execute("@+ < '" .. filename .. ":1:2::'")

F:execute("@*: A = false")

F:execute("@*: A = a or (".. start .." < x and x < ".. finish ..")")

F:execute("@*: guess ".. psf .. "")

F:execute("@*: fit")

F:execute("@*: info peaks_err >> '" .. report .. "'")

F:execute("delete @0")

F:execute("delete %*")

end

for f=1000,8000 do

    filename = string.format("file-%0d.xy", f)

    F:execute("@+ < '" .. filename .. ":1:2::'")

F:execute("@*: A = false")

F:execute("@*: A = a or (".. start .." < x and x < ".. finish ..")")

F:execute("@*: guess ".. psf .. "")

F:execute("@*: fit")

F:execute("@*: info peaks_err >> '" .. report .. "'")

F:execute("delete @0")

F:execute("delete %*")

end

[Marcin Wojdyr]

> -- start code. broken in to sections because you cannot do a for loop with
> leading zeros (to the best of my knowledge at least)
> for f=1,9 do
> filename = string.format("file-000%0d.xy", f)

You may specify number of digits in the format, and 0 to fill missing
digits with 0's not spaces, for example

> = string.format("%05d", 123)
00123
> = string.format("%05d", 1234)
01234

So the beginning of the loop could look like this:
for f=1,8000 do
filename = string.format("Mg20A_%05d.ge2.txt", i)

It may not help OP though, because version 0.9.8 has no Lua.

One workaround may be to first generate large fityk script that
repeats the same 8000 times, and then run this script.
For example, this Python script produces fityk script with 8000 x 5 lines.

#!/usr/bin/env python
f = open("big_script.fit", "w")
for n in range(1, 8000):
f.write("""
@0 < Mg20A_%06d.ge2.txt
guess %%f = Gaussian
fit
print %d, %%f.center, filename >> results.txt
""" % (n, n))

Cheers
Marcin

[William Paxton]

Thanks Marcin! Always a great help.

--
--
http://groups.google.com/group/fityk-users
---
You received this message because you are subscribed to the Google Groups "fityk-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fityk-users...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

William Paxton

President | MSE Graduate Student Association 

Graduate Assistant | Nanomaterials Laboratory

Department of Materials Science and Engineering, Rutgers University

will....@rutgers.edu | 732-421-7343

[grandr...@gmail.com]

Thank you all very much for the responses. I'll give them a shot and report back!