[mg14747] Re: config. data from internet database
P.J. Hinton
writes:
|> I have a question regarding how to extract and put into a list form,
|> numbers from an internet science database file that I have already
|> imported into my Mathematica 3.0 program using "ReadList". The
problem |> is as follows:
|>
|> I want to extract from a file named "1997.ADJ" (which is a list of
daily |> 10.7 cm solar flux values for the year 1997), certain columns
of |> numbers from the period Sept. 1, 1997 to Dec. 31, 1997 which
will form |> a list (time series) which I will use to plot, get a
Fourier |> Transform, a power-spectrum, etc.. |> The file comes from
the internet site: |> http://www.ngdc.noaa.gov/stp/stp.html and is
found after getting into |> this site by the following steps: |>
|> Click "Solar and Upper Atmosph."
|> Click "Get Data"
|> "" "Solar Radio"
|> "" " Flux"
|> "" "1997.ADJ"
|>
|> I am at this point where I have read in the file by using the
ReadList |> function as follows: ReadList["a\1997.adj", String] My
output is a |> table of average daily values for each month of 1997
with a header and |> footer section containing comments. I only want
to extract the data |> values from four columns (Sep., Oct., Nov, Dec).
I've tried using |> mathematica functions such as MemberQ,
Character,and Select and have |> not been successful. At this point,
I'd really appreciate some help! |> Got any ideas?
|>
|> Thanks for your time and consideration. I look forward to hearing
from |> you.
Because this table is slightly irregular, we'll have to do some fancy
footwork to get the data you need:
(* First, let's open the file as an InputStream[]. *)
In[1]:= stmp = OpenRead["1997.ADJ"];
(* Skim past the five header lines. *)
In[2]:= Do[Read[stmp, String], {5}]
(* Now read in the 37 lines of data and and vertical spaces. *)
In[3]:= daysString = Table[Read[stmp, String], {37}];
(* Then we map a function which converts each string into a stream and
breaks the strings up into lists of individual strings. *)
In[4]:= daysTable = Map[Block[{stmp = StringToStream[#], lst},
lst = ReadList[stmp, Word]; Close[stmp]; lst]&, daysString];
(* Close the data file. *)
In[5]:= Close[stmp];
(* Any vertical spaces will produce empty lists, so we can throw
those out. *)
In[6]:= trimmedTable = DeleteCases[daysTable, {}];
(* Now we even out the array be entering rows 29 and 30 of Febrary. *)
In[7]:= MapAt[Insert[#1, 0, 3] & , %, 29];
In[8]:= MapAt[Insert[#1, 0, 3] & , %, 30];
(* And do the same for day 31 of September, April, June, and November.
*)
In[8]:= finalTable = MapAt[ReplaceAll[#,
{date_, jan_, mar_, may_, jul_, aug_, oct_, dec_} ->
{date, jan, 0, mar, 0, may, 0, jul, aug, 0, oct, 0, dec}]&,
%, 31]
(* We can now get the last four columns and knock out the zero pads. *)
In[9]:= desiredData = Map[Map[ToExpression, #]&,
DeleteCases[Take[Transpose[finalTable], -4], 0, -1]]
(* The nested Map operation is needed to turn the strings into numbers.
*)
Hope that helps.
--
P.J. Hinton
Mathematica Programming Group pa...@wolfram.com Wolfram Research,
Inc. http://www.wolfram.com/~paulh/ Disclaimer: Opinions expressed
herein are those of the author alone.