How to delete the first 50 rows of individual files

[Yan]

Hi there,

I need to delete the first 50 rows and the last 200 rows from individual text files. I have over 400 individual tab-delimited files, all are matrix of 600 by 65.

Any help will be greatly appreciated.

Yan

[dpb]

Basically, the quickest will be to use dir() to return the list of
files, process it sequentially to read in the array; do the necessary
elimination of rows in memory and then save the result either
overwriting the original file if not needed/wanted or in a new file if
that's the choice wanted.

One could use textscan() w/ the 'headerlines' option to skip the first
50 lines on the reading part and also only read the next 200 lines, but
I suspect the overhead of doing it that way would make the
straightforward way noticeably quicker.

--

[Yan]

Thanks a lot for your response!
I am shamefully a beginning user of Matlab with no programming knowledge.
My friend wrote me the following script to take off the first two lines.
I guess that just a few changes to the script will make it read files from line 51 to line 450. However, I do not know how to twist it :(

Below is the script:


function filt
% Define parameters
path = '/Users/yuy3/Documents/MATLAB/PCA_CSD/Yan_diss_CSD_mul_all/mms'; % The folder contains data
overwrite = true; % Whether overwrite the data file, if not, a new data file with surffix '.new' will be written

% Search the directory to find all files
namelist = dir_search( path, char( '') );

for it = 1 : size(namelist,1)
% Form the name of the source file and the destination file
source = deblank( namelist( it, : ) );
if isempty( source )
continue
end
if overwrite
dest = source;
else
dest = sprintf( '%s.new', source );
end
% Read file, and filter it
file_filt( source, dest );
end
end

% Search directory
function newlist = dir_search( path, namelist )
% Read directory
direnv = dir(path);
newlist = namelist;
for it = 1 : size(direnv,1)
fname = direnv(it).name;
% Skip the parent directory and the directory itself
if strcmp( fname, '.' )
continue
end
if strcmp( fname, '..' )
continue
end
fullname = sprintf( '%s/%s', path, fname );
if isdir( fullname )
% If this is a directory, search it recursively
newlist = char( newlist, dir_search( fullname, char('') ) );
else
% If this is a file, add the name into list
newlist = char( newlist, fullname );
end
end
end

% File filter
function file_filt ( source, dest )
% Read source file
fid = fopen( source, 'rb' );
data = char( '' );
dummy = fgets( fid ); % Skip the first line
dummy = fgets( fid ); % Skip the second line
x = fgets( fid );
while ischar( x )
% Read the following lines
data = char( data, x );
x = fgets( fid );
end
fclose( fid );

% Write destination file
fid = fopen( dest, 'wb' );
for it = 2 : size( data, 1 )
x = deblank( data( it, : ) );
% Write each line, add return and enter at the end of line except
% the last line
if ( it < size( data, 1 ) )
fprintf( fid, '%s\r\n', x );
else
fprintf( fid, '%s', x );
end
end
fclose( fid );
end


Again, thanks a ton!

Yan



dpb <no...@non.net> wrote in message <knl5or$f3i$1...@speranza.aioe.org>...

[dpb]

On 5/23/2013 9:03 AM, Yan wrote:

...[Please don't toppost...hard conversation follow makes]...

> Thanks a lot for your response!
> I am shamefully a beginning user of Matlab with no programming knowledge.
> My friend wrote me the following script to take off the first two lines.
> I guess that just a few changes to the script will make it read files
> from line 51 to line 450. However, I do not know how to twist it :(

...

Pretty complicated for the job it seems...you _could_ fix it up to skip
the 50 lines one at a time as is done simply by changing a loop counter
and the same thing for the reading, but I'd go at it more like

L1=51; % set the two line limits desired to keep
L2=450;
d=dir('suitableWildCardNameString.txt'); % get the files desired
for i=length(d)
x=dlmread(d(i).name,'\t'); % read the tab-delimited file
x=x(L1:L2,:); % keep desired lines
dlmwrite(d(i).name,'\t'); % overwrite the existing file
% fixup the above by making a new file name first if need, obviously
end

The above does assume the files are in the working directory and don't
need to search subdirectories, etc., etc., etc., ... but those are
details. The guts of how (at least one relatively efficient way) to
truncate an existing file to a selected portion is herein above.

--

[dpb]

I blew syntax here by cut 'n pasting from read instead of thinking,
sorry..hard to output anything w/o anything in the call list.

dlmwrite(d(i).name,x,'\t'); % overwrite the existing file

--