Oldest file in subdirecties

[Cathy]

1. I am looking for a way to find the oldest file in a specified directory
and all subdirectoties within it

2. then delete this file

3. check how much space is left

4. if less than 2gb (or 9%) free space left rerun from 1.

5. end



Any bright suggestions will be greatly appreciated



Cathy

[foxidrive]

@echo off
:loop
check the diskspace free
if less than 2GB (
take a list of files with a for/f-in-do using a dir /s /b /a-d command
echo into a file the last modified date in yyyymmdd (and add hhmm if you need that granularity) followed by the drv:\path\filename
use sort to get the list in date order and then another for-in-do to extract the oldest entry, get the filename and delete it.
goto :loop
) else (
goto :EOF
)





--
Mic

[billious]

"Cathy" <M8R-n...@mailinator.com> wrote in message
news:LrmdnasxUMs0OPrS...@bt.com...

@echo off
:: delete oldest until freespace > specified size
setlocal enabledelayedexpansion
:loop
for %%i in (name1 name2 name3 size1 size2 size3) do (set %%i=)
for /f "tokens=3*" %%i in (
' dir /o:d /-c /a-d "C:\destdir" 2^>nul'
) do (
set name3=!name2!&set size3=!size2!
set name2=!name1!&set size2=!size1!
set name1=%%j&set size1=%%i
)
if not defined name1 echo Directory empty&goto :eof
set size1=000000000000000%size1%
set size1=%size1:~-15%
:: specified size as 15 digits, leading-zero-filled
if "%size1%" gtr "001681121345536" echo Done!&goto :eof
ECHO del "c:\destdir\%name3%" 2>nul
goto loop


C:\destdir is the target directory.
The number "00168..." in the third-last line needs to be adjusted to suit
your minimum required size, as leading-zero-filled to 15 digite. The figure
I used here is my remaining space during testing
The ECHO on the second-last line needs to be removed to perform the delete
after testing. If it isn't removed, the loop will execute endlessly. For
example, you could use another process while it is looping to delete
sufficient space for the loop to stop.
All the usual warnings about filenames containing poison characters should
be observed.
Note that the IF ... GTR ... is performing an ALPHABETIC comparison, not a
NUMERIC comparison since the values are beyond the 31-bit limit. This is the
reason for forcing the strings compared to be of equal length.

[foxidrive]

I didn't study your code - just wondering if it will do subdirectories too?

> The number "00168..." in the third-last line needs to be adjusted to suit
> your minimum required size, as leading-zero-filled to 15 digite. The figure
> I used here is my remaining space during testing
> The ECHO on the second-last line needs to be removed to perform the delete
> after testing. If it isn't removed, the loop will execute endlessly. For
> example, you could use another process while it is looping to delete
> sufficient space for the loop to stop.
> All the usual warnings about filenames containing poison characters should
> be observed.
> Note that the IF ... GTR ... is performing an ALPHABETIC comparison, not a
> NUMERIC comparison since the values are beyond the 31-bit limit. This is the
> reason for forcing the strings compared to be of equal length.
>
>

--
Mic

[billious]

"foxidrive" <foxi...@gotcha.woohoo.invalid> wrote in message
news:S7S9r.2557$Ce4....@newsfe21.iad...

> --
> Mic

Nope. Missed that clause.

Hmm - so the requirement is to pick off the oldest file in the subtree until
the freespace rises above a threshhold then?

[foxidrive]

On 20/03/2012 14:18, billious wrote:
> "foxidrive" <foxi...@gotcha.woohoo.invalid> wrote in message
> news:S7S9r.2557$Ce4....@newsfe21.iad...
>> On 20/03/2012 12:55, billious wrote:
>>> "Cathy" <M8R-n...@mailinator.com> wrote in message
>>> news:LrmdnasxUMs0OPrS...@bt.com...
>>>> 1. I am looking for a way to find the oldest file in a specified
>>>> directory
>>>> and all subdirectoties within it
>>>>
>>>> 2. then delete this file
>>>>
>>>> 3. check how much space is left
>>>>
>>>> 4. if less than 2gb (or 9%) free space left rerun from 1.
>>>>
>>>> 5. end

>>

>> I didn't study your code - just wondering if it will do subdirectories
>> too?
>>
>>
>> --
>> Mic
>
> Nope. Missed that clause.
>
> Hmm - so the requirement is to pick off the oldest file in the subtree until
> the freespace rises above a threshhold then?

I think so.


--
Mic

[billious]

"foxidrive" <foxi...@gotcha.woohoo.invalid> wrote in message
news:S7S9r.2557$Ce4....@newsfe21.iad...

> --
> Mic


OK - so - read in conjunction with previous posts, warnings, disclaimers and
so on.

Code is indented two spaces. Lines not starting with two spaces have been
wrapped and need to be rejoined.




@echo off
:: delete oldest from subtree until freespace > specified size
setlocal enabledelayedexpansion
set "tempfile=%temp%\temp##.###"
del "%tempfile%" >nul 2>nul
set "target=c:\destdir"
for /f "tokens=*" %%i in (
'dir /s /b /a-d "%target%" 2^>nul'
) do (
call :procdate %%~ti
>>"%tempfile%" echo !formdate! %%i
)
for /f "tokens=1*" %%i in (
'sort ^<"%tempfile%" ' ) do (
for /f "tokens=3" %%s in ( ' dir /-c "%%j" 2^>nul ' ) do set
size=000000000000000%%s
set size=!size:~-15!
if "!size!" gtr "001681000000000" echo Done!&del "%tempfile%" >nil&goto
:eof
ECHO del "%%j"
PING -n 15 127.0.0.1 >nul
)
goto :eof

::
:: Process the date and time supplied as parameters
:: destination envvar=formdate
:: destination format is
:: CCYYMMDDHH(:)MM or
:: CCYYMMDD[a/p](.)(m(.))HH(:)MM
:: if 12-hr clock used, pay attention to leading zeroes
:: and convert HH=12 to 00. That way, the sequence
:: 12:59am 01:00am 11:59am 12:00pm 12:59pm 01:00pm becomes
:: a0059 a0100 a1159 p0000 p0059 p0100
:: which sorts correctly.
:: Critical issue is consistency of length of subfields.
:: Colons, etc are superfluous.
::
:: date part - keep or generate consistent-length subfields,
:: observe order
::
:: might be a good idea to replace all spaces with zeroes
::
:: Fortunately for me, I use 24-hr time. Date format is dd/mm/yy
::
:: Not interested in dates pre-2000. If you are, make adjustments.
::
:procdate
set formdate=%1
set formdate=%formdate:~6,2%%formdate:~3,2%%formdate:~0,2%%2
goto :eof


The way I tested this was to start another session and copy a large file
into a dummy directory. Then start the batch where the "1681000000000" was
more than my current freespace. The files to be deleted were then presented
by name one at a time every ~15 seconds. This showed the correct
oldest-first order. Deleting the file I'd copied made the space avaiable
jump above the threshhold and the batch terminated.

Having checked that the sequence of files to be released is appropriate, to
activate - remove the ECHO keyword on the ECHO del ... line and delete the
PING line.

The only real complication is the date-conversion routine. What's required
is a consistent format with a consistent length. Without knowing the details
of OP's date/time format, that has to be left as an exercise.

[Timo Salmi]

On 20.03.2012 07:36 billious wrote:
> The only real complication is the date-conversion routine. What's required
> is a consistent format with a consistent length. Without knowing the details
> of OP's date/time format, that has to be left as an exercise.

Which the OP should alternatively find in the FAQ.

All the best, Timo

--
Prof. (emer.) Timo Salmi, Vaasa, Finland
http://www.netikka.net/tsneti/homepage.php
Useful CMD script tricks http://www.netikka.net/tsneti/info/tscmd.php

[Cathy]

Thanks for the response

I have tried this response although I dont understand how it works and what
it is doing where

I have ended up going back to Timo's original response and trying to work on
that as a solution.

Hope to finish this tomorrow and will post here what I come up with

PS. The date and time formats I use are the same as yours so it should work,
but I was not sure where the size of your free space came from until I
turned echo off and started debugging.

Thanks for your efforts so far.

regards
Cathy

[billious]

Hmm. strange - works as-is for me...

Good to see you're trying out the debugging facilities.

Here's the same source with [line numbers] for easier reference:

----- batch begins -------
[01]@echo off
[02]:: delete oldest from subtree until freespace > specified size
[03]setlocal enabledelayedexpansion
[04]set "tempfile=%temp%\temp##.###"
[05]del "%tempfile%" >nul 2>nul
[06]set "target=c:\destdir"
[07]for /f "tokens=*" %%i in (
[08]'dir /s /b /a-d "%target%" 2^>nul'
[09]) do (
[10]call :procdate %%~ti
[11]>>"%tempfile%" echo !formdate! %%i
[12])
[13]for /f "tokens=1*" %%i in (
[14]'sort ^<"%tempfile%" ' ) do (
[15]for /f "tokens=3" %%s in ( ' dir /-c "%%j" 2^>nul ' ) do set
size=000000000000000%%s
[16]set size=!size:~-15!
[17]if "!size!" gtr "001681000000000" echo Done!&del "%tempfile%" >nul&goto
:eof
[18]ECHO del "%%j"
[19]ping -n 15 127.0.0.1 >nul
[20])
[21]goto :eof
[22]
[23]::
[24]:: Process the date and time supplied as parameters
[25]:: destination envvar=formdate
[26]:: destination format is
[27]:: CCYYMMDDHH(:)MM or
[28]:: CCYYMMDD[a/p](.)(m(.))HH(:)MM
[29]:: if 12-hr clock used, pay attention to leading zeroes
[30]:: and convert HH=12 to 00. That way, the sequence
[31]:: 12:59am 01:00am 11:59am 12:00pm 12:59pm 01:00pm becomes
[32]:: a0059 a0100 a1159 p0000 p0059 p0100
[33]:: which sorts correctly.
[34]:: Critical issue is consistency of length of subfields.
[35]:: Colons, etc are superfluous.
[36]::
[37]:: date part - keep or generate consistent-length subfields
[38]:: and observe order
[39]::
[40]:: Fortunately for me, I use 24-hr time. Date format is dd/mm/yy
[41]::
[42]:: Not interested in dates pre-2000. If you are, make adjustments.
[43]::
[44]:procdate
[45]set formdate=%1
[46]set formdate=%formdate:~6,2%%formdate:~3,2%%formdate:~0,2%%2
[47]goto :eof
------ batch ends --------

So - to clarify (some of this will be obvious but might help others
interested)

[03] Allows use of delayed-expansion mode so !var! is evaluated as the
current value of VAR within FOR loops (%var% would be the value at parse
time.) As a bonus, ensures initial environment values are restored at
batch-end.

[04,05] Establishes a temporary filename and deletes the file if it exists,
suppressing messages
[07..12] Performs a dir /s /b /a-d on the target directory, producing a list
of the full names of each file (/b) except directorynames (/a-d) including
all subdirectories (/s)

For each line in this list, assign the FULL text (ie full filename) to %%i
and
1) call the PROCDATE subroutine passing the file's datestamp (%%~ti) as
parameters
2) append a line containing the current value of FORMDATE (which will be set
up by PROCDATE) and a space and the full filename to the temporary file

In consequence the tempfile should contain a lines of the form
YYMMDDHH:MM full-filename

[13..20] The tempfile is used as input to a SORT, and each line then parsed
according to the tokens. The first token on the line is assigned to %%i and
the remainder of the line to %%j.
With each line (=filename in %%j) :
1) perform a DIR on the filename found, using /-c to remove the
thousands-separators from the sizes reported. Assign the third token from
the line to %%s, prefix this with a goodly number of 0s and assign that to
SIZE.
The result should be that SIZE after executing [15] contains the remaining
space on the target drive, prefixed by zeroes, since the last line of the
DIR command reports the remaining-space as the third token.
2) [16] sets SIZE to its own last 15 characters, using the current value set
in the FOR..%%s just executed
3) compare that result to the desired minimum freespace, forcing CHARACTER
comparison which proceeds left-to-right. The values will therefore be
compared correctly if the two strings are of equal length. The comparison
can't be made in NUMERIC mode because the values involved may be larger than
the 31-bit computation capacity allowed.
If we have enough space, display done, delete the tempfile and exit the
batch
If we don't have enough space, delete the target file named in %%j. Note
that the SORT in [14] ensures that the lines are presented to the FOR...%%i
in order of timestamp since the first column is YYMMDDHH:SS, therefore the
deletion proceeds oldest-first until the freespace-required is reached.

the PING and ECHO are used for testing, as noted.

The PROCDATE procedure should present no problem.

PROCDATE receives two parameters - the date and time. It assigns the date
part to FORMDATE then re-assigns the value using substringing (which can
only be done on a conventional environment variable, not on a metavariable)
and tags the time on to the end of the rearranged date-string as it's in a
convenient form.

So - hope the explanation is clear and helps to clear up any problem you're
having.

[foxidrive]

On 21/03/2012 14:55, billious wrote:

> Cathy wrote:
>> PS. The date and time formats I use are the same as yours so it
>> should work, but I was not sure where the size of your free space
>> came from until I turned echo off and started debugging.
>>
>> Thanks for your efforts so far.
>>
>> regards
>> Cathy
>
> Hmm. strange - works as-is for me...
>
> Good to see you're trying out the debugging facilities.

Is there a regional locale setting here that is sneaking in?

temp##.###
20032116:36 d:\AAA\A.BAT
20032101:21 d:\AAA\Here-report-ALL.xls
20032101:29 d:\AAA\qt.conf
20032102:05 d:\AAA\rename.BAT


DIR output
21/03/2012 16:30 <DIR> .
21/03/2012 16:30 <DIR> ..
21/03/2012 16:36 1,651 A.BAT
21/03/2012 01:21 0 Here-report-ALL.xls
21/03/2012 01:29 60 qt.conf
21/03/2012 02:05 111 rename.BAT




--
Mic

[billious]

Certainly, as noted by Timo and lines 25..42 of the batch.

Since you're using 4-digit years, you'd need
[46]set formdate=%formdate:~6,4%%formdate:~3,2%%formdate:~0,2%%2

(or do the DIR with the /4 switch then ~6,4 if you normally use 2-digit
years)

Point is that there are so many different possibilities in representing date
and time - the sequence of the date elements, 12 or 24-hr clocks,
leading-zero or suppressed, am/pm or a.m./p.m. or some other signifier,
different separators...

The object is to establish the first column as CCYYMMDDHHMM. It doesn't
matter if CC is missing unless you're interested in pre-2000 files. It
doesn't matter if separators are included consistently in the same column.
It does matter that spaces are replaced by "0" in that column, otherwise the
for/f in [13] won't work properly.

The details of how to convert local-dependent data to a universal form
should be routine to any batch programmer. Covered by the FAQ, too.

[Cathy]

Hi and thanks once again for all the responses.

Sorry my previous mail should have said I ended up going back to
"foxidrive's" request, not "Timos"

Anyhow here is what I ended up with in the end

***deloldst.bat***
@echo off
::************************************************************

:: delete oldest from subtree until freespace > specified size

::************************************************************

set /A free_space=0
set /A space_required=2147483640
set "temp_file=%temp%\~temp#.###"
set dir_name=c:\a

:loop
del "%temp_file%" >nul 2>nul

:: check the diskspace free
for /f "tokens=3 delims= " %%a in ('dir "%dir_name%" /-C ^|find "bytes free"
') do ( set /A "free_space=%%a"
)
echo free_space = %free_space%

:: if less than 2GB
if %free_space% EQU 0 goto :EXIT
if %free_space% GTR %space_required% goto :EXIT

:: If No files in the directory then exit dir "%dir_name%" /s /b /a-d >nul
2>nul IF ERRORLEVEL 1 echo No files to delete IF ERRORLEVEL 1 goto EXIT:

:: take a list of files with a for/f-in-do using a dir /s /b /a-d command
:: echo into a file the last modified date in yyyymmddhhmm format followed
by the drv:\path\filename for /f "delims=" %%a in ('dir "%dir_name%" /s /b
/a-d') do (call subfoldr "%%a">%temp_file%)

:: use sort to get the list in date order sort %temp_file% /R /O %temp_file%

::and then another for-in-do to extract the oldest entry, for /f
"tokens=1,2delims=," %%i in (%temp_file%) do set file_name=%%j

:: get the filename and delete it.
echo del %file_name%
del %file_name%

goto :loop

:: Clean up set variables and exit
:EXIT
del "%temp_file%" >nul 2>nul
set free_space=
set space_required=
set file_name=
set temp_file=
set dir_name=
***end of deloldst.bat***

***subfoldr.bat***
@echo off
set filename_=%~1
if not exist "%filename_%" (
echo Files "%filename_%" not found
goto :EOF)
set fdate_=%~t1
set fdd_=%fdate_:~0,2%
set fmm_=%fdate_:~3,2%
set fyyyy_=%fdate_:~6,4%
set fhh_=%fdate_:~11,2%
set fmin_=%fdate_:~14,2%
echo %fyyyy_%%fmm_%%fdd_%%fhh_%%fmin_%,"%filename_%"
set fdate_=
set fdd_=
set fmm_=
set fyyyy_=
set fhh_=
set fmin_=
***end of subfoldr.bat****

[Cathy]

After transferring this to run on server I run into a small problem
regarding the amount of space I specified

I have speified 2gb, but the server fires an alert if it reaches less than
10% storage on the drive and the whole purpose of me writing this is to
avoid ever getting these alerts. The disk size is 277gb so the alert is
fired when less than 2.77gb free space is available

Unfortunately I dont seem to be able to go much higher than to gb in the
format I am using. Some message about only being able to use 32bit numbers

So back to the drawing board until I work that one out

Cathy

[billious]

Daddy's flown across the ocean...

[foxidrive]

You can use WSH to get the diskspace free and creating a flag for the batch file to use, or maybe write it all in vbs.

Billious used a technique comparing the trailing bytes and string lengths (I didn't look closely, billious) but it's probably going to confuse you or your co-workers down the road when it needs to be modified.



--
Mic