Recursive searching for subfolders in a folder
PowerShellFajita
subfolders within a folder. Performance of the script is terrible because
the logic that I've found to recursively search for subfolders in a folder
has to search through every item in a folder and filter out just the folders.
I'm using the following logic to get the collection of subfolders in a
folder:
foreach ($dir in get-childitem $path -Recurse |Where {$_.psIsContainer -eq
$true})
It takes a very long time to execute this on a large folder structure with
many files. Is there a better way to do this in PowerShell? I ended up
writing this in VBScript.
Thanks in advance!
Flowering Weeds
>
> It takes a very long time to execute this on a large
> folder structure with many files. Is there a better
> way to do this in PowerShell?
>
Remember, Windows PowerShell
is not a data parser, but is a
Windows-based automation tool,
meant to pass "data"
from tool to tool, to tool,
until output (or whatever)!
Mmm automate tools and there really are
lots of Windows-based tools available,
for Windows PowerShell to automateI
So for here perhaps (like so many IT Pros)
automate a data parser tool within powershell.exe!
Perhaps Microsoft's IIS's (local or remote)
data parser, Log Parser!
PS > LogParser.exe -h -i:fs
Input format: FS (FileSystem properties)
Returns properties of files and folders
"Log Parser" - Bing
http://www.bing.com/search?q=%22Log+Parser%22
And surely after all these years,
any PowerShell user can help
one automate Log Parser too!
As always enjoy the automation
of tools within the Windows-based,
.NET aware, WPF accessible,
admin's automation tool,
powershell.exe!
![Keith Hill [MVP]'s profile photo](http://lh3.googleusercontent.com/a-/ALV-UjXbO2NJsBQgbJ_JN6AmQdt4evxsCCjmpY7dkJCsHg0RzH3_rQ=s40-c)
Keith Hill [MVP]
I don’t have an answer for you. What you have is probably the best (only)
way to go for now short of shelling out and using "cmd /c dir /ad /b $path".
Let me suggest that you vote on this issue:
https://connect.microsoft.com/PowerShell/feedback/ViewFeedback.aspx?FeedbackID=308796
If Get-ChildItem supported a switch to specify only containers (or only leaf
items), each provider could optimize the search on the provider side and in
theory provide much better performance for scenarios like the one you
outline below.
--
Keith
"PowerShellFajita" <PowerShe...@discussions.microsoft.com> wrote in
message news:2ABE5D44-F912-4C63...@microsoft.com...
tojo2000
wrote:
> I don’t have an answer for you. What you have is probably the best (only)
> way to go for now short of shelling out and using "cmd /c dir /ad /b $path".
> Let me suggest that you vote on this issue:
>
>
> If Get-ChildItem supported a switch to specify only containers (or only leaf
> items), each provider could optimize the search on the provider side and in
> theory provide much better performance for scenarios like the one you
> outline below.
>
> --
> Keith
>
> messagenews:2ABE5D44-F912-4C63...@microsoft.com...
>
>
>
> > I'm trying to write a PowerShell script that recursively searches for all
> > subfolders within a folder. Performance of the script is terrible because
> > the logic that I've found to recursively search for subfolders in a folder
> > has to search through every item in a folder and filter out just the
> > folders.
> > I'm using the following logic to get the collection of subfolders in a
> > folder:
>
> > foreach ($dir in get-childitem $path -Recurse |Where {$_.psIsContainer -eq
> > $true})
>
> > It takes a very long time to execute this on a large folder structure with
> > many files. Is there a better way to do this in PowerShell? I ended up
> > writing this in VBScript.
>
> > Thanks in advance!
Whoah, not so fast. There is still nothing wrong with using the
Scripting.FileSystemObject model outside of VBScript.
$fs = New-Object -COM Scripting.FileSystemObject
This is a much more streamlined object model than the one used by Get-
ChildItem, even though it's not quite as PowerShell-ish.
Keith Hill [MVP]
as you say, it isn't quite as PowerShell-ish - which is all I meant. Using
the FSO is a good suggestion. Still, please vote on the connect issue. I
would dearly love to see this issue fixed in a future version.
--
Keith
"tojo2000" <tojo...@gmail.com> wrote in message
news:117049d6-6233-498f...@d9g2000prh.googlegroups.com...
> On Aug 25, 10:58 pm, "Keith Hill [MVP]" <r_keith_h...@hotmail.com>
> wrote:
>> I don�t have an answer for you. What you have is probably the best (only)
tojo2000
wrote:
> Using Scripting.FileSystemObject may not be technically "shelling out" but
> as you say, it isn't quite as PowerShell-ish - which is all I meant. Using
> the FSO is a good suggestion. Still, please vote on the connect issue. I
> would dearly love to see this issue fixed in a future version.
>
> --
> Keith
>
>
> news:117049d6-6233-498f...@d9g2000prh.googlegroups.com...
>
>
>
> > On Aug 25, 10:58 pm, "Keith Hill [MVP]" <r_keith_h...@hotmail.com>
> > wrote:
Definitely. Will-do.
PowerShellFajita
"Keith Hill [MVP]" wrote:
> Using Scripting.FileSystemObject may not be technically "shelling out" but
> as you say, it isn't quite as PowerShell-ish - which is all I meant. Using
> the FSO is a good suggestion. Still, please vote on the connect issue. I
> would dearly love to see this issue fixed in a future version.
>
> --
> Keith
>
> "tojo2000" <tojo...@gmail.com> wrote in message
> news:117049d6-6233-498f...@d9g2000prh.googlegroups.com...
> > On Aug 25, 10:58 pm, "Keith Hill [MVP]" <r_keith_h...@hotmail.com>
> > wrote:
> >> I dont have an answer for you. What you have is probably the best (only)
Larry__Weiss
BTW, the recent "Workaround" added to issue 308796 is not really a workaround at
all.
- Larry
Keith Hill [MVP] wrote:
> ... Let me suggest that you vote on this issue:
Martin Zugec
For Get-ChildItem, I got more complains:
1.) REMOVE PATH LIMITATION - today I had to rewrite 10 scripts to use cmd or
robocopy instead of Powershell, because it's not able to handle longer paths
:(
2.) Add option to specify depth - if I know that my items are in 1-2
subfolders, however there are 50 embedded folders, I would like to limit
search results. Currently I have to write my own wrapper functions to
achieve that :(
Martin
"Larry__Weiss" <l...@airmail.net> wrote in message
news:eJn0XY9W...@TK2MSFTNGP04.phx.gbl...
Larry__Weiss
I too hate it when a software abstraction like .NET imposes artificial limitations.
https://connect.microsoft.com/PowerShell/feedback/ViewFeedback.aspx?FeedbackID=276235
- Larry
Larry__Weiss
https://connect.microsoft.com/PowerShell/feedback/ViewFeedback.aspx?FeedbackID=276235
Posted by Stephen Mills on 1/16/2009 at 7:22 PM
This "Feature" of the .Net framework has also bitten the Windows Home Server
group. I really hope they get rid of that limitation since there were already
lots of companies using long path names (greater than 260 characters) before
.net even existed. Even some of the old Cmd tools supported very long file names
(robocopy did). We really need to be able to be able to copy the same files that
robocopy can or it is sort of a bad joke.
If this is dependant on the .Net framework, how many YEARS will we have to wait.
Unless the next version is in Windows 7/2008 Server r2, then it will probably be
at least 2 more years before .net has it and then we will probably have to wait
for the version of Powershell after that. So most likely it will be 3-5 years
for this critical feature to make it to PowerShell. This is sad, because it has
been a known issue for a long time. I just wish both groups would have decided
that it was unacceptable long ago and fixed it.
- Larry
Larry__Weiss wrote:
> I too hate it when a software abstraction like .NET imposes artificial
> limitations.
> https://connect.microsoft.com/PowerShell/feedback/ViewFeedback.aspx?FeedbackID=276235
>
Martin Zugec
Very, very annoying - and this limitation (logically) doesn't affect
Explorer or cmd, so there is nothing to prevent you from creating these
structures (well, NTFS supports 32,000 characters as far as I remember) :(
Most people will think that 260 characters is enough. It's not:
\\UKRAGTH1TD15.testlab.microsoft.com\Installations$\Office2003 is already 62
characters long and this problem affects not only copy-item or remove-item,
but also get-childitem - combined with missing -Depth argument for recursive
dir, it means that parsing such folders is not possible...
I hate the feeling that I can do miracles in Posh, but because of this
limitation I have to call cmd.exe again :(
Martin
"Larry__Weiss" <l...@airmail.net> wrote in message
news:uUfAQsKX...@TK2MSFTNGP05.phx.gbl...
Larry__Weiss
- That it exists at all.
- That is exists in an underlying abstraction so the POSH developers alone can't
fix it.
I would challenge the POSH developers to workaround this one somehow until the
real fix is in.
I'm afraid that Jeffrey Snover doesn't read these posts. He is active on
Twitter, so maybe I'll get his attention there someday.
- Larry
Larry__Weiss
http://blogs.msdn.com/powershell/archive/2009/11/04/why-is-get-childitem-so-slow.aspx
describes the current thinking about PowerShell's future with respect to the
.NET file system API, including a "Why Don’t We Fix It?" section regarding why
they are *not* going to emulate cmd.exe in the short-term.
- Larry
Martin Zugec wrote:
> Well, that bug report is dated 2007 :(
> Very, very annoying - and this limitation (logically) doesn't affect
> Explorer or cmd, so there is nothing to prevent you from creating these
> structures (well, NTFS supports 32,000 characters as far as I remember) :(
> Most people will think that 260 characters is enough. It's not:
> \\UKRAGTH1TD15.testlab.microsoft.com\Installations$\Office2003 is
> already 62 characters long and this problem affects not only copy-item
> or remove-item, but also get-childitem - combined with missing -Depth
> argument for recursive dir, it means that parsing such folders is not
> possible...
> I hate the feeling that I can do miracles in Posh, but because of this
> limitation I have to call cmd.exe again :(
>
> "Larry__Weiss"
>> Some pretty pessimistic comments schedule-wise to get this handled from
>> https://connect.microsoft.com/PowerShell/feedback/ViewFeedback.aspx?FeedbackID=276235
>> Posted by Stephen Mills on 1/16/2009 at 7:22 PM
>> This "Feature" of the .Net framework has also bitten the Windows Home
>> Server group. I really hope they get rid of that limitation since
>> there were already lots of companies using long path names (greater
>> than 260 characters) before .net even existed. Even some of the old
>> Cmd tools supported very long file names (robocopy did). We really
>> need to be able to be able to copy the same files that robocopy can or
>> it is sort of a bad joke.
>> If this is dependant on the .Net framework, how many YEARS will we
>> have to wait. Unless the next version is in Windows 7/2008 Server r2,
>> then it will probably be at least 2 more years before .net has it and
>> then we will probably have to wait for the version of Powershell after
>> that. So most likely it will be 3-5 years for this critical feature to
>> make it to PowerShell. This is sad, because it has been a known issue
>> for a long time. I just wish both groups would have decided that it
>> was unacceptable long ago and fixed it.
>>
Larry__Weiss
http://blogs.msdn.com/powershell/archive/2009/11/04/why-is-get-childitem-so-slow.aspx
describes the current thinking about PowerShell's future with respect to the
.NET file system API, including a "Why Don’t We Fix It?" section regarding why
they are *not* going to emulate cmd.exe in the short-term.
- Larry