Powershell and sync data

[Cyber-Guy]

Greetings,
Is there anyone that has tried to use powershell as a simple sync of data
for backup?
For example from PC to and external HDD

many thanks

[Brandon Shell]

This should be pretty simple. What is your goal?

Master/Slave?
Master/Master?
Conflict resolution?

"Cyber-Guy" <cybe...@netmadeira.com> wrote in message
news:%23oC5sf%236HH...@TK2MSFTNGP06.phx.gbl...

[Hal Rottenberg]

Brandon Shell wrote:
> This should be pretty simple. What is your goal?
>
> Master/Slave?
> Master/Master?
> Conflict resolution?

Yes, yes, and yes. :)

I just tried using foldershare.com the other day to sync my PS $profiledir and
some other things for work<->home syncing. And I've been using SyncToy from MS
for some local<->\\network syncing as well. Those work fine, but it's not
Powershell. And if it's not Powershell, it's CRAP! [read in a Scottish accent.]

--

Hal Rottenberg
blog: http://halr9000.com
powershell category:
http://halr9000.com/article/category/programming/scripting/powershell/

[Brandon Shell]

Hmmm... lemme see what I can put together.
Master/Slave is a no brainer
Master/Master we can use file Hash and date

"Hal Rottenberg" <h...@halr9000.com> wrote in message
news:%23eFlbN$6HHA...@TK2MSFTNGP03.phx.gbl...

[Brandon Shell]

Any thoughts on the best way to compare the source and destination? Anyone?
I have a function to get a MD5 of the files, but I'm just wonder what others
think is a good idea for the compare.

"Brandon Shell" <tshel...@gmail.com> wrote in message
news:OqxxF6$6HHA...@TK2MSFTNGP06.phx.gbl...

[Brandon Shell]

Here is what I got... recommendations for performance are welcome.

Basically it gets all the files/folders from both the Source and
Destination.
- If first checks for folders and creates them if they are missing.
- It Checks the files from the Source in the Destination
1) If they exists it copies which ever is newer.
2) If they are missing it copies them to Destination
- It Checks the files from the Destination in the Source
1) If they are missing it copies them to Destination

###################################################
###################################################
Param($Source,$Destination)
#$Source = "C:\temp\Source"
#$Destination = "C:\temp\Destination"
function Get-FileMD5 {
Param([string]$file)
$mode = [System.IO.FileMode]("open")
$access = [System.IO.FileAccess]("Read")
$md5 = New-Object System.Security.Cryptography.MD5CryptoServiceProvider
$fs = New-Object System.IO.FileStream($file,$mode,$access)
$Hash = $md5.ComputeHash($fs)
$fs.Close()
[string]$Hash = $Hash
Return $Hash
}
function Copy-LatestFile{
Param($File1,$File2,[switch]$whatif)
$File1Date = get-Item $File1 | foreach-Object{$_.LastWriteTimeUTC}
$File2Date = get-Item $File2 | foreach-Object{$_.LastWriteTimeUTC}
if($File1Date -gt $File2Date)
{
Write-Host "$File1 is Newer... Copying..."
if($whatif){Copy-Item -path $File1 -dest $File2 -force -whatif}
else{Copy-Item -path $File1 -dest $File2 -force}
}
else
{
Write-Host "$File2 is Newer... Copying..."
if($whatif){Copy-Item -path $File2 -dest $File1 -force -whatif}
else{Copy-Item -path $File2 -dest $File1 -force}
}
Write-Host
}

# Getting Files/Folders from Source and Destination
$SrcEntries = Get-ChildItem $Source -Recurse
$DesEntries = Get-ChildItem $Destination -Recurse

# Parsing the folders and Files from Collections
$Srcfolders = $SrcEntries | Where-Object{$_.PSIsContainer}
$SrcFiles = $SrcEntries | Where-Object{!$_.PSIsContainer}
$Desfolders = $DesEntries | Where-Object{$_.PSIsContainer}
$DesFiles = $DesEntries | Where-Object{!$_.PSIsContainer}

# Checking for Folders that are in Source, but not in Destination
foreach($folder in $Srcfolders)
{
$SrcFolderPath = $source -replace "\\","\\" -replace "\:","\:"
$DesFolder = $folder.Fullname -replace $SrcFolderPath,$Destination
if(!(test-path $DesFolder))
{
Write-Host "Folder $DesFolder Missing. Creating it!"
new-Item $DesFolder -type Directory | out-Null
}
}

# Checking for Folders that are in Destinatino, but not in Source
foreach($folder in $Desfolders)
{
$DesFilePath = $Destination -replace "\\","\\" -replace "\:","\:"
$SrcFolder = $folder.Fullname -replace $DesFilePath,$Source
if(!(test-path $SrcFolder))
{
Write-Host "Folder $SrcFolder Missing. Creating it!"
new-Item $SrcFolder -type Directory | out-Null
}
}

# Checking for Files that are in the Source, but not in Destination
foreach($entry in $SrcFiles)
{
$SrcFullname = $entry.fullname
$SrcName = $entry.Name
$SrcFilePath = $Source -replace "\\","\\" -replace "\:","\:"
$DesFile = $SrcFullname -replace $SrcFilePath,$Destination
if(test-Path $Desfile)
{
$SrcMD5 = Get-FileMD5 $SrcFullname
$DesMD5 = Get-FileMD5 $DesFile
If(Compare-Object $srcMD5 $desMD5)
{
Write-Host "The Files MD5's are Different... Checking Write
Dates"
Write-Host $SrcMD5
Write-Host $DesMD5
Copy-LatestFile $SrcFullname $DesFile
}
}
else
{
Write-Host "$Desfile Missing... Copying from $SrcFullname"
copy-Item -path $SrcFullName -dest $DesFile -force
}
}

# Checking for Files that are in the Destinatino, but not in Source
foreach($entry in $DesFiles)
{
$DesFullname = $entry.fullname
$DesName = $entry.Name
$DesFilePath = $Destination -replace "\\","\\" -replace "\:","\:"
$SrcFile = $DesFullname -replace $DesFilePath,$Source
if(!(test-Path $SrcFile))
{
Write-Host "$SrcFile Missing... Copying from $DesFullname"
copy-Item -path $DesFullname -dest $SrcFile -force
}
}

"Brandon Shell" <tshel...@mk.gmail.com> wrote in message
news:uC8XtFR7...@TK2MSFTNGP03.phx.gbl...

[Flowering Weeds]

"Brandon Shell"

> Any thoughts

> I have a function to get a MD5 of the files,

Habit #6: Don't Write Insecure Code
http://msdn.microsoft.com/msdnmag/issues/06/11/SecureHabits/#S6

[Hal Rottenberg]

Flowering Weeds wrote:
>> I have a function to get a MD5 of the files,
>
> Habit #6: Don't Write Insecure Code
> http://msdn.microsoft.com/msdnmag/issues/06/11/SecureHabits/#S6

The article is talking about using MD5 for encryption, isn't it? As far as I
can tell, this has no impact on security to use MD5 to compare files.

[Brandon Shell]

How is this insecure?

"Flowering Weeds" <flowering...@hotmail.com> wrote in message
news:eQ3iMxZ7...@TK2MSFTNGP04.phx.gbl...

[Flowering Weeds]

"Hal Rottenberg"

>
> The article is talking about using
> MD5 for encryption, isn't it? As
> far as I can tell, this has no impact
> on security to use MD5 to compare files.
>

System.Security.Cryptography.HashAlgorithm Class

"Represents the base class from which all
implementations of cryptographic hash
algorithms must derive."

So to most, concerned with security,
"with regard to cryptography" seems
to apply to "compare files" too. :)

[Brandon Shell]

I am just using MD5 to see if the files are the same. It has nothing to do
with encryption in my case

BTW... Script is done and posted on my blog
http://bsonposh.com/modules/wordpress/?p=38

"Flowering Weeds" <flowering...@hotmail.com> wrote in message

news:OB2AuH%237HH...@TK2MSFTNGP03.phx.gbl...

[Oisin Grehan]

On Sep 5, 1:22 pm, "Flowering Weeds" <floweringnoweed...@hotmail.com>
wrote:

As far as comparing files is concerned with security, it depends on
what you're trying to accomplish. If you're using md5 hashes to ensure
file integrity, e.g. you compare the hashes from the live files to
your pre-generated static list of hashes every day to check if someone
has replaced one with their own nasty version. Using md5 for this is
not really a good idea anymore. Older systems like Tripwire etc used
to use md5, but since then various problems have been discovered in
this algorithm that allow you to find what they call "collisions."
This is where you have two files with different contents with the same
md5 signature. Therefore it's difficult, but not impossible, to tailor-
make some malware that matches a real file as far as md5 hashing is
concerned.

So, if you're just trying to sync two folders, well, you just have to
weigh up the risks.

Hope this helps,

- Oisin