List and Remove Corrupted files reported by Data Deduplication with Powershell

I’ve been copying 7TB of data in about 100.000 files from an old fileserver to the new one, but I just noticed that some of the files are corrupted! Gahhh…

Chkdsk found some issues, but didn’t solve the problem. As this server is running Windows Server 2012 R2 with Data Deduplication I decided to have a look at that. Data Deduplication Errors

Yeah, unfortunately a lot of corrupted files with EventID 12800

So Data Deduplication is reporting a lot of corrupted files, and this error message didn’t really make me any happier.

Hopefully this quick and dirty powershell script that I just wrote can help you too.
As I still had the working fileserver with working files available, I decided to just delete all corrupted files with this script.

And then ran a robocopy script to recopy everything (it will skip any files that already exists making it a quite fast process).
robocopy /mir /copyall /r:1 /w:1 \\source\path \\destination\path

Updated 2014-05-22 16:22:  Added a full delete and copy script, which is a bit better written;

 

 

7 thoughts on “List and Remove Corrupted files reported by Data Deduplication with Powershell”

  1. Hi Jason,
    if you would just like to get a list of the files, you can use this script;
    It should only list the files and not delete them.

    ######
    $corrupted = Get-WinEvent -LogName Microsoft-Windows-Deduplication/Scrubbing | where ID -EQ “12800” | Sort-Object -Property Message

    foreach ($entry in $corrupted) {
    # Write-Output $entry
    $file = “$((($entry | select -ExpandProperty Properties)[4]).value)$((($entry | select -ExpandProperty Properties)[5]).value)$((($entry | select -ExpandProperty Properties)[6]).value)”
    write-output “Corrupted file: $file”
    }

  2. Markus,

    The last script is awesome, and I would love to add it to my toolbox. My one question is, if I just want to pull the list of corrupted files and not remove them, in your first script what line(s) do I need to omit or modify.

  3. You are correct, but it only applies if you use the D:\ (any volume root) or point to a \\server\d$ (any volume root) as the Store is in the root of each Volume.
    I was using \\server\share and \\server\share$’s directly, so robocopy would not wipe the System files.

  4. Ha, erm, just read again. Ok, you had a relatively small number of files and still had an issue… I suppose that corruption can just happen for whatever reason… Still, not cool when it happens in the deduplication chunk store.

  5. Horrible… I’ve had the same thing. We monitor some event IDs now to pick this up early when it happens. Out of interest, how big was the volume you had this on? Not just in size, but particularly the number of files and directories? If you remember…

    We had an insanely large amount of files and directories. Microsoft publish a maximum supported deduplication size, but don’t specify the number of files/directories. Maybe they assume the worst case?

Leave a Reply