重复使用计算的md5（或任何其他校验和）

Question

尝试计算深度目录树中所有文件的增量md5摘要，但我无法“重用”已计算的摘要。

这里是我的测试代码：

#!/usr/bin/env perl 
use 5.014; 
use warnings; 
use Digest::MD5; 
use Path::Tiny; 

# create some test-files in the tempdir 
my @filenames = qw(a b); 
my $testdir = Path::Tiny->tempdir; 
$testdir->child($_)->spew($_) for @filenames; #create 2 files 

dirmd5($testdir, @filenames); 
exit; 

sub dirmd5 { 
    my($dir, @files) = @_; 

    my $dirctx = Digest::MD5->new; #the md5 for the whole directory 

    for my $fname (@files) { 

     # calculate the md5 for one file 
     my $filectx = Digest::MD5->new; 
     my $fd = $dir->child($fname)->openr_raw; 
     $filectx->addfile($fd); 
     close $fd; 
     say "md5 for $fname : ", $filectx->clone->hexdigest; 

     # want somewhat "add" the above file-md5 to the directory md5  
     # this not work - even if the $filectx isn't reseted (note the "clone" above) 
     #$dirctx->add($filectx); 

     # works adding the file as bellow, 
     # but this calculating the md5 again 
     # e.g. for each file the calculation is done two times... 
     # once for the file-alone (above) 
     # and second time for the directory 
     # too bad if case of many and large files. ;(
     # especially, if i want calculate the md5sum for the whole directory trees 
     $fd = $dir->child($fname)->openr_raw; 
     $dirctx->addfile($fd); 
     close $fd; 
    } 
    say "md5 for dir: ", $dirctx->hexdigest; 
} 

上面打印：

md5 for a : 0cc175b9c0f1b6a831c399e269772661 
md5 for b : 92eb5ffee6ae2fec3ad71c777531578f 
md5 for dir: 187ef4436122d1cc2f40dc2b92f0eba0 

这是正确的，但不幸的是低效率的方式。 （见评论）。

阅读the docs，我没有找到任何方式重用已经计算出来的md5。例如如上面的$dirctx->add($filectx);。可能这是不可能的。

存在任何检查和总结方法，它允许在某种程度上重新使用已计算的校验和，因此，我可以计算整个目录树的校验和/摘要，而无需为每个文件计算多次摘要？

编号：试图有所解决this question

Answer 1

号没有什么，涉及MD5(initial data)和MD5(new data)到MD5(initial data + new data)因为位置在流事务中的数据，以及其价值的。否则它不会是一个非常有用的错误检查为aba，aab和baa可能都具有相同的校验

如果文件足够小，你可以做阅读每一个入内存，并使用该副本，将数据添加到这两个摘要。这将避免大容量存储读取两次

#!/usr/bin/env perl 

use 5.014; 
use warnings 'all'; 

use Digest::MD5; 
use Path::Tiny; 

# create some test-files in the tempdir 
my @filenames = qw(a b); 
my $testdir = Path::Tiny->tempdir; 
$testdir->child($_)->spew($_) for @filenames; # create 2 files 

dirmd5($testdir, @filenames); 

sub dirmd5 { 
    my ($dir, @files) = @_; 

    my $dir_ctx = Digest::MD5->new; #the md5 for the whole directory 

    for my $fname (@files) { 

     my $data = $dir->child($fname)->slurp_raw; 

     # calculate the md5 for one file 
     my $file_md5 = Digest::MD5->new->add($data)->hexdigest; 
     say "md5 for $fname : $file_md5"; 

     $dir_ctx->add($data); 
    } 

    my $dir_md5 = $dir_ctx->hexdigest; 
    say "md5 for dir: $dir_md5"; 
} 

如果文件是巨大的，那么唯一的优化左边是为了避免再度同一个文件，而是读了第二遍之前倒带回到起点

#!/usr/bin/env perl 

use 5.014; 
use warnings 'all'; 

use Digest::MD5; 
use Path::Tiny; 
use Fcntl ':seek'; 

# create some test-files in the tempdir 
my @filenames = qw(a b); 
my $testdir = Path::Tiny->tempdir; 
$testdir->child($_)->spew($_) for @filenames; # create 2 files 

dirmd5($testdir, @filenames); 

sub dirmd5 { 
    my ($dir, @files) = @_; 

    my $dir_ctx = Digest::MD5->new; # The digest for the whole directory 

    for my $fname (@files) { 

     my $fh = $dir->child($fname)->openr_raw; 

     # The digest for just the current file 
     my $file_md5 = Digest::MD5->new->addfile($fh)->hexdigest; 
     say "md5 for $fname : $file_md5"; 

     seek $fh, 0, SEEK_SET; 
     $dir_ctx->addfile($fh); 
    } 

    my $dir_md5 = $dir_ctx->hexdigest; 
    say "md5 for dir: $dir_md5"; 
}