读取文件到特定行php

Question

我试图读取一个大文件（大约500万行），它保持达到内存限制。有没有办法将文件读到特定的行，然后增加一个计数器并从下一行继续？

这是我正在使用的代码，我如何添加一个指向fgets起始行的指针？

$handle = @fopen("large_file.txt", "r"); 
if($handle){ 
    while(($buffer = fgets($handle, 4096)) !== false){ 
     //get the content of the line 
    } 
} 

我并不想读只是一个特定的行，我试图从比如说1号线读排队10,000，然后10,001行再开始另一个10000行，这样的。

Answer 1

尝试使用这个功能，你应该使用迭代和范围得到线。

$file = new SplFileObject('yourfile.txt'); 

echo getLineRange(1,10000); 

function getLineRange($start,$end){ 
    $tmp = ""; 

    for ($i = $start; $i <= $end; $i++) { 
    $tmp .= $file->seek($i); 
    } 
    return($tmp); 
} 

Answer 2

我想答案会在这里Reading specific line of a file in php

可以使用寻求获取特定的线位置

$file = new SplFileObject('yourfile.txt'); 
$file->seek(123); // seek to line 124 (0-based) 

Answer 3

批块处理大型文件可以在PHP中使用fseek()/ftell()和保存块之间的背景下完成的。 （SplFileObject::seek()可以直接寻找，但似乎有performance issues with large files。）

假设你有某种批处理器可用，下面的示例应该给你一个方法的想法。它没有经过测试，但是来源于生产中的代码。

<?php 

$context = array(
    'path' => 'path/to/file', 
    'limit' => 1000, 
    'line' => NULL, 
    'position' => NULL, 
    'size' => NULL, 
    'percentage' => 0, 
    'complete' => FALSE, 
    'error' => FALSE, 
    'message' => NULL, 
); 

function do_chunk($context) { 
    $handle = fopen($context['path'], 'r'); 
    if (!$handle) { 
     $context['error'] = TRUE; 
     $context['message'] = 'Cannot open file for reading: ' . $context['path']; 
     return; 
    } 
    // One-time initialization of file parameters. 
    if (!isset($context['size'])) { 
     $fstat = fstat($handle); 
     $context['size'] = $fstat['size']; 
     $context['position'] = 0; 
     $context['line'] = 0; 
    } 
    // Seek to position for current chunk. 
    $ret = fseek($handle, $context['position']); 
    if ($ret === -1) { 
     $context['error'] = TRUE; 
     $context['message'] = 'Cannot seek to ' . $context['position']; 
     fclose($handle); 
     return; 
    } 

    $k = 1; 
    do { 
     $context['line']++; 
     $raw_line = fgets($handle); 
     if ($raw_line) { 
      // Strip newline. 
      $line = rtrim($raw_line); 
      // Code to process line here. 
      list($error, $message) = my_process_line($context, $line); 
      if ($error) { 
       $context['error'] = TRUE; 
       $context['message'] = $message; 
       fclose($handle); 
       return; 
      } 
     } elseif (($raw_line === FALSE) && !feof($handle)) { 
      $context['error'] = TRUE; 
      $context['message'] = 'Unexpected error reading ' . $context['path']; 
      fclose($handle); 
      return; 
     } 
    } 
    while ($k++ < $context['limit'] && $raw_line); 

    // Save position of next chunk. 
    $position = ftell($handle); 
    if ($position !== FALSE) { 
     $context['position'] = $position; 
    } else { 
     $context['error'] = TRUE; 
     $context['message'] = 'Cannot retrieve file pointer in ' . $context['path']; 
     fclose($handle); 
     return; 
    } 

    if (!$raw_line) { 
     $context['complete'] = TRUE; 
     $context['percentage'] = 1; 
    } else { 
     $context['percentage'] = $context['position']/$context['size']; 
    } 

    fclose($handle); 
} 


// Batch driver for testing only - use a batch processor in production. 
while ($context['complete']) { 
    do_batch($context); 
} 
if ($context['error']) { 
    print 'error: ' . $context['message']; 
} else { 
    print 'complete'; 
}