os.walk创建一个内核恐慌

Question

我想走路的目录根据

$ find . -type f | wc -l 

我每次运行使用os.walk剧本中有大约400K的文件，它发送我的笔记本电脑变成一个内核恐慌。它非常一致。

OSX Mountain Lion，Python 2.7.4。

任何想法？

编辑：示例代码

import os 


def collect_files_for_path(path): 
    all_files = [] 

    for root, dirs, files in os.walk(path): 

     for f in files: 
      full = os.path.join(root, f) 
      all_files.append(full) 

    return all_files 


files = collect_files_for_path('some/path') 

编辑2：恐慌日志

Tue Apr 23 11:17:22 2013 
panic(cpu 0 caller 0xffffff800d4ee9a5): "hfs_lock: locking against myself!"@/SourceCache/xnu/xnu-2050.22.13/bsd/hfs/hfs_cnode.c:1751 
Backtrace (CPU 0), Frame : Return Address 
0xffffff813aca3390 : 0xffffff800d21d626 
0xffffff813aca3400 : 0xffffff800d4ee9a5 
0xffffff813aca3430 : 0xffffff800d518563 
0xffffff813aca34b0 : 0xffffff800d3127ff 
0xffffff813aca3500 : 0xffffff800d309001 
0xffffff813aca3540 : 0xffffff800d32584c 
0xffffff813aca35f0 : 0xffffff800d325ee5 
0xffffff813aca3780 : 0xffffff800d511279 
0xffffff813aca37b0 : 0xffffff800d4fc953 
0xffffff813aca3830 : 0xffffff800d312a21 
0xffffff813aca38c0 : 0xffffff800d2daa5e 
0xffffff813aca39b0 : 0xffffff800d312a7e 
0xffffff813aca39e0 : 0xffffff800d2dc68c 
0xffffff813aca3a10 : 0xffffff800d2dc7e7 
0xffffff813aca3a30 : 0xffffff800d5177d0 
0xffffff813aca3a80 : 0xffffff800d312410 
0xffffff813aca3ab0 : 0xffffff800d2eac05 
0xffffff813aca3b50 : 0xffffff800d2ea1a4 
0xffffff813aca3c10 : 0xffffff800d304a32 
0xffffff813aca3d90 : 0xffffff800d2fd40c 
0xffffff813aca3f50 : 0xffffff800d5e063a 
0xffffff813aca3fb0 : 0xffffff800d2cdd23 

BSD process name corresponding to current thread: Python 

Mac OS version: 
12D78 

Kernel version: 
Darwin Kernel Version 12.3.0: Sun Jan 6 22:37:10 PST 2013; root:xnu-2050.22.13~1/RELEASE_X86_64 
Kernel UUID: 3EB7D8A7-C2D3-32EC-80F4-AB37D61492C6 
Kernel slide:  0x000000000d000000 
Kernel text base: 0xffffff800d200000 
System model name: MacBookPro8,1 (Mac-94245B3640C91C81) 

System uptime in nanoseconds: 299018314244 
last loaded kext at 77719537176: org.virtualbox.kext.VBoxNetAdp 4.2.1 (addr 0xffffff7f8ef2b000, size 28672) 
last unloaded kext at 182522584862: com.apple.driver.AppleUSBUHCI 5.2.5 (addr 0xffffff7f8dbf3000, size 65536) 
loaded kexts: 
org.virtualbox.kext.VBoxNetAdp 4.2.1 
org.virtualbox.kext.VBoxNetFlt 4.2.1 
org.virtualbox.kext.VBoxUSB 4.2.1 
org.virtualbox.kext.VBoxDrv 4.2.1 
org.pqrs.driver.PCKeyboardHack 8.0.0 
org.pqrs.driver.KeyRemap4MacBook 8.0.0 
com.radiosilenceapp.nke.PrivateEye 1 
com.Logitech.Unifying.HID Driver 1.2.0 
com.Logitech.Control Center.HID Driver 3.5.1 
com.apple.filesystems.msdosfs 1.8 
com.apple.driver.AppleHWSensor 1.9.5d0 
com.apple.iokit.IOBluetoothSerialManager 4.1.3f3 
com.apple.driver.AudioAUUC 1.60 
com.apple.filesystems.autofs 3.0 
com.apple.driver.AGPM 100.12.87 
com.apple.driver.AppleMikeyHIDDriver 122 
com.apple.driver.AppleHDA 2.3.7fc4 
com.apple.iokit.IOUserEthernet 1.0.0d1 
com.apple.driver.AppleMikeyDriver 2.3.7fc4 
com.apple.driver.AppleLPC 1.6.0 
com.apple.driver.AppleUpstreamUserClient 3.5.10 
com.apple.Dont_Steal_Mac_OS_X 7.0.0 
com.apple.driver.ApplePolicyControl 3.3.0 
com.apple.driver.ACPI_SMC_PlatformPlugin 1.0.0 
com.apple.driver.AppleIntelHD3000Graphics 8.1.0 
com.apple.iokit.BroadcomBluetoothHCIControllerUSBTransport 4.1.3f3 
com.apple.driver.AppleSMCLMU 2.0.3d0 
com.apple.driver.AppleSMCPDRC 1.0.0 
com.apple.driver.AppleIntelSNBGraphicsFB 8.1.0 
com.apple.driver.AppleBacklight 170.2.5 
com.apple.driver.AppleMCCSControl 1.1.11 
com.apple.driver.SMCMotionSensor 3.0.3d1 
com.apple.driver.AppleUSBTCButtons 237.1 
com.apple.driver.AppleUSBTCKeyboard 237.1 
com.apple.driver.AppleIRController 320.15 
com.apple.AppleFSCompression.AppleFSCompressionTypeDataless 1.0.0d1 
com.apple.AppleFSCompression.AppleFSCompressionTypeZlib 1.0.0d1 
com.apple.BootCache 34 
com.apple.iokit.SCSITaskUserClient 3.5.5 
com.apple.driver.XsanFilter 404 
com.apple.iokit.IOAHCIBlockStorage 2.3.1 
com.apple.driver.AppleUSBHub 5.5.5 
com.apple.driver.AirPort.Brcm4331 614.20.16 
com.apple.iokit.AppleBCM5701Ethernet 3.6.0b1 
com.apple.driver.AppleFWOHCI 4.9.6 
com.apple.driver.AppleSDXC 1.4.0 
com.apple.driver.AppleUSBEHCI 5.5.0 
com.apple.driver.AppleAHCIPort 2.5.1 
com.apple.driver.AppleSmartBatteryManager 161.0.0 
com.apple.driver.AppleEFINVRAM 1.7 
com.apple.driver.AppleIntelCPUPowerManagementClient 196.0.0 
com.apple.driver.AppleACPIButtons 1.7 
com.apple.driver.AppleRTC 1.5 
com.apple.driver.AppleHPET 1.8 
com.apple.driver.AppleSMBIOS 1.9 
com.apple.driver.AppleACPIEC 1.7 
com.apple.driver.AppleAPIC 1.6 
com.apple.nke.applicationfirewall 4.0.39 
com.apple.security.quarantine 2 
com.apple.driver.AppleIntelCPUPowerManagement 196.0.0 
com.apple.iokit.IOSerialFamily 10.0.6 
com.apple.kext.triggers 1.0 
com.apple.driver.DspFuncLib 2.3.7fc4 
com.apple.iokit.IOAudioFamily 1.8.9fc11 
com.apple.kext.OSvKernDSPLib 1.6 
com.apple.iokit.IOSurface 86.0.4 
com.apple.iokit.IOBluetoothFamily 4.1.3f3 
com.apple.driver.AppleHDAController 2.3.7fc4 
com.apple.iokit.IOHDAFamily 2.3.7fc4 
com.apple.iokit.IOFireWireIP 2.2.5 
com.apple.driver.IOPlatformPluginLegacy 1.0.0 
com.apple.driver.AppleSMBusPCI 1.0.11d0 
com.apple.iokit.AppleBluetoothHCIControllerUSBTransport 4.1.3f3 
com.apple.driver.IOPlatformPluginFamily 5.3.0d51 
com.apple.driver.AppleGraphicsControl 3.3.0 
com.apple.driver.AppleBacklightExpert 1.0.4 
com.apple.iokit.IONDRVSupport 2.3.7 
com.apple.driver.AppleSMBusController 1.0.11d0 
com.apple.iokit.IOGraphicsFamily 2.3.7 
com.apple.driver.AppleSMC 3.1.4d2 
com.apple.iokit.IOSCSIBlockCommandsDevice 3.5.5 
com.apple.iokit.IOUSBMassStorageClass 3.5.1 
com.apple.driver.AppleUSBMultitouch 237.3 
com.apple.driver.AppleThunderboltDPInAdapter 1.8.9 
com.apple.driver.AppleThunderboltDPAdapterFamily 1.8.9 
com.apple.driver.AppleThunderboltPCIDownAdapter 1.2.6 
com.apple.iokit.IOUSBHIDDriver 5.2.5 
com.apple.driver.CoreStorage 296.16 
com.apple.driver.AppleUSBMergeNub 5.5.5 
com.apple.driver.AppleUSBComposite 5.2.5 
com.apple.iokit.IOSCSIMultimediaCommandsDevice 3.5.5 
com.apple.iokit.IOBDStorageFamily 1.7 
com.apple.iokit.IODVDStorageFamily 1.7.1 
com.apple.iokit.IOCDStorageFamily 1.7.1 
com.apple.iokit.IOAHCISerialATAPI 2.5.1 
com.apple.iokit.IOSCSIArchitectureModelFamily 3.5.5 
com.apple.driver.AppleThunderboltNHI 1.6.3 
com.apple.iokit.IOThunderboltFamily 2.2.6 
com.apple.iokit.IOUSBUserClient 5.5.5 
com.apple.iokit.IO80211Family 522.4 
com.apple.iokit.IOEthernetAVBController 1.0.2b1 
com.apple.iokit.IONetworkingFamily 3.0 
com.apple.iokit.IOFireWireFamily 4.5.5 
com.apple.iokit.IOAHCIFamily 2.3.1 
com.apple.iokit.IOUSBFamily 5.5.5 
com.apple.driver.AppleEFIRuntime 1.7 
com.apple.iokit.IOHIDFamily 1.8.1 
com.apple.iokit.IOSMBusFamily 1.1 
com.apple.security.sandbox 220.2 
com.apple.kext.AppleMatch 1.0.0d1 
com.apple.security.TMSafetyNet 7 
com.apple.driver.DiskImages 345 
com.apple.iokit.IOStorageFamily 1.8 
com.apple.driver.AppleKeyStore 28.21 
com.apple.driver.AppleACPIPlatform 1.7 
com.apple.iokit.IOPCIFamily 2.7.3 
com.apple.iokit.IOACPIFamily 1.4 
com.apple.kec.corecrypto 1.0 

Answer 1

你不应该能够惊慌的系统只是通过做os.walk。

你可能应该fsck你的文件系统（虽然卸载），并检查磁盘坏块。

Answer 2

如果find很高兴'走'的目录，这似乎更可能是问题是在内存中建立这个庞大的列表（all_files），这可能很容易占用4-8GB的RAM。

请问，如果你删除行仍出现问题...

all_files.append(full) 

...，是有你需要建立一个完整的列表，而不是做什么你打算与做任何理由使用迭代器列表返回os.walk()？

更新

打印单独的路径将系统送入内核崩溃。

很奇怪。如果在打印相同的文件名后它总是失败，那可能会给你一个关于哪个文件导致问题的线索。

否则，os.walk()的source code非常简单，并且使用纯Python编写，因此如果您在新文件中创建了该代码的副本并添加了一些调试消息，则可能有助于找出问题。尽管如此，最终，没有任何用户进程应该能够引起内核恐慌，这使得它成为一个OSX错误，所以你可能有更多的运气联系Apple客户服务 - 他们实际上得到报酬来解决这些问题。 ;-)

Answer 3

您的python代码不会导致内核恐慌。在Mac内核恐慌发生了极少数的原因：

1. 硬件问题
2. 错误在OS本身
3. 错误在安装内核驱动程序

Python不安装内核驱动程序，所以即使它是一致的，根本原因是你的python代码下面的东西。

我在您的恐慌追踪中看到virtualbox。这可能是你的问题。这里是描述如何阅读恐慌日志的页面：http://www.embracingchaos.com/2011/05/macbook-crashes-kernel-panics-and-coping-with-an-apple-genius.html