ZFS: the final straw
It's official. I am finally pronouncing ZFS 'buggy'.
I do not trust the integrity of my data with this filesystem, and I suggest that you do not use it for any purpose.
I've tried three platforms so far, trying to find a stable ZFS system. They are:
Didn't support any common SATA controllers. Crashed if you looked at it the wrong way. Actively user-hostile. Tiny user community, most of it hostile to Linux users and noobs.
Leaks memory and crashes about once a week, more if you use it. I can't complete a scrub or a complete copy of my storage pool (800GB) in one pass: the machine must be rebooted in the middle. Sometimes claims that the pools are fully intact but contain no data, which is somewhat disconcerting. Little hope for future updates as Sun has 'purchased' the developer that was working on the project.
User-hostile, but not nearly as bad as OpenSolaris. Also leaks memory and crashes if you use it a lot, but not as badly as Linux. Still has the precious-data-disappearing bug. Barfs if the drive arrays aren't just right when you boot up. An array that I created and copied data to just did not exist the next time I booted the machine.
So what's next?
I don't know. I thought my problems were specific to the OS implementation - maybe they were buggy because they're not from the ZFS authors - but the fact that the exact same bugs appear across multiple platforms suggests to me that the problem lies with ZFS, not particular ports of it.
I've had enough. I really, really want something that does checksumming; I'm using this array for long-term storage and ZFS has demonstrated that bits do rot over time.
XFS has the stability that I want, but no checksumming. Linux RAID is painful and dangerous. I'm tempted to just buy a hardware RAID controller, but I've found them to be even less reliable than software RAID, as well as costing a bundle. The promise of ZFS was for a simple, easy-to-administer disk storage system with redundancy and reliability on commodity hardware. Great idea, but poor implementation.
I've got my IntegriFS project, but of course, that's a long way off. And any filesystem that I write will be considered unstable for a long time anyway; I certainly wouldn't want to be keeping un-backed-up data on there.
Update 23 Feb 2008
Linux RAID6 + ext3 + USB drive boot. I've invested maybe three hours of labour into this (versus about a hundred for various ZFS schemes) and it works remarkably well. Linux RAID has come a long way since I last tried it - it's a whole lot harder to destroy an array with a typo. It's also quite flexible in the ways you can modify and resize an array after it has been created. Aaaand it has data scrubbing now, so you know (eventually) that there are checksum errors on your disks.
ext3 is surprisingly fast on the large-files workload - much faster than ZFS. Some directory listings used to take twenty seconds, and with ext3 they're instant.
USB booting works perfectly, of course.
So I'm very happy with the end result and will probably sleep a lot better.