mirror/zfs-inplace-rebalancing
1
0
Fork 1
mirror of https://github.com/markusressel/zfs-inplace-rebalancing synced 2026-02-04 21:14:09 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update README.md to denote hardlink support

Browse Source 
Adds details around the debug flag, hardlink support, removed --skip-hardlinks functionality, and temporary files used during the script processing.
This commit is contained in:
undaunt 2024-11-06 13:04:38 -08:00 committed by GitHub
parent 2dabc4bc4a
commit d5678de7a5
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 6 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
README.md
View File

@ -7,6 +7,8 @@ Simple bash script to rebalance pool data between all mirrors when adding vdevs
This script recursively traverses all the files in a given directory. Each file is copied with a `.balance` suffix, retaining all file attributes. The original is then deleted and the *copy* is renamed back to the name of the original file. When copying a file ZFS will spread the data blocks across all vdevs, effectively distributing/rebalancing the data of the original file (more or less) evenly. This allows the pool data to be rebalanced without the need for a separate backup pool/drive.
When the script detects an inode group of hardlinked files, it will proceed to copy one file in the inode group. The original file and all hardlinks are then deleted, the *copy* is renamed back to the name of the original file, and new hardlinks are generated from that copy to replace all other linked files that were removed.
The way ZFS distributes writes is not trivial, which makes it hard to predict how effective the redistribution will be. See:
- https://jrs-s.net/2018/04/11/zfs-allocates-writes-according-to-free-space-per-vdev-not-latency-per-vdev/
- https://jrs-s.net/2018/08/24/zfs-write-allocation-in-0-7-x/
@ -28,6 +30,8 @@ Since file attributes are fully retained, it is not possible to verify if an ind
1
```
The hardlink support process creates temporary files in the script location alongside `rebalance_db.txt` which are removed upon the end of each run. `files_list.txt` lists all files found in the given target location. `sorted_files_list.txt` lists all files sorted by inode number. `grouped_inodes.txt` lists all files by inode, but with all files from a given inode space separated on one line.
## Prerequisites
### Balance Status
@ -83,6 +87,7 @@ chmod +x ./zfs-inplace-rebalancing.sh
Dependencies:
* `perl` - it should be available on most systems by default
* `awk` - it should be available on most systems by default
## Usage
@ -100,7 +105,7 @@ You can print a help message by running the script without any parameters:
|-----------|-------------|---------|
| `-c`<br>`--checksum` | Whether to compare attributes and content of the copied file using an **MD5** checksum. Technically this is a redundent check and consumes a lot of resources, so think twice. | `true` |
| `-p`<br>`--passes` | The maximum number of rebalance passes per file. Setting this to infinity by using a value `<= 0` might improve performance when rebalancing a lot of small files. | `1` |
| `--skip-hardlinks` | Skip rebalancing hardlinked files, since it will only create duplicate data. | `false` |
| `--debug` | Shows additional output, including listing all files in the target location 3 times (list, inode sorted list, inode groupings) and more granular move/copy/link/count transaction information. | `false` |
### Example