mirror/zfs-inplace-rebalancing
1
0
Fork 1
mirror of https://github.com/markusressel/zfs-inplace-rebalancing synced 2026-03-11 22:16:29 +00:00
Code Issues Packages Projects Releases Wiki Activity
104 Commits 2 Branches 0 Tags
a8beeaf951a8e70804c19a30c2aafd70f96334cf
Commit Graph

104 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Colin Hebert
a8beeaf951 Print whole line in debug 2025-03-23 12:09:46 +11:00
Colin Hebert
d4f52ee5a7 Make grep compatible with BSD/MacOS 2025-03-23 11:46:02 +11:00
Colin Hebert
34b59acc95 Fix pasting issue 2025-03-23 11:45:31 +11:00
Colin Hebert
a2d282d483 Fix ShellCheck issues 2025-03-23 11:26:08 +11:00
Colin Hebert
45053f9bee Update tests to assert the hardlinks are still working 2025-03-22 18:08:32 +11:00
Colin Hebert
4c67429c58 Add ignore on temporary inode files 2025-03-22 17:54:45 +11:00
Colin Hebert
7f5896f395 Fix broken test due to | in the filename 2025-03-22 17:51:13 +11:00
Colin Hebert
f830ed9d5f Remove unused flag 2025-03-22 17:50:48 +11:00
Colin Hebert
ae7cfe645e Remove unnecessary -e flag 2025-03-22 16:37:07 +11:00
Colin Hebert
f1de0295e6 Reintroduce cp documentation 2025-03-22 16:35:11 +11:00
Colin Hebert
d785020f7e Introduce echo_debug 2025-03-22 16:34:52 +11:00
Colin Hebert
adf4cf60d0 Merge branch 'master' into HEAD 2025-03-22 16:23:09 +11:00
Adi
97d11fdb0b 1.5x speed increase by using cmp instead of crc32 (#61) 
* 2x speed increase by using cmp instead of crc32

* Remove redundant cmp check
 2025-01-18 11:48:12 +01:00
meo
d190faf49c Expand test (#59) 
* add(ignore): ignore of .vscode file

add the vscode config file to gitignore

* add(test): add set of test

- test 1000 1Ko file
- test 5 1 Go file
- test whit both

* fix: spellcheck

prevent wordspliting

* fix: remove of db

remove the db before testing for more accurate time

* fix: time in macOS and freeBSD

- use of gdate (coreutils) for macOS
- switch to `ns` in place of `ms` for freeBSD
 2025-01-17 22:50:33 +01:00
Adi
9b233fdeaf Replace MD5 with CRC32 for 2x checksum speed (#57) 
* Replace MD5 with CRC32 for 2x checksum speed

* Use % builtin instead of awk for better performance
 2025-01-09 17:00:01 +01:00
meo
9e5aadff04 fix: remove the caching (#60) 
the caching for the vm do not work so i remove it
 2025-01-09 16:58:26 +01:00
dependabot[bot]
21d718259e Bump actions/cache from 3 to 4 (#58) 
Bumps [actions/cache](https://github.com/actions/cache) from 3 to 4.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-01-06 16:39:07 +01:00
meo
393cbe2a12 FreeBSD fix and test (#55) 
* add: test on freebsd

* test fix vm

* fix: transform to lowercase

transform the `OSTYPE` to a lowercase and store it inside `OSName`

* add: caching of the vm
 2025-01-06 00:43:43 +01:00
meo
bd8d447150 Test mac os (#53) 
* add: test on MacOS

Add test for MacOS in Github action

Maybe see this to add test for FreeBSD :
https://github.com/marketplace/actions/freebsd-vm

* add: log to test

adding log to the test for better understanding

* fix: shellcheck

remove the yellow color that is unused in the test

* fix: fail on lower-casting

the `"${variable,,}"` seam to not work on macOS so y remove it for the `skip_hardlink_flag` because it's always lowercase

* fix: macOS

remove all the lower-case casting

* fix(perf): going to `stat` remove `lsattr` macOS

The command `lsattr` is not supported on macOS so I have remove it
switch to `stat` instead of `ls` for better performance and no subshell
use of parameter expansion with  `%` in place of `awk` for performance and no subshell

* fix: forgot to remove variable
 2024-12-17 00:33:35 +01:00
meo
922531829e refactor: add regularity to spaces (#52) 
1) switch all the miscellaneous indentation to 4 spaces

2) remove the space before the parenthesis of function

3) add indentation inside of if

4) remove miscellaneous spaces at the end of lines
 2024-12-14 02:13:11 +01:00
undaunt
d7c74441a5 Fix echo bug 
Removed the 'recreating hardlinks' echo for inode groups of 1 file.
 2024-11-06 13:48:53 -08:00
undaunt
3ae15e565a Fix space handling in paths 2024-11-06 13:37:42 -08:00
undaunt
46c14fae53 Flip default debug 2024-11-06 13:11:17 -08:00
undaunt
1111a4027f typo 2024-11-06 13:08:08 -08:00
undaunt
8f63fe0b29 Add additional hardlink group database notes 2024-11-06 13:07:39 -08:00
undaunt
d5678de7a5 Update README.md to denote hardlink support 
Adds details around the debug flag, hardlink support, removed --skip-hardlinks functionality, and temporary files used during the script processing.
 2024-11-06 13:04:38 -08:00
undaunt
2dabc4bc4a Support hardlink groups, add debug 
-Adds debug functionality with extended details
-Supports detecting inode groups for hardlink processing.
-Pulls files and sorts by, then groups by inode group with awk
-Checks all files in an inode group's counts when calculating skipping counts
-Removes existing skip hardlink flag
-Removes hardlinks and recreates them directly after the balance copy/delete/move operation per inode group to minimize 'downtime'
 2024-11-06 12:50:55 -08:00
Moss Hope
4a6fb83c3d freebsd and macos fixes (#44) 
freebsd and macos share stat and sed history with some differences from
gnu

tested on freebsd and linux, mac just verified with man pages and stack
overflow

Co-authored-by: mosshope <moss@lararium.houseofdusk.online>
 2024-10-04 19:07:04 +02:00
Karol Kozlowski
9cea80a9c7 Remove dependence on perl (#41) 
* Remove dependence on perl

* Fix shellcheck SC2004

* Fix shellcheck SC2004
 2024-08-29 17:06:00 +02:00
dependabot[bot]
6087250290 Bump docker/build-push-action from 5 to 6 (#39) 
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 5 to 6.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](https://github.com/docker/build-push-action/compare/v5...v6)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-06-18 22:41:47 +02:00
Oliver Conzen
15672d2a02 Remove extraneous flags (#38) 2024-06-17 03:17:41 +02:00
Markus Ressel
979edb265c add --reflink=never to cp on linux (#34) 2023-11-11 11:08:36 +01:00
dependabot[bot]
d6cbeb3f40 Bump docker/build-push-action from 4 to 5 (#31) 
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 4 to 5.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](https://github.com/docker/build-push-action/compare/v4...v5)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-09-18 17:38:55 +02:00
dependabot[bot]
71684f1770 Bump docker/login-action from 2 to 3 (#30) 
Bumps [docker/login-action](https://github.com/docker/login-action) from 2 to 3.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/v2...v3)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-09-18 17:38:47 +02:00
dependabot[bot]
c7705ccde9 Bump docker/metadata-action from 4 to 5 (#29) 
Bumps [docker/metadata-action](https://github.com/docker/metadata-action) from 4 to 5.
- [Release notes](https://github.com/docker/metadata-action/releases)
- [Upgrade guide](https://github.com/docker/metadata-action/blob/master/UPGRADE.md)
- [Commits](https://github.com/docker/metadata-action/compare/v4...v5)

---
updated-dependencies:
- dependency-name: docker/metadata-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-09-18 17:38:39 +02:00
Markus Ressel
4cc7eb9814 move stale.yml to correct location 2023-09-04 17:25:26 +02:00
dependabot[bot]
f478d95bf6 Bump actions/checkout from 3 to 4 (#27) 
Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-09-04 17:21:22 +02:00
Gregory Zdanowski-House
2276727d6f Add rebalancing start date (#25) 2023-09-04 17:21:11 +02:00
Gregory Zdanowski-House
709b1cc2c6 Update dependency documentation (#26) 
The project no longer requires "bc", but instead relies on more common "perl" being preinstalled.
 2023-09-04 17:19:54 +02:00
Kenneth Ballard
b550a57615 Using perl to generate the percent text. "bc" isn't available on TrueNAS SCALE. (#24) 2023-08-21 18:10:01 +02:00
Karl Henselin
8a76414b43 Update README.md (#23) 2023-07-21 00:13:26 +02:00
johnpyp
973855c049 --skip-hardlinks flag (#22) 
* feat: --skip-hardlinks flag

Adds a --skip-hardlinks flag that will not process any files that have an active hardlink.

Without a pre-scan to identify hardlink targets and relink them after copies, all this would result in is duplication.
You'd then have to run another de-duplicator, which doesn't have knowledge of the balancing, and could arbitrarily undo
the balancing work.

* doc: .rebalance suffix -> .balance suffix

* doc: add --skip-hardlinks parameter to README

* test: add unit tests for --skip-hardlink
 2023-06-16 16:36:43 +02:00
dependabot[bot]
3a96510387 Bump docker/build-push-action from 3 to 4 (#18) 
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 3 to 4.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](https://github.com/docker/build-push-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-02-12 01:00:27 +01:00
Markus Ressel
5a4a8b0e34 Create dependabot.yml 2023-02-12 00:50:41 +01:00
Markus Ressel
9faf865850 Fix help string 2023-01-27 00:44:42 +01:00
Markus Ressel
c6790e7e94 Create stale.yml 2022-11-15 23:50:35 +01:00
Markus Ressel
19caa50d47 Update README.md 2022-11-15 23:37:01 +01:00
Markus Ressel
4fe2c08b24 added docker-latest github action 2022-11-15 23:35:09 +01:00
Markus Ressel
548c488782 added info about Dockerfile to README 2022-11-15 23:29:34 +01:00
Markus Ressel
bebe233371 added Dockerfile 2022-11-15 23:27:21 +01:00