Friday 12 October 2018

linux - rsync hard linking bug when using -R?

Following on File copy tools that will preserve hard links at destination, I found the problem is not at copying hard linked files to another volume part, but the source hard linked files themselves.


Took me quiet a wild-chase to pin-point where the problem is. Let me show the normal case first:


Here it is:


/tmp$ ls -l hl-demo
total 0
-rw-rw---- 2 tong tong 0 2019-08-03 22:32 abc
-rw-rw---- 2 tong tong 0 2019-08-03 22:32 abc-hl
-rw-rw---- 1 tong tong 0 2019-08-03 22:32 cde
-rw-rw---- 2 tong tong 0 2019-08-03 22:32 def
-rw-rw---- 2 tong tong 0 2019-08-03 22:32 def-hl

rsync -a --link-dest=/tmp/hl-demo /tmp/hl-demo/ hl-dup/

$ ls -l hl-demo
total 0
-rw-rw---- 4 tong tong 0 2019-08-03 22:32 abc
-rw-rw---- 4 tong tong 0 2019-08-03 22:32 abc-hl
-rw-rw---- 2 tong tong 0 2019-08-03 22:32 cde
-rw-rw---- 4 tong tong 0 2019-08-03 22:32 def
-rw-rw---- 4 tong tong 0 2019-08-03 22:32 def-hl

$ ls -l hl-dup/
total 0
-rw-rw---- 4 tong tong 0 2019-08-03 22:32 abc
-rw-rw---- 4 tong tong 0 2019-08-03 22:32 abc-hl
-rw-rw---- 2 tong tong 0 2019-08-03 22:32 cde
-rw-rw---- 4 tong tong 0 2019-08-03 22:32 def
-rw-rw---- 4 tong tong 0 2019-08-03 22:32 def-hl

$ stat hl-demo/abc hl-dup/abc-hl
File: hl-demo/abc
Size: 0 Blocks: 0 IO Block: 4096 regular empty file
Device: 81dh/2077d Inode: 1223453 Links: 4
Access: (0660/-rw-rw----) Uid: ( 9999/ tong) Gid: ( 1001/ tong)
Access: 2019-08-03 22:32:27.000000000 -0400
Modify: 2019-08-03 22:32:27.000000000 -0400
Change: 2019-08-03 22:37:56.000000000 -0400
Birth: -
File: hl-dup/abc-hl
Size: 0 Blocks: 0 IO Block: 4096 regular empty file
Device: 81dh/2077d Inode: 1223453 Links: 4
Access: (0660/-rw-rw----) Uid: ( 9999/ tong) Gid: ( 1001/ tong)
Access: 2019-08-03 22:32:27.000000000 -0400
Modify: 2019-08-03 22:32:27.000000000 -0400
Change: 2019-08-03 22:37:56.000000000 -0400
Birth: -

Everything is normal, a second folder hl-dup is created, hard linking all files from the first, hl-demo. All good.


Now, let's redo the above rsync command with an extra -R, with everything else the same, after rm -rf hl-dup of course. The result should be the same -- second folder hard linking all files to the first, right? Let's see:


rm -rf hl-dup

$ ls -l hl-demo
total 0
-rw-rw---- 2 tong tong 0 2019-08-03 22:32 abc
-rw-rw---- 2 tong tong 0 2019-08-03 22:32 abc-hl
-rw-rw---- 1 tong tong 0 2019-08-03 22:32 cde
-rw-rw---- 2 tong tong 0 2019-08-03 22:32 def
-rw-rw---- 2 tong tong 0 2019-08-03 22:32 def-hl

rsync -a --link-dest=/tmp/hl-demo /tmp/hl-demo/ hl-dup/ -R

$ ls -l hl-demo
total 0
-rw-rw---- 2 tong tong 0 2019-08-03 22:32 abc
-rw-rw---- 2 tong tong 0 2019-08-03 22:32 abc-hl
-rw-rw---- 1 tong tong 0 2019-08-03 22:32 cde
-rw-rw---- 2 tong tong 0 2019-08-03 22:32 def
-rw-rw---- 2 tong tong 0 2019-08-03 22:32 def-hl

$ ls -l hl-dup/tmp/hl-demo/
total 0
-rw-rw---- 1 tong tong 0 2019-08-03 22:32 abc
-rw-rw---- 1 tong tong 0 2019-08-03 22:32 abc-hl
-rw-rw---- 1 tong tong 0 2019-08-03 22:32 cde
-rw-rw---- 1 tong tong 0 2019-08-03 22:32 def
-rw-rw---- 1 tong tong 0 2019-08-03 22:32 def-hl

$ stat hl-demo/abc hl-dup/tmp/hl-demo/abc-hl
File: hl-demo/abc
Size: 0 Blocks: 0 IO Block: 4096 regular empty file
Device: 81dh/2077d Inode: 1223453 Links: 2
Access: (0660/-rw-rw----) Uid: ( 9999/ tong) Gid: ( 1001/ tong)
Access: 2019-08-03 22:32:27.000000000 -0400
Modify: 2019-08-03 22:32:27.000000000 -0400
Change: 2019-08-03 22:41:55.000000000 -0400
Birth: -
File: hl-dup/tmp/hl-demo/abc-hl
Size: 0 Blocks: 0 IO Block: 4096 regular empty file
Device: 81dh/2077d Inode: 1223498 Links: 1
Access: (0660/-rw-rw----) Uid: ( 9999/ tong) Gid: ( 1001/ tong)
Access: 2019-08-03 22:43:34.000000000 -0400
Modify: 2019-08-03 22:32:27.000000000 -0400
Change: 2019-08-03 22:43:34.000000000 -0400
Birth: -

I.e., redoing the first rsync command with an extra -R will change everything -- second folder no-longer hard linking all files to the first any more.


My original setup was done using a mixed of using -R and not using -R, so I didn't noticed that hard linking are gone when using -R.



  • Can someone confirm please?

  • How to create hard linkings in the second case, when -R is necessary?


PS. setup:


touch abc def cde
ln abc abc-hl
ln def def-hl

No comments:

Post a Comment

Where does Skype save my contact's avatars in Linux?

I'm using Skype on Linux. Where can I find images cached by skype of my contact's avatars? Answer I wanted to get those Skype avat...