RH rpms, and installing using hardlinks vs symlinks

classic Classic list List threaded Threaded
31 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: RH rpms, and installing using hardlinks vs symlinks

Samuel Sieb
On 03/09/2018 03:06 PM, Todd Zullinger wrote:
> Samuel Sieb wrote:
> In the git package, there aren't symlinks.  Within /usr/bin,
> the git binaries which are identical are hardlinked to each
> other.  And separately, within /usr/libexec/git-core, the
> git binaries which are identical are hardlinked to each
> other.

But the split between /usr/bin and /usr/libexec is a new thing in F28
according to your earlier email, right?
_______________________________________________
users mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: RH rpms, and installing using hardlinks vs symlinks

Todd Zullinger
Samuel Sieb wrote:
> On 03/09/2018 03:06 PM, Todd Zullinger wrote:
>> In the git package, there aren't symlinks.  Within /usr/bin,
>> the git binaries which are identical are hardlinked to each
>> other.  And separately, within /usr/libexec/git-core, the
>> git binaries which are identical are hardlinked to each
>> other.
>
> But the split between /usr/bin and /usr/libexec is a new thing in F28
> according to your earlier email, right?

Yeah.  In current releases the files are hardlinked across
/usr/bin and /usr/libexec.  I've never seen any bug reports
about that causing actual problems.  I made the change after
noticing the cross-directory hardlink warning from rpmlint.

It should really be an unnoticable change to nearly
everyone, apart from those rare people that want to mount
/usr/bin and /usr/libexec on different file systems

--
Todd
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Historian, n. A broad-gauge gossip.
    -- Ambrose Bierce, "The Devil's Dictionary"


_______________________________________________
users mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]

signature.asc (555 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: RH rpms, and installing using hardlinks vs symlinks

Gordon Messmer-2
In reply to this post by Stephen Morris
On Fri, Mar 9, 2018 at 2:57 PM, Stephen Morris <[hidden email]> wrote:
> No, what I was mentioning here is what I have read as standard linux
> functionality with copying, when a file is copied, and it doesn't matter
> where to, rather than create a 2nd copy of the file, the "copy" is created
> as a hard link to the original file, for storage efficiency, and then when
> one of the files is updated the hardlink is broken and both files become
> physical.


The only process I can think of that works even remotely like that is
rsnapshot (and similar backup systems).  In those setups, rsync will
create new files for the first backup.  On the second backup, it
creates hard links to all of the files in the set, first, and then
runs rsync to refresh the directory holding the newest backup.  When
rsync runs, it does not modify files in place, it creates a new file
and merges data from the local file and data from the remote file in
order to save bandwidth, according to its specialized algorithm.  When
it's done, it moves the new file into place, atomically updating the
path in the backup.

As you describe, rsnsapshot uses hard links to conserve disk space and
breaks links when it updates files.  My guess is that at some point in
the past, you read about rsnapshot or a similar backup system, and
later remembered their mode of operation as standard functionality.
_______________________________________________
users mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: RH rpms, and installing using hardlinks vs symlinks

Stephen Morris
On 11/3/18 4:35 am, Gordon Messmer wrote:

> On Fri, Mar 9, 2018 at 2:57 PM, Stephen Morris <[hidden email]> wrote:
>> No, what I was mentioning here is what I have read as standard linux
>> functionality with copying, when a file is copied, and it doesn't matter
>> where to, rather than create a 2nd copy of the file, the "copy" is created
>> as a hard link to the original file, for storage efficiency, and then when
>> one of the files is updated the hardlink is broken and both files become
>> physical.
>
> The only process I can think of that works even remotely like that is
> rsnapshot (and similar backup systems).  In those setups, rsync will
> create new files for the first backup.  On the second backup, it
> creates hard links to all of the files in the set, first, and then
> runs rsync to refresh the directory holding the newest backup.  When
> rsync runs, it does not modify files in place, it creates a new file
> and merges data from the local file and data from the remote file in
> order to save bandwidth, according to its specialized algorithm.  When
> it's done, it moves the new file into place, atomically updating the
> path in the backup.
>
> As you describe, rsnsapshot uses hard links to conserve disk space and
> breaks links when it updates files.  My guess is that at some point in
> the past, you read about rsnapshot or a similar backup system, and
> later remembered their mode of operation as standard functionality.

I can't find the documentation any more, but I have found documentation
on how to use copy the create the target as a hard link or as a soft
link. It is possible I have incorrectly remembered what I had read, or
it is possible over time that the standard copy functionality has
changed and now you have to explicitly specify that you want that
functionality.


regards,

Steve


> _______________________________________________
> users mailing list -- [hidden email]
> To unsubscribe send an email to [hidden email]
_______________________________________________
users mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: RH rpms, and installing using hardlinks vs symlinks

Ed Greshko
On 03/11/18 09:24, Stephen Morris wrote:
> I can't find the documentation any more, but I have found documentation on how to
> use copy the create the target as a hard link or as a soft link. It is possible I
> have incorrectly remembered what I had read, or it is possible over time that the
> standard copy functionality has changed and now you have to explicitly specify that
> you want that functionality.


From "man cp"

       -l, --link
              hard link files instead of copying

       -s, --symbolic-link
              make symbolic links instead of copying


[egreshko@meimei tmp]$ ll -i source
11574505 -rw-rw-r--. 1 egreshko egreshko 14 Mar 11 10:06 source

[egreshko@meimei tmp]$ cp -s source source-s

[egreshko@meimei tmp]$ ll -i source*
11574505 -rw-rw-r--. 1 egreshko egreshko 14 Mar 11 10:06 source
11573763 lrwxrwxrwx. 1 egreshko egreshko  6 Mar 11 10:07 source-s -> source

[egreshko@meimei tmp]$ cp -l source source-h

[egreshko@meimei tmp]$ ll -i source*
11574505 -rw-rw-r--. 2 egreshko egreshko 14 Mar 11 10:06 source
11574505 -rw-rw-r--. 2 egreshko egreshko 14 Mar 11 10:06 source-h
11573763 lrwxrwxrwx. 1 egreshko egreshko  6 Mar 11 10:07 source-s -> source

So, this is probably what you're recalling


--
I believe all research assistants should be paid for their time.


_______________________________________________
users mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]

signature.asc (235 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: RH rpms, and installing using hardlinks vs symlinks

Stephen Morris
On 11/3/18 1:09 pm, Ed Greshko wrote:

> On 03/11/18 09:24, Stephen Morris wrote:
>> I can't find the documentation any more, but I have found documentation on how to
>> use copy the create the target as a hard link or as a soft link. It is possible I
>> have incorrectly remembered what I had read, or it is possible over time that the
>> standard copy functionality has changed and now you have to explicitly specify that
>> you want that functionality.
>
>  From "man cp"
>
>         -l, --link
>                hard link files instead of copying
>
>         -s, --symbolic-link
>                make symbolic links instead of copying
>
>
> [egreshko@meimei tmp]$ ll -i source
> 11574505 -rw-rw-r--. 1 egreshko egreshko 14 Mar 11 10:06 source
>
> [egreshko@meimei tmp]$ cp -s source source-s
>
> [egreshko@meimei tmp]$ ll -i source*
> 11574505 -rw-rw-r--. 1 egreshko egreshko 14 Mar 11 10:06 source
> 11573763 lrwxrwxrwx. 1 egreshko egreshko  6 Mar 11 10:07 source-s -> source
>
> [egreshko@meimei tmp]$ cp -l source source-h
>
> [egreshko@meimei tmp]$ ll -i source*
> 11574505 -rw-rw-r--. 2 egreshko egreshko 14 Mar 11 10:06 source
> 11574505 -rw-rw-r--. 2 egreshko egreshko 14 Mar 11 10:06 source-h
> 11573763 lrwxrwxrwx. 1 egreshko egreshko  6 Mar 11 10:07 source-s -> source
>
> So, this is probably what you're recalling

It could be Ed, I remember the documentation differently, so I could be
not remembering it correctly, so I'll keep quiet now.


regards,

Steve


>
>
>
> _______________________________________________
> users mailing list -- [hidden email]
> To unsubscribe send an email to [hidden email]
_______________________________________________
users mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: RH rpms, and installing using hardlinks vs symlinks

Patrick O'Callaghan
On Sun, 2018-03-11 at 19:59 +1100, Stephen Morris wrote:

> On 11/3/18 1:09 pm, Ed Greshko wrote:
> > On 03/11/18 09:24, Stephen Morris wrote:
> > > I can't find the documentation any more, but I have found documentation on how to
> > > use copy the create the target as a hard link or as a soft link. It is possible I
> > > have incorrectly remembered what I had read, or it is possible over time that the
> > > standard copy functionality has changed and now you have to explicitly specify that
> > > you want that functionality.
> >
> >  From "man cp"
> >
> >         -l, --link
> >                hard link files instead of copying
> >
> >         -s, --symbolic-link
> >                make symbolic links instead of copying
> >
> >
> > [egreshko@meimei tmp]$ ll -i source
> > 11574505 -rw-rw-r--. 1 egreshko egreshko 14 Mar 11 10:06 source
> >
> > [egreshko@meimei tmp]$ cp -s source source-s
> >
> > [egreshko@meimei tmp]$ ll -i source*
> > 11574505 -rw-rw-r--. 1 egreshko egreshko 14 Mar 11 10:06 source
> > 11573763 lrwxrwxrwx. 1 egreshko egreshko  6 Mar 11 10:07 source-s -> source
> >
> > [egreshko@meimei tmp]$ cp -l source source-h
> >
> > [egreshko@meimei tmp]$ ll -i source*
> > 11574505 -rw-rw-r--. 2 egreshko egreshko 14 Mar 11 10:06 source
> > 11574505 -rw-rw-r--. 2 egreshko egreshko 14 Mar 11 10:06 source-h
> > 11573763 lrwxrwxrwx. 1 egreshko egreshko  6 Mar 11 10:07 source-s -> source
> >
> > So, this is probably what you're recalling
>
> It could be Ed, I remember the documentation differently, so I could be
> not remembering it correctly, so I'll keep quiet now.

The original version of 'cp' (by which I mean the version going back to
the early days of Unix) didn't have any options. It simply copied the
file contents to a different file. There was no question of using
links, hard or soft (in fact symbolic links hadn't been invented).

poc
_______________________________________________
users mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: RH rpms, and installing using hardlinks vs symlinks

Gordon Messmer-2
In reply to this post by Stephen Morris
On 03/10/2018 05:24 PM, Stephen Morris wrote:
> I can't find the documentation any more, but I have found
> documentation on how to use copy the create the target as a hard link
> or as a soft link. It is possible I have incorrectly remembered what I
> had read, or it is possible over time that the standard copy
> functionality has changed and now you have to explicitly specify that
> you want that functionality.


I'd forgotten the other system that works as you described: reflink.

"cp" supports a --reflink option, and on a few filesystems that
implement it (btrfs, OCFS2, maybe xfs and zfs, I'm unclear on their
status), you can create a CoW copy of a file.

Again, not standard behavior, not widely implemented, and relatively
new, but this might be what you were thinking of.
_______________________________________________
users mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: RH rpms, and installing using hardlinks vs symlinks

Robert P. J. Day
On Sun, 11 Mar 2018, Gordon Messmer wrote:

> On 03/10/2018 05:24 PM, Stephen Morris wrote:
> > I can't find the documentation any more, but I have found
> > documentation on how to use copy the create the target as a hard
> > link or as a soft link. It is possible I have incorrectly
> > remembered what I had read, or it is possible over time that the
> > standard copy functionality has changed and now you have to
> > explicitly specify that you want that functionality.
>
> I'd forgotten the other system that works as you described: reflink.
>
> "cp" supports a --reflink option, and on a few filesystems that
> implement it (btrfs, OCFS2, maybe xfs and zfs, I'm unclear on their
> status), you can create a CoW copy of a file.
>
> Again, not standard behavior, not widely implemented, and relatively
> new, but this might be what you were thinking of.

  ok, that's cool, i had no idea that even existed.

rday
_______________________________________________
users mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: RH rpms, and installing using hardlinks vs symlinks

George White
In reply to this post by Ed Greshko
On 6 March 2018 at 10:45, Ed Greshko <[hidden email]> wrote:
On 03/06/18 22:30, Robert P. J. Day wrote:
>   so there's probably more admittedly trivial questions coming, but
> only because i'm being forced to look more closely at things than i
> have in a long time.


OK.  Then I feel compelled to ask the following in the spirit of full-disclosure.

In answering these queries are we contributing information to your knowledge base
that will be incorporated into a course for which you'll be compensated?

Actually, the information could be incorporated into courses by people lurking on the
list or maybe finding the thread using google.  Good questions are in themselves 
useful contributions, so are welcome unless they appear to come from a homework 
assignment.
 
If simple questions can be answered with a simple google query then why are there so
many of them?

Maybe because people have learned that answers to google queries are often 
incorrect.   If there is disagreement over the answer, forums let you see the reasoning
behind different proposed answers.  Some of the discussions in this forum have 
been educational.   

In the interest of a full discussion, there is: 

--
George N. White III


_______________________________________________
users mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: RH rpms, and installing using hardlinks vs symlinks

Ahmad Samir
In reply to this post by Robert P. J. Day
On 7 March 2018 at 12:34, Robert P. J. Day <[hidden email]> wrote:

> On Wed, 7 Mar 2018, Ahmad Samir wrote:
>
>> On 6 March 2018 at 14:34, Robert P. J. Day <[hidden email]> wrote:
>> >
>> >   i'm curious about RH packaging policy that dictates that some
>> > command variants are packaged for fedora to install with symlinks
>> > and others with hardlinks.
>> >
>> >   trivial example in /usr/bin on my fedora 27 system:
>> >
>> >   -rwsr-xr-x.   1 root root        52984 Aug  2  2017 at
>> >   lrwxrwxrwx.   1 root root            2 Aug  2  2017 atq -> at
>> >   lrwxrwxrwx.   1 root root            2 Aug  2  2017 atrm -> at
>> >
>> > so even though all of those "commands" are in the very same directory,
>> > atq and atrm are supported via symlinks, not hardlinks.
>>
>> I am not an expert, some commands act differently when called
>> differently; so executing /bin/atq would make the 'at' binary behave
>> differently than when it's executed as plain 'at'.
>>
>> The same logic applies to bash, /bin/sh is a symlink to /bin/bash,
>> but when bash is invoked as sh it acts differently than when invoked
>> as 'bash'. Have a look at the bash manual page for more info.
>
>   yes, i'm aware of this, and i'm pretty sure whether a hardlink or
> symlink is used would make no difference.
>
> rday

You are right, I missed the bit about hardlinks. Sorry for the noise.

--
Ahmad Samir
_______________________________________________
users mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
12