du Weirdness - how is this possible

classic Classic list List threaded Threaded
40 messages Options
12
Reply | Threaded
Open this post in threaded view
|

du Weirdness - how is this possible

Philip Rhoades
People,

I started deleting GBs of stuff from:

/dev/sdb1 /backup

but df did not reduce from 95% so I looked more closely and found this
weirdness:

# du -s -BG 20180216
43G     20180216

# du -s -BG 20180216/*
1G      20180216/naf_dirs
43G     20180216/phil
1G      20180216/root

# du -s -BG 20180216/phil/*
1G      20180216/phil/0
1G      20180216/phil/0_finance
1G      20180216/phil/0_naf
1G      20180216/phil/Maildir
1G      20180216/phil/txts
1G      20180216/phil/vimwiki

Where has ~37GB disappeared to?  There are no files held open and I have
successfully umounted and re-mounted the partition - what is going on?

Thanks,

Phil.
--
Philip Rhoades

PO Box 896
Cowra  NSW  2794
Australia
E-mail:  [hidden email]
_______________________________________________
users mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: du Weirdness - how is this possible

jd1008
How about checking the permissions of all files/dirs in phil?

Run
ls -lR ....phil

and report results.

On 03/11/2018 12:48 PM, Philip Rhoades wrote:

> People,
>
> I started deleting GBs of stuff from:
>
> /dev/sdb1 /backup
>
> but df did not reduce from 95% so I looked more closely and found this
> weirdness:
>
> # du -s -BG 20180216
> 43G     20180216
>
> # du -s -BG 20180216/*
> 1G      20180216/naf_dirs
> 43G     20180216/phil
> 1G      20180216/root
>
> # du -s -BG 20180216/phil/*
> 1G      20180216/phil/0
> 1G      20180216/phil/0_finance
> 1G      20180216/phil/0_naf
> 1G      20180216/phil/Maildir
> 1G      20180216/phil/txts
> 1G      20180216/phil/vimwiki
>
> Where has ~37GB disappeared to?  There are no files held open and I
> have successfully umounted and re-mounted the partition - what is
> going on?
>
> Thanks,
>
> Phil.
_______________________________________________
users mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: du Weirdness - how is this possible

Gordon Messmer-2
In reply to this post by Philip Rhoades
On 03/11/2018 11:48 AM, Philip Rhoades wrote:
> Where has ~37GB disappeared to?


"ls -la 20180216/phil"

Are there dot-files in there that aren't being matched by the shell
glob?  If so:

shopt -s dotglob
du -s -BG 20180216/phil/*
_______________________________________________
users mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: du Weirdness - how is this possible

Robert Nichols
In reply to this post by Philip Rhoades
On 03/11/2018 01:48 PM, Philip Rhoades wrote:

> People,
>
> I started deleting GBs of stuff from:
>
> /dev/sdb1 /backup
>
> but df did not reduce from 95% so I looked more closely and found this weirdness:
>
> # du -s -BG 20180216
> 43G     20180216
>
> # du -s -BG 20180216/*
> 1G      20180216/naf_dirs
> 43G     20180216/phil
> 1G      20180216/root
>
> # du -s -BG 20180216/phil/*
> 1G      20180216/phil/0
> 1G      20180216/phil/0_finance
> 1G      20180216/phil/0_naf
> 1G      20180216/phil/Maildir
> 1G      20180216/phil/txts
> 1G      20180216/phil/vimwiki
>
> Where has ~37GB disappeared to?  There are no files held open and I have successfully umounted and re-mounted the partition - what is going on?

That "du -s -BG 20180216/phil/*" is ignoring any "dot" files/directories under 20180216/phil .

It's better to leave out the "-s" option and use the "--max-depth" option to limit the depth of the display.
        du -BG --max-depth=1 20180216/phil

--
Bob Nichols     "NOSPAM" is really part of my email address.
                 Do NOT delete it.
_______________________________________________
users mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: du Weirdness - how is this possible

Philip Rhoades
JD, Gordon, Robert,


On 2018-03-12 06:13, Robert Nichols wrote:

> On 03/11/2018 01:48 PM, Philip Rhoades wrote:
>> People,
>>
>> I started deleting GBs of stuff from:
>>
>> /dev/sdb1 /backup
>>
>> but df did not reduce from 95% so I looked more closely and found this
>> weirdness:
>>
>> # du -s -BG 20180216
>> 43G     20180216
>>
>> # du -s -BG 20180216/*
>> 1G      20180216/naf_dirs
>> 43G     20180216/phil
>> 1G      20180216/root
>>
>> # du -s -BG 20180216/phil/*
>> 1G      20180216/phil/0
>> 1G      20180216/phil/0_finance
>> 1G      20180216/phil/0_naf
>> 1G      20180216/phil/Maildir
>> 1G      20180216/phil/txts
>> 1G      20180216/phil/vimwiki
>>
>> Where has ~37GB disappeared to?  There are no files held open and I
>> have successfully umounted and re-mounted the partition - what is
>> going on?
>
> That "du -s -BG 20180216/phil/*" is ignoring any "dot"
> files/directories under 20180216/phil .
>
> It's better to leave out the "-s" option and use the "--max-depth"
> option to limit the depth of the display.
> du -BG --max-depth=1 20180216/phil


Damn, I should have thought of that - normally, outside of .config,
there would not be much in my dot dirs - but I had been experimenting
with cryptocurrency nodes . .

Thanks for the useful tips!

Regards,

P.
--
Philip Rhoades

PO Box 896
Cowra  NSW  2794
Australia
E-mail:  [hidden email]
_______________________________________________
users mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: du Weirdness - how is this possible

Stephen Morris
On 12/3/18 10:48 am, Philip Rhoades wrote:

> JD, Gordon, Robert,
>
>
> On 2018-03-12 06:13, Robert Nichols wrote:
>> On 03/11/2018 01:48 PM, Philip Rhoades wrote:
>>> People,
>>>
>>> I started deleting GBs of stuff from:
>>>
>>> /dev/sdb1 /backup
>>>
>>> but df did not reduce from 95% so I looked more closely and found
>>> this weirdness:
>>>
>>> # du -s -BG 20180216
>>> 43G     20180216
>>>
>>> # du -s -BG 20180216/*
>>> 1G      20180216/naf_dirs
>>> 43G     20180216/phil
>>> 1G      20180216/root
>>>
>>> # du -s -BG 20180216/phil/*
>>> 1G      20180216/phil/0
>>> 1G      20180216/phil/0_finance
>>> 1G      20180216/phil/0_naf
>>> 1G      20180216/phil/Maildir
>>> 1G      20180216/phil/txts
>>> 1G      20180216/phil/vimwiki
>>>
>>> Where has ~37GB disappeared to?  There are no files held open and I
>>> have successfully umounted and re-mounted the partition - what is
>>> going on?
>>
>> That "du -s -BG 20180216/phil/*" is ignoring any "dot"
>> files/directories under 20180216/phil .
>>
>> It's better to leave out the "-s" option and use the "--max-depth"
>> option to limit the depth of the display.
>>     du -BG --max-depth=1 20180216/phil
>
>
> Damn, I should have thought of that - normally, outside of .config,
> there would not be much in my dot dirs - but I had been experimenting
> with cryptocurrency nodes . .
>
> Thanks for the useful tips!
>
> Regards,
>
> P.

Just one question on this, is the scaling that the -BG screwing around
with the results of du as shown below?

bash-4.4$ du -s -BG /home/steve
8G      /home/steve

bash-4.4$ du -s -BG /home/steve/*
1G      /home/steve/andrew
1G      /home/steve/config
1G      /home/steve/Desktop
1G      /home/steve/Documents
1G      /home/steve/Downloads
1G      /home/steve/gtk.css
1G      /home/steve/jxbrowser-browser.log
1G      /home/steve/jxbrowser-chromium.log
1G      /home/steve/jxbrowser-ipc.log
1G      /home/steve/jxbrowser-ipc.log.1
0G      /home/steve/jxbrowser-ipc.log.1.lck
0G      /home/steve/jxbrowser-ipc.log.lck
1G      /home/steve/l10n
1G      /home/steve/Music
1G      /home/steve/NetBeansProjects
1G      /home/steve/Pictures
1G      /home/steve/Public
1G      /home/steve/R
3G      /home/steve/rpmbuild
1G      /home/steve/Templates
1G      /home/steve/Videos
1G      /home/steve/workspace

du -s -BG /home/steve/workspace/*
1G      /home/steve/workspace/basics.zip_expanded
1G      /home/steve/workspace/jsf-blank.zip_expanded
1G      /home/steve/workspace/jsffacletstutorial
1G      /home/steve/workspace/libraries
1G      /home/steve/workspace/RemoteSystemsTempFiles
1G      /home/steve/workspace/Servers
1G      /home/steve/workspace/test-app
1G      /home/steve/workspace/whirlwind


regards,

Steve
_______________________________________________
users mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: du Weirdness - how is this possible

Robert Nichols
On 03/11/2018 07:19 PM, Stephen Morris wrote:

> On 12/3/18 10:48 am, Philip Rhoades wrote:
>> JD, Gordon, Robert,
>>
>>
>> On 2018-03-12 06:13, Robert Nichols wrote:
>>> On 03/11/2018 01:48 PM, Philip Rhoades wrote:
>>>> People,
>>>>
>>>> I started deleting GBs of stuff from:
>>>>
>>>> /dev/sdb1 /backup
>>>>
>>>> but df did not reduce from 95% so I looked more closely and found this weirdness:
>>>>
>>>> # du -s -BG 20180216
>>>> 43G     20180216
>>>>
>>>> # du -s -BG 20180216/*
>>>> 1G      20180216/naf_dirs
>>>> 43G     20180216/phil
>>>> 1G      20180216/root
>>>>
>>>> # du -s -BG 20180216/phil/*
>>>> 1G      20180216/phil/0
>>>> 1G      20180216/phil/0_finance
>>>> 1G      20180216/phil/0_naf
>>>> 1G      20180216/phil/Maildir
>>>> 1G      20180216/phil/txts
>>>> 1G      20180216/phil/vimwiki
>>>>
>>>> Where has ~37GB disappeared to?  There are no files held open and I have successfully umounted and re-mounted the partition - what is going on?
>>>
>>> That "du -s -BG 20180216/phil/*" is ignoring any "dot"
>>> files/directories under 20180216/phil .
>>>
>>> It's better to leave out the "-s" option and use the "--max-depth"
>>> option to limit the depth of the display.
>>>     du -BG --max-depth=1 20180216/phil
>>
>>
>> Damn, I should have thought of that - normally, outside of .config, there would not be much in my dot dirs - but I had been experimenting with cryptocurrency nodes . .
>>
>> Thanks for the useful tips!
>>
>> Regards,
>>
>> P.
>
> Just one question on this, is the scaling that the -BG screwing around with the results of du as shown below?
>
> bash-4.4$ du -s -BG /home/steve
> 8G      /home/steve
>
> bash-4.4$ du -s -BG /home/steve/*
> 1G      /home/steve/andrew
> 1G      /home/steve/config
> 1G      /home/steve/Desktop
> 1G      /home/steve/Documents
> 1G      /home/steve/Downloads
> 1G      /home/steve/gtk.css
> 1G      /home/steve/jxbrowser-browser.log
> 1G      /home/steve/jxbrowser-chromium.log
> 1G      /home/steve/jxbrowser-ipc.log
> 1G      /home/steve/jxbrowser-ipc.log.1
> 0G      /home/steve/jxbrowser-ipc.log.1.lck
> 0G      /home/steve/jxbrowser-ipc.log.lck
> 1G      /home/steve/l10n
> 1G      /home/steve/Music
> 1G      /home/steve/NetBeansProjects
> 1G      /home/steve/Pictures
> 1G      /home/steve/Public
> 1G      /home/steve/R
> 3G      /home/steve/rpmbuild
> 1G      /home/steve/Templates
> 1G      /home/steve/Videos
> 1G      /home/steve/workspace
>
> du -s -BG /home/steve/workspace/*
> 1G      /home/steve/workspace/basics.zip_expanded
> 1G      /home/steve/workspace/jsf-blank.zip_expanded
> 1G      /home/steve/workspace/jsffacletstutorial
> 1G      /home/steve/workspace/libraries
> 1G      /home/steve/workspace/RemoteSystemsTempFiles
> 1G      /home/steve/workspace/Servers
> 1G      /home/steve/workspace/test-app
> 1G      /home/steve/workspace/whirlwind

Of course it is. Any size greater than 0 and not exceeding 1GB will show as 1G. It's answering the question, "How many blocks 1GB in size would it take to hold this?" Anything of non-zero size will take at least 1 such block.

So, your "workspace" directory contains 8 things, none of which is larger than 1G, and the total space for all of them is also less than 1G.

--
Bob Nichols     "NOSPAM" is really part of my email address.
                 Do NOT delete it.
_______________________________________________
users mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: du Weirdness - how is this possible

Stephen Morris
On 12/3/18 11:29 am, Robert Nichols wrote:

> On 03/11/2018 07:19 PM, Stephen Morris wrote:
>> On 12/3/18 10:48 am, Philip Rhoades wrote:
>>> JD, Gordon, Robert,
>>>
>>>
>>> On 2018-03-12 06:13, Robert Nichols wrote:
>>>> On 03/11/2018 01:48 PM, Philip Rhoades wrote:
>>>>> People,
>>>>>
>>>>> I started deleting GBs of stuff from:
>>>>>
>>>>> /dev/sdb1 /backup
>>>>>
>>>>> but df did not reduce from 95% so I looked more closely and found
>>>>> this weirdness:
>>>>>
>>>>> # du -s -BG 20180216
>>>>> 43G     20180216
>>>>>
>>>>> # du -s -BG 20180216/*
>>>>> 1G      20180216/naf_dirs
>>>>> 43G     20180216/phil
>>>>> 1G      20180216/root
>>>>>
>>>>> # du -s -BG 20180216/phil/*
>>>>> 1G      20180216/phil/0
>>>>> 1G      20180216/phil/0_finance
>>>>> 1G      20180216/phil/0_naf
>>>>> 1G      20180216/phil/Maildir
>>>>> 1G      20180216/phil/txts
>>>>> 1G      20180216/phil/vimwiki
>>>>>
>>>>> Where has ~37GB disappeared to?  There are no files held open and
>>>>> I have successfully umounted and re-mounted the partition - what
>>>>> is going on?
>>>>
>>>> That "du -s -BG 20180216/phil/*" is ignoring any "dot"
>>>> files/directories under 20180216/phil .
>>>>
>>>> It's better to leave out the "-s" option and use the "--max-depth"
>>>> option to limit the depth of the display.
>>>>     du -BG --max-depth=1 20180216/phil
>>>
>>>
>>> Damn, I should have thought of that - normally, outside of .config,
>>> there would not be much in my dot dirs - but I had been
>>> experimenting with cryptocurrency nodes . .
>>>
>>> Thanks for the useful tips!
>>>
>>> Regards,
>>>
>>> P.
>>
>> Just one question on this, is the scaling that the -BG screwing
>> around with the results of du as shown below?
>>
>> bash-4.4$ du -s -BG /home/steve
>> 8G      /home/steve
>>
>> bash-4.4$ du -s -BG /home/steve/*
>> 1G      /home/steve/andrew
>> 1G      /home/steve/config
>> 1G      /home/steve/Desktop
>> 1G      /home/steve/Documents
>> 1G      /home/steve/Downloads
>> 1G      /home/steve/gtk.css
>> 1G      /home/steve/jxbrowser-browser.log
>> 1G      /home/steve/jxbrowser-chromium.log
>> 1G      /home/steve/jxbrowser-ipc.log
>> 1G      /home/steve/jxbrowser-ipc.log.1
>> 0G      /home/steve/jxbrowser-ipc.log.1.lck
>> 0G      /home/steve/jxbrowser-ipc.log.lck
>> 1G      /home/steve/l10n
>> 1G      /home/steve/Music
>> 1G      /home/steve/NetBeansProjects
>> 1G      /home/steve/Pictures
>> 1G      /home/steve/Public
>> 1G      /home/steve/R
>> 3G      /home/steve/rpmbuild
>> 1G      /home/steve/Templates
>> 1G      /home/steve/Videos
>> 1G      /home/steve/workspace
>>
>> du -s -BG /home/steve/workspace/*
>> 1G      /home/steve/workspace/basics.zip_expanded
>> 1G      /home/steve/workspace/jsf-blank.zip_expanded
>> 1G      /home/steve/workspace/jsffacletstutorial
>> 1G      /home/steve/workspace/libraries
>> 1G      /home/steve/workspace/RemoteSystemsTempFiles
>> 1G      /home/steve/workspace/Servers
>> 1G      /home/steve/workspace/test-app
>> 1G      /home/steve/workspace/whirlwind
>
> Of course it is. Any size greater than 0 and not exceeding 1GB will
> show as 1G. It's answering the question, "How many blocks 1GB in size
> would it take to hold this?" Anything of non-zero size will take at
> least 1 such block.
>
> So, your "workspace" directory contains 8 things, none of which is
> larger than 1G, and the total space for all of them is also less than 1G.
>
Thanks Robert, so basically what you are saying is that if you use that
parameter, the output is rounded up to the nearest integer
representation (in this case Gig) rather than displaying it as a
fraction? For example, for a file that is 600 MB in size I would have
expected the command to display it as 0.6G rather than 1G.


regards,

Steve

_______________________________________________
users mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: du Weirdness - how is this possible

Ed Greshko-2
On 03/12/18 09:41, Stephen Morris wrote:
> Thanks Robert, so basically what you are saying is that if you use that parameter,
> the output is rounded up to the nearest integer representation (in this case Gig)
> rather than displaying it as a fraction? For example, for a file that is 600 MB in
> size I would have expected the command to display it as 0.6G rather than 1G.


From the man page....

       -B, --block-size=SIZE
              scale sizes by SIZE before printing them; e.g., '-BM' prints sizes
              in units of 1,048,576 bytes; see SIZE format below

and

       The  SIZE  argument  is  an  integer  and  optional unit (example: 10K is
       10*1024).  Units are K,M,G,T,P,E,Z,Y (powers of 1024) or KB,MB,...  (pow‐
       ers of 1000)

Key word in the above definition of "SIZE" is "integer".  And, as you've noted, 0.6
isn't an integer.

I just happen to have a computer in front of me.  So, I can create a few files and
try things.

[egreshko@meimei misty]$ ll test-dir/
total 1110004
-rw-rw-r--. 1 egreshko egreshko 1024000000 Mar 12 10:21 test
-rw-rw-r--. 1 egreshko egreshko  102400000 Mar 12 10:27 test1
-rw-rw-r--. 1 egreshko egreshko   10240000 Mar 12 10:29 test2

[egreshko@meimei misty]$ ll -h test-dir/
total 1.1G
-rw-rw-r--. 1 egreshko egreshko 977M Mar 12 10:21 test
-rw-rw-r--. 1 egreshko egreshko  98M Mar 12 10:27 test1
-rw-rw-r--. 1 egreshko egreshko 9.8M Mar 12 10:29 test2

[egreshko@meimei misty]$ ll --si test-dir/
total 1.2G
-rw-rw-r--. 1 egreshko egreshko 1.1G Mar 12 10:21 test
-rw-rw-r--. 1 egreshko egreshko 103M Mar 12 10:27 test1
-rw-rw-r--. 1 egreshko egreshko  11M Mar 12 10:29 test2

[egreshko@meimei misty]$ du -s -BG test-dir/*
1G      test-dir/test
1G      test-dir/test1
1G      test-dir/test2

[egreshko@meimei misty]$ du -s -BG test-dir
2G      test-dir

[egreshko@meimei misty]$ du -s -BM test-dir/*
977M    test-dir/test
98M     test-dir/test1
10M     test-dir/test2

[egreshko@meimei misty]$ du -s -BM test-dir
1084M   test-dir


And, since there is a difference, the man page for "ls" should be consulted to find....

       -h, --human-readable
              with -l and/or -s, print human readable sizes (e.g., 1K 234M 2G)

       --si   likewise, but use powers of 1000 not 1024


--
Conjecture is just a conclusion based on incomplete information. It isn't a fact.

_______________________________________________
users mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: du Weirdness - how is this possible

Stephen Morris
On 12/3/18 1:41 pm, Ed Greshko wrote:

> On 03/12/18 09:41, Stephen Morris wrote:
>> Thanks Robert, so basically what you are saying is that if you use that parameter,
>> the output is rounded up to the nearest integer representation (in this case Gig)
>> rather than displaying it as a fraction? For example, for a file that is 600 MB in
>> size I would have expected the command to display it as 0.6G rather than 1G.
>
>  From the man page....
>
>         -B, --block-size=SIZE
>                scale sizes by SIZE before printing them; e.g., '-BM' prints sizes
>                in units of 1,048,576 bytes; see SIZE format below
>
> and
>
>         The  SIZE  argument  is  an  integer  and  optional unit (example: 10K is
>         10*1024).  Units are K,M,G,T,P,E,Z,Y (powers of 1024) or KB,MB,...  (pow‐
>         ers of 1000)
>
> Key word in the above definition of "SIZE" is "integer".  And, as you've noted, 0.6
> isn't an integer.
>
> I just happen to have a computer in front of me.  So, I can create a few files and
> try things.
>
> [egreshko@meimei misty]$ ll test-dir/
> total 1110004
> -rw-rw-r--. 1 egreshko egreshko 1024000000 Mar 12 10:21 test
> -rw-rw-r--. 1 egreshko egreshko  102400000 Mar 12 10:27 test1
> -rw-rw-r--. 1 egreshko egreshko   10240000 Mar 12 10:29 test2
>
> [egreshko@meimei misty]$ ll -h test-dir/
> total 1.1G
> -rw-rw-r--. 1 egreshko egreshko 977M Mar 12 10:21 test
> -rw-rw-r--. 1 egreshko egreshko  98M Mar 12 10:27 test1
> -rw-rw-r--. 1 egreshko egreshko 9.8M Mar 12 10:29 test2
>
> [egreshko@meimei misty]$ ll --si test-dir/
> total 1.2G
> -rw-rw-r--. 1 egreshko egreshko 1.1G Mar 12 10:21 test
> -rw-rw-r--. 1 egreshko egreshko 103M Mar 12 10:27 test1
> -rw-rw-r--. 1 egreshko egreshko  11M Mar 12 10:29 test2
>
> [egreshko@meimei misty]$ du -s -BG test-dir/*
> 1G      test-dir/test
> 1G      test-dir/test1
> 1G      test-dir/test2
>
> [egreshko@meimei misty]$ du -s -BG test-dir
> 2G      test-dir
>
> [egreshko@meimei misty]$ du -s -BM test-dir/*
> 977M    test-dir/test
> 98M     test-dir/test1
> 10M     test-dir/test2
>
> [egreshko@meimei misty]$ du -s -BM test-dir
> 1084M   test-dir
>
>
> And, since there is a difference, the man page for "ls" should be consulted to find....
>
>         -h, --human-readable
>                with -l and/or -s, print human readable sizes (e.g., 1K 234M 2G)
>
>         --si   likewise, but use powers of 1000 not 1024

Thanks Ed, this functionality seems to be counter intuitive to me. Given
that du shows space used in sub-directories, it seems to me the only
useful output is du without any parameters.


regards,

Steve


>
>
> _______________________________________________
> users mailing list -- [hidden email]
> To unsubscribe send an email to [hidden email]
_______________________________________________
users mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: du Weirdness - how is this possible

Patrick O'Callaghan
On Mon, 2018-03-12 at 21:41 +1100, Stephen Morris wrote:
> Thanks Ed, this functionality seems to be counter intuitive to me. Given
> that du shows space used in sub-directories, it seems to me the only
> useful output is du without any parameters.

'du' with no parameters recursively lists all the subdirectories and
their sizes, along with the grand total. When applied to my home
directory, I get over 30,000 lines of output. That's almost never what
I want. My usual call is 'du -hs'.

poc
_______________________________________________
users mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: du Weirdness - how is this possible

Stephen Morris
On 12/3/18 10:11 pm, Patrick O'Callaghan wrote:

> On Mon, 2018-03-12 at 21:41 +1100, Stephen Morris wrote:
>> Thanks Ed, this functionality seems to be counter intuitive to me. Given
>> that du shows space used in sub-directories, it seems to me the only
>> useful output is du without any parameters.
> 'du' with no parameters recursively lists all the subdirectories and
> their sizes, along with the grand total. When applied to my home
> directory, I get over 30,000 lines of output. That's almost never what
> I want. My usual call is 'du -hs'.
>
> poc

Thanks Patrick, taking this a step further, it seems to me that the only
parameter for du that, to me, provides the correct file size is -b as
shown below. I am listing my Desktop directory via ll, du -hs and du
-bhs. Just further to this is it a bug with du that the -a parameter
which is supposed to list all files not just directories, does not list
files prefixed with a '.'?:


bash-4.4$ ll /home/steve/Desktop/*
-rwxr-xr-x. 1 steve steve  466 Oct  3  2015 '/home/steve/Desktop/Acrobat
Reader.desktop'
-rwxrwxrwx. 1 steve steve  233 May  5  2014
/home/steve/Desktop/autoplus.desktop
-rwxr-xr-x. 1 steve steve  310 Dec 26  2016
/home/steve/Desktop/chrome-eppojlglocelodeimnohnlnionkobfln-Default.desktop
-rwxr-xr-x. 1 steve steve  374 Jan 11 07:54
/home/steve/Desktop/Eclipse.desktop
-rwxr-xr-x. 1 steve steve  417 Jan 16  2017
/home/steve/Desktop/Firefox.desktop
-rwxr-xr-x. 1 steve steve  358 Sep  8  2013 '/home/steve/Desktop/Google
Chrome.desktop'
-rwxrw-r--. 1 steve steve  412 Nov 30  2013
/home/steve/Desktop/jws_app_shortcut_1385783507561.desktop
-rwxrwxr-x. 1 steve steve  247 Jan 19 07:47
/home/steve/Desktop/netbeans-8.2.desktop
-rw-------. 1 steve steve  169 Mar  3  2015 '/home/steve/Desktop/Nova
Genesis.url'
-rwx------. 1 steve steve 2191 Mar 12 13:10
/home/steve/Desktop/steam.desktop
-rwxr-xr-x. 1 steve steve  370 Jan 12 06:57
/home/steve/Desktop/Thunderbird.desktop
-rwxr--r--. 1 steve steve  313 Oct  5  2013
/home/steve/Desktop/Vuescan.desktop
-rwxrw-r--. 1 steve steve  362 Jan 11 07:10
/home/steve/Desktop/yatta-eclipse-launcher.desktop


bash-4.4$ du -hs /home/steve/Desktop/*
8.0K    /home/steve/Desktop/Acrobat Reader.desktop
8.0K    /home/steve/Desktop/autoplus.desktop
4.0K
/home/steve/Desktop/chrome-eppojlglocelodeimnohnlnionkobfln-Default.desktop
4.0K    /home/steve/Desktop/Eclipse.desktop
4.0K    /home/steve/Desktop/Firefox.desktop
8.0K    /home/steve/Desktop/Google Chrome.desktop
8.0K    /home/steve/Desktop/jws_app_shortcut_1385783507561.desktop
4.0K    /home/steve/Desktop/netbeans-8.2.desktop
4.0K    /home/steve/Desktop/Nova Genesis.url
4.0K    /home/steve/Desktop/steam.desktop
4.0K    /home/steve/Desktop/Thunderbird.desktop
8.0K    /home/steve/Desktop/Vuescan.desktop
4.0K    /home/steve/Desktop/yatta-eclipse-launcher.desktop


bash-4.4$ du -bhs /home/steve/Desktop/*
466     /home/steve/Desktop/Acrobat Reader.desktop
233     /home/steve/Desktop/autoplus.desktop
310
/home/steve/Desktop/chrome-eppojlglocelodeimnohnlnionkobfln-Default.desktop
374     /home/steve/Desktop/Eclipse.desktop
417     /home/steve/Desktop/Firefox.desktop
358     /home/steve/Desktop/Google Chrome.desktop
412     /home/steve/Desktop/jws_app_shortcut_1385783507561.desktop
247     /home/steve/Desktop/netbeans-8.2.desktop
169     /home/steve/Desktop/Nova Genesis.url
2.2K    /home/steve/Desktop/steam.desktop
370     /home/steve/Desktop/Thunderbird.desktop
313     /home/steve/Desktop/Vuescan.desktop
362     /home/steve/Desktop/yatta-eclipse-launcher.desktop

regards,

Steve


> _______________________________________________
> users mailing list -- [hidden email]
> To unsubscribe send an email to [hidden email]
_______________________________________________
users mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: du Weirdness - how is this possible

Ed Greshko
On 03/13/18 04:26, Stephen Morris wrote:
> Just further to this is it a bug with du that the -a parameter which is supposed to
> list all files not just directories, does not list files prefixed with a '.'?:


Of course not.

Compare the -a opt of ls with that of du in their man pages.

ls 

       -a, --all
              do not ignore entries starting with

du

       -a, --all
              write counts for all files, not just directories

With du it is saying "all files found based on the pattern specified in [FILES]"   
And like all shell commands [FILES} follows the rules of "globbing".  Gordon has
already told you a method you can use to have du consider "." files.

This is by turning on the shell option "dotglob".

Run the "shopt" command by itself and notice the setting for "dotglob".  Then set it
with -s to see how this affects commands when you use "*".  And -u will unset options.

You would probably benefit from doing a bit of searching and finding pages such as

https://www.gnu.org/software/bash/manual/html_node/Filename-Expansion.html
https://www.gnu.org/software/bash/manual/html_node/The-Shopt-Builtin.html#The-Shopt-Builtin

and more generally

https://www.gnu.org/software/bash/manual/html_node/

--
Conjecture is just a conclusion based on incomplete information. It isn't a fact.


_______________________________________________
users mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]

signature.asc (235 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: du Weirdness - how is this possible

Stephen Morris
On 13/3/18 8:10 am, Ed Greshko wrote:

> On 03/13/18 04:26, Stephen Morris wrote:
>> Just further to this is it a bug with du that the -a parameter which is supposed to
>> list all files not just directories, does not list files prefixed with a '.'?:
>
> Of course not.
>
> Compare the -a opt of ls with that of du in their man pages.
>
> ls
>
>         -a, --all
>                do not ignore entries starting with
>
> du
>
>         -a, --all
>                write counts for all files, not just directories
>
> With du it is saying "all files found based on the pattern specified in [FILES]"
> And like all shell commands [FILES} follows the rules of "globbing".  Gordon has
> already told you a method you can use to have du consider "." files.
>
> This is by turning on the shell option "dotglob".
>
> Run the "shopt" command by itself and notice the setting for "dotglob".  Then set it
> with -s to see how this affects commands when you use "*".  And -u will unset options.
>
> You would probably benefit from doing a bit of searching and finding pages such as
>
> https://www.gnu.org/software/bash/manual/html_node/Filename-Expansion.html
> https://www.gnu.org/software/bash/manual/html_node/The-Shopt-Builtin.html#The-Shopt-Builtin
>
> and more generally
>
> https://www.gnu.org/software/bash/manual/html_node/
>
Thanks Ed, I'll check the doco out, I was just expecting the command to
do exactly what the help info said, output the information for all
files, not just a subset.


regards,

Steve


>
> _______________________________________________
> users mailing list -- [hidden email]
> To unsubscribe send an email to [hidden email]
_______________________________________________
users mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: du Weirdness - how is this possible

Ed Greshko
On 03/13/18 05:20, Stephen Morris wrote:
> Thanks Ed, I'll check the doco out, I was just expecting the command to do exactly
> what the help info said, output the information for all files, not just a subset.


It *does* do exactly what the man page says.  You just have to understand the
"context" in which it is saying it.

--
Conjecture is just a conclusion based on incomplete information. It isn't a fact.


_______________________________________________
users mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]

signature.asc (235 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: du Weirdness - how is this possible

Ed Greshko
In reply to this post by Stephen Morris
On 03/13/18 05:20, Stephen Morris wrote:
>>
> Thanks Ed, I'll check the doco out, I was just expecting the command to do exactly
> what the help info said, output the information for all files, not just a subset.

It *does* do exactly what the man page says.  You just have to understand the
"context" in which it is saying it.

Let me complete my thought.  Even take the "ls" command as an example.

[egreshko@meimei test-dir]$ ls
test  test1  test2

[egreshko@meimei test-dir]$ ls -a
.  ..  test  test1  test2  .test3  .test4  .test5

[egreshko@meimei test-dir]$ ls *
test  test1  test2

[egreshko@meimei test-dir]$ shopt -s dotglob

[egreshko@meimei test-dir]$ ls *
test  test1  test2  .test3  .test4  .test5

--
Conjecture is just a conclusion based on incomplete information. It isn't a fact.


_______________________________________________
users mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]

signature.asc (235 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: du Weirdness - how is this possible

Ed Greshko-2
On 03/13/18 05:47, Ed Greshko wrote:

> On 03/13/18 05:20, Stephen Morris wrote:
>> Thanks Ed, I'll check the doco out, I was just expecting the command to do exactly
>> what the help info said, output the information for all files, not just a subset.
> It *does* do exactly what the man page says.  You just have to understand the
> "context" in which it is saying it.
>
> Let me complete my thought.  Even take the "ls" command as an example.
>
> [egreshko@meimei test-dir]$ ls
> test  test1  test2
>
> [egreshko@meimei test-dir]$ ls -a
> .  ..  test  test1  test2  .test3  .test4  .test5
>
> [egreshko@meimei test-dir]$ ls *
> test  test1  test2
>
> [egreshko@meimei test-dir]$ shopt -s dotglob
>
> [egreshko@meimei test-dir]$ ls *
> test  test1  test2  .test3  .test4  .test5
>
Oh, when it comes to ls, I've been gently reminded about -A

[egreshko@meimei test-dir]$ ls -A
test  test1  test2  .test3  .test4  .test5


--
Conjecture is just a conclusion based on incomplete information. It isn't a fact.


_______________________________________________
users mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: du Weirdness - how is this possible

Robert Nichols
In reply to this post by Stephen Morris
On 03/12/2018 03:26 PM, Stephen Morris wrote:
> Thanks Patrick, taking this a step further, it seems to me that the only parameter for du that, to me, provides the correct file size is -b as shown below. I am listing my Desktop directory via ll, du -hs and du -bhs. Just further to this is it a bug with du that the -a parameter which is supposed to list all files not just directories, does not list files prefixed with a '.'?:

The du command is doing exactly what it is told to do. It is your _shell_ that is expanding the "*" into a list of names that does not (by default) include "dot" names. The du command is never asked to look at the current directory, and thus will never see the dotfiles, regardless of what flag arguments you use. What you are complaining about is like running "du -a [abc]*" and complaining that names beginning with "d" were not included.

Whenever you have questions like this, you should run "set -x" in the shell to see exactly how commands are being invoked. (Run "set +x" to turn that off again.)

--
Bob Nichols     "NOSPAM" is really part of my email address.
                 Do NOT delete it.
_______________________________________________
users mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: du Weirdness - how is this possible

Patrick O'Callaghan
In reply to this post by Stephen Morris
On Tue, 2018-03-13 at 07:26 +1100, Stephen Morris wrote:

> > 'du' with no parameters recursively lists all the subdirectories and
> > their sizes, along with the grand total. When applied to my home
> > directory, I get over 30,000 lines of output. That's almost never what
> > I want. My usual call is 'du -hs'.
> >
> > poc
>
> Thanks Patrick, taking this a step further, it seems to me that the only
> parameter for du that, to me, provides the correct file size is -b as
> shown below.

I think you have a misconception here. 'du' does not give file sizes,
it gives disk usage. A 1-byte file takes up at least 1 disk block, so
that's the size 'du' will give. I seem to remember that it also counts
indirect blocks and other housekeeping that corresponds to the file
without being included in the file's content, but I could be mistaken
(though I'm fairly sure early versions did do that).

> I am listing my Desktop directory via ll, du -hs and du
> -bhs. Just further to this is it a bug with du that the -a parameter
> which is supposed to list all files not just directories, does not list
> files prefixed with a '.'?:

It does list files beginning with '.'. The reason you aren't seeing
these files is because you're implicitly excluding them when you write
'.../*'. The Shell meta-character '*' doesn't match the initial '.'.

poc
_______________________________________________
users mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: du Weirdness - how is this possible

Rick Stevens-3
On 03/12/2018 03:37 PM, Patrick O'Callaghan wrote:

> On Tue, 2018-03-13 at 07:26 +1100, Stephen Morris wrote:
>>> 'du' with no parameters recursively lists all the subdirectories and
>>> their sizes, along with the grand total. When applied to my home
>>> directory, I get over 30,000 lines of output. That's almost never what
>>> I want. My usual call is 'du -hs'.
>>>
>>> poc
>>
>> Thanks Patrick, taking this a step further, it seems to me that the only
>> parameter for du that, to me, provides the correct file size is -b as
>> shown below.
>
> I think you have a misconception here. 'du' does not give file sizes,
> it gives disk usage. A 1-byte file takes up at least 1 disk block, so
> that's the size 'du' will give. I seem to remember that it also counts
> indirect blocks and other housekeeping that corresponds to the file
> without being included in the file's content, but I could be mistaken
> (though I'm fairly sure early versions did do that).

du (with no flags) gives disk usage. As Patrick says, a 1-byte file
uses one disk block (which is generally 4KiB) and that's what du is
reporting (after all, "du" means "disk usage"). The "-b" flag means
"set the block size to 1 byte and show the apparent size", which is what
"ls -l" would report (there may be differences between du and ls if
sparse files are involved).

Also, du walks down the entire current directory unless you give it
arguments to tell it what to look at. Note that the arguments you pass
it are interpreted by the _shell_, not "du" (even the man page says
"PATTERN is a shell pattern (not a regular expression)").

This is a common confusion point with many people. Unless you enclose
shell metacharacters in quotes (and dots and splats are metacharacters),
the shell WILL interpret them--sometimes in ways you aren't expecting!
By default, shell globbing does NOT expand filenames that start with a
dot (to the shell, a dot means "current directory").

If you want to list JUST files/dirs that start with a dot, you can use
"du [flags] \.* (so the shell interprets the dot as a dot, not "current
directory"). Or you can tell the shell to expand files starting with a
dot by using "shopt -s dotglob" (as Ed said). "shopt -u dotglob" will
restore the default setting.
----------------------------------------------------------------------
- Rick Stevens, Systems Engineer, AllDigital    [hidden email] -
- AIM/Skype: therps2        ICQ: 22643734            Yahoo: origrps2 -
-                                                                    -
-               The Theory of Rapitivity: E=MC Hammer                -
-                                  -- Glenn Marcus (via TopFive.com) -
----------------------------------------------------------------------
_______________________________________________
users mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
12