FreeBSD and Linux SSD Write Speeds

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

FreeBSD and Linux SSD Write Speeds

Tim Daneliuk
Disclaimer: Tests below run on lightly loaded systems, but results are ... surprising:

Test Case: dd if=/dev/zero of=foo bs=8M count=512

Linux 4.4.0-21-generic on a 2.66GHz Core2 Duo w/8GB memory, older OCZ SSD/ext4:  310MB/sec writes

FreeBSD 10-STABLE on an 3.2 GHz Quad Core i5 w/8GB memory, newer Kingston SSD/ufs:  210MB/sec writes

Results are repeatable.

So, what is the likely culprit making FreeBSD 1/3 slower?  The FreeBSD system does does / nfs exported (which I don't quite yet understand since all the nfs mount points are below it) at the moment, but there is little or no nfs traffic.


_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD and Linux SSD Write Speeds

Jov-2
Try add sync option to dd and write twice of memory size data(16gb for your
hw).


Jov


2017年6月22日 12:04 AM,"Tim Daneliuk" <[hidden email]>写道:

Disclaimer: Tests below run on lightly loaded systems, but results are ...
surprising:

Test Case: dd if=/dev/zero of=foo bs=8M count=512

Linux 4.4.0-21-generic on a 2.66GHz Core2 Duo w/8GB memory, older OCZ
SSD/ext4:  310MB/sec writes

FreeBSD 10-STABLE on an 3.2 GHz Quad Core i5 w/8GB memory, newer Kingston
SSD/ufs:  210MB/sec writes

Results are repeatable.

So, what is the likely culprit making FreeBSD 1/3 slower?  The FreeBSD
system does does / nfs exported (which I don't quite yet understand since
all the nfs mount points are below it) at the moment, but there is little
or no nfs traffic.


_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[hidden email]"
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD and Linux SSD Write Speeds

Valeri Galtsev
In reply to this post by Tim Daneliuk

On Wed, June 21, 2017 11:04 am, Tim Daneliuk wrote:
> Disclaimer: Tests below run on lightly loaded systems, but results are ...
> surprising:
>
> Test Case: dd if=/dev/zero of=foo bs=8M count=512

First, I would modify the test and include sync command inside timing,
something along these lines:

sync; date; dd if=/dev/zero of=foo bs=8M count=512; sync; date

( and your measure is the difference of output of two "date" commands, not
what dd reports )

Second, I would make the size dd sends really big, say 10 GB.

( your bs probably _is_ multiple of fs block size, but that is something
to watch for too).

Next, I would exclude filesystem at all, cut small ( larger than 10 GB)
partition off the SSD, and make dd destination that partition, and see
what difference does that make.

Incidentally, what filesystem is it on FreeBSD side, and how many open
files (owned by other processes) do you have on the fiesystems your dd
destination file is on?

Valeri

>
> Linux 4.4.0-21-generic on a 2.66GHz Core2 Duo w/8GB memory, older OCZ
> SSD/ext4:  310MB/sec writes
>
> FreeBSD 10-STABLE on an 3.2 GHz Quad Core i5 w/8GB memory, newer Kingston
> SSD/ufs:  210MB/sec writes
>
> Results are repeatable.
>
> So, what is the likely culprit making FreeBSD 1/3 slower?  The FreeBSD
> system does does / nfs exported (which I don't quite yet understand since
> all the nfs mount points are below it) at the moment, but there is little
> or no nfs traffic.
>
>
> _______________________________________________
> [hidden email] mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-questions
> To unsubscribe, send any mail to
> "[hidden email]"
>


++++++++++++++++++++++++++++++++++++++++
Valeri Galtsev
Sr System Administrator
Department of Astronomy and Astrophysics
Kavli Institute for Cosmological Physics
University of Chicago
Phone: 773-702-4247
++++++++++++++++++++++++++++++++++++++++
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD and Linux SSD Write Speeds

Adam Vande More
In reply to this post by Tim Daneliuk
On Wed, Jun 21, 2017 at 11:04 AM, Tim Daneliuk <[hidden email]>
wrote:

> Disclaimer: Tests below run on lightly loaded systems, but results are ...
> surprising:
>
> Test Case: dd if=/dev/zero of=foo bs=8M count=512
>
> Linux 4.4.0-21-generic on a 2.66GHz Core2 Duo w/8GB memory, older OCZ
> SSD/ext4:  310MB/sec writes
>
> FreeBSD 10-STABLE on an 3.2 GHz Quad Core i5 w/8GB memory, newer Kingston
> SSD/ufs:  210MB/sec writes
>
> Results are repeatable.
>
> So, what is the likely culprit making FreeBSD 1/3 slower?  The FreeBSD
> system does does / nfs exported (which I don't quite yet understand since
> all the nfs mount points are below it) at the moment, but there is little
> or no nfs traffic.
>

SSD speeds can vary greatly based upon models.  What steps did you take to
ensure the rates you are seeing aren't completely normal?


--
Adam
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD and Linux SSD Write Speeds

Tim Daneliuk
In reply to this post by Jov-2
On 06/21/2017 11:23 AM, Jov wrote:
> Try add sync option to dd and write twice of memory size data(16gb for your hw).
>
>
> Jov
OK, reran with:

    dd if=/dev/zero of=foo bs=8M count=1024 conv=sync


Results (fastest of several runs):

Linux:   265MB/sec

FreeBSD: 180MB/sec


The fact that there is some variability us unremarkable.  What is
remarkable is that the FreeBSD is more powerful and has a much
newer SSD drive.


----------------------------------------------------------------------------
Tim Daneliuk     [hidden email]
PGP Key:         http://www.tundraware.com/PGP/

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD and Linux SSD Write Speeds

Tim Daneliuk
In reply to this post by Valeri Galtsev
On 06/21/2017 11:27 AM, Valeri Galtsev wrote:
> sync; date; dd if=/dev/zero of=foo bs=8M count=512; sync; date

So I just did this:

   sync; date; dd if=/dev/zero of=foo bs=8M count=1024; sync; date

And got this:

Linux:

Wed Jun 21 13:30:22 CDT 2017
1024+0 records in
1024+0 records out
8589934592 bytes (8.6 GB, 8.0 GiB) copied, 33.523 s, 256 MB/s
Wed Jun 21 13:30:57 CDT 2017


FreeBSD:

Wed Jun 21 13:28:23 CDT 2017
1024+0 records in
1024+0 records out
8589934592 bytes transferred in 46.508336 secs (184696666 bytes/sec)
Wed Jun 21 13:29:10 CDT 2017

So ... still a pretty big disparity

I don't want to test against a raw device or partition because that's
not how these systems get used.  My FreeBSD FS is UFS and maybe that
simply is not as efficient on an SSD as ext4.

This isn't a big deal, just an eyebrow raiser for me.  I've used FreeBSD
as a production OS since 2.x and have always found it to be consistently
quicker and have a smaller footprint than Linux.  This was just a surprise.

--
----------------------------------------------------------------------------
Tim Daneliuk     [hidden email]
PGP Key:         http://www.tundraware.com/PGP/

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD and Linux SSD Write Speeds

Tim Daneliuk
In reply to this post by Adam Vande More
On 06/21/2017 11:45 AM, Adam Vande More wrote:
> SSD speeds can vary greatly based upon models.  What steps did you take to ensure the rates you are seeing aren't completely normal?

They may well be normal.  I need to go back and look at the specs, but
as I recall, the OCZ drive was rated in the 400-ish Mbit/sec range
and the much newer Kingston was in the 500-ish MBit/sec range.  But
I am not certain of this.


--
----------------------------------------------------------------------------
Tim Daneliuk     [hidden email]
PGP Key:         http://www.tundraware.com/PGP/

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD and Linux SSD Write Speeds

Mark Moellering-2
I guess, for me, to be a true experiment, you need to swap the drives
between the two machines and then see what happens.
The other thought, is wondering how big the sector sizes are.  If you
indicate the wrong sector size, i believe performance will take a hit.

It is an interesting experiment...

-- Mark


On Wed, Jun 21, 2017 at 2:35 PM, Tim Daneliuk <[hidden email]> wrote:

> On 06/21/2017 11:45 AM, Adam Vande More wrote:
>> SSD speeds can vary greatly based upon models.  What steps did you take to ensure the rates you are seeing aren't completely normal?
>
> They may well be normal.  I need to go back and look at the specs, but
> as I recall, the OCZ drive was rated in the 400-ish Mbit/sec range
> and the much newer Kingston was in the 500-ish MBit/sec range.  But
> I am not certain of this.
>
>
> --
> ----------------------------------------------------------------------------
> Tim Daneliuk     [hidden email]
> PGP Key:         http://www.tundraware.com/PGP/
>
> _______________________________________________
> [hidden email] mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-questions
> To unsubscribe, send any mail to "[hidden email]"
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD and Linux SSD Write Speeds

jd1008
In reply to this post by Tim Daneliuk


On 06/21/2017 12:33 PM, Tim Daneliuk wrote:

> On 06/21/2017 11:27 AM, Valeri Galtsev wrote:
>> sync; date; dd if=/dev/zero of=foo bs=8M count=512; sync; date
> So I just did this:
>
>     sync; date; dd if=/dev/zero of=foo bs=8M count=1024; sync; date
>
> And got this:
>
> Linux:
>
> Wed Jun 21 13:30:22 CDT 2017
> 1024+0 records in
> 1024+0 records out
> 8589934592 bytes (8.6 GB, 8.0 GiB) copied, 33.523 s, 256 MB/s
> Wed Jun 21 13:30:57 CDT 2017
>
>
> FreeBSD:
>
> Wed Jun 21 13:28:23 CDT 2017
> 1024+0 records in
> 1024+0 records out
> 8589934592 bytes transferred in 46.508336 secs (184696666 bytes/sec)
> Wed Jun 21 13:29:10 CDT 2017
>
> So ... still a pretty big disparity
>
> I don't want to test against a raw device or partition because that's
> not how these systems get used.  My FreeBSD FS is UFS and maybe that
> simply is not as efficient on an SSD as ext4.
>
> This isn't a big deal, just an eyebrow raiser for me.  I've used FreeBSD
> as a production OS since 2.x and have always found it to be consistently
> quicker and have a smaller footprint than Linux.  This was just a surprise.
By my prior use of BSD and after that FreeBSD, the UFS was never
a shining star of disk I/O speeds.

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD and Linux SSD Write Speeds

Polytropon
In reply to this post by Mark Moellering-2
On Wed, 21 Jun 2017 14:45:04 -0400, Mark Moellering wrote:
> I guess, for me, to be a true experiment, you need to swap the drives
> between the two machines and then see what happens.
> The other thought, is wondering how big the sector sizes are.  If you
> indicate the wrong sector size, i believe performance will take a hit.

Another question to ask would be this:

UFS and ext4 allow certain options for initializing a file system.
It's possible that those options have been set in a certain way,
for example, to have UFS run with suboptimal settings. I thin dd
operates on file system level here (instead of "on bare device"),
so those FS settings _might_ have an impact on the results.


--
Polytropon
Magdeburg, Germany
Happy FreeBSD user since 4.0
Andra moi ennepe, Mousa, ...
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD and Linux SSD Write Speeds

Valeri Galtsev
In reply to this post by Tim Daneliuk

On Wed, June 21, 2017 1:33 pm, Tim Daneliuk wrote:

> On 06/21/2017 11:27 AM, Valeri Galtsev wrote:
>> sync; date; dd if=/dev/zero of=foo bs=8M count=512; sync; date
>
> So I just did this:
>
>    sync; date; dd if=/dev/zero of=foo bs=8M count=1024; sync; date
>
> And got this:
>
> Linux:
>
> Wed Jun 21 13:30:22 CDT 2017
> 1024+0 records in
> 1024+0 records out
> 8589934592 bytes (8.6 GB, 8.0 GiB) copied, 33.523 s, 256 MB/s
> Wed Jun 21 13:30:57 CDT 2017
>
>
> FreeBSD:
>
> Wed Jun 21 13:28:23 CDT 2017
> 1024+0 records in
> 1024+0 records out
> 8589934592 bytes transferred in 46.508336 secs (184696666 bytes/sec)
> Wed Jun 21 13:29:10 CDT 2017
>
> So ... still a pretty big disparity

It is.

>
> I don't want to test against a raw device or partition because that's
> not how these systems get used.  My FreeBSD FS is UFS and maybe that
> simply is not as efficient on an SSD as ext4.

OK, at least it is not zfs on FreeBSD: zfs has its reasons to be slower.
As others mentioned, it can be one or more of: particular two devices
performance difference; less than optimal parameters of UFS; UFS itself
being not that "shiny" compared ext4 ( I somehow discount the difference
in scheduler, process switching, etc...)

Oh, well... I feel like devoting some time to experimenting too: curiosity
;-)

Valeri

>
> This isn't a big deal, just an eyebrow raiser for me.  I've used FreeBSD
> as a production OS since 2.x and have always found it to be consistently
> quicker and have a smaller footprint than Linux.  This was just a
> surprise.
>
> --
> ----------------------------------------------------------------------------
> Tim Daneliuk     [hidden email]
> PGP Key:         http://www.tundraware.com/PGP/
>
> _______________________________________________
> [hidden email] mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-questions
> To unsubscribe, send any mail to
> "[hidden email]"
>


++++++++++++++++++++++++++++++++++++++++
Valeri Galtsev
Sr System Administrator
Department of Astronomy and Astrophysics
Kavli Institute for Cosmological Physics
University of Chicago
Phone: 773-702-4247
++++++++++++++++++++++++++++++++++++++++
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD and Linux SSD Write Speeds

Jakub Lach
> I happen to have an unused 256G Kingston V300 SSD.  When I get a moment,
> I want to test speeds against the bare drive on both machines, and then
> via the filesystem on each.  That way the drive would no longer be the
> variable.

Only way to do it imho, but ideally should be the same machine too.



--
Sent from: http://freebsd.1045724.x6.nabble.com/freebsd-questions-f3696945.html
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[hidden email]"