Sennheiser HD800 S Impressions Thread (read first post for summary)
Jan 16, 2018 at 4:26 PM Post #3,001 of 8,573
Regardless of measurements…at the end of the day it all comes down on how the phones resolve and unravel the intricacies of the music and if their presentation works or does not work for the end user. Haven’t listened to it in quite a while but just sat down with Ry Cooder’s Buena Vista Social Club from 1997 for the first time with the HD800S and the session had no problems in checking off all my boxes. : )
Screen Shot 2018-01-16 at 4.25.33 PM.png
 
Last edited:
Jan 16, 2018 at 6:56 PM Post #3,002 of 8,573
@james444, you are correct that using an amp with low output impedance is going to be more ideal for measuring headphones, to minimize any chance the amp will affect the measurements. While we mostly use the APx1701, we also sometimes use the Neve RNHP (which also has low output impedance). We try to remember to always show (and link to) the instruments we're using for a given measurement, as it's certainly relevant. We have two entirely different headphone measurement fixtures in three possible configurations, so it's important information to know when you're looking at our measurements.

While I very much enjoy several tube amps for listening with the HD800 and HD800S, they would not be reasonable candidates for the purpose of measuring the headphones, for obvious reasons. And before using any device other than the APx1701 to drive headphones or earphones for measurement (like the Neve RNHP, which we use sometimes), we first measure those devices on theAPx555 to assure they're up to the task of both driving the headphone and staying out of the way (and to set levels).

As much as is reasonably possible, the idea is to measure only the device under test (in this case, in this thread, headphones), with the instruments being quiet enough to stay out of the way, and also precise enough to get the full measure of the DUT (device under test). By any measure, I think most would agree that both the Sennheiser HD800 and HD800S are high-precision, low-distortion devices. As such, you need instruments of even greater precision and lower distortion than these high-precision headphones to attempt to get their full and proper measure. To that end, we're using the best audio analyzer currently available and the some of the best measurement microphones and ear simulators currently available.

Given that these are also acoustical measurements, you also want to eliminate as much environmental influence (ambient noise, vibration, etc.) from the setup. If there is no attempt to isolate the headphone and measurement microphones from environmental influence, it will reflect in the measurements. At our office, then, we do also attempt to isolate using a laboratory-grade acoustic and vibration isolation enclosure:



This is something we should definitely examine in greater detail in the future, but, for now, read Dan Foley's article in audioXpress that discusses the importance of precision in audio measurements: “I Can Hear It. Why Can’t I Measure It?”

We've been discussing harmonic distortion and total harmonic distortion (THD). Let's look at Audio Precision's definition of THD in their APx500 manual:



Here's another interesting read at AP.com about THD: More about THD+N and THD

Now let's take a look at one of the first FFT measurements I posted in this thread (click on it to enlarge):



If THD is the measure of all the harmonic distortion products in the DUT's output, let's look at that in this FFT and the distortion products shown. In this measurement, the headphones measured were measured enclosed in the isolation enclosure shown in the above photo. Again, the fundamental in that FFT is a 40 Hz sine wave. H4 is 80 dBSPL below the fundamental, and H5 is more than 80 dBSPL below. H6 is about 93 dBSPL below the fundamental. While perhaps I'm oversimplifying here, I think it makes a relevant point: I've been asked repeatedly about comparative measurements from DIY headphone measurements rigs (that probably generate most of the headphone measurements you find on the web), but I just do not feel they are precise enough to give us the answers we're looking for in this discussion -- a discussion of the harmonic distortion of two very high-precision headphones. Are these DIY headphone measurement rigs precise enough to selectively measure these two headphones' true harmonic distortion amidst instrument noise (see Dan's article above) and environmental noise? If you're hemming and hawing about the answer, let me answer this for you: No.

Tyll also acknowledged the difficulty of properly measuring distortion:



Even with the use of a lab-grade acoustic and vibration isolation enclosure, we turn off our two HVAC systems when measuring. If we see environmental influence in a measurement, we either notate it or discard it entirely. If while a measurement is running, if we hear a loud car, truck, or airplane -- or even if someone flushes a toilet in the office next door -- we stop the measurement and wait. As Tyll mentioned, trying to properly measure a headphone's harmonic distortion means having to make a very sensitive acoustic measurement. And the more precise your device under test, the more precise your measurement setup needs to be. And these two headphones are precise.

Considering the repeatability of the HD800 and HD800S measurements at Innerfidelity, how similar in subbass frequency response the HD800 and HD800S are and that the Innerfidelity has lower distortion measurements at subbass frequencies of headphones like the Audeze LCD2 none of this would be relevant to the discrepancy.

I think people are overestimating how good your equipment needs to be do these bass measurements. As long as the environment is quiet, you repeat test for consistency and you measure at a high enough SPL you will be ok with subbass distortion measurements at -40dB (1%). People are perfectly capable of pulling out <1% distortion measurements with their DIY foam flat baffle setups and shown as much consistently across the <8Khz frequency range. If they couldn't the noise would be obvious in the measurements.

The discrepancy between FFT and THD+N measurements I think is going to be the issue. Innerfidelity has three different HD800 measurements taken at different times and multiple HD800S measurements all showing consistently the same thing. I think people are grasping at straws if they think it's a simple error considering how long Tyll has done this for. We are after all not talking about one outlier here but a consistent pattern.

Jude is getting a very small (but so far consistent) bump in second order distortion (H2) for the HD800S compared to the HD800. That's for a fundamental frequency of 40Hz, to match the FFT that Tyll posted in his review. Jude and Tyll both get higher H2 readings for the HD800S vs. HD800, but Jude's difference is only a fraction of Tyll's.

However, the 40Hz FFT is just a part of the picture. All other distortion measurements I've seen of the HD800S vs. HD800 deal with total harmonic distortion (THD). That's why I asked Jude for THD graphs a few posts back, to give us a more complete picture of his results. (You can't compare apples and oranges in that regard, you need to compare THD to THD)



Sounds like a no-brainer, doesn't it? And yet the matter MAY be a bit more complex still. Note the upper case, because I'm just a curious amateur and not a pro.

What I was trying to point towards in my last post, is that Jude not only has the better, more precise measuring rig, but also the technically better amp to drive these phones. The HDVD800 is Sennheiser's recommended amp for the HD800 / HD800S and certainly no slouch, but Jude's low output impedance amp is undeniably better in controlling the driver movement (= electrical damping) than Senn's higher output impedance amp. Which means, that some issues you may encounter with a higher impedance amp like the HDVD800, may only show up to a much lesser extent in measurements with Jude's amp, because it may be simply "too good".

Note that this is just me thinking out loud, and I may well be totally wrong. But as long as there's no plausible explanation why others are getting different THD results, all bets are off. It would be too easy imo, to ascribe this simply to inferior DIY rigs, at least not until we have more in-depth information.

Nobody uses an amp so bad that it would show distortion at high enough levels to affect distortion measurements of a high impedance headphone like a HD800. You would struggle to even find an amp that bad.
 
Last edited:
Jan 16, 2018 at 10:53 PM Post #3,003 of 8,573
Jan 17, 2018 at 1:51 AM Post #3,004 of 8,573
The discrepancy between FFT and THD+N measurements I think is going to be the issue. Innerfidelity has three different HD800 measurements taken at different times and multiple HD800S measurements all showing consistently the same thing. I think people are grasping at straws if they think it's a simple error considering how long Tyll has done this for. We are after all not talking about one outlier here but a consistent pattern

It's not only about Tyll's results, all other measurement I've seen show higher THD in bass for the HD800S vs. the HD800 as well. As I see it, if these were all simple errors due to inferior DIY rigs, results would have to be more random and not following a consistent pattern (i.e. showing the HD800S with higher THD).

Nobody uses an amp so bad that it would show distortion at high enough levels to affect distortion measurements of a high impedance headphone like a HD800. You would struggle to even find an amp that bad.

Just for clarification, I wasn't talking about amp distortion, but the effect of high output impedance on driver response and distortion. But I'd agree that the effect is not strong enough to explain a difference like the one we've seen between those FFTs.
 
Last edited:
Jan 17, 2018 at 7:28 AM Post #3,005 of 8,573
From Sennheisers website: “The concave glass reflects the sound waves from the rear of the transducer to an absorber,...........

Probably best left for discussion on the 820 thread https://www.head-fi.org/threads/sennheiser-hd820.846244/page-42
But....If indeed it is a symmetrical concave lens and not a banana : ) then that would be correct, otherwise it may be a question of semantics and an error or misunderstanding on part of the copywriter.
 
Jan 17, 2018 at 9:10 AM Post #3,006 of 8,573
It's not only about Tyll's results, all other measurement I've seen show higher THD in bass for the HD800S vs. the HD800 as well. As I see it, if these were all simple errors due to inferior DIY rigs, results would have to be more random and not following a consistent pattern (i.e. showing the HD800S with higher THD).



Just for clarification, I wasn't talking about amp distortion, but the effect of high output impedance on driver response and distortion. But I'd agree that the effect is not strong enough to explain a difference like the one we've seen between those FFTs.

And that is my point exactly as well...
It wasn't just Tyll who found consistently higher bass THD measurements when comparing the HD800S to the regular HD800, but others have measured it as well, back to back, and not just randomly.
I really cannot imagine all of this is just some coincidence.
Anyway, I still enjoy the HD800S regardless of whether it does or does not have higher THD in the bass :)
 
Last edited:
Jan 17, 2018 at 9:46 AM Post #3,007 of 8,573
From Sennheisers website: “The concave glass reflects the sound waves from the rear of the transducer to an absorber, which results in minimal resonance. Thus, the sound waves are effectively “gone” like they would be in open headphones.” ... https://en-us.sennheiser.com/high-end-headphones-audiophiles-hd-820
My picture means with same looking 820 and same function of reflexing sound, the shape as I drawed can be use.
So, dont know exact the shape. For light weigh, lens shape may not be used
 
Jan 17, 2018 at 4:52 PM Post #3,008 of 8,573

Considering the repeatability of the HD800 and HD800S measurements at Innerfidelity, how similar in subbass frequency response the HD800 and HD800S are and that the Innerfidelity has lower distortion measurements at subbass frequencies of headphones like the Audeze LCD2 none of this would be relevant to the discrepancy.

I think people are overestimating how good your equipment needs to be do these bass measurements. As long as the environment is quiet, you repeat test for consistency and you measure at a high enough SPL you will be ok with subbass distortion measurements at -40dB (1%). People are perfectly capable of pulling out <1% distortion measurements with their DIY foam flat baffle setups and shown as much consistently across the <8Khz frequency range. If they couldn't the noise would be obvious in the measurements.

The discrepancy between FFT and THD+N measurements I think is going to be the issue. Innerfidelity has three different HD800 measurements taken at different times and multiple HD800S measurements all showing consistently the same thing. I think people are grasping at straws if they think it's a simple error considering how long Tyll has done this for. We are after all not talking about one outlier here but a consistent pattern.

It's not as simple as thinking you're in a quiet-enough room to measure low levels of distortion. If it was that easy, why bother to isolate at all? Look again at Dan Foley's article in audioXpress:

Test and Measurement: “I Can Hear It. Why Can’t I Measure It?”

Look at the NC (noise criterion) curves, in which the measured sound pressure at each octave band is plotted. In a typical suburban home, noise levels of 30 dBA to 38 dBA can be expected, with a noise spectrum corresponding to the NC30 curve. You can see, then, if you're measuring THD -- and especially if you're measuring THD+N (total harmonic distortion plus noise) -- isolation is critically important, especially at the low levels we're examining here. Notice the shape of the NC curves as you move lower in frequency, and also keep in mind that plot only goes down to 63 Hz.

"No, but it's quiet where I am,"
you might say. You might think that, because perhaps you live in a yurt in the middle of the desert, far away from any civilization -- and/or it might seem that way because our ears follow A-weighting curves, so we don't tend to hear the low frequency energy in our environments at those levels. Microphones, however, will "hear" it.

Again, if it didn't matter, why bother isolating? Why would companies like Sennheiser, Shure, Sony, Audeze, Focal, and other manufacturers bother to isolate? I just received an email from OSSIC today updating Kickstarter participants on their shipping progress. In the email was this photo (which looks to me like a production test rig):

OSSIC.jpg


Why would Microsoft need the quietest place on earth? Why would Roush's NVH (noise, vibration, and harshness) labs go through the expense of building hemi-anechoic chambers large enough to drive trucks into? If you want to do acoustical measurements, isolation does matter if you want to work toward eliminating external influence on the device under test and the measurement system.

This is why we went to considerable expense and trouble to have a lab-grade isolation enclosure built, for as much acoustic and vibration isolation as we could reasonably afford and fit into our office. While I've seen and experienced better isolation at several labs and manufactories than we can achieve here, I don't personally know of anyone doing consumer-facing headphone measurements that has gone to the lengths we have to isolate the headphones being measured. When I want to measure the harmonic distortion of these headphones, I want to do the best I can to make sure I'm only measuring harmonic distortion.

As for the DIY measurement rigs, I'm not going to venture to explain their results. While they obviously have a lot in common when you look at them (since they're mostly guided by the same sets of instructions), their results are not as compelling or authoritative to me, certainly when it comes to distortion measurements, for the reasons I've discussed in this thread already.

If the precision was there in the DIY-type rigs, acoustical engineers the world over would save themselves the grief of trying to secure more budget from management (an age-old engineering ritual) to expand or improve their labs. I think it's very cool that an increasing number of enthusiasts are interested in audio measurements, and I think it's fun to share results and findings with friends. However, when you're going to post conclusions based on those measurements to thousands (even millions) of people, I do think we should expect and encourage greater precision.

Here's an example of what I'm talking about, this time with a completely different kind of audio measurement -- not acoustical, but this one measuring audio electronics (a digital-to-analog converter in this case):

An inexpensive (sub-$100) USB DAC was released. Here's a measurement of it that was posted online by an enthusiast:

DAC.png


This was posted for many thousands to see. It looks fine at first blush. Here's what the poster said about it:

Noise floor is higher here because I'm using a balanced probe with a /10 divider (less spurious garbage, but at the expense of moving the noise floor up 10db). I wouldn't worry about it because really, anything under 100db from the main signal... yeah, only John Atkinson cares to interpret stuff like that.

Now even if you agree with that sentiment, let's take a look to see if the DAC's noise floor is in fact only 10 dB lower:

FFT-Spectrum_USB-DAC_CORRECTED_LABELED.jpg


The pink trace is the one previously posted and overlaid on this plot, and the blue trace is the same measurement done here on our APx555. You can see that DAC's noise floor is not only 10 dB lower than the pink plot, it's actually closer to 35 dB lower through most of the audio band in this measurement. Is that pink measurement (or even a hypothetical one only 10 dB below that) a fair representation of the DAC's noise floor? I don't think so.

Now let's get back to acoustical measurements and the HD800 and HD800S:

Remember, the belief that Sennheiser is deliberately engineering 2nd order harmonic distortion into the HD800S started with one measurement in particular -- an FFT that showed the HD800 as having higher 3rd order than 2nd order distortion with a 40 Hz sine as the stimulus signal. Due to this, the difference in H2 between the two measured headphones (using this stimulus signal) was shown to be 20 dB -- and it's from that significant difference that this theory originated, and then propagated for the past couple of years.

At the ALMA conference, I talked to a few veteran acoustical engineers about this very subject (one of whom is a headphone audio engineer who has designed one of the lowest distortion headphone products I've yet measured), and another at CES (another experienced headphone engineer), none of whom work for Sennheiser. All of them said that they found it extremely unlikely that Sennheiser would do such a thing with this type of headphone (which is also consistent with feedback from Sennheiser).

This is why we focused more on the Sennheiser HD800 measurements. We had two used, older units, and later received three brand new units. None exhibited the higher H3 (versus H2) with a 40 Hz sine stimulus -- all showed substantially higher H2 than H3. Again, the original FFT measurement showing higher H3 (still the only one I've seen to show that) was the primary reason so many people have assumed this theory to be true for nearly two years, which is why we performed the same test so many times.

Sennheiser-HD800-HD800S-measured-units-00393.jpg


We also ran the same test on an HD800 and HD800S at the ALMA conference last week
, one with Audio Precision and other with Brüel & Kjær. I placed and adjusted the headphones in the Audio Precision measurements. However, at Brüel & Kjær, I simply placed the headphones on their new 5128 HATS (head and torso simulator), and the gentleman there took over for the final seatings and measurements. Lest you wonder about his qualifications, the man at Brüel & Kjær's exhibit who did this also helped set up and monitor the measurement system that confirmed Microsoft's anechoic chamber as the record-setting quietest place on earth -- I suspect, then, he's qualified to measure a headphone. Neither AP's nor B&K's measurements showed higher H3 than H2 with the HD800 (or HD800S), both being consistent with what we'd shown before that, and what we've shown since.

Finally, then, here are our HD800 and HD800S THD plots for all the new units (one HD800S and three HD800's). Unless putting these headphones on a human is part of Sennheiser's manufacturing process, none of these headphones had been worn by anyone before the measurements were performed.

The first graph is total harmonic distortion (THD). The second graph is distortion product ratio (H2). In each graph, there are four measurements per headphone. The gray lines are HD800's (serial numbers 49282, 49759, 49760), and the black plots are the HD800S (serial number 13134).

THD-Ratio_Sennheiser-HD800_Sennheiser-HD800S.jpg


Distortion-Product-Ratio-(H2)_Sennheiser-HD800_Sennheiser-HD800S.jpg


Here are our FFT measurements, posted in the order they appeared in this thread:


(Above) Sennheiser HD800 S/N 00342


(Above) Sennheiser HD800 S/N 00342 and Sennheiser HD800S S/N 00205, Seating 1


(Above) Sennheiser HD800 S/N 00342 and Sennheiser HD800S S/N 00205, Seating 2


(Above) HD800S (S/N 13134) and HD800 (S/N 00279):
80 Hz (H2) Δ = 3.331 dBSPL (HD800S > HD800)
120 Hz (H3) Δ = 0.598 dBSPL (HD800 > HD800S)


(Above) HD800S (S/N 13134) versus HD800 (S/N 00342):
80 Hz (H2) Δ = 1.920 dBSPL (HD800 > HD800S)
120 Hz (H3) Δ = 3.611 dBSPL (HD800S > HD800)

UPDATE: Here are three new FFT's (measured on 2018-01-19) showing the brand new HD800's and HD800S. These are overlays of the headphones being driven at both 90 dBFS and 105 dBFS (at 40 Hz). For the sake of this discussion, the most salient point is that all three of the HD800's (as well as the HD800S) show substantially higher H2 than H3 at both 90 dBSPL (40 Hz) and 105 dBSPL (40 Hz).

Sennheiser-HD800-SN-49282---FFT-90---FFT-105.jpg

Sennheiser-HD800-SN-49759---FFT-90---FFT-105.jpg

Sennheiser-HD800-SN-49760---FFT-90---FFT-105.jpg

Sennheiser-HD800S-SN-13134---FFT-90---FFT-105.jpg


Again, at ALMA International's annual conference a couple of weeks ago, I visited the exhibits of Brüel & Kjær (B&K) and Audio Precision (AP) to measure HD800 (S/N 00342) and HD800S (S/N 13134) using their headphone test fixtures. Again, these measurements are consistent with our findings so far, and do not support the original theory (that Sennheiser deliberately added H2 distortion to the HD800S).

The following are the measurements taken by Brüel & Kjær at the conference. I placed the headphones on the head, and Vince Rey from Brüel & Kjær did the final positioning and conducted the measurements using their latest HATS (head and torso simulator) and their PULSE analysis software. Here are the results:




Because it was in an exhibit hall, these B&K measurements were done at 100 dB (40 Hz sine).

Here's the measurement of the same headphones at the Audio Precision exhibit, which was performed by AP's Ryan O'Connor and me, also at 100 dBSPL:



UPDATE: We also ran FFT's on the GRAS KEMAR 45BB-12, as some were curious about using a completely different measurement fixture in our system, and what the results would be from that. Keep in mind that in addition to being a different type of fixture, the 45BB-12 also has unique ear simulators within it called the GRAS 43BB low-noise ear simulators. The GRAS 43BB simulators are highly sensitive, very low-noise, and extend the lower dynamic range below the threshold of human hearing. You'll see, then, that the noise floor is lower through much the audio band than our previous FFT's, which is why the floor will look different than our previous such plots.

Also keep in mind that the GRAS 45CA (because of its flat "cheeks") can (as I said in this video about headphone measurements) result in more measured bass than we'd get with KEMAR with some headphones, and the HD800 / HD800S are among those headphones this is true with. As a result, to achieve a certain bass frequency dBSPL requires substantially more output with KEMAR if you're within the KEMAR's bass roll-off zone with these headphones (and the 39 Hz sine stimulus we used for this test is in that zone). So, for example, I only needed >175 mVrms to achieve 90 dBSPL (40 Hz) with the 45CA, I had to go to >335 mVrms to reach 90 dBSPL (39 Hz) on KEMAR.

This next set consists of four FFT measurements, each comparing the HD800 (S/N 49759) and HD800S (S/N 13134): 39 Hz @ 90 dBSPL, 39 Hz @ 100 dBSPL, 78 Hz @ 90 dBSPL, and 78 Hz @ 100 dBSPL. Here they are:

KEMAR-FFT-HD800-HD800S-39Hz-090dBSPL.jpg


KEMAR-FFT-HD800-HD800S-39Hz-100dBSPL.jpg


KEMAR-FFT-HD800-HD800S-78Hz-090dBSPL.jpg


KEMAR-FFT-HD800-HD800S-78Hz-090dBSPL.jpg


To summarize the measurements: We now have a variety of measurements that include two different HD800S's (one of which is brand new, and one of which is early production, neither of which has ever been modified), and five different stock HD800's (also never modified, which may be relevant), three of which are brand new, one of which is early production, and one of which is early production.

At our office, we have measured with two completely different kinds of test fixtures (GRAS 45CA and GRAS KEMAR) with two completely different kinds of ear simulators (GRAS RA0401 and GRAS 43BB), both sets of which have been calibrated.

We also have included FFT measurements of HD800 S/N 00342 and HD800S S/N 13134 done by others
. Ryan O'Connor from Audio Precision and I did one set at ALMA's annual conference using the Audio Precision AECM206 test fixture. At that same event, Vince Rey from Brüel & Kjær did the other set on B&K's new 5128 HATS. (I had to let Vince handle this one, as I had no experience using B&K's PULSE measurement software.)

Most notably, none of the 40 Hz (or 39 Hz) FFT's -- whether by us, B&K, or AP (the latter two at ALMA) -- show HD800 H3 > H2, as the original FFT in question does.

Again, at the ALMA conference and CES, I talked to several veteran acoustical engineers (including headphone engineers) about this very subject (none of whom work for Sennheiser). And, again, all of them said that they found it extremely unlikely that Sennheiser would do such a thing with this type of headphone (which is also consistent with feedback from Sennheiser).

It seems pretty clear to me at this point -- based on the measurements (by us and others), based on feedback from Sennheiser, and based on feedback from several acoustical engineers in the audio industry not in the employ of Sennheiser -- that the assumption that Sennheiser did deliberately "add second harmonic distortion to the [H800S's] low notes to 'bloom' them up a bit" is not accurate. While I do personally feel (in listening) that the HD800S does have an overall fuller tone than the HD800 does, it would seem that it would be due to methods of tuning other than this, with a resulting headphone that in my opinion sounds no less precise.

Except where otherwise stated, the measurements in this post were made with the following system:


NOTE 2018-01-17 21:20 EST: I made a mistake on the DAC FFT plot. My FFT length and sample rate were not the same as the other FFT, which resulted in a noise floor on my plot on the graph I posted that's about 10 dB lower than it should be. I'm posting a corrected FFT shortly (and will also correct the values stated when I do).


NOTE 2018-01-17 21:50 EST: I posted the corrected DAC measurement above with the corrected FFT length and sample rate, and also corrected the values in the accompanying paragraph. (Click here to see the original graph that, for the purpose of a fair comparison, was not configured correctly.) In the future, for that type of measurement, we may use power spectral density (PSD) or amplitude spectral density (ASD) -- which we'll explain later, in the proper context -- as those will allow us to present noise data in an FFT spectrum with amplitude values independent of FFT length. Sorry about the error, guys.


NOTE 2018-01-22 04:18 EST: FFT's -- including ones by others with two other fixtures -- have been added to this post (above).


NOTE 2018-01-23 17:18 EST: We added FFT measurements from the GRAS KEMAR 45BB-12 (click here to see these above).
 

Attachments

  • upload_2018-1-17_13-57-19.png
    upload_2018-1-17_13-57-19.png
    67.8 KB · Views: 0
  • DAC-overlay.jpg
    DAC-overlay.jpg
    607 KB · Views: 0
  • FFT Spectrum - DAC - CORRECTED.png
    FFT Spectrum - DAC - CORRECTED.png
    381.2 KB · Views: 0
  • KEMAR-FFT-HD800-HD800S-78Hz-100dBSPL.jpg
    KEMAR-FFT-HD800-HD800S-78Hz-100dBSPL.jpg
    899.6 KB · Views: 0
Last edited:
Jan 17, 2018 at 5:56 PM Post #3,009 of 8,573
It's not as simple as thinking you're in a quiet-enough room to measure low levels of distortion. If it was that easy, why bother to isolate at all? Look again at Dan Foley's article in audioXpress:

Test and Measurement: “I Can Hear It. Why Can’t I Measure It?”

Look at the NC (noise criterion) curves, in which the measured sound pressure at each octave band is plotted. In a typical suburban home, noise levels of 30 dBA to 38 dBA can be expected, with a noise spectrum corresponding to the NC30 curve. You can see, then, if you're measuring THD -- and especially if you're measuring THD+N (total harmonic distortion plus noise) -- isolation is critically important, especially at the low levels we're examining here. Notice the shape of the NC curves as you move lower in frequency, and also keep in mind that plot only goes down to 63 Hz.

"No, but it's quiet where I am,"
you might say. You might think that, because perhaps you live in a yurt in the middle of the desert, far away from any civilization -- and/or it might seem that way because our ears follow A-weighting curves, so we don't tend to hear the low frequency energy in our environments at those levels. Microphones, however, will "hear" it.

Again, if it didn't matter, why bother isolating? Why would companies like Sennheiser, Shure, Sony, Audeze, Focal, and other manufacturers bother to isolate? I just received an email from OSSIC today updating Kickstarter participants on their shipping progress. In the email was this photo (which looks to me like a production test rig):



Why would Microsoft need the quietest place on earth? Why would Roush's NVH (noise, vibration, and harshness) labs go through the expense of building hemi-anechoic chambers large enough to drive trucks into? If you want to do acoustical measurements, isolation does matter if you want to work toward eliminating external influence on the device under test and the measurement system.

This is why we went to considerable expense and trouble to have a lab-grade isolation enclosure built, for as much acoustic and vibration isolation as we could reasonably afford and fit into our office. While I've seen and experienced better isolation at several labs and manufactories than we can achieve here, I don't personally know of anyone doing consumer-facing headphone measurements that has gone to the lengths we have to isolate the headphones being measured. When I want to measure the harmonic distortion of these headphones, I want to do the best I can to make sure I'm only measuring harmonic distortion.

As for the DIY measurement rigs, I'm not going to venture to explain their results. While they obviously have a lot in common when you look at them (since they're mostly guided by the same sets of instructions), their results are not as compelling or authoritative to me, certainly when it comes to distortion measurements, for the reasons I've discussed in this thread already. If the precision was there, acoustical engineers the world over would save themselves the grief of trying to secure more budget from management (an age-old engineering ritual) to expand or improve their labs. I think it's very cool that an increasing number of enthusiasts are interested in audio measurements, and I think it's fun to share results and findings with friends. However, when you're going to post conclusions based on those measurements to thousands (even millions) of people, I do think we should expect and encourage greater precision.

Here's an example of what I'm talking about, this time in measuring audio electronics (a DAC in this case):

An inexpensive (sub-$100) USB DAC was released. Here's a measurement of it that was posted online by an enthusiast:



This was posted for many thousands to see. It looks fine at first blush. Here's what the poster said about it:



Now even if you agree with that sentiment, let's take a look to see if the DAC's noise floor is in fact only 10 dB lower:



The pink trace is the one previously posted and overlaid on this plot, and the blue trace is the same measurement done here on our APx555. You can see that DAC's noise floor is not only 10 dB lower than the pink plot, it's actually closer to 45 dB lower in this measurement. Is that pink measurement (or even a hypothetical one only 10 dB below that) a fair representation of the DAC's noise floor? I don't think so.

Now let's get back to the HD800 and HD800S:

Remember, the belief that Sennheiser is deliberately engineering 2nd order harmonic distortion into the HD800S started with one measurement in particular -- an FFT that showed the HD800 as having higher 3rd order than 2nd order distortion with a 40 Hz sine as the stimulus signal. Due to this, the difference in H2 between the two measured headphones (using this stimulus signal) was 20 dB, and it's from that significant difference that this theory originated.

I talked to a few seasoned acoustical engineers at ALMA's conference, and another at CES, to discuss this topic. (None of these engineers work for Sennheiser, but all were quite familiar with the company.) All of them said that they found it extremely unlikely that Sennheiser would do such a thing with this type of headphone (which is also consistent with feedback from Sennheiser).

This is why we focused more on the Sennheiser HD800 measurements. We had two used, older units, and later received three brand new units. None exhibited the higher H3 (versus H2) with a 40 Hz sine stimulus -- all had substantially higher H2 than H3. The original FFT measurement showing higher H3 (still the only one I've seen to show that) was the primary reason so many people have assumed this theory to be true for nearly two years, which is why we performed the same test so many times.

We also ran the same test on an HD800 and HD800S at the ALMA conference last week, one with Audio Precision and other with Brüel & Kjær. I placed and adjusted the headphones in the Audio Precision measurements. However, at Brüel & Kjær, I simply placed the headphones on their new 5128 HATS (head and torso simulator), and the gentleman there took over for the final seatings and measurements. Lest you wonder about his qualifications, the man at Brüel & Kjær's exhibit who did this also set up the measurement system that confirmed Microsoft's anechoic chamber as the record-setting quietest place on earth -- I suspect, then, he's qualified to measure a headphone. Neither AP's nor B&K's measurements showed higher H3 than H2 with the HD800 (or HD800S), both being consistent with what we'd shown before that, and what we've shown since.

Finally, then, here are our HD800 and HD800S THD plots for all the new units (one HD800S and three HD800's). Unless putting them on a human is part of Sennheiser's manufacturing process, none of these headphones had been worn by anyone before the measurements were performed.

The first graph is total harmonic distortion (THD). The second graph is distortion product ratio (H2). In each graph, there are four measurements per headphone. The gray lines are HD800's (serial numbers 49282, 49759, 49760), and the black plots are the HD800S (serial number 13134).





We'll post more measurements later.

It seems pretty clear to me at this point -- based on the measurements (by us and others), based on feedback from Sennheiser, and based on feedback from other acoustical engineers in the audio industry -- that the assumption that Sennheiser added harmonic distortion (specifically, H2) to the Sennheiser HD800S is not accurate.

Good stuff Jude! I've had some reservations about your measurements for a little while now. I've been looking at others for years and their consistency has always been on point, though I have to admit you've really taken things to the next level. Not just at getting the right equipment but also understanding how to use it and present the information!

That said, you don't need to prove your self to any one of course, but I'm sure I speak for others when I say that after such exhaustive posting and experimentation from you guys I have far less reservations! I also 100% agree that manufacturers should be putting out all the necessary measurements for their products. In fact unique measurements for individual serial numbers would go a long way for a lot of these new Top of the Line Headphones! So thank you, for the work you've put in as well as the input you've presented to manufacturers in an effort to push the industry forward!
 
Last edited:
Jan 17, 2018 at 7:43 PM Post #3,010 of 8,573
It's not as simple as thinking you're in a quiet-enough room to measure low levels of distortion. If it was that easy, why bother to isolate at all? Look again at Dan Foley's article in audioXpress:

Test and Measurement: “I Can Hear It. Why Can’t I Measure It?”

Look at the NC (noise criterion) curves, in which the measured sound pressure at each octave band is plotted. In a typical suburban home, noise levels of 30 dBA to 38 dBA can be expected, with a noise spectrum corresponding to the NC30 curve. You can see, then, if you're measuring THD -- and especially if you're measuring THD+N (total harmonic distortion plus noise) -- isolation is critically important, especially at the low levels we're examining here. Notice the shape of the NC curves as you move lower in frequency, and also keep in mind that plot only goes down to 63 Hz.

"No, but it's quiet where I am,"
you might say. You might think that, because perhaps you live in a yurt in the middle of the desert, far away from any civilization -- and/or it might seem that way because our ears follow A-weighting curves, so we don't tend to hear the low frequency energy in our environments at those levels. Microphones, however, will "hear" it.

Again, if it didn't matter, why bother isolating? Why would companies like Sennheiser, Shure, Sony, Audeze, Focal, and other manufacturers bother to isolate? I just received an email from OSSIC today updating Kickstarter participants on their shipping progress. In the email was this photo (which looks to me like a production test rig):



Why would Microsoft need the quietest place on earth? Why would Roush's NVH (noise, vibration, and harshness) labs go through the expense of building hemi-anechoic chambers large enough to drive trucks into? If you want to do acoustical measurements, isolation does matter if you want to work toward eliminating external influence on the device under test and the measurement system.

This is why we went to considerable expense and trouble to have a lab-grade isolation enclosure built, for as much acoustic and vibration isolation as we could reasonably afford and fit into our office. While I've seen and experienced better isolation at several labs and manufactories than we can achieve here, I don't personally know of anyone doing consumer-facing headphone measurements that has gone to the lengths we have to isolate the headphones being measured. When I want to measure the harmonic distortion of these headphones, I want to do the best I can to make sure I'm only measuring harmonic distortion.

As for the DIY measurement rigs, I'm not going to venture to explain their results. While they obviously have a lot in common when you look at them (since they're mostly guided by the same sets of instructions), their results are not as compelling or authoritative to me, certainly when it comes to distortion measurements, for the reasons I've discussed in this thread already. If the precision was there, acoustical engineers the world over would save themselves the grief of trying to secure more budget from management (an age-old engineering ritual) to expand or improve their labs. I think it's very cool that an increasing number of enthusiasts are interested in audio measurements, and I think it's fun to share results and findings with friends. However, when you're going to post conclusions based on those measurements to thousands (even millions) of people, I do think we should expect and encourage greater precision.

Here's an example of what I'm talking about, this time in measuring audio electronics (a DAC in this case):

An inexpensive (sub-$100) USB DAC was released. Here's a measurement of it that was posted online by an enthusiast:



This was posted for many thousands to see. It looks fine at first blush. Here's what the poster said about it:



Now even if you agree with that sentiment, let's take a look to see if the DAC's noise floor is in fact only 10 dB lower:



The pink trace is the one previously posted and overlaid on this plot, and the blue trace is the same measurement done here on our APx555. You can see that DAC's noise floor is not only 10 dB lower than the pink plot, it's actually closer to 45 dB lower in this measurement. Is that pink measurement (or even a hypothetical one only 10 dB below that) a fair representation of the DAC's noise floor? I don't think so.

Now let's get back to the HD800 and HD800S:

Remember, the belief that Sennheiser is deliberately engineering 2nd order harmonic distortion into the HD800S started with one measurement in particular -- an FFT that showed the HD800 as having higher 3rd order than 2nd order distortion with a 40 Hz sine as the stimulus signal. Due to this, the difference in H2 between the two measured headphones (using this stimulus signal) was 20 dB, and it's from that significant difference that this theory originated.

I talked to a few seasoned acoustical engineers at ALMA's conference, and another at CES, to discuss this topic. (None of these engineers work for Sennheiser, but all were quite familiar with the company.) All of them said that they found it extremely unlikely that Sennheiser would do such a thing with this type of headphone (which is also consistent with feedback from Sennheiser).

This is why we focused more on the Sennheiser HD800 measurements. We had two used, older units, and later received three brand new units. None exhibited the higher H3 (versus H2) with a 40 Hz sine stimulus -- all had substantially higher H2 than H3. The original FFT measurement showing higher H3 (still the only one I've seen to show that) was the primary reason so many people have assumed this theory to be true for nearly two years, which is why we performed the same test so many times.

We also ran the same test on an HD800 and HD800S at the ALMA conference last week, one with Audio Precision and other with Brüel & Kjær. I placed and adjusted the headphones in the Audio Precision measurements. However, at Brüel & Kjær, I simply placed the headphones on their new 5128 HATS (head and torso simulator), and the gentleman there took over for the final seatings and measurements. Lest you wonder about his qualifications, the man at Brüel & Kjær's exhibit who did this also set up the measurement system that confirmed Microsoft's anechoic chamber as the record-setting quietest place on earth -- I suspect, then, he's qualified to measure a headphone. Neither AP's nor B&K's measurements showed higher H3 than H2 with the HD800 (or HD800S), both being consistent with what we'd shown before that, and what we've shown since.

Finally, then, here are our HD800 and HD800S THD plots for all the new units (one HD800S and three HD800's). Unless putting them on a human is part of Sennheiser's manufacturing process, none of these headphones had been worn by anyone before the measurements were performed.

The first graph is total harmonic distortion (THD). The second graph is distortion product ratio (H2). In each graph, there are four measurements per headphone. The gray lines are HD800's (serial numbers 49282, 49759, 49760), and the black plots are the HD800S (serial number 13134).





We'll post more measurements later.

It seems pretty clear to me at this point -- based on the measurements (by us and others), based on feedback from Sennheiser, and based on feedback from other acoustical engineers in the audio industry -- that the assumption that Sennheiser added harmonic distortion (specifically, H2) to the Sennheiser HD800S is not accurate.

Glad to see similar THD results for the HD800 and HD800S on your setup.

Manufacturers spend huge amounts of money on laboratory grade equipment because it provides high consistency in a factory environment with a lot of noise around them and that it is an asset to the development of multiple types of product over decades. Loudspeakers in particular require anechoic measurements for accurate spinoramas.

A fancy anechoic chamber or isolation box will provide a lower noise floor but the proof of DIY setups has always been that their results tend to match reference equipment orders of magnitude more expensive than them for aspects like THD. There will be nuances and limitations but headphone measurements have so many limitations that it's true for all setups. I don't want people to get the impression this is space age hardware vs stone tools because it isn't. At the same time if one setup is capable of measuring distortion down to 0.5% and the other to 0.1% it isn't actually going to mean much as neither of these correlate well to audibility where masking in music means that people tolerate very high levels of THD anyway.

Even the most expensive B&K setup isn't going to mean much anyway considering for frequency response considering just how much variety there is between people with different head sizes, pinna shape and ear canal shapes. Frequency response in bass regions can change significantly depending on seal and seal varies a lot from person to person. There are just so many factors at play that individuals should take some healthy skepticism towards how well measurements will actually correlate to what they hear.

Laboratory equipment also tends to be much more expensive because of R&D costs, low tolerances, small markets and the cost of getting required certification. A consumer mindset of improvement with price doesn't apply.

So the next question is why different setups such as the one at Innerfidelity are showing a difference and this one isn't. I don't know whose is correct but the answer to why there is a difference is worth knowing.
 
Last edited:
Jan 17, 2018 at 9:41 PM Post #3,011 of 8,573

NOTE 2018-01-17 21:20 EST: I made a mistake on the DAC FFT plot. My FFT length and sample rate were not the same as the other FFT, which resulted in a noise floor on my plot on the graph I posted that's about 10 dB lower than it should be. I'm posting a corrected FFT shortly (and will also correct the values stated when I do).

NOTE 2018-01-17 21:50 EST: I posted the corrected DAC measurement above with the corrected FFT length and sample rate, and also corrected the values in the accompanying paragraph. (Click here to see the original graph that was configured incorrectly / unfairly.) In the future, for that type of measurement, we may use power spectral density (PSD) or amplitude spectral density (ASD) -- which we'll explain later, in the proper context -- as those will allow us to present noise data in an FFT spectrum with amplitude values independent of FFT length. Sorry about the error, guys.

Thanks for the candid updates and corrections. I had assumed the graph was just from a random site but... it isn't. The continuous negative attention towards that same community over and over still leaves me with some reservations, far less than before but still some. Just let them sling their mud. As you continue to improve the quality of what your posting here the quality of the work alone will speak for it self, I want people to come to Head Fi not because we tell them that the the DIY communities aren't "precise enough" but rather because here at Head Fi it's obvious that the quality or precision is that much better!

Looking forward to the next Note with a few more HD 800S postings!
 
Last edited:
Jan 18, 2018 at 6:59 AM Post #3,012 of 8,573
DIY measurement really shouldn't hold any weight in any discussion for a high end stock product, they might be useful to help visualize differences made due to modding and such.

But when used as an absolute measurement and even using it to adjust their overall judgement of a product? How many of those rigs are properly calibrated, I do wonder? If the measurement seems near enough to another DIY measurement, does it mean that it is working perfectly already? Have anyone worked out the system's measurement uncertainty? If so, does the so call differences fall within the uncertainty range?

As someone that professionally take measurements for a living. Using DIY measurement solutions without real versification and taking it as the truth, it's simply unthinkable. Keep it as a tool to visualize modding differences, and leave judging headphone quality, engineering choices to your ears, it's already calibrated.
 
Last edited:
Jan 18, 2018 at 1:32 PM Post #3,014 of 8,573
It's not as simple as thinking you're in a quiet-enough room to measure low levels of distortion.
Thanks Jude for clearing that up. You put a lot into this research. I think we can put to rest the idea that they added bass with distortion, and that Sennheiser wouldn't admit that if they did. Users who'd claimed measuring distortion meant well, no hard feelings. That said, Sennheiser, add some damned bass to your open headphones please. Just don't even THINK of using distortion to get there. LOL
 
Jan 18, 2018 at 3:39 PM Post #3,015 of 8,573
I sincerely apologize for any affront to anyone my posts may have made. Though they are often in jest, hurting anyone with the strengths of my writing and words is something I apologize for with all my heart. Those of you affected, please forgive me.

Tari
 
Last edited:

Users who are viewing this thread

Back
Top