FSAF (Fast subband adaptive filtering) measurement

So far I have tested either with 5s pink noise, or 15s music clip, and two different speakers. Unfortunately I don't have another audio interface, and my only other XLR mic is a large condenser vocal mic, not a measurement mic but I suppose it couldn't hurt to try just for comparison.

Testing in my office space, the primary source of background noise is my PC, however it is a rather quiet one, I use Noctua fans throughout with optimized speed control, and there is a sound baffle on the front air intake. In my testing, the possibility of a "bad" measurement appeared to be completely random, not correlated to background noise, it doesn't matter if I try to keep very quiet or not. See post 115 above, noise floor is the same between the two measurements shown there. Other thing to note is that when listening to the residual files for my music tests, the "problem" exists throughout the entire 15s measurement. I thought perhaps some small timing errors were to blame, but I get the same result using timing markers or not.

FWIW, here's an RTA of my background noise this morning in a quiet house, so only computer noise, and my mouse movements and breathing.


I'd really like to get to the bottom of this so I can make the best use of FSAF. As it is, unfortunately I can't put much trust in the result of a single measurement, so I must take several and pick the best one.

Today I caught REW adjusting timing by -0.11ppm, I don't think it's done that before. This is possibly the worst selection of 10 repeats yet. Here I changed to measuring just a tweeter, it's a big AMT, Dayton AMTPRO-4, so keep in mind everything <700Hz is just noise. 500Hz LR2 high passed pink noise was used, SPL from 80-85dB/1m in the pass band.

Thanks for taking a look, John.

I have to think that this problem is somehow inherent to my use of Motu M4. I've changed all other variables and receive the same results.

I recall about a year ago, I discussed with the ARTA developer on some inconsistency with FR rolloff below the window limit when using a sine sweep, which was due to some subsonic noise in the way the response was being processed. Perhaps something similar is at play here?

Reviewing my measurements, low frequency noise is the only place I am seeing any real difference in measured result.


As a test, I reduced the IR length to its minimum value of 100ms, and believe I am getting more repeatable results. Same measurement as post 127 above, but with 100ms IR length. Not perfect, but definitely an improvement in repeatabilit, and consistently at a higher distortion level than the 500ms measurement.

Just to highlight the variability in low end noise. Below is 10 sine sweep measurements, and 10 FSAF measurements. The FSAF measurements stop at 10Hz. All are processed with right window of 500ms.

Both sine sweep and FSAF are roughly equivalent for the measuring room response. Not showing the noise floor is a GUI problem, not a method. If you knew that everything below 15Hz is just noise, you would not worry. If you do need to measure LF response, you'll need a mic like Rode NT1.

FSAF's "purpose" is to enable you to listen to the distortions of the loudspeaker "alone", unmasked, on the music you like, and be able to compare the results of any modifications, apple to apple.

I'm not trying to measure LF nor do I care about it at this moment. I'm having issues with FSAF measurement repeatability, and currently the LF noise is the only thing that's not constant from measurement to measurement. It's not easily apples to apples when any given measurement can vary by 10dB.

Equipment is Motu M4 and Line Audio OM1. I also have a large condenser vocal mic (Neumann U87 knock off) , the measurement variability did not improve with it. Loopback measurement is consistent / repeatable, but no acoustic measurement I've taken is.

Ok, got it. I did not test FSAF below 20Hz. Not tested - does not work:-) AFAIK, John applies some special tricks to make sine sweep work in low frequencies. I did not do anything special.

Generally, you need to put enough energy into the frequency band you are interested in and make sure your observation noise is low enough. You may try brown noise as excitation. Create it yourself in Adobe Audition or apps alike. Music (classical/jazz/acoustic) rarely has any content below 40Hz. If you continue having this problem, send the recordings to me, and I will see if anything can be done.

Good to see you from the West Coast.

Let me know if I can be of any support in your corner line array project. It reminds me of the Murphy Corner Line Array


I agree with what he says, and limitations of CTA2034A.

“The typical placement of point source, line or planar speakers in our listening rooms results in especially bad comb filtering because the reflected images are so distant from the sound source. Ultimately what really matters in a high performance audio monitoring system is the frequency response that the system achieves in-room at the listening area, not the anechoic response that WOULD occur IF the system were auditioned in an anechoic chamber...in mono.”


Best regards,
Thanh from Down Under

Thanks!
I've tried before to register on this forum to help John ... but it did not work with my outlook email. Now I tried another - and it worked.

@dcibel : I tried brown noise on my setup, and got the following (dotted lines are 2 sigma error estimates):

Hope it helps.

I've tried pink noise, and 15s music clip. I've sent you a PM with the same MDAT and residual files I sent John previously.

It doesn't matter what I use for stimulus signal, the variability exists in the measurements.

Set up for brown noise.


Result:

I've sent Mike the same MDAT and residual files that I sent to you previously.

I'll have to re-do the measurement with the other options selected. I'll plan for that this evening.

I run the same data through MATLAB. The results:

Here we also see the dashed line: internal FSAF estimate of the model errors. With some imagination:-), you may find a degree of correspondence to the FR variations. So, yes, these variations are to be expected. Nothing abnormal. The question is: what do they mean?
Some of the sources of errors and variability:
1. method
2. excitation aka stimulation
3. microphone noise & distortions
4. room noise and RT60
5. the loudspeaker itself, i.e. this variability is an indication of the loudspeaker/driver design or manufacture.

Let's go step by step.
The [exponential] sine sweep (ESS) method supplies constant power, drivers quickly come to a thermal balance, and it provides nice results. They may be too good to be true... Also, ESS drives tweeters nuts and ignores Barkhausen noise. The FSAF tries to find an approximation of IR that minimizes observation error (aka residual), on any stimulation. You can use FSAF on MLS or sine sweep. Performance is limited by Cramer-Rao bounds. On each frequency, you take the ratio of the energy of the stimulus you send and the observation noise+distortions. The Best Linear Unbiased Observator (BLUE) is Least Squares (LS) for LTI models. If a method claims to exceed the LS performance and Cramer-Rao bounds, it lies. FSAF uses a regularized (kernel-based) estimation in subbands. It is pretty close to BLUE. There are a few exceptions related to eigenvalue distribution singularities, but in general, if it shows that something is not perfect, it is worthwhile considering that something is indeed not perfect.

Let's look at the same picture in wideband:

Some measurements have extra noise and should have been discarded. Also, there are reasons to suspect the tweeter (>2k5 Hz) to be of concern. It should not vary so much. Is it AMT?
Let's compare this loudspeaker to another using the same stimulation on 80 dB SPL RMS @1m, Focal 908.1 circa 1994. It degraded a lot since, totally unsuitable for classical/acoustic music, but still ok for rock/pop.

Generally, there are fewer distortions and variability. In the LF:

... it also varies a lot below -25 dB, and FSAF estimated model error agrees with variability. But is it due to method or stimulation? Let's try the same FSAF on brown noise, the same 80 dB SPL:

Of course, I use a less noisy mic (12 dBA, 1% at 145 dB), and my room is treated (RT60 is ~250ms, with lower undermodelling errors), in a new building with good sound insulation, on a quiet street.

You may find it useful to have friends in audio retail, who may allow you to measure their best and worst loudspeakers. They may require strict confidentiality, etc. Then it would be much easier as you have a good base to compare with. Other people may submit their measurements too. If a set of stimulations is agreed upon, you are set.

FSAF said:

Let's go step by step.
The [exponential] sine sweep (ESS) method supplies constant power, drivers quickly come to a thermal balance, and it provides nice results. They may be too good to be true... Also, ESS drives tweeters nuts and ignores Barkhausen noise. The FSAF tries to find an approximation of IR that minimizes observation error (aka residual), on any stimulation. You can use FSAF on MLS or sine sweep. Performance is limited by Cramer-Rao bounds. On each frequency, you take the ratio of the energy of the stimulus you send and the observation noise+distortions. The Best Linear Unbiased Observator (BLUE) is Least Squares (LS) for LTI models. If a method claims to exceed the LS performance and Cramer-Rao bounds, it lies. FSAF uses a regularized (kernel-based) estimation in subbands. It is pretty close to BLUE. There are a few exceptions related to eigenvalue distribution singularities, but in general, if it shows that something is not perfect, it is worthwhile considering that something is indeed not perfect.

Click to expand...

Thanks for taking the time to analyze my measurements. I must apologize, a few sentences into this explanation gets over my head a bit.

FSAF said:

Let's look at the same picture in wideband:
View attachment 75062
Some measurements have extra noise and should have been discarded. Also, there are reasons to suspect the tweeter (>2k5 Hz) to be of concern. It should not vary so much. Is it AMT?

Click to expand...

This is exactly my problem, see past few pages of this thread. This Speaker is a Satori MW16P and a 28mm hard dome tweeter in a shallow waveguide. The speaker is of my own design, and crossover is at ~2kHz.


I had lengthy response written up as I completed testing this evening, including measuring at 1cm to prove a point that the variation had little to do with the room and background noise. However, I'll cut to the chase - the variance here is caused by myself, and I believe I've determined the cause. It's my own test jig. A simple jig that I use for testing all my speakers, when set up for "SPL" measurements, the signal simply runs through a switch - an industrial style rocker switch rated at 15A. It's provided perfectly good repeatability and consistency for my all my measurements in the past - ESS, HD, IMD, MLS, periodic noise, you name it, and I've just made a habit of leaving it in place for all my measurements whether I need it or not, and didn't even think to remove it in all my troubleshooting. Somehow this FSAF was doing something special, at seemingly random intervals. I have to think that the switch contact is not as good as it could be. Stupid me connected the amp straight to the speaker, and low and behold, the situation is much improved. I tested 2 different speakers to be sure.

As well, through testing, a "quiet room" seemed to not be good enough for this test on high end speakers, a "very quiet room" is required.

Tests below are 5s of pink noise.

Speaker 1 (same as previous):

Speaker 2:


Thanks @FSAF and @John Mulcahy for bearing with me through this, but I'm glad the problem was simple and less glad that it took me this long to figure it out for myself.


@John Mulcahy , for REW's display, I notice that FSAF above is showing the distortion levels relative to the "corrected" SPL for lack of a better term. In REW, if the scale is set to SPL, the fundamental on the distortion tab and SPL tab are quite a different story. Would it make sense to show the "corrected" SPL on the distortion tab?


vs

dcibel said:

As well, through testing, a "quiet room" seemed to not be good enough for this test on high end speakers, a "very quiet room" is required.

Click to expand...

Absolutely correct.

BTW, the spike in distortions in Speaker 1 at ~1.3kHz may indicate that the back wall at ~270mm is too reflective.

Tweeters are a hassle. Someday most music was acoustic with spectrum decaying as 1/f^2. Low-power tweeters were adequate then. Now it is 1/f, and the power in tweeters is about the same as in midrange. Some tweeters have blackened aluminum coils (no former) in the blackened gap and are capable of holding some power. Most - don't. For my current project, I plan on using 4 Nd tweeters with their waveguides cut below and above and putting them in an array, f >= 5kHz.

FSAF said:

BTW, the spike in distortions in Speaker 1 at ~1.3kHz may indicate that the back wall at ~270mm is too reflective.

Click to expand...

I believe this is a problem inherent to the Satori MW16P, caused by cone-surround interaction to my knowledge. Speaker 2 above was tested in the same room location.

The Satori shows a frequency response dip, as well as a 2nd order harmonic distortion spike at that frequency. Any test of this driver that you find will show it, including my own:

I'll test a bit more on that spike. It could be related to some other problem, perhaps a leaking baffle seal or something buzzing. It seems a little large to be solely the result of a fairly benign 2nd order distortion.

How should I interpret the SPL on the distortion chart as far as the absolute level? If I recall, the level was 90dB via the "check levels" prior to measuring. the Distortion fundamental peaks at 120dB, which seems like a bit much, and higher than 90dB + crest factor of 17.8dB = 107.8dB.

If a music clip is 5-60 seconds, with varying levels of SPL and varying frequencies over time, how does one make sense of the Distortion graph?

I can understand that the Noise graph is a captured moments before the recording was commenced, but how does one make sense of the Total Distortion + Noise graph, which is a static graph.

Also, is REW's Spectrogram equivalent to @FSAF 's spectrogram generated in Matlab?

Or are we just supposed to dispense with the graphs, and start listening to the Disrotion residuals? An MW16TX would be interesting comparison for @dcibel

I have new audio interface and/or computer arriving soon, so hopefully my problems with clipping samples are over. Like @dcibel , I can start measuring and publishing things without intermittent artefacts.

dcibel said:

I am still struggling on comprehension of measurement results unfortunately.

Below is a test of a pretty good speaker, a 2-way bookshelf using 6" Satori woofer. A hard dome waveguide tweeter is crossed at 2kHz.
FSAF test, full range pink noise, RMS level is fairly low around 78dB/1m
View attachment 74849
If I add in a 12dB LR high pass at 20Hz, the overall distortion level is decreased significantly. Is cutting the bass output below 20Hz really having such an impact to distortion levels >2kHz?
View attachment 74850

Click to expand...

If I could return to this original problem on pink noise with no filter vs pink noise with a 20Hz high pass. Now that I have much better confidence in my test results, I revisited this tonight.

The speaker being tested uses a 6" Satori MW16P woofer, in a ported cabinet tuned to 35Hz, and a crossover around 2kHz. I tested with 5 seconds of pink noise, with no filter, so the test includes spectral content right down towards 0Hz.

The speaker was tested again with a sock stuffed in the port. This reduced the driver excursion under the same test, and surprisingly enough the FSAF result showed a reduction in total distortion through the entire operating range of the woofer.

The next test things get interesting. I removed the sock, and added a LR2 high pass at 20Hz. The result of this FSAF measurement confuses me, as the total distortion is reduced through the entire spectrum of the speaker, right up to 20kHz, and significantly so as well. Is there a reasonable explanation for such a change in TD throughout with only a 20Hz high pass applied?





I realize this is a "synthetic" test, and real music has little content below 30Hz, but still I find the result confusing.