I didn't see mention what your output gain for various outputs are set at. Shown as +0dB below:

Everything is at 0dB, USB Volume is 127. Direct Monitor is on and it's set to the default 44.1KHz

I still have to wrap my head around the internal gain staging.

For example, I typically set the main outputs to -6dB because they go directly to my BM15a speakers and are otherwise was too hot even with the speakers set to the lowest amplification. But, I wonder if that means I have to also make sure the mix on the USB output is in line with what I hear from the main outs going to my monitors... not to mention if I want to use the internal recorder.

I also get the impression there's some digital limiting/compression/saturation going on before the analog output stage. You can abuse the master effect and eq quite brutally before causing audible clipping.

Question: so with that in mind, where exactly does the recorded mixdown come from? Pre- or post-master output offset?

There's not a "schematic" that shows where in the signal chain the Performance Recorder taps in. But you can experimentally determine if the output boost/cut is pre-recorder by setting the values to extremes (maximum cut -XXdB, maximum bost +XXdB). Use a Performance that has a fixed velocity like "Init Normal (AWM2)" which is a piano sound with no velocity sensitivity. You can play/record the same note with the two extremes on output settings and see what your audio software (Wavelab) shows in the output levels.

I wasn't sure what your settings were when I asked - but I was thinking IF the recorder takes the signal after the Main L&R gain boost/cut and the USB is after the USB output boost/cut - then a difference in these settings could explain results where the Perf. Recorder is a different level than the USB.

Also, analysis was done with Wavelab - but what recorded the USB Main L&R stream? If a DAW - then DAWs have gain staging themselves and you have to unity gain all of those various levels and ensure no other settings are affecting the recorded volume. For now, my goal is to ensure that the test setup (Montage settings, computer settings, software settings) is not introducing error.

The control you find on the back of powered studio monitors is “input sensitivity”. Usually a potentiometer that they want you to “set and forget” it is not to be used as a subjective Volume Control, you are supposed to use it to match the level arriving at the Monitor’s Input. Minimum setting is typically +4dBm (signals of this strength need less amplification), maximum is -10dBm (weaker products require more amplification). It looks to work backward... but the markings are to match the device Output with the Monitor’s Input.

You would only turn the Input Sensitivity up all the way to match a consumer level output -10dBm... it would require the most help from the amplifier.
Your Montage is Line Level by default. If your Main L&R are Set to +0dB Output, Set the Input Sensitivity on your Monitors low (counterclockwise)... start at +4dBm and turn clockwise to taste, if you like things louder than the typical 85dB studio norm...

Depending on how you are Monitoring will determine what you get. The MONTAGE sends audio directly to the Main L&R Outputs when “Direct Monitor” is On. When working with DAW software you may at times opt to monitor through the DAW. In such a scenario you would defeat the Direct Monitor pathway, and use the MONTAGE as audio interface.

There are two pathways for audio to travel from the MONTAGE Tone Generator, Direct and when acting as Audio Interface (via USB through the computer and back). It is very important to defeat one or the other of the two Monitoring pathways (if you Monitor both, it will give you a ‘not true’ sound).

None of what he's measuring/evaluating is the monitor output. It's comparing two digital recordings. One is the internal Performance Recorder and the other is the the digital out (USB Main L&R).

I'm not sure monitor gain or the response of his monitor would be involved with either.

He's visually looking at the difference between two audio files. Neither of which is from the analog Main L&R out.

Oh, I’m not worried about .38dB or .39dB (ever). I’m not sure what is being measured — the output or the accuracy of the meter. That small an amount I’m sure falls within the +/- error area when measuring whatever. (The Analog Outputs were mentioned as leaving an impression of being different from the digital Output... absolutely true)

Anyone claim to be able to hear that small a difference? Do tell... less than two-fifths of a dB (like a feather adds to overall weight, but not so you’d notice). If you’re two-fifths of a dB from clipping, you’re too darn close!

I’m not concerned with using every available bit all the time... it is not how to mix music material. Use meters as a very general guide, not as a rule. Finding a difference of less than 2/5 of a dB should not concern you... I’m not saying no one can hear that, but I do believe the majority of humans walking around can’t (on their best day).

Bad Mister - I wasn't concerned about the volume difference in general, but rather that the internal recording was distorted whereas the USB wasn't. That's all

Good news! I found the culprit:

In Windows 10 settings, under "Input - Line (MONTAGE)", it was set to 97 instead of 100. I don't recall ever going into that menu, but I've had my montage for 3 years so I can't guarantee it 😉

@ Jason: Wavelab recorded the main USB out. There are no input level controls.

Re. monitor settings - here's my signal flow as I mentioned in a different thread here:

The Dynaudio BM15a only have a -10/+4dB setting. They're on +4dB and the Main L&R is set to -6db. FWIW this gives me 1/3 of the master volume slider before it gets way too loud (I didn't measure the SPL).

For now, my goal is to ensure that the test setup (Montage settings, computer settings, software settings) is not introducing error.

Looks like "computer settings" (i.e. OS) got fixed and now you have parity. Glad that setup issue got sorted.