There are two sides to this - you have to get the "dry" audio into Mainstage over USB then you need to route the audio that has added effects back to speakers (presumably back to MODX again).

I see more potential for issues with the second part of this - but you appear to be having problems with the first step.

I'm assuming if you have the A/D output set to Main L&R that you will hear the analog synth. Is this correct?

If not, check your A/D input button and A/D gain.

I do not have Mainstage here to duplicate but can setup the same basic scenario under Cubase and see if I can get a keyboard connected with 1/4 inch TS cables to the A/D inputs and its audio routed to the DAW (using USB7&8) with a signal.

I was able to get this to work. I got lazy and just connected the output of my keyboard (left channel) to the A/D input left. Pressed the [A/D INPUT] to light it up. Turned the A/D gain knob to get enough signal and not to create feedback. Used "Init Normal (FM-X)" since I already had it up and changed the Common/Audio section "Audio In" -> "Mixing" screen so A/D In was routed out USB 7&8 (this tells the analog signal coming in from the A/D L/R jacks on the back to route to USB 7&8) and also so Digital In was routed to MainL&R. This Digital In section there's a volume just for it which is your "return" signal. I used this later to turn up/down the output of the return (with effect) signal out the speakers to prove this was working -- and it was.

On Cubase I opened an empty project and added an Audio Track. Set the input to be USB7&8. I could see the meters were receiving audio from the keyboard through USB -- which already seems to be further than you were able to get.

Then I routed the output of this audio track to Digital L&R. This allows for me to route the return signal which I already had ensured was "Main L&R" on the MODX side. Also in Cubase I added an insertion effect that had some tube overdrive so I could hear the effect.

All of this worked - if I turned the "Digital In" volume down to 0 I could not hear the A/D inputs. If I turned it up, then I could start to hear the Cubase-added effects coming back in and flowing out to my studio monitors.

Yes, if I have the A/D output set to Main L&R I will hear the analog synth.

I tried this in Ableton as well, and can see the audio level meters indicating that Ableton is receiving signal - but only when I set the input channels to 9&10 (I have set the A/D routed to channels 7&8 😮 )

Still can't hear the audio signal with the effect applied from the MODX L&R outputs, though - need to dig in deeper.

But at least I know it should be possible, thanks!

Ableton may count Main L&R as 1/2 and then start numbering the rest of the USB channels starting from 3/4 (which are really USB1&2 on MODX's side).

This video shows this: https://youtu.be/11ccxrU2QzY?t=1202 where it shows Stereo Inputs 1/2 = Main L&R and Stereo Inputs 3/4 = USB 1&2 from the keyboard and so on. Specifically, it's this time: https://youtu.be/11ccxrU2QzY?t=1359 that shows the screen.