The problem is most likely a ground issue. More information about the connected devices and your setup would be necessary for a definitive analysis and solution. But you can begin researching the topic by looking for USB ground noise.

https://www.soundonsound.com/sos/dec08/articles/qa1208_2.htm

Here is a related issue, the problem's cause can vary depending on where in the world you are located and electrical issues that maybe unique to your situation. It not typically any one piece of gear's fault - it is always due to interconnecting the gear. In general, when multiple devices are connected to the electric mains, differences can cause noise in the audio line... Particularly where computers are involved. The Ebtech Hum-X eliminator mentioned in the link does work wonders in my experience... Again your mileage can vary.

Yes, most likely ground loop. This won't happen with DIN MIDI because MIDI (thank goodness!) is required by the specification to be optically isolated.

You actually have three (or four, if you're running stereo) ground paths between your keyboard and your audio monitors: power, USB, and audio.

Furthermore, many computers have very sloppy USB connections, disregarding audio ground loop issues (which usually aren't an issue for the kinds of devices plugged into USB, like disk, keyboard, mouse, etc.) I've had a laptop where I could hear the disk drive being accessed thanks to noise on the USB connection. (Using the USB on the other side cured that particular problem.)

A simple potential solution in your case would be an "audio isolation" box, like an ART DTI (this is a 2-channel device, in case you're running stereo.) It transformer-isolates your audio. That removes one or two paths, but isn't guaranteed to solve the problem which might still be present thanks to USB and the power connection.

Also, make sure all your gear is plugged into the same power strip. That solves some cases.

Thanks folks. Appreciate the suggestions.