X7B is fairly well documented by the data list because it is a bulk dump with a header/footer that's not hard to figure out. But the full format is not documented.

X3A is not documented by Yamaha. I have not seen any formal documentation for the binary (non-bulk) formats of Yamaha keyboards.

I've advocated for a more open platform with more enablement for community involvement in development. This creates interest and a better ecosystem. However, Yamaha has never been particularly "open" as a company -- so this would represent quite a change in direction.

So I am guessing John Melas came to understand the formats by taking default dumps of them and comparing them to data sheets and other resources, and his tool(s) work simply by taking a template default file and patching specific bits in them that are always at fixed offsets.
Since they are both (?) just memory dumps, you can easily understand any bits/bytes from them by changing them on the system, making another dump, and comparing them.

That’s simple enough, but tedious. And I am not sure how that works with waveforms since they necessarily will be variable lengths.
But if the final answer is, “Hack it yourself,” then I guess that is what I have to do.

L. Spiro

Has either one of you asked? (I’m going to go out on a limb and say, no, you probably haven’t). Coming up soon, right here on YamahaSynth, will be an interview with Mr. Melas, and hopefully he will tell just how he got started and his relationship with Yamaha. But until then, feel free to speculate wildly about whether such support is available to those who ask. (Although most info needed is available in the Data List Booklets)

Please don’t make the request here, that’s not how you do it. (Like asking the ticket taker at Disneyland about getting coding info on the animatronics- you’d simply be asking in the wrong place). Thanks. Start by contacting your regional Yamaha Distributor...

over the years I’ve been with Yamaha, the company has worked with third party companies to help develop useful tools for the Yamaha hardware. Make no mistake, the company is all about the hardware.

Back when XGworks and SQ01v2 (Yamaha DAWs) walked the earth... Yamaha pioneered integrating external hardware with computer-based systems using STUDIO CONNECTIONS... basically, (a third generation of the original OPS systems), a way to manage all your external hardware settings and setups by establishing a network communication system via MIDI with the computer, that could capture-when-storing and restore-when-reopening your DAW Project. Some of these concepts were implemented in Cubase after Yamaha purchased Steinberg back in late 2004.

VST (Virtual Studio Technology) was a brilliant Steinberg concept to include external effects and external Synthesizers into the DAW environment, overcoming delay problems with new routing schemes. (“External” included anything external to the DAW itself, software or hardware). Being both Windows and Mac compatible this was ideal (from Yamaha’s point of view)... it should not be surprising that some info is not just posted on a website for anyone to download... not my thing but that doesn’t seem like a smart thing to do business wise (but then again I’m just a musician). Again if you have questions on how to get more info reach out to your regional Yamaha office, they can tell you what you must do and/or who to contact. (That goes for improvements to Waverunners and motorcycles, as well.)

Hope that helps.

Hi! Thanks so much for everything that you guys do! I love my Montage 8!!! It's truly a gorgeous and powerful beast!! I am looking forward to hearing from Mr. John Melas!

To the Honorable: **Bad Mister** : QUESTION: The Montage is now fully compatible with Motif XF/XS/MOXF as of V2. Is it known or can it perhaps be hinted at...if John Melas will reveal or sneak peak/hint at his planned future expansions for his fairly new Montage Tools Suite? Respectfully, There are a few functionality (mainly sample support, total importing of, creation of, manipulation and conversion of sampled sound instruments file data) holes that are in all reality still not all all together there and no real user friendly/sit down ease of use intuitive solutions have stepped up to the batting plate as of yet. Specifically, will there (I humbly ask with high hopes along with many other Montage Owners) be an addition to/porting over of a "Montage Waveform Editor" that's on par with or very similar in feature capacity with the tools John Melas already made available for his full featured Motif XF/XS Tools Suite in order to to bring more completion of sampled sound support features such as in that previously mentioned Suite to the new Montage Tools suite?

http://www.jmelas.gr/motif/wave.php

This is honestly and I say this with the deepest respect and admiration.. after speaking to many Montage Owners a very specific area that is still lacking the direct attention and existence of a user friendly total software support tool and or possible future onboard Montage OS Firmware implementation. This can be a little frustrating. l know it could be due to the differences in the design architectures of the Motif XF and the Montage platforms. Honestly, A lot of people though, truly need something that's very clear, and much less convoluted with an intuitive and user friendly interface to provide direct support relief for both Motif XF Owners with sampler content they wish to move or port to native Montage format and current newer Montage Owners with sampler content they wish to import/convert from native Motif XF to native Montage...perhaps even from other sources or sampled instrument platforms. This would immediately help and enable both Developers and Motif/Montage Owners to convert, import, create, custom edit, manipulate all parameters, effects assign, sound category assign, create custom User Banks and custom User Libraries with sampled sounds, arrange and organize them. Software features such as: key mapping/ spreading-stretching/assigning of multi-sampled files or even WAV files across the keyboard and then export and save the now fully mastered sampled instrument to be exported and imported into custom X7L Montage Libraries/User Banks. What say you fine sir?? Do you think John Melas might elaborate on this further and his development plans??

Again...THANKS A MILLION FOR FOR EVERYTHING YOU GUYS DO!!

BM,

I did submit a request on 9/21/2016. I can try again and see if the support model has changed.

I’m working on a new tool right now, following my question in this topic.
I’ve figured out much of the waveform format and am currently writing the first tool: an updated waveform editor.

Although I am focusing on my own needs first, I am being sure to write it in a generic way so that it could later be useful for anyone. For example, even though my pipeline produces only one type of WAV file for the samples, my WAV loader/parser is entirely generic and loads all WAV files. Since I had such a problem earlier, I will be adding some features to automatically remove 0-size loop points or loop points so early in the file that the sample data is too short.

For what it is worth, I was previously a senior graphics programmer at Square Enix (on Final Fantasy XV) and am currently a GPU performance engineer at Apple Inc., so it’s fun to note that likely the majority of you have used or are using my code right now (if you have played any of my games or are using any Apple device at all), but also important to note that this means Apple Inc. will technically own my code until I have a few meetings and cut through some red tape.

At this point it is too early to discuss a release schedule on my side, so I encourage you to keep your eyes peeled or even start your own project if you are able, but I still wanted to at least mention that at least one of us out here is working on something. At least.

L. Spiro

Shawn wrote:

For what it is worth, I was previously a senior graphics programmer at Square Enix (on Final Fantasy XV) and am currently a GPU performance engineer at Apple Inc. . . .

Wow! Very cool!

@Shawn - A few notes:

1) Technically, the attached WAV did not contain a zero-length ("0-size") loop. Small, but not zero.
2) The problem experienced with WAV import isn't necessarily "small" loops - but infinite loops with loop endpoints near the sample data start - which truncates the WAV sample data. In other words - the same "sample too short" error would likely not occur with a "small" infinite loop located later in the data stream (past Yamaha's threshold for such an error).
3) For a WAV editor - some may want "small" loops (for whatever reason) - so it would be best to warn and flag with an option to delete (or not). Software that thinks it knows better often does not (know better).

Shawn wrote:

All humans are born with knowledge of the John Melas waveform tools, tools for MOTIF, and more recently the tools for MONTAGE.

What no humans know is where he got the specs for all the file formats his tools generate. X3A and X7B are examples.
How does John Melas know the file formats for these? Are they publicly documented?

Where can I get information on these formats and waveform formats so I can make my own tools just as John Melas has?

L. Spiro

The Yamaha YSFC structure is available from several sources.
IMO it is likely undocumented and unsupported.
Start Here!
I can't image John being to keen to admit reverse engineering or any other of his efforts.
Yamaha give's you tools that work and no doubt reserves the right to change the specification with out notice.
John no doubt does the same.
In any event you can spin up python and play with the source on GitHub.

Cheers.

Reverse engineering gets you pretty far. The general skeleton of YSFC stays consistent. Poking at the instrument and seeing what changes gives you the answers since you know what you changed and the data is hardly obscured. You'll run into the same brick walls as JM such as WXC decode, etc - but the rest unfolds fairly well.

An "open" approach would have format information publicly online, possibly an SDK. I'm not necessarily impeded by the current approach (closed) - but there's a different kind of grass-roots engagement that happens when information is more readily available.

Although I advocate for this, I understand the other side.

Follow-up.

I asked on 9/21/2016 and re-asked a few times since. Recently I re-asked and received a response that this information is not available.

Has either one of you asked? (I’m going to go out on a limb and say, no, you probably haven’t). Coming up soon, right here on YamahaSynth, will be an interview with Mr. Melas, and hopefully he will tell just how he got started and his relationship with Yamaha. But until then, feel free to speculate wildly about whether such support is available to those who ask. (Although most info needed is available in the Data List Booklets)

As wild as the speculation may have been, it was correct.

Yamaha, as assumed, is a closed system without support for 3rd party development. Reverse engineering is, however, encouraged as Yamaha supports products which are fruits of reverse-engineering. Really, it's the only way to develop tools that read/write file formats unless someone slips a note under your door with additional information. Perhaps develop something popular enough and some (unofficial channel) hints will be provided.

Sorry for a late follow-up, but I'm the chap who spent a couple of afternoons playing with a Motif XF to write that Github gist. I'm really pleased it's been useful for someone other than me. If I'd thought anyone other than me would read it, I'd have made more effort to fill out some more detail. If I get a chance some time, I'll do a Montage version to reflect changes in the last six years.

If you dig into Montage files yourself, the content of most blocks should be obvious. Everything's still identified by sequential integers rather than being content-addressed, hence duplication issues, needing to keep references from one block matching the ids in another, etc.

Arps are probably the hardest thing to generate yourself. It might help to look at all the factory arps, which are stored unpacked but in the same format as the data block of a save file. You can find them in /usr/share/yamaha/contents/arpeggio if you arrange yourself a shell prompt on the Montage by one means or another.

Sorry for a late follow-up, but I'm the chap who spent a couple of afternoons playing with a Motif XF to write that Github gist.

Thank you so much! You saved me a lot of work.

You can find them in /usr/share/yamaha/contents/arpeggio if you arrange yourself a shell prompt on the Montage by one means or another.

Intriguing! How do you do that?