Untick the box entitled “using HDMI or optical audio headset”. Then in the HDMI drop down box (or optical, which ever you are using) under “speaker audio”, select bitstream out. Beneath that select Dolby digital as bitstream format.
This sets your primary output config for games and streaming apps (Bluray app has its own settings though) and determines what form your Atmos for headphone will take (Dolby digital or PCM). Now re-tick the “using hdmi or optical audio headset” box to apply Atmos for headphone. - Whatever your last set primary output config was is what Atmos for headphone is applied to when you re-tick the Atmos box.
However it should be noted that for Xbox at least, there isn’t much point in feeding the Yarra Atmos for headphones unless applied to actual discrete speaker configuration Atmos content such as a Bluray track or Netflix title or at least one of the rare games that actually has a discrete Atmos soundtrack intended for speakers.
For standard non-Atmos 5.1 and 7.1 content, better just to bitstream Dolby digital as is with no Atmos for headphone application.
Same for *pre-mixed* Atmos headphone content such as a mix built into a game’s audio settings (e.g. Overwatch) or pre-binauralised audio files or Streaming content (for example, Virtual Barber’s Shop or any of the Dolby surround demos on Youtube), best to have the Xbox transcode to vanilla Dolby Digital, forgo setting Atmos for headphone, and just let the Yarra take it from there.
In effect you are double binauralising by applying Yarra’s surround mode to already binaural content, but that is necessary in this instance, it’s just how the Yarra reproduces surround in that context. However, if you then proceed to apply Atmos for headphone on top of the same content then you will be double binauralising already binauralised content - in effect, triple binauralisation (once at mixing / recording as the track is already binaural, a second time by applying Atmos for headphone, and then a third time by engaging Yarra’s surround mode), which will compromise accuracy and quality so best avoided.
Re audio quality, if you are not using the Yarra’s surround function on binaural content or are just applying the surround mode to vanilla non-binaural stereo then PCM is better. But if you are using the surround mode to upmix a pre-binauralised 2 channel stereo feed, in my experience it is better to send it to the Yarra as Dolby Digital rather than PCM in terms of audio presentation. See my earlier posts for more details.
I don’t have Stefan’s settings to hand. I suggest you search post by his user name (Stefanthebass) on here and on the Avforums and AVSForum thread.