Thread by @LadieAuPair

@LesterCovax @brycej @LittleRabbtPaws @Twitch @awscloud @WatchMixer @Azure @ianhamilton_ 1/ Alright - to address some some of your questions/thoughts.

So separating audio, I can meant from the video itself. We already support separate audio channels (which is how we do multiple languages for our big shows) but ...

@LesterCovax @brycej @LittleRabbtPaws @Twitch @awscloud @WatchMixer @Azure @ianhamilton_ 2/ For streamers using Xsplit, OBS or Native Broadcasting on the Xbox, although the sources are individual on the streamer side, when they make it to us, it's all collapsed together, making it all jumbled. They aren't sending their voice along like we do for big shows.

@LesterCovax @brycej @LittleRabbtPaws @Twitch @awscloud @WatchMixer @Azure @ianhamilton_ 3/ This is why when people use an automated system right now, it would be more accurate than us trying to make captions from the whole video.

It would take work for streaming software & Xbox to split it before it comes to us, which is doable, just takes time &making it priority

@LesterCovax @brycej @LittleRabbtPaws @Twitch @awscloud @WatchMixer @Azure @ianhamilton_ 4/ In regards to limiting to just the speaker, it's what I would like to focus on based on feedback I've heard from deaf/hard of hearing viewers, because the game itself already has captions that can be turned on, and having both can sometimes drown out the streamer

@LesterCovax @brycej @LittleRabbtPaws @Twitch @awscloud @WatchMixer @Azure @ianhamilton_ 5/ Also terms of the accuracy of Azure cognitive services, limiting it down to the one voice I want to ensure gets captioned is vital. In Mixer, we would end up using the same model that is already on the Xbox, and I don't know if you've ever tried using it... but it's not great.

@LesterCovax @brycej @LittleRabbtPaws @Twitch @awscloud @WatchMixer @Azure @ianhamilton_ 6/ There are ways to get it more accurate for a particular person, and I've done some side projects with to make it work better for me, but there isn't a user interface for gamers/streamers themselves to be able to do it, and for some people, it just can't understand them

@LesterCovax @brycej @LittleRabbtPaws @Twitch @awscloud @WatchMixer @Azure @ianhamilton_ 7/ Moving onto costs, it basically comes down Mixer isn't currently using the Azure Cognitive Services, and to do so, it would cost money. We don't get it for free because we're part of Microsoft, there is still a cross charge. (This is the norm for big business)

@LesterCovax @brycej @LittleRabbtPaws @Twitch @awscloud @WatchMixer @Azure @ianhamilton_ 8/ If we wanted to have cognitive services running a separate instant for each channel live at any time, that costs money.

That's the cost I don't want to pass on. Right now people have to set it up, which cost time, but it doesn't cost them actual dollars.

@LesterCovax @brycej @LittleRabbtPaws @Twitch @awscloud @WatchMixer @Azure @ianhamilton_ 9/ I don't have answers on how to solve this yet, but I'm working on it. But putting an accessibility feature behind a paywall really doesn't sit right with me, and I really, really, really, don't want to do that unless there is clearly no other option.

@LesterCovax @brycej @LittleRabbtPaws @Twitch @awscloud @WatchMixer @Azure @ianhamilton_ 10/ I don't want to contribute to the disability tax.

But we agree on so much! I think that it totally makes sense for Mixer to do it. We resources available to help, it would be something to totally show off, etc etc.

@LesterCovax @brycej @LittleRabbtPaws @Twitch @awscloud @WatchMixer @Azure @ianhamilton_ 11/ BUT to be honest, our standard captioning and captioning ingest needs improvement first (that's a whole other story I'm not going into).

So that would be the next area of captioning we worked on (although I don't have a timeline available)

@LesterCovax @brycej @LittleRabbtPaws @Twitch @awscloud @WatchMixer @Azure @ianhamilton_ 12/ I hope this added come clarity and answered and questions you had!

Again, I would love to have automated captions as you described, and I think it's a great idea, we've just got stuff to sort out before we can get that far.