Thread by @kentbye

1/ THREAD
Facebook Connect 7 is under way, & I'm covering the news here in this thread.

Oculus Quest 2 already leaked, Zuckerberg is talking a bit about their augmented reality glasses plans.

I have many open privacy questions.

Livestream here:
facebook.com/facebookrealit…

2/ Facebook plans on releasing their first AR glasses next year. They didn't show any actual product, but they did show a number of pretty bulky prototypes in a promo video in their collaboration with RayBan

See episode #942 with @alexeheath for more.

https://twitter.com/kentbye/status/1305935725913714688

3/ The Oculus Quest 2 (64GB) will be sold for $299, and the 256GB for $399. It has 3 IPD adjustments and a single LCD screen at 1832x1920 per eye - just shy of 2K per eye.
Oculus Move is a built-in fitness tracker included as well.

4/ @boztank shows a research project Aria that explores many open questions with the AR glasses prototype. They need to test it in real-world environments. Look at how to leverage 3D spatial internet.

5/ I got an early look at the Oculus Quest 2 & invited @RtoVR's @benz145 to share his impressions along my own feedback.
I also have a lot of unanswered privacy questions as we won't get to see the new Oculus Supplemental Data Policy until October 11th.
voicesofvr.com/943-oculus-que…

6/ @boztank talking about some of the ethical considerations of their AR glasses prototype Project Aria:
about.fb.com/realitylabs/pr…

They have an RFP for inclusive AR & unintended consequences:
research.fb.com/research-award…

Privacy experts say talk is cheap, what is behavior? (Agreed)[

7/ Like @boztank said talk on privacy is cheap & it's worth looking at their behaviors.
I've been trying to look at & review the new Oculus Supplemental Data Policy for Quest 2, but Facebook isn't making it available for anyone to see until October 11th.

https://twitter.com/kentbye/status/1305563085680197632

8/ The New York Times now has an AR Lab to experiment with telling stories with AR delivered on Instagram.
See previous experiments from NYT here:
rd.nytimes.com

9/ Facebook is announcing a few AR titles that will be launching on Facebook Portal, and more info later in a Spark AR session.

10/ Over 35 Quest titles have generated more than a million titles with Onward selling more than a million dollars worth within 4 days.

11/ The embargo for all types of news lifted at 10:30a & so I'm going through @RtoVR & @UploadVR.
One piece of news is the Rift product line is being discontinued Spring of 2021.
Oculus Quest + Link (and eventually wireless streaming) will be PCVR
roadtovr.com/facebook-disco…

12/ Just announced that @BTSW_official is releasing a music pack for @BeatSaber with custom characters. I'm not sure that that actually means in the game, but I know there's lots of big fans of BTS out there.

13/ Jurassic World Aftermath uses an interesting toon-shaded aesthetic. Developed by @Coatsink

14/ Myst is being remade for VR.
Here's the trailer via @rtovr

15/ Interesting to hear Medal of Honor will be launching on Steam on December 11th as well on Oculus. It's PCVR, and so it'll be available via Oculus Link.

16/ Talking about the future of work, Facebook Workplace integrations, and "Infinite Office" announced to use the passthrough modes, Oculus browser, attach virtual panes to VR, logitech physical keyboard that you can see via passthrough. (Passthrough demo is in Spatial app)

17/ Chief Scientist Michael Abrash now giving his annual update talking about some of the latest research & insights into VR, AR, personalized AI, and the ideal AR interface (putting us at the center of technology) to get to that Engelbart Mother of All Demos moment.

18/ Abrash will talk about: Input/Output, Machine Perception, & Interaction.
Electromyography (EMG) uses muscles as a neural interface to XR. I did an interview with @CTRLlabsCo a few months before they were acquired by Facebook
voicesofvr.com/814-neuroscien…

19/ WOW! Abrash talking about training yourself to use a sixth finger in VR using a neural interface driven by EMG. Take off the VR headset, and then use your imaginal sixth finger detected by EMG device to click & interact in virtual spaces.

20/ AR glasses will enable sensory superpowers, specifically audio to do gaze-based beam forming to tune out ambient noise, personalized HRTFs, audio propagation simulation to simulate reflections more accurately.

21/ Not sure what I think about the implications of gaze-based audio beam-forming in IRL environments. Only being able to hear who you're looking at. I'm sure there's accessibility applications for those with hearing impairment but their demo feels like some new social dynamics.

22/ Now talking about Live Maps as a way to understand your spatial context without having to spatially scan your world. They're selling this as a way to save battery in AR Glasses, but there's also significant privacy implications of having a 1:1 scan of the world + content.

23/ Abrash using philosophy term of ontology to refer to the relational property of entities. Talking about the need to collect "Egocentric Data" to train AI assistant to be aware of your current context. Project Aria is their research project to start collecting training data.

24/🚨🚨🚨Facebook going all in with recording all the data with Project Aria research glasses. They say that they have "privacy controls" to blur faces & license plates, but I wonder how they're dealing with the third-party doctrine & a "reasonable expectation of privacy" here.

25/ Project Aria is a research project to capture training data for personalize AI. It's captures all of your local spatial context, your eye tracking data, & audio in your environment. There's MANY PRIVACY concerns here. How is not a Facebook surveillance device you wear IRL?

26/ So the new buzzwords release today are "Egocentric Data" and "Contextually-Aware AI." Capturing first-person perspective data in order to train personalized AI assistants to become aware of your local context.
I have MANY questions here.
What's the cost to our privacy here?

27/ The archive of the 2-hour Facebook Connect Keynote can be seen here:
facebook.com/facebookrealit…

28/ @boztank says "We built Project Aria with privacy in mind & we’ve put provisions in place around where & how we’ll collect data as well as how it will be processed, used, & stored."
Their privacy protocol was developed by @elidourado & @castrotech:

https://twitter.com/boztank/status/1306295638645026816

29/ I'm also getting a 404 error for this "EU/EEA/UK Consumer Right of Withdrawal Information" link:
oculus.com/legal/eu-consu…

I'm not sure what this is supposed to be & whether it's related to the GDPR "coupling ban" or not.
More:
arstechnica.com/gaming/2020/09…

https://twitter.com/Zimtok5/status/1306306789374337026

30/ Looks like the Right to Withdrawal has to do more to do purchasing goods online. More.

https://twitter.com/Zimtok5/status/1306312260881653762

31/ Facebook Connect is single-track this year with 9 sessions on a single livestream.
facebook.com/11355664044980…
@ID_AA_Carmack keynote begins at 5:30p.
Carmack could talk for 3 hours, but they'll likely cut him off in order for people to watch a Jaden Smith performance at 7p. ☹️

32/ At the #AppleEvent yesterday, Apple didn't announce any of their AR plans yet, but they may say more at another iPhone event here soon.

More context on Facebook vs Apple from @alexeheath:
voicesofvr.com/942-the-facebo…

My #AppleEvent thread from yesterday:

https://twitter.com/kentbye/status/1305915592361127937

33/ Oculus Venus is a ghost town. For a company focused on VR, Facebook did a *terrible* job leveraging VR in any meaningful way to gather for a virtual conference experience.

6 rooms:
1 livestream watch party feed
3 non-Connect events
1 Carmack talk 5:30p
1 Smith concert at 7p

34/ I watched the Keynote on my PC & folks are reporting that more people in Venues then.
Currently Venues has an instance limit of 8 people.
Makes it hard to feel like a conf gathering.
VoiP breaks down on Quest with more people in both VRChat & AltSpace.

https://twitter.com/hc_virtuoso/status/1306341302057840640

35/ @protocol's @jank0 talked with Facebook's @boztank, @schrep, & Jenny Hall about FRL's "responsible innovation principles" & "how to take values like privacy, inclusivity & accessibility into account while building Facebook's future AR & VR products."
protocol.com/facebook-ar-vr…

36/ Facebook Reality Lab's has a "set of principles for Responsible Innovation" that includes:

#1 Never surprise people
#2 Provide controls that matter
#3 Consider everyone
#4 Put people first

NOTE: They "won’t offer controls for everything" re: privacy.
about.fb.com/realitylabs/re…

37/ #4. Put people first...

https://twitter.com/__Ka1/status/1306287444984082432?s=19

38/ I'm going back over the keynote to pick up things I missed watching it real-time with no way to rewind.

@MegFitz says Oculus Avatars are introduced & "new Oculus Avatars will be rolling out soon - the Horizon & Venues avatars are an early iteration."

https://twitter.com/MegFitz/status/1306307615891152897

39/ Announced today at Facebook Connect is the Fit Pack, Elite Strap, and Elite Strap with Battery for the Oculus Quest 2 (didn't get to try these)
The default soft strap for Quest 2 is not really that substantial, & a degradation back to how the Oculus Go used a flexible strap.

40/ Zuckerberg said Facebook is collaborating with @Luxottica & have plans in 2021 to release some "smart glasses, they're not yet augmented reality glasses, but they're on the road there."

Here's a link to the 51-sec "The future will be a classic" video:
facebook.com/facebookrealit…

41/ @NiclasJ passed along this forum post from the Oculus Forums from December 24, 2019 asking to remove the Mandatory Facebook login in order to make friends.
Facebook has definitely Crossed the Rubicon on this front.

https://twitter.com/NiclasJ/status/1306381462728577035

42/ @boztank said Facebook Portal has increased usage, they're opening up AR filters soon, & integrate with lots of teleconference apps.

Portal is subject to the Facebook Data Policy:
facebook.com/policy.php

AND the Supplemental Portal Data Policy
portal.facebook.com/legal/data-pol…

43/ @boztank listed a number of "Productivity & Collaboration apps on Quest," but only 4 of 12 of these have launched on the public store: Virtual Desktop, Spatial, Gravity Sketch, & vSpatial
Search for yourself:
oculus.com/experiences/qu…
Maybe available for enterprise, but odd.

44/ Here's the nice 1-minute ad for Facebook Reality Labs Research that played during the keynote.
I wish Facebook would make more of their FRL researchers available for more on-the-record interviews to share their research findings.

45/ Watching annual @ID_AA_Carmack keynote in Oculus Venues. There are a lot of SUPER ANNOYING sound effects like firework "woohoo" confetti, clicks, bops, & boops. I'm not convinced anyone from Oculus has actually tried to watch anything in here otherwise there'd be global mute.

46/ @ID_AA_Carmack was ready to talk for 2+ more hours, but Facebook cut him off after 82 minutes. They're having him go into private Horizons beta room that can only hold 20 people(!) Hopefully someone records it since he had *a lot* more to say.
FB is YEARS behind in social VR.

47/ The Jaden Smith concert they cut @ID_AA_Carmack off for (which is in a totally separate virtual space) is a pre-recorded, normal 2D recording of Smith composited onto a static & monoscopic 180-degree video playing in the Oculus Venues space.
Very underwhelming & no where live

48/ There was a lot of speculative XR design that was showed throughout Facebook Connect. Here's the 95-second video of "Infinite Office" showing lots of speculative design on the future of work & productivity using passtthrough, virtual windows, keyboard.

49/ More speculative design of AR glasses shown by Facebook Reality Labs today including navigation wayfinding, notifications, talking to hologram projections, translation overlays.

50/ More speculative AR design from Facebook Reality Labs with finding your keys (& annotating what they keys are for which seems like a security vulnerability), geo-located metadata, product recommendations, & accessibility features. This vision has so many privacy implications.

51/ All of the #FacebookConnect developer sessions are all contained in this single, 5-hour livestream video. They'll likely be broken out at some point, but you can reference the video below + schedule (where these sessions started at noon PDT).
facebook.com/facebookrealit…

52/ There's not an official copy of @ID_AA_Carmack's Unscripted Live online yet, but @UploadVR livestreamed it. Lots of candid insights about the engineering tradeoffs, successes, & open problems yet to be solved.
Hopefully the post-chat was also recorded.

53/ During @ID_AA_Carmack's talk he said it's pretty embarrassing that Facebook doesn't have a viable social VR platform launched in the middle of a global pandemic.
All previous efforts have been shuttered & what they have is in beta & needs lots of work.

https://twitter.com/kentbye/status/1306394212645007360

54/ A few folks recorded portions of @ID_AA_Carmack's talk in Facebook Horizon after his Unscripted talk.

via @InsomniaDoodles (h/t @Navahk):


via @gemisisDev:
facebook.com/gerald.mcalist…

55/ Zuckerberg told @CaseyNewton "I think the first thing is we should just talk about more of the [AR ethical] issues up front."
theverge.com/interface/2020…

These ethical conversations are happening behind closed doors without journalists covering them.👇

https://twitter.com/kentbye/status/1191786583529271299

56/ Does Facebook Reality Labs prioritize "what's best for the most people in *the* community" or "what's best for *our* community?"
Are bystanding "non-users" w/o AR/VR devices not their community?

Sources:
about.fb.com/realitylabs/re…
tech.fb.com/facebook-reali…
research.fb.com/programs/resea…

57/ Sep 16th announcement of Facebook Reality Lab Research winners & finalists of the "Explorations of Trust in AR, VR, & Smart Devices" RFP to "better understand & address these nascent risk areas in our efforts to build trustworthy [XR] products."
research.fb.com/blog/2020/09/a…

58/ Lots of Japanese VR dev engagement in #FacebookConnect this year.

Carmack talk by @AmadeusSVX

https://twitter.com/AmadeusSVX/status/1306391249109229568

Unity + Oculus Quest tips @korinVR

https://twitter.com/korinVR/status/1306304575087849472

"Big in Japan: Best Practices for Japanese Developers" via @kiyoshi_shin

https://twitter.com/kiyoshi_shin/status/1306361290818908160