Hello World Building Augmented Reality for Snapchat

Fun fact – 30000+ lenses created by Snapchatters, leading to over a billion views of lense content
Table of Contents
Lens Studio
Hello World! Lens Studio Live Demo
High-Quality Rendering with Allegorithmic
Chester Fetch with Klei Entertainment
Cuphead and Mugman with Studio MDHR

Travis Chen

• Worked at Bad Robot, Neversoft, and Blizzard

Lens Studio

• Snapchat has always opened up to the camera, which has positively affected their engagement
• Pair your phone with Lens Studio
• A community forum on the site exists where devs q & a
• Has been out for less than 4 months today, and the lenses have resulted in over a billion experiences
• The tool has been used for a variety of things hamburger photogrammetry, full-screen 2d experiences,
  r/snaplenses

• Distributing your lenses is really easy

• Within, snap you can discover a lense where you can see more lenses by the same creators or you
  can pull up on the base of a story to figure out what lense was used.

Lense Boost – All users see the Snapchat carousel, a Lens Boost to get your lense into this carousel

Find which template best fits your creative intent

Templates

• Static object
• Animated object
• Interactive templates (tap, approach, look at)
• Immersive (look around, window)
• For 2D creators (cutout, picture frame, fullscreen, soundboard)
• Interactive path (idle, walk, and arrival states necessary) coming soon

Examples

• Brian Garcia, Neon Book
• Pinot, 2D textures, cutout template, then character animator to animate
• DFace, DDog, imported into lenstudio (from camera reflections feature)
• Jordan & Snapchat, ‘88 static Jordan 3D model
• Netflix & Snapchat, Stranger Things – turning on the TV, or spelling your name out,
  awakiening the demi gorgon

Hello World

Lens Studio is made up of panels:

• Live Preview, to see what it will be like, it includes tracked content and interaction support
• Objects panel, like the Unity scene view, it shows you what is in the preview
• Resources panel, all your resources and where you’d import stuff

Workflow

Start with the animated object template
Select an object in the resources panel and move it to the objects panel
Google blocks + mixamo + export free animations from Adobe and import your character animated from
mixamo
File import monitor and astronaut
Child the imported 3D model to the fox as a child and delete the fox
Add a shadow
Sprite

High-Quality Rendering

Substance Painter is an app to apply materials or paint textures for 2D or 3D.
Any material you bring in you can apply to an object, they apply uniformly, but there’s
also, smart materials which applies intelligently to geometry (rust example).

The layers tab is like the scene view the place to drag and drop

Alphas provides a cutout, you can apply materials to the cutout

Upon clicking export

lensstudio
Challenge: Rubber Ducky

Chester Fetch with Klei Entertainment

Games studio since 2005

Why is AR interesting for Klei?

• AR is about bringing the virtual world out to the player.
• Shareable
• Limited bandwidth
• Seems hard
• Would require too much time from others at the Studio

Cuphead and Mugman with Studio MDHR

• Cuphead and Mugman wanted to build and snap a boss battle
• All of the lenses used in cuphead were from assets created directly from the game
• Chains together 5 2D animations

 

Questions

---

• Within the Snap app, I noticed you can rent/create a lense “as a service” how does this pertain to lens studio?
• A question I had was, looking forward to a day when you can use targets like people for further interactable and shareable content like the examples shown in Mugman, when will person/object recognition be available to developers and users of Snap?   
• What is the github account for Snap?