Amazon Sumerian - First impressions

For those who know me and for those who doesn't...as I work as a Developer Evangelist...my main job is to learn, explore and evangelize new technologies and programming languages...so of course...AR/VR had been on my plate for quite some time...

I have played with Unity3D and Unreal Engine...and of course I have developed for the Google Glass, Microsoft HoloLens and Oculus Rift...

When the good folks at Amazon announced Amazon Sumerian you can figure out that I completely thrilled -:D

So yesterday, I finally got accepted into the Beta program, so of course I started to follow a couple of tutorials and get to know the tool -;)

Please be advised that I'm starting...so I haven't tried or used everything...I want to go step by step following the tutorials and trying to understand everything in the most positive way...

Have I mentioned that Sumerian runs on your browser? How crazy is that? No installation...just launch up your browser and start building AR/VR experiences...

When you first launch it, you will be presented with the following screen...

Where you can create a new screen or simply use a template.

Sumerian provides many tutorials, and so far I have only made my way through the first 3...

So here's how my TV room looks like...

As you can see...Sumerian is a full blown editor that provides all the tools that you can find on any other editor...plus many things that I believe are brand new and exciting...

Of course, you can preview your work...

As for the TV Room tutorial...the idea is that below the TV Screen, there's an Amazon Echo, so you can press it to change the videos presented on the screen. For this you need to use a State Machine and also create a script that will manage the different videos. For the scripting you need to use NodeJS...which is really nice as is the language that I mainly use when developing application for Alexa...

This is how my TV Room looks like when playing a video on render mode -:)

Before moving on to learn more about Sumerian...I need to say that the navigation system doesn't seem to be too good by now...you can use the mouse buttons, Tab and Shift...but control keys or AWSD doesn't seem to work like you would expect on Unity3D or Unreal Engine...I have forwarded my question to the Sumerian Team on Slack...so I will update this post as soon as I get an answer :)

*UPDATE* By following the "Lights and Camera" tutorial I found out that while the default camera doesn't allow fine grain navigation...the FlyCam does it! -:D All good in the hood -;)

Till next time,

Blag.
Development Culture.

Blender Lego Art for HoloLens

This blog was originally posted on Blender Lego Art for HoloLens.

Who doesn’t love Lego? And if you have used Blender before…who doesn’t love Blender? -:)

Combining both seemed like a great idea, so that’s what I did…Using Blender I create some pretty simple Lego pieces that can be used to build both simple and complicated models.

A single piece is 0.25 by 0.25 and it’s made out of a single vertex. In the image, the colors are just used to give an idea of the different pieces.

The main point is simply to create a new Blender file, append the different pieces and start building.

At first it’s kind of complicate because you need to deal with the X, Y and Z positions…but once you get used to it…it’s becomes a little bit addictive -:)

Now, the name of the blog is Lego Art, right? So…if you look online for Pixel Art instead, you will find a lot of nice images…like this one…

The perfect candidate for Lego construction! By putting the pieces together and simply assign them the right material…we can get this…

And with some more time and dedication…we can get this…

Anyway…with a collection of models…we can think up and bundle them together into a Microsoft HoloLens application -;)

The application itself is easy…you start by looking at all the models on a shelve…you can select one, make it smaller, bigger, turn it left or right or simply go back to the shelve.

Here’s a video showing how it’s look like.

Greetings,

Blag.

Development Culture.

Room with a View – HoloLens, Unity3D and Blender

This post was originally posted on Room with a View – HoloLens, Unity3D and Blender.

Since the last month or so I have been enlighten myself with an awesome Blender course…and while I had some previous experience by reading some books…nothing could get me to the point that I’m at now as this course…so…enough said…it’s more than highly recommended 🙂

So…one of the challenges was to create something just using primitives…that means…cubes, spheres, cones and so on…nothing fancy…no modifiers…no extra knowledge…so I was able to come up with this…

That was a nice start…but I knew it needed something else…beside getting textures for it of course 🙂 It looked pretty plain…so next step was making it nicer…and this came along the way…

Now…it looks fancy, doesn’t it? Everything hand made created in Blender -;)

So…next step was to figure out what to do with this…it looks too nice to just leave it there on my hard drive…so I start thinking and then remembered that the Holographic Academy has a really nice demo called Origami…in this demo you need to select an origami ball that will drop down a paper plane and then hit a fan that will explode and in turn open a crack on the floor where a nice underground world can be seen…truly amazing if you ask me…so…I had an idea 🙂 Why not have a d-shop frame that will explode when selected and then open a hole in the wall where the room that I designed in Blender could be seen…that sound like an impressive demo to me…so that’s exactly what I did 😉

Here’s the video for your viewing pleasure…hope you like it 🙂

At first I though it was going to be a lot of work…but it wasn't really like that…here are some highlights…

* I enclosed the room in a black box with a hole so it be looked through…

* In Unity3D I simply used an Unlit shader…as black is processed as invisible on HoloLens, having the model in a black unlit box gives the same impression as being invisible…hence…it looks like a hole in the wall…

* I used some spatial mapping to be able to pin the frame and the room to the wall…so it didn’t float around but instead looked like the room was inside the wall…

* When I first imported the model from Blender into Unity3D…none of my textures were available…which seemed odd to me…as they were there and were actually assigned…it turned out that I need to choose every single piece of the model and do a smart unwrapping…so the shader knows exactly how to implement the texture…

* I first I tested out with the emulator…which is good…but doesn’t really implement the same measure units that the real device uses…so while it looked fine on the emulator…it looked too far away on the HoloLens…so a lot of testing was needed in order to get it into the right position…

* To shoot the video, at first I tried using Camtasia…but of course…the rendering wasn’t was I was expecting…one thing is to look it from the HoloLens and other thing is looking from the laptop…so instead I used a recording from the Live Streaming of the HoloLens itself…and that was the trick…

As you can imagine…this now brings a whole world of new possibilities and demos…

Now it’s your turn to show us…what can you do 😉

Greetings,

Blag.

Development Culture.

SAP d-shop’s Virtual House – A journey from Physical to Virtual

This post was originally posted on SAP d-shop’s Virtual House – A journey from Physical to Virtual.

Some time ago…our good friends from SAP d-shop Newtown Square (Namely John Astill et all) built a IoT House for SAP Insurance. This little house (hand made by the way) used an Arduino Nano, a bunch of sensors and LED lights…and…which is pretty cool by the way…a 3D Printed washing machine with a water sensor…and of course…it was and it is…IoT enabled.

We thought it was pretty cool…so we have one at our own SAP d-shop at Silicon Valley and it had become a key part in all our d-shop tours.

Then…some time later, our friends from HCP Marketing (Namely Joe Binkley et all) and Intel build a Smart Building. A really nice building…controlled by Amazon Alexa that used an Intel Galileo, some Arduinos as well as servos, lights, a solar panel and even a fan…everything again…IoT enabled…but also as you may have guessed…voice controlled…so you can send the elevator up and down…open or closed the doors and even send the whole building on emergency mode…gladly…we had keep it on the d-shop for quite some time and it’s another of our “wow” factor demos every time someone comes to visit…

Having these two available for us…slowly sparked the fire of innovation and creativity…why don’t we build a Virtual House that can be used on the Oculus Rift and it’s controlled by Alexa?

Not an easy thing…but we for sure love challenges…and thanks to our previous experience Unity3D and Alexa working together we already knew how to start…

The architecture is pretty simple… The Heroku server is just an echo server, so it will repeat everything we pass to it as a JSON response. Our Unity app is constantly checking the Heroku server to see if there’s a message to respond to. Of course, for this to work as intended, we need to setup a skill on Amazon Alexa just to update the server. So, when we say “open door”, then Alexa will send a command to the Heroku server and this server then will produce an “open door” message in JSON. Our Unity app will read the Heroku Server, and act accordingly by opening the door…of course, we don’t want this to happened all over…so after Unity executes the action it sent a null message to the Heroku server, so next time the JSON response is going to be null as well and Unity will simply wait for the next valid command.

If you want to take a sneak peak of how the Virtual House looks like…here are a couple of screenshots…but don’t forget to watch the video, J You will get the full experience -;)

Now…this project started as a “Project in a Box” (for internal only…sorry about that) which means…all the source code and explanations on how to build it from the scratch should be provided…but…for obvious reasons…that didn’t happened L So instead…we turned this into a “Product in a Box” meaning that (Sorry again…internals only) you can download the compiled application and simply edit the configuration file to have it running on your own J No source code is provided by obviously a nice email can get you that -;)

Grab it from here

Now…that I got your full attention…please watch the video J It’s a nice journey from the IoT House to the Virtual House passing by the Smart Building…

Now…you may wonder about the 3D Models used for this Virtual House…as you can see on one of the images, most of them were downloaded but some of them were developed in house J using Blender…Like the Amazon Echo, the 3D Printed Robot and name tags, the Amazon Echo and obviously the house itself J For some other things like the plants and tables…those were imported into Blender and “hand painted” as the textures were not available.

Now…something that we believe it’s pretty important…is to list all the Pain Points and lessons learned while developing this application…

Pain Points and lesson learned:

• As this is a Product in a Box and not a Project in a Box, we’re not going to include the source code for this application, but what we’re going to do instead is let you know the pain points and lessons learned that came from this project.
• Unity uses the .NET Framework 3.5, which is already deprecated by .NET 4.0 so many things are not going to work simply because they haven’t been implemented…and why is that? Well…Unity uses Mono (which is .NET for Linux) and I guess they do it to maintain uniformity in all platforms. While Mono remains on .NET 3.5, Unity will not likely upgrade either.
• When loading scenes, the lighting gets all messed up…so you start in level one…more to level two and suddenly it looks like nighttime…the solution to that is simple…choose “Window à Lighting à Lightmaps”, uncheck the “Auto” checkbox and press “Build” to bake the light again.
• Coroutines are simply awesome. Normally, you can’t make your application wait or sleep…but by using Coroutines you certainly can…Coroutines are like threads.
• When using a light, make sure it’s turn off while the character is not in the room, because this will save some graphic processing and because even virtually…we need to be environment aware…
• Unity doesn’t have a wrap function or property for 3D Text…which is kind of problematic especially if you want to do a Twitter Wall…so your only chance is to build you own…although that’s not that hard…simply grab the incoming text, split it by space into an array…concatenate each word by checking first if the length of the string is lower than our threshold (which should be the maximum number of characters that fit where our 3D text is), is the string is bigger than the threshold, we simply add a carriage return (“\n”) before doing the concatenation.
• As your application grows you might feel the need to duplicate some assets, which is perfectly fine and doesn’t add too much processing (Especially if you create a Prefab and use that prefab), but don’t forget to assign them unique names, otherwise you’re going to have a headache if you application needs to interact with those assets.
• Sometimes you will download some 3D models from the web…other times you will create them using Blender…but don’t forget that sometimes just a simple sphere, cube or any other Unity primitive can work just fine by just using an image attached to it as its texture.
• When creating your Alexa skill…make sure not to make any spelling mistake…otherwise you will hit you head thinking why Alexa is doing what you’re asking her to do…
• When testing our your application both Debug.log() and Print() will become your best friends…nothing better than a printed value or message to realize what going wrong.
• When moving an object, always make sure to record its original position and then add the new value to that recorded position. Otherwise, something might provoke the values to go wrong…by having the original values recorded, you avoid having to recalculate the position but just call that variable and get things where they belong.
• When using 3D Text you will notice that even if you put another object in front of it…it will be always visible…which is not very likely…so we have two options…either create a shader to occlude it…or the easiest one…make the material that it’s in front of it transparent. That’s not perfect for all situations but at least will work.
• The biggest problems when making Unity and Alexa speak, is that when you ask Alexa to turn on the lights, she will respond “The lights are on” …but then if you ask a second time her response should be “The lights are already on” …to make this…we should need to use a Database or something to store state information…and when closing the application, we would need to clean up the states…while this might be doable…it’s a lot of work, and what happens if the application crashes? Would we need to go and reset the states manually? Not ideal…
• That leads me to the point of using the elevator…you can open the doors or sent it to any of the floors…for the main part…that’s easy…each floor is a scene, so you need to be on the first floor in order to make the elevator to floor two or three…but…what if you’re outside the elevator? You are on floor one…ask for floor three…and then you open the door…as your characters moves along with the elevator floor…when you open the door everything will look bad…solution? Simply using a cube without a mesh renderer, so it’s invisible…assign a collider with “is trigger” enabled…and validate that the player is colliding with the cube in order to make the elevator move…that way, even you ask for floor three and Alexa confirms that the elevator is going up…nothing will happen…when you open the door…we can assume that the elevator went down or up to your floor in order for you to hop in…just an illusion…but it works…
• Alexa doesn’t have an option to delay the re-prompt, so when exploring the Virtual House she will ask you “What else can I do for you?” and if we don’t respond the skill will just die…so we will need to wake her up again…that’s kind of sad due to the nature of the application…but nothing to be done unless Amazon releases a way on making the re-prompt to wait longer…
• As the whole Alexa-Unity3D relays on Heroku…expect some downtimes or responses from Alexa that are not actually replicated in the virtual world…might be an internet connection glitch or just Heroku glitch…

As I mentioned first…the environment gets affected by the weather…if it’s sunny…you will see a sunny clear sky…if it’s rainy you will see a dark and gloomy sky…and this involves using a Skybox…although not your regular Skybox…and what is a Skybox, anyway? Well…simply put…is a cube that covers your whole environment and has different images to simulate the environment…the problems is that the regular Skybox only allows you to assign six sides…which is of course not likely…you need to use a twelve side Skybox…then you can assign sunny image and also cloudy images…that way when checking the weather you can modify the luminosity and that will also affect the Skybox as it will use one or the other giving that nice effect on reflecting the outside weather…

Greetings,

Blag.

Development Culture.

LED is my new Hello World - Prolog time

As promised...here's my LED Numbers app written in Prolog...it took me a long time...a lot of research...a lot of headaches...so I hope you like it...still a complete Prolog newbie...so...no warranties at all -;)

LEDNumbers.pl
number(0,[[' _ '],['| | '],['|_| ']]). number(1,[[' '],['| '],['| ']]). number(2,[[' _ '],[' _| '],['|_ ']]). number(3,[['_ '],['_| '],['_| ']]). number(4,[[' '],['|_| '],[' | ']]). number(5,[[' _ '],['|_ '],[' _| ']]). number(6,[[' _ '],['|_ '],['|_| ']]). number(7,[['_ '],[' | '],[' | ']]). number(8,[[' _ '],['|_| '],['|_| ']]). number(9,[[' _ '],['|_| '],[' _| ']]). digits(0,[]). digits(X,[H|T]) :- (X/10 > 0 -> H1 is floor(X/10), H is X mod 10, digits(H1,T)), !. accRev([],A,A). accRev([H|T],A,R) :- accRev(T,[H|A],R). getDigits(L,R) :- digits(L,Y), accRev(Y, [], R). show_records([]). show_records([A|B]) :- print_records(A), nl, show_records(B). print_records([]). print_records([A|B]) :- format('~w',A), print_records(B). merge([L], L). merge([H1,H2|T], R) :- maplist(append, H1, H2, H), merge([H|T], R), !. listnum([],[]). listnum([H1|T1],[R|Y]) :- number(H1,R), listnum(T1,Y). led(X) :- getDigits(X,Y), listnum(Y,Z), merge(Z,R), show_records(R).

Wanna see it in action? Me too -;)

Back to learning -;)

Greetings,

Blag.
Development Culture.

My first post on Prolog

As always...I was looking for my next programming language to learn...and somehow...Prolog got in the way...

I had played with Logic Programming in the past by learning Mercury...but really...when it comes to logic...Prolog wins the pot...

Did you guys knew that the first Erlang compiler was built on Prolog? Me neither -:P

For learning...I'm using SWI-Prolog which seems to be the nicer and widely used...and I have to admit...it's pretty cool -;)

So...in a glance...Prolog reminds me of Mercury of course...but also Forth a little bit...and weirdly to   Haskell in the sense that recursion is a key component...

As happens many times when I'm learning a new programming language...I started off with my Fibonacci numbers application...so here it is...

fibonacci.pl
fibo(NUM,A,B,[H|T]) :- (NUM > 1 -> H is A + B, X is NUM - 1, (A =:= 0 -> fibo(X,H,B,T); fibo(X,H,A,T))). fibo(_,_,_,[]). fibonacci(NUM,R) :- fibo(NUM,0,1,X), !, append([0,1], X, R).

.pl extension? Yep...the same as Perl...but as you can see...it has anything to do with Perl at all -;)

Anyway...here's the output screen...

My LED Numbers applications is gladly ready and will come after this blog -;)

Greetings,

Blag.
Development Culture.