So, hi, everybody.

This talk will be about the Firefox OS tricorder app, which I made, which I launched on the phone as well here. You'll see a screenshot on screen later as well. Its main part of the talk will be about how you actually access device sensors

on Firefox OS devices. As Jana already mentioned, I'm Cairo Robert Kaiser. I'm not maintaining Sea Monkey anymore. That was the past. I headed that over to other people in the community. I'm working for Mozilla QA now as a project manager.

And the slides for this are already up at slides.cyro.at, which is my domain for all slides that I ever had. There are a lot of links in there to the web APIs and to the actual documentation of the functions on MDN.

So if you're interested in that stuff, you can look up that in the slides. And they should work in modern web browsers, at least in Firefox. Didn't test anywhere else. So first, what is a tricorder? It's a sensor device as seen in Star Trek.

And it displays data that is just needed for the story in the script. So it takes them at plot speeds and accuracies and resolutions, just whatever the plot for the story needs. So whatever is seen on screen is usually pretty useless,

but it looks fine and nice. The Firefox OS tricorder app actually has real data on it that it displays data from device sensors as exposed by the web APIs. It's available from the marketplace

just under the name tricorder. And the code is up on my GitHub account as well. So you can look it up there. I will have very abbreviated pieces of code in there if you want to see how it actually plays all together.

Just look it up at GitHub. I hope it's documented well enough that you actually can understand it. So this is how it looks, just so you see it looks the same on the phone itself. The most important thing for a Star Trek app

is it needs to have a start date, which is usually pretty nonsensical. And this one is just taking every year 1,000 units and starting with the point that the first Star Trek episode aired in the US.

Then it has an area for the title. The whole interface is based on the Star Trek Next Generation and Voyager TV screens, so the LCARS system. It has those navigation items for the modules that expose different kinds of sensors.

And it has a full screen button so that you can get the experience without the title bar on the phone, which just immerses you more in the experience. So I talked about those modules. And I'll go through the interfaces module by module.

What I'm calling a module here is some HTML that exposes the switch button and the actual UI, and a JavaScript object

that has two methods, activate and deactivate. Those are automatically called by the functions when you switch between the modules. So the active one is deactivated, the other one is activated. So we properly shut down everything that we had active from the last module.

So let's go into the actual code here. The position module is the first one in the list because it's the first one I did. I already had played with an application with some GPS stuff, Lantia maps, an OpenStreetMap-based GPS track

recording app for Firefox OS. So I knew a little bit about that. My main reason for doing this tricorder app was I wanted to play with the APIs we have available. And I thought, what better than to do just something that exposes all the sensor data. And as that matches the tricorder,

that's where it comes from. So position is just a GPS or Wi-Fi or mobile cell location. That's all hidden behind the geolocation API. So if we don't have GPS, we just send a request out to the Mozilla location service

that includes what mobile cell you're seeing and what Wi-Fi networks you're seeing. And if we have data for that point, we can send you back where you are. If we don't have data for that point, it's good if you're using the most Stambler app

to actually help us collect data for that. Our database is way more privacy sensitive than the alternatives out there from Google or Apple. So anyone who can help us there is welcome. So we're using the geolocation API and we need to specify the geolocation permission

in the app manifest for Firefox OS apps, even though normal websites just can use it. But for web apps, we want to have those things that they are using that could be sensitive to be mentioned, but will prompt the user in any case if they actually want to expose it.

This is how it looks in this case. Geolocation has two main functions, one to get the current position and one to continuously get the position when you move. This is the one I'm using here, so that's watch position on the navigator.geolocation object.

Yes, you see the blue there on the slides, that's actually the link to the documentation. And you get a call, you put a call back in there that has the one argument, which is the position. On that, you have a chords property

that has the actual data underneath it as latitude, longitude, accuracy, and a number of others. You also put an error function in case it doesn't work, which can happen a lot if the user does not acknowledge the prompt or something like that.

We're enabling high accuracy there to be sure we get a GPS position if it's available in some timeouts. One thing is important in all of those examples, I have that at the end, how to clear up, because this will continuously call that call back

as soon as we have any location change. So we need to stop that at some point when we deactivate the module in the tricorder, and we call clearWatch. And that's the reason why above there,

the uniforms were not made for microphones. That's the reason why we have this watch ID up there that we assigned in the beginning, because we needed to actually know what we're clearing when we deactivated.

The second thing I did was I called it Gravity Module, because it actually measures Earth's gravity as a side function from what it's doing. It accesses the accelerometer and the magnetometer, so magnetic compass in 3D.

With the APIs for that are actually events, the device orientation and device motion events, and those don't need any permission at all. You can even use that in websites without any prompt or anything. The code for that is, as I said, it's events,

so you need to add an event listener on the window, and you just add the event listener for this event with which function is being called. You'll see I have this dot in all of those things.

As I said, the module is a JavaScript object, so I'm setting this all on the JavaScript object of that module so that I don't pollute the namespace of anything else. The event that's being called, the function that's being called for that event,

and I never remember what the true is, but everybody specifies it everywhere, so I always have it in there. You have orientation, that's actually the magnetometer, so the orientation of the device with angles in all three directions, and motion is the accelerometer.

The orient event, that's the orient event function on the module that takes the event, gives you this orient data that you have as an argument on the function,

and you get alpha, beta, gamma angles in degrees there. The directions of those angles, there's a pretty good page on MDN that describes how those work, which directions those are, so it's best to look them up there, because it would take a longer time to describe that here.

It's better to look it up. They have graphics for that and to illustrate things like that. For the motion event, it's pretty similar, but on the events, you have actually a property. You have two properties. One is acceleration, and one is acceleration, including gravity. The acceleration, including gravity,

is the one that always works. The acceleration one only works if you have a gyroscope so that the phone actually knows how to subtract the Earth's gravity from the value that it's measuring. The acceleration, including gravity,

is the one that always works, and for a tricorder, if you go to a foreign planet, you want to know what the gravity of this planet is, so it sounds fun to have that included in this case, anyhow. On this acceleration, including gravity, you get x, y, and z coordinates in meters per second squared,

and of course, if you perform the calculation in 3D space to even those out, you get around 9.81 meters per second squared on Earth. And on the flame, it's even mostly accurate.

It depends on how good the accelerometer is, how accurate it is. You'll get lots of calls to those events, so if you're using that in an application, you might want to figure out how to smoothen over that or something like that, because you get really tons of those events.

I'm handling everyone and just displaying it, but in another thing, you might want to deal with that differently or leave out some of those or something like that. Of course, when clearing up, you need to remove those event listeners,

otherwise you would pull those functions in a never-ending loop until the battery's gone or whatever, or you close the app, of course. But remove event listener just mirrors exactly the event listener function. You just need to care that you're doing it.

The next one is probably the most complicated example I have in this, is the sound module. It looks pretty innocent, the microphone. Microphone is something that every phone has and that has to be easy, right?

Well, getting the microphone is easy, but what I'm doing is I'm displaying graphically graphically an analysis of the data that I get in from the microphone. That's not that easy. There's a few fun things in there.

For getting the microphone itself, you're using WebRTC or the get user media piece of WebRTC. For analyzing, I'm using Web Audio, which is a pretty new standard, but it's extremely powerful, so what you're seeing is a very small piece

of what it can do. The permissions you need is audio capture for the WebRTC get user media part to actually fetch the audio data from the microphone. And that looks like this. As I said, probably the most complicated piece I have in there, and what I feared is true

that we don't even see that completely here. The easy part is the navigator.getUserMedia call itself. We set audio true because we only want to have the microphone.

getUserMedia could fetch video from the camera as well, but, or a single picture from the camera. We only want an audio stream so that we hand over audio true, and then a callback function that is actually called

with the stream we get as an argument. Inside of that, we set up the audio context for Web Audio, and that's where it's a little bit more difficult. First, we assign that local stream

to a property. That's because of the cleanup. Of course, we want to turn off the microphone when we're leaving the module so we need to stop the stream at that point. Then we create a new audio context, connect, create an input where we,

for the input we specify a stream source, and that is the audio stream that we just did get. Then we create an analyzer so we can actually analyze the data that is coming in there, and you'll see there's spelling differences.

I'm using American spelling in my variables, but they're using British spelling in the API, so that's, be careful when you write that. You create an analyzer. You actually need to set up the FFT sites and stuff like that of the analyzer, but I don't want to go into those details here.

And then you connect the analyzer to the input, and then you're set up, basically. Then what you still need to do is, in my case, I'm displaying it graphically, so I'm doing a loop with request animation frame

so that every time we are able to display a frame, we are calling this function to actually display something. And in there, we create an array, a fixed typed array in this case,

and this is a C-inspired API, so it has those strange things that you actually hand over the array to the function, which is something you usually don't do in JavaScript. You hand over the array that the data will be put in

as an argument in the function. It's something that people doing C, C++ code do all the time, but in JavaScript, it's a bit of a strange construct. And then you have this data actually populated, and you can do something with it. I'm leaving that out and all the display,

because this piece is complicated enough for the short time we have here. If you want to see more details on that and how the code works in detail, just look up the GitHub page, and you get more of it. You saw that it's actually working,

so I have all the bits and pieces to connect there. I had some sensors that are interesting that we have them in there and that might have some creative uses that I hadn't covered with any other modules, like the light sensor and the proximity sensor,

and that was just two lines on there, so I realized I could maybe put an on-off for the flashlight on the camera in there as well. It doesn't work anymore on my nightly builds, but it works on older builds,

so I just thought, did you try it on the Tricorder app? Okay, yeah. On 2.1, it also worked on mine. Now it nightly changed something and it broke. So that's two events again, device light and device proximity.

It would be the camera API for the flashlight, and that would need a permission, the other two don't. I don't have the code for the flashlight on here because the camera API is kind of strange as well. So it's basically the same thing as we had with those other two events with the accelerometer and magnetometer.

You add event listeners, then you get the arguments in the functions, you get a value in lux for the lights, and you get proximity data, the value in centimeters, and the minimum and the maximum.

The flame only can tell you if it's more than five centimeters or less than five centimeters. But you get the minimum and maximum levels that tell you what the maximum value is you will get and the minimum value. And on the flame, you always will get either five, which is equal to the max, or zero, which is equal to the min.

But on other devices, you might get more if they have better sensors. Of course, you need to remove those, and as I said, the flash or torch works with the camera API, with the getCamera and so on. I won't go into this, but you can look it up in the code.

The camera API also is subject to change as we are trying to get it into standards and discussing with other players on the web how to do stuff correctly there. The last thing, and I will do this very fast because I'm, time-wise, also getting to the end.

The device module is pretty simple. I only have the battery status API there, so I want to, at some point, add free storage there, but I didn't get to that yet. The code for that, you don't even need

to do any complex functions there. You just have the value at navigator.battery.level, which is between zero and one, so multiply it by 100 if you want to get percent. You get a battery.charging that is true or false, depending on if a charger is on there or not.

If it's charging, you get a charging time in seconds. If it's discharging, you get a discharging time in seconds. How long it still will last, probably. Both of those have zero or infinity values when it's unknown due to different reasons.

Otherwise, you get a value in seconds. And with that, I hope we have time for one or two questions. That's Mr. Tricorder, if you ever saw the movie.

So do we have any questions? I think one or two we can do, yeah.

So the question is, I need to repeat them for the recording, if we can change the frequency of device orientation and probably device motion events. I haven't seen anything to change it, so you probably just need to select which ones you actually handle.

You get tons of them. I think any time that value changes and that time goes into a number of decimal places, I think any time it changes, you get something. And those sensors are noisy, so you actually get multiple calls a second. I think even per millisecond or whatever,

it's pretty fast. I haven't tried it, but you see, for a labyrinth game or something, you need it in high frequency. I saw a game like that that worked decently,

but yeah, I'm not sure. For display, it's good enough. For other causes, I didn't actually try. So yeah.

So the question is, as web pages have those capabilities as well, what it will help for web pages. So for a normal web page,

I don't think there's too much value in the number of those. The battery API can have some value if you're doing some intense stuff and say, hey, this person is out of battery, let's try to scale it down to save their power. Let's say a messaging app, let's slow down the refresh rate for something

to save their power. Something like that might be useful from the battery API. Stuff like orientation and motion, for a desktop computer, it doesn't change, so it's not really helpful. But for doing, as he said, games, it can be a quite interesting thing. There's also a vibration API that I'm not using here

because it's not really a sensor that you can make the device vibrate if you're the front most web page or app. For example, when you jump over something in a game, you can do a short vibration and things like that. So especially for games and things like that,

I think it's pretty helpful. For a normal web page, it depends. The light sensor might be interesting to actually change contrast, stuff like that. So yeah, we're out of time. Thank you, and live long and prosper.