Smart JS a tale of two platforms
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
Title |
| |
Title of Series | ||
Part Number | 17 | |
Number of Parts | 110 | |
Author | ||
License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
Identifiers | 10.5446/30927 (DOI) | |
Publisher | ||
Release Date | ||
Language |
Content Metadata
Subject Area | |
Genre |
00:00
Computing platformPlastikkarteAttribute grammarElectronic meeting systemSoftware developerLaurent seriesStandard deviationContext awarenessVideo trackingInternationalization and localizationPersonal digital assistantSource codeCASE <Informatik>Different (Kate Ryan album)Forcing (mathematics)Identity managementFlash memoryComputer hardwareArithmetic meanBasis <Mathematik>BuildingMereologyData structureBlock (periodic table)NumberEndliche ModelltheorieMultiplication signInformation securityMedical imagingForm (programming)PeripheralRight angleProcess (computing)FrequencyWorkstation <Musikinstrument>Cellular automatonComputing platformTraffic reportingSlide ruleCartesian coordinate systemComputer architectureCoprocessorPhysical systemSummierbarkeitBlack boxAddress spaceModulformScaling (geometry)Core dumpSoftwareCommunications protocolSoftware engineeringCross-platformOpen sourceServer (computing)Web 2.0Serial portComplete metric spaceFreewareData conversionAnalogyGoodness of fitInternet der DingeSoftware developerModule (mathematics)Reduction of orderExterior algebraBitProduct (business)Computing platformMountain passFirmwareArmBefehlsprozessorSet (mathematics)Open setCache (computing)Lecture/Conference
08:23
Attribute grammarPlastikkarteComputing platformView (database)StandortproblemSynchronizationCodeOrder of magnitudeOrder (biology)Convex hullComputer hardwareAddress spaceVoltmeterImplementationOpen sourcePersonal identification numberCommunications protocolFile systemDivisorRevision controlRange (statistics)WindowRight angleProjective planeComputer fileInternet der DingeComputing platformFlash memoryComplete metric spaceBlogSoftwareSpacetimeCurvatureModule (mathematics)WritingControl flowData storage deviceCodeComputer programmingPublic key certificateFunctional (mathematics)Declarative programmingPatch (Unix)BitGoodness of fitBinary codeSystem callSemantics (computer science)EmailRepository (publishing)Different (Kate Ryan album)Point (geometry)Reverse engineeringConnected spaceInternet forumState of matterEndliche ModelltheorieOrder (biology)FamilyQuicksortInformation securityProcess (computing)1 (number)ResultantEvent horizonMultiplication signCASE <Informatik>SphereLogic gatePhysical systemWeightVideo gamePhysical lawFood energyPattern languageSet (mathematics)Source codeLibrary (computing)Texture mappingIncidence algebraTheoryMereologyAutocovarianceWordRotationParameter (computer programming)Inequality (mathematics)PlanningStandard ModelLimit (category theory)Software developerColor managementForm (programming)Lecture/Conference
16:45
PlastikkarteComputing platformView (database)Electronic meeting systemAndroid (robot)Computer virusAttribute grammarConvex hullGoogolSynchronizationDean numberNumberFile systemShift operatorConfiguration spaceFlash memorySemiconductor memoryShared memoryLibrary (computing)MereologyMemory managementForestDivision (mathematics)Form (programming)Right angleSummierbarkeitDependent and independent variablesSoftware testingInteractive televisionQuicksortComputing platformMultiplication signVector spaceSpacetimeMarginal distributionExtension (kinesiology)Natural numberBitInformation securityIntegrated development environmentGame controllerPoint (geometry)Game theoryMappingMedical imagingPhysical systemProcess (computing)Food energyRule of inferenceAdditionProduct (business)Channel capacityOverlay-NetzVariety (linguistics)Module (mathematics)Computer configurationVirtual memoryConstraint (mathematics)SpeicherschutzLimit (category theory)CodeGraph (mathematics)WindowCore dumpFunctional (mathematics)Disk read-and-write head2 (number)Computer hardwareDialectComputer fileDifferent (Kate Ryan album)BootingCartesian coordinate systemStructural loadAddress spaceFirmwareLecture/Conference
25:08
PlastikkarteComputing platformMathematicsAttribute grammarCore dumpEnterprise architectureComputer animation
Transcript: English(auto-generated)
00:02
Okay, I'd like to introduce Diamit. So my name is Diamit and I'm a software engineer at Savannah. We are a startup...
00:23
So it's not amplified? So we are a recent startup. Some of you may know Mongoose web server. We are the startup founded by Mongoose web server and decided to go to
00:45
IoT to grow our products to provide full stack IoT platform based on JavaScript. And with the goal of... We learned from experience with Mongoose that people want more than just embedded HTTP server and we aim to provide that.
01:02
So in particular, we are aiming at the army of web devs. We want to bring them onto embedded programming, embedded development. For this, we found that the market lacked JavaScript engine. So we are writing one and there is separate talk about that tomorrow, which I encourage you to attend if you are interested in that.
01:22
We emphasize we are targeted in ECMAScript 5 compatibility and our emphasis is on small footprint, like kilobytes footprint. But this talk is not about that. The talk about that is tomorrow. The platform, we aim to provide a complete solution with device firmware and communication protocol between them.
01:48
But which you are not obliged to use, you could use whatever you want on the device. You could use all of that or each of those parts is tended to be separately used.
02:00
So the device part is open source under GPL and we are targeting multiple platforms and we started with ESP8266 and the SCC3200. We also have a POSIX port, which will work on your Raspberry Pi. And today I am going to talk about our experience porting on these two platforms, these two embedded platforms.
02:22
Because the reason is, when I look back, these platforms are as different as you can imagine and pretty much in all respects. And I would like to highlight those differences and how we tackle them. So I will give you an overview. Some of you may know, for those who don't, the SCC3200 is a system on a chip with Wi-Fi capability from Texas Instruments,
02:51
which is a major vendor. This is their IoT play. They very much, there is like all the resources, like resources for the big corporation behind us.
03:01
And this is their push into IoT. They use a well-known, they have well-known ARM, a well-known processor. It's an ARM Cortex-M4 core with an interesting architecture, which shows that it's having everything on one chip, which means a radio and application processor is not common.
03:22
Usually it's like you have an RF chip and you have your application chip. And this one integrates them on the same die. But they are still kind of separate, because you have two, actually two separate cores. One is called network processor. It's a SCC3100 network processor and application processor on the side.
03:43
They are connected internally by SPI. But otherwise, you do not actually get to program the network processor. It's like, it's a black box which you talk to over SPI. It has a fairly beefy 256K of RAM. It doesn't have any built-in flash.
04:00
It uses external SPI chip. So, yeah, it uses Wi-Fi. It actually has only, pretty much only the features you expect from this, from a chip like this. I2C, SPI interfaces, two serial UARTs, pulse width modulation timers, hardware accelerated crypto, RTC for analog converters.
04:26
So, all the peripherals, all the usual peripherals that they have. The SDK, whatever I say next, I'm going to recommend TI on your SDK, because it's really, really good. It has, it's well documented.
04:41
It's actually, all of the SDK is completely, apart from their proprietary TI, RTOS, which I don't know why, why would you use. They ship complete sources for free RTOS, all with build configuration, so you can rebuild. So, they ship binary blocks for sure, but you can rebuild all the components from sources, which is very recommendable.
05:04
And we use that to reduce footprint somewhat. So, that's all good. The hardware, I mean, datasheets, you know, what else can you ask? When I was implementing I2C driver, I actually could go and read everything about how to drive the module, the internal I2C module, like this is.
05:28
And this was after my ESP experience. This was like, ah. Anyway, so the price though is to build, so what you buy is, you can't buy a chip itself, but what you usually buy is a module that you plug on your PCB.
05:44
This one doesn't have integrated antenna, so you actually have to trace the antenna as well, which is a bit of a pain. But basically, it's $25 on Digikey and about half that, like kind of non-official alternative module on AliExpress.
06:01
So, that's kind of the industry standard. However, this is the newcomer. This is the interesting thing. They are kind of an accidental IoT success by a small Chinese vendor. I will show you how they started. This is from what they started. So, it's kind of obvious when you look into this that they didn't intend to become an IoT vendor, but they did.
06:24
So, it's a fully custom design. The silicon is fully custom designed by the company's CEO, who did a really good job. The hardware is top notch. It uses a modular, for the CPU part, so RF is all custom designed because this company's CEO is in the past an RF designer.
06:50
So, the CPU parts are kind of from the building blocks of a thing called Extensa. It's an 80 slash 160 MHz.
07:01
You can overclock to 80 MHz. It has a 96K of data RAM, instruction RAM. It also doesn't have any built-in flash. It is SPI only, but with a very, very useful feature that it does have a rudimentary MMU, by which you can translate directly, the address space is directly translated onto SPI with obvious performance impact,
07:27
but with some 32K cache behind it, so it's not terrible. In practice, it's fine. Being a relatively obscure instruction set, the open source, of course, the company behind Extensa, they sell proprietary compilers,
07:46
but we are not interested in that here, right? So, GCC toolchain is, up until very recently, was like a one-man effort by an employee of the company, but it's just one person. And GDB support is still kind of shaky.
08:03
One interesting thing it supports is it can run AP and the station at the same time, which is kind of useful on different channels also. So, it doesn't have as rich set of peripherals, because, as I said, I don't think it was intended as a general purpose.
08:23
What it was intended, really, is to provide a Wi-Fi shield for your IoT projects. It does have I2S, but it doesn't have I2C. Internally, it has SPI. It has 1.5 RX, because one of them you cannot really use, because it's multiplexed to pins that are used for SPI flash.
08:45
Well, you can only use TX, but not RX. So, it's kind of weird. It has one ADC that is 0 to 1 volt range. Not very useful, so we have to have an external divider. Now, the software side of it is completely, as I said, it's complete opposite of TI in quality, because plainly it sucks.
09:10
There are two SDKs, and both of them suck. One sucks more, the other sucks relatively less. But given the origins, I think it's explainable, because, as I said, it didn't intend to become.
09:24
I'll go into more detail about that. There is absolutely zero data sheets. Like, data sheets are unknown in this space. All you get, the best you can get is headers, and then kind of guests, and then some examples. So, you can kind of deduce what hardware does, but zero data sheets, that's just not unknown.
09:43
However, what's interesting about that is that the module, as I said, is that you plug into your project to get the Wi-Fi connectivity, and it includes antenna on the module as well. It's under $2 on AliExpress. So, that gets people interested. So, this is the ESP-01.
10:02
This is the very first device where this is what became popular. As you see, there is an SPI flash chip and the processor itself. As you can see, there are only, like, this is the Wi-Fi sheet. It has RXTX, it has the supply voltage, and the programming pin.
10:21
That's it. There is no, it's not intended to be connected by protocols. However, people got interested. They reverse engineered it, and the company actually picked up on it and went into IoT space, decided to release SDK. Their first efforts were really miserable.
10:41
So, what you get with this is really not enough. So, you get two GPIOs, and if you want more, people were doing this. So, this is what you get for GPIOs. There was enough interest that this was actually good enough. However, we are past that. So, this is the latest module that gives you all the, breaks out all the pins that you can really get from this.
11:07
And it has, they subsequently got the FCC certification and CE certification. So, it's all kosher. So, this is what you can buy for under $2. Now, so what I'm going to tell you is the origin.
11:22
As opposed to explicit IoT push by TI, this started with a module that got reverse engineered by the community. And it's just interesting that the company, so they didn't fight it. They decided, maybe there is something in this IoT thing. So, they started releasing SDKs, but they weren't very good.
11:43
Like, I mean, they just don't know what function declaration is. Like, I mean, it's obviously, they just don't declare the functions. Like, they don't need it. They just ignore the warnings that you have, that you use. WIL.
12:01
What? When you do WIL, you get all the warnings. You get warnings, like it's all, it's terrible. But you don't do that, and it's all fine, right? So, you just use a function, you know it's there. Ignorance is a bliss. Yeah, so, exactly. But, it's been going from version to version. They use a function, and they just don't declare it. The thing does not comply with all warnings.
12:23
So, you have to declare that yourself. Have you tried submitting patches to them, by the way? They are not very, they are not very, they don't have open source. Well, they are not very receptive to patches, I shall say. Anyway, so they use, so the call SDK is a bunch of binary blobs.
12:44
But it is based on things like X-TLS, LWIP, which they don't ship headers for. Instead, for example, for networking, they build something that's called ESP connection library, whatever. Which is another layer on top of LWIP, because LWIP is not good enough, I guess, with these two APIs.
13:04
I don't know why. Anyway, this thing just sucks. You just shouldn't use it. So, we finally, finally, one thing we persuaded them to do is to just release the headers for the LWIP. Yay! So, now you can completely bypass this thing and just use LWIP and it all works.
13:25
The state of WTF is really amazing. Like this, as I said, the documentation for this is spread all over the forums. Different code repositories with examples, reverse-engineered ROM. I mean, it's just, however, as I said, the community has been very excited by the, I guess, the price point.
13:44
And it's actually, the hardware is really neat. I'm commanding them. The TI is somewhat strange on hardware. The ESP is, like, it's beautifully designed hardware. I wish it was documented better.
14:00
So, let me go to CC 3800 so I can tell you. So, right from the beginning, with CC 3800, what we ran into is that you cannot flash the thing without some stupid Windows tool. Like, this is, I mean, so we used, we took, there is actually an open development platform called Energia, which is a fork of Arduino.
14:24
And they ship, like, kind of half-assed flashing tool, which lets you just flash one file and that's it. So, we weren't quite happy with that, although we did use it for, in the beginning. So, we reverse-engineered the protocol and we integrated, we shipped implementation in our own flashing tool.
14:47
So, at least, that's all. But it's kind of, like, why wouldn't it, there isn't even a Mac version. Okay, we are used to not having Linux versions, but not even a Mac version. What the hell, really? Now, the failure fast story is quite interesting.
15:02
So, the flash on CC 3200 is, it's like, it's not a flat address space with blobs. It's actually a file system. But the problem with this file system is it's really, you cannot do much. You have to, I mean, it's, so, first of all, you cannot access the flash other than through this file system.
15:22
This is a problem. If you want to, for example, store things that you don't know the size of in advance, and on this file system, you create a file and you declare the file, the size in advance. Also, you can only open files for writing with truncation. So, it's like, there is no append, there is no open for read and write.
15:42
No, just open for writing, it's open. From gates. So, that was really, and with SmartGS in particular, we provide a file API. So, for this, and there are also, there are strange, other strange things. For example, when you open the file for writing, it does look like, because it's on flash,
16:03
so there are certain semantics, it can only reset bits and not set them. It will let you do random writes, thankfully, but it will have this weird semantics that you can only reset bits. So, it's not really writing, resetting bits. But, never mind that. So, what we instead did, it's basically, it's very weird.
16:23
With SmartGS, our primary goal is to kind of, to smooth out the bumps, you know. So, we provide what looks like a POSIX file API, and to do that, we actually implemented SPIFFS, which is an SPI flash file system, inside a container of files on this fail fast.
16:44
And, yeah, that was, we have a blog post about that. But, basically, they acknowledge the problems with the file system, but they say they will not fix it, because this product is now, they don't add features. And, apparently, a file without truncation is a feature.
17:03
Right. Well, they are active on the forum, by the way. They do interact. Both of them actually have forums. Both of them interact, and TI is actually quite responsive. The problem is the responses are often, okay, wait for next. Not good enough. And, the biggest problem with TI is that the RAM.
17:25
256K seems like a lot. But, the problem is that it doesn't have an internal flash, so it loads all the code into RAM. So, you code and data, and RAM is shipped. So, 256K is on get, which is really not good enough for our purposes,
17:42
because our firmware is quite fat. We have to fit all the JavaScript interpreter, the libraries, the things, and still have some memory left for application use. But, there is really no way around it. So, what it does is the foot loader loads the firmware image,
18:01
lets it run, and then, well, you can, it's basically, there is no memory. There is no memory mapping of anything. There is no memory protection. So, you can implement some, it's basically, it's a dead end. We reached a dead end that we ran out of code space,
18:21
and now we cannot add features to SmartJS because of this. Or, it's either, so, we would have to go to something like, you know, DOS overlay days to implement something like that. But, we just don't want to do it. So, that's, that kind of sucks. It's like, it's a big problem for us.
18:42
The problem is that Cortex A4 core lacks any kind of memory management in it. They didn't think about implementing it. Usually, the controllers have embedded flash that just sits on the system bus, so it's like directly mapped, it's all good. But here, it's SPI. It doesn't have any direct mapping as the kind of, that ESP does.
19:02
And, yeah, it's really a big disadvantage in that way. So, and I actually went through everything pretty quickly. Let me see, I have five more minutes. So, yeah, that's, the option is that we managed to, in the end,
19:21
we managed to, to a certain extent, that you don't see all this, like all this terrible tragedy of ESP SDK. Or, like you get the same JavaScript environment on both platforms. And, we suffered, so you don't have to.
19:41
This is a graph of popularity, though. So, you see why it makes, it's, so see that both platforms started, but even the CCLR200 got a bit of a head start on ESP, but you can see the interest, the relative interest in both platforms. And, this is, I mean, the suffering through all the idiosyncrasies was worth it,
20:07
because it's, there is incredible community around ESP. And, we, so what's next? The ESP32 is the next chip from the, from Espressif Systems. They learned from their mistakes, and they are adding Bluetooth,
20:25
so it will be Wi-Fi and Bluetooth. They're adding more, so the chip has more pins, there is more GPIOs. They are added to I2C and the hardware of PWM. Second core. Yeah, there is, I didn't mention, yeah, second core.
20:41
Yeah, there is a second core, which, so we were first, when we heard that they were adding a second core, we said, oh great. Now we have this SDK and all the SMP issues. I mean, these people, I mean, the people don't know about function declaration, like what do they know about locking, I wonder. So, no, but fortunately, fortunately, it looks like it,
21:03
first it will be able to, it's possible to turn it off, run it as an independent core with a shared memory region, or run it in SMP configuration, like, please don't. And, yeah, so it has more RAM to begin with. It has the same SPI, transparent SPI mapping of memory regions to SPI,
21:27
and it looks like the memory regions will be even more flexible in the configuration, so we will be able to change the division between the amount of IRAM you have and DRAM, and how much goes to one core to the second core. The silicon is in beta, we actually do have beta silicon,
21:42
and we are starting to play with it, so we plan to be, the release date is in a couple of months, so we plan to have full play by the time it goes general availability, SmartGS should be there. Maybe not supporting, because some of the, for example, Bluetooth is currently completely undocumented and unsupported in the beta SDK,
22:03
so we cannot even start on that. But as much as we can, we will have play the day of the release, and it's, yeah, it's quite interesting. The SDK, I'm happy to say, the SDK on ESP32 looks like more mature, at least they started, looks like maybe they hired some new people that actually understand more, and it's looking much better.
22:25
So, yeah, that's, the future looks bright with ESP32. You have two minutes left for questions. Yeah, do you have any questions? Do you have any idea of the price point of ESP32? Do you have any idea of the price point? Yeah, price point.
22:42
Well, I don't imagine, like, they understand that their biggest advantage is under $2, otherwise people wouldn't, wouldn't, wouldn't put up with all the hassle. So, I don't, but I don't expect it to be much more expensive. We have one here to show you what it's like, just by coincidence.
23:04
Is it the 32? Yeah, it's 32. Well, no, this is the 8266, this is the 12E module. This is the old one. The one I can, yeah. Any other questions? The question about the SPI, you said that, so actually on the ESP, there is one flash chip.
23:26
The new, on the ESP01 was like 512K, on the new one it's 4 megabytes. On the TI you said that there had a difference between the SPI mapping, that on one, on one for the ESP was a...
23:44
There is no SPI, the SPI mapping I referred to is basically the new original address in a certain region gets transparently mapped into an SPI region, an SPI flash, an ESP which is quite handy. We do OTA updates that way, so we keep two separate images. On TI there is nothing like that.
24:01
I think they ship, it ships with a few variety of modules. But see the problem with, for us, at least for us, on TI the flash capacities are relevant because only part of that can be used for code. I mean you can write data, I think that it ships with 8 megabits, 1 megabyte flash. The code space is really the constraint for us on TI.
24:22
I was wondering what's the actual limit for the amount of flash that should be used. Because even now we have flash chips with 16 megabytes or 64 megabytes. I don't know if you have a problem with that. There is a limit I think on the size of the code window you can map.
24:43
And if I'm not, I didn't work with that in particular, but I heard, I think it's 8 megabits, so it's a 1 megabyte window. But you can access flash for data, you can access flash beyond that. Although I personally have not seen and have not attached flash more than 4 megabytes.
25:05
Thank you very much.