Last week I was on a business trip to Singapore, and stayed in one of the most prestigious hotels in the area, the Capella on Sentosa Island - Singapore’s man-made holiday park. Everything went swimmingly: the resort was beautiful, the service fantastic, our event fully booked and the participants loved the program.
As I do nowadays, I travel light to such events, carrying only my iPad Pro (now the 10.5’’ model), my TextBlade and a small bag of cables, chargers, and other accessories in my bag, and a carry-on suitcase with 1 suit, some shirts, socks, and clean undies for the week. As long as there is Internet I can do everything my work requires without any problems: I write in Word, I access our company’s admin systems using a VPN connection and Safari, I present using PowerPoint and a Bluetooth remote (Kensington PresentAir Pro). There are one or two things I need a Windows machine for, but I solve that by remotely logging in to my Windows laptop at home, using Parallels Access. If the Internet is even half decent that is usually fast enough to get the job done.
This time, however, I was caught off-guard by the hotel’s Internet’s strange behaviour. The speed was fine, much higher even than what I can expect in my home in rural Queensland, Australia. But some sites I needed to access either took ages to load, occasionally timing out, or they would not load at all, reporting back I was not connected to the Internet (which is not the same as ‘server stopped responding’, by the way, several of my company’s apps reported I was off-line, even though I could not have started them if I indeed had not been on-line). So I was a bit handicapped but didn’t feel it was big issue: when I’m presenting at an event I mostly focus on presenting, and far less on the other parts of my job. I am there, after all, to give the very best I can to our audiences, and I have learned that trying to multi-task during events distracts me so much it negatively impacts my presence and focus during my presentations and workshops.
It went really wrong, however, when after the event, with a day left in the hotel before traveling home, I decided to go through some of the emails that were rapidly piling up in my inbox. One of those emails was an announcement from WayTools about an update of the firmware, specifically targeting Bluetooth connections issues some TREGgers have experienced. I have had some of those issues myself when using my MacBook so I thought it would be a good idea to install the firmware update so I could test it as soon as returned home. Which turned out to be a dumb idea.
First of all, having worked in IT and traveling the world for over 30 years, I know from experience that updating software or firmware while on the road over public Internet connections is never a very smart idea. As long as your software and devices are working, don’t risk it, unless you either have alternatives, or really don’t care if you end up bricking things. Second, the strange Internet of the days before should have been a warning: intermittent disconnects in the middle of OTA updates can really, really screw things up, and I should just have been patient and wait to be back home before trying to update my TB.
But I have to admit: I’m a bit of a neophile, always ready to try the latest, and since I had some free time on my hands I decided to go ahead with the update. I also had some first-hand experience with the robustness of the TB’s firmware protocols, that seem to be smart enough to fall-back on the built-in modules when the flash-memory seems faulty or non-responsive. “What could possibly go wrong?” are the famous last words I must have had on my mind when I decided to press the Install button for the TB firmware update.
Well, what could go wrong became pretty obvious pretty quick: within seconds the installation process started throwing up error-messages relating to my on-line status. My TB’s identity could not be verified, the cloud assets could not be downloaded, and it then claimed it would ‘use cached assets’ instead, and then seemed to get stuck. When I saw the warning messages piling up and no visible progress, I decided to abort the process, leaving my TB in an undetermined state. I tried it again a few more times, with the same result, before giving up. I had other things to do, which did not require the use of my TB, so I folded it up and put it away, trusting I would be able to complete the update when I got home the next day.
And then I arrived home, got some sleep, and set up my TB and iPad to write some emails before having another go at the update. To my surprise, the TB wouldn’t type. Looking at its indicator lights it should have been functioning: it flashed the right patterns when using the jumps, it signaled shift, caps, and layer modes correctly, it just didn’t transmit any characters. So I decided to have a look at my Bluetooth connection. Instead of the usual TextBlade and Hex number identifier, it had paired itself as TBBootloader2C2F (if I remember correctly), something I had never seen before, and suggested it may have gotten stuck in its boot-up routines. I turned the iPad’s Bluetooth off and on, without success. Then I pulled apart the TB and put it together again - which usually restarts the device from scratch. This time, it made it disappear completely. I could not get it to show up in any Bluetooth device list on any of my devices, including iPhones, iPads, MacBook and Windows laptop. Strangely, the TB did seem to think it was functioning fine: normally when it can’t pair to a device, it will signal this with a flashing pattern or a blinking dot, depending on the circumstances, and it will not respond to keypresses. Now it was not showing anything out of the ordinary, and kept lighting up things like shift, caps, and all the layer modes, as if everything was fine.
So, I managed to well and truly brick my TextBlade. A completely new and unexpected experience. And since this happened on a Saturday I fully expected this situation to remain unsolved until after the weekend. After all, I would not expect the WayTools team to be still working (my Saturday afternoon would be their Friday evening). So, I did post a message inside the app explaining my situation and decided to go back to old-fashioned full-size keyboard pain.
For those who think the TB should have been released already this perhaps shows that things are never quite as simple as they might seem. The TB is a complex piece of equipment, with sensitive electronics inside, and a very complex interplay between different hardware and software components that have to function under widely varying conditions. To make matters worse, the OTA process to update its firmware has to be able to successfully negotiate or recover from flakey Internet and Bluetooth connections the TB itself has very little control over. There are so many ways things can go pear-shaped it’s almost a miracle the TB and its associated eco-system works as well as it does. As an engineering friend of mine once said: for every character of code there is only one way it can be right, and thousands of ways it can be wrong. In the case of the TB I guess there are millions, if not billions, of ways it can get things wrong. The TREG program is meant to find as many problems and exceptions as possible by exposing the device to as many different environments and situations as possible. Based on the feedback received, the hardware can be refined, and the firmware and software hardened, until it is considered robust enough to be released to the general public. What WayTools has found - by their own admission - was that the complexity of hardware, Bluetooth and Internet connections combined with the OTA complexity and the fact that there are many flavours of Bluetooth implementation, some of them downright buggy (yes, Apple, I’m talking to you) made properly hardening the code, the updates, and the recovery mechanisms a much harder task than anticipated. I think the team has done an amazing job and have come a long way creating an almost fail-proof system. But my case proves there are still situations that can wreak havoc and defeat even the most robust of protocols.
Which does not mean, by the way, that my particular set of circumstances will now automatically become another gating issue. That depends on what they find when the team does their post-mortem analysis. If it turns out to be a random fluke, not very likely to occur again (it could be, for instance, be caused by the precise millisecond in which I interrupted the OTA process, or pulled the blades apart when it was trying to recover from its boot failure), then they may decide to take a note of it, and park it for future consideration. If it does point, on the other hand, to a weakness that could occur more often it may mean they need to find a solution. After all, you don’t want to leave a vulnerability that can brick the TB through an OTA failure. Personally, I believe my problem was a fluke, and not very likely to be replicated. What I do think, however, is that this may call for a - remotely triggered perhaps - forced factory-reset option to forcibly return the TB firmware to its original state. I have no idea how hard or easy that would be to implement, but it would at least prevent future TBs from ending up bricked.
(This has already become quite long, but there is more to the story, so ... To be continued).