... and how to restore it

As noted a couple of days ago, I broke my Nexus 5 and managed to get a backup via adb.

Finally, UPS managed - as usual with their second attempt and one call in between - to deliver the new Nexus 5 yesterday.
So, unbox, switch on, log into Wifi, etc etc... and have the system update to 4.4.3.

What?!?
Say Again!

"... and have the system update to 4.4.3."
No matter how often I repeat this, it's still weird that a brand new Nexus 5, ordered and shipped directly from Google, comes with 4.4.2 the end of 2014.

Anyway, update to 4.4.3, reboot, update to 4.4.4, reboot, update to 5.0....
Nope.
No 5.0. No Lollipop. As we know, rechecking every 5 seconds for an Android over-the-air update is as useful as hitting the elevator call button 10 times.

Well, so enable developer options with the 7x tap on the Build Number. Enable USB debugging.
Connect to PC, authorize, find the device with "adb devices". Voila.

Let's try the "adb restore". Enter the password on the mobile, this time without the help of the "adb shell input" workaround. The restore starts, and ends without any further message after about 2-3 seconds.

Not good.
Lets check the log with "adb logcat". There's something about mismatched header versions, so lets google this.
Well, as can be expected you cannot restore a 5.x backup to a 4.x device.
How silly of me to even consider it.

Recheck the over-the-air upgrade.
Nope, still not 5.0.

So screw it, download the factory image for 5.0.1.
Unlock bootloader, apply image, as described.
Reboot, log into Wifi, ...
Then the new Lollipop Setup Assistant kicks in and allows me to actually place my old phone to he back of the new one to NFC pair them, and transfer the basic settings (mainly account and Google settings) directly.

So I plug in the USB mouse again to the old, broken Nexus 5, and try to find the NFC settings (which are of course in the broken part of the display). Google around a bit to find there in the settings they are, and manage to half blindly enable NFC.
A couple of seconds later I have my account and (Google) launcher settings on the new Nexus 5.
Nice.
A lot of the app icons on the home screen are marked to show that the app itself is (still) missing behind the icon.

So I tried to restore the backup.
Again enable developer options (I just loaded a new system image, so everything I did an hour ago was of course gone) and USB debugging (and sideloading).

And now the "adb restore" really did work. Since there is no proper status on the device itself, I started a "adb shell logcat -s BackupManagerService" in another window to see some progress.

And about half an hour later I was done.
Most apps had their settings again, but not all.

Considering what last-resort hack this was, pretty good.
Considering how easy it is, to upgrade from one iPad to the next (via iTunes) this was pretty ugly.

By the way:
The Setup Assistant from Lollipop now supports a lot of options, even restoring data from a previous backup.
But I was not really inclined to test this right now.
If someone wants to sponsor yet another Nexus device for me, I'd be happy to test this for you.

How to Backup a Broken Nexus 5 via USB

So, the other day I decided to drop my beloved Nexus 5 - display first - on a concrete (or stone, whatever) floor, right in the middle of Vienna's most busy shopping street. And got a result like this:

Not only was the glass broken, but the display itself was defect in large areas as well. Touch of course did not longer work.

I visited a repair shop just down the road, and guess what, they only have Samsung and Apple spares on stock, and I got the tip to use a cover next time. Yeah right, Thanks a lot.
So with the broken display and all, I decided to purchase a new one (via Play store), which is en-route to Vienna right now.

In the meantime I was trying to save whatever I can from the old device. My Nexus is unrooted without even a hint of Titanium Backup or the likes activated. Luckily I enabled the developer options as a first step after I got it.
So I connected it via USB to my PC and saved whatever as accessible this way.

I have the Android Developer Kit / SDK Tools installed on my PC, so I have adb at hand - the Android Debug Bridge, which is needed for the following steps.

Then I tried an "adb backup", which prompted for a confirmation on my device.
Of course, with no touch and the broken display I could not enter one.

So, the ultimate hack of 2014 to the rescue:

Step 1.
Connect a USB mouse via USB converted to the phone.
Yes, I really wrote that. A regular (PC) USB mouse. And you'll also see a regular mouse pointer on the display then. How cool is that.

Unlock the phone by drawing the swipe pattern with the mouse.

Then immediately (with the mouse again), go to settings and change the pattern to a PIN.

The PIN can be easily sent via USB from the PC, the pattern less so. Now we can disconnect the mouse again.

Step 2.
Connect the phone to the PC again.

Run "adb devices" just to be sure you are connected (all adb commands are of course from the windows command line or whatever your "host" operating system is).

If you are stuck with the lock screen, you need to swipe up with the following adb command[1]:
adb shell input swipe 500 1900 500 900

Then you should get the PIN prompt.

Unlock it with a sequence like this:
adb shell input text 1234 
adb shell input keyevent 66

The first line sends the PIN (1234 in this example), the second line sends the ENTER after it.  The event codes can be found in the Android Developer docs or here. The POWER, ENTER, BACK and HOME keys/events are quite useful :)

Step 3.
Start the backup with
adb backup  -apk -obb -shared -system -all

(or whatever options you require)
Then you'll get the full backup confirmation prompt, which needs to be answered again with the help of  adb shell input commands.
You can set a password with
adb shell input text password  [2].

However, it does not accept the keyevent 66 to confirm this dialog; this literally inserts a new line at the end of the password.
We have to tap on the "BACK UP MY DATA" button at the right bottom of the screen. Which leads us to

Step 4.
Luckily you can also emulate touch events via USB. Some research, some guessing, and trial and error showed, that this button is roughly in the area of coordinates 800,1750   (so in the right third and just above the button bar). The display is 1080x1920 in size, so that was easy to guess and test.
So we send a
adb shell input tap 800 1750

And the backup starts.
Magic.

Tomorrow I shall hopefully receive my new Nexus 5 (if the UPS guy for once delivers on the first attempt) and then I can report on how the migration to the new device works.

--

[1] The screen is 1080x1920; so the start position for our swipe is 500/1900 , i.e. quite in the middle, almost on the bottom, the end is 500/900, i.e. quite in the middle, quite in the middle, and almost perfect swipe to unlock.
[2] substitute your real desired password for password here, right after text.

Wakelock Detector for Nexus 5 non-root, Lollipop

Earlier this week I had a good discussion and exchange with Max  - mainly on the Moto 360 watch, but also on a lot of Android / phone topics.

We were sort of mentally debugging a battery/power problem he had on his device (1+1), while my own Nexus 5 was lying around as idle as possible: I had to put it into power saver mode because my own battery was down to 5% at 7pm already.

So after all this discussion about Android power consumption etc etc, I decided not only to do some cleanup on my own device:

• delete two of the 3 calendar apps, because Lollipop Calender + widget is quite OK now (I removed Cal and Sunrise)
• change email sync frequency in Boxer from Automatic(Push) to 15 minutes.

but also to install the Wakelock Detector app (WLD), to check if any app is blocking my phone from going to sleep properly.

Since my phone is not rooted (and I have not intention to root it - for the time being), I needed to provide a little help and trick to WLD: essentially through local tpcip connection to the adb on the device. Very well documented here.

So, let's see what I'll find out.

BeyondPod - per feed playback speed

I have two podcast devices:

• an iPod Nano, where I have the less time sensitive podcasts, because I only synch it through iTunes every second week or so
• and BeyondPod on my Android Phone (Nexus 5 with Lollipop as of this writing), where I listen most podcasts, especially the daily ones.

A couple of months ago I changed the playback speed in BeyondPod to x1.3, i.e. 30% faster. I found this the ideal speed to accommodate all the daily (mostly tech) podcasts into my daily commute schedule, while still being at a good pace for listening.

However, some feeds, I like to keep on regular (=x1.0) speed. I guess, the less techie the podcast is, the more narrative it has, the more I like to listen to it at it's original, intended pace. Take the Serial Podcast, for example.

So I always went into the BeyondPod app, and changed the speed - simple.
I was pretty sure, that there is a better way for this, like setting the speed on a feed level. Today I remembered to look for this.
And - of course - BeyondPod can do that.
In the feed settings ("Edit Feed"), when you open the Advanced Settings you'll find it:

Nice one.

However, sometimes you need to be very confident that a certain feature/option "just must be there" in order to find it (especially outside the Apple environment).

Podcast meets longform - Serial

Ok, not strictly a technical topic. But a couple of weeks ago I stumbled over the Serial podcast.

Serial is a new podcast from the creators of This American Life, hosted by Sarah Koenig. Serial will follow one story - a true story - over the course of a whole season. We'll follow the plot and characters wherever they take us and we won’t know what happens at the end of the story until we get there, not long before you get there with us. Each week we'll bring you the latest chapter, so it's important to listen in order, starting with Episode 1.

I really perfectls combines the podcast format/medium with long form journalism (one whole season!). And the podcast is technically perfect as well.

Hint: it is about an investigation into a Baltimore homicide from 1999. So, if that's not your kind of content, bad luck :)

New Google Calendar Provider for Thunderbird/Lightning

Finally a Google Calendar Provider for Thunderbird/Lightning has been released that's based on the gdata api, and not just a network XML link.
Problem with the later approach was - to me - that you needed to add each of your calendars separately via their URLs. That seemed to have performance impacts as well. I had about seven such calendars, and it significantly (with my other CalDAV calendars) slowed down Thunderbird start-up time.

When the new provider was released a couple of days ago, I first really noticed, because it forced me - with a separate pop-up - to authorize it for my Google account. With seven calendars this step kept popping up - seven times, of course - but still for the old calendar URLs.
Performance was really nasty...  Thunderbird was unusable essentially.

For some weird reason, I tried a "New Calendar" > "On the Network" > "Google Calendar" and found, that it was actually letting my select Google calendars by account, instead of just letting me add the XML link per calendar like the previous version.

So first thing, delete all my old Google calendars in Lightning, and try the "New Calendar" > "On the Network" > "Google Calendar" again, with my Google account, and voila, it presented all the calendars I had on Google. With the color as on the web. I only needed to select the ones I wanted in Lightning, and that was it.

Should have been like this from the start :)

Thunderbird start-up performance is back to a couple of seconds now (from about 1 minute+)

Broken Google Maps Rendering on Chrome on Windows

The other day, I noticed that Google Maps rendering on my Chrome (v 36.0.1985.125) on Windows (7) was so broken, that Google Maps essentially became useless.
Streets/Roads were mostly invisible - looks like the line for the street has been overwritten by the street name:

At the same time it worked fine in Firefox and Internet Explorer (yes, I admit to having IE on my machine, just to test stuff like this).

So some testing and googling ... ok a lot of googling, since "google maps broken chrome" reveals essentially every rant on google maps there ever was...

Finally with some refining of search terms, I found the following hint.
Turns out, in this case you should disable the video hardware acceleration in Chrome.
Just open the settings, then search for "accel", and disable it, i.e. unmark the checkbox, if it is marked.

Restart chrome, and it works fine again:

USB Debugging a Nexus 5

It is described all over the internet, so let me repeat it here :)

To USB debug your Nexus 5, first enable USB Debugging in the Developer options.
If you don't yet have the developer options visible in Settings, then enable them like this:

Go to Settings and About Phone, then scroll all the way down to "Build Number" and then tap 7 times on the item "Build number"  (yes, weird)

Once done, in the Settings menu under the Developer options enable "USB Debugging" and connect to the PC via USB.

From a command line check if you can see the phone with "adb devices".
If none shows up, you might want to kill any running adb.exe instance from the task manager and retry.

If this still fails, open the Windows Device Manager, and look for the Nexus device right at the top under "Android devices".

If this shows an error or warning, you need to install the proper driver.
Right-click on the "faulty" android entry in the list and select "Update driver software". Point the update to the android-sdk path on your computer.
So check where you installed the Android SDK into. From the SDK path it is  under extras\google\usb_driver
(e.g.  %appdata%\..\Local\Android\android-sdk\extras\google\usb_driver)
It should find the "Android Composite ADB Interface" driver.

Once this is installed, your Nexus 5 should be visible to the debugger. Again, kill any running instance of adb.exe first, just to make sure)

If not, google is your friend :)

How sorting gets ruined.

Frequent readers - or close followers - of this blog will know, that the share feature (SEND intent) is one of my favourite features on Android.

To recap: it allows any application to share content via (or send content to) any other application, e.g. email, facebook, google+, sms/text, bluetooth, ... You name it.
All that needs to be done is for the target app to register itself for handling certain content types, and then it can be picked by the user, whenever a suitable content type needs to be sent. Android will automatically generate the list of all apps that are available for this content.

So far, so god.

Now in order to be at the top of that list, most apps "fake" and "A" into this name. But this is getting out of hand.

Let me draw your attention to exhibit A:

Exhibit A - the share menu

So this makes the alphabetical sorting totally pointless.
This has to stop.

Next app will probably register itself with an underscore....

Reminds me of my times at Sun Microsystems not too long ago. Obviously a lot of things started with S there ... Sun, Solaris, SPARC, ... but then they created lines of business called Server, Storage, Software, Services.... So even when you sorted a list alphabetically, 90% would cluster under S...

Pointless.

Say Hi to my Nexus 5

So, I finally did it.

The lack of Android updates and the (inevitable) decline in battery duration - my HTC One S was already 2 years old - made me go to the Google Play store and get a Nexus 5 (32GB). With the wireless charger.

tl;dr: happy happy happy.

Buying it
Apart from UPS's blunder on literally the last mile (the guy cannot read building numbers or name plates) the buying and delivery process was excellent. Really close to one-click :)

Basics
It came with 4.4.3 and this night updated to 4.4.4.

This time I went for un-rooted and no bootloader.

Migration
Transferring apps and data was easy...Google-Login and then the apps just re-install from the play store, 99% of the data is in the cloud anyway... The most cumbersome part was getting the login and account credentials into all the apps.
Even this was easy with LastPass on Android - expect a separate post on this.

Only for two apps I needed to move local data; one was BeyondPod, my podcast manager and player. However, this was totally easy: do a backup on the old phone, choose to share the backup via PushBullet to my Nexus 5 (also an upcoming story), within a second receive a notification and the backup on the new phone, and open it in BeyondPod. That's it. All subscriptions incl listened-to state are now on the new device.
The other one was my weight recorder, and copying was easy as well, with the help of DropBox and Astro FileManager. I'll spare you the details of the file :)

The call history, sms/text and Wifi settings I got with a Titanium XML Backup (does not require root) and restore. Not too important for call log, but great for Wifi settings.

UI Experience
So now I'm sorting out where I really miss HTC sense, but it's not a lot.
Turns out the HTC contacts, mail and calendar apps had some nice advantages over stock Android.
Also turns out, that the (stock) Google Now Launcher does not have badges anymore, so I'll miss the little red number on my email/gmail and others.

The Google Now launcher is OK, especially with the Google Now just being there.
But - by Max's recommendation - I'm now testing the Aviate launcher... Sigh... 3rd story that I need to write. Suffice it to say, it's free, by Yahoo, intergrates well, and - most importantly - guesses your context (i.e. work, home, travelling, ...) and adjusts the app/widget groups accordingly.  Let's see in some weeks how this works.

So, stay tuned for some more stories on it.

Google Now on Chrome vs. Celsius

A couple of weeks ago, I enabled Google Now notifications in my (desktop) Chrome browser.

One thing I noticed, that on the weather card the temperature for Vienna, Austria, was given in Fahrenheit, instead of Celsius. Weirdly, there was no setting for this.

My PCs are definitely all running with Austrian locale (so it should be Celsius), this particular PC even with German language (and the combination of German and Fahrenheit is rather odd). And nowhere could I change this.

My mobile devices are also all on Celsius, so I thought it could not have been picked up from there.

Oddly, however, on my other PC (laptop), it displayed in Celsius. And even more oddly, this PC had English as language.

So did some Googling and found suggestions that you need to change this on the iPhone (iPad in my case), and that maybe switching to F and back to C could work.

That's what I did. Go to Google app on iPad, then into Settings > Google Now > Weather > Temperature.
Change for Fahrenheit, save.
Repeat with Celsius.

And voila, it is Celsius on the Desktop Chrome as well ...
Weird, but part of my software developer brain actually understands this.

My Next Android?

What should my next Android be?

HTC One M8 
As much as I hate HTC for dropping further support on the One S, they still have the best hardware (IMHO) of all the Android manufacturers. As this is their flag ship product, they wont kill support for it soon. Also, there is the Google Play Edition of it, so support of future Android releases is secured.

Google Nexus 5

Plain google, good HW, maybe not as fancy as the M8 but solid. A lot cheaper (in both senses of the word). Good choice for the money, I'd say.

And I think I'd love the wireless charging.

1+ One
OnePlus seems to have fancy HW and a weird go-to-market approach: selling hardware by invitation only... Only make me want to have it. You won :)

Google Timer

Just discovered this:
Enter timer in the Google search or awesome bar of your favorite browser, and Google will present you a nice little timer app/widget.

Just then enter whatever time you want/need and start.

You can also go fullscreen, e.g. if you need to project this during presentations or so.

Cool thing is, wherever you have google, you have this timer.

There are some nice short cuts, try them:
timer 10 minutes
timer 16:30
timer 5:30 pm

Where are the Mesh Networks?

With all the (justified) uproar about the NSA and GHCQ, which should actually be an uproar against the Telcos and ISPs (and backbone providers playing along) one wonders, why there is no rush to mesh networks, giving us a bit more independence of centralized networking providers.

REALLY!?! - On the Problem of Incomplete Data

A couple of months ago, the Austrian Federal Railways (ÖBB) opened their data for Google (maps). Whether this can be already considered "open data" or not, is not todays subject.  Wiener Linien (let's call them the Vienna Transport Authority) did not.

My concern is, that Google now has the time table data of the railways, but not of the (local) public transport in Vienna (underground, tram, bus).

Which leads to incomplete data for commuters in and around Vienna (within Google Maps), or a segment, which I in reverence of SNL like to call

REALLY!?!

So, because there is no underground/metro/tube data. Incidentally ÖBB also operate the airport transfer bus, which happens to have a station right next to my office. And thus... Google Maps fills the gap in Vienna with a nice little transfer to the airport and back to my office... Because that's the only routing data (apart from walking) the find.

REALLY!?!

My regular commute time is around 50 mins.

So please, all, get your act together.

Lightning Drap'n'Drop Problem

In my earlier post (and comment discussion) about Opening ICS Files with Thunderbird/Lightning we mentioned that you can also drag the .ics link directly from the browser to the calendar pane in Thunderbird/Lightning.
I also found, that this is not always working.

So - as promised - I looked into this a bit.
First, narrow down the "not always" to something reproducible:

I took a link that I knew was working with drag&drop (actually from the timetable of the Austrian Federal Railways), and verified this. Yep, working nicely

Then I just downloaded the resulting .ics file to my local hard disk into a folder that was mapped into my webserver (Tomcat 7). I wrote a tiny .html file that mainly included a link to the .ics file.

Then open the html file in Chrome/Firefox and drag the download text/link into (or is it onto?) the Lightning calendar pane in Thunderbird. Ignore the second line for a second (no pun intended).

Nothing.

Well, that was surprising... So I went on to waste a lot of time with checking the mime-type etc etc.

But it was fine by responding with "text/calendar" - see:

So I took some Java drag&drop code I had, and took a closer look at the data included in the DnD object. Also everything fine there (I spare you the details here).

By chance I looked into the access log of the webserver and found the following:

GET /test/download.icsdownload HTTP/1.1" 404 1001

Which essentially says, that Thunderbird requests a broken URL. It should GET /test/download.ics and not /test/download.icsdownload

No wonder it gets a 404 and cannot import anything.

Quick test to include a "?" in the a href (see above, the second line I asked you to ignore earlier).
If you drag this one, you'll get a
GET /test/download.ics?download? HTTP/1.1" 200 949



As we are handling static files here, the "?download?" appended to the filename will gracefully get ignored and the file will get served (as can be seen by the 200 response code)
And with this link, Thunderbird really imports the file and opens the event dialog.

Fine there.

What Lightning does, is - wrongly - append the text of the html anchor ("download" or "download?") to the URL. You can test with naming the anchor asdfgh and will see that /test/download.icsasdfgh will be requested.

Why does it do it ?

It seems to be that Lightning picks up the HTML fragment data type that gets passed by DnD. The browser passes the full anchor, from begin tag to end tag (see above), and Lightning stupidly parses until the first newline (\n), and throws away the HTML tags.
This is wrong.
Should not happen.
I'll file a bug in bugzilla.

Google Maps Navigation now finally for Croatia

Since mid December Google Maps Navigation is finally available for Croatia.
I drive from Vienna to Zagreb every month or so (for business reasons) , and, yes indeed, I do know the way by heart now, but within Zagreb itself, it really annoyed me that I only had the maps, but no turn-by-turn navigation.

Well, this week when I went there again this week, it was just nice to have it available