Saturday, April 8, 2017

Android Upgrade Report Card: Grading the manufacturers on Nougat

Six months after Nougat's release, how have different Android manufacturers done at delivering upgrades to their devices? A sobering analysis.

Android Upgrade Report Card -- Nougat

'Tis the season for tech companies to start hawking high-end phones. Over the coming weeks and months, we'll be hearing about numerous new Android devices from practically every manufacturer imaginable.
One thing you can bet none of them will mention, though, is their commitment -- or lack thereof -- to supporting and updating those smartphones after you buy 'em. Plain and simple, that's not something most Android device-makers like to draw attention to or discuss.
Don't let that lack of focus fool you. A company's commitment to timely and ongoing software support is arguably the most important thing to consider with any Android phone purchase. The problem is that it's hard to know what to expect when manufacturers aren't upfront about their policies.
I'm here to help. I've been tracking Android manufacturers' commitments and performance with OS upgrades since the platform's earliest days -- and this year is no exception.
After all, while Google's deconstruction of Android and introduction of standalone security patches have helped make traditional updates less critical than they once were, our devices still have significant foundational improvements that only full OS upgrades can provide. Timely ongoing upgrades aren't everything, by any means, but they are without a doubt a significant and valid factor to consider.
So let's arm ourselves with knowledge, shall we? We've now reached the point where we're six months past the launch of the most recent major Android release, Android 7.0 Nougat -- and that means it's prime time to step back and look at who's making post-sales support a priority and who's treating it as an afterthought.
This Android upgrade cycle is especially interesting, as Google took the unusual step of introducing a preview version of Nougat all the way back in March of 2016 -- nearly three months earlier than it had with the previous year's release. It then delivered the final version of the software in August, a month and a half ahead of its 2015 schedule. All in all, that gave Android device-makers a full 36 more days with the software ahead of its release than they'd had with both of the previous two (Marshmallow and Lollipop) cycles.
You'd think that would mean the scores this go-round would be higher than ever. But -- well, that isn't exactly how things panned out.
Enough with the suspense. Time to see who's making the grade and who's coming up short.
(Want the full nitty-gritty on how these grades were calculated? You can find a detailed breakdown of the formula and every element taken into account at the very end of this article.)

Google

Android Upgrade Report Card -- Nougat: Googlejr
  • Length of time for upgrade to reach current flagship: 0 days (60/60 points)
  • Length of time for upgrade to reach previous-gen flagship: 42 days (27.6/30 points)
  • Communication: Mediocre (5/10 points)
With the Pixel, Google officially became a full-fledged Android phone manufacturer. But even before, with its Nexus devices, it played the same role as far as updates go -- acting as the sole provider of all rollouts and software-related matters.
And in an unusual twist, Nexus devices are actually the only ones that are relevant for this analysis. The Pixel didn't go on sale until October 2016, well after Nougat's debut -- a change from Google's typical approach of launching its flagship phone alongside its latest Android version. So for the purposes of this report card, we'll consider the Nexus 5X and 6P to be the "current" flagships and the Nexus 6 -- which came out way back in November of 2014, nearly two full years before Nougat's arrival -- to be the "previous-gen" model.
The results are mostly good, though not perfect: At least some owners of the 5X and 6P received updates on the first day of Nougat's release, when Google started its Android 7.0 rollouts. Folks with the Nexus 6, meanwhile, didn't see their update until early October -- an unusually long wait by Google's standards (unofficially caused by some lingering technical issues), though still quite respectable by all other Android ecosystem metrics.
It's worth noting that Google, rather infamously, rolls out its updates "in waves" -- which means some people end up waiting days or even weeks longer than others to receive the software. That's a deliberate process designed to keep unexpected bugs or issues from reaching large groups of users before they can be addressed. It can understandably be a source of frustration for eager device-owners in the later parts of the rollouts, but for our purposes, the start of the rollout is what counts.
The real issue with Google's update performance -- and it's one we've seen pretty consistently over time -- is that the company's communication could stand to be better. Following its initial August 22 announcement of a general Nougat rollout beginning for Nexus devices (which broadly indicated that numerous models, including the Nexus 6, would receive the software that day and "over the next several weeks"), Google didn't provide much else in terms of official info about its process. That means those users who were waiting -- including Nexus 6 owners -- were essentially in the dark, with no sign of the upgrade and no idea what was going on or how long the wait might be.
Between its less-than-stellar communication and the frustrations that sometimes result from its staged rollout process, Google's Android devices are by no means flawless when it comes to OS upgrades. Even so, they stand in a league of their own -- without question the most reliable way to receive ongoing updates in a timely, if not always immediate, manner.

HTC

Android Upgrade Report Card -- Nougat: HTCjr
  • Length of time for upgrade to reach current flagship: 95 days (47.4/60 points)
  • Length of time for upgrade to reach previous-gen flagship: 105 days (23.7/30 points)
  • Communication: Fair (6/10 points)
HTC is still solidly in second place when it comes to upgrade reliability -- a position it's held since 2015's Android 5.0 Lollipop release. That's certainly commendable in terms of placement alone. At the same time, however, this year marks the company's first drop in score in the past two years, with worse performance on both delivery time (for its current and previous-gen flagship) and communication.
After two years of promising growth, that's disappointing to see. And while a 77% score may be good enough to slip into second place in this lackluster lineup, a 9-point drop from last year isn't exactly spectacular.
HTC started rolling out Nougat to the unlocked version of its 2016 HTC 10 flagship on November 25, just over three months after the software's release. Though the carrier-connected models of the device didn't start seeing their rollouts until early January, the unlocked 10 was readily available for purchase from HTC in the States -- and so its rollout counts as the first time the software became available to U.S. consumers.
That being said, it's worth noting that the Sprint model of the 10 just started getting its Nougat update earlier this week, while the Verizon model has yet to be updated as of today. It's a good reminder that given the option, going with a carrier-connected phone model is rarely the best choice for speedy OS upgrades.
(Another reminder, especially if you're wondering how I decide what date to use for this grading and why: You can find a detailed discussion of the scoring system at the end of this analysis.)
HTC's previous-gen One M9 flagship started receiving Nougat in early December, about three and a half months from the time of Nougat's arrival. That was the unlocked model; rollouts for carrier-specific variants are said to be in the works for sometime "early" this year.
In terms of communication, HTC is still doing better than most. To its credit, the company did confirm which of its devices would be getting the Nougat update way back in June, well before the final Android 7.0 software had even arrived. It then alerted customers via Twitter as rollouts got underway for various models of its phones but provided no further official guidance about progress or delays along the way.
What's particularly vexing about that is the fact that HTC used to maintain an exemplary "Software Updates" page, with detailed info on the update status of every model and variant in its catalog (complete with a huge chart explaining every step of the process so you could follow along as things progressed) -- but that page appears to have been abandoned sometime over the summer. As of this writing, it still has a vague message at its top about the "upcoming" launch of Google's Android 7.0 software, and its phone-by-phone listings show only long-out-of-date details about the Android 6.0 update. (The HTC 10 isn't even included as an option.)
It's a far cry from the level of excellence HTC once maintained in this department -- and while there's something to be said for being the only non-Google phone-maker to earn a satisfactory grade, it's still a shame to see the company sliding backwards in its efforts.

OnePlus

Android Upgrade Report Card -- Nougat: OnePlusjr
  • Length of time for upgrade to reach current flagship: 131 days (43.2/60 points)
  • Length of time for upgrade to reach previous-gen flagship: 131 days (21.6/30 points)
  • Communication: Poor (0/10 points)
With its popularity among Android enthusiasts and its bold proclamations of its phones being flagship-caliber (and then some), I thought it only made sense to add OnePlus into this analysis despite its niche-level status as an Android player. The results, unfortunately, aren't great.
The company's two most recent flagship phones -- the OnePlus 3 and OnePlus 3T, both released in 2016 -- started to receive the Android 7.0 update on December 31. That's just over four months past the date of Nougat's release.
Some reports suggest that the initial build of the software was buggy on at least one of those devices and that OnePlus may have stopped its rollout prematurely as a result. An updated version of the software started rolling out on January 20. For the sake of consistency, I'm sticking with the first rollout date for the official score -- since that's when at least some owners of both devices received the software -- but truthfully, with either date, we're looking at an underwhelming "D"-level grade. It almost doesn't matter.
OnePlus's pokey progress isn't aided by its poor communication with customers. The company didn't provide much in the way of official guidance, with the first indication of timing and availability coming by way of a random report by a blog in November. Even the company's Twitter account has remained mostly silent on the subject, aside from a single tweet touting Nougat's availability in early January.

Motorola (Lenovo)

Android Upgrade Report Card -- Nougat: Motorolajr
  • Length of time for upgrade to reach current flagship: 88 days (49.2/60 points)
  • Length of time for upgrade to reach previous-gen flagship: Still waiting (0/30 points)
  • Communication: Poor (0/10 points)
Oh, Moto. Moto, Moto, Moto. The company once lauded for its outstanding Android upgrade performance just keeps getting worse with every passing year. (So much for that "nothing will change" transition in ownership, right?)
Under Lenovo's wing, Motorola went from being at the head of the class to earning an ever-slipping range of embarrassing scores. It got a 65% "D" grade in 2015, a 52% "F" grade in 2016, and now -- a mere three years after Lenovo took the company's reins from Google -- it's down to a shameful 49% failing score.
The first Moto Nougat update arrived to the Verizon version of the 2016 Moto Z flagship on November 19. (In an odd twist, the unlocked version of the phone -- which, logically, should receive updates ahead of or at least at the same time as any carrier-connected model -- didn't start getting its Android 7.0 update until earlier this month. Again, though, we'll use the first-available date for our official score.)
As for the previous-gen Moto X Pure flagship, six months after Nougat's launch, it's still waiting. For Motorola in particular, with its near-stock Android software and its history of excellence in updates, that's just inexcusable.
Equally inexcusable is the lack of any meaningful communication from the company about its Android update plans or progress. All in all, it's painfully clear this isn't the same organization it was a few years ago.

LG

Android Upgrade Report Card -- Nougat: LGjr
  • Length of time for upgrade to reach current flagship: 91 days (47.4/60 points)
  • Length of time for upgrade to reach previous-gen flagship: Still waiting (0/30 points)
  • Communication: Poor (0/10 points)
After some sloooooow but steady improvements over the past couple of years, LG is yet another company seeing a slip in its Android upgrade performance -- and its resulting score -- for the 7.0 Nougat cycle.
The company's first U.S. Nougat rollout was with the Sprint model of its G5 flagship on November 21, about three months after Nougat's debut. The T-Mobile model followed about a week later, with the Verizon version coming in a couple days after that and the AT&T G5 trailing in earlier this week.
(Not working in its favor: LG doesn't sell unlocked versions of its phones in the U.S., so there's no carrier-free option available to speed things up. That being said, the company's first international rollout for unlocked phones was only slightly ahead of its U.S. schedule, with a starting date of November 8.)
LG still hasn't made a peep about its plans for the previous-gen G4 flagship, meanwhile. Forums around the web are filled with messages from frustrated G4 owners asking for any info about if or when Nougat might be headed their way.
And that gets at LG's other persistent upgrade downfall: its essentially non-existent efforts at communicating with its customers. Aside from its traditional self-serving press release boasting about a symbolic limited-scope rollout ("FIRST!!!!!!"), the company has stayed silent on both its plans and its progress.
All in all, it's not a situation worth celebrating. And, believe it or not, things only get worse from here.

Samsung

Android Upgrade Report Card -- Nougat: Samsungjr
  • Length of time for upgrade to reach current flagship: 179 days (37.2/60 points)
  • Length of time for upgrade to reach previous-gen flagship: Still waiting (0/30 points)
  • Communication: Poor (0/10 points)
Samsung has never excelled at Android upgrades, but this year marks -- yup, you guessed it -- a new low for the company. The first sign of Nougat on a U.S. Samsung flagship showed up only last week, with the rollout of Android 7.0 for the T-Mobile Galaxy S7 and S7 Edge. That's just a hair less than six months since Nougat's release.
The AT&T and Sprint versions of the phones saw the software within a few days of that, meanwhile, while the Verizon version is still waiting as of this writing.
Samsung's previous-gen flagships -- both the 2015 Galaxy S6 and 2015 Galaxy Note 5 -- are also still awaiting their Android 7.0 upgrades.
I should note a few things here: First, I'm aware that Samsung offered a beta program this year that allowed users of certain devices to try out an early build of the software. That's a positive step -- but beta software isn't what we're assessing here. We're looking at final, official software delivered to regular consumers.
Second, I typically include both the Galaxy S and Galaxy Note as co-flagships for Samsung, since the company essentially treats them as equal in flagship status. Since the Note 7 was cancelled and recalled this year, however, that obviously isn't relevant or possible for the current-year flagship category.
And finally, Samsung, like LG, does not officially offer unlocked models of its phones to consumers in the States -- which is a shame, as its devices did experience faster rollouts internationally. Well, to a degree, anyway: Even the international unlocked Galaxy S7 didn't see Nougat until late January, nearly five months after the software's release. (Correction: Evidently, Samsung did actually offer an unlocked S7 model specifically for the U.S. market this year. My mistake. That phone, however, has yet to see a Nougat upgrade -- so...yeah.)
On the communication front, Samsung continued its policy of keeping its customers mostly in the dark about its upgrade plans and progress. Once upon a time, the company did actually provide detailed info about that stuff -- but after it repeatedly failed to keep up with its promises (and,ahem, received some strongly worded negative publicity over the way it handled those failures), Samsung sealed its corporate lips firmly shut. And it's kept 'em mostly glued together ever since.

BlackBerry

Android Upgrade Report Card -- Nougat: BlackBerryjr
  • Length of time for upgrade to reach current flagship: Still waiting (0/60 points)
  • Length of time for upgrade to reach previous-gen flagship: Still waiting (0/30 points)
  • Communication: Poor (0/10 points)
It's now six months since Nougat's release, and neither BlackBerry's 2015 Priv flagship nor its curiously named 2016 DTEK60 flagship has seen any sign of the software. Worse yet, the DTEK60 actually launched in October -- two months after Android 7.0's arrival -- and yet still shipped with 2015's Android 6.0 software out of the box.
On top of that, BlackBerry hasn't provided a word of official info to its customers about what's going on -- and if or when their devices will see last year's "new" Android software.
It's the same situation we saw with BlackBerry last year. The company has, by most counts, done a commendable job of getting Google's monthly security patches out to its Android phones quickly -- and there's absolutely something to be said about the significance of that. When it comes to actual OS updates, though, BlackBerry has been consistent only with its disappointing performance.

Closing thoughts

The best way to sum this up, I think, is to borrow a line from my past report cards: We can -- and should -- do better.
But you know what? At the end of the day, that's not up to us. You and I can't control what manufacturers do or how much of a priority they make timely and ongoing software support. All we can do is educate ourselves about their practices, decide how much that matters to us, and then make our future purchasing decisions accordingly.
I've said it before, and I'll say it again: With Android's open nature and the level of diversity that allows, Android OS upgrades are never going to be completely consistent across all devices. That's par for the course. I often say that Android presents you with a lot of choices, and if quick and regular upgrades are important to you, you most certainly can have them. You just have to choose a phone that provides that type of experience.
Google's own Android devices -- first Nexus and now Pixel -- continue to be the most reliable options. They truly are in a league of their own. If you were to make any comparison to iOS, Google's phones would be the closest equivalents to iPhones in that their software, updates, and entire user experiences are controlled completely by a single company. It's a holistic arrangement, and the end result speaks for itself.
Continuing the trend from the past couple years, HTC is about the only other manufacturer making timely OS upgrades any sort of priority. Its speed and communication have both slipped this cycle -- which is disappointing to see, given its impressive improvements over my past two report cards -- but compared to basically everyone else out there, it's at least still making somewhat of an effort.
And as for everyone else, what can you say? Regardless of who's to blame, there's no excuse for keeping customers in the dark for months while flagship phones sit idle. This year, the manufacturers had more time than ever to prepare -- and yet they somehow still took longer than ever to deliver. At the end of the day, it's up to each manufacturer to decide what level of resources it wants to devote to the upgrade process and how it wants to treat customers along the way. That reality has never been more apparent than now.
The one bit of reassuring news is that, as always, the power is ultimately in your hands. We may not be able to make manufacturers do better, but we can make ourselves educated consumers -- and then avoid down-the-road disappointments by making the right buying decisions for our own personal needs.

In detail: How these grades were calculate
This report card follows the same grading system used with last year's upgrade analysis -- which features precise and clearly defined standards designed to weigh performance for both current and previous-generation flagship phones along with a company's communication efforts.
Each manufacturer's overall grade is based on the following formula, with all final scores being rounded up or down to the nearest full integer.:
  • 60% of grade: Length of time for upgrade to reach current flagship phone(s)
  • 30% of grade: Length of time for upgrade to reach previous-gen flagship phone(s)
  • 10% of grade: Overall communication with customers throughout the upgrade process
Upgrade timing often varies wildly from one country or carrier to the next, so in order to create a consistent standard for scoring, I've focused this analysis on when Android 7.0 first reached a flagship model that's readily available in the U.S. -- either a carrier-connected model or an unlocked version of the phone, if such a product is sold by the manufacturer and readily available to U.S. consumers.
(To be clear, I'm not counting being able to import an international version of a phone from eBay or from some random seller on Amazon as being "readily available to U.S. consumers." For the purposes of creating a reasonable and consistent standard for this analysis, a phone has to be sold in the U.S. by a manufacturer or a carrier in order to be considered a "U.S. model" of a device.)
By looking at the time to Nougat's first appearance (via an over-the-air rollout) on a device in the U.S., we're measuring how quickly a typical U.S. consumer could realistically get the software in a normal situation. And we're eliminating the PR-focused silliness of a manufacturer rushing to roll out a small-scale upgrade in somewhere like Lithuania just so they can put out a press release touting that they were "FIRST!" The same analysis could be done using any country as its basis, of course, and the results would vary accordingly.
All measurements start from the day Android 7.0 was released into the Android Open Source Project: August 23, 2016, which is when the final raw OS code became available to manufacturers. The following scale determined each manufacturer's subscores for upgrade timeliness:
  • 1-14 days to first U.S. rollout = A+ (100)
  • 15-30 days to first U.S. rollout = A (96)
  • 31-45 days to first U.S. rollout = A- (92)
  • 46-60 days to first U.S. rollout = B+ (89)
  • 61-75 days to first U.S. rollout = B (86)
  • 76-90 days to first U.S. rollout = B- (82)
  • 91-105 days to first U.S. rollout = C+ (79)
  • 106-120 days to first U.S. rollout = C (76)
  • 121-135 days to first U.S. rollout = C- (72)
  • 136-150 days to first U.S. rollout = D+ (69)
  • 151-165 days to first U.S. rollout = D (66)
  • 166-180 days to first U.S. rollout = D- (62)
  • More than 180 days to first U.S. rollout (and thus no upgrade activity within the six-month window) = F (0)
There's just one asterisk: If a manufacturer outright abandons any U.S.-relevant models of a device, its score will default to zero for that specific category. Within that specific category (be it current or previous-gen flagship), such behavior is an indication that the manufacturer in question could not be trusted to honor its commitment and provide an upgrade. This adjustment will allow the score to better reflect that reality. (No such adjustments were made this year, though there was one instance where it happened in the past.)
Last but not least, this analysis focuses on manufacturers selling flagship phones that are relevant and in some way significant to the U.S. market. That's why Sony is no longer part of the analysis and why BlackBerry (which is certainly trying to make a major play for American consumers and is a company plenty of people are watching) and OnePlus (which is quite popular among Android enthusiasts and explicitly touts its phones as being flagship caliber) are.
Most of the other players are either still relatively insignificant in the U.S. market or have focused their efforts more on the budget realm in the States so far -- and thus don't make sense, at least as of now, to include in this flagship-focused report.
That's All Folks

No comments:

Post a Comment