Flagfox slow performance question [RESOLVED]

Any feedback, suggestion, bug reports, problems....
Locked
thomgarv
Posts: 4
Joined: Wed Mar 03, 2010 9:27 pm

Flagfox slow performance question [RESOLVED]

Post by thomgarv » Mon Apr 04, 2011 5:38 pm

Hi,

Just seen that according to this Slow Performing Add-ons page, Flagfox makes Firefox start 16% slower than normal. Is this just due to the size of the add-on? Or maybe I should first ask, is this even really true?

Thom

User avatar
DaveG
Flagfox Developer
Posts: 723
Joined: Wed Oct 03, 2007 9:06 pm
Location: Philadelphia, USA

Re: Flagfox slow performance question

Post by DaveG » Tue Apr 05, 2011 6:35 am

First of all, I'm a little confused as it said 12% when the page first launched and no Flagfox update was posted since then. I have no clue where that new +4% came from, and since it doesn't tell you what is slowing down the startup, I can't use any of this info to make anything better.

Secondly, the page has a few problems; more precisely, its explanation is wrong. It says it's listing the addons with the largest startup slowdown, which it does not. It left out a rather crucial bit of information that is only stated on the blog post announcing the new list. The list is only out of the top 100 most used extensions, showing the 50 with the most startup time impact (10 until you click "more"). There are plenty of other addons that may have far worse of a performance impact on startup, but if they're not in the top 100 they're not listed at all there. This leads to the false implication that these on the list are all bad, whereas the average is around 10% (stated in the same blog post) and their stated ideal is around 5%. I filed a bug for the bad explanation on the page, but as of yet haven't had a reply.

As to what the startup impact actually is for Flagfox, even on an older computer it's not really noticeable. Mozilla is trying to point out the cumulative effect of installing lots of addons, each with a negligible impact but adding up to a larger issue. One of the other problems here is the difference between cold and warm startups. I'm pretty sure the list is measuring cold startups (startup right after boot of computer) and not warm startups (startup after previously having loaded before and still waiting in memory). Flagfox has a much faster warm startup than cold, as does everything, but most of the impact being measured here is in cold startup I suspect. (actually, for all I know it's measuring startup on first install, which would be quite wrong as that would count slowdown from the installation itself) Though, as I said above, they aren't actually giving me any real information on this, which is a bit annoying.

I'm fairly certain that even with some possible problems with the numbers that Flagfox may have an above average startup impact, but not a very large one. I did improve things a little bit in some past updates and I'd love to try and find ways to reduce it more, but I'm already following their stated "best practices" guidelines and have done my own testing to see that what I do on Flagfox startup doesn't impact much of anything. The big hit is probably coming from loading of the files themselves, not by anything in my code. I'd like to try a few things, but I can't for the life of me get consistent enough testing results to try new things and see what helps. In my testing, Firefox startup times vary so much that I can't measure anything, and testing cold startup in any remotely reliable way, i.e. rebooting constantly, is a nightmare. They say at some point they'll add the ability for developers to upload a version to run through the test they're using for their performance page, which would help, but that's not available yet.

User avatar
DaveG
Flagfox Developer
Posts: 723
Joined: Wed Oct 03, 2007 9:06 pm
Location: Philadelphia, USA

Re: Flagfox slow performance question

Post by DaveG » Sat Apr 09, 2011 5:38 am

Now the page says 16% for Flagfox, again, with no changes. It appears to be giving bad results. Wladimir Palant is looking into this and hopefully some people at AMO will figure out what's going on soon.

User avatar
rleeden
Site Admin
Posts: 117
Joined: Mon Oct 01, 2007 3:13 pm
Location: United Kingdom
Contact:

Re: Flagfox slow performance question

Post by rleeden » Sun Apr 10, 2011 8:36 am

Back to 14% today!

User avatar
DaveG
Flagfox Developer
Posts: 723
Joined: Wed Oct 03, 2007 9:06 pm
Location: Philadelphia, USA

Re: Flagfox slow performance question

Post by DaveG » Sun Apr 10, 2011 9:36 pm

Wladimir Palant is doing a wonderful job testing this new system and filing bugs. His list of bugs is here, with 13 filed by him, the one from me, and now a new tracking meta-bug to collect them all and a couple new ones.

There are quite a few problems with their testing system, so for now the numbers are basically useless until many of them get fixed.

thomgarv
Posts: 4
Joined: Wed Mar 03, 2010 9:27 pm

Re: Flagfox slow performance question

Post by thomgarv » Mon Apr 11, 2011 8:12 pm

Thanks for all the feedback on this Dave. You confirmed by suspicions that things weren't quite as cleare cut as they seemed.

I've seen that Wladimer has updated his blog with more information and has also put together an overview page for AMO’s add-on performance testing results. Interesting to see the raw figures and that on average Flagfox slows down my Windows 7 machine my 66 milliseconds at startup. Seeing the percentage I'd assumed it was going to be a bit more dramatic than a mere 66 milliseconds! I think I can cope with that somehow ;)

User avatar
DaveG
Flagfox Developer
Posts: 723
Joined: Wed Oct 03, 2007 9:06 pm
Location: Philadelphia, USA

Re: Flagfox slow performance question

Post by DaveG » Mon Apr 11, 2011 8:47 pm

Yes, I just read Wladimir's latest post and took a look at his new chart. If you check the tooltips for the color-coded cells in the chart, it even says which bug is causing that number to be wrong. On Windows Flagfox and many other addons have their numbers inflated because the testing system is handling internal JARs wrong, so that 66ms measurement is higher than what it really should be reading. (these are most of the red cells in his chart) Also, Flagfox is affected highly by whatever problems are causing inconsistent readings, which we already noticed. (these are the yellow cells in his chart)

User avatar
DaveG
Flagfox Developer
Posts: 723
Joined: Wed Oct 03, 2007 9:06 pm
Location: Philadelphia, USA

Re: Flagfox slow performance question

Post by DaveG » Mon Apr 11, 2011 9:06 pm

Also, the bug I filed has been fixed. Whenever they release the next set of updates to the AMO slow addons page, it'll say "Add-ons with Slow Start-up" instead of "Add-ons with Slowest Start-up" and "a significant impact" instead of "the most impact". The page will also no longer list any addon with a tested impact less than 25%, so Flagfox won't be on it anymore.

Edited to add: The updates to the page are live. There's now a grand total of 6 addons listed on the AMO page. Only the worst offenders now.

User avatar
DaveG
Flagfox Developer
Posts: 723
Joined: Wed Oct 03, 2007 9:06 pm
Location: Philadelphia, USA

Re: Flagfox slow performance question

Post by DaveG » Sun May 01, 2011 7:06 pm

They must've fixed a few more bugs in their testing system. While Flagfox still isn't listed on the official page because it's way less than a 25% hit, it's still listed in the full results list from Wladimir. It's now showing at 11.2% down near the bottom of the list of tested addons that didn't break the test outright (which is still most of them). There are a few single digits below it, but some of them are bugs; not sure how many. There are still a couple yellow cells for Flagfox meaning bugs still affecting things, though.

Locked