FERDY CHRISTANT - AUG 31, 2014 (11:41:14 AM)
As a web developer, I am very much aware and pro-active in performance tuning websites. JungleDragon uses many techniques to optimize for a speedy experience, both in the back-end and the front-end:
- On the back-end, the most important database queries are tuned to the bone.
- Images are large and retina-ready, yet still light in file size
- The number of HTTP requests is minimized
- Static resources are compressed and heavily cached
- Resources are loaded in a smart order, avoiding blocking requests where possible
But...there's always room for further improvement, and I found quite a spectacular one that I will be telling about in this post. Last week, this A List Apart article popped up into my RSS reader. It details 3 relatively new techniques of prebrowsing. I had heard of the term only once before, but this article triggered me.
Prebrowsing is smartly loading resources up front, based on a prediction of what the user is going to do next. The article discussed 3 ways to do this:
- DNS prefetching
- Resource prefetching
- Page prerendering
Having studied the article in detail, it had me thinking of how this possibly can be applied to JungleDragon. I have taken a deep dive into these 3 techniques, and now I am using all 3 of them. I'm not going to repeat the article here, rather I'm going to share how I've implemented them, and what the effect is.
Many websites use secondary domains to load resources. For example, one can have a img.mydomain.com to load images for the site. For browsers to load images from that sub domain, it needs to resolve the domain, translating it into an IP address. The way this occurs is complicated, it depends on the browser's strategy, browser cache settings, and DNS cache across many hops in the chain.
Resolving a domain can take from 1ms to several seconds, depending on these factors. Without cache, say a "fresh" resolve in practice will likely take a few dozen ms at least. DNS prefetching aims to do this resolve action upfront. Here's what I placed in the head section of every JungleDragon page:
<link rel="dns-prefetch" href="http://static.jungledragon.com">
<link rel="dns-prefetch" href="http://media.jungledragon.com">
<link rel="dns-prefetch" href="http://fonts.googleapis.com">
This prefetches the domains for JungleDragon static resources (JS, CSS), photos, and the header font. Unfortunately, it is nearly impossible to objectively measure the performance effect of this technique. For example, Chrome has its own intelligence in DNS pre-fetching regardless of what any web page does. When starting up Chrome, it will automatically DNS prefetch the 10 most visited sites you use.
If my theory is correct, this should avoid the high cost of DNS lookups in those cases where it wasn't cached. This should be a saving a few dozen ms or more. But as said, it's hard to measure. It is cheap to implement though, and a proven technique on websites such as Amazon.
Browser support: currently Firefox and Chrome, and perhaps a few smaller browsers.
A more spectacular performance improvement may be found in resource prefetching. A resource would be a file (JS, CSS, img) in this case, although it can be used for pages as well. You would use this technique when it is very likely that a user will need that resources on the next action or page. By prefetching it, it will instantly be ready when that next action occurs.
With resource prefetching, you are giving the browser a hint, not an instruction, to load that resource. It will typically be loaded during browser idle time, thus the preloading in no way blocks current page loading and rendering.
I found a very interesting case in JungleDragon where using this technique makes sense: slideshows. In a slideshow, it is extremely likely that users will navigate to the next image. Have another look at the opening screenshot of this blog post. That's the JungleDragon slideshow. It features one big image, followed by 9 small thumbs.
In the before situation, navigating to the next image would start loading that next image only on the spot. The next image was not loaded yet, thus there is a wait time. That wait time even in good conditions may be 2 seconds or more. Whilst that is not a huge delay, it is still a bothersome delay.
In the after situation, all 9 large images are prefetched, as it is very likely that they are needed later on. As said, this prefetching is non-blocking, it delays nothing on the current page. Here's a small video demonstrating the slideshow with prefetching enabled:
As you can see, next (and previous) images load instantly. That's because they're ready before you clicked on them. It is a dramatically different slideshow experience because of this.
How is this done? It's as simple as inserting prefetch links like these:
<link rel="prefetch" href="http://media.jungledragon.com/images/1234.jpg" />
Any prefetching article will warn developers not to overuse this technique, as it may consume a lot of bandwidth that isn't actually always used. I agree with that, but I find the slideshow scenario a perfect example of where this technique does make sense.
Browser support: currently supported by most modern browsers. Note though that these are browser hints, not browser instructions. For example, Chrome is much more aggressive in taking the hint, opposed to Firefox, which prefetches more slowly.
The 3rd and last prebrowsing technique is the most powerful, but also the most controversial and risky. With page prerendering, currently only supported in Chrome and IE11, an entire page with all its resources is prefetched and prerendered. It is as if a user is previsiting the next page entirely. The browser has that next page fully ready, it's simply hiding it until the user actually decides to visit that next page.
This technique is controversial because:
- Bandwidth misuse in case the user does not visit the page you predicted
- It can skew website statistics
- I am not sure, but perhaps it also messes with ad mechanisms
- If the preloaded page contains malware, you're actively taking away end-user control
This technique is to be used with great care, so let's see what it can do for JungleDragon. JungleDragon has no ads and serves no malware, so that risk is not applicable. Bandwidth misuse is the only potential risk. I decided to give this technique a try as an experiment, and the results are so awesome that I am going to stick with it.
What is the use case? JungleDragon on many pages shows a list of things. A list of photos, a list of members, a list of tags, and so on. In these lists of "things", a fixed amount of entries are shown per page, followed by a "more" button to load the next set.
And here is the very simple idea: why not preprender the page behind the "more" button? What would that do? Before we dive into the risks, let us first check the performance benefit. In this video, I'm navigating these "more" buttons on lists of photos, members and tags:
This is quite incredible. Normally, when loading a new page of thumbnails, you would see 24 grey boxes, followed by a few seconds of the images loading one by one. With prerendering, performance is instant. Note that this concerns 24 retina-size thumbnails, appearing in a snap. As if you're navigating locally, without network. That's because we are in fact navigating locally, all network work has been done already because of prerendering.
The 2nd part shows me navigating lists of members. As the user avatars come from gravatar.com, this usually is a page that slowly builds up, since it makes many requests to gravatar.com. With prerendering, it is instant. And finally, you can see that the list of tags also loads instantly.
What I'm doing here is a pretty aggressive form of prerendering, so let us see what any negative side effect may be. The main concern in this case is what happens if the user does not use the "more" button. In that case, I have prerendered it needlessly and bandwidth is wasted.
The heaviest use case are lists of photo thumbnails. The bottom line here is that the pre-rendering on average adds about 400KB in bandwidth costs, for loading the images of the next page. On mobile, I load smaller images, so the costs is far less. The loading of the extra 400KB, whether needed or not, is non-blocking, thus there is no wait costs, only bandwidth cost.
I find that number, 400KB, to be very acceptable. Note that despite being image heavy, JungleDragon image pages even with prerendering enabled is still below the average web page size of the world (1.6MB). After a first visit to JungleDragon, a full image page with prerendering drops to only 800KB, due to heavy caching.
Given the extreme performance benefit and the very reasonable non-intrusive side effects, I'm going to keep this prerendering experiment for now.
DNS prefetching, resource prefetching and page prerendering are extremely powerful techniques that when used with care, can dramatically speed up select parts of your web site and application. They are little known features of the modern web, so I hope that my article on applying these techniques in practice inspires you to investigate these techniques as well. My recommendation for now:
- DNS prefetching: always use it. Easy to implement and no negative cost.
- Resource prefetching: use in select cases of highly likely "next actions"
- Page prerendering: use in select cases of highly likely "next actions"
In all cases, be sure to measure the effect of your experiments. And to make matters just a little more complicated, note that some browsers cancel prefetch actions when using their inspector window.
FERDY CHRISTANT - AUG 22, 2014 (01:26:02 PM)
Another week, another series of JungleDragon site updates, such has been the rhythm for a while now. This week is no different, so let's see what changed this time:
Style upgrade for activity feeds
I changed the look of items in the activity feed, which are found on many pages inside JungleDragon:
In the new look, the items themselves do not have a background of their own, they simply lay on the page, which has a general background. I find this less distracting and boxy. The text has a light shadow giving it an inset effect. Padding has been improved and made more consistent. User avatars are slightly rounded, but not much.
In JungleDragon it is possible to make your own lists of photos, and include any photo on it (by you or others), based on a theme of your choice. A good list has at least these qualities:
- It has a proper title and description
- It has a specific weight, a minimum amount of photos
- It concerns an original dimension of photos, and interesting topic
JungleDragon will now actively guide list owners in creating a useful list by providing guidance:
It concerns the block in red. It will only be visible to the list owner, and only when signed in. As you can see, essentially it is a todo-list where items taken care of are striked through. If the list meets all 3 conditions, it will dissapear.
The idea of more proactive quality guidance will eventually also be applied to individual photos, but in a different way.
Promotion awards and notifications
At JungleDragon, moderators decide which photos shine on the homepage, by means of photo promotions. I made some changes in this area. The first change concerns karma. As of now, if your photo is promoted, you will receive 100 karma points. More importantly, your image will also receive 100 karma.
This karma reward on the image itself, means that your image will appear "higher" in several areas of JungleDragon:
- On the popular photos overview
- Inside tags
- On species pages
Getting your photos promoted is now much more rewarding, it goes beyond being listed on the homepage. Note that user and image karma for promotions is only rewarded:
- When it does not concern a self promotion
- When it is the first time the image is promoted, not during repromotions
The 2nd change regarding promotions is that it now also shows up on activity feeds:
This means it is now fully transparant which moderators promotes which image. This activity item does not appear on the general community feed, it only appears on the feed of individual users, moderators in this case.
And the 3rd change is that a new notification is linked to the event of images being promoted:
Above is a test email of the email that a photo owner would receive if his/her photo is promoted. This email is only sent when the promoter and image owner are different persons.
That's it for this week, I hope you appreciate these changes.
FERDY CHRISTANT - AUG 15, 2014 (02:41:07 PM)
Hereby the 3rd site update for JungleDragon this month. This is only a small one, but every improvement counts.
How fast JungleDragon is for you depends on many parameters: your location, connection speed, the load on the server, etc. In turn, the total page load time depends on multiple components as well: DNS, image download time, database query time, etc. It's a complicated puzzle.
In this update I focused on improving the database performance of JungleDragon's most important pages. With great success:
Above is diagnostic output of JungleDragon's homepage, which requires 5 database queries. This is the live site. As you can see, each query now only takes 5ms or less. I've done this on many pages, often the total time spent on the database for the entire page is in the range of 20 - 50 ms, which are really good numbers.
As explained, the database time does not dictate the total load time of a page. Most time will be spent now on the downloading of images, which is something I cannot speed up much further without adding substantial costs. It is also something one can except from an image-heavy site like JungleDragon. To put that in perspective:
The average web page size as per July 2014 of the top 1000 websites in the world is 1.6MB, and some expect this to grow to over 2MB per page by the end of 2014. JungleDragon's homepage, one of the heaviest pages on the site, is only 1.2MB in size, and much smaller even when you revisit with a full cache. And to add to that, realize that this homepage loads 24 large photos, in Retina-like quality.
Even on mobile, JungleDragon is doing well. Sure, if you're on a slow mobile connection, its' speed will not blow your mind. But even that can be put in perspective: nothing will blow your mind in such a situation. For example, opening Facebook in such conditions takes me twice as long as opening JungleDragon.
I believe JungleDragon is optimized for speed quite well by now, but I'll always look for opportunities to improve it further, as long as they are reasonable.
It's a small improvement, but on photo pages next to the capture date, you can now also see the capture time:
It's an interesting piece of information, for example to see how early the photographer was up, and what that means for the light we see on the photo. Note that this capture time is coming from EXIF, thus it comes from your camera. If the date/time on your camera is wrong (often happens on travel), so will this time be. Furthermore, no time zone is added, since most cameras have no idea in which timezone they are, thus they do not capture that information.
Related to the info panel on photo pages:
If your camera is GPS-enabled and embedded both the GPS coordinates and time in the photo, in JungleDragon a special link will appear below the map "View light conditions of this photo". This was already implemented before, but when clicking this option, the header is now different:
See above, the header can be seen in light brown. Coming from a photo, this header shows a different instruction compared to visiting the daylight planner directly. A subtle change, but it provides more guidance this way.
- Layout issue on "load original" button in full screen viewer: fixed.
- Layout issue on daylight controls for long dates: fixed.
- Breadcrumb navigation for species browser further improved for small screens
FERDY CHRISTANT - AUG 9, 2014 (10:00:56 AM)
It's good times at JungleDragon. We're seeing fantastic contributions by members, and we're crossing new content milestones rapidly. As a result, I am eager to keep improving the site. Here's the 2nd site update in August.
Slideshow titles and ambient backgrounds
My obsession with full screen viewing has not yet ended. I've improved slideshow viewing in particular this time. A slideshow is when you open any series of images in JungleDragon in full screen mode (by clicking the "play" button).
There's 2 improvements. The first one is the slideshow title. Before there was none. Typically, you'd be in JungleDragon on a page showing a series of photos, and then open it as a slideshow. That way you'd know what type of slideshow it is. However, what if you would open a slideshow link directly, from outside of the site? Except for the window title, there's no way to know what slideshow you're really playing. This is now changed:
(click to enlarge)
See above. Note the new slideshow title "Marvelous Macros" in this example. As you can see, it is in the same font as the JungleDragon logo, clearly separating it from the photo title in the bottom. In this case, we're playing a slideshow based on a custom list made by a member. Here's a second example, this time playing a slideshow of all photos of a particular species:
(click to enlarge)
The 2nd slideshow improvement can be seen from the above screenshots as well: ambient backgrounds. Ambient backgrounds dynamically change the photo background based on the average color of the actual photo. Before I had it enabled only on single photo full screen viewing, not it is also enabled on slideshows.
Other bits and pieces
The other updates for this week consist of several smaller items:
- Fixed some typos in the moderation panel
- Fixed an issue where the "share" button does not work on species maps
- Fixed an issue in Chrome which would truncate the species navigation breadcrumb too aggressively
- On touch devices, opening a photo full screen required an extra click/tap, as the device first shows the hover state of the element. This is now fixed.
- I've removed the social footer from email notifications. I find them distracting and I rather have you visit the actual JungleDragon site, instead of our social channels. You can still find the social links in the site footer.
- Lists (member-created collections of photos) are now sorted by the most recently added photo. In particular this makes the bigger/most popular lists more dynamic, easier to see what's new
- Fixed an issue where the species photos and species photos slideshow have a different sorting
- Both the forum body field and the photo description field now allow you to post much more text, 10 times more than before.
- I've improved the speed of the homepage by 0.5s. Doesn't seem like much, but every bit helps.
FERDY CHRISTANT - AUG 1, 2014 (08:51:18 PM)
Here's another few updates to JungleDragon.
Ambient Glass aftermath
Last monday I deployed a crazy fun new update to JungleDragon: Ambient Backgrounds. In a nutshell, these concern dynamically calculated background colors based on the photo you are viewing full screen. It's a feature that got me quite excited.
However, frustration followed after the initial deployment. It didn't work as well as it did on my test environment. The reason: CORS, differences in browser caching mechanisms and event orders, things you don't really want to know.
Luckily, I was able to develop a fix, and now ambient backgrounds not only work reliably, the calculation part now also works in far more browsers, including mobile browsers.
It's a tiny change, but I'm further deploying the "glass" theme. As of now, all thumbs have it applied to their caption:
JungleDragon's footer is now dark instead of light, more clearly marking the end of the page:
Please view me full screen
When on a photo page and hovering the actual image, a hint appears in the bottom of the image, emphasizing that you can launch it full screen (as some people may not know that):
And yes, again it is themed like "glass". Let's click the image and open it full screen:
Note the ambient background, chosing blue based on the photo. New in the full screen viewer is the hint in the top bar, which says "Use your mouse wheel to zoom in and out, drag to pan around". Not everybody knows that JungleDragon has a desktop-quality full screen photo viewer, which allows you to zoom up to 10 levels deep, so hopefully this makes it more clear. The hint will fade out as soon as you zoom one step.
- Notification emails about awarded medals now have a better image, the old one was quite outdated.
- The header of the community activity panel on the homepage had a styling issue. Fixed.
- When playing a slide show and zooming and dragging, releasing your mouse button would move it to the next slide, which is unexpected. Fixed.