As Constant Contact produces more and more video content, we’ve started to run into some unlikely bottlenecks in the video creation process. One of the big ones turns out to be the creation of preview thumbnails, which requires compositing a play button over a frame of video and saving out the resulting image. This simple, but still time-consuming, procedure is a problem we’ve solved for Constant Contact customers through our new Insert Video feature, but it’s still been eating up significant time internally! Rather than continue to create these images manually, Labs took a look at the problem and came up with an automated solution that uses some pretty cutting-edge browser features. Read on for details and a demo.
The Problem
Creating a preview thumbnail for a video is pretty simple, conceptually. All you need to do is take a frame of video:
And composite a play button over it to make it look playable:
That’s really all there is to it! Unfortunately, though, performing this simple image composition normally requires image editing software, a bunch of intermediate files, and careful scaling and positioning of the play button. This is a prime candidate for automation, especially when the alternative involves Photoshop and a call to our creative services team!
What the Solution Looks Like
We’d like a way to create these preview thumbnails that’s:
• Fast
• Convenient
• Easy to Learn
• Easy to Deploy
We concluded it’d be ideal if there were some way to perform this image composition in the browser – because everyone at Constant Contact has easy access to a web browser, and everyone already knows how to use one. What wasn’t clear was that this was actually possible to do in a browser. Here are the operations that need to be performed:
1. Load an image file from the user’s local disk.
2. Scale, position, and composite a play button overlay onto the user’s image.
3. Save the resulting image back to the user’s disk.
A few years ago, this was impossible without some heavy lifting on the back end. We’d need to receive an image upload from a user, store it on a server, perform some server-based image processing on it, and then present the result. That’s .. doable, but seems like a lot of effort. We’d need to juggle temp images (and namespaces), find and configure a server, set up file uploading, find a server-side image processing library, and etc. Not very elegant. It’d be great if we could just do this all in-browser instead, and use the user’s own processor to do the work. It’d be the fastest option by far, and it would mean we’d just need to serve a single static HTML file. Nice. But is it doable?
Yes, It’s Doable
A late-breaking addition to the increasingly-amorphous “HTML5” collection of web technologies is the File API. It’s new enough that it isn’t supported by all browsers yet, but it has been added to recent Chrome (6+) and Firefox (3.6+) builds, and that’s a good place to start. The File API provides a way for browsers to access, manipulate and represent local files from a user’s machine inside the DOM – which is exactly what we need to do to load an image file entirely on the client side. So, the File API takes care of step 1.
Step 2 – scaling, positioning and compositing images together – can also be done client-side, using the HTML canvas tag. The file we read into the DOM using the File API can be pulled into a canvas rendering context and composited with another image using the canvas context’s drawImage() call – and by specifying a PNG overlay with an alpha layer, the alpha channel is incorporated seamlessly into the composition operation.
Finally, Step 3 requires transforming the canvas’ rendering context back into the DOM as a downloadable image, and again, this has recently become possible. The canvas tag’s toDataURL() call does precisely this, allowing the canvas’ rendering context to reappear in the DOM as a ready-to-download PNG file.
Putting It Together
The final hurdle, then, is to invent an easy way to present this workflow. Long-suffering Web app users are familiar with the ever-awkward <input type=“file”>, but that’s never really ideal, and certainly not ideal here. Drag-and-drop using Javascript event listeners would be a lot nicer, and since we’re already specifying HTML5 features that only work on recent browsers, there’d be no further compatibility problem to use drag-and-drop as well.
With that final problem taken care of, let’s recap the solution:
1. Receive a user-provided image using drag-and-drop and the File API.
2. Use the canvas tag to composite that image with a scaled, transparent overlay.
3. Render the result back out as a new image, so the user can drag + drop again to save it.
The power and flexibility of these new browser technologies not only make this operation easy and quick for end users, but also lets the solution scale well (as all processing is client-side), use very little server resource, and work across all major platforms. This solution meets our criteria that the automation be fast, convenient, easy to learn, and easy to deploy. Take a look at the result:
Try the demo page here!
The code’s pretty rough, but it definitely gets the job done, and it’s always nice to take a peek into the future of web technologies. Widespread adoption of these APIs – especially the File API, which is in no way limited to image files! – will lead to some pretty interesting options for client-side data manipulation. Images are just the beginning.
What do you think of the demo? Is client-side image manipulation an interesting idea, or is it taking the web browser too far? And, what other applications can you imagine for new APIs like File? Let us know in the comments below.
Continue the conversation by sharing your comments here on the blog and by following us on Twitter @CTCT_API
The post Client-side Image Composition with HTML5 appeared first on Constant Contact Tech Blog.