Identifying Blank Thumbnails

When a video is imported or uploaded, the system automatically generates 3 thumbnails of the video (beginning, middle, end). A lot of videos begin with a sort of fade up from black, and someone pointed out during my presentation last Friday, it might be neat if there was a way to avoid adding empty (all black) thumbnails.

To accomplish this, I compare the generated thumbnail with a plain black image that I store using RMagick, a Ruby wrapper for ImageMagick. Here’s the snippet of code that makes it happen:
def black(args = nil)
require 'RMagick'
src_black = Magick::ImageList.new(RAILS_ROOT+'/public/images/black.jpg')
source = Magick::ImageList.new(args[:path])
black = src_black.scale(source.columns, source.rows)
if source.difference(black)[1] < 0.01
return true
else
return false
end
end

The difference command only works when the images are of equal size, which is why I have the intermediary resize step. The difference method returns an array of values, and I found the ‘normalized mean error’ to be a pretty good indicator. When comparing a black thumbnail with the black image that value was on the order of 1 E -5 and a regular thumbnail generated a number in the range of .50-.90… so I opted for the 0.01 threshold. That should be enough to test for all black images, but still allow fairly dark shots thumbnails to pass through the system.

Thumbnailing Adjustments

The current thumbnail generation code runs in the same process as the web request to generate the thumbnails. Its much easier to extract a thumbnail image in the same code that creates the thumbnail object, and it worked pretty fast for a lot of my smaller test files. Now that I’ve started to work with larger and longer files, I’ve found the thumbnail generation process (which is handled via this command ffmpeg -i file.avi -y -deinterlace -f image2 -ss 00:12:34 image.jpg) has been too slow to be acceptable. I imagine that as video codecs get more complicated to compress video and maintain quality, the method to extract a single frame in the middle of a stream gets harder.

To fix this I’m going to have to re-write the thumbnail generation to be handled like video conversion and run as a background job. It unfortunately won’t present users with their thumbnails as quickly as I’d like (no one like to see a page saying “In Progress…”) but it should fix the long load times and subsequent timeouts caused by thumbnail generation.

Importing, Uploading, Thumbnailing

Last week was a busy week for me on the programming end of things… sorry about the lack of a blog post!

I decided to temporarily halt work on the video “collection” concept (grouping similiar videos together isn’t as important as the videos themselves!) and focus back on the workflow for uploading a video. I’m happy to say things looks and flow much better now. Let me walk you though what happens.

1. Visit the Add Video page
2. Give the video a title
3. Choose to Upload a file or Import a file on the server. Importing works best for really large files, where it might make sense to ftp or scp them over manually before bringing them into the software.
4. I use Mediainfo to read some metadata about the video file, like duration, codecs, etc and store those back on the Video model.
5. Then, I generate 3 thumbnails at the beginning, middle and end of the video. They are also stored in 2 smaller versions (like thumbnail, and preview).
6. Presto! Upload complete.

The thumbnail functionality is finished, such that you can manually specific a timecode anywhere in the video to generate a new thumbnail.

I have run into a few bumps with really high quality files, the thumbnail generation takes longer than the web-browser is willing to wait so the browser times out. The process continues on in the background, but its still not the cleanest implementation.

Next on my list of things to work on is video conversion.

Oh yea, here are some screenshots of the current interface: