Images for TutorialGrad

My partner , John Ward, and I have been working on a lot of new projects that we think are great and they are starting to finally come together. In this post I am going to cover some details of what went into making TutorialGrad; our new tutorial site that monitors, updates, and posts tutorials all on it’s own. This site took a while to figure out how to make it work and make it as easy to use by end users as possible. Let’s begin.

We started this project because of the ridiculous changes that have been made to a once highly respected tutorial site that caused it’s traffic and overall outlook to become terrible. They have let a lot of people down and there should be sites to counteract what was done. I am talking about Tutorialized. More about what they have done and where they went wrong can be read on John’s detailed blog post.

To make this site work, several things needed to be done. In this post, we are going to look at something that I wanted to be done automatically that you can pay to have done, but I felt this was something we could achieve for free. The feature I am talking about the ability to take a screenshot of a page and make it the thumbnail of a newly added site automatically with absolutely no intervention from us. On the main page of the site, there is a bar to add a URL of an RSS feed that we will use to scrape the content of your site and post them on our site automatically. Once a URL is entered, a row is put into a job queue table where the image creation begins.

The server this runs on is a command line only install of CentOS and does not have X windows or anything installed to produce graphical output. To achieve the browser view and take a screenshot, I used a virtual display using X Windows Virtual Frame Buffer This is a powerful tool that will allow you to create, control, and even view the graphical output via command line. You can use VNC over SSH to have visual remote control to a machine as well. Once this was configured on the server and I installed the necessary modules for firefox and some necessary font files we were ready to roll. I decided to use a cron job that calls the PHP interpreter and runs a PHP file to make like easier. The PHP file is in a secure location not access by the apache processes so it wont be accidentally called for any reason.

Some of the more interesting code in the PHP follows. We grab all the rows that currently need processed form the database and begin processing them. For each row, we run the following code:

 //get url from database
$url = $row['site_url'];

//Calls the virtual frame buffer and creates a virtual display and tells it what to run
$shellfile .= "DISPLAY=:3 firefox '$url' &\n";

//give the browser time to load
$shellfile .= "sleep 12\n";

//capture the screenshot of what is currently being displayed (id passed from database)
$shellfile .= "DISPLAY=:3 import -window root '/home/tutorial/public_html/avatar/$id.jpg'\n";

//wait for the image to be captured
$shellfile .= "sleep 3\n";

//kill firefox (don't want hundreds of instances running)
$shellfile .= "pkill firefox\n";

//wait for firefox to be killed
$shellfile .= "sleep 3\n";

//add final shell script line to kill the X virtual frame buffer
$shellfile .= "pkill Xvfb\n\n";

//compile the above strings into an actual file on the hard drive of the server
$shell = "autoimage.sh";
$fh = fopen($shell, 'w') or die("can't open file");
fwrite($fh, $shellfile);
fclose($fh);

//process the file via the shell interpreter
system("sh /jobs/autoimage/autoimage.sh");
sleep(3);

//delete the shell file
unlink($shell);

Note that the above code is all passed to a file on the hard drive and then processed by the shell interpreter and then deleted. This has then created the image in the images folder that we can use to show as the site thumbnail. We then run it through a resize and crop function and then update the file with the updated image. We verify the image is not black (something may have failed) and if it isn’t it is moved into the production database and ties to the new sites account.

This goes to show that with a little thinking outside the box and using the systems at your disposal, anything can be done. You just have to be willing to try new things. This system has worked flawlessly since implementation and has been a great success. Thanks for reading and look back for more updates.