Ed's Tech Tips

FLOSS stands for "Free Libre Open Source Software" and is one of many regular podcasts that are part of the TWiT network produced by Leo Laporte and friends. Leo was a host on the now defunct TechTV Screen Savers show and hosts a wide variety of weekly tech orientated podcasts. My favorites are This Week In Tech, MacBreak Weekly, Jumping Monkeys (tech and kids), Windows Weekly and FLOSS. You can easily subscribe via iTunes, which is what I do and I listen either while I am working or at home doing various chores around the house with my iPod.

The shows are both informative and entertaining, although I do get annoyed sometimes with all of the talk about Twitter which I think is really only popular among the commentators and pundits on the show and not with real people trying to get real things done. Who has time to read inane chatter? But when you talk about getting things done I think wikis (may favorite is Deki Wiki) are a great tool and the inventor of the wiki is Ward Cunningham. He wrote the first wiki in Perl  to help keep track of and share ideas with is co-workers back in 1995. Since then wikis have been written in almost every computer language and have given birth to one of the most popular web sites on the planet, Wikipedia. 

Not only is Ward the father of the wiki but is heavily involved in the extreme or agile programming movement. It is a software engineering methodlogy intended to lead to a development process that is more responsive to customer needs ( "agile" ) than traditional methods, while creating software of better quality. Deki Wiki is developed with the "agile" programming approach and features an update or release almost every month.  

Besides all of that, Ward comes across as a real down-to-earth humble nice guy. 

If you are interested in technology and like to listen to hour long conversations (ala Charlie Rose style) then check out TWiT. 

 

Apple released the Software Development Kit (SDK) for the iPhone/iTouch last week. It is a pretty much a complete package. It is based on Xcode the same development tools used for Mac OS X. It gives you access to all of the hardware of the iPhone including the camera and accelerometer - the device in the phone that responds to tilting the phone. There is even a iPhone simulator so that you can test your apps on a virtual iPhone which allows you to develop for it without having to own one. It even allows you to simulate the pinching touch feature. This opens the device to some cool gaming possibilities.I won't go into detail because other sites have done that, as well as, Apple's own pages dedicated to the SDK. I'll just relate some issues from my personal experience.

First, why is this cool or important. Well, initially it appeared that Apple was not going to open the platform for outside developers, but now it has and it opens the door to thousands of applications. If you have used a Palm you know that there are many possibilities for small applications on a portable device and even more so on the iPhone because of it is screen resolution, multimedia playback capabilities, touch interface and connectivity to the Internet. Applications will range from currency converters to pharmacological encyclopedias and beyond. ReadWriteWeb offers some more sophisicated possibilties.

Also, the cost of entry is fairly low. The SDK is free. It is a 2 GB download that expands to take up 5 GB of space. It is not compatible with PowerPC processors, which I'm a little miffed about. It  only works on Macs with Intel processors. Like other features in Leopard, the PowerPC processor is not fully supported, but that was inevitable with the move to Intel's dual core technology. The G5 and lessor PowerPc chips just cannot keep up with the demands of the newer software. But, I'm on a tangent...

While the cost of the software is $0, the developer program cost $99 and $299 for proprietary, in-house developers. Applications can only be made available through Apple's App store which will be available on the iPhone and iTunes.  The developer sets the selling price. Apple takes it cut of 30%, but handles all related costs of distributing the application, including credit card processing and bandwidth.If you are giving away your application there is no cost beyond the developer fee. I think it sounds like a pretty good deal, especially for small developers to get their products in front of thousands of potential customers.   (More)

How many times have you taken a photo that is over or under exposed? Either the highlights are blown out and it is too bright or there are too many shadows and the image ends up too dark. This especially happens with sunrises and sunsets or when taking photos indoors during daylight near a window or open doorway. You usually cannot capture the details in the shadows and the highlights at the same time or at least not in the same shot. This is where those exposure and bracket settings on you digital camera come in handy.

Most digital cameras have a often overlooked bracket feature which when set, will allow you to take three or more quick successive shots when you press the shutter key where one image will be underexposed (dark), another will be moderately exposed and the last will be overexposed or too bright. You will want a steady hand or a tripod when using this feature. This can be a useful option to use when you are not sure how the camera will interpret a scene or when you want to insure every bit of detail is captured at a variety of exposures to be combined together later in a single tone-mapped image that is a combination of the original three photos.

Do a search for "tone mapping"  on Flickr for a variety of images which are either hyper-realistic or on the other extreme dreamlike or surreal. This photo of sunflowers almost looks like a painting to me:

Here is a example of an interior of a church. With tone-mapping, the detail of the wood work and the stained glass is captured in a single image that originally started out as six seprate images:

A digital camera can only capture a limited range of lights and darks from the full spectrum that we can perceive. Even the monitor that is displaying the words you are now reading can only display a limited range. No matter how many mega-pixels your camera has or how fine your camera lenses are, your camera is no match to your eyeballs.

You could compensate for the limited dynamic range by using extra lights and reflectors to reduce the shadows, but it is not always practical or feasible. Fortunately, through some software trickery or complex algorithms, there are programs (some free or not too expensive) that can combine your multiple images of various exposures into a single image. You can do this to a certain extent with Photoshop by combining or blending the multiple exposures in multiple layers. See this technique explained at Luminous Landscapes and Eric Krause's web site. Now there are free or inexpensive software applications that you can use for this specific purpose without needing to purchase and learn PhotoShop:

• Bracketeer: A Mac OS X only application that cost $19.95. Just drag and drop your images into the program and let it do it's magic. This application is actually a graphical front-end for thefree free command line utility called Enfuse. Bracketeer always outputs a loss-less TIFF file. Bracketeer is for Mac OS X 10.4 or later. It is a Universal Binary application, so it runs native on either a PowerPC or Intel based Mac. You an almost instant preview of the image before it is rendered.
• Xfuse: This is another Mac OS X only graphical front-end for Enfuse. It is not as polished as Bracketeer, but it is free and offers some more control over the process. You can save in JPEG format and import more than 6 images, the limit of Bracketeer. Like Bracketeer, there is no auto-alignment so you do need to use a tripod to shoot your images. Even with the SteadyShot feature on my Sony DSC-H1, the final output by hand was blurry.
• Hugin: This free open source tool is primarily a panorama photo stitcher, but does integrate Enfuse as well which is appropriate when you think about trying to capture a panoramic scene and balance the exposure of the image. It has options to help align your images if they were shot by hand instead of using a tripod. Unfortunately, at the time of this writing there was not a stable version of 0.7.0 for Mac OS X. 
• Qtpfsgui: This is an older multi-platform (Mac, Windows and Linux) free open source program designed for processing and developing HDR (high dynamic range) images. It supports more image formats and offers even more control over the process, however, it is  about 5x slower than Enfuse and the results may not be as desireable without some considerable tweaking. See the article on Linux.com for details. You also need QT4 installed which makes installation more of a hassle.

If you have some money burning a hole in your pocket there are some commercial applications that are specifically geared toward tone-mapped and HDR images and are more complete packages than the above applications. Although these cost upwards to $99, they are still less than PhotoShop or a set of lights and tripods and may offer more support and hand holding than the open source applications. 

• EasyHDR: This is a Windows only application available for 30 euros. They sponsor a EasyHDR Flickr group where you can see numerous HDR images created with EasyHDR.
• Photomatix Pro: This $99 program appears to be the higher end HDR utility which is available for both Mac and Windows. The site features numerous examples, FAQ and links to resources. The same company offers a PhotoShop plug-in. There is a free trail version to check out if you want.
• Hydra: Briefly mentioned in the April 2008 issue of MacWorld, this is probably the most elegant HDR program I have seen with a slick interface that integrates well with Apple's iLife suite. Like PhotoMatix and EasyHDR, this prgram also attempts to line up images that may not perfectly match up if the camera was not on a tripod.  The cost is $59.95 and it requires Leopard, Mac OS X 10.5. You can download and test drive the program for free which is what I did. The alignment feature worked very well. You get an instant preview of the HDR image. It saves in JPEG and TIFF. However, I thought the output of Xfuse was better as far as contrast and color, but the image was not as sharp. I'm looking forward to a stable version of Hugin to how well the alignment feature works. 

Here is a video posted on YouTube that provides a succinct explanation of HDR with some examples that were processed with Photomatix:

Hopefully, this inspires you make a little extra effort to take better photos and make your family think that you took a photography class. 

Apple briefly stole the spotlight from CES (the international Consumer Electronic Show in Las Vegas) again this year, although not to the extent it did last year by announcing the iPhone. This time around the announcement was  about an update to the Mac Pro model which now features as standard equipment dual Intel Xeon quad core processors which adds up to 8  CPU cores.  This is a whopping  amount of power for a desktop tower computer. However, it is a large tower and you would need a large desktop. It is probably more suitable for the floor. Dual quad core processors have so far been regulated to servers for Windows PC's.

In addition to the increased CPU power you can now slide up to 4 terabytes of storage in four drive bays. There is also an option for the NVIDIA Quadro FX 5600 with 1.5GB of graphics memory for an additional $2,850 - slightly more than the recommended configuration of the Mac Pro. 

This is a machine designed to do some heavy computational lifting including high definition video rendering and encoding and is overkill for most folks.  

At the same time Apple updated the Xserve with the same dual quad core cpu's which is targeted to small and medium businesses, however for the small businesses of under 10 people in a office the new Xserve would be  overkill unless your small business involves decoding DNA, video broadcasting, digital animation or similar tasks. Many small business need a server than can store office documents, connect to a printer and maybe host a web site and act as a mail server.

So, instead of the Xserve or Mac Pro I suggest you consider the Mac Mini as a possible server. You'll have to add an external firewire drive or two, but over-wise it could handle the job and you would not even need to install server version of Mac OS X which cost $499 for the 10 user license. The standard version of Mac OS X can easily handle many of your basic server needs.

The Mini consumes very little of everything: space, power and the knowledge needed to run it. Would a Mac mini be reliable enough? It does not have redundant power supplies or hardware RAID, but then again it costs much less, so much less that you could by two Mac Minis, one as a back up of the other - a Mac Mini cluster. Some web hosting companies are offering co-location services using Mac Mini's, such as, Macminicolo which would attest to it's reliability. There are instructions for setting up your Mac Mini as a web/mail server. Another blog, Nerd Vittles, boast 50 projects centered around the Mac Mini. 

Here is a nice example of Mac Mini server setup on the top shelf of a closet in a home office:

 

 

For the external firewire drives I would consider the models from Newertech which are the same dimensions as the Mac Mini and feature additional Firewire and USB ports for daisy chaining additional drives or connecting other accessories.

 

 

 

YAML stands for "Yet Another Multicolumn Layout", which is a little understated. It is an extensive CSS framework for creating a wide variety of web page layouts that work with a variety of different web browsers. It takes a lot of guess work and hassle out of developing a custom web page design or template. I used it for some of the class and club web site temapltes and just recently for the new upcoming Alumnae Quarterly Blogazine. The Blogazine layout was inspired by a WordPress theme, but I use LifeType which utilizes a superior Smarty based template system. To save time, I use a basic YAML fixed 3 column layout for the basis of the template and built it up from there. Check out the cool online YAML builder which allows you to dynamically create a page layout online.

Another tool you may want to consider is Yahoo's YUI Grids CSS although it may not offer as many layouts.  There is a online YUI Grids builder, too.

These are great tools for saving time andletting you concentrate on the design and content of your design. Don't reinvent the wheel.  

 

I use a great little open source program called Logwatch that sends me a daily analysis report of who accesses our web server and from what IP address they are coming from. There is at least one list that shows up each day listing numerous failed log in attempts with user names from A to Z (see a typical list below). These are from hacker programs or scripts that go through a list of typical names and various simple passwords or lack of passwords. It's an attempt to find a weak link in my defenses.  This is not a clever attack. It is a blunt brute force attack. Don't think that no one would be interested in your server or web host account, because someone somewhere is interested. Your basic defense is a good password that you change often, atypical user names and none standard port addresses for SFTP and SSH. Do not use telnet or plain FTP.

Here is a list of failed login names from October 15th, 2007:

 (More)

Warning: This article is technical, but it pertains to something I have been working on for the past week and is pretty important: the backing up of data on the network.

You may have heard it said before, but it bears repeating: It is not a matter of "if" your hard drive or computer is going to crash but "when".  Therefore, you should be prepared if you care about the data on your computer. Our file/print/intranet server has four 250 GB hard drives configured in a RAID-5 configuration, so that if one drive fails the other drives pick up the slack. It works, I experienced a failed drive in a 3 drive array. Because of the redundent data the drives actually store 650 GB. The data is backed up to another four drive terabyte RAID-5 server that is remotely located. There are many backup strategies, but today I am just going to cover what I do with a free open source program called Rsync. 

From the rsync web site:

rsync is a file transfer program for Unix systems. rsync uses the "rsync algorithm" which provides a very fast method for bringing remote files into sync. It does this by sending just the differences in the files across the link, without requiring that both sets of files are present at one of the ends of the link beforehand.

Some features of rsync include

• can update whole directory trees and filesystems
• optionally preserves symbolic links, hard links, file ownership, permissions, devices and times
• requires no special privileges to install
• internal pipelining reduces latency for multiple files
• can use rsh, ssh or direct sockets as the transport
• supports anonymous rsync which is ideal for mirroring

This command line utility is very powerful and is relatively easy to use. The following command will backup up one directory to another: rsync -av /src/foo /dest. However, for useful daily incremental backups more options are required. There are some very complex scripts written around rsync, however, with the backup option the following script will backup multiple directories to a remote server via ssh (a secure connection), retain 29-31 days of incremental backups (depending on the month), store information in a log  and email the network admin when the job is complete. The first backup may take a while, but the following ones will be much faster since only the items that have changed will be transferred.

Here is the script with comments. You'll want to schedule it via cron, a scheduling application. Here is an article on how to generate and share a ssh key.

#!/bin/bash
# usage: backup.sh [ -d ]
BACKUPS=user@domainname:/c/archive
TIME_STAMP=$(date %d)
RSYNC_OPTS="-avz --timeout=600 --force --ignore-errors --delete --backup --backup-dir=/c/archive/$TIME_STAMP"
DEBUG=0
LOG_FILE=/var/log/your_rsync.log     ## Keeps a copy in /var/log
TMP_LOG_FILE=/tmp/your-tmp-rsync.log  ## Mails the current session's log
rm $TMP_LOG_FILE

CURRENT_DATE=`date`


# this option allows me to easily do a test run when I make changes
if [ "x$1" == "x-d" ]; then DEBUG=1; fi
if [ $DEBUG -eq 1 ]; then RSYNC_OPTS="$RSYNC_OPTS --dry-run"; fi

# the following line clears the last months incremental directory
[ -d $HOME/emptydir ] || mkdir $HOME/emptydir
rsync --delete -aq -e "ssh -i /your-ssh-key" $HOME/emptydir/ $BACKUPS/$TIME_STAMP
rmdir $HOME/emptydir


#Clearly shows me in the log the the start of each backup
echo "Starting Backup of your server on : $CURRENT_DATE" >> $TMP_LOG_FILE


#this is a for loop to go through each of the directories I want backed up
for DATA in /etc /var/flexshare /var/www /home
do
rsync $RSYNC_OPTS -e "ssh -i /your-ssh-key" $DATA $BACKUPS   2>1 1>>$TMP_LOG_FILE
done

#separate lines for mysql database backup, because mysql needs to be stopped 
#This step backs up th entire  database
/etc/init.d/mysqld stop
rsync $RSYNC_OPTS -e "ssh -i /your-ssh-key" /var/lib/mysql $BACKUPS 2>1 1>>$TMP_LOG_FILE
/etc/init.d/mysqld start

#appends the log for the current session to the main log file. 
cat $TMP_LOG_FILE >> $LOG_FILE
cat $TMP_LOG_FILE | mail you@domainname.com -s  "Rsync Backup Results"

exit $?

I would have included the ssh command into the RSYNC_OPTS variable, but when I did I always got the error from rsync that "-i" is not a known option. And yes I did try various combinations of single qoutes inside double and vice versa to no avail. However, the above solution works fine.  The reason for the ssh command is that your connection is secure and you do not have to manually enter your password and the job can be scheduled to run any time of the day. 

This job only takes a few minutes to run after the initial backup (which took a few hours) and I run it nightly, however if you are really paranoid you could set it up to run every hour. If you do so, change the time stamp to include the hour, as well as the day to preserve your incremental backups of which you will have many more so make sure you have the free space. 

Miro, formerly known as Democracy, calls itself the only video player you need and I think they are right, but it is more than that. It is a tool for subscribing and automatically downloading videos from across the Internet. You no longer have to go out to many different video web sites. You can pull them into a single location and easily organize them into categories. You can watch classic Popeye cartoons or video podcasts from PBS. They even have a section devoted to High Definition videos, as well as, a built in Bitorrent client. You may also download videos from YouTube, DailyMotion, Google Video, among others. The software is free and open source. It works on Windows, Mac and Linux computers. The interface is well designed. Miro is the iTunes for Internet video. 

Miro is a project of the Participatory Culture Foundation, a non-profit based in Worcester, MA.  I am very impressed with the organization of the web site, the extensive documentation and technical support, although I did not run into any problems with the application. It just worked. Everything is clearly labeled in the application and is fairly intuiative. The online guide is helpful in finding channels and videos and subscribing is very easy. There is an excellent related web site called MakeInternetTV that is a guide that has step-by-step instructions for shooting, editing, and publishing online videos that can be watched and subscribed to by millions of people.

Here is a video demo of the Miro software running on Windows:

 

 

Don't take my word for it and check it out for yourself.  

CUPS? It stands for Common Unix Printing System and is an open source application. From the CUPS web site:

CUPS is the software you use to print from applications like the web browser you are using to read this page. It converts the page descriptions produced by your application (put a paragraph here, draw a line there, and so forth) into something your printer can understand and then sends the information to the printer for printing.

Now, since every printer manufacturer does things differently, printing can be very complicated. CUPS does its best to hide this from you and your application so that you can concentrate on printing and less on how to print. Generally, the only time you need to know anything about your printer is when you use it for the first time, and even then CUPS can often figure things out on its own.

CUPS is distributed with many Linux distributions and has been bundled with Mac OS X since the Jaguar edition (10.2). Apple had initially attempted to create it's own printer system from scratch, but then shortly adopted CUPS. This past February Apple quietly hired the creator of CUPS and bought the code, but continues to make it available as an open source project. The announcement was just made on the CUPS web site July 11. 

Hopefully, Apple will continue to keep CUPS open. This should be good for CUPS, since it assures financial backing for the project. There was a commercial version which is no longer available, although the reference book can still be bought but they are not accepting any orders while they move from Maryland to California. It is the best system on Linux for managing printers which is done via web browser administrative interface.  It is what is used on our Linux file and print server which runs ClarkConnect.