Above is a small part of my music listening history as reported to last.fm over the last year and a half. Time is plotted left to right, overall number of tracks by the width of the shape, and the colors represent individual artists. I used LastGraph3, which if given your user name will make a set of graphs from your data.

If you click on the image above, you’ll see the full history. It looks like I go through periods where I listen to a fair bit of music, and then stop, and start again. I think there’s a fair amount of smoothing of the data. I think my history would look even more jagged without smoothing.

I like plots like this because they show multidimensional data using colors and shapes in an intelligent way. Of course the classic example is Minard’s famous depiction of Napoleon’s 1812 Russian campaign. I think everyone should have to learn how to make good plots, and understand how to read one. When I was a TA, I constantly had to remind students the point of making graphs – I think nearly all of them felt it was busy work rather than a way to organize and visualize data; a way to recognize a physical effect.

Just like significant figure errors (I am bothered enough by those to contact newspaper reporters: I’ve done it in the past), I cringe at the sight of misleading or poorly organized graphs. The worst offenders tend to use Excel, whose plots are instantly recognizable as probably being garbage. I also dislike the USA Today charts and many plots seen on the various network evening news shows. Too much artistic influence from graphics artists (no offense K.P.!), and not enough substance.

My first two-year prepaid period with Site5 went up in December last year, and I seriously thought about moving. I looked at other shared hosting companies, but I felt I would probably have the same problems on a new shared host. I looked into hybrid solutions, but that too didn’t seem a guaranteed improvement. I liked the idea of Virtual Private Servers (VPS), but I couldn’t find one with enough disk space in my budget.

A few months ago, my lab mate Rick pointed me towards s3fs, which intrigued me. s3fs puts your data on Amazon S3, but allows the data to appear to be local to the server, like another hard drive. You pay for only what you use with S3, and it has virtually unlimited space. Suddenly, a VPS hosting solution fit into my budget. I could pay for a VPS with less disk space than I needed, but still get the power of VPS. It was also an upgrade because now me and my family could upload as much data as we wanted, and it would be much more secure from disk failure than before.

This website and other sites that were on the old server are now being hosted on a machine from linode.com. I’m using their lowest option, which has 10GB of space. I installed Ubuntu Hardy Heron which seems like a solid Linux distribution. s3fs has proven to be reliable and fast enough, although it’s much slower than having the data on a local disk. Using Apache rewrites, my father and I have made it such that when a web browser asks for items on a page that exists on S3, the request goes there instead from this server, which saves lots of time. I’ve also figured out how to shoehorn Gallery2 into using S3.

So far I am very happy with the new server. Oh, and you may not have noticed, I moved my blog to stephenskory.com, although stephen.phratry.net will get you the same thing.

But Yahoo! maybe trying harder. There is now a preference to add the greater-than signs on replied to messages:

Which is great. Until you try to use it. Here is a message I sent myself:

Here is what I get when I hit ‘reply’ (this is a screen shot of the compose window, the text is editable):

Yes, each and every word of the message I’m replying to gets its own line. But it gets worse! Here’s what I get when I send the replied message without touching anything:

Here each word of the replied to message gets its own line separate from the greater-than signs. I hope this is just a simple bug (I will submit a bug report about this) but this is simply ridiculous.

Donuts are truly the gateway to understanding his soul. It does a slightly worse job with me. I don’t own anything made by Nikon.

And it misses Melissa by a wide margin.

In particular, the newest version of Gallery2 provides site-wide watermarking capability. This means that all the images (except thumbnails) now will have a “©Stephen Skory” in the corner, as seen below.

Let me know, dear readers, what you think about this. Do you think it’s ugly? Distracting? Pointless? Comment away!

My hosting plan is on a shared webserver. Here is an informative page about my machine, named iaso. It has four Intel 2.8Ghz Xeon processors, and four gigabytes of RAM. It runs Linux which is a solid server OS.

My big problem are periods when this very website is unresponsive. There are lots of other websites on the same computer. And by lots I mean on the order of 600 websites:


-bash-3.00$ cat /etc/passwd | grep -v nologin | wc -l
646

The number of users with logins is not a one to one list of websites served, but it’s probably a good estimate. All it takes is one of the 600 users with a bad webpage to clog up the machine.

Below is a plot of the load level for iaso over the course of 15 days last month. Without going into specifics, a load level of one means that there is one process needing a processor at any given moment. It will be many different processes, and that’s fine. Practically, a machine can stay responsive with up to about a load of three or four per processor. So, on a four processor machine like iaso, a load level of 10-15 is about the highest comfortable level.

What you see above are many occasions when iaso went well above load levels of 25. The highest peak was a load of 230. In my experience, once the load reaches 25 my website becomes more than slow: it doesn’t work anymore.

For comparison, below is a plot of the same thing on one of the nodes of the supercomputer I use, Datastar. This is the node where scientists do heavy-duty analysis on their datasets. For instance, I use this node to process my multi-gigabyte datasets using IDL. People also run Mathematica and other very computationally intensive tasks on this machine. It’s got 32 1.7Ghz Power4 processors and 256GB of RAM (what do you have on your workstation, huh?). It runs IBMs AIX 5.3. As you can see below, for the first four days, the load level stays below one process per processor. In the fifth day something happens and it goes above 60 for a while, before the machine gets rebooted and things return to normal.

The kind of processes that run on the two computers above are very different. However, the supercomputer is supposed to run big jobs and get beat on. A webserver isn’t. Anytime the webserver’s load goes above 25, it’s like the supercomputer’s load shooting to 256. At no time did the supercomputer shoot to 256, while the webserver goes above 25 many times. Of course, I’m comparing 15 days to 5 days in the two plots, but I think the differences are clear.

Site5 pays a third party to monitor their webservers, with results listed here. iaso has 99.8% uptime overall and 99.4% over the last month. This is bad enough that apparently I’m due a 5% credit on my next billing cycle. iaso isn’t even living up to Site5′s own service standards.

Every time I catch my website being slow, I contact Site5 tech support. I know that this is a common problem with shared hosting. I’m sure that Site5 is aware that these outages, and does what they can when they happen. But, when it does happen, it’s annoying. It shouldn’t happen in the first place. Sometime this summer, Site5 is changing their hosting solution which may help with these problems. We’ll see.


<?PHP
ini_set('display_errors', 0);
$data = yahoo_reg_login_setup();

if ( $data === FALSE )
{
exit();
}
else if ( ! isset( $data['DISPLAY_FORM'] ) )
{
error_log( "yahoo_reg_login_setup didn't set the DISPLAY_FORM field" );
header( "Location: http://login.yahoo.com/");
exit();
}


Here is the rest of the PHP code. It is obviously the PHP code that runs the base of Yahoo! Mail, which decides where to forward you: either to your inbox or to a login page. At any rate, there's no reason why Yahoo! Mail's servers should be sending me raw PHP.

When I get this page, if I hit 'refresh' I'll get the correct page, so things aren't totally broken. This slightly annoying, and it's obviously a problem with Yahoo! Mail's servers. My web browser is incapable of producing raw PHP code, much less with a Yahoo! Mail bug tracking number (see the full text).

Trying to be proactive, I sent this code to their help department. I also wrote "Please don't tell me it's my problem, as there is no way my browser can generate raw PHP." They wrote back telling me:

We understand that you're receiving a HTML error message in your account ... To help us troubleshoot and assess the issue, please take a screenshot of the entire page when the issue occurs next.

They then proceed with instructions on how to take screenshots in Windows. As a Mac OS X user, I found this help insulting and useless. I am no common Windows user, and they shouldn't assume I am. Furthermore, providing a screenshot is even less helpful than my original post, as the PHP code covers more than one contiguous window - I have to scroll to see all the PHP code. It's clear that either my message wasn't read, or the person reading it had no idea what I was saying and put in the most general pre-written reply they could find.

It says something about your company when this kind of 'help' is allowed. This person should have forwarded my message to someone who had a clue. They shouldn't have sent me anything, much less nonsense, when they no idea what I was writing about. Perhaps there is no conduit for this kind of thing between the Yahoo! Mail help staff and the actual programmers. If that's the case, it's a shame, and it's more evidence why GOOG is kicking YHOO's butt.

Shifting the lens laterally has the effect of keeping the edges of a vanishing object stay parallel. Buildings or door frames often suffer from this, here is an example of this vanishing effect. This happens because the film is not parallel to the plane the parallel vanishing lines define. Conveniently, wikipedia has some nice graphics showing that the goal in shifting the lens is to keep the image plane parallel to the object while also keeping the image on the film.

Another attractive ability of a shift/tilt lens is shown below. By tilting the lens, the sides of the shot can be brought out of focus in creative ways. There is a stripe of the photo which is in focus, and on each side the image is out of focus. Obviously, by rotating and shifting the lens, the location of the stripe can be moved on the image.

At first glance, this image above looks like it’s of a scale model. It is in fact not a model, rather it’s an aerial photograph (or from a tall building) using a shift/tilt lens. Here are more examples of this kind of photography from the same site. I’m fairly certain that these “the bitter* girls (place in your life)” (#) photos have been photoshopped to enhance the colors because they look just too saturated to be real. This has the effect of making things look more like a scale model with clean, unweathered paint. I think there’s also been some down-sampling because even where the photo is in focus, there’s not very much detail.

Through extensive searching online, it seems the widest variety of shift/tilt lenses are Russian-made. Here are a couple lenses which claim Minolta compatibility: MC 35 mm ($600) and MC 80 mm ($400) tilt/shift lenses. If any of you are feeling particularly generous, go ahead and buy me either lens. If you’re feeling especially generous, buy me both!

(#) The Japanese are weird. I can’t even imagine how that makes sense in Japanese. Perhaps it’s mangled English for the sake of mangled English.

Here are some sites that use Google Maps in ways I find interesting.

Wikimapia

This site allows you to tag, with rectangles, any area of the earth at a wide range of scales. The image above shows the San Diego Velodrome which I tagged myself. This is a great time waster even if you’re only exploring.

Solar Eclipses

This site shows the path of all the total solar eclipses for the next thirty years. Clicking on the map brings up statistics for that spot on the earth. Start planning for the next one in the USA, in 2017!

Earth Sandwich

Ever wonder how to make an earth sandwich? First, put a piece of bread down where you are, and then quickly (before it gets eaten by your house mice), using this tool, put a piece of bread exactly 1/2 the way around the world. This tool tells me that my second piece of bread would get very soggy in the Indian Ocean.

It was in Georgia that I learned of the utility of the best weather site on the internet, wunderground.com. Their best feature, was, and still is, doppler radar that updates every six minutes when there’s rain. Five years ago all the other websites I searched updated the radar no more quickly than every half hour. In Georgia I would often log onto the website to see if I could dodge between thunderstorms and walk to the Physics building, or go for a bike ride. The radar was accurate and current enough to do this.

In the five years since the radar has just gotten better. Now you can zoom in, make animations, and use many other cool tools. Since I’m a paying customer (I believe it’s $10/year, and highly worth it) I get no advertisements and expanded radar tools.

Below is an example animation, fresh off the presses for today. It’s Saturday morning, a time when I usually go for a bike ride. In fact, the sun was out this morning when I woke up. But my trusty wunderground.com weather page told me that I’d better wait. I live just a bit north of the black cross with a circle around it, which you can see is about to get pummeled by some thick rain.

The other, more popular weather websites have gotten better in the last five years, too. However, I still think that wunderground.com is still the best by far.

Lo and behold, Yahoo! came out with an updated version of Mail Beta a month ago, and more recently my server farm received the update. Let’s see how Yahoo! fares this round…

1. Fixed-width fonts. Huzzah! Numbah one gets addressed. This is big. Fixing this almost is enough for me to start using Beta every day. But only almost. Yahoo!, you get a nice green check:

2. Message replying format. Nope. Nothing new here. Same, lame behavior as before. Give us some freedom, Yahoo!. Stop putting the minority with good etiquette down! This earns you a Big Red X, and red is never a good color for anything.

3. Message quoting. Nope, again. There is still no way to differentiate the message I’m replying to and what I’ve written. Another BRX.

Yahoo!, you’re getting beat up, down, left and right by Google. They just took YouTube out from under you this week! Shape up!

1. Fixed-width fonts. There is still no option for showing/composing messages in fixed-width font. Starting off weakly, Yahoo!

2. Message replying format. It still puts my signature at the top of the message. Again, they should provide the option of putting it where I want it. Uh oh, another red x.

3. Message quoting. This also has not been fixed yet. Since I use good email etiquette, having no differentiation between what I’m replying to and writing is not an option.

4. Signature new lines. Finally, something they’ve fixed. Of course, without fixed-width font, my signature still looks wrong.

5. Bugs. Perhaps there are other bugs that have gone unfixed, but the one I identified in the previous post has been fixed. So you get a green check, Yahoo!

In sum total Yahoo! is batting .400, which is an excellent baseball average, but it is a poor average for things so simple to fix. Moreover, the list is in roughly descending order of importance to me. Therefore, Yahoo! is batting more like .150, having tackled none of the things crucial to me.

I’ve tried to tell Yahoo! about these shortcomings. I’ve submitted feature requests to the appropriate place several times over the last seven months. I can’t believe I’m the only one with these concerns. But since nothing has changed, perhaps I am.

During lunch today I added a module to my photo gallery that allows me to tag photos or albums with latitude and logitude such that they are marked on a map. Using Google Maps along with the Google Maps Module for my photo gallery software, in less than 15 minutes I was able to mark my photos and albums on the earth.

I can tag photos or albums by type, so that when browsing the map, different types can be turned off or on. You can also define places to quickly zoom in on. There is a facility to map routes (like this), but haven’t tried that yet. When you click on one of the marker pins, a preview window will pop up, with a thumbnail for that photo or gallery, a link to the item, and the title or description.

It’s all very slick, and it worked on my first try. If your browser works with Google Maps, it will work with this. Go check it out!

(p.s. Right now not all browsers show the fonts the correct size, Firefox in particular. I suggest trying a different browser until I fix that problem.)

Limited time offer: Save a click! Just for this post, you can subscribe below. In the future, you’ll actually have to go to the Email Updates page. Act now!

^* Okay, many is too strong a word. Nonexistant is probably more accurate.

WPG2 is a WordPress plugin to allow Gallery2 to fit ‘inside’ of the WordPress look and feel. I tried their version 1 some time ago, but I didn’t like it because it didn’t work very well. I tried writing my own WordPress-Gallery2 wrapper, but I couldn’t get that to work, either.

Now if you click on the My Photo Gallery link, the photo gallery will keep my rocky shoreline header image and the borders that this page has. There are still some CSS-related issues, so when you browse the gallery, the text may be too small to read. It’s too small in Firefox, but not in Safari. I have yet to figure out which browser is wrong.

Ever since I saw the Google mapified routes for the Tour of California (here’s an example), I’ve wanted to do the same thing on the UCSD Cycling routes page.

I recently stumbled across GPS Visualizer.com. It’s an amazing website full of all kinds of GPS tools. The one I found to be useful was the Google map tool. It can take all kinds of GPS data files, and it outputs the route superimposed on a draggable Google map. Since I use a Garmin Forerunner 301, I can use the GPS data to make a Google map route.

There are two ways I can go about this:

• Upload my data to Motionbased.com, which then allows me to download the route as either a KML or GPX file. I then give that to GPS Visualizer.
• Use LoadMyTracks to download the data off my GPS to my iMac, and then (after some parsing) upload that file to GPS Visualizer.

So far I’ve only done a few of the routes, but here are some examples. So for all you fans of me out there in cyberspace, keep checking back for new Google map cycling routes!

My avid readers may not have noticed that I have added a Cool Websites category to this non-blog blog. There I have put all my navel-gazing website posts, as well as my posts on Yahoo! Mail Beta. This new category will be used for posts about cool websites, like this one, or any other I feel inclined to write about.

I have just discovered a new website, digg.com that I think is pretty cool. It’s not that new, so you’ve probably heard of it already. I was introduced to it as an alternative to Slashdot, and I like what I’ve seen so far. The links seem fresher than Slashdot, and if you make your own account it keeps track of the stories you like, or that you “dugg.” You can “digg” a story by clicking on the “digg it” button next to any posting. The more people that digg a story, the more likely it is to show up on the homepage.

I suggest you go check it out and see what digg is all about. I think I’ll add it to my list of daily websites, along with Slashdot.

Back to my main point. If you look at the Phratry Nexus you’ll see thumbnail sized screenshots of the various Phratry.net webpages (four as of this writing). Due to the already mentioned fluid nature of the web, keeping those thumbnails current would be pretty tedious if done by hand. That’s where Applescript comes in. Ideally, I would have liked to write some kind of script and put it in crontab on the webserver, but doing screenshots is difficult from the command line. The Applescript I wrote (along with some help from my father, Mike) uses four apps, which either come with Mac OS X or are free: Cyberduck (for FTPing to the server), Freesnap (screencapture utility), Image Events (image editing) and OmniWeb (web browsing). OmniWeb isn’t free, actually. Firefox would work too, but I got the script to work better with OmniWeb.

The specifics of the script aren’t important, but the cool thing is Applescript allows me to do a 15-minute process of screenshot editing and uploading with just one double-click. It still takes about a minute for the whole thing to work (most of the time is from built-in delays in case of slow webpage loads), but that’s still fifteen times better than before, and it’s done right every time.

You may be wondering what phratry means. According to Dictionary.com it means

1. A kinship group constituting an intermediate division in the primitive structure of the Hellenic tribe or phyle, consisting of several patrilinear clans, and surviving in classical times as a territorial subdivision in the political and military organization of the Athenian state.
2. Anthropology. An exogamous subdivision of the tribe, constituting two or more related clans.

Since I wanted to have other family members & possibly friends use this domain & hosting service, I wanted to think of a domain that wasn’t too specific to any of us, but was descriptive. Although none of us are a member of a Hellenic tribe, the general idea is phratry refers to an extended family group, which describes the situation. I also kind of like the word because it’s not very common and has an odd spelling.

The cool thing is everything is so fast! Downloading photos off of the old hosting situation was pretty painful. This should be much nicer.

This is either an artifact of the beta not having all of the features of the old Yahoo! Mail yet, or that Yahoo! has something special planned regarding Mail. I’m willing to bet that unlimited mail is impractical and would be abused, but we’ll see!

Okay, spammers, give it your best shot!

Click for full screenshot

I just got added to the list of Yahoo! Mail users who get access to the new Beta interface. All in all, I actually like it, even compared to Gmail’s interface. For a while now I’ve been hoping that Yahoo! would give users IMAP access to their email, so I could use a good email client with my own computer at home, but still have access to all my messages anywhere. This interface just might curb that desire. It has Gmail’s trick of simple keyboard shortcuts, but also has drag-and-drop ability for messages, RSS feeds and cute icons. One really neat feature is you can have multiple ‘tabs’ open. Tabs can have messages you’re composing, reading, or your inbox. Very often I am writing a long email when I want to send a quick note to someone else. With tabs it should be very painless.

This new interface has the potential to be just as good as many email clients. I however, have some complaints:

1. Please, please, please use fixed-width fonts. Email is not supposed to look good. If I wanted it to look good, I’d send a PDF. There are many reasons why fixed-width fonts are a good idea, foremost being ASCII art (of course!). My signature has a cute little cyclist guy nestled between my name, email & web address. With a variable-width font it looks terrible.

But seriously, though, if I ever wanted to email myself some sort of crontabbed system log, the unix-style of formatting with spaces between items would completely not work. Please, Yahoo!, at least give us the option to have fixed-width fonts. I know I could fix that using site preferences on my browser, but that’s not very practical on every single computer I use.

2. When replying to messages, I like to reply to each of the person’s points by quoting what they wrote, and then writing below that. Example:

> What do you think about the design? Too much?
> I think less red and more blue.

Yeah, I agree. Red is pretty much the worst color you could ever choose.

The way the beta does it is it puts the replied to message with about 4 carriage returns above it AND puts your signature above the message. This means that if I reply to a person’s message the way I like to, the first thing my message will have is my signature. Dumb!

3. As far as I can tell, right now there is no way to differentiate between what I’ve written and what I’m quoting. Usually there are the greater-than signs (>), but in beta there aren’t. Some email programs put colored lines. I’m asking for something to tell them apart, Yahoo!, are you listening? Don’t take a step backwards!

4. The signature doesn’t seem to recognize carriage returns. That’s just silly. I have a three-line signature that gets turned into one very long line. Even if I didn’t have ASCII art, it would butcher a simple signature.

5. Fix the bugs! Editing a replied-to message is buggy. Before I can delete any of the copied message (the one I’m replying to) I need to type something at the very top. The interface could be faster, too. But then again, old Yahoo! mail isn’t that fast, either.

I’m sure I can think of more things to gripe about it in the next few days. I’ll add to this list as I use it.

I didn’t want to make a whole website from scratch and I wanted something a little slick looking. And I’ve heard about this blogging thing. Although I don’t think I’m going to blog, per se, I think this software has some advantages over alternatives. I like to play, too.

I guess my main goal of this site is for my own purposes. I’ll post things here that interest me. I think it’s primary purpose will be to post notices when I add new photos to the gallery. Let’s see what I do with this, huh?