You know how in the movies, the protagonist starts off his day just like any other day, only to find that some some evil supervillain or global catastrophe prevent him from getting home at 5:30 like normal? That’s sort of how it’s been with gravatars recently. What I thought would be a relatively simple architecture improvement has turned out to be a lengthy, frustrating experience for everyone involved. Gravatars pose a unique challenge, especially when faced with the limited budgetary constraints that I currently have. I’m exploring several options with regard to this and hope hope hope to have gravatars being served properly again very very soon. I am actively developing the Gravatar 2.0 website (screenshots will be ready soon), and once this whole thing settles down, there will be some really cool stuff to play with.

I’m not in the habit of asking for donations anymore, as I will be properly monetizing G2, but if you love gravatars (despite the recent shortcomings), it would bring a huge smile to my face, and help me have a little wiggle room in server configurations, if you wanted to donate a few bucks. Any donations can be sent via PayPal to donate@gravatar.com. Huge thanks!

Once again, my sincerest apologies for how things have been, I’m spending all my free time on getting this sucker purring again!

The pre-renders are complete, the mod_magnet script for lighttpd is done and tested, I’m just waiting on the setup of the new servers. Once setup is complete, I’ll get this sucker deployed, and we’ll see what happens. I’m still working on the Gravatar 2.0 admin site, I’ll keep you posted. Thanks for your continued patience.

First I want to thank everyone who has shown support during the last several weeks. It’s been a very stressful time for me, and you are what makes this whole thing worthwhile.

Just to give you an idea of what’s going on, gravatars are currently served in an unscalable way via a php script. This worked fine 3 years ago, but is not ideal at all for today’s situation. For the past two weeks, I’ve been working on a way to take php completely out of the picture. I’ve devised and tested a setup that will allow gravatars to be served as static files from lighttpd at 5000 req/sec. This requires prerendering all gravatars, and as you can imaging, generating 800,000 images can take some time. I hope to have this new setup running as early as the beginning of next week. This will hopefully eliminate lag time for gravatars, and the world shall rejoice!

On the website side of things, I’m going to have to ask for your patience. I’ll have it rolled out just as soon as I can!

And don’t worry, Gravatar is NOT going to shut down.

A recent 50% increase in gravatar requests has pushed the current server to the limits, and response times have slowed accordingly. I have disabled signup and login while I migrate the database to the Gravatar 2.0 schema and setup the new and improved gravatar servers. Once the new servers are in place, you should see gravatars being served with unprecedented speed! After the serving issue is solved, I will deploy the Gravatar 2.0 website, full of usability improvements and new features.

Great things are ahead!

Kip from zenpax.com has released a beta version of a WordPress plugin that improves upon Skippy’s original plugin. Here’s the word from his site:

—————-quote—————–

Gravatars2 is a WordPress plugin, modified from Skippy’s Gravatars with the following enhancements:

• Caches “No Images” and Errors (in addtion to images, of course)

• Separate cache times for all 3 cache types
• Greatly speeds up page loads which are slowed down by having to check the gravatar.com server (and removes a LOT of load from the gravatar.com server)

• Unix-type “system cron” job can be used to have all cache refreshes performed by the server, without user interference
• Skippy’s “WP-Cron” can be used if a true system cron is not available (Gravatars2-WPCron updated for clean integration)

Various other things which may or may not be noticeable to you, but are good

—————-quote—————–

Get your WordPress 2.0 tested Gravatar plugins now, at http://zenpax.com/gravatars2/

I want to take a moment to thank all the volunteer guest raters that make getting your gravatar rated a quick affair. With upwards of 300 ratings taking place each day, it’s a job that can quickly lead a single man down a path of babbling madness. If you see a guest rater on the forums, please take a moment to say thank you. Here’s a few links to our guest rater’s sites for you to wander around on your web browsing journey:

chris :: wired http://blog.maltese.net
http://spoken-for.org
http://www.thereasoner.com
http://www.lookanothersite.com
Álvaro Castaño http://desinformados.net/blog
http://www.geeklimit.com/

Thanks again, raters! You keep me sane.

Coding has begun on version 2.0 of gravatars. The new codebase will help alleviate a lot of the problems that are cropping up on the current PHP site. It will also incorporate a better business model that will hopefully allow me to run multiple servers and eliminate the resource bottlenecks that are the root of these problems.

Thank you all for your patience, and until 2.0 is out, I will continue to do my best with the tight resources available to me. Thanks again to all the volunteer gravatar raters that free me up to work on making gravatar the best system possible!

As many of you may be aware, I’ve been falling a bit behind on rating gravatars. About 200 gravatars need rating each day, and it’s become a bit overwhelming for me. I’m looking for a few gravatar lovers who would like to help me in my daily rating tasks. With a team of people, the individual taskload will be greatly diminished and the likelihood of speedy approval will be greatly increased. If you’d like to help, send an email to tom at gravatar dot com with your name and a few reasons why you’d like to help out gravatars. Rating is quite simple, it just takes time when they pile up.

Thanks in advance for your support!

Some fiddling by my host have left things a little unstable around here. I have disabled logins, as uploads are failing. I am in communication with my host to have these problems solved as soon as possible. Thanks for your patience.

If you like gravatars (and I know you do), then you should keep tabs on my company, Cube6 Media, as we just relaunched our company site (built with Ruby on Rails) and have some new and exciting things on the cooker. Subscribe to the weblog feed to stay up to date with Cube6 news!

http://www.cube6media.com

It’s been a bumpy ride, but I am happy to announce that the server problems have been resolved, and logins are enabled, so get those gravatars in there!

Here’s the full scoop on what happened behind the scenes over the last several weeks:

In order to optimize the performance of gravatar.com, my host moved the site to some new hardware and set it up to run across three different machines. Due to miscommunication on both our parts, this led to the gray/black gravatars as my code was ill equipped to handle different document roots. Unable to get ahold of my host, I eventually figured out how to fix the gravatar serving script, but was left helpless as session affinity was not enabled, leading to the disappearing session problem that forced me to disable logins. Due to continued miscommunication and my emails not reaching my host (paranoid spam control), this problem was unresolved for several weeks (because my host was unaware of the problem). Once I finally got in contact with my host, the session problem was fixed promptly, only to be followed by futher hardware (hard disk performance) problems, which was why the site was down yesterday. That problem is now resolved, and everything seems to be fully functional.

I rated what gravatars made it through the fiasco, but most of the pending avatars were lost, so I apologize to those of you who will have to re-upload your avatars. If this happend to you, you should have gotten an email saying as much.

Once again, I apologize for the turbulence, and appreciate the support of everyone who offered it. You are why I keep doing this.

Only a matter of hours until gravatar.com serves its one billionth gravatar! Wow. One billion. That number doesn’t even register in my brain as real. I only wish that I could have had the site fully operational for this occasion.

So thank you to all the gravatar supporters out there. You’ve made gravatars a raging success, and I hope to be able to continue providing the service far into the future!

Ok boys and girls. I believe I have finally fixed gravatar serving for real this time. I did some digging into the open_basedir problem and discovered that my host is now serving gravatar.com from two different internal servers, each one with a different location for the document root (and presumably different copies of all the files!!!). Strange, I know. Still trying to get ahold if this guy to straighten things out. So what this means to you, the end user, is that gravatars are now being properly served from both places, which I believe is what was causing the intermittent black/gray problem (one server could find the files, one could not). However, it also means I can’t enable logins yet, because of the dual file system issue. Patience everyone, patience. All will be healed in time. And thank you once again to everyone who has been supportive!

On a side note, is there anyone out there who is experienced with FreeBSD server administration and willing to donate their expertise to the gravatar project? I’m hoping to find someone to help me out with setting up and maintaining server(s) for G2. Go on, spread the word!

I spent a bunch of time today working on the avatar serving script, trying to optimize it a little and get the gray/black box issue sorted out. While it seems to work for the most part now, I’ve still been experiencing some strangeness from it on occasion. Often clearing your cache or a refresh or two will bring a munged avatar back to life, which makes no sense at all. I’ve implemented an If-Modified-Since handler that will return 304 Not Modified if the requested avatar has not been changed since the last time your browser downloaded it (assuming your browser supports IMS headers). Hopefully this will cut bandwidth usage quite a bit and free up some server resources so this thing will survive until G2. Thanks to everyone who was supportive during the last week, it means everything to me; and a special thanks to all who have donated recently despite the quirkiness of the gravatar universe. You guys rock!