Google just lifted the restriction that App Engine’s data store query can only return 1000 results at a time.

I just updated this blog (built on App Engine) to take advantage of this functionality. It nicely simplifies some things. For example, before I had to write contorted code like the one below to get all Articles:

def get_all_articles():
    query = Article.gql('ORDER BY __key__')
    articles = []
    while True:
        got = query.fetch(201)
        for article in got[:200]:
            articles.append(a)
        if len(got) <= 200:
            break
        query = Article.gql('WHERE __key__ > :1 ORDER BY __key__', got[199].key())
    return articles

Now it’s much simpler:

def get_all_articles():
    query = Article.all()
    articles = []
    for article in query:
        articles.append(a)
    return articles

Another improvement they did is support for query cursors, which allow bookmarking a position in the query. It makes implementing pagination of results easier.

It’s nice to program against a platform that improves.

This idea of "release early, release often" might actually work. A day after releasing first public version of VisualAck, my visual grep tool for mac, I'm releasing another version which includes an improvement suggested in the forum: to not highlight matches with control sequences when not printing output to a tty (e.g. when output is redirected to a file).

To complement that, I've added ack-compatible options --[no]color and --[no]colour to force or disable highlighting matches.

This, of course, only applies to command-line vack and doesn't apply to UI.

ack is a good improvement on grep. When I discovered it, I decided to write a similar tool in Cocoa, exclusively for Mac OS X and with (optional) UI. VisualAck is a result of that decision (and a testimony to my unimaginative naming skills).

It’s a first public public release with a rather arbitrary version number. The only important part about the version number is that it’s not 1.0, which means that this program is a work in progress.

So try it out and let me know how it works for you.

Obligatory screenshots:

7.92 times cheaper, to be exact, if my calculations are right. According to this page, you can buy 1TB of storage for $256/year. According to this calculator, 1TB (1024 GB) of S3 storage costs $169 per month. A simple math tells us that $169 * 12 / 256 is about 8 and that’s how much cheaper Google’s storage is compared to S3.

I know it’s not exactly the same service, but the difference in cost is simply astonishing.

Is Amazon overcharging? Is Google 8 times more efficient than Amazon? Or is Google willing to loose its shirt in short term but setting up a stage for the world of cheap, interchangeable, Chrome OS-running laptops without local storage, whose users mostly use Google’s on-line apps.

I don’t really care why – at $256/year for a terabyte of cloud storage it starts to look like an interesting alternative to drobo.

It happens all the time:

• I try to read an article or a paper on a programming technique, say B-Trees
• I read and understand the first few pages about motivation behind the technique and other expositions
• when it comes to the details, my eyes start to glaze over, I read but I rarely understand

I end up being aware of techniques, but not really understanding them at the same level I understand linked list (which I’ve implemented many times in my programs). The time I spent reading the paper feels wasted.

For example, I’ve read about skip lists several times in the past but today is the first time I can say that I understand skip lists well enough to implement them. Why? Because today I have actually implemented them.

You have to implement to truly understand.

The war on spam on the web is ongoing.

Smart people at big companies come up with ever sophisticated ways to prevent automated submission of spammy content to forums or blogs.

Smart sleazy people come up with ever sophisticated ways to break those schemes (or captchas, as we like to call them in the industry).

In fofou, my forum software, I use a trivial captcha that I copied from CVSTrac: it asks a user to enter the result of a simple arithmetic expression (e.g. “5+3”). It even provides the answer.

I have no doubt that this scheme would pose no challenge to people that are able to crack google’s captcha, yet I haven’t had a single automated spam in more than a year of running my forum.

My theory is that this captcha scheme is not used frequently enough to catch the attention of spammers. Even though it is probably easy to break, spammer’s time is better spent trying to break latest scheme used by google or yahoo than waste even 5 minutes cracking one-off scheme used by a few obscure websites.

Hence my advice: instead of using sophisticated but frequently used captcha, you can instead use a trivial scheme that no one else is using for 100% spam proof website.

And what if your website becomes really, really popular and is worth the attention of spammers? First of all, congratulation, you’re the lucky one. Second, switch to sophisticated captcha that all other really popular websites use.

I have a mac pro (early 2008 model) running OS X 10.6 and recently it has been plagued with mysterious slowdowns: after a couple of hours it would develop a huge latency when responding to mouse clicks.

For example it would take literally seconds to show menu after clicking on a menu bar or switch to an application after clicking its window. Not an experience I would expect from 8 core machine with 12GB of RAM.

A reboot would fix it but only for a few hours. After several days of frustration I figured out the problem: Drobo Dashboard 1.6.1 was the culprit.

There are two solutions.

One is to kill DDServiced process when the slowdown starts happening (sudo killall -9 DDServiced). It’ll restart itself so things will keep working and the slowdown will be gone for several hours.

Another one is to uninstall Drobo Dashboard (at least until Drobo fixes their software) following those instructions. Drobo runs just fine without the dashboard and its usefulness is lost on me.

I found a few twitter mentions about this problem. According to this tweet upcoming 1.6.6 release will fix this problem.

I’ve made a small update to 15minutes for windows to fix few small things that were never the less annoying in frequent usage.

First, when the timer finishes, the window no longer steals focus from whatever it is you’re doing.

Second, the window is smaller – important when running on lower resolution screens.

And finally, when the timer is running, it gets slightly different background color so that it’s easy to tell if timer is running or not (I would sometimes forget to start the timer).

And here’s how it looks like:

I’ve released 1.0 version of Sumatra. Compared to 0.9.4 it has bunch of bug fixes and small improvements. Each of them is too small to single out, but together they improve Sumatra quite a bit.

This version gets the 1.0 version and officially is out of beta because, well, it’s time.

This version has received great many improvements from Simon Bünzli (zeniko) and some more TeXSync support fixes from William Blum so they deserve much credit for this release.

It’s available here

If you need a simple web server for .NET, this is a solid piece of software.

I’ve needed an internal web server for my C# application and re-using someone else’s code certainly beats writing it from scratch.

It’s not spelled out on the web page, but it comes in two flavors:

• lite, with only http serving core. When you download the code, look for it in branches directory
• standard, which includes additional capabilities like templating engine

When I added it to my project, I didn’t know about the lite version, so I manually added the files to the project and removed files I didn’t need. If I were starting from scratch, I would just use the lite version.

So far I’m happy with the code – it does exactly what it advertises and saved me a bunch of time I didn’t spend writing web server code.

It’s distributed under Apache 2.0 license so can be used in commercial software as well.

I’ve made two small improvement to my 15minutes app for mac (which is meant as a simple productivity tool that helps to focus on doing things).

Improvements:

• time display is bigger
• it now has non-awful icon

I also came to a realization that it should really be more unobtrusive app that lives in the menu bar and has even more minimal UI. However, at present I don’t have the time to do that. 15minutes has been something that I wanted to get done as quickly as possible. It would be ironic if my productivity dropped due to spending a lot of time working on productivity-enhancing application.

I just installed Visual Studio 2010 Beta 2 and tried it on Sumatra. Unlike Beta 1 it properly converted project file from vs 2008. After playing with it for a few minutes it looks like a worthy upgrade to 2008 which is why it stings even more when a useful functionality is missing.

Lenovo’s thinkpads have nice feature of combined TrackPoint and UltraNav that allows easy scrolling in many apps (I think they generate scroll-wheel events). Unfortunately the new, rewritten editor in 2010 doesn’t seem to support this kind of scrolling and it’s one of those features that I use very often and became so dependent on it that it’s hard to use an application that doesn’t support it.

Here’s me hoping that it was just lack of time in beta 2 cycle and it’ll be supported in final release, since it would be really painful to use the editor on a ThinkPad laptop without it.

I spent a couple of hours tracking and fixing this problem.

A user reported an error in my fofou forum software for appengine: a forum that had german text in tagline couldn’t be edited. All appengine told me was this mostly useless error message: “UnicodeEncodeError: ‘ascii’ codec can’t encode character u’\xdc’ in position 0: ordinal not in range(128)” coming from template.render.

By trial and error I narrowed down the culprit to this part of the template: ”{% firstof forum.tagline “Tagline.” %}”. forum.tagline comes from a datastore, so it’s unicode. Error message tells me that (for whatever reason) template rendering is trying to convert that unicode string to ascii using ascii codec and (rightfully) failing.

Using just ”{{ forum.tagline }}” works so it seems that the problem is limited to firstof statement. The only solution I found was doing the firstof logic in python code and removing firstof from template.

The bottom line is that firstof is useless if you ever hope to use it with non-ascii strings.

I’ve opened a bug for this.

EverNote was the best note taking application I found so far. It’s free and it works on Mac and Windows. Unfortunately the best does not mean it’s good.

While EverNote packs a lot of advanced features, like recognizing text in images, the basic functionality is riddled with bugs and unpolished experiences that I encounter on a daily basis. A real shame for software that has been in the market for so long.

I’ve been using Google Wave for a few days and it looks like it might be a good note taker. It’s inherently cross-platform by virtue of running in the browser, it has a pleasent user interface and all the basic text formating functionality that I need.

It does have its share of bugs, but I’m more forgiving given that it’s a preview release and Google is very clear it’s not a finished product.

ack is a nifty improvement over grep written in perl. However, it’s a command-line app and those days dumping results into terminal window isn’t very inspired.

What would be nice is a Visual Ack, which would present the results in an easier to digest way i.e. an app with native UI with results in a new window, nicely presented.

That’s my lazy web request for today.

I’ve released a new version of SumatraPDF, my PDF reader for Windows.

It’s been 9 months since previous release.

This version mostly improves PDF compatibility (more PDFs can be opened and rendered), fixes lots of bugs reported in the bug tracker and adds UI polish.

Many improvements in this version were contributed by Simon Bünzli and William Blum.

The most visible changes are:

• improved PDF compatibility (more types of documents can be rendered)
• added settings dialog (contributed by Simon Bünzli)
• improvements in handling unicode
• changed default view from single page to continuous
• SyncTex improvements (contributed by William Blum)
• add option to not remember opened files
• a new icon for documents association (contributed by George Georgiou)
• lots of bugfixes and UI polish

My simple productivity application 15minutes used to only run on Windows. I just finished writing Mac OS X version so a blog post announcing it seems appropriate.

I’ve started programming in Cocoa again. I’m only an advanced beginner in that framework so I often have to look up how to achieve even simple things. To make programming a little bit faster, I started working on a Cocoa quick reference.

It lists the most frequently used methods and classes in Cocoa, with links to official Apple docs.

On a side note, I think that existing quick reference guides on the web aren’t very good. I have a few ideas on how they could be improved and hope to use those ideas in my Cocoa reference.

In March I wrote about an idea for tagging links with their sha1 checksum for better caching in web browser. Now I’m not alone.

I have Windows 7 installed on my MacBook under VirtualBox. To keep things simple, I keep my code on mac side, sharing the directory with Windows side.

Tip #1: VirtualBox 2.2.4 has weak implementation of shared folders. For example, Visual Studio 2008 sees the files on shared drive as write-protected (even though other applications don’t). Visual Studio 2010 doesn’t have this problem, but it can’t finish compilation, complaining it cannot write some intermediary build files to a shared folder. Also, trying to mark shared folder as trustworthy for .NET doesn’t seem to work (see below).

Solution: use Mac’s built-in file sharing instead and shared things out using built-in samba (smb) sharing in System Preferences/Sharing.

Another problem is that .NET by default doesn’t trust code on network drives and Visual Studio will tell you that “The Project Location is Not Trusted” if you open a project from a network drive. The magic incantation to make a given shared folder (in my case it’s drive z:) trusted is:

caspol -q -machine -addgroup 1.2 -url file://z:/* FullTrust

This command has to be executed as Administrator.

One way to have caspol.exe available in the PATH is to start Visual Studio cmd-line (shortcut is installed in Start menu).