Trogdor login screen

I think in general, it is bad security practice to display a lot of identifiable information at an SSH login prompt. If there's a known exploit for distro X version XYZ, you don't want to give someone a fast path to utilizing it.

I was setting up a new test VM at work, so I decided to have fun...and came up with (possibly) one of the greatest things ever. The trogdor ASCII art login screen.

Step 1 - search alt.ascii-art for "trogdor" on google groups
Step 2 - copy paste and save to a file on your server
Step 3 - configure SSH server to display a banner

 

The only down side is that my PUTTY window needs to be long enough to see the full awesomeness of this trogdor ascii art. I take zero credit for the ascii art, which you can find by clicking the google groups link above.

I'm kind of a newbie at configuring SSH, so I'm not sure if banners always display after you type your username...

Solution to a slow syncing iPad

I noticed sometimes my iPad takes FOREVER to sync. Specifically, my ipad takes forever to backup before it starts syncing. I recently stopped syncing it to my macbook pro and began syncing it on a windows 7 system - iTunes has been behaving awesome for me on Windows, contrary to frequent opinion. At first, I was careful, and only ever used my iPad cable... but it's just a USB cable, so eventually I slipped and starting using my iPhone cable.

I'm not sure what's special about the iPad cable, aside from combined with the iPad charger it is a higher power charger, but it appears that backing up your iPad (even when it has practically nothing on it - as I recently cleaned mine out) can take ages if you use the wrong cable. Facepalm. It took some reading up on this to realize what I as doing wrong. It's a double facepalm for this even being an issue - I am not sure why an iPhone 3G or 4 cable would cause slow syncing...but it looks like it does. In my case it's slow backing up...like 30 mins to an hour when nothing has even changed. Problem solved though - use the iPad cable.

Oracle: How to find the service name of a database instance (SID)

At work I ran into a surprising issue. It's hard to find a service name for an Oracle database when all you know is the SID if you don't know exactly where to look. Hard, even for large, corporate Oracle deployments with DBAs you can ask. I found it hard to believe, but we've had issues several times now where it has been difficult for customers to determine a service name for their oracle database instance. Now, of course our application lets you enter SID or Service Name and handles URL formats for you and everything - that is not the problem. The problem is when administrative work needs to be done. It usually isn't an oracle DBA handling administrative tasks for our software at a customer site, since the DBAs are a precious resource.

Now, first a quick refresher:

SID - System Identifier uniquely identifying a database instance. Each database instance has an SID - one and only one.
Service Name - An alias to one or more INSTANCES (useful for clustering, failover, without changing end-user configurations) - introduced in Oracle 8.

One SID could have a hundred service names pointing to it if you wanted to...but I'm pretty sure at a minimum it will have one. When you create an instance, Oracle makes you name them both. The 10g Configuration Assistant on Windows has a field named Global Database Name, which maps to Service Name. The SID is
frequently, but not always, the same as the initially specified Global Database Name.

After the creation of a database instance, Oracle tools refer strictly to the SID - after all, that is the name of the instance you'd be configuring. Since Service Name is just an alias, you wouldn't configure the character set or memory options of an alias.


Why do I care what the service name is if I have the SID?

Because knowing it lets me avoid relying on properly configured tnsnames.ora on each client machine. Sometimes, tnsnames.ora files appear to be controlled by a dark magic - the same tnsnames.ora file works fine on one machine, but fails inexplicably on another client machine. Sometimes TNS doesn't work right - and for a Java application that's going to be using JDBC, needing to configure tnsnames.ora is unnecessary work. For the command line tools like SQLPLUS and the import/export tools, you can use the EZCONNECT URL format and avoid TNS completely. There's a catch - EZCONNECT only accepts Service Names.

So, how do you determine the service name? Here's one way...

This assumes you have access to the oracle server. Provided you do, simply use the command:

lsnrctl status

This will print out various status information, as well as a compact listing of configured service names and which instances they point at. You could also use lsnrctl services for slightly more verbose services output - but depending on how many service names and instances there are, it may be harder to look at.

It's kind of anti-climatic, but that simple command ended the confusion. I thought Service Name was preferred (as it is more flexible than directly specifying the SID), but apparently it isn't always the case.

Booklist updated, no Tuesday post

No Tuesday post, but I did spend time working on a longer post that just isn't quite ready yet. Sleep is more important...

I did update my bookshelf the other night though. Eventually I'll better separate it into a recommendation section, what I've read, and my current queue of books awaiting reading. I often have bigger eyes purchasing books than I do reading, so I always have a long backlog.

Stumbleupon is neat

A coworker recently told me he finds some really cool stuff using Stumbleupon. Now, I had heard of Stumbleupon before, but I had never tried it. I'm not sure why. The concept is cool.

I gave it a spin today, and here's my experience so far. Account sign up? Fairly easy, though it did kind of nag me to use a facebook account and import some friends or something. Once that was setup, it was fairly straightforward.

So far I've found a couple of really cool links, after telling it I liked ~35 things, and disliked 5, and skipping a few.

• 28 Rich Data Visualization Tools - some really cool graph and chart tools
• UX Movement - a great site about user experience. I'm already learning some useful things, like better ways of presenting form data and search buttons. 

So, Stumbleupon is pretty neat. That's it for now.

November month in Writing: Week 2 summary

I was able to broadly stick to my writing plan last week, as well as converted a couple of my drafts into finished posts. I read just shy of the first 50 pages of "Release It!", and I am enjoying it so far. Every weekday post was scheduled for 8:00AM. This doesn't appear to make a difference when you have no readership :) I need to write some meatier posts that I can submit to Dzone - that does makes a difference.

I failed at pursuing a Hudson / Opera issue that has plagued me. I'll restate the goal for this week... I just need to go through history, merge some modifications to prototype.js into prototype 1.6, then create a patch file and see if anyone is willing to take my patch for a spin (as well as stage it in my environment for a while).

As for general week's news:

Oracle and Apple Announce OpenJDK Project for Mac OS X  - the future of Java on OS X seems assured now. Cool.

Mozilla released Firefox 4 Beta 7 - this added JIT to their Javascript engine, and it's now faster than everything except Opera 11 alpha at Javascript. Woot - I was beginning to think IE9 would be faster, but I am glad to see Firefox 4 gain some serious speed. Competition is good, and it is my #2 browser.

Gold Nanoparticles Could Transform Trees Into Street Lights - this is just amazing. I spotted this on twitter and then later on engadget...

No specific ideas for topics this week yet...we'll see what happens.

muCommander - the ultimate cross platform utility

Despite NTFS supporting very long path names, Windows frequently has issues with long filepaths. Sometimes you need to bring in another tool for the job.

My favorite tool for dealing with long filepaths is muCommander. It is a Java app that mimics the traditional "Norton Commander" user interface. Java uses the API in Windows that supports longer filepaths, so muCommander has no trouble dealing with extra long file paths. It's not just for Windows, though - it's available on a lot of platforms.

muCommander is much more than a better file explorer, too. It has built in support for loads of protocols:

Virtual filesystem with support for local volumes, FTP, SFTP, SMB, NFS, HTTP, Amazon S3, Hadoop HDFS and Bonjour

You mean I can have my local filesystem on one side, and an SFTP session on the other? Awesome.

It can even open and edit zip files in-place.

Plus - it has a sweet logo. 

 

muCommander is an awesome utility to keep in your toolbox.

Replacing Notepad with Notepad2

I've been using Notepad2 as a light-weight replacement for the notepad.exe that comes with Windows for several years. Let's face it, the original notepad.exe falls short on many levels - no auto-tab, no multi-line tabbing/untabbing, poor line endings support to start... I've always felt it was inferior to even the old MS-DOS EDIT, which is a pretty sad statement. In comes Notepad2, fixing all of its shortcomings while remaining lightweight, and adding some other nice features as well.



It's always been somewhat of a pain to perform the replacement to Notepad2, since notepad.exe is considered a protected operating system file in XP and Windows 7. Fortunately, Kai Liu wrote an installer that does this for you automatically! I just learned about this this week and figured I would share.




To download the installer, just go to http://code.kliu.org/misc/notepad2/ and scroll to the bottom, "Custom Notepad2 Builds", for both 32-bit and 64-bit machines.

Desktop Java Apps - to bundle or not to bundle a JRE?

First - here is my definition of bundling the JRE. Shipping your application with a JRE, such that it is the one used by default from your application startup scripts, regardless of the environment on the target system. The bundled JRE is not registered with the system, so it should not affect other applications on the system.

There are probably some good reasons not to bundle a JRE. Taking a moment to ponder, I can think of at least the following:

• Bloat
• No automatic security updates
• Less portable

There may be more - but I believe the pro's will outweigh them regardless, once I add some additional constraints. Specifically, I am concerned with Java on the Windows desktop. Further, I am talking about larger, high engineering effort commercial software deployed to big companies. Not a small, independent Java app deployed to individual consumers. (although there may still be cases where that simplifies things). At a minimum, I would argue for stating "Use 1.6 Update X or higher" somewhere in a README, even for a small app. I've just seen too many weird issues not to.


On to some reasons for bundling a JRE:

• Control over major Java version (1.4,1.5, 1.6) - this is less of an issue today, since 1.5 has reached its end-of-life and 1.4 is far behind that. Thank goodness for that.
• Control over the exact version (i.e. 1.6 Update 20) - to ensure mandatory functionality works properly
• So the end users don't need to install something additional
• So your program is not influenced by the system environment
• So your program does not influence other applications in the environment 
• Less variables for when customers report bugs
• No automatic updates - which could break functionality
• Less to test

See a theme? Better control. Also, better isolation from the environment and other things that can go wrong.

For instance, in the 1.5 days, I recall one release that had very misbehaved tabbing through components in a UI. In the era of Windows 7 - you need a minimum of 1.6.0 Update 18 - or you have missing icons in Open/Save File Dialogs. It is not acceptable to have a UI defect like that in a shipping app - any less, and you don't have full Windows 7 support. Except on 64-bit systems Update 18 has a bad habit of crashing the JVM a lot...so you need a more recent update. Then there's the Swing/AWT changes in Update 12 that alter behavior. Then I'm pretty sure depending on the version, applets load differently, if that's a factor (in one of our cases, integration with another process - it is).

You can avoid all that by picking a version you know works for your required situations. This means less to QA. Less potential headaches. The trade-off is losing faster security updates, increased application size (which believe it or not - still matters even in the age of 3TB hard drives), and portability. If you are in a Windows-only market, as I have seen several applications be, then portability isn't affected. Even if there are other target platforms - it's just a matter of repackaging for other platforms.

I believe there are some good reasons to bundle a JRE - on Windows, anyway.

Usability Fail - Confirm Then Swipe

I consider a fundamental goal of software, and technology in general, to make people's lives easier. To simplify, automate, and perform useful functions. When the above holds true, technology is a success. When technology gets in the way, is a hassle to use, or does not otherwise improve an existing process - it is a failing and does not have purpose.

First, a success story. Trader Joe's is a very pleasant place to shop, all the way through. The checkout process is one reason for this great experience. The first time I went to one was several years ago, and I was pleasantly surprised during checkout.

The Card Swipe device:

1. Made it immediately obvious that I could swipe my card at any time
2. Allowed me to enter DEBIT, PIN, Cashback options, all before the cashier had even finished scanning my items
3. Provided a final confirmation once all items were scanned

This is pure win, on all levels. I have true respect for the makers of this device. Others get 2 & 3, but some fail to make it obvious to the user that they can swipe early. No one likes waiting in line - this helps people wait in line for less time - both the person using the card swipe, as well as anyone behind them.

Imagine my surprise when my local Market Basket got new card swipe devices a year or two ago, and they did not share this amazing feature. On the contrary - it required the exact opposite. Enter the "Confirm Then Swipe", a giant usability fail.

Here is a basic rundown of the Bad Card Swipe Device (TM):

1. You must wait until the cashier has scanned the final item
2. The cashier then usually asks credit or debit.
3. The cashier reminds you to "Confirm then swipe"
4. You confirm the amount
5. You can then swipe your card
6. Debit PIN
7. Oops - you got your PIN wrong, try it again 
8. Final confirmation (?)

What went wrong for a device with such a basic purpose to have been made this way? Is it short-sightedness of the developer? Is it due to the other vendor holding software patents? (more on software patents another day...) Whatever the cause - it really bums me out. How many wasted words are there on a daily basis, just to explain to the customers that they must confirm first? (plus the time that could have been saved if the swiping process was completed while items were still being scanned) How much time is lost by the patrons due to this every day? Per Year? This should not be the case, but sadly it is.

Thoughts on OpenID

I've had an OpenID for over two years now, since I joined StackOverflow during the beta. A lot has probably happened since then. If memory serves correct, at least one OpenID provider went out of business. Others may have changed ownership - and new ones have surely emerged.

How have my first 2 years with OpenID been? Well, I've only set it up on a whopping 2 websites, both of which are technically oriented. I have recently started seeing OpenID as an option on a few more websites, but not a lot.

I attempted to set it up on a third (DZone), and although it allowed me to login, I could not verify my account, or join it to my non-OpenID account. This is probably just an issue with DZone, but if it was important enough, I would hope it'd be resolved.

What implications are there to third party authentication such as OpenID over time? (say, 5-10 years). Quite a lot can happen to a company in that timeframe. Especially tech companies.

Is it easy to switch OpenID providers? It seems like that is limited by whether every site you use supports adding a second OpenID or not. That may be a requirement of participating, I have no idea - so far, the two I care about have supported this.

What happens if my OpenID provider, say, starts getting hosted in China. What happens if my OpenID provider goes down for good? Will I ever be able to reclaim my account?

What happens, for instance, if my OpenID provider's SSL certificate expires? I can't get to my websites unless I accept an expired cert?

For the record - back in August (when first typing this), the MyOpenID login page was doing just this - showing expired certificate messages. Even though it wasn't apparently needed for authentication (because rest assured - I reject expired certs), it was still alarming. I hadn't even seen the domain name before - which was also worrisome.

This is what got me pondering OpenID. The concept is nice, but is it succeeding?

I am glad Google is a provider - would OpenID be at all usable still if they weren't? I at least feel safe that my Google account isn't going anywhere anytime soon. Also, it is rather convenient as I'm usually logged into gmail and just need to confirm to login.

I am curious what others think. Do you use Open ID? Do you think it is succeeding - is it convenient, or a pain? What provider(s) do you use? What websites use Open ID for the sole authentication system?

November month in writing: week 1 summary

One week down - three to go. This year I am aiming to write my posts a day early, scheduling them for 8:00 AM the following day. That worked last week - but I am off to a late start this week. Fortunately, as of this writing, I already have the next two days' posts finished and scheduled to appear.

• Tuesday will be some thoughts on Open ID
• Wednesday will be a usability post
• Thursday - goal is to discuss JRE bundling for Java on the Desktop
• Friday will remain a mystery

As for last week, I felt OK but didn't think I wrote as much as I wanted to. Hopefully I can find room for a recent software experience and maybe a tutorial in the coming week or so.

Goals for this month:

• Finish reading two books (not succeeding so far)
• Finish tracking down a Hudson / Opera issue and hopefully submit a patch
• Finish converting more of my drafts into completed posts

Distributed Source Control Rocks

Hudson core still uses Subversion, and the java.net svn server is kind of slow. I needed to do some diffs when investigating a fix for an issue I encountered. I had to wait a couple of minutes for every single diff I wanted (from the log, not local diff).

It just so happened it was related to Prototype, a popular Javascript library, which is hosted on GitHub. I took a look on that end as well. I cloned a local repository - simple, fast, and instantaneous SCM operations.

I'm still a Git / Mercurial newbie... but it's obvious that if you're a remote user, centralized vs distributed source control isn't even a fair comparison. It's that much better.

News and Links Nov 6 2010

Marco Arment had a great post about technology - Developers don’t rush to new platforms. Everyone considering getting new smartphones or tablets should definitely read this. It's short and to the point.

I've been looking at our database performance at work recently, so I've been digging around a lot of MySQL and related database info this week and found this blog - EXPLAIN EXTENDED. In particular, this week's post:
10 things in MySQL (that won’t work as expected) - I'll be subscribing to this one, and maybe you should too if you use MySQL.

iOS 4.2 went to GM seed this week. That should mean 4.2 will release for iPads next week if no criticals problems are found. I can't wait.

Seven Languages in Seven Weeks: A Pragmatic Guide to Learning Programming Languages by Bruce Tate is now out. I can't wait to read it! I better make some progress on "Release It!"...

Browser Benchmarks November 5 2010

It's been a few months since I last benchmarked web browsers, so I figured it was time for an update. There's just something innately fun about running benchmarks. These are first and foremost for my curiosity and amusement, not because I think it matters if Browser A is 0.3% faster at Sunspider than Browser B. However, at least last time there were very measurable gaps in performance and results of each browser - more than I would have thought.

This time I'm going to separate the results of official stable browser releases and beta releases.

Browser releases continue to be frequent, and I don't think it is going to slow down. Chrome in particular appears to now plan on major releases every 6 weeks. Last benchmark was Chrome 5 - this time it's Chrome 7. Opera has some modest improvements with 10.63, Firefox 4 is on beta 6 now, up from beta 1. IE 9 Platform Preview has also gone through some more iterations (though it is hard to find)

System specs are the same as last time, except for new video card drivers, which may affect 3d accelerated HTML5 tests.

Environment:

• Intel Core i5 750 @ 3.36GHz (Turbo Disabled)
• 4GB RAM
• 80GB Intel SSD (G1)
• ATI Radeon HD 4850 (Catalyst 10.10 drivers)
• Windows 7 64-bit Home Premium

Fresh boot. No other apps or system tray programs running aside from Microsoft Security Essentials.
Each browser was run by itself, with only one tab for the benchmark itself.

I'll start with Sunspider again. Only this time with Sunspider 0.9.1 (I didn't realize that was available last time):

• Chrome 7.0.517.41 - 208.3ms +/- 0.5% [Full Result]
• Opera 10.63 - 224.4ms +/-0.5% [Full Result]
• Safari 5.02 - 291.3ms +/- 0.6% [Full Result]
• Firefox 3.6.12 - 638.1ms +/- 2.9% [Full Result]
• IE8 - 3614.1ms +/- 0.2% [Full Result]

 Separately, beta browsers:

• IE9 Platform Preview 6 - 249.1ms +/- 1.0% [Full Result]
• Firefox 4 Beta 6 - 364.4ms +/- 3.5 % [Full Result]

No real surprises - Chrome continues to lead, but IE9 and Firefox 4 betas are catching up to the 3 leaders. There is much less spread than last time.

I also decided to run The Kraken, a new benchmark the Mozilla team released in September.This will be the only benchmark Firefox 4 beta will overtake the others in. I am curious how accurate of a representation this benchmark will be of the future.

• Opera 10.63 - 10515.7ms +/- 0.5% [Full Result]
• Chrome 7.0.517.41 -12046.8ms +/- 0.4% [Full Result]
• Firefox 3.6.12 - 14467.1ms +/- 0.1% [Full Result]
• Safari 5.02 - 14589.5ms +/- 0.1% [Full Result]
• IE8 - I did not wait for this to finish.

Separately, beta browsers:

• Firefox 4 Beta 6 - 9768.9ms +/- 0.1% [Full Result] 
• IE9 Platform Preview 6 - 21015.6ms +/- 0.2% [Full Result]

Firefox 4 beta 6 leads Opera 10.63 by a slight margin. Chrome trails by a bit more, followed by Safari and Firefox 3.

Peacekeeper:

Firefox 3 appears twice because I didn't realize 3.6.12 was out at first. All browsers except Opera see gains from last time. This could be due to my updated graphics driver.

ACID3:

• Chrome 7.0.517.41 - 100/100
• Safari 5.0.2 - 100/100
• Opera 10.63 - 100/100
• Firefox 3.6.12 - 94/100
• IE8 - 20/100

Beta Browsers:

• Firefox 4 Beta 6 - 97/100
• IE9 Platform Preview 6 - 95/100 (noticeable improvement from preview 3)

CSS3 Selectors Test:

• Chrome 7.0.517.41 - Passes 574/574 tests
• Safari 5.0.2 - Passes 574/574 tests
• Opera 10.63 - Passes 574/574 tests
• Firefox 3.6.12 - Passes 574/574 tests
• IE8 - Passes 345/574 tests

Beta Browsers:

• Firefox 4 Beta 6 - Passes 574/574 tests
• IE9 Platform Preview 6 - Passes 574/574 tests

I must say, the IE9 team seems to be taking web standards pretty seriously. I recommend taking a look at their test center. This will be good for everyone. I'm not a web developer currently, so I can't say if they are truly implementing everything that's important, or just a higher amount of obscure features, but in the end it is a net win. One of the reasons I've always liked Opera is that they strive to adopt web standards.

I noticed something weird in Firefox 4 and Firefox 3.6.12 (it must be a recent thing). By default, when I type into the search box after a fresh install - nothing happens. The browser appears to now require search providers, a la IE, except it doesn't come with any by default.

UPDATE: This is why beta browsers are listed separately. I installed Firefox 3.6.12 on my work PC and it does not have this issue - neither does Firefox 4 beta 6, once it is installed The above must be a fluke. My guess is that one of the Firefox 4 betas somehow corrupted or cleared out the search engine extensions, and this also affected Firefox 3.6.12 on the same machine.

Overall the browser situation is getting better and better. When IE9 comes out, IE8 can finally start going away - it is really the only browser that I can't recommend. Most of the rest come down to personal choice. 

Do you want the absolute fastest browser, but with a few less features? Chrome.
Do you want the ultimate in customization, if you want to download some extensions? Firefox.
Do you want more features built in, and almost as fast as Chrome? Opera.
I'm not sure if there is a compelling reason to use Safari 5 on Windows, but it also isn't a bad choice.

Until next time.

Java Roadmap Disappointment

Am I the only one disappointed with the new Java 7 road map? It seems both late and lackluster.

First, let's look back at past Java releases (courtesy of the first google result I saw with actual release dates)

J2SE 1.3 2000-05-08 Kestrel
J2SE 1.4.0 2002-02-13 Merlin
J2SE 1.5.0 (5.0 to salesmen) 2004-09-29 Tiger
J2SE 6.0 (1.6.0) 2006-12-12 Mustang

What has been happening the last 4 years? Apparently a few things I didn't know about (politics, JCP, Apache Harmony, etc). Most of which I'd need to read up on a lot more before I could write about it...so I'll stick to two things. 1) Apple OS X Java thing, 2) The updated Java 7 release timeframe and scope.

On the Apple deprecating the shipping of Java with OS X - I think Ted Neward summed it up best. What is potentially most interesting is depending on how Apple handles phasing out maintaining Java on OS X, and if Oracle picks up writing a JRE for OS X. If so, will it impact the Java 7 release date? I think he has a point though - is there even a large demand for Java on the Mac? (And I say this recalling having compiled landon fuller's openjdk6 port on my Mac prior to Java6 shipping on my MBP almost 3 years ago...)

On the updated timeframe and scope given by Mark Reinhold...

I was really hoping Lamba expressions, often referred to as closures, would make it into Java 7. A mid-2011 release (another delay) without them is pretty disappointing. Java feels stagnant. The primary strength of Java for me is the tooling, not the language...but it would still be nice if the language was developing faster. Maybe it's time I started checking out some other languages that run on the JVM.

There should still be a few neat things though...looking at the list:

• invokeDynamic bytecode - should be great for all the dynamic language people running on the JVM
• related to the above...method handles. First I've heard...this requires more reading. 
• Strings in switch statements - at last
• NIO.2

Ending my rant on a positive note...I suppose there are still some cool new things. I'd install an openjdk7 build, but it would just make waiting harder :)

iPad Thoughts, Part II

There are a couple of topics I forgot to mention in my first iPad thoughts post.

More favorites apps

The new Twitter app for iPad is awesome. There are so many great UI concepts happening, it is hard to describe. Pulling and sliding and locking important things in place. Kick. Ass.


Another app in my favorites for iPad is iSSH. It is a very solid SSH client - I can SSH into critical servers from anywhere using my iPad. While also for iPhone, the iPad's onscreen keyboard is a lot more usable (and iSSH has extra keys available). It's possible to get meaningful work done when SSHing from the iPad. And that is cool.

More games:

• Angry Birds. 'Nuff said.
• Solipskier - very simple, entertaining game where you draw slopes to ski on. 

Upcoming games:

• Epic Citadel - I can't wait until Epic releases the actual game
• Ditto for id Software and their Mutant Bash TV 

The iOS Notification System Is Lame


My only other gripe with the iPad, and this is iOS in general, is notifications. I haven't historically been a heavy enough user, so they didn't bother me much in the past. With both iPad and the iPhone 4, Apple's notification system is showing its age. It's annoying even without multitasking (e.g. current iPad - to be remedied later this month) as it will just stop whatever you are doing when they pop up. On iPhone 4 it is even more noticeable - oooh yes, you thought you could multitask away from this app...except I just froze the app you are using to show you this nice notification dialog! Foiled!


Notifications need to be fixed. Pretty please, Apple?

At this point we're at GM seed of iOS 4.2 for iPad, so iOS 4 on iPad should be a reality in the next few weeks as planned. Any day now...

Bogus file corruption issues with TortoiseSVN on Windows 7 64-bit

Last Thursday I encountered a strange and frustrating issue. There are a wealth of blog entries out there about this, for anyone who bothers to search, but I am going to summarize it regardless. I myself didn't google it until after it happened to a coworker as well - at that point I knew something was up.

I've used Windows 7 64-bit with Microsoft Security Essentials at work for over a year now. For the last 4 months, I've had an Intel SSD in my workstation at work as well, and did not hit this issue until last week, when it hit hard.

I could not successfully checkout a new Subversion working copy to my SSD - period. Every attempt gave me the following message, in my case after 5-10 minutes time. Repeat attempts (update to attempt to resume the checkout) would cause it to happen again until finally the workspace could not be updated or cleaned.

Each time I would see the following error:

Error Can't move
Error '...\path\to\.svn\tmp\entries' to
Error '...\path\to\.svn\entries': The file or directory is corrupted and unreadable

Following this, the next reboot windows would schedule a CHKDSK on my SSD drive. Despite it not finding any issues, I still got a sinking feeling. Checkouts to my regular hard disk worked fine, aside from crawling in comparison. Hmm... As it turns out - my SSD did not have corruption problems. It also isn't TortoiseSVN's fault. It's a bug in Windows 7.


Fortunately, there is a secret agent style hotfix (email registration, download link with password that expires in a few days) that came out in June. It will apparently be included in Service Pack 1 when it is released.

Symptoms listed from the hotfix described exactly the behavior I was seeing:

For example, you run an application that uses the MoveFileEx() function to replace files. If you use this function more than one time in a short time, you may receive an "ERROR_FILE_CORRUPT" error message. Additionally, you may encounter one of the following issues:A warning message is displayed in the notification area. This warning message indicates that you should run the Chkdsk.exe utility to check the disk.
A disk check is scheduled for the next time that you start the computer.

Subversion uses the MoveFileEx() method for replacing files in place. The reason this didn't happen on my regular HDD, is that the issue is specifically when MoveFileEx() is called in fast succession. SSDs perform the actions must faster. They are apparently so fast, they uncover obscure filesystem implementation bugs in NTFS. Have I mentioned SSDs rock for development?

For the curious - file indexing and anti-virus could also apparently make this problem worse / more apparent, but disabling Antivirus did not help in my case. I didn't try disabling indexing, but I did not have whole-drive indexing enabled anyway. The hotfix completely resolved the behavior on my machine.

Here are some other reference links I found helpful. Hurrah for ServerFault, which had the first useful result in Google.

http://serverfault.com/questions/72561/64-bit-tortoisesvn-on-windows-7-says-file-or-directory-is-corrupted-and-unreadab

http://tinyurl.com/w7corrupt

It both amazes me, and ceases to surprise me that bugs like this shipped in a production OS. I mean...think of the madness one could go through thinking your HDD was corrupt, or software faulty, if you just believed the messages and didn't search. It is due to this that I post this entry, hopefully it will save at least one person a few wasted minutes.

This is also the second hard drive related issue I have encountered with Windows 7 so far. The other had to do with my 1TB hard drive disappearing after resuming from standby - it was something along the lines of Windows 7 not waiting long enough for drives to respond on wakeup, and larger drives can take longer to wake up and respond. I forget if I had to apply a hotfix, or update a driver or registry setting to fix that problem.

This isn't a rant about Microsoft, this is just another good example that writing software is HARD.

It's November again...

Last year I did my own take on http://www.nanowrimo.org/ by writing in this blog every day for a month. I plan on doing it again this year. Hopefully the writing habit will stick.

I intended to post this last week, but as it is already November 1st - I guess I am already behind.

I have quite a backlog of miscellaneous, semi-completed blog entries in the past month alone, so the first week may be completely random. I would like to come up with some topics that I can focus on in the weeks following - we'll see how it plays out.

There is a lot to talk about right now in the Java world.
- Where is Java headed? Feelings on the Java 7/8 roadmap
- Is Oracle scaring off developers, and what will happen to Android with Oracle vs Google?
- What is the future of Java on the Mac?

The web also continues to be full of interesting advancements, especially due to the iPad and mobile devices.
- Some very impressive things can be done with HTML5 + CSS3. What are some examples?
- Fierce web browser competition - it's hard to go wrong nowadays.
- Smart phone competition (Android, iOS, WebOS, Windows Phone 7, etc)

In addition to (hopefully) talking a bit about the above, and more, I'd like to lay out some other goals for the month:

- Start and finish reading Release It!, by Michael T. Nygard

I'll add more goals and topics as I think of them. Time to start writing Tuesday's post...

Ant DirectoryScanner memory problems with large directory trees

I encountered a very interesting build problem this week. Hudson running out of memory when trying to parse test reports. It wasn't actually even getting to the parsing part though. Thread dump looked innocent, Ant...hmm... so I took a heap dump and examined that a bit. Top memory use? Ant's DirectoryScanner. Ant's DirectoryScanner was running out of memory just trying to round up the test report XMLs. Even with a 1.3GB max heap, it was not sufficient to scan our workspace anymore. Peculiar.

From looking at the heap dump, DirectoryScanner holds on to unincluded directories. This doesn't seem right to me... I may dig further when I have time. Simple searches showed some archived mailing list discussions, but I don't think it's related. Ant 1.6.5, 1.7.0, 1.7.1, 1.8.1 all observed this behavior for me...so I do not think it is a regression. A simple ant goal run on this workspace - pure ant, couldn't scan the dir until it had a 1400m heap, and to do so took it 96 minutes because it was still memory constrained.

The pattern being used was fairly innocent looking:
koko/dev/projects/**/target/test-reports/TEST*.xml

Except it isn't that innocent when the root workspace dir has 300k+ files, and 1,000,000+ directories in it, with some very long path names on top of it. As it turns out, that ** is expensive and should be used with great care. Even though it is only traversing our code projects, it is still a very large directory tree and it runs out of memory.

More important than the workspace dir count, which should be getting skipped with the start of the pattern, the dir count within /projects/ is still almost 300k dirs.

Maybe this is a newbie mistake with Ant...but I'm not so sure. There are times where ** may be necessary, and I have a newfound awareness of how expensive this can be (in time taken, obviously, but more importantly in memory - it's no good if it fails because it runs out of heap...)



For a smaller directory tree on my dev machine, I wrote a short ant target that would touch 1 file, found using an include pattern similar to the above. The ** version took 24 seconds to execute on an Intel x25-m 160GB (G2) SSD. the * version took 0 seconds. The time penalty of ** would be much larger on slower, traditional hard drives...

Bear in mind, for Hudson users - Findbugs, Warnings, and other plugins typically use Ant include patterns (and I am sure Ant's DirectoryScanner), and in those cases sometimes ** is unavoidable.

I need to fire up a Hudson workspace and ensure that it supports multiple includes, so users (me!) can avoid ** in more cases...

Don't Filter Without Being Aware of the Scunthorpe Problem

I just happened to stumble across this question today while browsing StackOverflow. In it, a user is searching for a library that will filter profanity. Some suggestions are posed - as well as a problem with one of them. The Scunthorpe Problem. The gist of it is that naive filtering implementations will often filter innocent words that contain sequences of letters that are considered dirty by themselves.

I highly recommend reading the full wikipedia page - it is highly entertaining.

If you ever find you need to implement a profanity filter - make sure you don't forget about the Scunthorpe problem, or you may cause your users unnecessary strife.

My favorite word resulting from the scunthorpe problem: buttbuttinate.

More on e-reading, and the Pragmatic Bookshelf

I had some ranting the other day on my first experimentation with e-reading, but afterwards I learned something cool to offset the potential downer. I had completely forgotten that I have PDF versions of lots of books from the Pragmatic Bookshelf.

I love the Pragmatic Bookshelf. Not only is it full of awesome books, but books I bought 5 years ago in print+pdf I can now convert and put on my iPad at no additional charge. This is the definition of sweet. All I had to do is log into my account, tell the gerbils which books I wanted in what formats, and waited a minute. There are a couple of old ones that are still only available in PDF - but I can still view those on my iPad, it just isn't as convenient yet. The Pickaxe is a notable example...but I'd say more than 75% of my books had e-reader formats available. Again, iOS 4 for iPad can't come soon enough - in this case for iBooks PDF reading.

At a bare minimum, I know that I can enjoy a good set of books on my iPad going forward. Exciting. Especially since there are still a couple I haven't read yet, and some I intend to re-read soon. There are countless more that I desire...

It's high time I create a bookshelf on this site, come to think of it... I'll put one up in the next week.

Adventures in E-reading

I read my first book and a half on the Kindle reader for iPad last weekend.

Specifically I enjoyed:

• Masters of Doom - a great book primary about John Carmack and John Romero. If you are a fan of the genre, it's a great read. I stayed up past 4am reading, it was so engrossing. I remember being on the Software Creations BBS, which is mentioned in the book.
• Peopleware - Productive Projects and Teams (Second Edition) - a highly recommended book. I'm only half way done, so a full write up will have to wait, but it is a very insightful (and also quick) read. 

Why did I choose the Kindle reader over iBooks? Kindle has a much better selection of books. So far iBooks is 0 for 3 on books I have wanted. Also - the Kindle reader is available on other platforms.

I love reading on the iPad. I can read with a light on - or without. The text was very readable and my eyes did not get put off by the backlit screen even after reading for 5 hours straight as I tore through Master of Doom.  Even though I already owned Peopleware in print, I paid the 10 bucks for the Kindle version anyway. That's a testament to how much I enjoyed reading on the iPad.

All is not well in the land of e-reading though. While I had no quality issues with Masters of Doom, Peopleware seems to be full of missing punctuation, at least one blatant typo, and a chapter that is in the wrong place. I did a quick search - Apparently I  am not  the  only one. Since I have the print version, I have verified that none of the issues I have seen occur in my print copy. What is going on? Could it be that physical books are being OCR'd, or worse, re-typed by hand? What on earth??? Shouldn't digital copies already exist at the publisher?

Here is an example from the Table of Contents in Peopleware:

Actual TOC from the Paperback copy

Kindle version

The chapter appears in the wrong place in the book. Intermezzo should be between chapters 9 and 10, not between chapters 8 and 9. Also, less importantly, the titles seem to have been truncated, and in the case of #8, the quotes are missing.

I found at least several instances of missing punctuation while reading so far, as well as one glaring typo:

"The only acceptable interruption there was a fire alarm, and it had to be for a real Tire."

Somehow a lowercase f turned into a capital T...

Maybe some more veteran e-readers can tell me if they run into this a lot. I find it seriously distracting when sentences are incomplete or I find typos in books. Especially if they are an artifact of the e-book translation, and not something the original editors missed.

Is this the state of ebooks these days? I hope not, or my adventures in e-reading will be short lived.

Stage. Stage. Stage. Always Stage.

It doesn't matter what software we're talking about.

• MySQL
• Hudson
• Java releases
• thirdparty libraries
• Subversion client
• you name it - if it can be upgraded or replaced, it can and needs to be staged.

 Always stage. ALWAYS.

If you don't stage, you're just punishing yourself (and potentially your coworkers and/or users). So be smart - always stage your upgrades. You'll get burned if you don't. It's just Murphy's Law applied to technology, really.

This isn't in response to recently getting burned myself, it's been a post that's been brewing for a while.

Many years ago, a friend of mine once wondered why an admin wouldn't upgrade PINE (a popular terminal email client at the time) as soon as it came out. Now I know why - because that admin was wise. They wanted to be sure it worked and didn't cause any regressions before subjecting their users to it.

I am continually surprised not just by how many things can go wrong, but also how frequently they do, despite the best efforts and intentions of developers. Just the other week, the tinest name change by Oracle in JDK 1.6.0 Update 21 caused Eclipse to fail to launch. Who'd have thought? And yet it happened. There are numerous other examples. I am sure you can think of plenty in whatever software you use. I can certainly think of several, just in the past couple of months.

Before your whole office upgrades to the latest version of Visual Studio, or the latest Subversion client, or any other software. Make sure someone tests it out first.

Stage, because anything that can go wrong will go wrong.

iPad Thoughts

I bought an iPad in early May. This post is a summary of my thoughts when i first got it, up till now, 3 months later

Opening Thoughts and The First Month

First off - you really need to use an iPad to appreciate their coolness. They are incredibly slick.

I found immediately after getting an iPad that I almost never needed to turn on my desktop at home. I also didn't really need to turn on my laptop for much. It satisfies most browsing, email, RSS, and other media needs. The battery life is incredible, especially for watching video. You can read books from iTunes, Kindle, and Nook stores. There are lots of fun games for it. It's a great web browser.

It has really changed how I consume media. Let's not forget it also does Comic books - although selection is still not excellent, I have enjoyed one or two series that way.

My iPad is my first choice entertainment device wherever I go. It is also my note taking device in meetings at work. It's handy for note taking + browsing our issue tracker + continuous integration + email all in one. It rocks.

Favorite Apps

Below are some of my favorite apps.

• The Early Edition
• Twitterific
• Comixology
• Instapaper
• Netflix 
• ABC Player
• We Rule 
• Cogs HD
• Plants vs Zombies HD
• Leap Sheep!

Problems

The iPad is a device I use every day, but there are a couple of clear annoyances popping up.

1. The iPad needs the iOS 4 update BADLY. It feels like it's a second class citizen now that the iPhone 4 is out and has iOS 4 and multitasking. Apple really better have some extra features up their sleeve, or the wait just is not justified (and is crummy for early adopters)
2. It needs flash. As much as I dislike flash, I still hit websites that require it - some sites even have videos, half of which are HTML5, the other half of which appear to be flash and won't play. This is an issue for both Apple and Adobe to sort out. Adobe because I still don't think they (although as I write this, Froyo is finally out on some phones potentially) have a good, working flash implementation out on smartphones. Also - where's Linux flash support these days? 64-bit anyone? You can't claim it "just works" when basic platforms have had problems for YEARS. Also, 

 Summary

The iPad is an amazing device. Considering it is not only a first generation device, but also the first of its class - a true tablet with great battery life for browsing, reading, and watching video, I am definitely impressed. It isn't perfect - but iOS 4 should remedy my biggest issue, and it can't come out soon enough. Being able to multi task on my phone but not my iPad just isn't right - it should have launched with it. My hope is that printer support and/or a more sophisticated multitasking are coming. Time will tell...

Browser Benchmarks - July 8, 2010 - Ready. Set. Fight!

Update: More recent benchmark

Yesterday my Opera browser at work auto-updated to 10.60. Opera's auto update is finally coming in line with Firefox...still not quite as nice, but it's getting there. More interesting was the dev blog about 10.60, and I saw elsewhere that Firefox 4.0 beta 1 was out, and also checked up on IE9 to see how things were progressing. Somewhere along the way, I also found the Peacekeeper benchmark and decided I wanted to benchmark some browsers to see what the current state-of-play is. I will be using that for overall (HTML5 / DOM / Javascript) performance, and Sunspider for pure Javascript performance. I'm also throwing ACID3 in there for a sense of where each browser is standards-wise. 

Environment:

System: Intel Core i5 750 @ 3.36GHz, 4GB RAM, 80GB Intel SSD (G1), ATI Radeon HD 4850

OS: Windows 7 64-bit Home Premium

Fresh boot. No other apps or system tray programs running aside from Microsoft Security Essentials.

Each browser was run by itself, with only one tab for the benchmark itself.

First off, I'll start with Sunspider. These are the final numbers, but I have linked the full results for each. I don't have any fancy graphs, so I will order them fastest to slowest.

1. Chrome 5.0.375.99 - 224.0ms +/- 2.0% [Full Result]
2. Opera 10.60 final - 231.4ms +/- 1.5% [Full Result]
3. Safari 5.33.16.0 - 273.4ms +/- 2.3% [Full Result]
4. IE 9 Preview 3 (1.9.7874.6000) -  293.6ms +/- 0.7% [Full Result]
5. Firefox 4.0 Beta 1 - 406.8ms +/- 1.9% [Full Result]
6. Firefox 3.6.6 - 575.6ms +/- 1.1% [Full Result]
7. IE 8 - 3555.8ms +/- 0.6% [Full Result]

Chrome has the lead on pure Javascript performance with its V8 Javascript engine, but Opera is not far behind. It's only trailing by 3.3%, barely more than the margin of error. Next up is Safari, taking 22% longer. IE9 Preview is showing great promise, at 31% behind Chrome. 

Firefox 4 beta is not in the same ballpark currently, at just more than half the speed of Chrome. It is a good increase over Firefox 3.6.6, which would've been almost a 3x difference, but it is a ways off. The big takeaway is that the IE9 team appears to now be ahead of the Firefox 4 team on Javascript performance...times are certainly changing.

IE8 is included as a baseline, and because a lot of users are still on IE.

Next up is Peacekeeper, for this I have screenshots and the full results here.

Opera 10.60 leads here overall. Chrome is close, at 16% lower score. It's worth noting just how far ahead Chrome and Opera are over every other browser in the overall category. Safari 5 is next, at less than half the score. Firefox 4b1 is on the heels of Safari, and then there are the rest.

What's interesting to note here is that even though Safari & Chrome are both webkit based browsers, clearly the Chrome team is going the extra mile on performance.

I want to drill into what makes up these scores, as this benchmark is new to me, but I think that will be in a follow-up post as this post is already almost a week late if you look at the date..

Finally, lets look at ACID 3.

• Chrome 5.0.375.99 - 100/100
• Opera 10.60 - 100/100
• Safari 5  - 100/100
• Firefox 4 Beta 1 -  97/100
• Firefox 3.6.6 - 94/100
• IE 9 Preview 3 -  83/100
• IE8 20/100 (FAIL)

Chrome, Opera, and Safari all receive full marks. I did not compare pixel to pixel, but they have had good track records with ACID tests. Firefox 4 Beta 1 is getting close to passing, slightly better than Firefox 3.6.6. IE9 has come a long way from IE8 but still has a ways to go in order to pass ACID 3.

Regardless of your preferred browser, there is some intense competition now in Javascript engines and general rendering performance, resulting in the experience improving for everyone. It is an exciting time, and I don't think you can go wrong with Chrome, Firefox, Opera, or Safari. Rendering and performance are more than good enough in all of them, so it comes down to other usability / features. Once IE9 is closer to final, I may even be able to recommend that.

Update: More Recent Benchmark 

My Nana Broke My iPhone Calendar

The other day I was following Google's instructions on how to sync my Google Calendar with my iPhone. I've been a big fan of Rainlendar for a long time, and it just so happened that every important Birthday is in my Rainlendar calendar. First, I setup the syncing. Piece of cake - I chose to go the CalDAV route, rather than Google Sync.

Great - events are showing up on my phone. Ok - let's import my Rainlendar birthdays iCal (.ICS) file to Google. Also easy - now everyone's birthday is in my google calendar, and should now make their way to my iPhone.

There's just one problem. Now every day on my calendar has a dot, even when there are no events. I check my Google Calendar - it looks fine...so I do some searching, and what do I find out? 

My Nana Broke My iPhone Calendar.

You see, there's a bug in the iPhone calendar. It cannot handle with recurring events that start prior to December, 1930. Nana, I've made you 3 years younger to make my iPhone happy - I hope you don't mind.

Eclipse and Mockito

Mockito is great. A big part of it is the syntactic sugar, which relies on using static imports. Eclipse, by default, removes static imports when you use 'Organize Imports' (Ctrl+Shift+O). This saddened me, so I found a way to fix it. Thanks go to this post on static import content assist and the post it links on organize imports. I will repeat both instructions in here in one place.

Part 1: Tell Eclipse Not To Remove My Precious Static Imports

1. Open Preferences (Window-> Preferences)
2. Browse to Java -> Code Style -> Organize Imports
3. Change 'Number of static imports needed for .* to '1' 
4. Click Ok 

Part 2: Get Eclipse to Create the Mockito Static Import for Me

1. Open Preferences (Window -> Preferences)
2. Browse to Java -> Editor -> Content Assist -> Favorites
3. Click 'New Type...'
4. Enter org.mockito.Mockito
5. Click Ok twice. 

Now you can write unit tests with Mockito in Eclipse without issues. You can now type mock and ctrl+space and Eclipse will add the static import for you. It will also leave it alone when you Organize Imports going forward. No more trying to remember not to Ctrl+Shift+O when editing a mockito test.

Cool concurrency feed and some links

Google Reader recommended a new feed to me today: Concurrency. I have to say, that was pretty smart of it. Sure, that word appears a lot in some of the feeds I frequent - it's still nice that it brought it to my attention, as I otherwise was missing out on it.

Right off the bat there is a sweet post: Concurrency in Maven. This then lead me to see there are other cool posts at In Code We Trust Inc, like this one on concurrent junit test execution.

Google Reader has made my day with this find, and I thought people might find the links interesting.

UI Prototyping Tools

I have been a fan of paper prototyping for many years. Paper prototypes are easy to create,  easy to edit,  and easy to scan and distribute the same as if it was created with some software UI designer. They also put your mind in a different mode - writing with a pen or pencil and paper is a lot different than sitting in front of the computer using a mouse.

There's just one issue - my office doesn't have a scanner. Now, that shouldn't be too hard to remedy, but along with my list of benefits to paper prototyping, there are also a couple of drawbacks.

• When there are many states to a given UI screen, it can result in a lot of repetitive drawing
• It can take a long time to sketch detailed layouts of more complicated UI components

This has made me once again curious about UI prototyping tools. I probably last looked at the available options 4 years ago.

To start things off, a coworker discovered Balsamiq Mockups. This looks like a pretty fast and easy to use mockup tool. It's quite possible I could create mockups faster in Balamiq than by hand, and they have a sort of notebook-sketch quality to them - they don't try to look like a real UI, and I like that. It also has JIRA integration (as well as some other issue tracking systems like FogBugz and some wikis), which could be a big win for us if we adopt it.

I'm curious what else is out there. I've read a couple of questions about this on StackOverflow to get some ideas, and WireframeSketcher also looks promising (it's an Eclipse plugin).