muCommander - the ultimate cross platform utility

Despite NTFS supporting very long path names, Windows frequently has issues with long filepaths. Sometimes you need to bring in another tool for the job.

My favorite tool for dealing with long filepaths is muCommander. It is a Java app that mimics the traditional "Norton Commander" user interface. Java uses the API in Windows that supports longer filepaths, so muCommander has no trouble dealing with extra long file paths. It's not just for Windows, though - it's available on a lot of platforms.

muCommander is much more than a better file explorer, too. It has built in support for loads of protocols:

Virtual filesystem with support for local volumes, FTP, SFTP, SMB, NFS, HTTP, Amazon S3, Hadoop HDFS and Bonjour

You mean I can have my local filesystem on one side, and an SFTP session on the other? Awesome.

It can even open and edit zip files in-place.

Plus - it has a sweet logo. 

 

muCommander is an awesome utility to keep in your toolbox.

Desktop Java Apps - to bundle or not to bundle a JRE?

First - here is my definition of bundling the JRE. Shipping your application with a JRE, such that it is the one used by default from your application startup scripts, regardless of the environment on the target system. The bundled JRE is not registered with the system, so it should not affect other applications on the system.

There are probably some good reasons not to bundle a JRE. Taking a moment to ponder, I can think of at least the following:

• Bloat
• No automatic security updates
• Less portable

There may be more - but I believe the pro's will outweigh them regardless, once I add some additional constraints. Specifically, I am concerned with Java on the Windows desktop. Further, I am talking about larger, high engineering effort commercial software deployed to big companies. Not a small, independent Java app deployed to individual consumers. (although there may still be cases where that simplifies things). At a minimum, I would argue for stating "Use 1.6 Update X or higher" somewhere in a README, even for a small app. I've just seen too many weird issues not to.


On to some reasons for bundling a JRE:

• Control over major Java version (1.4,1.5, 1.6) - this is less of an issue today, since 1.5 has reached its end-of-life and 1.4 is far behind that. Thank goodness for that.
• Control over the exact version (i.e. 1.6 Update 20) - to ensure mandatory functionality works properly
• So the end users don't need to install something additional
• So your program is not influenced by the system environment
• So your program does not influence other applications in the environment 
• Less variables for when customers report bugs
• No automatic updates - which could break functionality
• Less to test

See a theme? Better control. Also, better isolation from the environment and other things that can go wrong.

For instance, in the 1.5 days, I recall one release that had very misbehaved tabbing through components in a UI. In the era of Windows 7 - you need a minimum of 1.6.0 Update 18 - or you have missing icons in Open/Save File Dialogs. It is not acceptable to have a UI defect like that in a shipping app - any less, and you don't have full Windows 7 support. Except on 64-bit systems Update 18 has a bad habit of crashing the JVM a lot...so you need a more recent update. Then there's the Swing/AWT changes in Update 12 that alter behavior. Then I'm pretty sure depending on the version, applets load differently, if that's a factor (in one of our cases, integration with another process - it is).

You can avoid all that by picking a version you know works for your required situations. This means less to QA. Less potential headaches. The trade-off is losing faster security updates, increased application size (which believe it or not - still matters even in the age of 3TB hard drives), and portability. If you are in a Windows-only market, as I have seen several applications be, then portability isn't affected. Even if there are other target platforms - it's just a matter of repackaging for other platforms.

I believe there are some good reasons to bundle a JRE - on Windows, anyway.

Java Roadmap Disappointment

Am I the only one disappointed with the new Java 7 road map? It seems both late and lackluster.

First, let's look back at past Java releases (courtesy of the first google result I saw with actual release dates)

J2SE 1.3 2000-05-08 Kestrel
J2SE 1.4.0 2002-02-13 Merlin
J2SE 1.5.0 (5.0 to salesmen) 2004-09-29 Tiger
J2SE 6.0 (1.6.0) 2006-12-12 Mustang

What has been happening the last 4 years? Apparently a few things I didn't know about (politics, JCP, Apache Harmony, etc). Most of which I'd need to read up on a lot more before I could write about it...so I'll stick to two things. 1) Apple OS X Java thing, 2) The updated Java 7 release timeframe and scope.

On the Apple deprecating the shipping of Java with OS X - I think Ted Neward summed it up best. What is potentially most interesting is depending on how Apple handles phasing out maintaining Java on OS X, and if Oracle picks up writing a JRE for OS X. If so, will it impact the Java 7 release date? I think he has a point though - is there even a large demand for Java on the Mac? (And I say this recalling having compiled landon fuller's openjdk6 port on my Mac prior to Java6 shipping on my MBP almost 3 years ago...)

On the updated timeframe and scope given by Mark Reinhold...

I was really hoping Lamba expressions, often referred to as closures, would make it into Java 7. A mid-2011 release (another delay) without them is pretty disappointing. Java feels stagnant. The primary strength of Java for me is the tooling, not the language...but it would still be nice if the language was developing faster. Maybe it's time I started checking out some other languages that run on the JVM.

There should still be a few neat things though...looking at the list:

• invokeDynamic bytecode - should be great for all the dynamic language people running on the JVM
• related to the above...method handles. First I've heard...this requires more reading. 
• Strings in switch statements - at last
• NIO.2

Ending my rant on a positive note...I suppose there are still some cool new things. I'd install an openjdk7 build, but it would just make waiting harder :)

Practice your TDD

Misko Hevery made a great blog entry Tuesday. He posted a sample project for practicing TDD. We know that Beginners Need Steps, and that's exactly what he provided. Testing is a skill that is not innate - we as developers need somewhere to start.

In his post, he provides a starter project that can be grabbed from a Mercurial repository and used to practice TDD. The basic code is already layed out, including the first few tests. He even describes how to turn on a setting in Eclipse that will run your tests every file save. Slick - I didn't even know that was possible. That is serious TDD.

I consider this a personal testing homework assignment - I encourage anyone reading to do the same.

I will try to remember to post my resulting source & test code when I finish.

Silent JDK Deployment on Windows

It's pretty easy to find documentation on how to silent install JREs, but for some reason it's hard to find a full example for silently deploying recent JDKs. Today I needed to deploy 20 JDKs or so, so I decided to tackle this. I needed to deploy JDK 1.6.0_17 (both 32-bit and 64-bit varieties). Since they're build machines, I don't really want the public JRE installed, which may have auto update notifications.

Doesn't Hudson know how to automatically install JDKs now, you ask? Why yes, it does, but it doesn't always have the latest JDK updates. Apparently update 18 fixes some obscure issue that one of our customers has, and according to Ninite it is already out, but I can't yet find it on java.sun.com.(Apparently snapshot releases can be found here.)

After deducing the JDKs use an MSI installer (although packaged within an exe), I enabled verbose logging and found out the names of the feature groups:

• PublicjreFeature
• JavaDBFeature
  
• DemosFeature
• SourceFeature
• ToolsFeature

These are the important part, as they are what you specify using ADDLOCAL=list,of,features to determine which features are selected during silent install. To find any other general install options, all you need to do is run "msiexec" at a command prompt and it will pop up a dialog of options. Thanks to AppDeploy's page on the JDK I also found out some other info (like where the JDK .msi gets extracted to - you can use Orca to examine the MSI file if you're really curious).

On to the point, to perform a silent installation (in this case jdk 1.6.0_17), simply do:

jdk-6u17-windows-i586.exe /qn ADDLOCAL=ToolsFeature,SourceFeature,JavaDBFeature INSTALLDIR=c:\jdk1.6.0_17 REBOOT=ReallySuppress

• /qn means "No UI"
• INSTALLDIR is self explanatory
• REBOOT has a few options, I chose ReallySuppress
• ADDLOCAL - Specify a comma-separated list of features you want from the list shown earlier, or you can use ADDALL
  

I saw a lot of examples listing: /s /v"/qn ADDLOCAL=ALL ... ...", this seems to spring from the Java 5 JRE silent deployment guide, but it really need not be that specific. Maybe /s and /v provide some value if an error occurs, but the  /v"/qn OPTIONS..." seems a bit silly, and I am surprised it even works that way.

At last, I have my auto install script that I can remotely run on a bunch of machines with PsExec.

A Quick Look At Gradle

I'd say up until a month or so ago, I knew about Gradle but I had no real idea what it offered. I knew it was more elaborate than Gant, but that was all.

It's about time I took a closer look at Gradle. The most obvious cool feature is that you can write your build scripts using Groovy. This is sweet. Beyond that, it is probably easiest to list Gradle's own summary, as of this writing:

• A very flexible general purpose build tool like Ant.
• Switchable, build-by-convention frameworks a la Maven, for Java, Groovy and Scala projects. But we never lock you in!
• Groovy build scripts.
• Powerful support for multi-project builds.
• Powerful dependency management (based on Apache Ivy).
• Full support for your existing Maven or Ivy repository infrastructure.
• Support for transitive dependency management without the need for remote repositories and pom.xml or ivy.xml files (optional).
• Ant tasks and builds as first class citizens.

So... dependency management? check. multi project builds? check. Easy to write general purpose scripts? check.

What I think has extremely high potential is the idea of pluggable build frameworks, a la maven support. Use all of the best pieces of tools from one - Gradle. I think they could go a really long way with this, and it will remain a strong contender as I also take a closer look at some of the new Maven 3 features.

Downloading and extracting gradle-0.8-all.zip, one thing is immediately obvious. Comprehensive documentation and examples FOR THE WIN!

I can't even describe how frustrating playing with maven 2 was three years ago. Documentation was hard to understand, and the basic example tutorials just didn't cover my needs. I ended up banging my head a lot, when all I really needed was some good sample poms illustrating all of the tool's features. Gradle provides this out of the box and scores major points for it. I haven't even looked at the website docs in detail yet, but they seem easy to follow.

Some Maven 3.0 Info

I do intend to investigate Gradle, but I can't shake the feeling that Maven 3 will really rock. On that note, about all I have time for tonight is to pull together a few links. Nothing a couple minutes of searching couldn't accomplish, but hopefully it is of value.

Here's a presentation that happened all the way back in April. I'm sure there are more recent ones, so I'll try to update with additional links. There are (not surprisingly) quite a lot of good posts on the Sonatype blog. In particular, there is a series of posts titled "Paving the desire lines". It's very satisfying reading a developer talk about the small finishing touches to a project with real passion. I haven't read the whole post yet (and I spot 2 parts so far), but it has my appetite whet.

Maven 3.0 alphas have been coming out recently, with Alpha 3 coming out last Friday. I just grabbed it tonight (seems to be alpha 4 now) and took it for a quick spin. To test it, I built Hudson/main, which has 4.0.0 POMs.

This merits close following...

UPDATE: Walter linked a more recent video+slide presentation by Jason van Zyl

Java Build Tools

There are a lot of build tools for Java. More than I have seen for any other language I've used. When working with Java, I've used Ant, Maven 1, and Maven 2. I know of at least a few more, and I recently found this post with a link to a poll on what build tools people are using this year (for Java).

If you navigate to the 'What Build Tool are you using in 2009?' poll (I can't post a direct link, sorry, but there are only 3 polls to choose from), you will see quite a selection of tools.

Unfortunately, right now I'm that single Maven 1.x user (along with some Ant). It's tough - things work well enough, but there is no direct path to Maven 2.x for us (oh maven.xml, you looked so helpful, but you've trapped us!). In its defense, this project was using Maven 1 back in 2004. It's not surprising that in 5 years some cruft has accumulated.

I am fed up with having to write build scripts in XML, so even if a Maven 2 migration path was simple, I'm not sure it's desired. Gant would get rid of the XML while keeping familiar territory - but I feel like I need something richer than basic Ant. Gradle looks very promising, and I've been meaning to read up on some of the threads on where Gant and Gradle are going to get a better idea of things, but I haven't yet.

Lo and Behold, while pulling up links for this blog post... I stumbled upon a post about Maven 3.

Support for several build script file formats? YAML, Groovy? Hell yeah! I am going to have to do my research on this one.

I wasn't quite sure where I was going with this post, but I think I'll sample some of the build tools I've mentioned (including early maven 3 builds) and report in on them. If I'm motivated, maybe I will do a 'Hello World' project in each build tool. Gant, Gradle, and Maven 3.

Szczepan Faber is my bartender

I've known about Mockito for a little while, but I've only just now officially tried it today, and Sczepan Faber (and friends) are now my bartenders, and when it comes to mocking, a nice cold mockito is my new drink of choice.

Wow. Seriously, wow.

The syntax is exceptionally sweet, and additionally quite clever. A coworker and I dived into how it figures out what method to stub today and that alone was enjoyable.

Mockito will definitely be a valuable tool to have in the testing toolbox.

Windows file path FAIL

I'm going to take a break on testability to talk about file paths - specifically long file paths, and the care one must take when dealing with them on Windows.

Now, while quite a lot of technologies are involved in development, for me (right now), the most important to mention when dealing with files are: Java, Subversion, Windows, and Tortoise SVN. I'll throw in one more - the filesystem (NTFS).

Using the command line is part of my nature, so when I get into the office in the morning one of the first things I do is open up a command prompt and 'svn update' my working copy.

Imagine my surprise one day when I couldn't update, because Subversion claimed one of its .svn/... files was corrupt or missing. Cleanup didn't help - it wasn't actually a Subversion problem. It was a file path problem. The svn FAQ actually has great information on this.

Enter the windows API maxium file path limitation of 260 characters. What's annoying is that it isn't ALWAYS the limit, in fact the unicode version of the API has a much more flexible limit (thousands of characters). Why then, does Windows Explorer, even in Windows 7, still use the API that has the 260 character limit? *

I deal with this issue again and again. I generally don't create file paths this long, but it doesn't matter - because I work as part of a team. I think it's reasonable for people to have descriptive, rich naming for their stuff - I just wish it wasn't so painful. Once files are in version control, then build tools, installers, and all kinds of things MIGHT have an issue depending on where the files are checked out and what file API they use. It's just not an issue I should have to deal with.

I don't want to worry about long file paths - this is a simple request. Of course, simple problems for computers aren't always simple. In fact, they usually aren't, but the fact that the API exists but that it isn't already the defacto, default API used by Windows itself frustrates me.

I want to be clear - this isn't a limitation of NTFS, Java, or Subversion. They are all quite happy to deal with long filepaths. It's the Windows file API. It's just easier than normal to get into bad situations when you have tools that will very happily (as they should!) create and deal with long filepaths, only to have your operating system tell you the file doesn't exist or can't be accessed.

* Probably backwards compatibility

That's my rant for the day.

More on Testable Code, and the Law of Demeter

I started looking at a simple enhancement task today. Like many simple sounding tasks, when you peer at the 
system and look at what is actually involved, there may be ugly parts. Especially in ensuring testability, as I am committed to unit testing this code.

In my quest to test more (and test FIRST), I looked at what would be required to add the functionality, and specifically, how to test what I was adding.

Here are the issues with testability and excuses I came up with:

• No unit test exists for the immediate classes in question (shame on me - it's my code) 
• The code has a GUI - this usually makes testing harder
  
• Too many external dependencies!
  
• I haven't tested this exact type of code before (aka: I don't know how)
  

Tonight I'm going to focus on the dependency issue.

Taking a look at my model, it already relied on more external objects than it needed to for the required functionality. The requested enhancement would (if the pattern continued) require further addition of some heavy weight dependencies. Bye bye testability!

Fortunately, I had both the testability presentations still in my mind, and a coworker who had already solved this problem.

The feature in question is for administering users in a system. Among the things being manipulated are name, manager, credential reset, licensed features, and now - a user's preferred currency code (at the end of the day - a String). Great - slap in a few getters and setters, and we're done! Well not quite - currency code lives on a currency unit which is supplied by a unit system, well, which comes from this other big thing. You get the point. If I was lazy, I'd just pass this big other thing to my model so it could do its job. I'm not going to be lazy - I'm going to pay attention to the Law of Demeter.

This model don't really care about the 'other big thing', or the 'unit system', or anything other than that being able to set a user's preferred currency to Australian Dollars, therefore they have no place in this particular model.

Instead of that, my model will take a data source that can give it valid currency codes and set a user's currency code. Just a simple interface with a couple of methods that will be easily mockable for testing. While I'm at it, I'm going to pull out the existing extra dependencies (there was fortunately just one) that weren't needed. Now my interface has four methods, and I don't need any hulking database-requiring objects to get my model moving. A little shuffling here and there, and very little additional code, and my model is now much more easily tested. Suddenly the "I don't know how to test this" feeling has weakened a bit now that my class is simpler.

I want to write further on my take on the Law of Demeter, but it's 11:45PM and I need to submit an entry daily so I'll have to wrap this up tomorrow.

Afterwards, I plan to cover making testing the GUI easier.

Java path mystery...solved

For ages, I've wondered how in most cases, java will mysteriously be in your path in windows, even though there is no Java install directory in the path. (it will certainly be a version that was installed with a proper installer, at some prior point, but not necessarily the most recent version you've installed) To fix this, I generally put %JAVA_HOME%\bin; at the front of my PATH environment variable after defining JAVA_HOME to point to the Java install I care about (something Ant, Maven, and other dev tools often care about having defined...).

I've always wished I had the 'which' command in Windows, so I could locate which Java executable was in the path. Finally, by chance, a coworker was debugging an OpenGL crash with an Intel GMA graphics card and substituting DLLs in what he thought was the 'run' directory, and he realized that it wasn't picking it up when he put it in the directory of the JDK bin he thought it was using... When he checked process explorer, lo and behold... C:\WINDOWS\SYSTEM32\java.exe is what was running. What the heck??? I find this strange, yet there it is.

I have always thought it was registry magic, or some other trick that caused Java to end up in the system path unless you put it as the FIRST path entry. Instead, FIRST just means AHEAD of the System32 folder. (Several apps likely to be in a dev environment often put themselves at the front of the path, subversion, oracle, etc...so I never noticed that the usual first entry in the PATH is this...and I typically put entries at the END of the path except when I know there will be other entries fighting for this position)

It's funny how sometimes mysteries like this go on for many years - maybe it's a simple Eclipse feature nobody knows about, or some maven quirk, that we never take the time to fully investigate and understand...we just come up with the rule and follow it - at least it's what I've observed.

If anyone can shed some light on this, I'd love to hear it. Is this practice generally acceptable to pollute the SYSTEM32 folder with executables? For now, I am satisfied to now know why I need to put my java/bin at the front of my path...

SSDs for developers - my experience so far

I'm sure a lot of people have read the Anandtech articles on Solid State Drives by now, and maybe even Joel Spolsky's story (the combination of the two is what convinced me, after all). Just how big of an impact will an SSD have on your day to day development performance? I can't answer the general case, but I can at least share my personal experience.

I've been using an 80GB Intel x25-m SSD for the last 4 months, and everything about my workflow is faster. The main tools I currently use daily: Eclipse, Maven, and Subversion, on Windows XP 64-bit.

building our full app with Maven (javac, lots of file copies, jar creation, and other related actions - a lot more than just compile):

500GB x 2 (RAID 1): 10 min 45 secs

80 GB SSD (after filling it up): 3 min 25 secs

full build w/o clean:

500GB x 2 (RAID 1): 8 min 32 secs

80 GB SSD (after filling it up): 1 min 42 secs

We're talking 3-4x faster, after the drive is full and at the reduced performance level it will remain at. If TRIM were present, this would be an even wider lead. Initially a full clean build took just under 2 minutes, as compared to 3 min 25 secs now.

Aside from rebooting to ensure nothing was cached and disabling antivirus scan, I took no special effort to make this a 'real' benchmark. It is only what I've observed, but is in no way scientific.

For Subversion I do not have any way to measure time, suffice to say - if you are familiar with opening a commit window in TortoiseSVN only to wait minutes while it calculates differences followed by finally listing your modified files, that is mostly a thing of a past with an SSD. 10-20 seconds is the usual max wait. For reference, this is on a 3GB workspace with 30,000 files. Note that the .svn directories double amt of files and size due to storing a 'pristine copy' of each file. The workspace, if exported, has 15,000 files.

Similarly with Eclipse, if refresh/clean/autocompile seemed to take a really long time - it is also a night and day difference on an SSD. Rarely am I waiting for Eclipse to finish now.

Having an SSD eliminates a lot of time I'd otherwise have to waste waiting for something to finish. I think they are a worthy investment, and they are getting cheaper every month. I bought mine only 4 months ago, and the price has dropped from $400 to $229 for an 80GB Intel SSD. There are also a lot of competitive drives in similar price ranges now.

Here are a couple of things to check when looking into an SSD:

Make sure your motherboard's SATA controller is modern enough and supports AHCI mode. The original, older machine I tried it in (it was a 3 1/2 to 4 yrs old Athlon X2) could not see the real benefits of the SSD. I think any Core 2 Duo/Quad or more recent motherboard will be fine.

Buy a drive with a firmware that supports TRIM, or from a vendor that has released a manual TRIM utility. I would not trust any vendor that promises to deliver TRIM firmware or a utility after-the-fact. I had the impression TRIM was coming with my drive, and it looks like I'm out of luck with my first gen Intel drive. Who knows what the timeline is for TRIM on the second gen Intel drives. I believe all recent drives w/ an Indillix controller are fine, and I believe OCZ is even working on a background-TRIM firmware that works in XP. (It will be some time before my work environment is Windows 7...). Also, if you are a non-windows user, pay special attention to the TRIM firmwares. I believe a lot of them may only support TRIM on NTFS.