Fuse for the Idle Fellow – Short read of block errors when starting a container

The red triangle of death is banished once more,
I may have won this battle,
But I doubt I’ve won the war

I do not for one second pretend to understand the behemoth that is JBoss Fuse. Even having done battle with it for several months I feel I’ve barely scraped the surface of this 600Mb accumulation of open source technologies, or as Red Hat describe it, “lightweight” Enterprise Service Bus.

Fortunately I’m a monkey of two halves. On some projects the anal-retentive specialist gets to play, on others the spur-rattling generalist instead holds sway and resolving symptoms gives me as much satisfaction is nailing a root cause ever did.

And fortunately, for all its mighty heft, JBoss Fuse is open source through-and-through. And that means I stand a half-decent chance of fighting its foibles myself.

Like this baby, for example.

Not Yet Another Provision Exception

As my current project won’t go live for a few more months, the powers that be thought it a jolly wheeze to grab the freshly minted JBoss Fuse 6.2 release and upgrade as early as possible. Apparently tearing my pile of hard-won get-out-of-gaol cards into tiny little pieces is a good idea.

And, sure enough, I found myself without a card to play when confronted with this particular 87-line stack trace:-

Provision Exception:
io.fabric8.common.util.MultiException: Error
  at io.fabric8.agent.download.impl.MavenDownloadManager
   $MavenDownloader.(MavenDownloadManager.java:93)
  …
Caused by: com.google.common.util.concurrent.UncheckedExecutionException:
org.eclipse.jgit.api.errors.JGitInternalException: Short read of block.
  at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
  …
Caused by: org.eclipse.jgit.api.errors.JGitInternalException: 
    Short read of block.
  at org.eclipse.jgit.api.CheckoutCommand.call(CheckoutCommand.java:318)
  at io.fabric8.git.internal.GitHelpers.checkoutBranch(GitHelpers.java:178)
  …
Caused by: java.io.EOFException: Short read of block.
  at org.eclipse.jgit.util.IO.readFully(IO.java:248)
  …

Stack traces are wonderful things and this one is peppered more liberally than most with useful clues. Fabric profiles are maintained in a git repository with each container checking out its own copy to work with. This stack trace is pointing straight at a problem with this container’s cloned copy.

Gitting back up and running

Open source is a wonderful thing too. Since it’s just a plain old git repo that’s the problem, we can use plain old git to investigate.

The cloned git repository for Fabric profiles lurks under each container’s data directory in git/local/fabric. If you’re working with child containers this will be under instances/<container-name>/data within your Fuse installation directory. If you’re working with SSH containers it’ll be buried under the containers folder in the user’s home directory – e.g. /home/jboss/containers/<container-name>/fabric8-karaf-1.2.0.redhat-133/data.

And since this is a plain old git repo, plain old git status seems like a good place to start:-

# git status
fatal: index file smaller than expected

Index file smaller than expected?

# ls -l .git/index
-rw-r--r-- 1 jboss jboss   0 Jul 22 13:28 index

No shit!

So we have a broken git repository and, whilst we’re addressing the symptom rather than the cause, we can use the git command to fix it. With the container shut down, delete the broken index file and reset the repo:-

# rm .git/index
# git reset
# ls -l .git/index
-rw-r--r-- 1 jboss jboss 71040 Jul 22 13:30 index

Restart the container and all is well.