/etc/init.d/hack start

Jersey and Maven Share (i.e. Über Jars)

2011-03-04T20:53:00.000-08:00

For the past year or so, much of the work I've done with Java has used Maven's Shade plugin. This plugin does a really excellent job of mashing dozens to hundreds of Java dependencies into a single jar file. This not only makes deployment easier, but for things like Hadoop jobs makes dependency management in general much easier.

Unfortunately, using the Shade plugin does occasionally have some negative side effects. Namely, on some occasions I've had jar files stomp on each with non-obvious outcomes. Today was one of those times.

The issue in this case only appeared in the production environment but not in test and not when running from my IDE under a debugger. The core service would start fine, but on any request, the code streaming data would throw an exception similar to:

javax.ws.rs.WebApplicationException
at com.sun.jersey.spi.container.ContainerResponse.write(ContainerResponse.java:240)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:593)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:514)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:505)

This particular exception was particularly difficult to diagnose because Jersey writes a critical piece of data to stdout and not to the conventional logging infrastructure:

SEVERE: A message body writer for Java type, class java.lang.String, and MIME media type, application/octet-stream, was not found

When I finally realized this was the case, a Google search turned up this discussion where the difficult diagnostics had already been published by Gaëtan Sheridan . From there, it was a matter of excluding specific portions of the jersey-server plugin from the shade-generated artifact and all was well.

The relevant portion of the POM file looks like so:


    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-shade-plugin</artifactId>
      <version>1.4</version>
      <executions>
        <execution>
          <phase>package</phase>
          <goals>
            <goal>shade</goal>
          </goals>
          <configuration>
            <filters>
              <filter>
                <artifact>com.sun.jersey:jersey-server</artifact>
                <excludes>
                  <exclude>META-INF/services/javax.ws.rs.ext.MessageBodyWriter</exclude>
                </excludes>
              </filter>
            </filters>
          </configuration>
        </execution>
      </executions>
    </plugin>

Making that change fixed the issue.

The other painful reminder from this exercise was that common tools such as mvn jetty:run and launching from the IDE are no substitute for testing with the exact binaries and classpath used in production. Because my IDE and maven's jerry:run command manually assembled the classpath, it turns out that they did not experience the issue. Only when running directly from the über jar did the problem surface.

Jive Product Launch Videos

2009-03-10T23:02:00.001-07:00

Our product marketing team undertook a huge new push this week for Jive's 3.0 product launch, new web site included.

Most fun though are the YouTube videos the team and other Jivers created:

http://www.youtube.com/watch?v=0Y6aceiLzCI

http://www.youtube.com/watch?v=RThcPOmVC9o

Next on My Reading List - Against Intellectual Monopoly

2009-03-10T22:37:00.000-07:00

http://www.newswise.com/articles/view/549822/?sc=dwhn

What intrigues me most is that its an economist-driven view. Not the normal FOSS evangelist vs. IP lawyer-type rant, but a different perspective, hopefully with some quantitative backing behind their assertions. Always hard to argue with numbers.

Portland's clear.com Not Up to the Task

2009-01-13T22:48:00.000-08:00

I was excited to see WiMax coming to Portland via clear.com but that excitement has ended now that I've learned more and that the service has been rolled-out:

My main interest was for mobile access. I love the 3G on my iPhone, but without a way to tether to my laptop, I'm still forced into situations where I want to get online but cannot or have to pay yet another provider for service. Unfortunately, clear.com does not support Macs for their USB devices which I find stunning considering I see more Macs in downtown PDX than practically anywhere else I've traveled in the last few years. I'm feeling like they didn't research that one at all.
Support reps for clear.com's online chat is uneducated on their actual products. I asked if they support Mac and if there were Linux drivers for the USB dongle and the support resource happily told me "yes" to both. In talking with two different reps at clear.com kiosks around town, both confirmed only Windows is currently supported (again, in a town where Windows probably has the least market share of any urban area in the US for personal devices).
I suspect that clear.com is blowing up my satellite radio signal. Others in PDX seem to suspect the same. All I can say for sure is that since clear.com's service came on line, my satellite radio equipment that was previously working just fine stopped holding a signal. The same equipment did not move, just mysteriously stopped functioning with a similar experience in my car's unit.
For home service, the speed isn't really that impressive. Its slower than Qwest's high-end DSL at roughly the same price point, and definitely slower than Verizon's FIOS if you're lucky enough to have access to it - I for one can only dream.

In general, beware Portland WiMax and specifically clear.com.

Prog-iona

2008-10-03T00:45:00.000-07:00

Some how the Progress acquisition of IONA slipped under my radar. Funny, because that brings many things on my periphery full circle.

I used to work for C-bridge Internet Solutions. eXcelon bought C-bridge after the bubble burst in 2002. Fast forward to 2005 and I'm in Portland working for a now-defunct company trying to break into the ESB space via an open model. That space leaned towards JBI. I'd still argue that we bet correct and that JBI has done a whole lot of not much, offering "vendor independent" layers on top of things that people didn't care about vendor independence for, but in 2005 we gave it a serious look.

LogicBlaze used to be one of the primary FOSS backers of JBI and from the looks of the mailing list, their core team continues to be heavily involved in the development of ActiveMQ on which their JBI implementation was based.

By April 2007, LogicBlaze was acquired by IONA. Around that time, Mike started talking with Larry who was our boss in the latter part of the eXcelon days. Mike and I were both at LM, and we regularly worked with Stephen who doesn't blog (sorry no link) but formerly worked for IONA. Mike's talks with Larry didn't go anywhere, but the IONA/C-Bridge/eXcelon/Progress/LM circle was complete.

Now, last month, Progress buys IONA and Larry again works for Progress. I'm also again doing a deep dive into ActiveMQ, whose IP is now I guess owned by Progress who I used to work for. So in some sense, the company who owned my butt as far back as 1999 is still influencing my role today.

Add to that mix a couple of interactions via TSS, OSCON and IRC with Hiram and James (which they probably don't recall but did result in some attribution), and things get even more tangled.

It never fails to amaze and amuse me how small this business is, despite being so big. The networks I started making 10 years ago still come into play even today in subtle ways and keep overlapping. Six degrees of separation is nothing, I have trouble keeping it to two.

Classes of iPhone Users

2008-08-08T23:20:00.001-07:00

I've had an iPhone for some time now. While it does bother me that Apple doesn't really give two shakes about contributing back to the open source community on whose shoulders they have built, I'm also pragmatic. The device is simply the best of its kind out there.

I like to consider myself a student of other iPhone users. Any time I see someone with one, "classic" or 3G, I like to snoop on what they are doing. Over the last few months, I've noticed a few classes of iPhone user emerging. What follows is my untrained attempt at social anthropology mixed with humor.

The Frustrated Designer - a Mac user through and through. They always buy 1st gen Apple hardware, even though rational people know better. Most easily identified by their propensity to verbally express a tangible frustration attempting to use 97 nested DIVs, advanced CSS selectors and 12578 lines of JavaScript to center an image in the iPhone Safari interface. Would write an ObjectiveC native app, but doesn't know what a compiler or socket or ObjectiveC or a native apps is.
The Trophy - doesn't really know why they have an iPhone, just that their wealthy partner gave them one. Most easily identified by well-manicured nails attempting to tap out a text message with the sound at full volume for each key press. Never types faster than twelve keys per minute. Commonly sits next to me on airplanes.
The Showman - Doesn't really know why he needs an iPhone, just that he does. Identify the showman by his inability to do much other than talk on the iPhone, set it on the table during meetings so you know it's there, and by the fact that his emails still end with "Sent from my iPhone" despite having the device for nearly a year.
The Tech Luminary - Installs every possible application from the iTunes store hoping to gain status as someone qualified to comment on the state of iTunes Store/iPhone technology while at the same time not knowing a thing about the underlying platform. The luminary mocks Windows users for arbitrarily installing shareware from .ru domains, but never questions why he has installed 47 iPhone applications, 43 of which are never used.
The Frustrated Developer - Has an iPhone application in the works, but can't talk about it due to an unenforceable NDA. Identify these users by their regular and sanctimonious references to their NDA despite never having really finished an application or by their smug looks at local user groups.
The Marketing Director - Only knows that they should have an iPhone because their underlings said so. Loves the speaker phone, thinks it's the major advance of the device. Most easily identified by the fact they are still running the same firmware they bought the device with.
The Commuter - Found in urban areas, this individual cannot stop texting, browsing, or listening to podcasts long enough to exhibit any awareness of those around him/her and actually move out of the path of egress for everyone else on the transit device. Likely to accidentally enter altercation with local meth addict while engrossed in something Twitter-related.
The Power User - The most elusive of the iPhone users, understands device limitations (most Bluetooth profiles lacking, no Copy/Paste, no Search, AT&T partnership) along with a healthy distrust of anything Steve Jobs claims. Generally refuses to tout the virtues of the device unless overhearing someone rant about how much better a Nokia or RIM device is. Understands there are no better alternatives and that using the device sticks it to "the man" by making consumer opinion matter again. Hopes Android kicks major ass and that people will take Openmoko seriously.

It's probably obvious that I associate myself mostly with the latter category, but there is plenty of commuter in me as well.

There is a snobbery around having an iPhone. It's sort of like riding a fixed-gear bike in Portland; you feel like it makes you cool, you usually go out of your way to let people know you have one. Then you see all of the other DBs using one and realize its way too much machine for them. Some of them even run into walls while using the device you admire. You can't give up your baby, so you're left trying to quietly blend back into the masses hoping nobody noticed and that you don't get associated with the new user population. I'm sure there's a Gartner graph to prove it all out somewhere...

I Can Ride a Bike With No Handlebars

2008-04-16T22:01:00.000-07:00

This post on slashdot blew me away. I'm typing this on a Hardy/Kubunty alpha (maybe beta by now?) installation and it has been more difficult to install than several of my Gentoo installations - that's saying something.

Granted, I went with KDE4 which is simply asking for problems, but really, this was a doozie. Issues of note:

Now famed Ubuntu "Black Screen of Death" - I attribute this to my having some slamming video cards, but seriously, the people's distro should *never* get itself into a place where you can't fall back to the kernel fb. When good ole' Ctl+Shift+Del can't save you from X+driver maddness, something is fundamentally wrong. For the interested, I had to drop to a recovery console and install nvidia-glx-new, then hack xorg.conf to use nv drivers, then "activate" properietary nvidia drviers. Not pleasant.
Java UIs fail - mouse-over for certain Eclipse SWT widgets cores java. That is generally difficult to do - SWT is one of the more stable Java UIs out there and it dies *hard*.
Probably most important, my shell gets into some wierd sound death-trap. Yes, that's right, I said shell and sound are problems. Alarm bells going off for anyone yet? Basically, the DCOP issues on the Ubuntu forums don't seem to be fixed for me anyway, my shell is trying to do something sound related (even with sould off) and hanging until I start a corresponding shell to recover the first. The scenario looks something like - start System -> Konsole -- see DCOP error message, see Konsole hang. Start System -> Konsole 3 -- watch Konsole work while Konsole 3 hangs. The two seem to be fighting for control of the KDM and one but not both work at a time. I spend ~40% of my compute time in a shell. When something that fundamental doesn't work, I freak out.
FF3 widgets look like crap. That's not really Hardy or KDE4's fault, but it's true. Tabs, drop-down decorations both look simply aweful.

In all fariness, Gusty didn't install properly on this hardware. Still, I'm not seeing how things in XORG land have gotten any better, infact they may have gotten worse. Complain all you want about Windows, at least Vista on the same hardware displayed a UI without me dropping to a rescue console.

Stale Cake - With Frosting!

2008-03-16T23:28:00.001-07:00

Funny comments by Microsoft product manager Lawrence Liu on Mike Gotta's blog. Removing some noise between the lines:

"I am asking customers to step back and assess what business problem(s) they’re trying to solve... Jive and IBM are trying to wedge ... themselves into the SharePoint 'pie' by focusing on feature-to-feature comparisons while we’re working hard with our partners to provide the right frosting (or Cool Whip) to solve our customers’ problems."

I'm confused by Lawrence's response here for a couple of reasons:

Bad metaphor is always difficult to read - Who puts frosting on pie? Who takes a "wedge" of pie? Good pie is best served hot by a knowledgeable baker, not after the original baker has stepped away from the pie and waited for six months to see what other pie makers are selling, then asking some other pie maker to fix the six-month-old, stale pie.
Certainly nobody is asking customers to *not* solve a business problem. That would be equally as insulting - Microsoft does not have a monopoly on fixing business problems. If anything, they have a monopoly for causing business problems (early EOLs for XP, patch practices requiring substantial outages and reboots, Vista protocol design that causes networking headaches all translate into BCP issues in my mind).
More comical, Lawrence is trying to convince customers that alternatives to *their* products are rash and that buyers are *safer* sticking with the platform and ignoring what other vendors are doing in the space. In my mind, this is akin to "only following orders", but why take orders from the used car salesman at the end of the block? Sure, eventually Microsoft will figure out what is important for their most profitable customers, or alternatively customers can buy a number of flexible solutions today that meet their needs.

My translation: "We're Microsoft. You will do what we say is important, when we say it is important. Thank you, your next invoice is in the mail."

The term "Microserf" comes to mind...

Erlang Musings

2008-03-13T23:46:00.000-07:00

Completely bogged down buy our 2.0 release push at work, generally not a lot of time to blog these days. That combined with the Twitter micro-blog phenomena and I'm neglecting this blog. Sorry for the two people who read this :)

I did want to capture some thoughts on the recent discussion on the Erlang mailing list around Damien Katz's post about Erlang.

To preface my comments, I've been using a decent bit of Erlang over the past few weeks and have been learning the language for several months now. My current endeavors are in the distributed load-testing arena (Tsung didn't meet my needs). I'm by no means an expert and have nowhere near the experience of Damien or the members of the Erlang mailing list, but I'm beginning to feel like I'm not a total n00b.

Initial thoughts on Damien's post:

I've also found refactoring a bit difficult. Over time, defining an entirely different function to elicit a slight behavior change generally results in some parameter munging then a call to an existing function. Problem is, in development, the behavior of that existing function often changes thereby affecting all the calling functions.
I was also intrigued by the response to Damien's post on the Erlang mailing list where I lurk but rarely post. Some highlights:

"sometimes giving a whining child a lollipop is a rational thing to do." - Richard O'Keefe clarifies this comment in subsequent posts, but the condescending attitude towards other languages continues to discredit him IMO. I generally get this sense from the Erlang community - "we know functional and you don't, so don't bother us with your critique of our superior language." Not sure why early adopters of most languages feel a need to succumb to Napoleonic urges and lash out at successful platforms. Java bashing is so very original these days...

Robert Raschke also jumps on the Java-bashing bandwagon but with less valuable critique other than that he doesn't want to write Java in Erlang - not sure who asked him to do so.

As usual, Ulf comes through with solid, valuable feedback.

My experience has been similar to NAR's and I related to his comments for the life of the thread. In particular, defining dozens of one-line functions has made flow control difficult to follow in some circumstances. Probably my lack of understanding towards the language idioms but building on Damien's points, there aren't consistent examples of those idioms so it's easy to lose your way. I stand by my claim that Wide Finder proved this for file I/O in Erlang. It shouldn't take a committee to help you write good file handling code.

I found Alpár Jüttner's comment interesting: "I think, the major obstacle for newcomers is not the syntax, but the immutable data structures". As a newcomer, this was not an issue for me at all. In fact, having spent more time than I care to remember worrying about dirty reads of shared memory, this is a welcome addition to my programming routine. Rather, I struggled with multi-byte strings (referring people to ejabberd is not sufficient documentation for an entire language), function proliferation, syntax ( =:= ), and most importantly lack of libraries.

Robert Raschke's clarifications as line endings was a huge help here.

The fact that POST'ing a form with http:request remains an enigma (eh, where are my param encoding functions?) really makes me feel like the language is a DSL for networking gear. Yes, Erlang is more expressive and concise than Java, but that doesn't mean much to me when I can write a Scala function to make an HTTP POST in four lines of code and have it run screamingly fast where as the same Erlang takes two dozen lines (and based on the documentation is questionably not process safe). I have the sense that the Erlang community would happily provide me with 10 different functions to solve the parameter encoding issue, but that misses the point - I don't have to worry about such things in most other languages I work with.

In summary, I actually agree with most of Damien's original post. Having written a good bit of C/C++/Java/Python/Ruby, his observations are generally representative of how I feel approaching Erlang from that perspective.

Like Damien, I have a lot of issues with Erlang. I kind of hate Erlang. I hate it in the sense that I wish I had a better, more familiar solution to programming with actors. At the same time, it's immensely powerful and expressive. Every time I use it, I think "wow, I love functional programming" and "actors are so important, it will be amazing when the programming public realizes why we all need to do this".

At the same time, the syntactical and idiomatic issues I have with Erlang constantly nag me. I'm sure these will ease over time, but I have yet to reach a point where I've stopped looking/hoping for alternatives. Given some of the attitude displayed on the mailing list, I won't really have a lot of community loyalty if given a comparable option.

S3 Outage

2008-02-18T23:46:00.000-08:00

A bit surprised this didn't make more news, but I find the S3 outage (originally via Tim Bray) fascinating. From what I can tell reading between the lines, it seems as though a handful of demanding clients ground the service down with bad authentication requests. Makes sense on some level, auth requests are computationally expensive and in this case, they came from within EC2. My guess is that Amazon treats EC2 as a more privileged network and ultimately allows a higher QoS level between EC2 nodes and S3 nodes. So, while it may be en vouge to diss "architects", it's also important to have people around who understand the fundamentals from TCP stacks through the crypto and application layers and who build reliable systems across all three.

My prediction is that we'll see more of these incidents from EC2-hosted nodes, then growing out to bad S3 requests across the public network launched from various bot nets. It's not an easy problem Amazon is undertaking with EC2 or S3 to begin with, it's an even more difficult one to protect from DDoS and DoS attacks.

The Power of Simple, Online Collaboration

2008-02-13T23:34:00.000-08:00

I won't attempt to use any fancy market-speak or terminology. I've long been a believer in how blogs and wikis can drive collaboration inside a company. One of the things Jive is really good at is capturing ROI around open collaboration in quantitative terms and telling a compelling story. Sam's latest blog is a great example. In short, based on a self-organizing set of categorizations, we can track cross-departmental collaboration and interactions. Cool stuff, not the type of data you can see from Share Point, that's for sure.

Rhino with readline - Hurray for rlwrap

2008-02-01T23:52:00.000-08:00

Stumbled across dr.bob's sweet post on getting a readline wrapper around rhino.

Prior to reading his post, I had no idea rlwrap even existed, I'm simply amazed. GNU readline is IMO one of the most useful abstractions in all of *nix-dom. If I had known that I could use it even with tools that weren't readline-enabled, my previous lives would have been much easier. For example, looks like some foo-equipped DBAs (inspired by one of my heroes Tom Kyte) have found a way to use rlwrap to get a command history for sqlplus. It's been ages since I've actually used sqlplus (most time these days is in psql thankfully which natively supports readline), but this would be step 0 in my Oracle client install.

Anyway, back to Rhino. rlwrap + rhino is lethal. Gives me vi key bindings in rhino.

My setup ended up like so:

* Installed Rhino to ~/java/rhino
* Created script in ~/bin which was already on the path
* chmodded said script
* Added vi editing mode to ~/.inputrc
* Edited rhino commands with vi bindings
* Lived the good life

The various files looked like so:

~/.inputrc
set editing-mode vi

~/bin/rhino
#!/bin/bash
for jar in `find ~/java -type f -iname '*.jar'`; do
export CLASSPATH=$CLASSPATH:$jar;
done
rlwrap /opt/java/current/bin/java -cp $CLASSPATH org.mozilla.javascript.tools.shell.Main -strict "$@"

Drop an exploded rhino anywhere into ~/java, and you're good to go.

Of course, GNU Readline supports hundreds of options including other bindings like emacs if your are so inclined.

REST Sprinkled into my XMPP

2008-01-31T23:54:00.000-08:00

Been doing some low-level IQ packet handling in XMPP lately and interestingly, RESTful philosophy seems to be influencing me from afar. We started out the packet design to send a list (roster, but not an XMPP roster) of users associated with an artifact to the remote replica. The initial protocol design had elaborate Add Document, Add User, Remove User, Delete Document packets with corresponding XMPP IQ responses and errors to each. It quickly became apparent that there wasn't really a need to actually have different packets for Add User, Remove User and even Delete Document. When we started envisioning the resource as a document, and simply hammering the list of users along with the document, much the way we would expect an idempotent PUT to work in REST, the entire protocol got much more simple - just send the list every time. When there are no users, send an empty document packet. Otherwise, just send the authoritative list. That's much easier than the book-keeping needed to track if I've delivered a notice for one user and if it's been acked - just send the whole thing every time.

Even better, if the destination misses a packet, the sender doesn't care, it can just send the current one because the new packet is authoritative at the time it's sent.

All said and done, this eliminated a ton of state maintenance on both ends of the protocol. The sender didn't have to remember what it had fired off to the recipient and if it was adding or removing someone, nor did it need to manage the corresponding ack/error packets in detail - if a packet failed, just resend current. Likewise, the receiver was able to simply take the incoming update and make it fact - the reconcile process would ultimately be far easier than the alternative which amounted to checking if a user existed on the document, removing if they did and erroring if they didn't. That level of detail was immaterial to the sender, the sender simply wanted to publish *the* authoritative list of users and nothing more.

Too bad it took a practical implementation to realize most of the early protocol design wasn't needed :) Kind of reminds me of the recent SOAP vs. REST debates.

Really Deep Dynamics

2008-01-28T21:36:00.000-08:00

Patrick loses me with this statement:

"Face it. The history of programming is one of gradual adoption of dynamic mechanisms."

I read the post as saying the equivalent of "automatics gradually replaced manual transmission". Sure, it's true depending on how you look at the problem, but I'm not convinced it is really a practical, applied endorsement of dynamically-typed languages. I'm not sure it makes the most sense if I'm worried about demonstrating a performant, maintainable, visually-appealing result to my customers. You may like that automatic for giving average Joe customer a test drive, but it's not what you take on the Autobahn and it's not what you use to haul a big load cross country. More likely, the automatic feature is what you sell the masses who can't use the more precise, focused machinery.

Looking at some of Patrick's points:

"The problem is they are rarely asked by open-minded people." - seems ad hoc and condescending from the beginning. Pot, kettle meet off-white. I'll try and ignore that tone for the rest of the response although it does resonate through the rest of the post.
"I know large systems can be built statically. I've done it in more than one language. I've also done it dynamically in more than one dynamic language. Have you?" - I'm guessing Perl and Python meet this requirement on the dynamic side; pick your poison on the static side, C, C++, Java even C# or VB if you like to make me suffer. From my experience, in code bases of roughly 100 KLOC in size or more, having a compiler in place to check that you haven't made any stupid mistakes was actually helpful. Note that I said helpful, not sufficient. The problem I have with relying on tests for code bases of size is that you assume programmers are capable (as a fallible piece of wetware) of writing near-perfect tests. This is simply not true and as the code grows larger, the chance that your tests are inaccurate increases - this is a function of human nature and deadlines, not any specific programming language, static, dynamic or otherwise.
As a counter point, I spent roughly $2000 of client consulting hours debugging an issue in a commercial product built on Python that looked like this:

[foo,bar] = getImage(file, iso_varriant): ...

In normal circumstances, this returned a tuple of two items. However, in certain file system encodings, this returned a tuple of three items. Python, being its dynamic self, chose to ignore the third item in the return tuple under those conditions. In a full unit and integration test suite, the test for this method passed. In the runtime, it failed because the third argument was ignored. There was no good way to simulate the test because you could only reproduce the pain when you physically had a CD in the drive (due to the way the python interpreter handled the ioctls). In other words, having the world's most awesome, dynamic test suite failed but was still syntactically correct. Having a dynamic language made the failure pass silently where as a statically checked return type would have barked before the program made it to test or bailed with a stack trace in the runtime. Conversely, the Python interpreter quietly chugged along until the failure occurred > 20 lines down in the stack making it even more difficult to diagnose. Not always the case to be sure, but I think you can make equally valid arguments on both sides of the fence if you've used both static and dynamic languages in the wild. Relying on tests for refactoring and coverage of codebases is a luxury of small to mid size code bases. Inevitably, if your code is big enough, it will touch enough edge cases that the tests do not cover all conditions and changing a method will result in runtime errors, not test failures. That is a function of our cognitive limits as humans, not any programming language.

"On the other hand nothing eases the pain of bad projects." - absolutely. Personally I don't see anything about static languages that makes this more likely. From my experience, bad PM can railroad any initiative, static or dynamic.

"Face it. The history of programming is one of gradual adoption of dynamic mechanisms." - so true. Except, it is true in the context of static languages becoming dynamic, not in the sense that dynamic languages are conquering the world. Consider the success of static languages since the 70's:

The browser I'm using to post this blog is written in C++. Cross-platform, dynamic C++ but statically-typed C++.

The system libraries this browser uses are built on C.

The windowing libraries this browser is using are built on C and C++.

The OS the windowing libraries and system libraries used by this browser are built on C and expose strongly-typed C APIs.

The iTouch I used to read the original post was written almost entirely on C.

Nearly every device, feed reader, web server and browser that parses this content will be written in C or C++.

The browser you are using to read this was almost certainly written in C or C++. It might use some JavaScript, but that is optionally typed (based on the current ECMAScript spec and is interpreted by a C-based interpreter).

Conversely, I'm not using a browser built on Smalltalk. My system libs are not built on Scheme. In fact, to be honest, none of the utilities I use on a daily basis are built on a dynamic language (ObjectiveC being the potential crossover and its intents are not to be dynamic). There may be a few Java applications on my MacBook Pro that I use regularly, but even those are heavily reliant on kqueue - a kernel facility - and native windowing libraries. Firefox is proud to announce new Mac-native widgets, not new Smalltalk plugins.

Are dynamic languages influencing all that native code? Well maybe. I can reload kernel modules and I can link to DLLs or SOs, but loading all that static, native code is done through more static native code. I don't use a Ruby linker, the linker that makes my OS do its dynamic magic is proudly compiled to a low-level set of assembly instructions and usually machine-specific instructions, coded by a handful of really smart people at Microsoft or the GNU Foundation.

I'm hard-pressed to come up with any end-user applications that I find useful which do not directly depend on a statically-compiled something. Yes, some of the web applications I use leverage Ruby, Seaside, etc., but the browser or RESTful client library that access them is (for me anyway) immediately based on a statically-typed library (Firefox, libcurl, etc).

So, I'm probably missing Patrick's point. Most of the innovative apps I use don't depend on dynamic programming languages. In fact, if you took away many of the languages Patrick sites - Scheme, Lisp and Smalltalk, I would be able to go about my day without a single glitch. Conversely, take away C, C++ or Java, and I'm pretty sure I'd notice (i.e. no OS, > 75% http traffic, ~50% of http applications servers respectively).

XMPP For Integration

2008-01-25T07:23:00.000-08:00

Integration bloggers have been talking about using XMPP for years now. The XMPP community has entire specifications dedicated to things like RPC, Service Discovery and PubSub that make weaving together distributed systems easier (I think the native presence information helps as well). Matt Tucker gives a solid overview of XMPP in the integration space with a new post that is getting a lot of press coverage and shot to the top of Digg. I feel like XMPP's potential as an integration tool is finally starting to get the attention it deserves. Maybe a result of the imploding WS-Death-Star stacks and heavy-weight standards that have plagued integration developers for years? Maybe a natural gravitation towards a better technology stack - hopefully.

River Disappoints Again

2008-01-20T18:42:00.000-08:00

I had a desire to see if I could get Jython or JRuby working with Apache River this weekend for some distributed monitoring and task execution I want to try. Need something quick, reliable and with code mobility and ideally can integrate easily with Java. I did see some new updates on the River site which was initially encouraging, but after digging, looks like little has changed. There were no source distributions which was concerning, but I'd used the older Sun distributions of Jini before, so I wasn't scared about building from source. I proceeded to SVN check-out. Sadly, the listed SVN URL doesn't work - there is nothing there.

Walking up the tree to the /asf/ root, there is no sign of the source or the river project. I'm sure it is around somewhere, but seriously, how much less approachable could this project be? This is no way to attract new users. I didn't think it could get worse from the old Sun site; I was wrong. What a shame.

Looks like it's back to Rinda then. I've avoided Rinda in the past for important data given that there is no reliable storage of tuples. At this point, probably easier to bolt that on to Rinda than to pull anything meaningful out of River. Alternatively, maybe a better approach would be to make smarter Rinda clients that can survive a failure of the master ring server with some local persistence and eventual consistency approach.

CSRF is the new XSS

2008-01-17T23:55:00.000-08:00

I've been looking at Chris Shiflett's CSRF GET to POST converter and have to say that it's got me a bit freaked out. I don't normally do the annual prediction thing, but the more I look at it, the more I think we'll see 2008 as the year of the CSRF, particularly if social networking sites continue to grow in popularity among less-technical users.

I've seen mention that the attack can be mitigated by using a nonce of sorts in the form and session data, a value that must be posted back with the form for a valid request. But as I look at how Chris executed his redirector (loading an iframe on click), I can't help but think that once I have an unprotected and confused security context on the browser, I'm able to work around such nonces when:

1) I can make a reasonable guess that the user has an authenticated session. I don't always have to be right, just every couple of times will do fine.

2) I can parse out a response from the server such that I can snag the nonce parameter. Not hard once I have a confused, unprotected scripting context in an iframe that the browser is mistakenly trusting.

#1 is not a big deal. I can make some reasonable guesses, target my links appropriately using a bit of social engineering. Doesn't have to be terribly accurate, just moderately successful and I can grab some "seed" accounts from which to stage future attacks.

#2 is easy - if I can execute script in an iframe (as Chris' demonstrates), then I can string together multiple XHRs. As long as the user has a session I can piggy-back on, extracting a form nonce is no more difficult than submitting the post in Chris' redirector.

Would love to hear some thoughts on how others are dealing with the issue in the face of user-provided content.

Oh, and if you're wondering how to execute one of those attacks, you have a Blogger account *and* you clicked on the link to Chris' account above, consider yourself vulnerable - that's about all it takes.

+1 To Yahoo! Games '08 Predictions

2008-01-13T01:07:00.000-08:00

Yahoo! Games is spot-on for their predictions of the upcoming gaming year.

In particular, I think we'll see the following:

1) The Wii will jump the shark - I played the Wii for several hours over the holiday season and while it was fun, the novelty wore off quickly. I tried like mad to will Wii Sports into an online mode but to my dismay, that doesn't exist. You can only swing a virtual tennis racket in the confines of your own living room for so long while your neighbor is playing madden with friends across the country on a system two generations ahead of the Wii.

2) In spite of being the worst-marketed, most technically advanced gaming platform ever devised, the PS3 will rebound. For $200 more than the cost of a Blue Ray player, you also get a multi-cell processor on a linux-driven beast of a gaming system that helps society by lending it's processing power to protein-folding analysis in the off hours. I have a ton of stories about how its setup was better than the Wii's or XBOX 360, but those will have to wait for another post. In short, PS3 continues to lag in online content, but eventually I still believe that the hard-core gamers who spend the most money will continue to gravitate towards the PS3 and away from the Wii and the developers will follow. The early adopters I know (including me) are moving towards the better platform, in spite of Sony's seemingly insane approach of delivering the platform. I'm convinced devs will follow, nobody wants to build on the gaming equivalent of Visual Basic when the most advanced 3D platform ever is growing it's install base.

3) PC gaming will continue to deteriorate. I love my PC-only games. I think most games being developed today actually play better with a mouse and keyboard rather than two awkward analog sticks under-thumb. And while I've really loved playing WoW for the last few weeks, games like Crysis are simply making me miserable. By all accounts, I have plenty of hardware to play Crysis, but I still get Vista core dumps on a regular basis. As a result I'm rebuilding my awful NVRAID mirror on a regular basis (I'll never buy an NVidia RAID solution again, that monster of BIOS + Vista + Software RAID has cost me more time and pain than a decent hardware RAID solution ever would). Crysis is the best FPS I've ever played. It's immersive, well-designed and capable of making even the most modern GPUs sweat. None of that scares me, it's Vista's instability in 64-bit gaming and the resulting constant recovery exercise I'm forced to do that will push me closer and closer to purchasing console-only games.

Everybody Hates the Popular Kid

2007-12-21T21:54:00.001-08:00

Via Patrick, I just read Steve's latest blog post about code complexity. I think the vast majority of Steve's points are spot-on. Complexity in a codebase ultimately leads to disproportionate support and maintenance costs, complexity when on-boarding new developers, and a general productivity hit. When it takes me 30 minutes to get "in the zone", all it takes is one call from the boss to tear that down. That's a 60-minute productivity hit for a benign interruption and no tooling in the world will help you keep it all in programmer "RAM".

I was talking with a colleague of mine about this (Alok Singh) and we both agree that certain levels of complexity simply extend beyond the limits of most people's cognitive skills, it's simple and complicated all in one.

I agree with many of Steve's core assertions in this piece. But, I can't quite get past the Java-bashing and stereotyping that permeates the post and seems en vouge these days. Yes, there are bad java programmers out there, but is he really hitting on the limitations of the Java language and/or Java programmers in general?

A couple of things that stuck out for me:

"Design Patterns are programmers, and only programmers who use certain languages. Perl programmers were, by and large, not very impressed with Design Patterns."

I'm not sure I see this as an endorsement. I don't really know any non-Perl programmers who find another programmer's Perl easy to maintain. I've written a decent amount of it, and two of the top ten most miserable experiences in my programming career have been trying to decipher CPAN modules. Terse code does not make for manageable code and Perl programmers tend to pride themselves on brevity, readability be damned.

"It's obvious now, though, isn't it? A design pattern isn't a feature. A Factory isn't a feature, nor is a Delegate nor a Proxy nor a Bridge."

I don't really know what Steve's background was prior to Google, but I can say that as someone who works on commercial software which is regularly extended, our customers do actually think that having a factory is a feature. If we swap out authentication mechanisms, customers extending or writing plugins want to use a factory - they want one way to access a single API in Java. Lots of them. We could say, "no, use the REST API" but that hasn't actually seen much uptake (although it's there) and quite simply doesn't scale for core functionality like authentication.

Next is:

"Dependency Injection is an amazingly elaborate infrastructure for making Java more dynamic in certain ways that are intrinsic to higher-level languages. And – you guessed it – DI makes your Java code base bigger."

Actually, I can't see how this could possibly be correct so I'd love to hear more detail. IoC approaches like Spring reduce my code because I'm not using those design patterns that Steve is so against. I don't really care about creational logic any more because the reference I need is injected for me. I don't see Singletons or Factories any more, just references. I may have more configuration files than I did before, but I certainly have less code. Mature libraries like Spring do other nice things for me like hiding unnecessary unchecked exception handling, transactions across multiple resources and making an object remote when it has no inherent remote-ness. For example, I can publish any interface across Hessian, Burlap, JMX or RMI using spring configuration, the code doesn't know the difference. IoC has its issues to be sure, but code bloat is not one of them.

Moving on:

"I recently had the opportunity to watch a self-professed Java programmer give a presentation in which one slide listed Problems... The #1 problem he listed was code size: his system has millions of lines of code."

Ok, so the guy is a bad programmer, or he is bad programmer by virtue of being a Java programmer? I don't see the correlation. If he were giving a talk on VB would this even make the blog rounds? There are lots of bad programmers out there, but I feel like there is a real tendency to use guilt by association against the Java community. At the end of the day, one thing that is generally not disputable is that aside from the core *nix and C library collection, Java is *the* most ubiquitous set of libraries available. Steve likely has the ability to write a native hook against a native C library, I quite simply don't have the time and the Java libraries are my second-best resource.

As a counter-point, I looked at the amount of code in the mainline stable Linux kernel (2.6.23.12), which by count runs at 768119 in just the C - no headers, macros, etc. That's pretty big, and if you look at the committers, there are hundreds of people and companies (but not the almighty Google I might add) contributing towards that code base. But, for the life of me, I can't find a blog post slamming the amount of code in the Linux kernel or that it's too complex, or that it isn't contributing towards the greater good. Similarly, Apache HTTPD core seems to be running over 250K for just the HTTPD C code and surely requires a more skilled developer pool than your average Java developer would fill.

At the end of the day, I don't think Java is a programming panacea. But, if I put on my agile hat, and see that customers want solutions that they can run in their existing environments without worrying abut low-level library dependencies, and reuse a crazy amount of existing code, and that they can hire decent programmers to fulfill, then Java meets those tests and might actually be the simplest solution that delivers business value. C might be faster, Perl more "cool", but neither necessarily makes the most business sense. Business value isn't just about what the technology Illuminati tip their hat to. It's grounded in accessibility and more importantly, can I hire people to implement and support it. In that sense, Java is a decent solution these days. I tend to think business viability is a decent measure of a language, regardless of how it got there.

Socializing Eventual Consistency

2007-12-20T21:19:00.000-08:00

I've followed Werner's blog for a while now and his latest on eventual consistency is one of my favorite posts. I recently advocated for an eventual consistency approach for a service offering we're putting together and lost. Part of that I think was that the technology to build an eventually-consistent client could really use some improvement and I'll talk to that in a later post. More at play though was the general issue of pushing people's comfort zones with new concepts.

I was somewhat conflicted after the discussion because the Agile part of my brain wanted to believe that the simplest thing we could do was flow with out skill set and just write to an SQL database. The systems experience in me was screaming the whole way at building a tightly-coupled, highly intertwined system when staring at a green field that didn't really have highly-structured relational data.

As always, I could have done a better job in making my case, focusing more on the technical and less on the emotional, but I sure wish I would have had read Werner's blog before the discussion. Two things really stood out:

"Eventually consistency is not some esoteric property of extreme distributed systems. Many modern RDBMS systems that provide primary-backup reliability implement their replication techniques in both synchronous and asynchronous modes."

When I talk to people about things like eventual consistency, I generally get a response along the lines of "that's cool, but we're not Amazon". No, most companies aren't, but I'm not sure that we shouldn't leverage all the brain power and learning from Amazon to our benefit. You can only make buggy whips for so long while thought leaders speed ahead.

The other piece that stood out was:

"Inconsistency can be tolerated for two reasons: for improving read and write performance under highly concurrent conditions and for handling partition cases where a majority model would render part of the system unavailable even though the nodes are up and running."

Customers are always going to want the C, A and P from commercial software, but experience tells us that they are not all possible. If you buy Werner's arguments, the P is impossible for distributed systems. Most business-focused users I've encountered, when really pressed, almost always choose the A over the C.

When trying to socialize concepts like these, there is truly an art to helping people out of their comfort zones. I'm not as good at it as many people, posts like Werner's go a long way towards helping with that.

Now, if they can just fix that SimpleDB API...

Cheers to Adobe

2007-12-13T21:53:00.000-08:00

This is great news from Adobe and should really help them distance themselves from other Ria platforms by providing the middleware to help build better apps without the hefty price that used to come with LiveCycle DS. While I always liked LiveCycle DS, the price and proprietary nature always made recommending it difficult. No more! Now, if we can just get them to keep opening up and give the community the player :)

I also read this announcement to mean that the AMF spec is fully open as well which is also great to hear. I was always unclear on exactly what was available surrounding AMF, no more ambiguity there is equally big news.

Atom over XMPP

2007-11-29T00:38:00.000-08:00

I'm currently experimenting with the idea of using Atom over XMPP, at a minimum to push out a pointer to an entry behind a firewall into a more public staging area.

The thought would be that the Atom "pointer" to the entry is pushed into the public space, and when a client of that entry wants more detail, there is an XMPP service associated with the entry that can be used to retrieve the details.

Exactly half of me thinks that this is what Atom was meant to do - get the notion of an entry out there with enough information to make a decision about weather you want the detail, and let the interested parties decided if they want to fetch the remainder. The other half thinks that this is just the geek in me wanting to use Atom when there really isn't a need for it and that XMPP and less-structured XML stanzas can work just fine, that I can demand pull the list of entries when needed as well as the details.

There are some general rumblings out there on the interwebs, but no great success stories. Anyone care to share?

St. Peter seems to be thinking along similar lines. I would love to hear some real-world usages before I'm sold on Atom in this context.

Update: ralphm points to a newer draft on xmpp.org. Thanks!

Having and Eating Cake

2007-11-27T01:15:00.000-08:00

Fuzzy asks about options for disconnected client alternatives.

I started a reply, but it got too long so I decided to post and link instead.

I've now done this analysis a few times for a few different scenarios and as with most things, the devil is in the details. At the end of the day, although not quite so dogmatic, what usually matters most is ease of programming and that is generally driven by your server-side. It probably shouldn't be so, I'd love to say "Just put a set of RESTful services out there and I'll handle the client thanks", but in reality the separation is rarely that clean with more rich (i.e. offline) clients.

Fuzzy asks about a couple of technologies that I've worked with and others are mentioned in the comments.

I'd stay away from Tibco GI if you want to have any hope for advanced functionality like offline access, XMPP or direct socket manipulation. More rich APIs are available in XUL and AIR if you need to go there, and when I last worked with GI it was quite brutal (I.E. for development environment, no FF support, massive code base). Most of these have changed, but I feel like with something like GI, it almost doesn't matter that it's BSD licensed, you won't be able to grok or change the code anyway, it's too complicated and if you need what it does, you are almost certainly pounding a square peg into a round hole.

In the comments, Brian advocates a combination of Gears and GI - that scares me to death. Two technologies that have no knowledge of each other using the browser as a binary integration bus via JavaScript - I've yet to see that work on any effort of size and it's bound to cause some serious headaches, especially when you can't control what a user does from either side of that integration. Doesn't pass my sniff test.

Most importantly thought, I'd look at the community story there, it's not very compelling. GI strikes me as almost abandon-ware by Tibco and seeing as how they were so late to the OSS game, I have doubts about their commitment to sustain the community. Conversely, AIR has far more momentum and Adobe worked hard towards building community from the beginning.

For what it's worth, when I was talking with Tibco about these concerns and pressing them to open the code, they laughed and told me it would be really expensive. Guess I was on to something. I'll continue hammering Adobe to do the same with the Flash Player with every opportunity I get.

Having worked with XUL, I found it quite painful. The entire effort is under-documented to say the least. For Fuzzy's purpose, it obviously Mozilla-specific which may or may not be an issue. I can't say I've ever seen anyone build anything with any serious eye candy on XUL, just basic chrome and mostly tree views. If you have half an inkling that you might need some eye candy to sell your product, XUL probably isn't for you.

More importantly though, If it doesn't meet your lower-level needs for things like images or networking, be prepared to write some cross-platform C++ and COM-like IDL. You will also need to deal with version differences (i.e. XUL on FF 1.5.x is not the same as in FF 2.x). That said, if made to choose between GI and XUL, I choose XUL.

My personal bias is still for AIR. The community has good momentum, programming model is the most simple of the other options (including Sarge's Tcl comment - have fun with that return to DLL-hell). Not open though, so if you go that route make sure it's critical to your story and that open-ness is not.

Lazlo can "compile" to Flash as Mike W. mentions in Fuzzy's comments, but can also remain in AJAX/DHTML land if you so choose - Flash simply performs better. In all the testing I've done, Lazlo will not perform consistently nor render consistently if you do not transform to FVM bytecode, so those are likely not options.

One other potential option is haxe. Saw a presentation on it at OSCON a few years ago, seemed like a solid technology but hasn't really caught on in the states anyway. It's been around for several years now and people who use Neko seem to love it. Can't speak from experience with that one, but I'd give it a look.

Worst Measurement Ever

2007-10-26T22:02:00.000-07:00

I've been resisting posting about it for some time, I think since Patrick pointed me at the link almost six months ago. I've reached my breaking point. In my quest to learn Erlang, I've come across at least three blogs and/or articles that actually site this "measurement" with attribution as if it were some sort of legitimate claim as to the scalability of Erlang.

Given that I'm reading his book, I would really expect more from Joe Armstrong and by attribution Ali Ghodsi.

The Apache vs. Yaws measurement is one of the most useless pieces of information produced by the Erlang community, to the point that I'd argue it does a disservice to the Erlang community and the language.

In any sort of quasi-scientific measurement (or primary school science experiment for that matter), I would expect to see:

the actual code used to test the server
the actual Linux kernel version
the actual yaws server code

Instead, we see a graph of an under-documented experiment that creates conditions for a DoS test at best, not a web server scalability test.

From the looks of it, this "measurement" is not:

documented to any reasonable extent - what kernel was used? what was the exact Apache configuration? what was the Yaws code used to serve the files?
repeatable - no source, little documentation, little detail on the environment
peer reviewed - without the above, nobody else can discuss in detail or attempt to repeat the same results
valid - it simply does not repeat a real-world environment faced by any modern web server

Allow me to support some of those assertions from real-world experience helping large web sites:

Not knowing what Linux kernel version was used, how Apache was configured (in detail) and how Apache was built, it's impossible to know if Apache "was configured for maximum performance". Seeing as they decided not to share any of that vital information, I consider the entire experiment invalid. These things matter, just like if I were to run BEAM on an SMP system without SMP, it's easy to misconfigure the runtime. For example, Apache started using epoll in 2.6.x kernels. Anything earlier than that would be an inappropriate kernel to use with mpm_worker. We wouldn't know from the sparse detail if it is a valid configuration or not.
In a real-world environment, if I saw 100s of connections inactive for 10 seconds at a time, I would simply set the connection timeout to five seconds. Not sure how you would do the same on Yaws.
Most production web servers don't serve one byte files (the "load" requested by the clients in the measurement). I can't actually recall the last time I saw a high-volume server dish out more than one or two one byte files. Instead, real web servers serve files that measure in the KB or even MB in size. In fact, given Erlang's empirical difficulty handling basic file I/O, I'm not surprised that they chose a one byte file to use in simulating "load". If the file were any larger, Yaws would likely have been swamped under it's own unbuffered I/O implementation and exhibited substantial latency per request.
A one byte file would be cached upstream of the server in a high-volume site, particularly if your clients were operating over latent connections where one character per ten seconds was realistic (i.e. dialup).
If this were a real DoS attempt, it would be choked off at the ISP router, well before the web server saw the intentionally slow request.
I can personally type an HTTP request faster than one character per ten seconds, this is simply not a realistic access pattern from a benign client.

I'm not going to say that Erlang does or doesn't scale. My point is that this diagram and the corresponding writeup are utterly useless and that to site it as a valid study is irresponsible at best (especially when it's your own thesis).

I put this in the same category as Microsoft FUD - there is an agenda here, and people will read it and say "Oh, yea - told you we rock" without questioning the details. Most however, should dismiss it as pure FUD, and FUD served from an Apache 2.2.3 server no less (maybe Yaws wasn't up to the task).

My Bow-Legged Master - Comment

2007-10-23T23:48:00.000-07:00

I tried to comment directly on Bill's Blog but comments were broken again. What I wanted to say was:

I think the real tragedy of this legacy is that we currently have installed, in nearly every networked node in existance, the potential for a simple, iterative, resource-driven client that can't really speak the protocol it was designed to handle. I repeatedly find myself grudgingly writing a curl wrapper to test a RESTful API in a pinch while my browser looks at me and laughs.

Oh, and the irony that your comments button says "Post" rather than "Put" is not lost.

Hopefully comments will be fixed soon and I'll cross post, er cross PUT.

Scale in an Ontology Vaccum

2007-10-21T23:32:00.000-07:00

I had the pleasure today of reading Clay Shirky's Ontology is Overrated. Being in the collaboration software business, understanding how people classify artifacts is not just a science, it is vital to the evolution of the software we build.

For our purposes, I think it's a dual-edged sword. Make things too wide open, and users won't evolve their own classifications and people will stray away from the true strengths of the tools. Make things too ontology-driven and users will naturally migrate away from the structure, save everything locally and keep emailing and searching with Google Desktop. We have to build a better mouse trap to survive.

For me, a big take-away from Clay's piece was that any attempt to define classifications will fail. The Dewey 200 category is a fantastic example and one that really drives home our inability as simple humans to step out of context.

So, to paraphrase Clay's hypothesis a bit, any attempt I make at trying to define an ontology will fail, mostly because there is no way I can out-think the masses. Ok, fair enough, so how to build software that better helps the masses? Not too difficult at face value, things like tag clouds work great. Unfortunately, the implementation problems grow exponentially under scale. As the volumes of information available electronically continue to grow, and as information compounds on top of itself through links, blogs, etc, the notion of aggregating tag clouds, and more importantly, doing things like suggesting similar clouds shifts from a usability problem into an architectural one.

For collaboration software to innovate, it is faced with several interesting dilemmas. First, how do we expand beyond the bounds of our own control and pull in relevant data outside our domain? It's scary, but if you believe that collaboration software must go down that path, then we're faced with more integration concerns and less of the simplified workflow and text processing which is how most people think of their collaboration tools today.

Assuming you can go that far, how do you actually make sense of all that data? You start to run into scale problems real fast. Customers want one node to run everything and at the same time roll-up data from five internal sources, and ten external sources with partial federated identity all while being responsive. That's a tall order, and one that starts to make the "C" in "CAP" increasingly difficult to archive.

My gut tells me that Werner Vogels' and Amazon's approach to eventual consistency described in Amazon's Dynamo will win out. When you're aggregating and analysing terabytes of data looking for relevance, you have to make sacrifices.

plan for $display_id will be renewed for another year

2007-10-18T23:42:00.001-07:00

I received an email from Yahoo! today with this in the subject line "Your Yahoo! Custom Mailbox plan for $display_id will be renewed for another year". I've seen recently that Yahoo! has been hurting and that they have been losing customers, but I can't imagine why.

I used to host all my small business domain registrations and email with Yahoo! Starting a year ago, I moved nearly all my paid services away from Yahoo!. It became clear to me over time that they didn't really intend to actually support their small business services. I challenge anyone to send an email to Yahoo! Small Business support, you can't. There is no means to contact them. At best, you can fill in a generic form that gets lost in the ether (I've tried that a few times too and nobody has ever responded).

No thanks, I'll take my money somewhere that has staff capable of correctly generating a mail merge and that will actually respond to my emails when I have a hosting issue.

Yahoo! is and will be my portal, but come on guys. Maybe Jerry can correct the course, I hope so.

Mind the Gap

2007-10-08T22:22:00.000-07:00

I was recently fortunate enough to actually be offered a choice for my hardware and operating system at my current job. It's unfortunate that such choices aren't more common for engineering-types these days. I'm a big believer in the best tool for the job mentality and for me that's clearly some derivative of Linux on the desktop (and likely x86 Solaris on the server, depending on what I'm doing with it).

Faced with said choice, I went with Linux/x86 over Mac (that's another post all together). Having had good success in the past with Ubuntu, I decided to go with Gutsy Gibbon beta. Unfortunately, it was more beta than I would have liked.

First attempt at installation laid down the OS as normal. Being a sucker for eye candy, I wanted to get Compiz working ASAP in hopes of making the Windows users around me think twice. This turned out to be a big mistake. I ended up installing the binary ATI drivers and patching all in the same shot, and from there Gusty went into some sort of video card config hell. Despite my best attempts to hack /etc/X11/xorg.conf, I couldn't rescue my UI and had to start all over from scratch. Thankfully, I had a second hard drive available and copied all my userland tweaks there before rebuilding the whole rig.

This time around, I did a normal install. After making sure that was stable, I went after the binary drivers and got them working. Then, I allowed my updates to run, first system packages, then X11/Xde packages and binary drivers, backing-up my xorg.conf after each step so I could revert.

At the end of the day, the whole thing still isn't all that stable. I could bail on the eye candy, but that would be to admit defeat and that's not how I roll. Still though, I expected more. This experience was not all that much better from my previous Gentoo encounters with multiple reboots to the live CD to bail myself out. I was hoping for more, and didn't get it.

Finally, it's worth mentioning that Java UIs seem to struggle somewhat under my config with Compiz. IntelliJ's menus often don't show up and other Java-based UIs are also lacking.

Speaking of IntelliJ on Linux, it could really use some work. Their reliance on Java's fonts is plain painful, especially with so many nice anti-aliased fonts out there in Linux land. It's a pity that what is generally such a mature product couldn't keep-up with Eclipse and at least match SWT's handling of fonts.

Downsizing

2007-10-01T21:24:00.000-07:00

I'm both happy and sad to announce that I'll be downsizing my career. I'm leaving behind my work as a Technologist for Liberty Mutual and assuming a core engineering position with Jive Software here in Portland.

I'm sad to be leaving the team at LM. They are immensely talented and capable of great things, I sincerely hope they are given an opportunity to act on that talent.

That said, I'm looking forward to joining Jive, more so than most of my previous career moves. If you follow the news stories around Jive, they are doing good things and doing them well. They are building on open protocols and generally lean towards Open Source Software as well - both things I highly value.

Have a Seat on the Couch

2007-09-27T21:12:00.001-07:00

No idea why it took me so long to find CouchDB, but I will be watching its development closely and will start probing the source once I have a better handle on Erlang. Highlights include a RESTful API, a bevy of clients to talk with that RESTful API (Ruby, Python, Java, JavaScript), data replication for reliability, and off-line capabilities. I can't think of a much better use of Erlang on the server side and I love the use of SpiderMonkey for a Query language , that's plain brilliant and something people should use more of.

Flexy Flex

2007-09-14T00:13:00.000-07:00

I will be speaking at one of the Portland Flex User's groups on Sep. 20. The topic will be integrating Flex with RESTful applications, something that really should be easier than it is.

Apache Is My Service Hub

2007-09-13T21:59:00.000-07:00

Ok, so I nicked the idea of this post from mnot's Squid is My Service Bus and tried to extend the concept based on ways I've used Apache HTTPD in the past.

In general, I'm a big fan of Apahce HTTPD. It's immensely powerful and has been my "Swiss Army Knife" of the web for some time now, proxying this, caching that, dynamically serving things all the while never once becoming the primary constraint in any distributed system. That's saying a lot as a perimeter service responsible for all ingress traffic.

It occurred to me in discussions with Mike and Patrick that Apache's utility makes it an ideal Service Bus of sorts. Service Hub might be a better term. Most of Apache's default functionality is geared towards moving resource requests around and not protocol mediation (a key criteria of a bus). However, nothing stops the Apache developer from making the server more bus like if the requirements so dictate.

The idea goes like this

Service consumers, RESTful preferably (although SOAP will work too), desire to perform operations on a resource. The consumer has a client that is smart - it knows that it can rely on finding one or more Service Hubs in the local vicinity, but doesn't care exactly where that hub is.
The client performs some measure of discovery, a DNS lookup backed by BIND zones, a Zeroconf search for the Service Hub, a broadcast search, something that lets the client discover, at runtime, it's friendly local Service Hub. The important part is that all clients search for their local Service Hub in the same way and in a manner that lets us have one or twenty hubs depending on our needs. We can scale up or down and the client behavior doesn't change.
When the service client finds a hub, it makes a request of that hub. Next time around, it might use a different hub, or it might hold a keep-alive for the HTTP connection to that hub.

So, what makes Apache particularly well suited for this role?

Apache HTTPD is ubiquitous - Apache is capable as acting as an origin, a proxy or a gateway for resources all in one set of configuration while being 99% HTTP 1.1 compliant. It runs on all Unixes I know of, Windows and other platforms. I generally don't think of Apache as middleware, but if you abstract Apache up a layer and treat it as a RESTful gateway, it does fill a role traditionally reserved for middleware.
Apache HTTPD can reverse proxy - Apache has an added advantage that it can perform crude load balancing in a reverse proxy configuration. This means, you can have service clients consume HTTP resources without knowing that they are talking with one or twenty back-end origin servers. Sure, F5s have been doing this for some time now, but try installing thirty F5s in a large integration, one for each broadcast domain and see what it does for your budget. Additionally, Apache can reverse proxy for loads of protocols out of the box, HTTP(S), FTP and AJP open worlds of opportunities for the systems integrator.
Performance - Apache HTTPD builds on Apache's portable runtime (APR) which is a fairly low-level abstraction on top of most OS system calls. This gives it tremendous performance benefits (like using scalable non-blocking I/O ala epoll on Linux) while still remaining relatively platform-agnostic.
Extensible - Not long ago, writing Apache modules was a poorly-documented black art. That is improving substantially with books and better reference documentation. But, writing a module isn't necessary at all. With dynamic language alternatives like Python and Ruby that can run in-process with Apache workers, you can extend Apache as a Service Hub however you like and without writing C.

I don't expect to see wide adoption of this type of approach any time soon. If RESTful architectures catch on and people start to see the web as a series of resources, Apache HTTPD's role will continue to grow. It's high time we stopped looking at Apache HTTPD as just a web server and started viewing it as an HTTP Gateway.

Async Beauty

2007-09-06T18:17:00.000-07:00

Wow, came across AsyncWeb today and have to say it's a thing of beauty. Excuse me while I tout my own greatness for a while but about two years ago I had an idea to write a NIO/async REST engine for wicked fast, non-servlet cruft, async REST implementation and AsyncWeb + RESTlet is headed in that exact direction. RESTlet is equally impressive and has come a long way over the last six months. Love the pluggable connector design for RESTlet, should really allow it to move in ways that the Servlet spec hasn't been able to.

If you are trying to do things like COMET with traditional threaded servers, you're absolutely mad and need to look at this approach.

New Toy

2007-08-27T20:02:00.000-07:00

I have a new toy on back-order with Amazon.

In short, I'm after most of the iPhone features without the phone part. I want:

A Video Player (including DIVX)

A Music Player (including OGG)

A browser over WiFi in a pinch

A small form factor with decent battery life

WiFi connectivity

It's not an ideal device, but it's a start.

I'd like to see an iPhone-like browser, but Opera will do - this isn't my main browser
If I need a full-power browser I'll bust out a laptop.

I wish I had a competitive device that would play iTunes content

As an additional plus, the Archos will allow me to play Flash content. In general, I'm not seeing a compelling reason to purchase an iPhone over the 605. At a minimum, until until Apple fixes the iPhone's Bluetooth stack to let me sync calendar and contact over Bluetooth, I'll be looking elsewhere.

Enough with the Thai Already

2007-08-25T21:59:00.001-07:00

I love living in Portland for many,many, many reasons. But enough already with the lack of food diversity. A local sketchy Teriyaki place just closed down in my Hollywood neighborhood. What opens in it's place? Another Thai restaurant. I won't be going there, I already have a Thai rotation of some really good places, I don't need another. I can count at least ten in a three mile radius, Google says there are more. So, if you're opening a new restaurant in Stumptown, I submit:

No more Thai

No more Pho

A strong Indian effort would make a killing

OMG What Happened to My H3@d3r?

2007-08-24T00:01:00.000-07:00

Wow, read this from Scott Chapman which got me down. Yet another attempt to do REST from a Flex app with significant hackery bolted on, fails. I've said it before, when you are concerned with your HTTP headers, you are fighting the wrong problem. I don't spend much time thinking about my TCP window scaling, why should I have to constantly focus on HTTP protocol-level craps?

I'll say it again, someone needs to step-up and write an ActionScript HTTP client. Until that happens, the browser and it's FVM cohorts are the wrong platform for RESTful requests.

Toss the Bath Water and the Baby

2007-08-23T23:35:00.000-07:00

Bill Bill de hÓra picks up on the broken-ness of the Flash VM's ability or lack thereof to expose headers to the fault callback. He also pulls in to hell with bad web clients. As I read that, I'm convinced that Flex is actually my escape hatch from browser insanity. I'm heads down in CSS Mastery at the moment trying to make my nav bar behave sanely across browsers for some non-profit work I'm doing. It's sick the amount of work I'm doing to make sure A is left of B in IE 5.5 on Mac *and* IE 7 on XP x64. Comparatively, the time spent doing the same thing in Flex would have been 1/10th the effort if that. So, I actually agree with Bill on this, we should get rid of bad clients, those bad citizens of the web that break and hurt our beloved web.

That's right, we need to toss out the browser all together. Em, or even better, let's actually make our browsers behave as good citizens in the HTTP multiverse that is bearing-down on us like a freight-train made of UTF-8 printable characters (and none of those binary, non-readable ones either thank you very much). As an open message to FF, IE, OP, KQ, etc., I want DELETE. I want access to headers in my XHRs, I want full control. You want to hide it from me, the ignoramus developer because I'm not ready? Ok, wget, here I come!

XMPP MIA?

2007-08-22T23:36:00.000-07:00

I've been trying to hit-up xmpp.org for two days now and nothing, nada, zilch from their authoritative DNS server. My dig for the www looks like so:


--> dig www.xmpp.org

; <<>> DiG 9.4.1-P1 <<>> www.xmpp.org
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 54001
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 6, ADDITIONAL: 8
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;www.xmpp.org.                  IN      A

;; AUTHORITY SECTION:
org.                    45762   IN      NS      TLD2.ULTRADNS.NET.
org.                    45762   IN      NS      TLD5.ULTRADNS.INFO.
org.                    45762   IN      NS      TLD4.ULTRADNS.org.
org.                    45762   IN      NS      TLD6.ULTRADNS.CO.UK.
org.                    45762   IN      NS      TLD3.ULTRADNS.org.
org.                    45762   IN      NS      TLD1.ULTRADNS.NET.

;; ADDITIONAL SECTION:
TLD2.ULTRADNS.NET.      45863   IN      A       204.74.113.1
TLD5.ULTRADNS.INFO.     1251    IN      A       192.100.59.11
TLD4.ULTRADNS.org.      45762   IN      AAAA    2001:502:100e::1
TLD4.ULTRADNS.org.      45902   IN      A       199.7.67.1
TLD6.ULTRADNS.CO.UK.    477     IN      A       198.133.199.11
TLD3.ULTRADNS.org.      45863   IN      A       199.7.66.1
TLD1.ULTRADNS.NET.      45762   IN      AAAA    2001:502:d399::1
TLD1.ULTRADNS.NET.      45863   IN      A       204.74.112.1

;; Query time: 62 msec
;; SERVER: 209.244.0.1#53(209.244.0.1)
;; WHEN: Wed Aug 22 23:49:06 2007
;; MSG SIZE  rcvd: 344

I like XMPP, I do. But if *they* can't get their DNS right, I'm scared (XMPP depends heavily on DNS working properly).

Easy Peesy

2007-08-22T09:33:00.000-07:00

Making it stick.: The "Ease" Argument Again?

I've got Patrick's back on this one, I have a hard time believing that anything that starts with WSDL is going to make my life easier.

I like Dan's project, I do. I think XFire is the way to go if you have a hard and fast requirement to expose a Java-based service via SOAP. But make sure you actually need to go there before you do. If your biggest customer says "we use SOAP and only SOAP" for integration with trading partners, hello XFire.

Thus lies the path to Kakotopia! In my experience, you won't be pointing any tools at each other that actually grok that WSDL and can talk to each other without serious intervention on your part. My last experience trying to do this very thing was a Java/.NET integration for a project was about two years ago, the experience went something like this:

1) Expose XFire service by configuring Spring adapter booya
2) Check WSDL - looks OK, not that I'd know without studying the spec
3) Use VisualStudio .NET Stub generator wizard to create a C# class for invoking the service.
4) Try and run the class from step 3
5) Witness failure, something about unsupported binding
6) Read this.
7) Change API to XML-RPC.

I have about eight of these stories, some of them end with me snooping HTTP Headers because of obscure server errors. Some, like a recent attempt to create a Flex service to consume Confluence's WSDL end with obtuse client-side schema errors. That's about all the CPU cycles I give to these types of things, after that I fall back to things I can control and see. Maybe some day it will work, but I haven't seen it materialize yet.

Given recent episodes with REST, I'm not sure it's a clear winner from the browser either, because I again find myself looking at HTTP Headers to try and figure out what is broken. I don't want to spend my time doing that. Seriously, I'm done breaking out Wireshark to troubleshoot protocols that work until people try and get cute with them.

RESTful Flex Angst

2007-08-16T22:07:00.000-07:00

Following on the previous post about issues encountered getting Rails and Flex to play together, we've now observed that certain popular browsers do not correctly interpret the HTTP specification for status codes on HTTP responses. Namely, IE 6 (haven't tested 7) treats any 200 status code other than 200 as an error. Perform an overloaded POST to a RESTful service in IE and get a 201 back, IE exposes a status of error to interested applications.

Why does this matter? When trying to do Flex integration with a Rails back-end, we are required to rely on Flex's callback mechanism to tell us if we get a result or a fault. Flex apparently, relies on the browser's plumbing to determine the success of the request. So, updating a RESTful resource via a Flex HTTPService for example appears as a failure at the Flex layer because IE tells the FVM that it got an error. This was the source of great angst for us because most of us test our Flex applications in Firefox.

The problem is further compounded by the fact that Flex doesn't expose any of the raw response data to us, simply a FaultEvent with little useful information (one of the dreaded stream errors actually). No headers, no body content, nothing. I've seen blog posts that hint at this being a limitation of the FVM's status as a lowest common denominator runtime, Flex included - some browsers don't expose headers to plug-ins so neither can the FVM.

When I find some time, I'm going to fire up the ActionScript Socket API and write an HTTP client that doesn't require overloaded POST, that exposes plumbing of the HTTP, and that allows *me* to determine what is or is not an error.

Alternatively, I've got a more simple tool in the works that will allow you to test HTTP methods of all sorts and status codes propagate back into the FVM.

In a future post I'll dig into some of the interesting behaviors we found examining these behaviors from the stand-alone player and how it does it's networking.

RESTful Rails Angst

2007-08-07T20:25:00.000-07:00

I've been working with Rails to provide RESTful services to a Flex application recently and a number of issues have come up that people need to know about dern it.

Today's story entails a battle with Rails' scaffold_resource. The RESTful generator makes a noble effort to create RESTful controllers. However, a few minor issues pop-up that need repairing in the current incarnation of the generator.

The most glaring issue is that the default code doesn't handle ActiveRecord find errors gracefully. For example, if I have a FoosController servicing requests to "/foos/1.xml" and that model :id=1 doesn't exist, the default FoosController will throw a RecordNotFound which up the call stack results in a 500 error being returned to the client. This, is unfortunately not "RESTful" as they say. The correct behavior would be to return a 404 indicating that the desired resource does not exist, not that the server could not handle the request. Adding this behavior is not difficult, but it's somewhat tedious in that each generated controller exposes four find calls that can fail in this way.

Additionally, as Peter Armstrong's solid PDF-only book on Flexible Rails points out, the default implementation of to_xml used throughout the controller is quite painful for certain clients. Most programming languages don't allow "-" in member names and as such, mapping tools that attempt to make XML traversal easier by exposing the XML structure as a choke on the default XML marshaling. As Peter points out, you can change this behavior by adding the ":dasherize" option but again, this is all over RESTful Rails controllers.

I could fix both of these through some hacking on the Rails distro, but then I have to remember to keep my changes across upgrades. Another option would likely entail injecting some app-specific base classes into the hierarchy, again not an idea I'm crazy about. If I come up with a better approach, I'll post it here. Or maybe back as a patch :)