Monday, January 28, 2008

Really Deep Dynamics

Patrick loses me with this statement:

"Face it. The history of programming is one of gradual adoption of dynamic mechanisms."

I read the post as saying the equivalent of "automatics gradually replaced manual transmission". Sure, it's true depending on how you look at the problem, but I'm not convinced it is really a practical, applied endorsement of dynamically-typed languages. I'm not sure it makes the most sense if I'm worried about demonstrating a performant, maintainable, visually-appealing result to my customers. You may like that automatic for giving average Joe customer a test drive, but it's not what you take on the Autobahn and it's not what you use to haul a big load cross country. More likely, the automatic feature is what you sell the masses who can't use the more precise, focused machinery.

Looking at some of Patrick's points:

  • "The problem is they are rarely asked by open-minded people." - seems ad hoc and condescending from the beginning. Pot, kettle meet off-white. I'll try and ignore that tone for the rest of the response although it does resonate through the rest of the post.
  • "I know large systems can be built statically. I've done it in more than one language. I've also done it dynamically in more than one dynamic language. Have you?" - I'm guessing Perl and Python meet this requirement on the dynamic side; pick your poison on the static side, C, C++, Java even C# or VB if you like to make me suffer. From my experience, in code bases of roughly 100 KLOC in size or more, having a compiler in place to check that you haven't made any stupid mistakes was actually helpful. Note that I said helpful, not sufficient. The problem I have with relying on tests for code bases of size is that you assume programmers are capable (as a fallible piece of wetware) of writing near-perfect tests. This is simply not true and as the code grows larger, the chance that your tests are inaccurate increases - this is a function of human nature and deadlines, not any specific programming language, static, dynamic or otherwise.
  • As a counter point, I spent roughly $2000 of client consulting hours debugging an issue in a commercial product built on Python that looked like this:
[foo,bar] = getImage(file, iso_varriant): ...
  • In normal circumstances, this returned a tuple of two items. However, in certain file system encodings, this returned a tuple of three items. Python, being its dynamic self, chose to ignore the third item in the return tuple under those conditions. In a full unit and integration test suite, the test for this method passed. In the runtime, it failed because the third argument was ignored. There was no good way to simulate the test because you could only reproduce the pain when you physically had a CD in the drive (due to the way the python interpreter handled the ioctls). In other words, having the world's most awesome, dynamic test suite failed but was still syntactically correct. Having a dynamic language made the failure pass silently where as a statically checked return type would have barked before the program made it to test or bailed with a stack trace in the runtime. Conversely, the Python interpreter quietly chugged along until the failure occurred > 20 lines down in the stack making it even more difficult to diagnose. Not always the case to be sure, but I think you can make equally valid arguments on both sides of the fence if you've used both static and dynamic languages in the wild. Relying on tests for refactoring and coverage of codebases is a luxury of small to mid size code bases. Inevitably, if your code is big enough, it will touch enough edge cases that the tests do not cover all conditions and changing a method will result in runtime errors, not test failures. That is a function of our cognitive limits as humans, not any programming language.
  • "On the other hand nothing eases the pain of bad projects." - absolutely. Personally I don't see anything about static languages that makes this more likely. From my experience, bad PM can railroad any initiative, static or dynamic.
  • "Face it. The history of programming is one of gradual adoption of dynamic mechanisms." - so true. Except, it is true in the context of static languages becoming dynamic, not in the sense that dynamic languages are conquering the world. Consider the success of static languages since the 70's:
    • The browser I'm using to post this blog is written in C++. Cross-platform, dynamic C++ but statically-typed C++.
    • The system libraries this browser uses are built on C.
    • The windowing libraries this browser is using are built on C and C++.
    • The OS the windowing libraries and system libraries used by this browser are built on C and expose strongly-typed C APIs.
    • The iTouch I used to read the original post was written almost entirely on C.
    • Nearly every device, feed reader, web server and browser that parses this content will be written in C or C++.
    • The browser you are using to read this was almost certainly written in C or C++. It might use some JavaScript, but that is optionally typed (based on the current ECMAScript spec and is interpreted by a C-based interpreter).
Conversely, I'm not using a browser built on Smalltalk. My system libs are not built on Scheme. In fact, to be honest, none of the utilities I use on a daily basis are built on a dynamic language (ObjectiveC being the potential crossover and its intents are not to be dynamic). There may be a few Java applications on my MacBook Pro that I use regularly, but even those are heavily reliant on kqueue - a kernel facility - and native windowing libraries. Firefox is proud to announce new Mac-native widgets, not new Smalltalk plugins.

Are dynamic languages influencing all that native code? Well maybe. I can reload kernel modules and I can link to DLLs or SOs, but loading all that static, native code is done through more static native code. I don't use a Ruby linker, the linker that makes my OS do its dynamic magic is proudly compiled to a low-level set of assembly instructions and usually machine-specific instructions, coded by a handful of really smart people at Microsoft or the GNU Foundation.

I'm hard-pressed to come up with any end-user applications that I find useful which do not directly depend on a statically-compiled something. Yes, some of the web applications I use leverage Ruby, Seaside, etc., but the browser or RESTful client library that access them is (for me anyway) immediately based on a statically-typed library (Firefox, libcurl, etc).

So, I'm probably missing Patrick's point. Most of the innovative apps I use don't depend on dynamic programming languages. In fact, if you took away many of the languages Patrick sites - Scheme, Lisp and Smalltalk, I would be able to go about my day without a single glitch. Conversely, take away C, C++ or Java, and I'm pretty sure I'd notice (i.e. no OS, > 75% http traffic, ~50% of http applications servers respectively).


Patrick Logan said...

Hey Erik, "loosing" you was the hardest thing I've ever done. ;^)

Patrick Logan said...

"I spent roughly $2000 of client consulting hours debugging an issue..."

Yeah, that's *way* more than anyone ever spent debugging something stupid in a static language program!

Erik Onnen said...

Doh, lame way to start a rant :) Fixed in the published version.