Quantcast

Ruby on Rails, Io, Lisp, JavaScript, Dynamic Languages, Prototype-based programming and more...

Technoblog reader special: $10 off web hosting by FatCow!

Monday, November 13, 2006

A Fatal Flaw in Opinionated Software

David Heinemeier Hansson has written a lot about opinionated software and syntactic vinegar. At first glance, the term opinionated software sounds perfectly reasonable. It seems, at least to me, to mean that the software looks and acts in a certain way... it has opinions like DRY and convention over configuration. That is not what opinionated software means to David. To David, a piece of opinionated software is written in such a way that makes it easy to do things one way and ugly and difficult to do things another way.

What does this mean? Glad you asked. Let's look for example at the ActiveRecord find method.

User.find(:first, :conditions => "active = 1")
# => # "1", "active"=>"1"}>

User.find(:first, :with => {:active => 1})
# ArgumentError: Unknown key(s): with

Ok, so the find method does not accept the key :with. That is fine, but let's say you and I are clever hackers and we wanted to enhance ActiveRecord to accept the :with key in order to do some neat magic that reduces the amount of code we have to write. For a Ruby developer, this type of modification sounds like it should be easy, but this is what I found when I first looked at the Rails source.

def validate_find_options(options) #:nodoc:

options.assert_valid_keys([:conditions, :include, :joins, :limit, :offset, :order, :select, :readonly, :group, :from])

end

That is right, the keys were all hard coded, meaning that if I wanted to add my own :with key, I would have to completely overwrite the undocumented
validate_find_options method, which inevitably would break things down the line as Rails added its own new keys.

When I first saw this I thought to myself: ah, must be an oversight. I wrote a patch in a few seconds that abstracted the array passed to options.assert_valid_keys into something mutable. I submitted the patch and had a lengthy discussion with DHH in which I was slapped by the opinionated software stick. Apparently, David thought that if I wanted to hack ActiveRecord, I was going to have to be punished by writing ugly code. It was the "opinion" of ActiveRecord that people shouldn't add a key to the find method, so David intentionally made it much harder than necessary to accomplish.

That is right, this was not a simple oversight, this type of code was completely intentional. It is David's opinion that Rails be harder than necessary to hack around and play with. That if I was going to stretch the limits of Rails, take it places nobody else had thought of, that I should be punished for doing so, and that punishment is ugly code.

This is the fatal flaw of opinionated software... in order to make design decisions that limit people's ability to do things you don't think they should do, you must punish them. The form of punishment that David is interested in is syntactic vinegar, like having to override a method instead of append to an array.

The problem with this approach is that punishment is only appropriate for children and criminals. By actively working to make my life as a hacker more difficult, David is treating me like a child or a criminal. Saying either that I am not old enough to know what I am doing and must be punished for doing it, or that I am wrong to think that way at all. Punishment is a very dangerous tool to use because it is fundamentally condescending.

Rails is David's baby, so he can do whatever he wants to it, and that is fine, but to treat a seasoned professional as a child only goes so far in my book. Some might argue that it is for the greater good... that it makes people who don't know what they are doing write better code, keeps people in their place. First of all, the people who end up being punished the most are the ones who want to push the envelope--a beginner is never going to try to add a keyword to the find method. Second of all, I don't want to give up my work productivity for the productivity of those who do not know what they are doing, I want a tool that works best for me. Every time I run into "opinionated software" in Rails, it makes the back of my ears burn reminding me of how much I can't wait to find, start, or help the next generation of highly productive web development software.

Opinionated software is fine until you want to do interesting, new, and innovative things with ease. I am not interested in being treated like a child or a criminal for doing so, which means those opinions which David holds most dear are the ones that will eventually drive out the people that Rails initially attracted... the hackers, the innovators, the bleeding edge developers. Give us freedom or give us another tool that provides freedom.

As a side note, apparently I may have been the first but certainly not the last to complain about the particular form of syntactic vinegar presented with validate_find_options because I see now that they ended up abstracting it and you can now add accepted keys by modifying the VALID_FIND_OPTIONS array. If you would like to see my failed attempt over a year ago to add this feature, check out this ticket.

You should follow me on twitter here.

Technoblog reader special: click here to get $10 off web hosting by FatCow!

37 Comments:

Anonymous Martin May said...

That is right, the keys were all hard coded, meaning that if I wanted to add my own :with key, I would have to completely overwrite the undocumented validate_find_options method, which inevitably would break things down the line as Rails added its own new keys.

Mmm...how about aliasing the method, and then wrapping it in your own method which adds the desired keys?

Something like this:

alias_method :old_validate_find_options, : validate_find_options

def new_validate_find_options(options)
options.assert_valid_keys([:where]) or validate_find_options(options)
end

alias_method :validate_find_options, :new_validate_find_options

2:09 PM, November 13, 2006

 
Blogger Lucas Carlson said...

That is incredibly ugly and quite convoluted. It is punishment even though it works.

2:15 PM, November 13, 2006

 
Anonymous Anonymous said...

the next generation of web development tools is in Python, and theres a wide variety of options. While we have a dictator as well, hes not nearly so micromanaging.

2:18 PM, November 13, 2006

 
Blogger jamie said...

I have come across this kind of problem on numerous occasions with Rails. I hate having to alias methods etc. as you inevitably end up in a big mess with aliased-aliased methods when things need extending further.

2:24 PM, November 13, 2006

 
Blogger Lucas Carlson said...

Saying that the next great framework is going to be in Python is like saying: "the next great novel is going to be written in Italian."

2:26 PM, November 13, 2006

 
Blogger Michael said...

First of all, ActiveRecord already supports passing a hash for your :conditions, though I don't remember as of what version. (I remember there being discussion about it on the rails-core mailing list a good while ago.) So this would work as is, I think:

User.find(:first, :conditions => {:active => 1})

Even if that feature didn't exist, aren't you just translating a hash into a condition string? If that's the case, why couldn't you just use the :conditions key instead? Is there something particularly with-like about the finding you're doing? If so, perhaps you ought not to go through "find"? Perhaps declaring a method named like "find" like "find_with_whatever" would be more appropriate? Then you have the advantage of keeping your stuff further separate from the core.

3:11 PM, November 13, 2006

 
Blogger Lucas Carlson said...

Michael, your comment is off topic, you are not addressing any of the issues I brought up. But for you edification, the example I used for this post was chosen because it is easy to see why it could be useful to add keys to the find method. It was chosen for illustrative purposes and was not what I happened to be trying to do at the time.

3:19 PM, November 13, 2006

 
Blogger Kevin said...

The next great web dev tool is going to be Grok. Or if not Grok specifically, any large, rich, mature framework in a dynamic language that then has a layer of rapid development Models n' Views wrapped around it. When you want to grow the complexity of the software beyond the rapid development phase it's good to have something there that manage additional complexity.

3:25 PM, November 13, 2006

 
Blogger Lucas Carlson said...

"large, rich, mature framework in a dynamic language"

1) Rails was not large, rich or mature when it started to take off

2) OCaml is not a dynamic language, but I would certainly not be surprised to see OCaml be the base of the next great web dev tool

3:32 PM, November 13, 2006

 
Anonymous Adam Jones said...

I fail to see why this kind of discouragement is necessary. If you don't want ActiveRecord to support :with, don't accept patches that implement it. It is that simple, and still allows those with motivation and capability to make it happen if they want/need it.

Not to be an annoying advocate, but this is one of the biggest reasons turbogears is my web framework of choice. Community participation is actively encouraged, and I have yet to see any code that is deliberately obtuse.

8:10 PM, November 13, 2006

 
Anonymous Mr eel said...

Personally I think the Rails team should be encouraging hacking of ActiveRecord where ever possible. My opinion of AR has dropped lower and lower as I've had to struggle with it's performance issues.

Allowing folks to hack the code will encourage them to find better solutions for the problems that it has — eager loading for example is awful awful awful.

IMO opinionated software is saying 'this is what we think is the right way'. It works because it limits the choices you can make. Rather than having to cook up your own way of doing something there is The Rails Way.

In this case I think DHH is wrong. Locking down the code isn't opinionated at all. If anything it undermines the current position of the Rail Core regarding patches. Make a plugin they say, while measures like this make it difficult to actually do so.

You're not encouraging people to make a better choice, you're just making it more difficult for them to probe and possibly improve Rails.

9:43 PM, November 13, 2006

 
Anonymous Anonymous said...

Try Django. Those boys have mastered loose coupling and abstraction. The 'pieces' from which the framework is built are all easy to extend or change, and it doesn't feel like you're hacking something rigid; it feels like you're extending something very pragmatic.

9:56 PM, November 13, 2006

 
Blogger Lucas Carlson said...

I have tried Django and am not a fan, but I will keep trying it and might change my mind at some point.

9:59 PM, November 13, 2006

 
Blogger Michael said...

It's more SPECIFIC than off-topic I think, and I apologize for taking your example so literally, though maybe it would better to use one that's not simply a rephrasing of an existing feature?

However, to restate my question, (the second part of my post) why bother mucking with Rails internals if you don't have to? Is it really necessary to add complexity to an existing feature when adding something beside it would do? This is like complaining that HTTP doesn't have a SITEMAP method or something. It's my opinion that core functionality should be kept as spare as possible, I guess.

10:14 PM, November 13, 2006

 
Blogger Lucas Carlson said...

why bother mucking with Rails internals if you don't have to?

Because I am a hacker and I want to. I have ideas that don't fit into a box, I look at what I am given and wonder how I can do more with it. For example, how would you build an entire multi-and-disparate-model website with a single database table? This would allow you to create new data types as data at any point. How might you associate disparate-table models with single-table models? What are the benefits and pitfalls of these types of systems... especially with regards to creating wikis.

I have not only thought about that problem but solved it with ActiveRecord and it took a lot of hacking. It would have been good clean and maintainable hacking too if it weren't for opinionated software.

But really, essentially, the answer to that question is the same as the answer to the more general question: why bother much with anything's internals if you don't have to? There are some people that are happy to play inside sandboxes and some that aren't. Some things like Unix encourage playing outside the sandbox, other things like Windows do not.

10:32 PM, November 13, 2006

 
Anonymous Anonymous said...

Because I am a hacker and I want to. I have ideas that don't fit into a box, I look at what I am given and wonder how I can do more with it. For example, how would you build an entire multi-and-disparate-model website with a single database table?

Then maybe the "here's a solution to a problem, but not every problem" approach in Rails means it definately isn't the platform for you.

build an entire multi-and-disparate-model website with a single database table. Hmmmm.... :)

10:47 PM, November 13, 2006

 
Blogger Michael said...

I guess I'd say I'm a hacker too, but I like my put my ideas in my own box so I know who to blame when something goes wrong. I'm referencing your original example to death, but the fact that it's difficult and error-prone to hack something seems like a clue that another approach is necessary, which could be as simple as using a different name. (Um, I'm gonna quote DHH here, but "if something isn't easy, you're not cheating enough.")

Maybe here's a question: why bother using ActiveRecord? If you have a need that it doesn't satisfy, maybe there's another ORM that can, or maybe you can write your models yourself. Rails doesn't force you to use ActiveRecord for your models.

My web-store, if it were not a static HTML page generated by Camping, could be a Rails app, but the model I use to represent products is a simple class that loads data from a single YAML file. Using ActiveRecord would be overkill in this case, since I don't need an interface for editing products, I just edit the YAML file.

When compared with the "multi-and-disparate-model" example you mention, that's the other end of the spectrum, I guess! It seems like there's a need out there for greater flexibility w/ models and such, but I don't think that's the aim of ActiveRecord.

11:14 PM, November 13, 2006

 
Blogger Lucas Carlson said...

michael, as I said, I have implemented it in ActiveRecord, and it has worked very well. Moreover, it could have been maintainable if it weren't for the syntactic vinegar in my way. If DHH hadn't deliberately made it harder to hack than necessary, for no other reason that because he didn't think there was a good use, I could have easily abstracted my work into a ActiveRecord plugin.

It seems like there's a need out there for greater flexibility w/ models and such, but I don't think that's the aim of ActiveRecord.

Of course it is not the aim of ActiveRecord, but I am telling you that if DHH's software wasn't opinionated, it would be a simple change that would not need to be maintained.

11:54 PM, November 13, 2006

 
Anonymous Gabor Farkas said...

> David is treating me like a child or a criminal

:)

as it's said in the python community:

"we're all consenting adults"

12:57 AM, November 14, 2006

 
Blogger Brandon Corfman said...

The main question I have is, how are you supposed to identify syntactic vinegar over lousy design? Notice that you had to have DHH *tell you* that it was syntactic vinegar, because you (as a uninformed hacker) couldn't identify it yourself.

6:42 AM, November 14, 2006

 
Anonymous Anonymous said...

Not to mentioned that the "opinionated" Rails framework is oriented to produce pre-IPO web applications, rather than "web sites". Any framework which requires a server per app has some serious myopia.

7:16 AM, November 14, 2006

 
Anonymous Anonymous said...

Your argument is fatally flawed. You refer to a disincentive as a "punishment", claim that punishment is only appropriate for criminals and children, and therefore conclude that you're being treated as a criminal or child.

The fact is disincentives between non-criminal adults occur all the time -- within a household, at the workplace, etc. Incentives and disincentives are part of the social fabric.

Now you may very well have a point about DHH and his opinions. But there's no need to play word games and detract from your point.

8:22 AM, November 14, 2006

 
Blogger Lucas Carlson said...

Notice that you had to have DHH *tell you* that it was syntactic vinegar, because you (as a uninformed hacker) couldn't identify it yourself

I thought that it was just poorly written code, a mistake. A mistake like that only becomes syntactic vinegar with intention and without any comments in the code, there is no way to glean intention until I try to fix it and the fix is refused.

9:02 AM, November 14, 2006

 
Blogger Lucas Carlson said...

Disincentives and punishment are cut from the same tree, the line is most certainly more vague and subjective than you describe it.

9:19 AM, November 14, 2006

 
Anonymous Anonymous said...

Have you tried Nitro? It's another Ruby web programming framework that seems to be philosophically different from Rails.

10:47 AM, November 14, 2006

 
Blogger Lucas Carlson said...

Nitro is too much like Rails, and not enough like Rails to hold my interest. It is too much like Rails in design and everyday use and not enough like Rails with regards to the featureset and community support. Nitro's offering is not compelling to me.

10:58 AM, November 14, 2006

 
Blogger zzzeek said...

hey lucas -

people have mentioned here other frameworks and languages disparate from rails and/or ruby, all of which would love to have new people on their project contributing new ideas, but you seem disinterested in them. You'd like to explore your own ideas about architecture and web development, and are coming up against the brick wall that is "ruby has only one viable framework, which is totally owned by DHH, who generally doesn't care what you think" (at least thats the impression I get every time i read a discussion thread with DHH...im sure hes a nicer guy in person).

I cant help but think about the sidebar on your blog, "I found Rails in September of 2004 and fell in love"...because this sounds like pre-breakup angst.

3:00 PM, November 14, 2006

 
Blogger Lucas Carlson said...

I have loved Macintosh for over a decade now, but I certainly have a list of things that piss me off. Macintosh is great but some parts make me very angry. I disdain the dock, I don't know why the GUI ftp support is read-only, etc. On a more abstract level I am pissed at Steve for deciding to close the source of Intel-based Darwin. All that said, you couldn't rip my Macbook Pro + OS X from my cold dead hands.

Regarding web frameworks, I am constantly on the lookout for the next best thing, the next set of tools to make my life easier. I am constantly reviewing web frameworks in search of something better for me and my purposes. On that level I have always, even in September of 2004, had pre-breakup angst. I am not going to defend Rails tooth and nail blindly, I am always ready to jump ship.

This article has less to do with my angst and more to do with my anger. Rails is really great, but some parts make me very angry.

3:15 PM, November 14, 2006

 
Anonymous Anonymous said...

Disincentives and punishment are cut from the same tree, the line is most certainly more vague and subjective than you describe it.

Uh, where did I describe it?

If DHH gives you a time out, spanks you, grounds you, says you can't watch TV or use the Internet, incarcerates you, makes you pick up trash on the side of the highway, fines you, etc., then you can claim to be punished.

But all that happened here is that you want things to be one way, DHH wants them to be another way, and you mistakenly think you've enhanced your argument by claiming to have been punished.

8:41 PM, November 14, 2006

 
Blogger Lucas Carlson said...

Punishment can be thought of as going out of ones way to cause harm to others in order to teach them they are wrong.

One way of punishing a child is by taking away their allowance, making sure that they do not receive as much money as they would otherwise. I work for a living and when it takes me longer than necessary to do my job, time is money. Since I am paid a salary, this means that my company has paid for me to spend more of my time than necessary to do my job. Since I have stock in my company, that means that by taking my time, it devalues my stock.

David takes his time and energy to knowingly program worse than he would otherwise in order to make people like me spend more time writing uglier code and more time maintaining that code than, thus causing me harm.

Knowingly writing bad code and refusing to accept patches to clean it up on various occasions by various community members, not based on time constraints but based on the philosophy that it should stay ugly to their my work harder than it need be is certainly a form of punishment in my book.

8:58 PM, November 14, 2006

 
Anonymous Zach Baker said...

This is a matter of feelings and taste, is it not? Surely I knew before that DHH is unafraid of hurting feelings and a fellow of very particular taste that people do not necessarily agree with. If his code and projects honestly reflect just that, I find it hard to consider it punishment or condescension. That implies a motive not in evidence here and so is perhaps being speculated upon unfairly.

1:51 PM, November 17, 2006

 
Blogger Lucas Carlson said...

The motive can be found both in DHH's own writings and my personal contact with him that re-enforce his writings.

1:57 PM, November 17, 2006

 
Anonymous Alex said...

Lucas, If you have an issue with Rails then write your own framework. I did four months ago for similar reasons as you mention, and I haven't looked back since.

I can hook you up with a guy that has built a framework container if you like, to make life a little easier. Also you may want to consider Og for the db layer. It's immature but highly hackable.

All the above in Ruby.

4:26 PM, November 20, 2006

 
Blogger Rob said...

Lucas, out of curiousity did you find the changeset where the change actually made it in?

11:19 AM, November 30, 2006

 
Blogger Lucas Carlson said...

I never did track it down, but svn blame should help for anyone curious.

4:53 PM, November 30, 2006

 
Anonymous carmen said...

the flaw you mention isn't fatal - the point of an Apple or rails-style vertically-integrated stack is so you dont need or want to change the internals..'Experts' have decided it all for you, and youre left to ogle your $2000 slab of chinese parts or write a few lines in TextMate.

opinionated defaults and easy customization/overrides aren't mutually exclusive, but as in the Apple case (how do you change the window-manager again? you mean the 3 checkboxes they decided to put in the GUI arent the ones you want and you have to drop down to a cmdline netinfo tool?), the main reason i switched off rails was i didnt agree with its opinions on the defaults, and replacing them was more trouble that it was worth.

take ERB. since markaby was 10x preferable to me, i figured it would be a straight mixin/include via plugins.

well, no. since rail's view rendering has J2EE levels of gatekeeping and abstraction, it was impossible to do things like pass a block to a template, without rewriting parts of ActionView. that, and calling a template recursively (to treeize some graph data into DOM nodeS) was causing bizarre errors that i couldn't diagnose. pasting the nonworking rails code into camping solved both these problems at once.

or take the opinion that Prototype is the way to go for JavaScript. you have all sorts of methods for generating dom nodes, that dont even correspond to the element names (eg link_to, instead of 'a', and weird argument order that you constantly have to refer to the docs to figure out), and the same is true for the javascript. or take the fact that generating javascript programaticaly from ruby at request time is unnecessary and a waste of resources, and RJS specifies a very specific design style (shuttling over ruby-generated scripts to be executed client-side) along with the inlining of unreadable generated javascript on element 'onclick' attributes and similar, requires you to seek out 3rd party unobtrusive plugins where theres a chance they will break any time, and even sooner if youre using something besides ERB, or the way the new CRUD/REST magic specifies as very specific way of structuring your URLs (and even that you shouldnt use accept-headers and instead changing the URI itself with a non-query-string format extension) and if youre writing javascript and SQL in ruby, why not HTML and CSS?. and what about the dichotomy of having to specify the model in 2 places - your model code, and your migration code..

overall it was this kind of lack of loose coupling and layers of assumption that led me away from rails. by the time i had replaced with ActiveRecord with my own RDF ORM, ERB with Markaby, Prototype with JQuery, and their REST routes with my own (which incidentally updates to their routes code broke, and my bug report was closed saying i should rewrite my routes), and struggled to fix obscure scoping bugs with already-fragile include/alias nonsense, it seemed the only 'opinion' of rails left affecting me was the fact that i had to have said hoop-jumping in the first place to get it how i want.

for the record, i do believe the framework i built my 'no-code' app-development framework in benefited from the fact that i can read its entire source code without even hitting the pageup/pagedown key..in terms of developer grief and refactoring simplicify.

2:11 PM, February 06, 2007

 
Blogger micko said...

his post provides the light in which we can observe the reality. That is very nice one and provides in-depth information.sizegenetics discount

11:09 PM, March 14, 2014

 

Post a Comment

Subscribe to Post Comments [Atom]

<< Home

 

If you like this blog, you might also like top photography schools.