Categories
computer

5 Roadblocks To Enterprise Rails Acceptance

rails.pngI love Rails for its pragmatic design and agile culture: two qualities not usually associated with the large, enterprisey systems of Fortune 500 companies. In my last formal position I was part of a small internal movement to drive the Rails train upward through the IT ranks, but the effort was met with limited success. The unfortunately reality is that Rails currently lacks several key qualities to which enterprise project leaders have become accustomed. Here are five reasons of varying significance to start us off.

Insane Query Support

Most documentation you read about ActiveRecord will take you through tidy, minimalistic examples which are squeaky clean and really fast. Complex queries, however, will be easier to do using Model.find_by_sql, which accepts a raw SQL query. Ordinary dynamic finds with deep loading behavior may require you to hard-code names in the query to avoid issues with the generated SQL. ActiveRecord is way easier to use, but far from Hibernate. I’d say that over 95% of the queries issued by a larger application are of trivial or medium complexity, but a lot of time and your best developers go into that last 5%, and this is where the heavier OR/M frameworks start looking better than ActiveRecord.

Distributed Transactions

The rise in SOA interest over the last couple years has led to more applications using multiple data sources. While it is possible to nest transactions, “Rails doesn’t support distributed two-phase commits (which is the jargon term for the protocol that lets databases synchronize with each other).” (From Agile Development with Rails, 2nd Edition.) In many situations, simply nesting transactions will suffice; however, many situations should really have the safely and reliability of two-phase semantics, and this factor alone could be a deal breaker.

Data Integrity

Database Designers (DBDs) like FOREIGN KEY constraints, CHECKs, high levels of normalization, and are the natural enemy of null fields. In other words, DBDs don’t like Rails. While I’m certainly no Pedantic Data Nazi (PDN?), there should at least be a basic set of built-in mechanisms for generating such simple self-defenses against naughty applications. Frankly I’m surprised that the community isn’t pushing harder for solid constraint support within migrations.

IDEs

This isn’t technically an issue with Rails itself, but a roadblock to its adoption nonetheless. Most Rails developers (including myself) appear to be using TextMate. A smaller population use RDT, Emacs, or numerous other packages. But there isn’t yet an application which comes close to the basic core feature of the popular Java and .Net IDEs. The currently broken breakpointer is another swift kick in the pants. What I can do with Eclipse on a remote application server isn’t in the same universe of functionality as the Rails breakpointer, even when it worked.

Top-Down Push

For whatever reason, CTOs and CIOs haven’t yet become seriously interested in Rails, and without this air of implicit exploratory approval, managers seem reluctant to give in to antsy developers. I would love to see Rails become a flagship of agile enterprise projects, but that’s not going to happen until management sees the real ROI of a project done by experienced Rails developers.

None of these things are insurmountable, but there are many more challenges to overcome if Rails will ever sit on the same application servers as Java and .Net. What challenges have you faced with Rails at your organization?

17 replies on “5 Roadblocks To Enterprise Rails Acceptance”

First of all I hope that rails never gets accepted by the enterprise. The enterprise should stick with java and .net and leave ruby, python, and whatever else we are playing with alone. We don’t need all your bloat for our apps.

Having said that….

What’s wrong with find_by_sql?

What’s wrong with putting data integrity in the database (where it belongs).

What’s wrong with writing stored procs or views for the 5% of the complex queries?

As for IDEs try radrails, eclipse dynamic language toolkit, komodo, arachno, freeride, mondrian or whatever. I prefer jedit for all my ruby coding BTW.

Distributed queries? Again don’t really care all that much.

I’m a little confused about a few things
Your point on foreign key constraints in the database. Frankly I think all databases should pass the ACID test(you never know what app will be accessing that data in the future), and I write all my databases for Rails apps accordingly(in Postgres of course).
What exactly is wrong with writing an execute like:
execute “ALTER TABLE products ADD CONSTRAINT products_category_id_fkey FOREIGN KEY(category_id) REFERENCES categories(id) ON DELETE CASCADE”
in your migration?

While Rails encourages convention over configuration, it is not all that much harder to configure it.

Also I do agree with the the above comment re: views in the database, if you’ve got a query that is that complex, don’t ask any app to do it, let the database does what it does best….. An that’s deal with data.

I just feel that your giving enterprise even more ammo, if you feel that rails has problems(which I agree in part it does), then write solutions.
It’s open source and easily extendable, write a gem or a plugin.

I work at a growing company that works with a variety of technologies, both open-sourced and shrink-wrapped. At this point, the shiny new object is .NET, either with ASP.NET or WinForms. Therefore, it is difficult to introduce something like Rails when people are still getting their feet wet with .NET. Moreover, to handle our move to .NET, we are basically only hiring Windows/.NET engineers these days.

Other than that, here are other roadblocks that I see at my job:
– Warm body theory => If a project is running late, leaders need to be able to throw warm bodies at it (where have I heard that before?). It is hard to throw warm bodies at a rails project who only know .NET.
– Windows Integrated Authentication => most of our employees use IE6 on Windows workstations and we can seemlessly track who, what, when, etc. from our applications…Big Brother is watching.
– Separation of Duties => When a company grows, they begin to have different departments and put in processes to slow down development in the name of safety and security. Therefore, we are unable to take advantage of things like capistrano deployment. Also, hardware engineers already support a number of environments, so adding rails with mongrel, lighttpd, or apache fastcgi would be met with some “push back.”

With all of this, I am still able to use rails as a prototyping tool on projects where I am the only developer. Then, after developing the prototype, it is fairly straight forward to convert it to an ASP.NET app with the help of other engineers. This is not the ideal situation, but I’ll take it for now.

cam,

Nothing is functionally wrong with manually creating ALTER statements. I actually do that via helper methods when I need/want them. But it diminishes much of the purpose of having an abstraction layer if every time you want to adjust a constraint you have to revert back to raw SQL (which your database understands). And if you do this long enough–which happens in even small projects–your migrations become a mix of clean abstract Ruby code, and fugly looking SQL.

Pointing out these things certainly shouldn’t be viewed as “ammo”. Everybody needs to understand them to make the best choices for their project and company. A bad opinion formed by a failed or perceived inadequate Rails project is longer lasting than a decision to hold off on trying Rails.

Mal,

Let me clarify that when I guestimted >95%, that would be for a typical Java project. But when using Rails I’ve found that this number is much lower, and I’m forced to thinking about the *relational* model far more than an application developer should be. Remember that designing a database from scratch is a luxury many developers don’t have 🙂 I also believe in leveraging database features, but my first attempt is usually to try something at the application layer, especially if I’ll need to manipulate the objects in code and would like to use the lazy-loading of already present models.

Yes, I am aware of the various other IDEish package out there. I’ve experimented with FreeRIDE and Komodo in particular, and have looked over several of the others you mention, but I stand by my point that none of these come close to the power of the popular IDEs in The Other Languges.

I don’t understand the comment, “What’s wrong with putting data integrity in the database (where it belongs).”

“but I stand by my point that none of these come close to the power of the popular IDEs in The Other Languges.”

Lots of people seem to be able to build pretty complex rails web sites using nothing but textmate. As I said jedit does the job for me. I honestly don’t understand what you hope to get from an IDE that radrails does not provide. I gather you are a windows programmer if so you should look into ruby in steel which is a visual studio add on for ruby. If eclipse or visual studio don’t count as a “real” IDE for you then you are just trolling.

“I don’t understand the comment, “What’s wrong with putting data integrity in the database (where it belongs).””

Simple. Data integrity rules belong in the database. Foreign keys, referential integrity etc don’t belong in your application.

As for your other point about existing schemas you should learn a little more about ruby and activerecord. There is nothing you can’t do with it, it just becomes more and more painful as you deviate further and further from the rails way. If your database schema is massively different then the way rails thinks it should be then it will be a lot of work to make it work with rails. Although it will work it will not be fun and it’s probably not worth the effort.

If you have a massive legacy schema then you should not use rails. You can use other Ruby toolkits and you can use Ibatis http://ibatis.apache.org/ for the ORM.

If that doesn’t work you should use java because no other platform has more robust ORM layers then java.

Dude.. either I’m not being clear enough or you’re just not paying attention. I’m an old Eclipse plug-in hacker on OS X and Linux, and know what I’m looking for in an IDE. You could build your site by carving bits of UTF-8 into marble, but once you’ve used more powerful tools such as Eclipse it’s hard to settle for less. Why would you think I don’t like Eclipse? The fact that you put jEdit on the same playing field as Eclipse is downright laughable.

Your point about referential integrity being a database concern is exactly what I said. “.. there should at least be a basic set of built-in mechanisms for *generating* such simple self-defenses against naughty applications.” Why on Earth would foreign key constraint SQL generation methods not be appropriate in a migration? That’s what they’re here for.

I know existing schemas can be supported. I’ve done it, and never claimed you can’t. I agree that it becomes more painful to use Rails in complicated legacy situations, which would be another thing on the list of Rails roadblocks from an “enterprise” perspective.

how about the lack of a governing body? the community left to its own doesn’t unite.

That’s an interesting point. I don’t read all the RoR Core emails, but have generally been impressed with their progress. I do wonder, however, what the future holds for the maintenance of non-core gems and plug-ins. (salted_login_generator for example.)

Just a note on the IDE front, Netbeans is a fantastic Ruby IDE (in development, but still, fantastic.)

Also, Intellij Idea is supporting Ruby… so two of the biggest Java “Enterprise” IDE’s support ruby/rails 🙂

I mostly disagree with this article. My disagreement is probably just part of the continuing miscommunication about what “Enterprise” is supposed to mean. We’re finishing up the 1.0 (i.e., the version we move the organization to) of a health record application written in Rails. Is it an “enterprise” app, or an app “for the Enterprise”? I don’t know, but I would guess, “yes”: we have 1100+ users in-house spread over ~150 separate locations, across the majority of a state (TN). We deal with insurance and governmental reporting and auditing requirements, handling roughly 40,000 medical clients a year (and growing). We will deploy to 6 other similar organizations in the next year, some of whom we will host and some of whom will self-host. We have decades of existing data to manage and convert. We had/have to manage 45 separate “stakeholder” groups with domain knowledge and concerns. I think this is “enterprise” by any but the most common definition (that being “completely screwed organization that likes to say ‘you can’t service a company as screwed up as ours'”).

We’ve been using foreign keys in migrations for almost a year now, so we open sourced them a long time ago:

http://svn.caboo.se/plugins/svn/atmos/activerecord_foreign_key_extensions

We also do ridiculous auditing via our migrations, with support in migrations for table changes. By auditing I mean tracking all creation/update information on tables, including roll-off tables. This is a bit trickier, but, still, not rocket science. We need this for HIPAA, SOX, and clinical trials requirements for data integrity. We declare the triggers for auditing in migration 001 on a per-database level (we have subdirs for each database type we support). We then just add, “:audit => audit_level#” in our table creation migrations. We currently run the entire application on two different database engines, PostgreSQL and Oracle, and have support for SQLServer except for the last bits of the auditing support.

In this application with 188 models we have exactly one (1) find_by_sql, and that is simply to speed up a query that we could probably index better and get without the find_by_sql. This is a system that includes a complete Accounts Receivable engine (including medical claims processing) in addition to all the clinical functionality, etc. We run separate transactional and reporting databases and will build out a data warehouse for clinical data in the coming year.

We had so-called “top-down support” for Java from day one (2 years back) and started with that as our implementation language, using JBoss (and some now-deprecated Java frameworks), which was ultimately worth dick-all. We could choose to have C*O support for our choices and burn through $10M to produce some Java slag or we could produce a working application for cheap and educate our C*O’s on why we wanted to choose what we ultimately chose. If C*O Top-Down push is a problem then I would argue that the definition of “enterprise” is the “common” one cited above. Why? Because, if C*O’s are deciding your implementation technologies without understanding what is at stake, then you’re dead before you start. Or maybe they just don’t trust their technical staff, and perhaps that’s for good reason — but then you’re just screwed for other reasons, because your C*O’s won’t address the core problem (incompetent or inappropriate staff).

Are distributed transactions a killer for your “enterprise”? It could be possible. Every instance I’ve ever seen of this requirement, however, has been a symptom of systemic bad decision making.

I worked for what is arguably the most successful hedge fund on the planet, who at various points in time have accounted for over 10% of daily trade volume on the NASDAQ (and huge volumes elsewhere), and they have never needed distributed transactions. I am familiar with the operations of more than one major credit card transaction processor (hundreds of thousands to millions of transactions per day) who also do not do distributed transactions. There are plenty of other examples of which I am aware and they all point in the same direction — distributed transactions, according to people who are successful in large-volume environments, are not where you want to be.

Is it possible your enterprise is somehow special though and needs distributed transaction when those guys do not? Possible, yes, but more likely ignorance has won the day. Call up your vendor and ask them what’s “The Next SOA” so you can go ahead and throw your money away in advance and simplify your ledger sheet. I’m just saying.

As for IDE’s, you’re correct, most people seem to be using TextMate (I do, personally and love it), which doesn’t have the number of function points you’ll find in other IDEs. In fact, TextMate bills itself as an editor, though it’s a poweful one, and is hard to distinguish from an IDE once you learn some of the thousands of things under the hood. There are some folks who find other tools useful, but, probably the core of the matter is that the things that folks do with .NET IDEs and Java IDEs are not the same sorts of things that people would do with Ruby IDEs.

Now, in one sense there are probably some failings as it would be nice to have more refactorings in Ruby tools, but I think a larger part of the issue is that it’s yet another case of YAGNI. When I started learning Ruby back in 2000 I kept looking for the right way to do “destructors” or “finalizers”. It turns out there is a way to do them, though they are pretty nondeterministic due to the GC system. After playing with them a bit I never used them again. Looking back, I’ve never had a desire since that “well, how can you possibly not use a destructor?” moment. There are lots of similar fears that people have coming into Ruby and Rails use from outside. Most of them seem to be “all other things being equal, how can you survive without … ” fears. The thing is, nearly nothing is equal, much less all things, and what you think you’re gonna need — you ain’t gonna need it. The same holds true (in my opinion) for IDE’s, which is why you see people switching out of Eclipse when they come in from Javaland, picking up textmate and being *happier*. It’s not that TextMate is “better” than Eclipse, but, overall package, lots of developers are assloads happier using TextMate to write Rails than anything else to write Java or .NET.

Rick,

Thank you for sharing your experience. It’s very encouraging to hear! I really wish I’d had the same eases.

I’d like to know more about your companies infrastructure. Every larger company I’ve been exposed to has had an slew of various databases, usually more fragmented than desirable across divisions/business units and various other teams asserting “ownership” simultaneously, at the cost of the organization as a whole. Regardless of whether or not this correlates to bad decision making, it’s a reality of what large organizations face. And when you’re dealing with fragmentation issues, you have to worry about semantic data corruption due to the data duplication stupidity across systems. It sucks, but such is the beast. SOA I feel is in part a reaction to fragmentation by aggregation; a way of not really solving the core issues but giving people a single team to come to for information, thus masking a fragmented, ugly underbelly. I’m in complete agreement that it sucks to support an application with multiple data sources. It can be a horrible nightmare where *nobody* wants to be, but again, I’m not arguing for it.. just pointing out that it’s where many organizations are, and it’s not a strength of Rails to handle this type of environment.

Also note that I tend to work for technology companies aware of current trends and market status, and even if upper-level management can only influence the ranks by setting goals, yearly priorities such as “stabilize current platforms over the next year” can prevent significant technological innovation atop the data of “current platforms”.

On a technical level, I’m surprised that your application gets by with so few find_by_sql queries. Do you have performance/responsiveness requirements? Reports and other screens which deal in data from a large number of tables seem to have needed much ActiveRecord TLC in my experience.

[…] Numa empresa grande além de tudo aquilo que se faz na empresa pequena é preciso de números, a coisa aqui complica camarada. Quem conhece bem o Rails sabe que faltam várias coisas para que ele se torne uma aplicação de nível “enterprise”. As grandes empresas geralmente estão presas nas tecnologias mais usadas no mercado e querem correr o mínimo de risco (isso é meio contraditório, pois hoje em dia você tem que inovar se quiser sair na frente). Se o projeto em questão é algo novo e experimental (algo não-crítico para a empresa), fica mais fácil você usar o argumento da inovação e importância de se estudar tecnologias novas. […]

Leave a Reply

Your email address will not be published. Required fields are marked *