Monday, December 14, 2009

Why Zope won't Zip

I really wanted to like Zope. So much, in fact, that I had one of my developers spend nearly three weeks preparing a case for why it was the proper platform for a new product we are creating. Imagine my dissapointment when three weeks later, this developer told me his recommendation was to not use Zope for our project.

No. That's not what I asked for. But I learned a long time ago to trust the people I've hired, and I wasn't going to start changing that now. But I did ask a lot of questions.

No surprise - since we hire such smart cookies - this developer had lots of great answers. In the end, it came down to two primary reasons. The first was ZODB. I realize that Zope really isn't all that much without ZODB, and that very fact is what nixed it for me.

We create Enterprise software. This means that we need clearly defined install, backup, migration, and validation steps for our software and data. We also need lots of our data in a tool that supports easy reporting: typically a SQL database. As soon as we have some of our data in ZODB (because Zope won't let you store certain things anywhere else) and some of our data in SQL, things have just gotten ugly.

Strike one.

The other major problem is the CPython GIL (Global Interpreter Lock) problem. In a nutshell, the typical Python interpreter only allows a single python thread to execute in the interpreter at any time. There are lots of pro-Python sites that will show you examples of how single-threaded execution is hyper-fast and that the GIL isn't a problem. However, on an eight-core server that is running multiple simultaneous web-sessions, having a single-thread of execution is like using only one-eighth of your expensive server. Sure it's not truly that bad because of I/O locks and other things that release the GIL, but it's nowhere close to what it should be.

In short, the traditional Python interpreter is a terrible implementation for multi-core servers. Given that this now includes pretty much every server out there, there is no way to justify using the current Python interpreter to create a server process like Zope. It's just wrong. Before long, a typical server will have 16 or even 32 cores. Pretty soon, people depending on Zope are going to start asking a lot of questions about why their killer new server doesn't seem very fast.

Stikes two and three. (Just not at the same time on two different cores.)

So, in a nutshell, three weeks of developer research uncovered two critical elements of Zope that have dashed any hopes of using it as the platform for a new server application. That's too bad, because at a high-level Zope has all the components we really needed.

So what have we decided to use? We're pretty sure it's going to involve Groovy. And quite possibly Grails. We've got another two-week spike planned for first thing in January. I'll update things here once that research is complete.

3 comments:

mosabua said...

Grails won't disappoint you ;-)

Ken Wasetis said...

I would think that if you develop 'Enterprise' software, then you'd also be running your prototypes/experiments in a clustered environment, whether you're evaluating Zope, Java/J2EE, or anything else you need to have really scale.

Since Zope has shipped with ZEO (Zope Enterprise Objects) and the ability to run numerous Zope clients (application server instances, where your business logic runs - the equivalent of your Pythonic JBoss, if you will), each taking advantage of one of your number of processors, this allows you to leverage your CPU to the utmost and you surely would be clustering if you truly are building Enterprise software, with needs such as fail-over and redundancy (not having a single-point-of-failure.)

Did I mention that Zope has shipped with this capability for over 10 years and that it doesn't cost a dime?

We've implemented truly Enterprise Zope deployments for large federal agency customers as well as entertainment/media sites, handling millions of web application visits (running the Plone CMS on top of Zope) for busy websites that have sporting events on CBS TV every weekend during the Winter/Spring. We do truly 'Enterprisy' things such as implement a load balancer, multiple servers, and multiple instances of our (Zope) application server. It works great and has for years!

As for the ZODB, did you ever stop to ask why you need a relational database? I'm not going to push an object-oriented database down anyone's throats, but I will point to the quite active trend toward ORM models in recent years (your folks are surely familiar with Hibernate on the Java side.)

Have you ever thought about the extra overhead of having this ORM (Object-Relational Mapping) layer that maps your nice object-oriented code to your relational data model?

Have you thought about the fact that much of the time, the performance bottleneck in an enterprise application is between the application server and the database?

Do you care about security at all and if so, do you realize that SQL Injection is one of the more common areas of breach for enterprise applications?

The ZODB addresses both these performance and security-related issues quite well, in my opinion, but you don't have to believe me on that. Zope is on the approved list of open source apps at the Department of Defense, and the Plone CMS that we run on top of it has been deemed the most secure CMS in the market (see details at http://cve.mitre.org.)

In any event, I don't doubt that you've hired bright people, but there are a lot of good things about Zope that it seems you're ditching for flawed reasoning.

Would I rather have Zope without the GIL, sure - but really only for NON-Enterprise clients/needs (where quad- or 8-core servers aren't in place), since I would be running multiple instances of Zope for my Enterprise clients anyhow.

I hope you give your smart team some additional things to review before you come to a final decision.

Best of success!

Cliff McCollum said...

Ken, those are all excellent points. However, we weren't looking at Zope because we needed ANY of the things you identified. We have ALL of those things already with JBoss, Java, Groovy, etc.

We were looking at Zope to help us build those kinds of applications faster and easier than we already can. In the end, there just wasn't enough better about Zope to make us leave the JVM environment behind.