The migration from DB4O to NeoDatis was very straightforward, the only difference I had to figure out was on querying objects by class:

//db4o version:
ObjectContainer db = Db4o.openFile("CARS.DAT");
ObjectSet results = db.queryByExample(Car.class);
//returns also objects Cabrio, Sedan, Hatchback, ...

//NeoDatis version
ODB odb = ODBFactory.open("CARS.DAT");
CriteriaQuery query = new CriteriaQuery(Car.class);
query.setPolymorphic(true);	//in order to return the subclasses of Car
Objects results = odb.getObjects(query);
//because simple odb.getObjects(Car.class) doesn't return the subclasses of Car,
//it needs to be explicitly set as polymorphic query.

There are probably more differences than that, and even deeper ones when it comes to features, robustness, reliability, support, but I am just trying to find the right tool for my work, not one hammer for everything.
Speaking of which, I am getting more and more convinced that we might be rushing to use the relational databases a bit too much, even when it is not necessary or not really appropriate. The OODBs for storing objects is one alternative option, Amazon SimpleDB, Apache Hadoop, CouchDB or Google’s BigTable are other data storage options with different benefits and applicability.

Let’s imagine a hypothetical system that monitors a building, which generates an event (ZoneEnteredEvent, ZoneAbandonedEvent) whenever a person enters or leaves a zone (by swiping a card, RFID, face recognition, whatever). The eventProcessor definition looks like this:

<bean id="eventProcessor" class="com.nailsandhammers.espring.processor.EventProcessor">
<property name="statements">
	<list>
			<ref bean="tooMuchExposureStatement" />
			<ref bean="duplicateEntryStatement" />
		</list>
	</property>
</bean>

When the eventProcessor bean is instantiated by the application it also instantiates all objects defined in the statements list.

The statements are then defined like this:

<bean id="tooMuchExposureStatement" 
class="com.nailsandhammers.espring.statement.TooMuchExposureStatement">
<property name="expressionText"
value="select * from pattern[every s=ZoneEnteredEvent -> (timer:interval(30 minutes) and not f=ZoneAbandonedEvent(s.subjectId=f.subjectId and s.zoneId=f.zoneId and s.zoneId='A'))]" />
</bean>
<bean id="duplicateEntryStatement" 
class="com.nailsandhammers.espring.statement.DuplicateEntryStatement">
<property name="expressionText"
value="select * from pattern[every s=ZoneEnteredEvent -> (f=ZoneEnteredEvent(s.subjectId=f.subjectId and s.zoneId!=f.zoneId)) and not a=ZoneAbandonedEvent(s.subjectId=a.subjectId and s.zoneId=a.zoneId)]" />
</bean>

The first statement tooMuchExposureStatement ensures that nobody stays in the Zone A longer than 30 minutes (let’s say it’s dangerous there).
The second statement duplicateEntryStatement controls that somebody entered a Zone, they cannot enter another Zone without abandoning the current one. Otherwise the person is clearly in two places at the same time, or it means somebody’s identity has been compromised, or the system is not working correctly. In any case, it is something to be looked at.

The statement could be any object that implements com.espertech.esper.client.UpdateListener and of course before the eventProcessor starts accepting any kind of events, it needs to make sure the statements get registered with the Esper CEP engine. In order to make the statements expressions configurable as well I created an abstract superclass EsperStatement that contains the expressionText and the register method that adds the statement to the Esper engine:

public abstract class EsperStatement implements UpdateListener {
    private String expressionText;

    public EsperStatement() {
        super();
    }

    public void register(EPServiceProvider serviceProvider) {
        EPStatement epStmt = serviceProvider.getEPAdministrator().createEPL(
                expressionText);
        epStmt.addListener(this);
    }

...

}

When eventProcessor starts up, it needs to initialize the Esper engine (see the documentation and samples) and register the statements:

public class EventProcessor {
...
	EPServiceProvider serviceProvider;
	private void registerEsperStatements() {
		if (statements == null || statements.size() == 0)
			return;
		serviceProvider = initServiceProvider();
		for (int i = 0; i < statements.size(); i++) {
			EsperStatement stmt = (EsperStatement) statements.get(i);
			stmt.register(serviceProvider);
		}
	}
...
}

So in the end the statements and the expressions that are used to process events are configurable components, that can be added, removed and changed. With a little bit more coding it would be possible to force the eventProcessor to reload the config file and reinitialize the CEP engine with the changed statements without even restarting the application process.
Happy experimenting with CEP and Esper!

Of course there are several refactoring solutions that might help,  but Gerard Meszaros in his xUnit Test Patterns describes another way how to deal with this situation – a TestSpy pattern. TestSpy is a class that I can ‘inject’ to the tested class in order to instrument and capture its behaviour. I used the TestSpy to persist objects that were being created in the tested method so I can verify them later.

I had wide number of choices how to implement the persistence – using a file, a regular sql database, embedded database like HSQLDB or H2.  I really wanted something simple, no extra server administration, table creation or  O/R mapping definition. I wanted to store and retrieve objects, so I decided to give it a try with an embedded object-oriented database . I chose db4o for no particular reason except that it seems to be pretty well established as a product and is coming with an open source license (GPL, though).

Despite having not much of former exposure to db4o, it turned out pretty quick and simple. I followed the tutorial and I had the whole testing up and running in about 30 minutes.

Here is the ‘TestSpy’ class:

public class TestStore {
	private static String dbFileName = "DB4O.DATA";
	private ObjectContainer	db;

	public TestStore() {
		db = Db4o.openFile(dbFileName);
	}

	public void close() {
		db.close();
	}

	// store object to the database
	public void store(Object dataObj) {
		db.store(dataObj);
	}

	// retrieve stored objects using an example object
	public List queryByExample(Object example) {
		ObjectSet resultSet = db.queryByExample(example);
		return resultSet;
	}

	// retrieve stored objects of the same class
	public List queryByClass(Class clazz) {
		ObjectSet resultSet = db.queryByExample(clazz);
		return resultSet;
	}
}

That’s it, that’s all the db4o-specific code I needed.

And here is the tested method:

public class EmailSender {

	// send emails to all recipients using the NewYearGreeting email template
	public void sendNewYearGreetings(String[] recipients) {
		for(int i=0; i<recipients.length; i++) {
			NewYearGreetingEmail email = new NewYearGreetingEmail(recipients[i]);
			smtpServer.send(email);

			//testability support
			if(AppConfig.ALLOW_TEST_SPY) {
				testSpy.store(email);
			}
		}
	}

	private TestStore testSpy;
	//set the TestSpy
	public void setTestSpy(TestStore testStore) {
		testSpy = testStore;
	}
}

Finally, here the testing and verification code from the TestCase:

public void testSendNewYearGreetings() {
	try {
		String[] recipients = new String[]{"joe's_addr", "bob's_addr",
								"ann's_addr"};
		TestStore testStore = new TestStore();
		EmailSender emailSender = new EmailSender();
		emailSender.setTestSpy(testStore);
		emailSender.sendNewYearGreetings(recipients);

		List resultSet = testStore.queryByClass(NewYearGreetingEmail.class);
		if(resultSet.size()!=3)
			fail("Unexpected number NewYearGreetingEmail objects!");
		//TODO - add further verification
	}
	catch(Exception ex) {
		fail("testSendNewYearGreetings Failed");
	}
}

Of course, this testing method changes the tested code somewhat. As usual in this world, there is no free lunch and adding more testability to the code introduces some features that may need to be removed in production. If this is an issue, then you can use some of the ‘conditional compilation’ techniques :

• use ant to remove the test code from the production code as described for example here.
• you can even use the gcc preprocessor to have ‘real’ conditional compilation as described here.
• or you can use a static final variable that drives the execution of the test logic. This feature takes the advantage of the fact that the compilers do not generate unreachable code, so in fact your class would not even contain calls to the Test Spy methods. See the AppConfig.ALLOW_TEST_SPY variable being used in the code above.

In this case the OODB did exactly what I needed – allows me to store and retrieve my objects without too much dancing around. Pretty clean and easy to set up. I will see if this is the right hammer for the testing nail.

The book shows some interesting what-could-go-wrong scenarios,  and outlines some solutions and patterns how to address them, but the main value is that it illustrates how apparently unrelated things could just come together and bring the whole system down.  Check the recent Attack of Self-Denial post on Michael Nygard’s blog.

Some of these system failures cause significant damage to company reputation, or a hard economical impact, but it makes me think how much of other software is running out there, controlling nuclear plants, medical care systems, military systems, air-traffic control, government. These systems are still developed by people and what I am asking is – do they have some kind of a secret recipe how to create reliable, available and secure systems? If yes, why not to share it with others? And if not, then maybe there is a reason to be scared!

Of course there isn’t a simple guideline how to ‘create a system that never fails’, but there certainly are practices that lead to better, more reliable software that at least fails less often. To get started, I would certainly recommend to read Michael Nygard’s Release It book. Check out the next blog entry!

Maybe it’s because at the end of the day it’s the people that get to use those technologies like tools to build things, and as people say: If the only tool you have is a hammer, the next thing you are going to hit might be your fingernail!