Monthly Archives: February 2014

If you pay the piper …

Posted on by

…you can choose any filter implementation you like.

Thanks a huge lot to Steve Bates for his excellent blog series about pipes and filters:

It was interesting to follow his thoughts and the comments of all the readers. Mostly I found myself in agreement with everyone while reading it.

Some time later, I attempted my own pipe implementation  – and ended up somewhere else entirely. Not more object-oriented but almost completely without them!

It’s also at least as clean as any code I’ve seen so far and manages the whole composition thing in about 20 lines of code. I’ll come to the details but first a few comments on Steve’s postings and their comments:

His first implementation put the whole work of the filter into the constructor of the filter. Filter execution looked like

new Filter(Message);

And that was it. One object creation and construction per filter invocation. And like most other readers there I shook my head over it. What was this supposed to be? A training exercise for the garbage collector?

Well, as it turns out, Steve wasn’t the only one who went for short-lived objects. One of the more prominent examples is the Windows Communication Foundation’s per call instance context. But there’s more:

  • A type doesn’t use up any resources. It is completely compile-time and just sits around being there.
  • It exists or doesn’t. And if it doesn’t the compiler will tell you in no uncertain terms. Great for development.
  • If the pipe creates the object it can be totally sure that  no one else holds a reference to it. In multithreaded scenarios this makes things a lot easier. Scalability as well, see?
  • The pipe imposes no requirements about resource lifetime, references and so on on the filter beyond the absolute minimum that the filter must be in working condition once and for one time, right after it’s been created. No checking of whether any resource references are stale, depended on objects still there or, worse, objects may have been gotten rid of by the application and the filter may be holding on to an object that would have been  collected if not for the filters reference to it.
  • The pipeline doesn’t force a filter implementation that has a filter deal with state. Yes, many filters will modify application state, but but filter state is not application state and they should use some kind of repository for it. The pipeline does not have any state requirement.  Having a filter have a significant lifetime always creates the temptation to make assumptions about it (just one pipe, no concurrency are two examples) and use that a an excuse for keeping state in it. That can and will go wrong sooner or later.

So, on third glance, I think his idea isn’t half as bad as it looked at first. Still, when trying to implement this in a generic way, I bit on granite, much like some commentors on Steve’s postings. In an ideal world we would be able to write:

public void RunFilter<TFilter, TMessage>(TMessage pMsg)
	where TFilter: new(TMessage)
	{ new TFilter(pMsg); }

But Microsoft won’t let us. So if I wanted to use the type-as-a-filter principle I had to use an interface and a method too:

public interface IFilter<TMessage> { void ActOn(TMessage pMsg); }

I can’t say I like it very much. Because the filter isn’t supposed to keep state, it’s not required to be an object and therefore ought to be gotten rid of. The only reason I ended up with an object is because I wanted the type, not because I wanted to read or write properties or do any “objecty” things with it.

So, what does .net have to offer? Delegates. And this is where it gets to be fun:

A filter is supposed to do something with a message, so the minimum we need is

Action<TMessage>

and a way to turn the filter into a delegate. No problem here:

public Action<TMessage> FromType<TFilter, TMessage>()
	where TFilter : IFilter<TMessage>, new()
	{ return pMsg => new TFilter().ActOn(pMsg); }

does the job just fine. Nothing else required. No object hanging around for three weeks until someone finally gets around to connect to the service and run the filter. So far so good, but, as others have already pointed out, a single filter isn’t much. But now that I got an action, composition is not a problem. Here we go:

  • Sequences. This is the original “pipeline”: Something that takes a bunch of actions and turns it into one action which executes them all one by one. Sounds easy? Is easy: 
    public Action<TMessage> Sequence(
	params Action<TMessage>[] pSequence)
	{ return pMsg => Array.ForEach(pSequence, pI => pI(pMsg)); }

    And we’re done.

  • Branches. A filter that should execute one action when some predicate is true and another when it is false. Sounds ok, and there were some suggestions along this line. Only – should I let some stupid enum dictate the number of choices I am allowed to make? An outright attack on the freedom of choice of programmers everywhere! No way I was going along with that. So, here is my branching action creator: 
    public Action<TMessage> Switch(
	Func<TMessage, int> pSelector,
	params Action<TMessage>[] pAlternatives)
	{ return pMsg => pAlternatives[pSelector(pMsg)](pMsg); }

    There, plenty of choices. Restrict to uint if you like. For those of you who can’t be bothered to match the output of the selector to the amount of choices, here’s the kids version with just two branches and a boolean function:

    public Action<TMessage> IfThenElse(
	Predicate<TMessage> pIf,
	Action<TMessage> pThen,
	Action<TMessage> pElse)
	{ return pMsg => ( pIf(pMsg) ? pThen : pElse )(pMsg); }
  • Aspects took about 5min longer but in the end I did two:
    • Blackbox aspect: It takes an action on a message and executes that action. Here’s the aspect creator: 
      public Action<TMessage> FromAspect_BlackBox(
	Action<Action> pShell, Action<TMessage> pBody)
	{ return pMsg => pShell(() => pBody(pMsg)); }

      Nice and clean – and no good at all on my first attempt to use it. I was after a logger I could use to surround actions with but any logger that logs no more than “hello from the world’s favourite logging aspect” is of rather limited use. I needed a logger that could access the message and log things like its id or parts of the content. Back to the drawing board and on to the

    • Whitebox aspect: It takes an action on a message, and the message and executes that action. There: 
      public Action<TMessage> FromAspect_WhiteBox(
	Action<Action<TMessage>, TMessage> pShell,
	Action<TMessage> pBody)
	{ return pMsg => pShell(pBody, pMsg); }

And this is it.

Put it all together into a static class and you get your pipe factory in 19 lines of code:

namespace PipeWorks
	{
	public interface IFilter<TMessage> { void ActOn(TMessage pMsg); }
	public class PipeFactory<TMessage>
		{
		public static Action<TMessage> FromType<TFilter>() where TFilter : IFilter<TMessage>, new()
			{ return pMsg => new TFilter().ActOn(pMsg); }
		public static Action<TMessage> FromAspect_BlackBox(Action<Action> pShell, Action<TMessage> pBody)
			{ return pMsg => pShell(() => pBody(pMsg)); }
		public static Action<TMessage> FromAspect_WhiteBox(Action<Action<TMessage>, TMessage> pShell, Action<TMessage> pBody)
			{ return pMsg => pShell(pBody, pMsg); }
		public static Action<TMessage> Sequence(params Action<TMessage>[] pSequence)
			{ return pMsg => Array.ForEach(pSequence, pI => pI(pMsg)); }
		public static Action<TMessage> Switch(Func<TMessage, int> pSelector, params Action<TMessage>[] pAlternatives)
			{ return pMsg => pAlternatives[pSelector(pMsg)](pMsg); }
		public static Action<TMessage> IfThenElse(Predicate<TMessage> pIf, Action<TMessage> pThen, Action<TMessage> pElse)
			{ return pMsg => ( pIf(pMsg) ? pThen : pElse )(pMsg); }
		}
	}

How do you like it?

The filter importer is just an example, there obviously are plenty of ways to get at an Action delegate, and they are all mixable, but in my opinion that one offers the most benefits.

Next time I am going to do some examples and try to deal with IOC issues.