Anything you missing there? anything you don’t like?

I’m trying to compile a list of suggestions for it, put a comment or drop me a line….

Every now and then I am drawn to implement a call to a web service using the SOAP adapter and without adding a web reference.

I wrote about it in the past here, as did others, for example consider Richard Seroter, Saravana Kumar and Jon Fancy’s articles on the subject.

Last time I had to do this, though, a couple of months ago, I bumped into a puzzling error, which took some time to understand.

In the scenario I had I needed to modify the message in the pipeline, and - being the good citizen that I am (?) - my pipeline component created a new message and did not return the same message it had received.

When my message hit the adapter it got suspended with the error -

Failed to load "" type.

Please verify the fully-qualified type name is valid.

Details: "".

The type must derive from System.Web.Services.Protocols.SoapHttpClientProtocol.

The type must have the attribute System.Web.Services.WebServiceBindingAttribute.

In my scenario the proxy was provided for the SOAP adapter via its configuration in the send port explicitly as per Richard’s post, so I was very confused about the empty type definitions in the error message.

Some investigation revealed the cause – the send port configuration is read at the beginning of the pipeline and these settings are set in the outgoing message’s context.

My pipeline component created a new message and did not copy these properties to the new message, so when it arrived at the adapter, which tries to read them, the empty definitions were found.

Some observations (I’ll you to draw your own conclusions, for a change) -

• When you import an MSI, any default binding in it will always be imported, then any binding matching the environment selected in the import MSI wizard (if selected)
• If the second binding contains any settings that were already created as part of the first binding, they will be overwritten
• When you export an MSI, a binding file is created with the existing configuration as the default binding, even if you did not add one as a resource explicitly. This binding will be imported first, before any other binding.
• In the case of receive locations – if two bindings in the MSI contain receive locations for the same port each marked as primary, the last one imported will be the primary receive location.

Why am I writing this?

We had an odd case (which shouldn’t really happen) where we created an application with a particular receive location.
We have then added a binding file for another environment in which we have misspelled the receive location name (the URI was the same)

When we tried to import the MSI to the second environment, selecting the environment name in the import MSI wizard we received an error that looked something like this -

(for the search engines: “Cannot update receive location”, “Address should be unique for Receive Location”)

The reason was that the wizard imported the MSI and initially applied the default binding creating the receive location with the first name. It then went on to import the second, environment specific, binding file, now trying to add a new receive location with the different name but failing as the URIs are the same. had the name been the same the receive location would have been overwritten, had the URI been different a second one would have been successfully added, but given this weird combination the error was produced.

A couple of months ago I published this post suggesting that pipeline component developers should look to design those in a streaming fashion which avoids loading the message into memory.

Quite a few people contacted me asking for an example so eventually I sat down to create one.

Creating the sample warranted writing down an explanation for it, which turned out rather long and (hopefully) elaborate, so i turned that into an article and, I’m proud to say, has now been published on MSDN.

In the article I wanted I wanted to demonstrate a realistic case, which doesn’t necessarily take the “easy path” and yet is clear enough to demonstrate the principles without overwhelming the reader; primarily I felt it was important to show a sample that modifies the stream (and not just using information from it) as these tend to be more complex.

Hopefully I have managed to provide a good demonstration of the principle. Let me know…

A few months ago I found out that a chap called Jason Hyland, who is another BizTalk consultant here in the UK, has nicked the contents of my web site to his own. word for word.

Despite the hidden compliment (I guess it means I write reasonably well) I was not very happy and so I thought I’d drop Jason a polite note.

I never received a response for my email, but several days later his web site was down, so I assumed he accepted this wasn’t nice and is working on his own wording. I guess I was wrong as today I was reminded of that and decided to check what he has done, and what do you know - http://www.needlecast.co.uk/services still uses, word for word, the wording from my web site.

How bad is that?!

Jason – at the off chance that you’re reading this – here’s some free publicity on my blog (currently averaging around 160 unique visitors per day) – write your own stuff, I’ve done it, so it can’t be that difficult, and it will be the decent thing to do.

Yossi

Recently we’ve done some performance tuning on aspects of our solution, and as part of that we’ve looked at configuring the system.net maxCnnections setting (see Darren Jeffords post on this subject) to increase the number of concurrent calls our application can make to various services it uses.

We’ve looked at this for our BizTalk solution but also for a complex, multi-threaded smart client we’re building.

Initially some have been somewhat sceptical about the effects of this, despite all the information on the web, but when I tried to figure out why the reason became apparent – their observations did not match the expectation – no matter they set in the MaxConnections setting, the amount of time it took, and the number of concurrent calls to the external service remained the same.

Trying to figure out why, we’ve used two scenarios – in one we ran our smart client application and monitored using Fiddler we examined how many concurrent requests are sent to the service; on the other we ran a test application we created specifically for this purpose. Confusingly there were two separate reasons for why the observations did not match our expectations -

In the test scenario, the application and the service we used to test were all done on the local machine. Of course in this case the request to the service does not actually go through the network, and so the system.net MaxConnections setting does not play a part; as soon as we’ve moved the service to another machine the results were as expected.

The smart client application, however, was already pointing at a service on a remote machine, and so there must have been something else there; as timing was a little bit more tricky in this case the guys used Fiddler to look into the requests made where they saw all the requests were getting sent out at the same time, despite setting the MaxConnections to 1.

It turns out, and that I did not know, that Fiddler was the cause of the unexpected behaviour.

I don’t know exactly how Fiddler works, but I suspect it hijacks the requests on the local machine so that from our client point of view the requests never actually went through the network and so the MaxConnections setting has been ignored. Fiddler itself does not adhere to this (I’m guessing it is not a .net application) and so multiple concurrent requests have been forwarded by it to the remote service.

As soon as we’ve removed Fiddler the behaviour was as expected.

Lesson learnt.

Here’s something I learnt from Guru Venkataraman on my one of my visits to Redmond last year -

Like most seasoned BizTalk developers, I suspect, I usually follow the best practice around host planning nicely summarised by Marcel here.

What I did not think about (and nor did Marcel, it appears) is the actual queues behind these hosts, how they are used, and the impact on performance; in comes Guru’s wisdom-

Each host has a queue, implemented as a table in the message box database, so - loosely speaking – when a message is published to the message box, BizTalk determines which subscribers are interested in the message, and places the message (well – logically) in the queues of the relevant hosts.

Each hosts polls its own queue periodically (default is 500ms) to get any pending messages and acts on them – start a new instance or correlate to an existing instance, potentially rehydrating it.

Now imagine you have one host running two different processes and that two messages are received both of which should trigger a new instance of process A.

The host picks up the first message (for this discussion to be clear enough we’ll have to pretend it’s working on a single thread, although I suspect several aspect of this are) and passes it to a new instance of process A.

Imagine now that process A publishes a message intended for process B; as process B runs under the same host, the message gets into the very same queue in which we already have the second message queued for process A.

Now – process B cannot start until the host reads the pending message and starts the second instance of process A. it is only at this point that the message intended for process B is at the ‘front’ of the queue and can be read by the host and an instance of process B can be started. effectively the execution of process B has been delayed unnecessarily because there were messages queued for process A.

Now – of course BizTalk much of this is indeed multi threaded and so the problem is not that severe. BizTalk was clearly designed to host many processes under any single host and there’s definitely not a requirement (or a recommendation) to have a host per-process, that would be ridiculous, however -

Where you have a flow where process A often publishes messages that end up in another process (B, C or D to make some names), and you have a reasonable throughput, you would get better performance if the publisher of the messages (process A) is hosted separately from potential subscribers (B, C or D).

With the subscribers configured with a different host they can pick up their messages without being affected by other messages awaiting for process A, and vice versa – the same goes for any replies published by them.

Two things worth remembering in this context:

1. When you ‘call’ an orchestration, as opposed to ‘start’-ing an orchestration, the host of the calling orchestration is used, regardless of what’s configured in the admin console
2. Whilst I’ve specifically mentioned processes, the same considerations apply to send ports, and even receive locations, and when looking at the former it is worth remembering that dynamic send ports, where used, always use the default host for the adapter used.

Back in December I’ve posted ‘This should not be a routing failure’ about an experience we’ve had on our BizTalk 2006 environment that I found very weird.

Tim Wieman, a senior program manager on the BizTalk Customer Advisory Team, was kind enough to drop me an email letting me know that this behaviour was indeed identified as incorrect, and was actually fixed.

A hotifx was created for BizTalk 2006 which later versions already incorporate, so if you’re using 2006, and you’re having this issue - get that hotfix, if you’re using a later version – you probably don’t need to worry about it.

I have managed to test the hotfix on BizTalk 2006 and it does allow my scenario to run without a problem (don’t forget the registry change for each host, as described in the link to the hotfix); I haven’t been able to try this on later versions yet, but there’s no reason to think that would not work.

Thanks Tim!

This always seemed like an obvious one to me, but as I do see it out there occasionally, I guess it should be mentioned in my little ‘series’ – think carefully whenever you are hard coding any data into your code, as it is almost guaranteed to change one day, however remote you think that chance is.

This post could probably end here, but that would be almost rude, so I’ll give an example -

A custom disassembler is being developed to identify a message type before parsing it.

The messages are expected to be received over POP3, and the decision which message it is is done by identifying the sender of the email (rather than by looking in the actual message), which is fair enough.

So – the component has code to read the context property storing the sender’s email address and then a switch statement on the property’s value determines the message type required.

Of course this works very well. until a third party decides to change the email address, or another one is added. now you have to update the code, re-deploy and re-test. why? because you did not want to spend another day moving this to configuration?

Even a simple AppSetting would be better than hard coded, but of course usually you’d be looking at least at defining a custom configuration section if not using a database or other store (SSO?)

Important update: it looks like it’s not after all! and I’ve posted the details here; I suggest you check it out after reading this post.

Some routing failures are straight forward – you have a message, it has context, and no one subscribes to it. Routing Failure Report notifications are very useful on helping troubleshooting these – and most likely someone has mistyped a property’s value, a subscription value or forgot to ensure that the relevant properties are promoted and that relevant services are enlisted.

Some routing failures are a little bit more tricky to map out -

Consider a simple scenario where you have a two way receive port publishing a request, which is being picked up by a two way send port -

Now add a second two-way subscriber to the mix…

In this case BizTalk cannot know which of the two potential responses it may get (from the two different two way send ports) it should deliver to the [send side of the] receive port, and so it opts to not even publish the request and instead deliver a routing failure.

This is fair enough - race conditions are not a good thing and it’s pretty clever that BizTalk identifies this up front and prevents this.

However - the same routing failure will happen, it appears, even if the publisher is NOT a two way port, and that – I did not expect.

If the publisher of the original message (the receive port) was not a two way port, but one-way, it would not expect any response from anybody, which removes the race-condition situation, and therefore should have been supported in my view, but it does not appear that it does.

Imagine a scenario like this -

1st Process publishes message A, through a one way port
1st Send port (2-way) picks up message A and calls a service, publishing the service’s response - Message B.
2nd Send port (also 2-way) picks up message A as well and calls a different service, publishing this service’s response - Message C.

As far as I can tell, this is a valid scenario – call it message enrichment via external services.

Subscribers should be able to subscribe to message A, B or C as they wish, and all is nice and loosely coupled, if you only ignore the fact that this does not seem to work!

Building such a scenario I'm getting a routing failure for message A, which - unless I'm not seeing correctly - I can only explain according to the rule mentioned above, which I find really frustrating.

The next ‘in-person’ meeting for the UK SOA/BPM User Group will be held on the 27th of Jan in Paddington – London, check out the details here, and if you’re coming – be sure to come and say hi – I’ll probably be the loudest chap in the room. again.

And even before that, on January 12th, an online meeting will be held with the title ‘Developing Composited Solutions with Sharepoint and SAP’

Microsoft UK have published the details of this event taking place in the MS Campus in Reading, see the details and register here

The event has a very promising agenda -

I had the pleasure of meeting Robert Hogg several times through the MVP program, and he is probably one of the most suitable guys I know to be talking about pragmatic approach to projects with a track record of successful “rescue projects” under his belt.

I also have the pleasure (although I’m not sure it’s mutual) to be working closely with Paul Saggar and Jon Simpson, and if anybody can talk about implementing complex business processes in an ever changing, and highly competitive, environment it’s these two.

And of course, as many know, I work closely with PNMsoft, and so its only natural that I’d say it’s always very interesting to hear from their vast experience in the BPM field in which they’ve been working for many years.

See you there!

This one is fairly common, I suspect, and I can certainly see why – the temptation is simply to big – but too many pipeline components start by reading the message into memory, when, with a little bit more effort this could have been avoided.

One pipeline component I’ve seen, for example, receives a flat file, and needs to remove records already processed (duplication elimination) – quite a good thing to do in a pipeline, and I also liked the approach of doing so before the disassembler, to make the xml produced smaller.

V1 of the component used a memory stream – the incoming stream was read line by line, each line was assessed, and – if was not a duplicate – it would get written to the memory stream.

When the component had finished going through the entire incoming stream, the memory stream would be assigned to the message, replacing the original stream, and the message would get returned to the pipeline.

There are two downsides to this approach – the first is memory consumption – the component will always consume at least as much memory as the size of the (outgoing) message; done properly BizTalk would then clean this memory, but only after completing the processing of the message; the second downside is potentially unnecessary delay in further processing of the message – one of the huge benefits of the pipeline, in my view, is its streaming fashion, where subsequent components, if developed in the correct manner, can start working on parts of the message before preceding components completed their processing; basically each component passes back to the pipeline the portion of the message it already processed, whilst working on the next portion.

It appears that the customer in question encounter memory issues as the component’s code was changed to use virtual stream instead of memory stream; a virtual stream is effectively a stream that uses disk for storage instead of memory.

This solves the memory consumption issue, but merely replaces it with IO operations which may have an even bigger impact on the server’s overall performance (and does not address the processing delay point at all).

What would have been the correct way to implement this in my view?

The component should have create a custom stream, wrapping the original stream from the message; It would then replace the message’s stream with the custom stream immediately returning the message back to the pipeline. Note that so far the component hadn’t touched the message stream – zero bytes have been read.

As BizTalk (and not the component!) would read the message (for instance when persisting it to the message box), the custom stream’s read function would be called which would contain that reads the underlying stream (the original stream received by the component), probably buffering reads until the end of a line for simplicity (although in many cases this is not necessary) and assessing whether the record is a duplicate or not; if it is a duplicate the function will simply read the next line and so on until a non-duplicate record is found, at which point the line would be returned as the output byte stream from the read function.

This effectively means that the next component, or the message box, will receive the message line by line, duplicate records removed, without having to wait for the component to process the entire message, and with only a maximum of one line ever loaded into memory.

I’ve been doing some work recently with the LOB adapter SDK.

The adapter I’ve been creating is effectively a wrapper around a web service we’re using extensively, to which we often need to make several consecutive calls.

Doing so from an orchestration, using a send port, is inefficient, as each request involves a pub/sub cycle; on the other hand – I’m not a big fan of ‘inline sends’, which was one of the options considered.

Building a custom adapter would give us the best of both worlds, if you like – the process would create a message containing all the requests to be sent and publish it; a send port, configured with the custom adapter, would pick this message and deliver it to the adapter which would, in turn, debatch the requests, send them to the service, aggregate the responses back and return the aggregated message.

This way we still have all the tracking and management BizTalk provides but only one pub/sub cycle.

The LOB adpater SDK does a very good job in lowering the barrier of creating adapters (I’ve created ‘native’ adapters before) which is a good thing.

Several points I’ve been left with from this, very initial, exercise -

Implementing the metadata browser, resolver and search handler was a bit tedious; I ended up creating an xml file with my metadata, embedded this in my project and wrote code to generate the metadata off that. I wish this was part of the SDK.

Once the metadata was set I needed to write the outbound handler (in my case), for which I needed to know how the incoming messages and outgoing responses should look like; with no generated samples this was a bit too painful to my liking – figuring out the request format is easy although tedious again – I’ve ended up consuming my adapter from a client, making a call, setting a breakpoint and going over the message at runtime.

This is, of course, not possible for the response – where I need to create the message in the adapter, so trial and error was the only way (ok – and reading the docs, and applying some common sense) – in my case I actually made a mistake in my metadata configuration which took me a while to figure out (and Rupert’s help, thanks!)

Bottom line is that if there was a tool to look at the metadata and generate request/response samples for each operation this work would have been much easier. I might be tempted to create one if I get the time.

The last thing that phased me was working with metadata from the client.

I needed the client to provide some information outside the message; I knew the mechanism in BizTalk would be to use a context property, for which I will need to deploy, with the adapter itself, the property schema, but I wasn’t sure if and how the adapter gets access to that or how I could consume this from a .net client.

Figuring out the adapter access was easy enough, turns out all the BizTalk message’s context properties are added as properties to the request WCF message received from the framework, so

token = message.Properties[PROPERTY].ToString();

where PROPERTY is the context property’s namespace#node would return the value of the property requested.

I was a bit stumbled with how to provide this property outside BizTalk, this time Manas Garg came to the rescue in this thread

Turns out you can simply do -

ConcurrentPrograms_ARClient client1 = new ConcurrentPrograms_ARClient(binding, address);

using (new OperationContextScope(client1.InnerChannel))

{


          OperationContext.Current.OutgoingMessageProperties.Add("Property Name", "Property Value");


          client1.OPERATION(params...);


}

So – here’s another real-world they shouldn’t have done this –really! item; somewhat related to my previous post about projects, but a different slant -

MyP had used pipeline components fairly extensively, a very good idea in my view, especially in this case.

But, as discussed in that previous post all pipeline components, regardless of where they were used, were bundled together in a single assembly.

To make matters worse, a single pipeline component often served more than one purpose, for more than one client.

So – for example – a pipeline initially created to remove an unwanted trailer from a message from a particular sender, ended up also converting the message to xml, and then extended to support another format, from another sender, only that the two don’t share any code – the execute method of the pipeline component has a switch statement on the sender name, and runs two separate functions.

Now – considering all the components are in the same assembly already, how can this make matters worse?

Well – single responsibility principle is one that I generally like – I’m a new developer working on this project, I see a component called TrailerRemover, used in a pipeline called <someSender>_Receive I assume this is processing messages from <someSender> and that it removes a trailer.

I eventually discover it does a lot more, and processes messages from another sender as well.

One of the side effects of this is time wasting – it is much more difficult, in my experience, to maintain systems that don’t follow the single responsibility principle.

This is aggrevated by the fact that this is usually a symptom, if not a cause, for bad architecture – I shouldn’t be able to mix logic for two different senders, not unless specified common logic is factored out and reused properly, from a shared assembly.

Now when I come to change some code I find it difficult to know what the impact may be – where exactly is this thing used?

So – what not to do?

Here’s one – don’t take 15 different interfaces, from 15 different providers, and the canonical format, and your internal process and bundle it into the same set of projects.

My predecessor (MyP in short) in this project had followed the best practice and ensured (for the most part) that schemas, maps, pipelines and processes each have their own project.

This is good – to start with, in the old days, mixing them used to cause all sorts of build issues, but – although I haven’t tried in several years now – I assume that’s all in the past, but – more importantly – this is bad practice because different artifacts have different ‘resilience to change’ – if you need to add a small shape to a small process that gets instantiated as a result of a receive shape (and not being ‘started’ or ‘called’) you can usually un-deploy it fairly easily.

Change a schema, on the other hand, and usually there’s a whole raft of artifacts you now have to un-deploy with it.

For that reason – mixing two artifact types in the same assembly, whilst not technically problematic, usually suggests you will have maintenance nightmare in the near future (unless you don’t mind down time, and regression tests, that is).

Anyway, as I was saying, MyP did separate the different artifact types to different assemblies, but equally MyP only had one assembly of each; so – when we are processing a message arriving from partner A, for which we have a schema, a pipeline, a map or two and an orchestration – these were split across four assemblies; equally - when we are processing a message arriving from partner B, for which we also have a schema, a pipeline, a map or two and an orchestration, all different – these were also split across THE SAME four assemblies.

On the surface not much wrong with this, the problem is what happens when you want to change a small thing in, say the schema for partner B, used from within a map, which in turn gets used within an orchestration – you now have to un-deploy the orchestration assembly, so that you can un-deploy the maps assembly, so that you can un-deploy the schema assembly, so that you can redeploy the new version of the schema assembly followed by the rest. (ok – so I’m assuming versioning and side-by-side deployment are not being used – which I have to – for my story);

So – all that is the general pain that is part of the BizTalk developer’s day, why am I bringing this here? well because you’ve taken a BizTalk ‘challenge’ and doubled it - for one – you took process A down due to a change in process B; why? had they existed in separate assembly sets you wouldn’t need to… and two – as you’ve released new code – you have to test new code; only that now you have to test two sets of code, including one you haven’t intended to change (but may have, by mistake or otherwise); again – had there been two sets of assemblies, you could probably get away with testing just the scenarios related to the ones you’ve changed.

Now multiply this by about 15 partners, and you see how it can be quite wrong. I hope.

Again I find my self having to apologise for not writing for a while (it’s been one month since my last post, two since my last ‘real’ post), the usual suspects are to blame – too much work, too much bureaucracy, kids, new xbox games….

Work wise I’ve taken on a few more small engagement recently, one of which was to enhance/fix/maintain a small-ish solution someone else built.

I could not resist the temptation to take some notes of all the stuff I would have done differently, and am in the process of compiling them as a proper report for the client.

As I was not posting for a while, partly due to this work, I thought its only fitting that I publish some of these ‘recommendations’ here.

For obvious reasons, I will not name names and will try to generalise any samples/explanations provided; I can’t resist mentioning, though, that this other person, I learnt as I looked up his/her name on the web, had also nicked the entire text off my web site and used it on his/her own, which I found rather annoying (I believe the pages have since been removed, after a polite nudge…)

Anyway….keep in mind that many of my notes may come down to style, or MY best practice; they don’t necessarily mean that the other approach is completely wrong -  I don’t presume I know better than anyone else (oh well….) but that there’s value in another point of view – in this case- mine.

So – a few posts coming, likely to be very short and to the point, hopefully someone will findthem useful.

Yossi

This time Global 360 will introduce themselves to the group members, if you’re curious – check the details here

Check out tomorrow’s ONLINE SUBG meeting in which Jean-Pierre Auconie, the engineer behind the famous MessageBoxViewer tool talk us through it; details here

During the last few weeks I’ve been asked to review two separate projects, for two separate companies, developed – naturally – by two separate teams.

The two things both projects had in common were that they both had to deal with legacy “flat files” and they both chose to process these files outside BizTalk using custom code.

In both these cases I completely agree with the decision to use custom code to parse the incoming files – as good as the out-of-the-box flat file support in BizTalk is (made significantly better with the introduction of the flat file wizard, once one gets the hang of using it) – there’s no avoiding from writing custom code to parse flat files every now and then – some file formats are pretty challenging with different records types, conditional records, interleaving segments etc.

I do not agree with, however, the decision to perform this custom parsing outside BizTalk.

I’m pretty sure I would not even bother posting this point had I not seen two of these in the same month, but the fact that I did suggests it may be worth posting a quite note.

One of the projects had the code in a console app, called from a windows scheduled task; the application would pick up files from a folder, parse them, and drop the xml representation in another folder, for BizTalk to consume.

The other had a windows service monitoring a folder and pick up any files, parse them to a different, simplified, flat file format (!), and drop them in another folder for BizTalk to consume.

Both of these introduce another component to the mix; such component needs it’s own error handling, it’s own monitoring, deployment strategy, operations manual etc. similarly it includes a fair bit of re-inventing the wheel – writing code to monitor folders, read files, and write files – stuff that BizTalk is doing out of the box.

What would have been the correct approach then? quite simple – a custom disassembler in the receive pipeline -

Writing a custom disassembler it quite simple - at the end of the day, it boils down to developing a class library, which implements a few simple interfaces, the main one – IDisassemblerComponent defines two methods - Disassmeble and GetNext (the other interface are even simpler, almost insignificant in terms of effort)

Disassemble gets the source message and potentially parses it up front, GetNext is called repeatedly by the pipeline to receive 0 or more parsed messages, until it returns null. simple.

In one of the projects I’ve since taken their existing code (console app), refactored it into a class library, and wrapped it in a custom disassembler class that calls it; converting the scenario to a BizTalk pipeline and performing the key “developer testing” took less than a day.

Why did they not do it to begin with? whilst sometimes there are valid reasons, technical or otherwise, I suspect that in this case it was just unfamiliarity with with BizTalk and some lack of confidence in the development team’s ability to learn and implement (or their belief in themselves); these are valid concerns to any project manager, but I would suggest that a better course of action would have been to spend some time looking at what it takes to implement a custom disassembler, seeing that’s its not at all that scary, and by doing so learning more about a product used in the solution (BizTalk) and achieving a better architecture, and more maintainable approach.