shuttle / shuttle-esb Goto Github PK

REQUEST
It would be nice to have a way to:

• choose which connection string to use
• to have a database connection cache for a web-session

ISSUES
Also i'm trying to use shuttle-esb with NHibernate (with SQL 2008 R2 and .net 4.0). But there are some issues.

scenario: In transaction scope, some items are persisted via nhibernate, then a shuttle esb message is sent (writed to the db). after that another nhibernate write should be performed and the transaction should be commited.
#1) even if we're using > .net 3.5 (currently .net 4.5) and sql 2008 r2, and we have exactly the same connection string (which normally is enough to avoid DTC even if we open 2 separate connection inside transaction scope) - shuttle-esb will somehow force the transaction to be distributed
#2) if we use shuttle-esb transactionscope with enable=false, the transaction will be aborted, cause the write operation after sending the shuttle esb message will fail, because the original connection associated with nhibernate session is closed.
#3) whit enabe=true i've got: The operation is not valid for the current state of the enlistment..

question: if i set transationscope enabled to false, what will happen? will the original transaction be supressed? btw: what's the default behaviour?

Hi,

I use exactly the same connection string for DB connection as for the Subscriber. Even so, the transaction is promoted to a distributed one. Since the conditions from the text below are met (assume you use sql ado.net provider and you do not change/capitalize/trim the connection string from app.config), there is actually no need to have DTC.

Some providers, (like SQL2008 in .NET 3.5+) recognizes when a new connection is created in a transaction scope that refers to the same connection string, and will not result in DTC work. But any variance in the connection string (such as tuning parameters) may preclude this from occuring - and the behavior will revert to using a distributed transaction.

Currently the idempotence service only tracks message handling. When messages are sent it would be helpful to ignore messages that were sent in a transaction scope where the transaction scope failed.

Assembly scanning may interfere with normal application operation in some instances.

Make the scanning optional and perhaps have the ability to explicitly specify the queueFactories (type to load / assembly to scan).

There is a configuration for DurationToIgnoreOnFailure on OutboxPipeline but the DeferTransportMessageObserver is not included. This makes the outbox feature less tolerant with network failure.

It may be necessary to strip out the username/password for the sender uri from the TransportMessage. This could be configurable.

However, the server would then need credentials to be able to bus.Reply to the client. The credentials could be specified for the uri in the RabbitMQConfiguration.

Another use would be where the credentials of the received message are no longer valid and should be replaced.

Should a server endpoint need to bus.Reply to a client that is using a sql transport the host name connection string name would need to be defined. A possibility may be to pass the actual connection string with the transport message. The server would then use that.

Currently the service bus configuration is loaded and registration tasks are execute before the StartUpPipeline starts. This leads to queue factories being called too late to apply certain functionality on initialization.

This bug applies only if a queue factory implements IRequireInitailization to access configuration data prior to queue registration/create.

Currently MsmqQueue always sent messages with MessageQueueTransactionType.Single. This defeats the purpose of TransactionScope. I suggest that it should be like this:

    if (Transaction.Current != null)
    {
        queue.Send(sendMessage, MessageQueueTransactionType.Automatic);
    }
    else
    {
        queue.Send(sendMessage, MessageQueueTransactionType.Single);
    }

BTW, I would like to ask if I should post issue like this here or in the shuttle-esb-msmq.

Currently the SqlQueue uri 'host' portion contains the connection string name. Change this to contain the actual connection string.

Since this will undoubtedly lead to duplication this will typically be used with a QueueHostResolver,

There should be an option to turn on dead-letter per end point in MSMQ. Newbies are easier to get stuck with the permission issue:
http://stackoverflow.com/questions/8519606/msmq-send-to-remote-private-queue-does-nothing

The bus may work in dev environment but when going production, somehow the queue created without Anonymous Logon permission and the message delivery is just failed without any exception or even nothing left on the outgoing queue. Storing dead-letter by default will be a rescue to ensure that the message won't be lost entirely.

When an exception is thrown by an exception-handler, the handler is never passed to MessageHandlerFactory.ReleaseHandler().
This causes the Dispose() method of the handler never to be called.

I suggest the calling of the message handler in to be changed to:

try
{
InvokeHandler(bus, handler, transportMessage, message, pipelineEvent, message.GetType());
}
finally
{
bus.Configuration.MessageHandlerFactory.ReleaseHandler(handler);
}

fails: rabbitmq://shuttle:shuttle!@localhost:15672/shuttle-server
works: rabbitmq://shuttle:shuttle!@localhost/shuttle-server

It appears as though the two properties are not being used. Investigate and remove if not required.

Currently a warning is logged but it would probably be more accurate to throw an exception since the publisher expects a message to be published, yet no subscription manager exists.

After reviewing documentation, samples as well as implementing a prototype for myself, I had a question when it comes to architecture for a live/production environment. Out of the box (using the defaults) message routing and subscriptions are stored via the consumers and producers application configs (app/web.config). Which leads me to believe that the applications become couplied of other applications "awareness" (location, known-being, queue), would it be a good idea of preventing this coupling when expanding outside of the sample applications towards enterprise (via broker/router)?

Instead of having various options to send messages, rather have a fluent builder interface to construct the relevant TransportMessage and then Send() or Publish() it.

Hello,

   I have a question about Pub/Sub.It almost like this:

PubServerA publish a message named EngineEvent.
SubA and SubB subscribe to EngineEvent and handle the event by implementing some relevant functionality.

Now,PubServerA publish the EnginEvent,while the service bus of SubA does not start up.And when the service bus start up,I don't want to receive any messages.
Now I've no idea about this.Can you give me a hand?

Regards,
MoonSpring

Since the host contains the actual connection information that may be sensitive a host resolver may be useful so that the uri need not contain the actual host connection information. The uri would change to, e.g.:

rabbitmq://host:ServerA-MyHost/queue-name
sql://host:ServerB-Connection/queue-name

Hi,Eben,

I've solve that problem that I asked you last time.It is about starting two service bus in one process.Now I've already changed it to one service bus,it's ok now.

today,I also need to develop a windows service.When the computer is open,the service will start up,too.

And the windows service will contain a Shuttle service bus and some handlers.And the windows service can't start up because of the Shuttle ESB Service bus.

Why did this happen?

Regards,
MoonSpring

The TransactionScopeObserver is no longer used in the StartupPipeline and can be removed.

I am not sure if I am posting my question at right place.

Configuration Question:

Is it possible to add above configuration through codes when we create a new bus i.e
bus = ServiceBus
.Create()
.SubscriptionManager(subscriptionManager)
.AddModule(new SystemExceptionModule())
.Start();

Thanks in advance. I apologize if I posted this message at wrong place.

All configuration sections are implemented as individual sections. It should also be possible to group all the sections under the group.

Initializing queues in the constructor leads to unpredictable behaviour, especially when testing. Operations in the constructor may use existing queues when new queues, with different options, may be required.

The IRequireInitialization seems to be a likely candidate but the queue implementation will have to ensure idempotence; for instance when the same error queue is used more than once it must only be initialized once.

I tried to implement object IDatabaseConnectionFactory but in the class constructor SqlQueueFactory I find used "DatabaseConnectionFactory.Default ();"

public SqlQueueFactory()
{
scriptProvider = ScriptProvider.Default();
databaseConnectionFactory = DatabaseConnectionFactory.Default();
databaseGateway = DatabaseGateway.Default();
}

This invalidates my custom implementation. How can I do?
Thanks.

Any final solution with full source code sample application ? IMHO, better samples for minimize learning curve are real applications with full source code and good patterns.

There should be 2 defer options:

Using a deferred endpoint and service:

In order to send messages at a later stage this module will enable messages to be sent via a deferred message endpoint. The SendMessagePipeline should be modified to alter the RecipientInboxWorkQueueUri to point to the DeferredMessageService endpoint. This will register the TransportMessage in a data store along with the IgnoreTillDate value and the actual RecipientInboxWorkQueueUri that will be stored in a header on the TransportMessage. Once the message should be sent the service will alter the TransportMessage RecipientInboxWorkQueueUri and send the message off to the relevant endpoint.

Sending to the local queue:

The second option is to simply change the IgnoreTIllDate option on the TransportMessage and send the message normally.

Should be able to specify which connection string to use.

Hi,Eben,
How are you?

I have three questions that I have to ask,please help me.
1、do you remember the question I had asked you before about Oracle supporting?
Now I know the Shuttle ESB support the MSMQ and the SqlServer queue.I want to ask:Can I just use the MSMQ queue without SqlServer in the Pub/Sub demo?

2、I know,if I want to run the Pub/Sub,when the Pub publish a message,Where is the message?It is don't saved in database.It is in the pub Computer side,or the sub Computer side?

3、Now,Shuttle ESB can run normally in my system.But when my project go on-line,it have to keep running for 10 yes.So I want to build a clusters environment.What if the ESB pub have something wrong,I can change it to a normal one quickly. But I don't know too muck about Clusters and load balancing.Please help me.

Regards,
MoonSpring

When a transport message cannot be deserialized the UnderlyingMessageData is enqueued in the Error queue. However, this does not really help as nothing will be able to ever deserialize the message.

A better option may be to have a 'CorruptMessagesFolder' configured in the ServiceBusSection with a default of '.\CorruptMessages' and then simply write the stream to an arbitrary guid named file.

Hello,

Today, I changed the project build path,it sends me this wrong messages.Do you know why?

At System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
At System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
At System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
At System.RuntimeType.InvokeMember(String name, BindingFlags bindingFlags, Binder binder, Object target, Object[] providedArgs, ParameterModifier[] modifiers, CultureInfo culture, String[] namedParams)
At System.Type.InvokeMember(String name, BindingFlags invokeAttr, Binder binder, Object target, Object[] args)
At Shuttle.ESB.Core.Pipeline.RaiseEvent(PipelineEvent event, Boolean ignoreAbort)
At Shuttle.ESB.Core.Pipeline.Execute()
At Shuttle.ESB.Core.InboxProcessor.Execute(IThreadState state)
At Shuttle.ESB.Core.InboxProcessor.Shuttle.ESB.Core.IProcessor.Execute(IThreadState state)
At Shuttle.ESB.Core.ProcessorThread.Work()
At System.Threading.ThreadHelper.ThreadStart_Context(Object state)
At System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
At System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
At System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
At System.Threading.ThreadHelper.ThreadStart()

Although a subscriber may require a message type it may not always want a particular instance. It may be worthwhile to remove subscriber uris based on some condition. We may need a pipeline here and the routing could be done using a module; else a simpler model may be used such as a filter using a delegate or even a specification (list) of sorts.

The DeferredMessageElement is not being used at all.

Also refactor the remaining elements that still use the more complex configuration setup.

The uris are passed in clear text in the TransportMessage. Since something like RabbitMQ may require the username/password as part of the message it may be worthwhile encrypting the uris.

Currently the Msmq configuration section can specify queue-specific configuration also. This means entering the entire uri for the queue and then the settings.

An option may be to implement it to include the transactional attribute in the uri:

msmq://./my-queue?transactional=[true|false]

The default behaviour is to use a transactional queue if the transactional value is not specified.