Setting up Flower services

The section "Setting up a directory, a processor and deploying a workflow" covers the basics of how to install and setup Flower services. This section will help you to fine-tune and customize them.

Flower service host

All Flower services except the processor are hosted in the Windows service called Flower services host. It is is configured via two files: Flower.Services.Host.exe.config and Flower.Services.Host.config. The first is a standard application configuration file where you can setup WCF endpoints' configurations; the second is a Spring.NET IoC container where the services host object is defined.

In Flower.Services.Host.config you will see a lot of objects defined, but in most of the cases you will need only the 'service' object.

<object id="service" type="Flower.Services.Host.Service, Flower.Services.Host">

  <!-- The directory. -->
  <property name="Directory">
    <object factory-object="directoryProxyProvider" factory-method="MakeProxy">
      <constructor-arg>
        ...
      </constructor-arg>
    </object>
  </property>

  <!-- The state service. -->
  <property name="StateService">
    <object type="Flower.StateService.StateService, Flower.StateService">
      ...
    </object>
  </property>
</object>


The 'service' object exposes the properties for each service that can be hosted. So, to get only the services you really need to host on a particular instance, just comment out the properties for the services you don't need.

Setting up the directory and changing the directory implementation

First, check the WCF service endpoint configuration in 'Flower.Services.Host.exe.config'. The configuration used by the directory is 'Flower.Services.Host.Directory'. By default, the directory uses netTcpBinding and the endpoint 'net.tcp://localhost:10000/Flower/Directory'.

Then edit the 'Directory' property of the 'service' object.

<object id="service" type="Flower.Services.Host.Service, Flower.Services.Host">

  <!-- The directory. -->
  <property name="Directory">
    <object factory-object="directoryProxyProvider" factory-method="MakeProxy">
      <constructor-arg>
        <!-- The default in-memory directory implementation. -->
        <object type="Flower.MemoryDirectory.Directory, Flower.MemoryDirectory">
          <constructor-arg name="scriptEnginePool" ref="scriptEnginePool" />
          <constructor-arg name="directoryProxyProvider" ref="directoryProxyProvider" />
          <constructor-arg name="emulateSerialization" value="false" />
          <constructor-arg name="processesDispatcher" ref="processesDispatcher" />
        </object>

        <!-- The SQL server directory implementation. -->
        <!--
        <object type="Flower.SqlServer.Directory.Directory, Flower.SqlServer.Directory">
          <constructor-arg name="settings">
            <object type="Flower.SqlServer.Directory.Settings, Flower.SqlServer.Directory">
              <! The name of the connection string to use to connect to a SQL server instance. >
              <property name="ConnectionStringName" value="FlowerDirectory" />
            </object>
          </constructor-arg>
          <constructor-arg name="authorizationProvider" ref="authorizationProvider" />
          <constructor-arg name="scriptEnginePool" ref="scriptEnginePool" />
          <constructor-arg name="directoryProxyProvider" ref="directoryProxyProvider" />
          <constructor-arg name="processesDispatcher" ref="processesDispatcher" />
        </object>
        -->
      </constructor-arg>
    </object>
  </property>
  
</object>


To change the implementation, you need to change the object passed as an argument to the directory proxy provider.

Directory implementations depend on four important objects: the authorization provider, the directory proxy provider, the script engine pool and the processes dispatcher.

The directory implementations which support authentication use authorization provider to determine user roles. The default provider obtain roles from the directory, caches them and monitor the directory requests to update the cache if the roles have changed in the directory. The provider almost always can detect a role change by intercepting directory requests, however, in some cases it can't do that. Therefore, it polls the directory roles with some interval. This interval is specified as the constructor argument (it shouldn't be too short).

  <object id="authorizationProvider" 
          type="Flower.Security.AuthorizationProvider, Flower.Security">
    <!--
        How frequently the changes in the roles should be checked
        (once per minute in this case).
    -->
    <constructor-arg value="00:01:00" />
  </object>


The directory proxy provider is used as a factory to instantiate the directory and it's passed as a constructor argument to the directory as well. This is required to allow the directory to make its own proxies for running scripts without getting circular references.

As the script engine initialization is too costly, the directory host pools the engine instances. You can specify the maximum number of script engine instances in the pool. Keep in mind that this number effectively limits the number of parallel script requests.

  <object id="scriptEnginePool" type="Flower.Directory.Host.Default.ScriptEnginePool, Flower.Directory.Host.Default">
    <!-- How many scripts can be executed simultaneously. -->
    <constructor-arg type="int" value="1000" />
  </object>


The processes dispatcher tracks the processors' throughput and dispatches the processes to make the load evenly distributed across the processing instances.

Setting up a processor

Like the services host, the processor has two main configuration files: Flower.Processor.exe.config and Flower.Processor.config.

Edit Flower.Processor.exe.config if you need to change the WCF endpoint configuration of the processor or the directory client endpoint used by the processor. You specify the directory endpoint name in the IoC container and it is 'Directory' by default. The service endpoint of the processor is 'Flower.Processor.ProcessorService'.

The Flower.Processor.config is Spring IoC container with processor objects. In most of the cases you will change only 'processorSettings' object.

<object id="processorSettings" type="Flower.Processor.ProcessorSettings, Flower.Processor" singleton="false">
  <!-- Name of the processor in the directory. -->
  <property name="ProcessorName" value="DefaultProcessor" />

  <!-- Number of threads in the I/O threads pool. Default 10. -->
  <property name="BlockingThreadsCount" value="30" />
  
  <!-- Number of threads performing non-blocking calculations. Default 4. -->
  <property name="NonBlockingThreadsCount" value="4" />

  <!-- The time a waiting process is kept in the cache before unloaded. Default 30 mins. -->
  <!--<property name="ProcessUnloadTimeout" value="00:30:00" />-->

  <!-- How often the processor will check pending processes without notification. Default 30 secs. -->
  <!--<property name="PendingProcessesCheckingInterval" value="00:00:30" />-->

  <!-- The timeout of the activities' transactions. Default 1 min. -->
  <!--<property name="ProcessorTransactionTimeout" value="00:01:00" />-->

  <!-- 
    When an activity that can be retried throws an exception, the processor, first,
    waits this number of seconds before the next retry. Then, doubles the period 
    before each next retry. Default 30.
  -->
  <!--<property name="MinWaitOnErrorSeconds" value="30" />-->

  <!-- How many times the processor retries failed activity. -->
  <!--<property name="MaxRetries" value="3" />-->

  <!-- How long a non-blocking activity can run before aborted. Default 10 mins. -->
  <!--<property name="NonBlockingActivityTimeout" value="00:10:00" />-->

  <!-- Maximum number of processes the processor can run simultaneously. -->
  <!--<property name="MaxProcesses" value="0" />-->

  <!-- The list of custom advisors. -->
  <!--
  <property name="CustomAdvisors">
  </property>
  -->
</object>


The most important setting is 'ProcessorName'. This is the name of the processor as it registered in the directory i.e. the name of the entry under '/Processors'. If the processor is not registered or registered with a different name, it won't work.

Flower processor schedules blocking (I/O) and non-blocking activities to different thread pools, so that activities performing calculations are never blocked by the I/O. For the non-blocking pool the number of threads should not be more than the number of CPU cores in the system; for the blocking threads the more is the better.

Setting up a state service

In a production environment it is recommended to install local state services on each processor node and setup access to them via named pipes.

To setup a state service, install the Flower the state service implementation on a processor node (service host will be installed as a dependency).

msbuild /t:SqlServerStateService


Then, edit Flower.Services.Host.config to set the 'StateService' property of the 'service' object.

<object id="service" type="Flower.Services.Host.Service, Flower.Services.Host">

  <!-- The state service. -->
  <property name="StateService">
    <object type="Flower.StateService.StateService, Flower.StateService">
      <constructor-arg name="flowerClient" ref="flowerClient" />
      <constructor-arg name="settings">
        <object type="Flower.StateService.StateServiceSettings, Flower.StateService">
          <property name="StateTimeout" value="00:10:00" />
          <property name="DirectoryPropagationTransactionTimeout" value="00:00:30" />
        </object>
      </constructor-arg>
      <constructor-arg name="storage">
        <object type="Flower.SqlServer.StateService.StateStorage, Flower.SqlServer.StateService">
          <constructor-arg name="connectionStringName" value="FlowerStateService" />
        </object>
      </constructor-arg>
    </object>
  </property>
  
</object>


By default, the state service uses netPipeBinding and is available at 'net.pipe://localhost/Flower/StateService'. You can edit Flower.Services.Host.exe.config to change it.

The state service can use various persistent storage providers, but currently Flower comes with only one - SQL Server based (Flower.SqlServer.StateService.StateStorage, Flower.SqlServer.StateService). To use this implementation, you need SQL Server to be installed on the processor node. For the most of the cases, SQL Server 2008 Express is enough. In the Scripts folder of the installation package you will find a script Flower.SqlServer.StateService.sql. You need to create an empty database and run the script in it. Then, edit Flower.Services.Host.exe.config to specify connection string for the state service (by default, it is FlowerStateService connection string).

Finally, make sure an object 'stateService' is defined in Flower.Processor.config of your processor and there is a corresponding WCF client endpoint configuration defined in Flower.Processor.exe.config.

<object id="stateService" type="Flower.Client.StateServiceClient, Flower.Client">
  <constructor-arg name="endpointConfigurationName" value="StateService" />
</object>

Setting up a queue service

To setup a queue service, install the queue service implementation (service host will be installed as a dependency).

msbuild /t:SqlServerQueueService


Then, edit Flower.Services.Host.config to set the 'QueueService' property of the 'service' object.

<object id="service" type="Flower.Services.Host.Service, Flower.Services.Host">

  <!-- The queue service. -->
  <property name="QueueService">
    <object type="Flower.SqlServer.QueueService.QueueService, Flower.SqlServer.QueueService">
      <constructor-arg name="flowerClient" ref="flowerClient" />
      <constructor-arg name="connectionStringName" ref="FlowerQueueService" />
    </object>
  </property>
  
</object>


By default, the queue service uses netTcpBinding and is available at 'net.tcp://localhost:10002/Flower/QueueService'. You can edit Flower.Services.Host.exe.config to change it.

The queue service can use various persistent storage providers, but currently Flower comes with only one - SQL Server based (Flower.SqlServer.QueueService.QueueService, Flower.SqlServer.QueueService). To use this implementation, you need SQL Server to be installed on the queue service host. In the Scripts folder of the installation package you will find a script Flower.SqlServer.QueueService.sql. You need to create an empty database and run the script in it. Then, edit Flower.Services.Host.exe.config to specify connection string for the queue service (by default, it is FlowerQueueService connection string).

To make a queue service the default queue of a processor, specify its client in the processor's configuration in the directory. Define and object 'queue' with the type 'Flower.Client.QueueServiceClient, Flower.Client'.

<object id="queue" type="Flower.Client.QueueServiceClient, Flower.Client">
  <constructor-arg name="binding">
    <object type="System.ServiceModel.NetTcpBinding">
      <!-- Binding properties. -->
    </object>
  </constructor-arg>
  <constructor-arg name="remoteAddress">
    <object type="System.ServiceModel.EndpointAddress">
      <constructor-arg value="net.tcp://hostname:10002/Flower/QueueService" />
    </object>
  </constructor-arg>
</object>

Last edited Oct 28, 2013 at 7:27 PM by dbratus, version 8

Comments

No comments yet.