Hey Readers!  I have somewhat of an academic post for you all today to help document a topic that has come up from time to time around System Center 2012 Orchestrator and service dependencies.  Explained in detail below but the gist to this scenario is basically that servers are rebooting faster due to improvements in bootup processes, faster hardware, etc.  Due to this, certain core services (in this case the Runbook Server Service) may not start properly after reboot due to other dependent services starting in either a delayed fashion or just not quick enough.

Service Configuration for System Center 2012 Orchestrator

System Center 2012 Orchestrator by default will install with all services set to Automatic. Dependencies are lacking due to the fact that SQL cannot always be guaranteed to be installed locally on the Management Server. The following breaks down some advanced configuration that will need to be done post installation to ensure that core services for Orchestrator will properly start upon a reboot of a Orchestrator Runbook or Management server (given your SQL server is installed on the same server as your management and Runbook Server).

Note     Who would do this you ask (install everything on one box)?  Depending upon the environment, it may make sense to co-locate all components on one server.  Consider a development environment or a single purpose / project all in one Orchestrator configuration.  If nothing else, this pulls back a bit of the information necessary to understand when the Runbook Server Service fails to start upon reboot.  

Additional Important Note   This is a documented workaround that will need to be tested within your own environment and mileage may vary.  It worked great for the scenario I needed it for  - so I’m passing it on as something you may want to try if facing the same situation under similar circumstances.

Service Dependency Configuration

The Orchestrator Runbook Service depends on SQL to be running before it can launch.  The issue comes into play when the Management Service (which controls restarting Policies) starts up and the Runbook Service (controls what Runbook server to run the policies on) doesn’t.  You end up with policies firing and appearing to be running but not actually doing anything.

Therefore, going into the Runbook Designer, you can see the server connection up (SQL started eventually) and the Management Service running (lets you into the designer without issue).  You look at your Runbooks and see them running but if you go to the “events” tab, you’ll notice an error stating that the Runbooks are not actually running because no Runbook servers are available.

Additionally, the Runbook service itself has no dependencies built in to require SQL to start first.  Finally, the service itself is not set to Automatic (Delayed Start) but instead set to just Automatic.

Starting with Windows Server 2008 (to improve boot up performance), certain services will load out of order or later on to speed up the boot time process.  Certain services now perform better if configured for Automatic(Delayed).

How to Setup the Orchestrator Runbook Service to be More Resilient

1. Create a service dependency on SQL Server for the Runbook service.
   
   sc config orunbook depend= MSSQLSERVER
   
   Note        The space in the command above is important after the equal sign!  More information: http://technet.microsoft.com/en-US/library/cc990290.aspx
   
2. Set both SQL and the Runbook Service to delayed start to ensure they start once all other services are up and running
3. Update the primary Runbook server to also have a delayed start on its service since SQL is not local to it.
4. Finally put a recovery option on the service to restart on first and second failures to ensure they have an even better rate of success in service startup.

That’s it!  Quick and simple.  Hopefully it helps at least one automation admin out there looking to remediate this pesky issue .

Till next time, Happy Automating!