How CIOs View Workload Automation

New research into workload automation

Not so long ago, enterprises turned to job scheduling platforms primarily to execute well-defined tasks based on static schedules. Posting a job to run at 3:00 a.m. every other Wednesday was often as challenging as things got.

Job scheduling is still handling such tasks, of course, but a new era in business computing is upon us—and more advanced workload automation applications are helping lead the change. Today these newer solutions, going beyond the background solutions of the past, are used for increasingly strategic and mission-critical purposes as enterprises look to consolidate their IT automation, increase agility, and decrease operational costs.

The new workload automation approach is dynamic and supports more heterogeneous workloads. Unlike traditional date or time scheduling, these workloads can be triggered by any number of events, executing jobs across disparate systems in near real-time.

As workload automation has advanced, so has the thinking behind it. And the more modern method is the “architectural approach” whereby individual tasks are placed in the context of the broader computing landscape. Under this advanced strategy, one-off tasks are managed nimbly and with a sense of an enterprise’s total needs—one that puts systems and processes first. Even Big Data processing jobs can be executed by assembling a cost-effective mix of internal, virtual, and cloud resources on an ad hoc basis, according to specific business rules.

In an effort to measure just how deeply the architectural approach has impacted IT, Advanced Systems Concepts Inc. conducted a survey of CIOs and other IT decision makers at over 100 of the top companies across the U.S. The results of the survey demonstrate how widespread of a change is occurring in workload automation use:

Workload Automation is Now a Core Enterprise Application. Among those surveyed, 63%—nearly two out of three respondents—reported that their organizations use their workload automation solution on a daily basis to automate their business and IT operational processes. Nearly nine out of ten organizations report working with it at least several times each week.

Considering the need for real-time/near real-time data processing in today’s enterprise, it’s easy to see why workload processing is moving to the center of IT governance. Big Data, self-service computing, and the proliferation of mobile devices are all on the rise. As a result, SLAs are being imposed on IT management to make sure critical functions are carried out as expected. Workload automation can ensure that such requirements are met consistently and reliably.

Workload Automation Brings Efficiency to More Processes Than Ever Before. When asked what types of processes they use their workload automation platform to manage, over a dozen classes of processing were mentioned. Scripts were number one (mentioned by 85% of respondents), followed by Batch Processes (84%), File Operations (77%), Managed File Transfers (73%), Business Processes (68%), IT Operations (64%), Reporting/BI (50%), ETL/Data Warehousing (47%), Data Center Processes/Backups (40%), Runbook Automation (39%), and DevOps/QA/Release Management (17%).

Due to the large number of scripts, workload automation users benefit most from a solution that can absorb the complexity of custom script creation. Importantly, the solution must offer a wide variety of job steps as well as integrations with mission critical apps like SAP, PowerCenter, Oracle (E-Business Suite, PeopleSoft), Microsoft, and Web Services, so that their methods can be included within vendor-provided template jobs or activities. These “template jobs” or “job steps” can then be used to avoid scripting in the first place. Since the job steps contain all the necessary proven logic, all IT needs to do is add the required data.

An important note: while Business Intelligence (BI) may have ranked a bit lower in the survey results, it is among the fastest-growing and most vital IT functions. Line managers in every corner of business today are using BI to make important decisions. As such, there is huge pressure on workload automation to provide the complex workflows necessary to deliver current and accurate data.

The democratization of BI and reporting solutions, combined with the increasing volume and formats of data, means that the underlying ETL/data warehousing processes are more important than ever. In response, enterprises are leveraging workload automation to ensure data quality, improve time-to-insight, and reduce time spent managing resources.

Enterprises Continue to Seek New Workload Automation Opportunities. While job scheduling or reactive automation involving bolted-on dependencies and scheduling will never completely go away, workload automation or predictive task scheduling based on historical analysis is gaining ground. This strategy, which looks at precedent, ensures that the necessary combination of internal, virtual, and cloud resources are assembled at the precise time needed to meet workflow SLAs.

In the more intelligent “predictive” model, resources are assembled and provisioned as needed to accomplish tasks under circumstances that best fit business needs. Lower-priority jobs may even be delayed or re-assigned in order to accommodate more pressing tasks as the need arises. Respondents in the survey affirmed their desire for information about these advanced techniques, rating Tips and Best Practices (83%) and Knowledge Base Articles (77%) as either “very important” or “extremely important” to their future needs. While provisioning is usually considered within the context of meeting workflow SLAs, the term has now evolved to include Cloud and Virtual resources on a cost-efficient basis.

In the new era of workload automation, rules and strategic priorities, rather than schedules and structures, are driving innovation. Vendors simply cannot solve the complexity problem by introducing more complexity, i.e. building and maintaining custom scripts. Rather, vendors need to provide a simpler, easier, and more scalable/streamlined approach that allows the use of these integrations with little to no scripting.

The new workload automation overcomes the static and manual nature of job scheduling to provide a unified automation solution that can manage dependences across multiple platforms and applications. This architectural approach is enabling IT organizations to dynamically manage any number of point solutions as well as handle policies and service-level objectives. Intelligent workload automation is improving visibility, control, and responsiveness across the organization—and better IT governance is the result.


Ben Rosenberg is President and founder of Advanced Systems Concepts, Inc., the maker of ActiveBatch®, one of the world’s leading workload automation software products