While humankind has come a long way in automating software development and quality assurance processes, the current trends are rather worrisome. The singular purpose of automation is to remove a requirement for the error-prone manual intervention, freeing valuable human time and cognitive capacities to do bigger and better things, i.e. creating the actual product for which the enterprise was established. We assert that the two main paths the industry is taking are failing to deliver on that purpose.

With the first path, there is a plethora of open-source tools and utilities that offer a wealth of standalone functionality albeit requiring integration. While some of these open-source projects are well-maintained and thoroughly thought through for a long-term general applicability, a major portion of these are spur-of-the-moment projects, with no declared quality or longevity goals whatsoever. In fact, some open-source projects pride themselves on community and component churn and frequent obsolescence of plugins and components, emphasizing lack of governance, resulting in extremely poor quality control.

As a result, a large percentage of open-source projects go defunct rather quickly, often not due to the lack of utility in and of itself but due to narrowness of scope limiting said utility or simply the pursuit of ongoing fads of the moment. An open-source project is often started to satisfy a specific and immediate need at hand. Once the need is satisfied the majority do not progress to evolve and are often abandoned as the needs of the businesses and personal interests change. If a large stable contributing audience is not established, the maintenance lapses even if the consuming audience is substantial, leading to the latter slowly dissolving and the project going dormant.

If you rely on a project like this in implementing your automation, you’ll be either stuck maintaining it yourself or having to spend time moving to something else. Lather, rinse, repeat ad nauseam.

Even if the open-source project is well-tended, the modern patterns of rapid development result in constant component and dependency versioning churn, with frequent rapid abandonment of backward compatibility (the Agile way).

If you’re integrating multiple open-source products, product plugins, and augment your solution with a healthy sprinkle of standalone tools, the end result is always the same: as different components evolve, die-off and break at rapid rates you will end up having to build a team maintaining large process infrastructure, plugging the gap in lapsing functionality with your own expensive expert staff, experiencing unexpected outages every upgrade, security compromises etc. Now that you’re spending so many resources maintaining what you have, you will be afraid to touch your infrastructure to continue to optimize your development and delivery processes. Then the implementation and architectural entropy kicks in, people leave, and after a while nobody knows how your automation works.

This scenario repeats itself over and over again across the industry, especially in large shops and for products that undergo rapid expansion. You have simply shifted the burden from your system administrators having to maintain scripts and intervening manually to more expensive automation engineers maintaining someone else’s and their own code and scripts and intervening manually. This result is not automation. This is anti-automation.

With the second path, you have large hosted service providers that provide “automation” in a form of loosely coupled services – queues, databases, IaaS, PaaS etc. Such services, being loosely coupled, inherently requiring integration. Often not being open-source, they evolve in the direction that delivers most revenue to the provider and have limited, if any, customizability. Such services, inherently, also bind you “for life” to that specific service provider, unless, of course you invest in switching.

Since you have to develop all your service integration and required flexibility by yourself, you’re now stuck with a team of automation engineers supporting your third-party integrations with orchestration, provisioning, monitoring, unified security realm etc. The lack of flexibility further limits your ability to innovate and optimize your processes, relying on the graces of your service providers. This too, of course, is anti-automation.

So, is there a better way? We believe there is. And we intend to answer this question by demonstrating that flexible, highly available, self-provisioning, self-healing, secure and open-source automation infrastructure with impeccable quality control is possible.

Karellen is ushering in the Golden Age of Automation. There is no need to be afraid.