HCI - Holistic CI

What is software development?

Software development is about writing code, building, testing and deploying. You have to manage source code repositories, compilers, dependency libraries, infrastructure with computers, operating systems, issue tracking and storing release artifacts.

Later on you have to update the development environment with new versions of the compiler, library updates and security upgrades on the build server. You probably hesitate, as you have a big release to your new customer and the time schedule is already tight.

Your customers are happy, life is great!

But eventually, one of your big customers, running the stable old release, finds an ugly bug. The customer demands a fix, and insists they can’t upgrade to the new release, as that risks adding new behaviors and problems. You just have to solve the bug! The problem is, that you can’t use the new development tools, as the old release is built with something else. The old versions have to be dusted off and installed, without colliding with the new ditto.

Handling many variants of your product soon becomes a pain. Handling many variants of the environment is nearly impossible. Your developers grumble, it’s not fun anymore.

The above story is typical. The traditional way development environments evolve, is that they grow by the demands from the development department. You start small and step by step add new parts and processes, solve problems, upgrade some server OS, etc. Eventually the entire environment grows into an old troll that you rather not touch.

The holistic approach

Holistic CI (HCI) takes the comprehensive view approach to Software Development. It considers the bigger picture of what a CM really should be doing.

With HCI you have

• a model of your infrastructure, as it evolves.
• a model of your application development environments, from writing code to deployment, as they evolve.

Benefits

The biggest benefit is the vastly lowered risk of disruptions for your software development projects. You also get:

• Easier to visualize your environment (easier, not easy)
• High configurability - choose your own software development tools
• Maintainability - every change is recorded
• Deployment directly from models

What can be achieved

• Controlled changes in the infrastructure
  • OS-versions/patches/distributions
  • Number of computers
  • File server mounts
• Backup/restore of:
  • Issue tracking database
  • Version control repositories and metadata (such as pullrequests)
  • Artifact database
• New/other toolchain
• Preparation of changes of infrastructure - CM workbench/test environment
• Going backwards
  • Reverting toolchains
• Multiple development environments
• Multiple infrastructure sets
  • Virtual
  • Physical

Solution

How is this possible?

HCI achieve encapsulation of infrastructure installations, toolchains and applications by using lightweight containers. The management and deployment of environments are done with a provisioning tool.

Infrastructure model

The infrastructure model helps you manage the computers, used for development. You may setup multiple sets of infrastructure, e.g. for production and for test, virtual machines or physical ones.

The possibility to setup multiple infrastructure sets eliminates the risk of the CM (build engineer) introducing bugs into the production environment, which affects the developers and testers of your product.

The evolving infrastructure model is version controlled.

The hard requirements on the infrastructure nodes are that they can run Docker, Python 3 and host an SSH server.

(In the first release HCI will require Debian or Ubuntu as node operating system.)

Application environment model

The application environment is modelled separately from the infrastructure. It enables controlled upgrades and additions to the environment. The environment consists of the source code version control server, issue tracker, build servers, artifact servers, etc.

Your product might require additions to this, like extra test applications.

Application Build Recipe

The Application Build Recipe (ABR) defines the toolchain, source code and any external build dependencies. The ABR is a concept that enables you to have different toolchains for a specific source code version, or control the versions of any dependency function library.

Deployment

Deployment of HCI is accompanied by training of your development department, so that it can operate efficiently in the new environment, and develop with it over time.

License

HCI is based fully on open source products, which prevents lock in. HCI is modifiable by the customer. If your product development requires any closed source, proprietary tools, you manage them in separated Docker containers.

All source code of HCI is provided.

Example

The following is an example deployment diagram.

Demonstration

Send an email to info@pipegrep.se to call for a demonstration.