Ever wonder what actually happens behind the scenes when I spin up a new server or workstation? In this video, I’m giving you a behind-the-scenes look at my full deployment workflow — covering how I provision cloud VPS servers, configure them with Ansible, deploy workstations, and manage it all through a consistent configuration management approach.

Ansible is a key piece of this puzzle, but this video is really about the bigger picture: how all the pieces fit together to make server and workstation deployment fast, repeatable, and low-maintenance.

My Automation System

To get us started, I want to spend a bit of time talking about my automation system. It’s called Config-a-ma-jig, or Camj for short, and it’s an Ansible-based solution that automates just about everything in my environment.

Now, I’m not going to go over it in exhaustive detail here, mainly because I already created a dedicated video some time ago where I walked through the entire system in depth. Instead, I want to focus on some of the highlights — not for bragging rights, but because understanding the design decisions behind a system like this can actually teach you quite a bit about automation and infrastructure planning in general.

Also, throughout this video I’ll probably reference Camj several times, so I figured it makes sense to give you a bit of background on it first.

It’s Based on Ansible

The first thing I’ll mention is that Camj is built on top of Ansible. That probably won’t come as much of a surprise, since I’ve covered Ansible extensively on the channel over the years.

Before settling on Ansible, I previously used solutions like Puppet and Chef. And while those tools are certainly capable, they eventually felt too bloated and overly complex for what I wanted to accomplish. Over time, Ansible ended up becoming my automation platform of choice because of its simplicity, flexibility, and the fact that it doesn’t require a heavy infrastructure footprint.

For my implementation specifically, I decided to use ansible-pull, which is actually a lesser-known feature of Ansible itself. Instead of relying on a central control node that pushes configurations out to machines, ansible-pull works the opposite way — each machine pulls down its own configuration locally and applies it directly on the machine itself.

I really like this approach because it eliminates a central point of failure. Each machine is capable of configuring itself independently, and the workflow ends up being very resilient as a result.

The way I have everything set up is that whenever I make changes to my configuration repository, each machine will pull down the latest configuration and apply it automatically according to a schedule.

It’s Fully Configurable

Another major design goal for Camj was flexibility. Every machine in my environment can behave differently depending on its role.

Each machine has its own variable file, and if I don’t create one, a default configuration is used automatically instead. That gives me a ton of flexibility without requiring duplicate playbooks or complicated logic everywhere.

For example, I can configure things like:

• which applications are installed
• when automatic updates occur
• whether ZRAM is enabled
• how frequently the machine checks in
• and much more

That last point is especially useful because some systems are more critical than others, so I can have certain machines check in much more frequently while allowing less important systems to run on a slower schedule.

It Tries to Waste as Few Resources as Possible

Another thing that was important to me was efficiency. I didn’t want machines constantly burning CPU cycles for no reason.

So one of the optimizations I implemented is that a provision run will only occur if changes are actually detected in the configuration repository. That way, systems aren’t repeatedly running automation tasks throughout the day when absolutely nothing has changed.

This helps keep resource usage low and avoids unnecessary CPU spikes caused by redundant provisions.

Alerting

Another major design consideration was alerting. I wanted complete visibility into what the system was doing at all times.

Whenever a provision runs successfully, I receive a notification on my phone. And of course, I also receive alerts whenever something fails.

But one thing I specifically wanted to protect against was silent failures — situations where a system quietly stops checking in and you don’t notice for days or even weeks.

To solve that, I integrated Camj with Healthchecks.io which alerts me whenever a machine hasn’t checked in within an expected amount of time.

I also wanted to avoid stale configurations building up over time, so I configured Camj to wipe its cache on a weekly basis. That forces a full provision run at least once each week, even if no configuration changes were made.

It’s Extensive

And finally, the last thing I’ll mention is that Camj is extremely extensive at this point.

I’ve been building and refining it for well over a decade, so over time I’ve automated just about everything I could think of.

On a completely fresh Linux installation, running Camj can result in well over 300 individual changes being applied automatically.

And it’s not limited to servers either — it fully configures my workstations as well. Things like GNOME keyboard shortcuts, desktop settings, wallpaper configuration, mouse cursor speed, application preferences, and much more are all handled automatically.

At this point, provisioning a new machine has become less about configuring things manually and more about simply letting the automation do its job.

Deploying a Server

So, what we’ll do now is take a look at the actual process I go through whenever I deploy a brand new instance within my environment. I’ll show you the workflow on both a server and a workstation, but let’s start with a server first.

Now that I have a fresh server deployed, the next step is deciding which distribution group it belongs to within my automation environment.

Right now, I maintain three different groups:

• Development
• Production
• Stable

The Development group is where all active development happens. Any machine assigned there will receive the in-development version of Camj, including whatever new features or changes I’m currently working on.

Production receives finalized configurations that have already been tested in Development. So by the time something reaches Production, it’s generally considered safe and reliable for day-to-day use.

And then finally there’s Stable, which intentionally stays one version behind Production. That may sound strange at first, but it’s actually incredibly useful for mission-critical systems where stability matters more than getting the latest changes immediately. By delaying updates there, I get an additional layer of long-term testing before important systems receive modifications.

Once I’ve decided which group the new server belongs to, the next step is creating a profile for it on my configuration server.

Every machine in my environment has its own profile, which contains machine-specific settings and variables. To make things easier, I usually start by copying an existing profile that’s similar to what I need and then simply rename it for the new server.

One important detail here is that the hostname has to match properly, since the automation uses that information to determine which configuration belongs to which machine.

After that, I’ll also update the node assignments so the machine is associated with the correct distribution group and automation policies.

And once all of that is done, the final step is bootstrapping the machine itself. To do that, I’ll securely copy my kickstart files over to the new server and execute the bootstrap process.

At that point, the automation takes over and begins transforming a completely fresh Linux installation into a fully configured system according to the role I’ve assigned it.

Deploying a Workstation

When it comes to workstations, I actually follow almost the exact same process. But honestly, this part is going to be a lot more fun to watch because you’ll get to visually see Camj configure an entire machine right before your eyes.

Right now, what we have here is a completely fresh installation of Fedora Workstation with everything still at its defaults. No customizations, no tweaks, no applications installed beyond the standard defaults — basically a clean slate.

And I want you to pay close attention to how everything looks right now, because in a few minutes this system is going to look very different.

So at this point, what I’ll do is copy over my kickstart files and start the bootstrap process, just like I showed you earlier on the server deployment.

And actually, to make this a bit more fun, before I press Enter I’m going to open up a few windows so you can more easily watch the changes happen in real time. I’ll open the file manager, the settings application, and a few other things so we have something visual to keep an eye on during the process.

Then once I kick off the provision job, I’ll speed up the footage in post-production all the way up to the point where you’ll start noticing visual changes happening automatically.

And trust me, once the automation starts applying everything, you’ll very quickly begin to see just how extensive this setup actually is.

Additional Thoughts and Lessons

To close out the video, I want to return to the main theme we’ve really been focusing on throughout all of this, which is design philosophy and long-term thinking when it comes to automation.

You’ve now seen my automation system in action, you’ve seen how I deploy servers and workstations, and you’ve gotten a behind-the-scenes look at how my environment works overall. But beyond the technical details, there are also some important lessons to take away from all of this.

Understand the Scope

And the first lesson is this: never underestimate the scope of automation.

Building a good automation system is a massive undertaking. Over the years I’ve worked professionally with Linux and I’ve also helped companies implement automation solutions of their own. And honestly, one of the biggest reasons these projects fail is because people assume automation is a one-time project that can be fully completed within a year.

In reality, automation is never truly finished.

New Linux distribution releases will come out and require adjustments. New software versions will introduce changes. You’ll constantly think of new features and improvements to implement. And over time, scope creep naturally happens because your ideas continue evolving as your environment grows.

So maintaining automation becomes an ongoing process rather than a project with a final endpoint.

Now the good news is that once things are in place, maintaining an automation platform usually doesn’t require an overwhelming amount of work. But it absolutely does require consistency and ongoing attention.

Whether you’re building automation for your workplace or simply for your homelab, this is the approach I personally recommend.

Start by creating a master list of everything you’d eventually like to automate. Then, anytime you think of a new idea, add it to the list instead of immediately jumping into implementation.

After that, pick only five items from the list to focus on at a time. And don’t allow yourself to work on additional features until those five tasks are complete.

Once those are finished, pick five more.

Then repeat the process over and over again.

This approach works incredibly well because your automation platform continues growing steadily over time, but at the same time you avoid overwhelming yourself or constantly getting sidetracked by new ideas. Instead of chasing every new thought immediately, you maintain focus while still capturing future improvements for later.

And before you know it, years go by and you suddenly realize you’ve built something incredibly powerful.

Implement CI/CD

Another extremely important lesson is implementing CI/CD, which stands for Continuous Integration and Continuous Deployment.

In simple terms, this means you should always test your configuration changes automatically before they ever touch production systems.

The way I personally handle this is by maintaining reference servers for each Linux distribution supported by Camj. These systems pull configuration changes directly from the development branch and check for updates very frequently.

Whenever I commit configuration changes, those systems automatically test everything and report whether or not the provision run succeeded.

Once all of those systems report success, only then do I promote the changes into production.

And trust me, this is incredibly important.

A single typo inside an automation system can completely break a machine. I’ve absolutely had situations where bad configurations caused systems to fail, which is exactly why my reference servers are disposable by design. If something breaks there, it’s not a problem.

Now, enterprise CI/CD systems are often far more advanced and sophisticated than what I’m doing here, but the core principle is the important part:

Production systems should never be the first place your configuration changes are tested.

Use Variables — A Lot

Next, I want to quickly talk about variables, because variables are even more important in automation than many people initially realize.

At a basic level, variables give you placeholders for values that can differ between machines.

For example, maybe you want certain users to exist only on particular systems. Rather than creating separate playbooks for every scenario, you can simply use variables to define which users belong on which machines.

Now, that’s the obvious use case, and it’s usually one of the very first things people do when they start learning automation.

But honestly, you should go much further than that.

Use variables everywhere you reasonably can, and actively look for creative ways to make your automation more flexible and reusable.

A great example is package management across different Linux distributions.

For instance, if you want to install Apache on AlmaLinux, the package name is typically httpd. But on Debian, the package name is usually apache2.

What you should not do is create separate Apache playbooks for every distribution you support.

Instead, create a variable for the package name itself. Then your automation can dynamically substitute the correct package depending on which distribution the machine is running.

That keeps your automation cleaner, easier to maintain, and dramatically more scalable over time.

Be Consistent

And finally, one of the most important mindsets you can develop when building automation systems is consistency.

Come up with a consistent directory layout. Use predictable naming conventions. Organize things logically and keep that structure uniform throughout your environment.

Because over time, your automation platform is going to grow much larger than you initially expect. There will eventually be files, roles, scripts, and configurations that you don’t touch for months or even years at a time.

And when that happens, consistency becomes incredibly valuable because it allows you to quickly find what you’re looking for without having to mentally reverse-engineer your own infrastructure every time you revisit it.

Good organization may not feel exciting when you’re first building automation, but years later you’ll be very thankful you took the time to structure things properly from the beginning.