The Technical Blog of James: Recent Episodes
None
I’ve decided to move my blog to a static site. It’s now available at: https://purpleidea.com/blog/ If you find any issues or would like to make any improvements, all of the code is available in git: https://github.com/purpleidea/purpleidea.com/ I’ve replicated most of … Continue reading →
As you may have noticed, we recently re-licensed mgmt from the AGPL (Affero General Public License) to the regular GPL. This is a post explaining the decision and which hopefully includes some insights at the intersection of technology and legal … Continue reading →
I have a short movie that I imported into a libreoffice presentation. I wanted a copy of that movie back, but I couldn’t figure out how to extract a copy. In desperation, I figured I’d try opening the file with … Continue reading →
Recently, while operating two different remote-controlled appliances, I realized that it was high time for a discussion about declarative and imperative paradigms. Let’s start by looking at the two remotes: At first glance you will notice that one of these … Continue reading →
LaForge and the fine folks at Osmocom (Sysmocom) recently had a femtocell giveaway. I didn’t expect to have much time to hack on things, but they were still quite generous in sending me one. It arrived, and I took some … Continue reading →
In mgmt we have meta parameters. They are similar in concept to what you might be familiar with from other tools, except that they are more clearly defined (in a single struct) and vastly more powerful. In mgmt, a meta … Continue reading →
I’ve been hacking in golang since before version 1.4, and the speed at which my builds finished has been mostly trending downwards. Let’s look into the reasons and some fixes. TL;DR click-bait title: “Get 4x faster golang builds with this … Continue reading →
I’m old school and still rocking an old X220 laptop because I didn’t like the new ones. My battery life isn’t as great as I’d like it to be, but it gets worse when some “webapp” (which I’d much rather … Continue reading →
I previously published “A revisionist history of configuration management“. I meant for that to be the intro to this article, but it ended up being long enough that it deserved a separate post. I will explain Send/Recv in this article, … Continue reading →
I’ve got a brand new core feature in mgmt called send/recv which I plan to show you shortly, but first I’d like to start with some background. History This is my historical perspective and interpretation about the last twenty years … Continue reading →
Bootstrapping a cluster from your laptop, or managing machines without needing to first setup a separate config management infrastructure are both very reasonable and fundamental asks. I was particularly inspired by Ansible‘s agent-less remote execution model, but never wanted to … Continue reading →
The mgmt config project got a logo! The full commit is here. Thanks to Sarah Jane Cox for creating it. Happy Hacking, James PS: I might have a few stickers to give out too! Ask me next time you see … Continue reading →
All good sysadmins eventually learn about using tail -F to tail files. Yes upper-case F is superior. Around the time I wrote that article, I remember wanting to stream dmesg output too! The functionality wasn’t available without some sort of … Continue reading →
The Technical Blog of James was seen on an outdoor electronic display in downtown Montreal! Thanks to one of my readers for sending this in. If you’ve got access to a large electronic display, put up the blog, snap a … Continue reading →
I was stuck on a 14 hour flight last week, and to my disappointment, only one of the two headphone speakers were working. The plane’s media centre has an audio connector that looks like this: The hole to the left … Continue reading →
In mgmt, deploying and managing your clustered config management infrastructure needs to be as automatic as the infrastructure you’re using mgmt to manage. With mgmt, instead of a centralized data store, we function as a distributed system, built on top … Continue reading →
I’m thrilled to tell you that I’ll be speaking about mgmt in Hong Kong and South Africa. It will be my first time to both countries and my first time to Asia and Africa! In Hong Kong I’ll be speaking … Continue reading →
I have a GNU/Linux server which I mount a few LUKS encrypted drives on. I only ever interact with the server over SSH, and I never want to keep the LUKS credentials on the remote server. I don’t have anything … Continue reading →
In this post, I’ll tell you about the recently released “automatic grouping” or “AutoGroup” feature in mgmt, a next generation configuration management prototype. If you aren’t already familiar with mgmt, I’d recommend you start by reading the introductory post, and … Continue reading →
It’s been two months since I announced mgmt, and now it’s time to continue the story by telling you more about the design of what’s now in git master. Before I get into those details, let me quickly recap what’s … Continue reading →
On today’s issue of “one hour hacks”, I’ll show you how you can stop your git drive-by’s to git master from breaking your CI tests… Let’s continue! The problem: Sometimes I’ve got a shitty one-line patch that I want to … Continue reading →
I’ve been writing a lot of golang lately. I’ve hit painful problems in the past. Here are some debugging tips. Hopefully they help you out. I bet you don’t know #2. #0 Use log.Printf: This should go without saying, but … Continue reading →
It’s no secret to the readers of this blog that I’ve been active in the configuration management space for some time. I owe most of my knowledge to what I’ve learned while working with Puppet and from other hackers working … Continue reading →
I’ve got a few upcoming speaking engagements. If you’ll be attending one of these events, come see me or any of the other excellent speakers! Please remember to check the official schedules in case there are any changes! I’ll be … Continue reading →
Daniel P. Berrangé wrote about trying out a single node ceph cluster. I decided to take his article and turn it into an Oh-My-Vagrant omv.yaml file. It took me about two minutes to do so, and two hours to debug … Continue reading →
We’re constantly clicking on all sorts of different URL’s throughout the day. These clickable links appear in webpages (including in “web apps” like gmail) in mail clients like Evolution, in terminals such as GNOME-terminal, and any other GTK+ app on … Continue reading →
The Oh-My-Vagrant project became public about one year ago and at the time it was more of a fancy template than a robust project, but 188 commits (and counting) later, it has gotten surprisingly useful and mature. james@computer:~/code/oh-my-vagrant$ git rev-list … Continue reading →
My employer keeps paying me, which I appreciate, so it’s good to spend some time to make sure RHEL7 customers get a great developer experience! So here’s how to make vagrant, vagrant-libvirt and Oh-My-Vagrant work on RHEL 7+. The same … Continue reading →
I am definitely not an RPM expert, in fact, I’m afraid of it, but with recent tools such as COPR, and my glorious Makefile, some aspects of it have become palatable. This post will be about a recent journey I … Continue reading →
I’ve been hacking in golang for a while, but I’ll admit that I didn’t get too deep into some of the language nuances until more recently. Since some of them have started to bite me, here’s a little post-mortem of … Continue reading →
Git submodules are actually a very beautiful thing. You might prefer the word powerful or elegant, but that’s not the point. The downside is that they are sometimes misused, so as always, use with care. I’ve used them in projects … Continue reading →
Making Oh-My-Vagrant (OMV) more developer accessible and easy to install (from a distribution package like RPM) has always been a goal, but was previously never a priority. This is all sorted out now. In this article, I’ll explain how “mainstream” … Continue reading →
In this new crazy world of containers and immutable hosts, one might still want to run previous generation software such as Puppet on a current generation Atomic host. This article will explain how you can do that, and offer some … Continue reading →
I’ve added the ability to deploy a Kubernetes cluster with Oh-My-Vagrant (omv). I’ve also built an automated developer experience so that you can test your Kubernetes powered app in minutes. If you want to redeploy a new version, or see … Continue reading →
The Oh-My-Vagrant (omv) project is an easy way to bootstrap a development environment. It is particularly useful for spinning up an arbitrary number of virtual machines in Vagrant without writing ruby code. For multi-machine container development, omv can be used … Continue reading →
With Oh-My-Vagrant (omv) you can set up a dev environment in seconds. (Read the omv introduction if you’ve never used it before!) Since everything is defined in a single omv.yaml file, it is easy to share your cluster prototype with … Continue reading →
Here are two quick git tricks that I’ve added to my toolbox lately… I wanted to create a git alias that takes in argv from the command, but in the middle of the command. Here’s the hack that I came … Continue reading →
Vagrant is a great tool for development, but Red Hat Enterprise Linux (RHEL) customers have typically been left out, because it has been impossible to get RHEL boxes! It would be extremely elegant if hackers could quickly test and prototype … Continue reading →
You might want to write code that can tell how many iterations have passed since some action occurred. Alternatively, you might want to know if it’s the first time a machine has run Puppet. To do these types of things, … Continue reading →
I use the Evolution mail client. It does have a few annoying bugs, but it has a plethora of great features too! Hopefully this post will inspire you to help hack on this piece of software and fix the bugs! … Continue reading →
Recently, when I tried to access http://slashdot.org/ in Firefox, I would see my browser title bar flash briefly to “AT&T GUI”, and then I would get redirected to: http://slashdot.org/cgi-bin/redirect.ha which returns slashdot’s custom error 404 page! What is going on? (Read … Continue reading →
The fact that we manage the switches in our data centres differently than any other server is patently absurd, but we do so because we want to harness the power of a tiny bit of silicon which happens to be … Continue reading →
I’ve wanted a feature in Evolution for a while. It was formally requested in 2002, and it just recently got fixed in git master. I only started publicly groaning about this missing feature in 2013, and mcrha finally patched it. … Continue reading →
I had an itch to scratch, and I wanted to get a bit more familiar with Openshift. I had used it in the past, but it was time to have another go. The app and the code are now available. … Continue reading →
I just patched puppet-gluster and puppet-ipa to bring their infrastructure up to date with the current state of affairs… What’s new? Better README’s Rake syntax checking (fewer oopsies) CI (testing) with travis on git push (automatic testing for everyone) Use … Continue reading →
I usually avoid proprietary cloud services because of freedom, privacy and vendor lock-in concerns. In addition, there are some excellent libre (and hosted) services such as WordPress, Wikipedia and OpenShift which don’t have the above problems. Thirdly, there are every … Continue reading →
If you’re a reader of my code or of this blog, it’s no secret that I hack on a lot of puppet and vagrant. Recently I’ve fooled around with a bit of docker, too. I realized that the vagrant, environments … Continue reading →
Seagate has just publicly announced 8TB HDD’s in a 3.5″ form factor. I decided to do some rough calculations to understand the density a bit better… Note: I have decided to ignore the distinction between Terabytes (TB) and Tebibytes (TiB), since … Continue reading →
(Note: this hybrid management technique is being demonstrated in the puppet-ipa module for FreeIPA, but the idea could be used for other modules and scenarios too. See below for some use cases…) The error message that puppet hackers are probably … Continue reading →
Master SDN hacker Flavio sent me some tunes. They were sitting on my desktop in a folder: $ ls ~/Desktop/ uncopyrighted_tunes_from_flavio/ I wanted to listen them while hacking, but what was the easiest way…? I wanted to use the nautilus … Continue reading →
Configuration management is an essential part of securing your infrastructure because it can make sure that it is set up correctly. It is essential that configuration management only enhance security, and not weaken it. Unfortunately, the status-quo of secret management … Continue reading →
Earlier this year, R.I.Pienaar released his brilliant data in modules hack, a few months ago, I got the chance to start implementing it in Puppet-Gluster, and today I have found the time to blog about it. What is it? R.I.’s … Continue reading →
When GNOME shell breaks, you get to keep both pieces. The nice thing about shell failures in GNOME 3, is that if you’re able to do a restart, the active windows are mostly not disturbed. The common way to do … Continue reading →
Vagrant has become the de facto tool for devops. Faster iterations, clean environments, and less overhead. This isn’t an article about why you should use Vagrant. This is an article about how to get up and running with Vagrant on … Continue reading →
This is a quick trick for making working with git submodules more magic. One day you might find that using git submodules is needed for your project. It’s probably not necessary for everyday hacking, but if you’re glue-ing things together, … Continue reading →
So this happened: RedHat made me an offer, and I am happy to say that I have just started this week! I am proud to have joined a company that employs many of the worlds foremost, highly professional and clever … Continue reading →
I’ve been afraid of RPM and package maintaining [1] for years, but thanks to Kaleb Keithley, I have finally made some RPM’s that weren’t generated from a high level tool. Now that I have the boilerplate done, it’s a relatively … Continue reading →
Puppet is missing a number of much-needed features. That’s the bad news. The good news is that I’ve been able to write some of these as modules that don’t need to change the Puppet core! This is an article about … Continue reading →
I use gnome-terminal for most of my hacking. In fact, I use it so much, that I’ll often have multiple tabs open for a particular project. Here’s my workflow: Control+Alt+t (My shortcut to open a new gnome-terminal window.) cd ~/code/some_cool_hack/ … Continue reading →
I’ll be giving a talk at SCALE today about automatically deploying GlusterFS with Puppet-Gluster and Vagrant. I’ll be giving some live demos, and this will cover some of the material from: Automatically deploying GlusterFS with Puppet-Gluster + Vagrant! and it … Continue reading →
To give you a break from the usual GNU/Linux/DevOps/Puppet/GlusterFS drab, I’ve decided to have a go at writing a different kind of technical article. This article will show you how to build the traditional Canadian snow dwelling known as a … Continue reading →
I’m using a Lenovo X201 with 8GiB of RAM. Apart from some minor issues, I’ve been very satisfied with this laptop. It’s over four years old, and so I decided to see what’s available on the horizon. I did not … Continue reading →
As an update to my earlier article, a friend gave me an idea of how to make my $PS1 even better… First, the relevant part of my ~/.bashrc: If you haven’t figured it out, the magic is that the trailing … Continue reading →
I decided to record some screencasts to show how easy it is to deploy GlusterFS using Puppet-Gluster+Vagrant. You can follow along even if you don’t know anything about Puppet or Vagrant. The hardest part of this process was producing the … Continue reading →
I needed a base image “box” for my Puppet-Gluster+Vagrant work. It would have been great if good boxes already existed, and even better if it were easy to build my own. As it turns out, I wasn’t able to satisfy … Continue reading →
The GlusterFS community is having a “test day”. Puppet-Gluster+Vagrant is a great tool to help with this, and it has now been patched to support alpha, beta, qa, and rc releases! Because it was built so well (cough, shameless plug), … Continue reading →
Puppet-Gluster was always about automating the deployment of GlusterFS. Getting your own Puppet server and the associated infrastructure running was never included “out of the box“. Today, it is! (This is big news!) I’ve used Vagrant to automatically build these … Continue reading →
Some fun updates for vagrant hackers… I wanted to use the venerable clustered SSH (cssh) and screen with vagrant. I decided to expand on my vsftp script. First read: Vagrant on Fedora with libvirt and Vagrant vsftp and other tricks … Continue reading →
As I previously wrote, I’ve been busy with Vagrant on Fedora with libvirt, and have even been submitting, patches and issues! (This “closed” issue needs solving!) Here are some of the tricks that I’ve used while hacking away. Default provider: … Continue reading →
Apparently lots of people are using Vagrant these days, so I figured I’d try it out. I wanted to get it working on Fedora, and without Virtualbox. This is an intro article on Vagrant, and what I’ve done. I did … Continue reading →
I’m now syndicated on the Fedora Project planet. If you haven’t read through my blog yet, let me introduce myself, I’m James, and I write The Technical Blog of James. I’m a sysadmin, DevOps/Puppet hacker, I.T./network architect and physiologist. Hi! … Continue reading →
As a follow-up to my original article on recursion in Puppet, and in my attempt to Push Puppet (to its limit), I’ll now attempt some more advanced recursion techniques in Puppet. In my original recursion example, the type does recurse, … Continue reading →
Ironically, one of the reasons that I started writing Puppet code, was so that I could spend more time designing and building, and less time writing documentation. I suppose I’m a victim of my success, because Puppet-Gluster has grown large … Continue reading →
People often ask how to do iteration in Puppet. Most Puppet users have a background in imperative programming, and are already very familiar with for loops. Puppet is sometimes confusing at first, because it is actually (or technically, contains) a … Continue reading →
Hi there, I hope you enjoyed my “Pushing Puppet (to its limit)” talk and demos from Puppet Camp D.C., LISA 2013. As requested, I’ve posted the code and slides. Here is the code: https://github.com/purpleidea/puppet-pushing This module will require three modules … Continue reading →
I’m here at LISA 2013 at the Gluster Community Day. I’ve been asked by Joe Brockmeier to give a little recap about what’s been going on. So here it is! Wesley Duffee-Braun started off with a nice overview talk about … Continue reading →
I’m speaking at LISA 2013, the “Large Installation System Administration” conference. This conference runs all week in Washington. I’ll be giving two talks during the week, and attending at least one BOF. My first talk is on Monday during the … Continue reading →
Here’s a one minute read, about a trick which I discovered today: When running an strace, it’s common to do something like: strace -ppidof <process name> … Continue reading →
Oh, hi there. In case you’re interested, I’ve just made a first release of my puppet-shorewall module. This isn’t meant as an exhaustive shorewall module, but it does provide most of the usual functionality that most users need. In particular, … Continue reading →
I’m usually on IRC, but I don’t often notice incoming pings until after the fact. I had to both write, and modify various scripts to get what I wanted, but now it’s all done, and you can benefit from my … Continue reading →
This October 12th to 14th Montreal hosted the GNOME boston summit. Many thanks to Canonical for sponsoring breakfast, Savoir Faire Linux for hosting a great 6 à 10 with fancy snacks, and RedHat for sponsoring a pool night. What follows is … Continue reading →
Short post, long command… I’ve decided to start showing the current git branch in my PS1. However, since I don’t want to know when I’m on master, I had to write a new PS1 that I haven’t yet seen anywhere. … Continue reading →
In my attempt to push puppet to its limits, (for no particular reason), to develop more powerful puppet modules, to build in a distributed lock manager, and to be more dynamic, I’m now attempting to build a Finite State Machine … Continue reading →
I just shelled out to wordpress.com to buy the No Ads upgrade. I think they’re good people. I hope you all appreciate it. Let me know if you do. Happy hacking, James
I just realized that my Fedora 19 installation didn’t have any of the GNOME games installed by default any more. I guess there’s no love for nibbles. Here’s a quick one-liner to get them all back: $ sudo yum search … Continue reading →
Theron Conrey writes about using: BitTorrent Sync as Geo-Replication for Storage We got a chance to talk about this idea at Linuxcon. I’m not entirely convinced there aren’t some problem edge cases with this solution, but I think it will … Continue reading →
I’m here in New Orleans hacking up a storm and getting to meet fellow gluster users IRL. John Mark Walker started off with a great “State of the GlusterFS union” style talk. Today Louis (semiosis) gave a great talk about … Continue reading →
After hacking away on Monday and Tuesday and meeting fellow nerds IRL, I’ve landed even more changes to puppet-gluster. My git master branch now sits at 47 commits. $ git clone https://github.com/purpleidea/puppet-gluster.git Cloning into 'puppet-gluster'... remote: Counting objects: 317, done. … Continue reading →
Continuing on from yesterday, I’ve met even more interesting people. I chatted with Dianne Mueller about some interesting ideas for gluster+openshift. More to come on that front soon. Hung out with Jono Bacon and talked a bit about puppet-gluster on … Continue reading →
I’m here in New Orleans at Linux Con, hacking on puppet-gluster and talking to lots of interesting folks. I’ve met gluster hacker Theron Conrey, and my host John Mark Walker, Fedora and Raspberry Pi experts Spot and Ruth Suehle, and many … Continue reading →
Hey there, I’ve done a bit of puppet-gluster hacking lately to try to squeeze some extra features and testing in before Linuxcon. Here’s a short list: SELinux fixes to keep Dan Walsh happy :) Ping and status checks before volume … Continue reading →
John Mark Walker, (from Redhat) has been kind enough to invite me to speak at the Linuxcon Gluster Workshop in New Orleans. I’ll be speaking about puppet-gluster, giving demos, and hopefully showing off some new features. I’m also looking forward … Continue reading →
I love tabs, they’re so much easier to work with, but YAML doesn’t like them. I’m constantly adding them in accidentally, and puppet’s error message is a bit cryptic: Error: Could not retrieve catalog from remote server: Error 400 on … Continue reading →
It was time to take the plunge and upgrade from Fedora 18 to Fedora 19. Fedora 18 was one of the worst releases ever, so I figured it could only get better. I ran my backups as usual, however this … Continue reading →
A simple hack to add a user type to my puppet-ipa module turned out to cause quite a stir. I’ve just pushed these changes out for your testing: 3 files changed, 1401 insertions(+), 215 deletions(-) You should now have a … Continue reading →
I’ve been a little slow in making release announcements, so here’s some news: I’ve just released the third stage of my puppet-ipa module. At the moment it now supports installation, managing of hosts, and managing of services. It integrates with … Continue reading →
So I just rolled a new vm to hack around with FreeIPA. Here are some things that I’ve come across so far. I was planning on configuring LDAP, and Kerberos manually, but the included webui looks like a lovely tool … Continue reading →
In response to some discussion in the gluster community, I am releasing my puppet-lsi module. It’s quite simple, but it is very useful for rebuilding machines. It could do a lot more, but I wanted to depend on the proprietary … Continue reading →
I could probably write a long design article explaining why identical duplicate resources should be allowed [1] in puppet. If puppet is going to survive in the long-term, they will have to build in this feature. In the short-term, I will … Continue reading →
I don’t like repetitive work, and installing matlab counts doubly as so. Once I figured out the correct steps, I automated it with a puppet module. The downside is that the install takes a while because puppet needs to copy … Continue reading →