Software Engineering 16/08/2021

We, the Merry Maintainers – Aerospike Vault Plugin

Written by Alex Scammon, Head of Open Source Development

Being a modern, responsible, software-defined citizen can mean making the lives of those around you easier and better. It can mean making subtle improvements that make things run more smoothly, consume fewer compute resources or take less time. Improvements can be subtle or profound, but they move the ball forward for anyone using what you’ve created. It’s good to make things better.

Sometimes, though, those most resistant to change systems and processes are the same people who created the systems themselves. Nowhere is that more evident than in the world of software development and design. Developers can be haunted by the ghosts of their efforts, desperate to hold fast to what they once deemed to be a critical feature, or necessary function – blind to how their software is actually used in the wild, by actual users. They can grasp tight to how they initially imagine “users” might interact with their product or service, despite all evidence to the contrary. 

Open source development helps make transparent design choices and, though it’s by no means a perfect way of building software, it shines light into the minds of software-makers. The easiest way to create an open source project is to start it right from the beginning in the open, as I discussed in my previous writing. And if the users don’t end up using it in the way the developers intended, well, they can go right into the code and tinker with the functionality themselves until they get it just the way they want.

It may seem natural that we ‘give away’ projects all the time in the course of the open source work we do at G-Research. In many senses, open source work is all about giving code and ideas and time and energy to the common good. However, code can feel very personal, just like all other human creations. When it actually comes time to give code away, it can get very emotional and it isn’t always easy for maintainers to let go. 

Plate-spinning maintainers

It’s easy to imagine the work of a maintainer as stolid, stationary – boring, even. Not so! Better to think of the work of a maintainer as being most similar to that of a professional plate spinner, using the momentum and gyroscopic effect of software development to carry it forward into whatever new demands the future holds. It requires a deep knowledge, careful balance and a gentle touch.

Few do it better than Jonathan Giannuzzi on our open source team, who managed to carry the Aerospike-Vault plugin beyond what would suit the needs of our organisation alone, into something the wider community could find value in. It didn’t make sense merely to create a Vault plugin for Aerospike in-house and keep it all to ourselves: we knew that members of the Aerospike community would want this eventually. Heeding the call, Jonathan took up the charge, built the plugin, wrote tests, set up some CI, wrote the documentation, verified that everything worked in our production environment, and then dutifully rolled out the repository into public view. He kept the plates spinning too – giving his teams what they needed – while building something for everyone else.

As Jonathan did all of this, we couldn’t help feeling that the plugin needed a better venue. Why, we asked ourselves, would anyone come looking for an Aerospike plugin in the Github org of a quantitative research firm? Isn’t this plugin so generically useful to enough people that it should have a more prominent place in the Aerospike or Vault community pages?  

We did all the basic marketing: adding ourselves to the community-contributions pages for Aerospike and Vault. This was a good start and we even had some early positive feedback from our efforts. It’s not exactly wrong for a third party company to author and maintain useful glue like this – it happens all the time, and just as often from individual contributors.

But it still didn’t feel right to us.

Fortunately, our internal teams had already established a good relationship with the people at Aerospike, so we took the opportunity to reach out. When we did, we discovered that they knew about the plugin and were hoping to piggy-back on the groundwork that Jonathan had laid down. We floated the idea that we move the repository into their organisation so that it would get more visibility and to nobody’s surprise but ours, they thought it was a great idea.  

Furthermore, they expressed an idea that we were thinking ourselves: this plugin should eventually live in a Hashicorp organisation repository itself. We couldn’t agree more. But we all also agreed that it might take a few steps to get there and that moving the plugin to the Aerospike community was a positive step towards that end goal.

The opportunity was just too much to ignore, so we moved the git project from the G-Research repository to the Aerospike community repository:

A few short months later, we’re even more pleased to see that this work is now recognised as an official part of the Vault project itself:

While this movement of code from one location to another may seem trivial, the real effort was for us to overcome our emotions around “letting go” of “our” project. Of course, we recognised this was an opportunity for our work to prosper and grow into something more than we could achieve by ourselves, but moving the project over felt like handing the keys to your red Porsche 911 to your teenager. They’ll wreck it! They’ll destroy the transmission! The tires! Ack!

It was important for us to remember that this software isn’t static, or carved in stone, or even “ours” once it’s in the world as an open source project.  It’s a living thing that grows with its community and it needs increased collaboration and contribution to strengthen its roots and flourish. Opening it up to outsiders actually gave the project the space it needed to breathe and brought others in by virtue of its utility. And when others used it, they were able to see opportunities for improvement that we never would have.  With this plugin, we had enough rationalisations, realisations and observations of this truth that we were able to overcome our emotional attachment to the project.

Doing open source isn’t just simple philanthropy. Who we are and what we think of ourselves is all wrapped up in what we create and it can be a real struggle to let go of our creations. But it is very much worth it. We are really pleased that the Aerospike-Vault plugin has moved on beyond us, and we’re thrilled that the Aerospike team can derive immediate benefit from it.  

Stay up to-date with G-Research

Subscribe to our newsletter to receive news & updates

You can click here to read our privacy policy. You can unsubscribe at anytime.