Episode 15
Between Product and Engineering
In this episode we explore the way that technology, tools and architecture can affect the product that you are building. As engineers do we need to think more about product side? As product marketers what can we do to understand how the technology choice informs or limits our product possibilities? How do we deliver software in an open source project vs how do we deliver software in a corporate banking environment? What are the constraints that we need to be aware of in our decisions and how the customer need informs that. How we need to enjoy what we’re building - and make sure that we continue to enjoy building it as an individual, a team or as a company.
Once again I really enjoyed recording this one and asking myself some hard questions. Perhaps you have more questions that from it? Please get in touch with me via the Software Delivery Club. I’d love to hear from you!
SHOW NOTES
Dave Farley - Modern Software Engineering (https://twitter.com/davefarley77):
https://www.amazon.com/Modern-Software-Engineering-Discipline-Development/dp/0137314914
April Dunford’s Obviously Awesome (https://twitter.com/aprildunford):
https://www.aprildunford.com/obviously-awesome
Rosegarden Linux Audio and Midi Sequencer and Notation Editor:
http://rosegardenmusic.com/resources/authors/
You can also download a compilation of my best blog posts in PDF form. I call it:
Transcript
Welcome to the software delivery club.
Speaker:Every week, I explore different aspects of the business of designing and delivering
Speaker:production grade software, talking about subjects that I find crucial to
Speaker:good software delivery , as well as to industry experts about how they get to
Speaker:keep their production software running.
Speaker:I want to know whether they build or buy how they cope with new features
Speaker:and product changes, how they keep the customer satisfied while
Speaker:also making that engineers happy.
Speaker:I'll also talk about what tools we like to use and which ones we don't
Speaker:as well as tales from the frontline when it comes to delivering and
Speaker:supporting production grade software.
Speaker:Thank you for joining me.
Speaker:Welcome.
Speaker:I'm your host, Richard Bown.
Speaker:This is episode 15, it's all about the crucial interplay between
Speaker:product and engineering or what does building the product have to
Speaker:do with the tools technology and the architecture that we use to design it.
Speaker:It's been vacation time.
Speaker:And that means I got to spend the first few days of this month, busy scheduling
Speaker:a load of emails for my daily newsletter.
Speaker:If you've not checked it out, then I thoroughly recommend that you do.
Speaker:It's a daily thing, so I don't imagine everything may land with you, but at
Speaker:least not straight away, but I think it's turning into a great resource for
Speaker:those who think hard about the subject of software products and software delivery.
Speaker:In fact, I'm going to create a digest of it all soon.
Speaker:And make that available as a free download to give you a taster, check the links at
Speaker:the end of this episode for more details.
Speaker:In the meantime to read more about the newsletter and sign up for it,
Speaker:you can go to software delivery.club.
Speaker:That's dot C L U B.
Speaker:Come along and see what you missed out on in the archives, and please sign
Speaker:up , I promise you won't regret it.
Speaker:I spent some of my holiday reading some books about software delivery strategies.
Speaker:Not all of it.
Speaker:It was a holiday after all.
Speaker:Two in particular, I've been reading have been Dave Farley's "Modern
Speaker:software engineering" and also April Dunford's "Obviously Awesome".
Speaker:I've not finished either of them yet, to be honest, I was on holiday.
Speaker:So I'll leave off making any comments about them at this point,
Speaker:but they are two fascinating books.
Speaker:And really useful for those building production grade
Speaker:software for real customers.
Speaker:We want to build software the right way.
Speaker:But we don't want to build software that people ignore.
Speaker:For me, the product versus the engineering in my view, doesn't really exist.
Speaker:We are all people who need.
Speaker:some kind of affirmant in our lives.
Speaker:We want to make our creations strike a chord.
Speaker:So to do both, to engineer and be a product person, is possible.
Speaker:Despite what your job descriptions or the industry may tell you.
Speaker:And reading both of these books really brought that home to me.
Speaker:The older I become, the more experience I get, the less, I see
Speaker:the distinctions between skill sets, particularly at a senior level.
Speaker:For developers who are real developers.
Speaker:For product people who see themselves as real product people, that's fine.
Speaker:Keep plowing your furrough.
Speaker:But if you see yourself as an engineer with a product vision, then for
Speaker:goodness sake, run with that vision.
Speaker:Don't be afraid to stand out.
Speaker:This is the core of what I believe makes great software.
Speaker:It's not enough to just build stuff.
Speaker:Well, you have to have users.
Speaker:You need to have a community in order to validate your ideas and your execution.
Speaker:When I talk about software delivery strategies, I mean, everything, it
Speaker:takes to get an idea from inception.
Speaker:To production.
Speaker:That's the span of software delivery.
Speaker:It includes software development practices.
Speaker:But also product development practices.
Speaker:For many years now, I've been building software professionally i.e.
Speaker:getting paid for it.
Speaker:As well as building software on the side.
Speaker:The side staff, the side projects is of course, initially always less profitable
Speaker:or zero profitable and hard work.
Speaker:I've been doing it for so many years now that indeed I got a bit bored
Speaker:of just writing the software and now I prefer to talk about it instead.
Speaker:That's not to say I don't code.
Speaker:I do still, and I do enjoy it still.
Speaker:And I still have a few big projects in me I think.
Speaker:But I like to make sure that any coding I do is for the right reasons.
Speaker:I prefer not to code if I can avoid it.
Speaker:In fact, I got to the point a few years ago when I built a prototype
Speaker:for something and then realized that I need to be able to support that thing.
Speaker:And, spinning forwards thinking further down the line.
Speaker:I decided not to take it any further.
Speaker:The acid test being.
Speaker:Can you see yourself committing to that business model for the next so
Speaker:many years to make a success of it?
Speaker:I'll return to this point again, at the end of this episode.
Speaker:But I believe it's crucial for building excellent software.
Speaker:While it seems kind of obvious .i.e you have to actually enjoy using
Speaker:the thing that you're building.
Speaker:Testing it and using it must give you something that you can't get
Speaker:from something else, a thrill or a feeling that you're making something
Speaker:worthwhile, something better.
Speaker:I have an ashamed confession to make here.
Speaker:I started working for a SaaS company and regrettably hadn't really used the
Speaker:product in anger or enough detail before I started that in a pretty senior capacity.
Speaker:When I finally got my training on how to use it, I realized
Speaker:that it completely sucked.
Speaker:What's worse is in my time there, we didn't manage to make it less
Speaker:suckier due to debt, inertia, poor systems, lack of interest.
Speaker:No one building the product appear to care enough that it completely sucks.
Speaker:And that in itself was a major warning sign.
Speaker:Okay, so what else have I built that I'm actually proud of?
Speaker:Well, one of the first things was a big thing and it's still going.
Speaker:It's a Linux, audio and MIDI sequencer and notation editor, a
Speaker:mouthful, called Rosegarden.
Speaker:A couple of friends of mine at university built the original in
Speaker:1993 for one of their projects.
Speaker:It's almost 30 years old.
Speaker:I got into Linux around 95.
Speaker:And, me and one of the other originators decided to put this to Linux at the time.
Speaker:And then a few years later, it completely rewrite it.
Speaker:I've always had an interest in making music.
Speaker:And I learned sequencers such as Logic Audio and Cubase.
Speaker:And so using this massive project as a basis for my enthusiasm,
Speaker:for music, as well as for writing software seemed like a logical step.
Speaker:We didn't just stop at doing MIDI we added audio.
Speaker:VST support.
Speaker:Uh, loads of different interfaces.
Speaker:An extensible file format that allows sharing of device files
Speaker:all in a massive project, which is still actually going strong.
Speaker:The proof of the pudding is in the eating, as they say.
Speaker:It's been an incredible journey for Rosegarden.
Speaker:Tens, if not hundreds of thousands of downloads, perhaps even
Speaker:had an extra zero who knows.
Speaker:Many devoted fans across the world, and it's still there.
Speaker:It's still in its niche, but still going.
Speaker:I'd like to think that the current version.
Speaker:Which in essence hasn't changed since we rewrote it in early 2000s is a
Speaker:testament to the original design decisions we made, as well as some of the
Speaker:engineering that went into building it.
Speaker:It's definitely down to the tenacity and patience of many
Speaker:tens of developers over the years.
Speaker:In fact, the authors list on the rose garden website is 87
Speaker:individual contributors long.
Speaker:Year by year, new people have come into the Rosegarden fold.
Speaker:I don't contribute much anymore.
Speaker:The windows build a few years ago.
Speaker:Some automation stuff and that's about it.
Speaker:But what there is now is a wonderful community and there
Speaker:is a need for it to exist still.
Speaker:Therefore it exists.
Speaker:Product development is understanding what your customer wants.
Speaker:Software development is the practice of building and delivering that thing.
Speaker:The decisions that we make at all stages of these practices.
Speaker:Have impact and implications on the products.
Speaker:And therefore the experience that the customers have.
Speaker:There's a tension, always in building software.
Speaker:The need to create something which scratches an itch, both for
Speaker:the customer and the developer.
Speaker:This has to be balanced with a viable roadmap to delivering features.
Speaker:And this is particularly keenly felt when it comes to delivering open
Speaker:source projects like Rosegarden.
Speaker:I've worked on many pieces of software in the last 30 years,
Speaker:both in and after college.
Speaker:The most enjoyable and longest lasting pieces of software have
Speaker:been planned, but not too carefully.
Speaker:They have borrowed ideas and then reworked them and have also been
Speaker:relatively technologically agnostic.
Speaker:Bolting new pieces on as they grow.
Speaker:Maybe rewriting sections in completely different languages.
Speaker:In order to do this, a modular and flexible design must
Speaker:be the core of what you do.
Speaker:Having an excellent basis for your work, whether you build or buy the core is the
Speaker:key to making it extensible in the future.
Speaker:You need to keep your options open, listen to what your customers are doing.
Speaker:And be able to add those features without having to throw everything away.
Speaker:When we first discussed rebuilding Rosegarden from version 2.1, which was
Speaker:the university exit version two version 3.
Speaker:We were in the middle of an architecture discussion for quite a while.
Speaker:I remember this was the time when the Common Object Request Broker Architecture
Speaker:(CORBA) was particularly in vogue.
Speaker:And we did discuss using that, thankfully, that didn't get very far that version.
Speaker:I mean, we jumped straight to version 4
Speaker:. Additionally backwards compatibility
Speaker:supporting industry standards, all ensure that you appeal to a
Speaker:wide range of potential customers.
Speaker:In my professional career, I've also worked on many financial transformation
Speaker:projects are banks and insurers.
Speaker:These have often been based around a single piece of technology
Speaker:or platform, which provides a business rules type approach to
Speaker:data ingestion and transformation.
Speaker:These are essentially smart or sometimes not so smart ETL tools.
Speaker:That means Extract Transform and Load tools, which allows the developer to
Speaker:transform large quantities of data.
Speaker:At speed to provide defined outputs of data, taking some data,
Speaker:munging it into another type.
Speaker:In this case.
Speaker:The platform is the modular approach.
Speaker:It provides a standard interface to a whole range of data sources, such as
Speaker:relational databases, text files, file streams, rest API, XML, et cetera.
Speaker:These are all plugged into each other via a platform engine, which
Speaker:provides a familiar language.
Speaker:These machines we build are required to work quickly.
Speaker:Optimized for speed at the database.
Speaker:Or elsewhere.
Speaker:This was crucial in a production system.
Speaker:But also, it was crucial to make sure that the interfaces and the rules
Speaker:applied were consistent and repeatable.
Speaker:Therefore, there was a lot of testing required.
Speaker:Deployment had to be redone, redeployment, retesting, lots of manual work everywhere.
Speaker:This we learned to automate, even with some complex database systems,
Speaker:we hand-built basic CI/CD pipelines.
Speaker:ourselves out of the tools we found around us.
Speaker:Even Ms.
Speaker:Project got used as a basic front end for defining our scheduling
Speaker:dependencies and automating out of that.
Speaker:What you learn is when data integrity is vital, you need
Speaker:to ensure that quality is high.
Speaker:Quality can be improved by manual testing and deployment, but automation
Speaker:of this makes it quicker, more reliable and less difficult for the developers.
Speaker:So looking at this.
Speaker:There is more engineering technique required in building something
Speaker:for a bank, where you need predictability and precision, then
Speaker:there is when you're building a creative tool for creative people.
Speaker:There you need to focus more on features and extensibility.
Speaker:Both need a minimal level of engineering, knowledge and capacity.
Speaker:But the banking system will require more attention when it comes to deployment.
Speaker:The same is true, of course, for the user base.
Speaker:Finance systems have a relatively small number of users compared
Speaker:to people who use music software.
Speaker:Even those on Linux.
Speaker:So the technology choice and the effort you make to test and deploy your code
Speaker:must match the needs of the user base.
Speaker:If your end customer is paying for your service to be always available and
Speaker:always reliable, then you need to pay attention to testing and deployment.
Speaker:You can choose, for example, infrastructure that will
Speaker:grow with your users' needs.
Speaker:Auto scaling on public clouds.
Speaker:Possibly architecting for microservices.
Speaker:And deploying into a Kubernetes managed service if your customer requires
Speaker:that . If on the other hand, you have a single client and they need a rest
Speaker:API for their mobile app , then just deploy infrastructure appropriately.
Speaker:Your architectural choices will inform the speed at which you can move the
Speaker:people you can hire and the functionality and quality of service you can provide.
Speaker:Returning to the subject of the acid test that I mentioned earlier.
Speaker:How do you know that your product idea is one that you will continue to commit to?
Speaker:As an individual as a group, as a company.
Speaker:You must get excited about what you do.
Speaker:You must want to deliver real change to those who you support.
Speaker:How can you commit to this?
Speaker:We can be smart, us software types.
Speaker:But we have to admit that we're not as smart as the product marketers.
Speaker:The real product marketers know what the user needs.
Speaker:And they are willing to put a bet on it.
Speaker:They will build something that works.
Speaker:Sure, as an engineer, you can stumble around and come up with a solution
Speaker:that might fit a specific need.
Speaker:You might get lucky.
Speaker:However you increase your chances of getting lucky.
Speaker:If you don't just blindly build stuff, but make an educated or
Speaker:sometimes very educated guess at what will turn your customers on?
Speaker:Walking that line, finding what works is the place we need to be
Speaker:every day when we deliver software.
Speaker:Because software doesn't sleep.
Speaker:We need to change it, to improve it, to fix it when it breaks, to show
Speaker:its compliance when we're asked, to secure it , to integrate it.
Speaker:We are creating a living thing.
Speaker:We commit to it.
Speaker:Every day, we need to consider what this means to us.
Speaker:And what this means to the customer.
Speaker:There is so much more we can talk about here, about how we can measure customer
Speaker:interaction, how they use our software.
Speaker:But for the moment, I would like you to consider that technology and
Speaker:product are two sides of the same coin.
Speaker:You can't have one without the other.
Speaker:And you need to be proportionate when you make your choices,
Speaker:as they do affect each other.
Speaker:I hope you've enjoyed the thoughts today.
Speaker:I've really enjoyed thinking about it myself.
Speaker:And I would urge you to get in touch with me via the Software Delivery Club.
Speaker:You can go to the page software delivery.club and sign up for the
Speaker:newsletter and also reach me on Twitter @richardwbown, that's.
Speaker:B O W N or on LinkedIn or via my website.
Speaker:Until next time, keep coding.
Speaker:Keep dreaming and keep thinking software.
