3 Lessons Learned While Paying Off Technical Debt

For almost the past year, I’ve been part of a team at my company building a new web application. Because we’ve operated using agile methodology, we generally only do things very incrementally, building the smallest features or making the smallest adjustments one at a time. While there are a lot of benefits to building software this way, it has unfortunately left our codebase bloated, and our technology stack fundamentally misaligned with our technical needs. About two weeks ago, me and the other developers were able to persuade our manager to let us take two weeks to switch our application from a monolithic Ruby on Rails application into a front end application using Facebook’s new React.js framework, and an API backend in Ruby on Rails. Here’s what I’ve learned so far.

1. It’s Really Fun

Normally, I will just do a small story to add an incremental feature to our application. The pattern of such a story usually goes along the lines of a user going to a page, clicking on a button, saving something to the database, and then notifying the user that their intended action was performed. Besides these small stories, there are also lots of bugs that crop up that need to be squashed. However satisfying it is to complete these small tasks, as I discovered last month when building a side project, it’s even more satisfying to code large swaths of logic all at once. Since we aren’t (supposed to be) adding any new features, all we are doing is in essence refactoring our application - that is changing how the code is implemented without changing how the application functions. This has freed me up to go into our application’s code base and just bulldoze things left and right, and has put a smile on my face the entire time.

2. Be Aware Of Scope Creep

Transforming a Ruby on Rails monolithic application to a pure API isn’t terribly challenging (although, it did help that I spent most of my recent vacation learning about this). However, as I was going through old code I kept noticing areas that could be improved. At first these were small things, such as fixing a few false positive tests. As I dug further and further, and had more time to think (and dream) about the application it became apparent that there were a few big areas that needed a total overhaul. As a result, I went ahead and did a lot of additional refactoring that I didn’t anticipate doing at the onset of this project. Furthermore, there were a few features in the application that weren’t providing our users with a good experience that we decided to enhance, instead of rebuilding them identically. In the long run this will save us time, but just like the unforseen additional refactoring, it has expanded the scope of this project, and thus extended it's time period.

3. It Will Probably Take Longer Than You Think

We promised our manager that it would take two weeks to pay off this technical debt that we had accrued, and switch our app over to our newly chosen technology stack. Well, these two weeks are just about up and it’s going to be at least another week - possibly more, before the transition is actually completed. This seems to always happen when building software, but I can identify a few reasons why it happened in our case.

First, there was the unanticipated refactoring and feature enhancement that just felt right to do while we were already making the Rails app into a pure API. Second, our team is not as familiar with React.js as we are with Rails, and when we've run into problems using React.js, they take us longer to solve than we are used to. Finally, our team is not really good at estimating larger projects (which is normal - estimating software engineering projects is notoriously difficult), and also has no desire to play office politics and puff up the estimate so that we win some silly expectations game.

I feel really bad about underestimating the project. I always like to underpromise and overdeliver - but in this case it’s been the opposite. I suppose I can take solace in the fact that the work we’ve done has made our application dramatically faster, and our codebase leaner and more maintainable. Overall I've learned a lot by undertaking this project, and hope the next time I do something similar I'll be even better at it.