Project Retrospective: link-service

When a project is finished, or comes as close to finished as possible, it is nice to spend some time pondering what has been achieved. Reflecting on the things learned as well as the successes and failures allows for an opportunity to grow. I will call this a retrospective, but it is also known as a post-mortem.

According to R. Ryan Nelson, professor at the University of Virginia, project retrospectives offer a myriad of benefits including:

• Organizational learning
• Continuous improvement
• Improved recognition and reflection

While Dr. Nelson's writing is intended for significant projects and organizations, performing one on a hobby project can be just as beneficial. In agile development, sprint retrospectives are not uncommon so let's try extending the idea despite how challenging a project retrospective may be... it is important to remember the goal is for improvement - who can be a better critic of you than yourself?

Let's go over a project I've recently moved into a public GitHub repository, link-service.

What is link-service?

Mircoservices are all the rage right now. Perhaps it is just a fad - I'm not sure, but I am sure I like the premise. I like the idea due to the scaleable and composeable nature of the architecture. Being able to keep an entire service within your mind as you work is also neat. I doubt this project could be considered a true microservice, but it is a really tiny project and does provide a service so...

Here we have a microservice that fulfills the need for a curated collection of links and the ability to randomly return one of those links. It is written in Clojure. The intention is to have a chat bot (or anything, really) hit the endpoint /random. This endpoint returns a randomly selected link from within the service's database and sends it to the chat bot via JSON. From there the bot can do whatever it likes.

It is similar to the service Hubot's pugme script hits when you ask for a pug gif. The only difference being that you could add and remove links by:

1. Sending properly formed JSON with fields message and token to the endpoint /add or /delete.
2. Having a valid token field that the link-service knows about.

After that it is just a matter of sending the GET request and seeing if the response returned indicates success or failure.

TL;DR: Toy service for requesting a random link, but you can also add and remove links as desired.

Content With

Going over the good parts is unusually difficult... I find it much easier to pick on my work than to say anything positive. In this project keeping organized and writing tests paid dividends.

Organization

Throughout the project I felt I knew the direction to go and the steps needed to get there. As I was not initially sure if the project would see completion I chose to make use of Atlassian's Bitbucket. They offer free public and private repositories. Using their issue tracking was a huge boon to productivity and not going in random directions or otherwise veering off course.

Establishing well-defined tasks cannot be overstated. They are useful not only in capturing what broke or what to add, but the thought process at the time. Perhaps you submitted a task to create a new feature but by the time you get to the task the logic lost. By simply fleshing out the idea you have more information to go off of when you try to jog your memory. It also can serve as a way to test your idea before submitting. If you try to put a new feature into writing but are having difficulty then it would be a good idea to spend some more time thinking.

Testing

Looking at past personal projects I am not known to use much testing. These projects have always been for personal use and never available to others. This changes with link-service. Unsurprisingly, unit testing in a project pays off. In this case, code could be left untouched for weeks at a time and it is easy to forget details. On multiple occasions running lein test before committing caught something unexpected.

Continuous Integration also played a part through the use of Semaphore. Overall I am pleased with their service. The last time I used them was for the Clojure Cup in 2014. It is likely that having CI on this project was overkill, but it was still nice to experiment with. Being able to deploy through Semaphore was a time saver.

Could be Better

In any project there are things that go well and do not go well. For link-service two significant issues come to mind: time estimation and trying to use libraries before I am familiar with them.

Time estimation

The project was not expected to take long. I had expected it to take approximately one month of very limited spare time to complete. When I first came up with the idea it seemed simple enough. And to some extent it was... but I was pilling new library after new library onto the project. Learning multiple libraries and trying to get them to play well proved more difficult than anticipated.

In the middle of development I became focused on things unrelated to the project. Not a significant issue as it is a personal project with no deadlines. However, this still caused a delay as I had to become reacquainted with the code base. Overall, from start to finish, the project ran from the end of July to early October. While from my estimation of a month even when accounting for two months of not working on it.

Integrating Korma and lobos

Assumptions about the library Korma really threw things for a loop. For those not familiar with it, Korma is a domain specific language (DSL) for Clojure that gives a smooth interface for working with relational databases. It even has a pleasing website with documentation.

The library is rather nice to use once understood, and I will use it again, but one thing Korma that does not do is create the tables in a database. Looking back, I am not sure why I thought it did. Nowhere does it suggest that being a feature. Hours were spent trying to debug the problem. Maybe I'm using Korma wrong? Is something configured incorrectly? Does it have permission problems to the location? all raced through my head in a progressively absurd decline into possible things wrong. Then, after rereading the documentation, I noticed it mentions another library: lobos.

Duh! It became so obvious after reading about lobos. A library that creates databases and migrations for tables. Things significantly improved after putting the two together. Progress resumed and things went well from there.

For Future Use

A conscious effort will be made to start simple and fully understand a library before using it. Not trying to digest multiple new things at once should be obvious but it is all too tempting to try and do as much as possible in as little time. "Festina lente" means "make haste slowly" in Latin and proved to be true with this project.

Another abstract take-away from this project is to better plan the project before writing code. Again, this is another obvious point, but it would have saved significant amounts of time. Seeing the decisions made at the start of the project directly impact the end result was a good lesson.

Final Thoughts

The future of link-service may not be much beyond personal use, but I intend to maintain it as needed (anyone is welcome to make use of it). Regardless, it was a fun and rewarding project that heightened my enjoyment of Clojure and software development as a whole despite the self-induced issues. I feel more disciplined in my abilities and choices as a result of this project.