Yesterday I spent all day debugging a few unit tests and I learned a lot

Published August 1, 2024

I am not used to debugger. I am more of a printf kind of dev. A couple of times a year I use them for various reasons. Sometime because I think good developers should know how to use them. Sometime because the madness required to run a piece of code is so long and complicated that the best way for me is to do it via a debugger, sometime because it is kind of a nice place when you are there. It takes some time to set it up if you don’t use it very often.

Yesterday I needed a quiet place. Here in Italy we are in the middle of summer, even where I am in Piedmont the humidity is pretty high. Ludovica will turn 6 months next weeks, she is starting to play and sometime taste various kinds of foods, it is awesome, but I am pretty tired and stay focused is not easy as it used to be.

I am developing an Apache Arrow Flight server on top of Redis Streams that will be used to feed training sessions for generative AI, not LLMs but time series and tables.

Next year at this point I hope to have AC installed because I didn’t feel the need for one so much as in these last two years. I work from home from 7 years, but summer looks harder than usual.

Anyway, Apache Arrow Flight is a protocol on top of gRPC that can be used to put, get and list streaming sessions. It requires some coordination, some async work that needs to be validated. I started as I usually do, writing some untested code, but I quickly realized that I was not going anywhere. I needed a way to add save points to features that I know are ready and should not break. So I wrote kind of high unit tests. With that I mean that I am not running a single function, but I spin up the flight server, with Redis via testcontainers and I use the client to simulate what a user will actually do, as I said, the unit is pretty big but still a unit.

Then when the easy scaffolding was done and validated I started to get feedbacks from my colleagues that will consume such server and some corner cases where not that easy to figure out. What happens when multiple clients are consuming a stream that is still open with another client writing chunks? What if a client at some point disconnects? Will the connection stay open?

All legitimate questions but I didn’t have the experience required to answer, so I decide to write code to simulate multiple clients doing different things and to follow what was going on step by step I used delve.

Delve is the standard when it comes to debugging Golang, there is also an extension for GDB but delve is what you are looking for. I spend half an hour remembering how to use it but when I was there things turned pretty great in my life.

You can even set breakpoint in the old way from the shell itself or with you can do write in your program `runtime.Breakpoint()` and the debugger will stop there. Very easy.

And when you have delve running your test or your binary the only things you need to do is:

That’s it. It was a productive day, I learned a lot I am going to do it again today.

I side effect of this is that now I run rootless Docker, for laziness I never switched, I always used sudo because I don’t like the idea to add myself to docker group and since testcontainers spins up Docker containers running it with sudo added a bit of complexity that was ruining my dev ergonomics.

It was one line change, and then a reboot (but only because I am lazy I am sure it is not required).


As a side note for you, I am really into Apache Arrow. I think you should check it out. I am using data fusion in rust over s3, arrow flight in Go and python and various others things. If you are into it or you need some help let me know. Influx Data my previous employer is big into it, and I am kind of upset that they kicked my out earlier because of the idea to write their own query language because now they are doing cool things again! A bit more enterprise but still cool!

Are you having trouble figuring out your way to building automation, release and troubleshoot your software? Let's get actionables lessons learned straight to you via email.