Learn data manipulation posted on 07 June 2025

One of the most common skills I use as a software engineer isn’t some syntax about threads, promises or coroutine, but how to quickly manipulate data – this can be being able to quickly extract the files responsible for test failures from a large log, parsing a json blob to extract only one field, removing duplicated entries in a file etc.

This is something you should be able to do from multiple tools – from vim, from your terminal, from a google spreadsheet etc. What tools (e.g. sed vs awk) or formula (countif vs exists) you use doesn’t matter much as long as you can quickly use them (or at least know they exist such that you can properly prompt an AI tool to give you the answer).

What’s your favorite tool? What’s the one you still struggle to use? It’s definitively JQ for me as I always screwed up the syntax :)

LinkedIn post