Let me tell you a #
story about #
challenges, #
self-doubt, #
panic, #
eureka moments and #
satisfaction:
I wouldn't classify myself as a #
programmer, but one of the skills I have developed over the year is doing #
statistical analysis in the #
R #
programming #
language. I was recently asked, for the first time in my career, to carry out the analysis of another research group (thus, hired as the 'statistical expert'). I took on the #
challenge, not without fears. When I work with my own #
data and know the data very well, and I have time to really think on how to do the #
analysis. When working with someone else's data, you are supposed to just look at some tables and know exactly what you have to do. It's a whole different level.
As I started working on the data-set, which was quite big, I realized I had to merge some matrices (jargon for putting different data-sets together) and so I did, thinking I knew what I was doing. I ended up with millions of observations, which is not common when doing #
experiments on humans (after all, humans can only take a certain amount of trials before they become bored during the experiment). But I thought: oh well, #
data don't lie... This was the first warning sign. I couldn't merge all the data! The computer would crash, R would freeze, and I would get all sort of error messages which could basically be summarized as R telling me: "Yo wtf are you doing, I can handle so much data!". This was strange. After all, it was all just a bunch of ones and zeros!
I then went on and tried to run the statistical analysis with the data I
did manage to merge together. Again, same problem. R was telling "Dafuq man, no is no! Data analysis needs to be consensual, and I never agreed to sallow all this data!". By this time I started to panic. I have (yes, have. This is an ongoing story) an approaching deadline to hand this in, and R decided to start wearing a chastity belt and won't let me data it like I wanted to).
I then made the mistake of blaming the instrument rather than how I was using it. I thought: "Ok, this is not my fault, it's my stupid office computer. Old piece of crap it's too old for the fancy stuff I'm trying to do". So I started working on my own laptop. A fairly modern one that runs at half capacity because Linux still hasn't been made compatible with the hardware it has, but still, more powerful than my old office computer. Things slightly improved, I was able to merge a bit more data, but then I again run into the same problems. You wou;ld think by now I would've realized something was wrong, but good old stubborn me insisted on the 'it's not me, it's you' interpretation and asked some computer technicians at my work for the most powerful, modern, state-of-the-art supercomputer they had. And I got it! I had to go play a gig in one hour, so I rushed to the lab where these computers are stored, and I started hammering away at the data-analysis. "That that R! With this processors you won't be able to say no, muahahaha!!". But low and behold, the computer choked yet again!! I was contemplating jumping out of the window by then.... But I thought "Ok, fuck it... I'll just go to my gig, play some music, frink some beer, and forget about this piece of shit analysis that won't budge".
And so I did. I went to my gig. Probably the last gig in a long time (we just went into lockdown again here in the Netherlands). I played, I danced, I drank, and had a good time. That night I came home, ate something, watched the finale of RuPaul's drag race UK season 3, and went to bed.
There I was, lying semi-awake when it hit me! I had such a big eureka moment it almost felt like there was an orgy in my brain. I suddenly realized what I had done wrong. I won't bother you with the technical details, but it was something so obvious, so easy to fix! There have been few times in my life where I have felt such an intellectual satisfaction. Last time was when I was finally able to connect Heidegger's phenomenology with the study of consciousness in cognitive science, or when I was able to link Chomsky's theories to what I knew about language processing and acquisition. If these words mean nothing to you, I'm basically saying that only very challenging intellectual feats had given me such a feeling of satisfaction, and they were all in the fields of philosophy and empirical, experimental science. This is the first time I experienced something like that in the field of programming.
As I wrote this, my statistical model was running, and now I am finished, my statistical model just finished running as well... I made it! I fucking made it!
Take-home message: Take challenges when they come to you. You might fail, make stupid mistakes, think everything is lost... But just push through. The satisfaction of overcoming all those feelings and finally prevailing is worth it all.
*drops mic
