menuI was sitting in a room full of developers. Initially we had met to discuss what we needed to do to address some of our latest production issues but the discussion quickly devolved into another back-and-forth about the best way to log these issues so we had better visibility into what was happening in production.
We had a home-grown logging system but it was crude and, to be honest, abused by a number of our developers. We had so much shit poured into that system it was virtually impossible to find the issues you actually needed to get to. In addition, no alerting. So basically we waited for someone to complain about an issue and then we’d check it out and see what happened. Not ideal.
So during this meeting one of our smarter developers (not me unfortunately) threw out the idea of getting a pre-built package online instead of having some of us try to free up time to get our internal system up to snuff. Seemed like a good idea so we broke out Google and got to searching. Our criteria was pretty simple: real-time logging, REST-ful interface, no components to install, configurable alerting, not ridiculously priced. Simple, right?
While I didn’t think about getting an online system for our logging initially I have to admit I thought it was a good idea. But I was shocked to find out how few online logging systems there were! I expected pages of results but nope, there were only 3-4 who had SEO good enough to rank in the upper level of search results. We investigated those and none of them (seriously, not one of them!) was really what we were looking for. They were all these bloated, over-complicated systems that seemed more designed for capturing server logs than real-time application logging.
Funny side note: we did end up going with one of the online systems. After we signed up the company arranged a video session with us to get introduced to their product. I’m thinking “ok, this should be 15-20 minutes of basic best practices”. Nope. Fucking 90 minute meeting to cover all sorts of shit like their home-grown “query language” and all sorts of other stuff. Seriously, 90 minutes for their “intro” meeting and they recommended at least one additional meeting to cover everything.
So I set about creating JustLog.IT. I had built a number of logging systems in the past so I had a pretty good idea of what not to do. But I also had a number of code bases to draw from so it didn’t really take too long to get this up and a-runnin.