Batteries Not Included

There are lots of features that Logmunch doesn't have!

Dump Logs to S3

Lots of log providers give you the opportunity to periodically bundle up and export your logs to a S3 bucket!

This is a great way to:

  • Spend a lot of money on S3.
  • For logs that you will never, ever look at.
  • I've been working at a company where we have been socking away 1 full year's worth of logs in the S3 archives for the past 9 years, and I can count on zero fingers the number of times we've wanted to go gallivanting around in a bunch of .tar.gz archives bundled together by hour looking for something. The needle to haystack ratio there is just insane.

Are you the kind of person who dutifully boxes up leftovers, only to put them in the fridge and let them sit for a month, so that you can throw them out at the end of the month? You're exactly the sort of person who needs to archive logs in S3 for a very long time.

Really, Any Kind of Backup At All

Most of Logmunch's config lives in its environment variables - so, if you can boot up a fresh Logmunch, you're back in the game.

Worst case scenario? You lose some logs. Logs are ephemeral.

Hot Backups/Read Scaling

I think it might be possible to use ngx_http_mirror_module to send all of your logs to a bank of identical Logmunch servers, but I haven't tried it.

Cluster Mode/Write Scaling

Eventually - eventually - you will reach the point where you simply produce more logs than can be written to a single server.

I want to say that Logmunch has probably got your back for quite a long time before this becomes the case, but it's going to happen eventually.

Building a sharding write engine to randomly send logs to a bank of Logmunch servers would scale up those writes would be fairly trivial, but the difficult part comes when you want to search those logs: your search machinery needs to be able to query multiple servers and merge the results.

I know how I would build this, but, uh - it's a big project. So Logmunch can't do it.

Graphing Search Results

It can be really, really useful to see a chart of how many log hits you're receiving over time.

It is unfortunate that I have not built that.

Operational Stats & Graphs

RAM usage? Sisk usage? The amount of logs-per-second that Logmunch is processing? Some indication that Logmunch is under stress if you're sending it too many logs? These would be wonderful things to know.

Unfortunately, Logmunch doesn't surface any of these valuable details!

Lots of Log Ingest Formats

The more ways that Logmunch can accept logs, the better.

As of right now, it just accepts the splunk HEC format and nothing else.

Alerts

It makes a lot of sense that you might want alerts to fire when certain, particularly terrifying or interesting logs are discovered.

Can't do that with Logmunch, though. No alerting infrastructure in there.

Logmunch-Term

Hey, it'd be nice to download a little cli, maybe write a .logmunch-config with a key in it, and do stuff like:

#> logmunch --tail "Metadata on mountpoint"
2024-06-04 23:47:40 marquee stderr [2024-06-05 06:47:40] INFO admin/command_metadata Metadata on mountpoint /chill.mp3 changed to "Nobuo Uematsu - Kefka"
2024-06-04 23:48:05 marquee stderr [2024-06-05 06:48:05] INFO admin/command_metadata Metadata on mountpoint /groove.mp3 changed to "Barry "Epoch" Topping - Welcome"
2024-06-04 23:50:18 marquee stderr [2024-06-05 06:50:18] INFO admin/command_metadata Metadata on mountpoint /groove.mp3 changed to "Unknown"
2024-06-04 23:50:19 marquee stderr [2024-06-05 06:50:19] INFO admin/command_metadata Metadata on mountpoint /groove.mp3 changed to "Parov Stelar - Soul Fever Blues (feat. Muddy Waters)"
2024-06-04 23:51:05 marquee stderr [2024-06-05 06:51:05] INFO admin/command_metadata Metadata on mountpoint /chill.mp3 changed to "Homestuck - Walk-Stab-Walk (R&E)"
2024-06-04 23:57:06 marquee stderr [2024-06-05 06:57:06] INFO admin/command_metadata Metadata on mountpoint /chill.mp3 changed to "Tycho - See"
2024-06-04 23:58:51 marquee stderr [2024-06-05 06:58:51] INFO admin/command_metadata Metadata on mountpoint /groove.mp3 changed to "Toby Fox - Queen"
2024-06-04 23:59:48 marquee stderr [2024-06-05 06:59:48] INFO admin/command_metadata Metadata on mountpoint /groove.mp3 changed to "PROTO·DOME - THE BEST 2 Minutes 14 Seconds OF YOUR LIFE (Peanut Plains)"
2024-06-05 00:02:02 marquee stderr [2024-06-05 07:02:02] INFO admin/command_metadata Metadata on mountpoint /groove.mp3 changed to "Bonnie Tyler - Total Eclipse Of The Heart"
2024-06-05 00:02:25 marquee stderr [2024-06-05 07:02:25] INFO admin/command_metadata Metadata on mountpoint /chill.mp3 changed to "Homestuck - Clockwork Sorrow"
2024-06-05 00:03:30 marquee stderr [2024-06-05 07:03:30] INFO admin/command_metadata Metadata on mountpoint /chill.mp3 changed to "Mazedude - Mummy Dance (Underground Forest Area)"
2024-06-05 00:07:36 marquee stderr [2024-06-05 07:07:36] INFO admin/command_metadata Metadata on mountpoint /chill.mp3 changed to "Nobuo Uematsu, Aki Kuroda - Suteki Da Ne"

Wow, that seems like a cool thing to be able to do. Can't do it, though.

More Better Timezone UI

The logmunch UI just assumes you're cool with it converting logs from UTC into whatever your browser timezone is. That's... not always the right assumption?

Hide Search Highlighting

I just want to see logs, I don't want to see where the logs match the searches! That's crazy!