Even though I’ve been using UNIX for over 20 years, there are some simple, elegant, tools that I’m really now coming to appreciate:
cronandcrontabfor automatically running things at scheduled times, especially the GNU extension that supports@rebootallowing normal users to run scripts at machine startuplogrotatelets me keep dumping data into a single file, but break it into smaller, more manageable chunks. Combined withcrontabI can easily collect gigabytes of data in a straightforward manner, but have the results in well organized, compressed multi-megabyte units. That expedition with Python andargparse? That was driven by usage oflogrotate.curlfor grabbing anything off the Web. Okay, it’s simple and elegant like a Swiss Army Chainsaw, but it always works, never crashes, and can be configured six ways to Sunday.