Crawl and save a website as PDF files

The web is constantly changing and sometimes sites are deleted as the business or people behind it moves on. Recently we removed a few sites as we were doing maintenance and updates on the many sites we run at work. Some of them had interesting content - for personal or professional reasons, and we wanted to make a static copy of the sites before deleting the sites completely.

I have not found any easy, simple and well-working software, which can produce and an all-inclusive downloaded copy of a website (including all resources sourced from CDNs and 3rd party sites (to actually make them browsable offline). As I needed to make the copy reasonable fast, I choose to try to capture the contents of the site (a text/article heavy site) as PDFs.

Linux - No space left on device, yet plenty of free space

My little server ran into an issue, and started reporting the error:

No space left on device

No worries, lest figure out which disk has full and clean up…

Using the df command with the -h (for human-readable output) it should be easy to find the issue:

root@server:~# df -h
Filesystem Size Used Avail Use% Mounted on
udev 483M 0 483M 0% /dev
tmpfs 100M 3.1M 97M 4% /run
/dev/vda 20G 9.3G 9.4G 50% /
tmpfs 500M 0 500M 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 500M 0 500M 0% /sys/fs/cgroup
cgmfs 100K 0 100K 0% /run/cgmanager/fs
tmpfs 100M 0 100M 0% /run/user/1000

Strange. Notice who the /dev/vda is 50% fillled and all other disk devices seems to be finde too. Well after a little digging, thinking and googling, it turns out device space consists of two things - space (for data) on the device and iNodes (the stuff used to mange the space - where the data go - simplified).

DNSSEC and switching nameservers

I’ve switched nameservers for all my domains yesterday. During the past many years I’ve been free-riding on GratisDNS and enjoying their free DNS service (and luckily never needed support in their forums).

Yesterday I switched to Cloudflare and I’m using them for DNS for this (and other domains). I don’t have any particular requirements, and the switch was mostly easy and automated to the extent possible. Two domains went smooth, but the last my mahler.io domain went a stray a few hours during the switch.

Updates...

It’s been quiet here for a while, but be things have been happening behind the scenes. In case your wondering the site (and surroundings) have been seeing a number of updates which eventually may make it into separate posts.

  • I’m running on a Digital Ocean droplet. It was provisioned as an Ubuntu 12.04 LTS, which is dead by now (as in no more updates including security updates). The server has now been roll up to an Ubuntu 16.04 LTS in place.
  • As I was messing around with the server, I’ve added IPv6 support.
  • The DNS has been updated to have full support for DNSSEC.
  • My Let’s Encrypt Certificates now has automated certificate renewals and I’ve upgraded to CAA support.
  • The Webserver has been switched from Apache to NGINX.
  • The PHP has been switched from PHP 5.6 series to a modern 7.0.
  • I’m adopting full Git-backed backup of all server setup and configuration using BitBucket.org. It’s not complete but most config files have been added and managed using GitHub.

These was the majority of changes on the site and server the past few months. With these updates in place, I might get back to producing content for the site.

Devops: You build it; you run it… sort of

DevOps seems to be sweeping through IT departments these years and for most developers it seems to be sen as a way of getting those pesky gatekeepers from Operations away and ship code whenever any developers feels like it.

The problem is however, that in the eagerness to be a modern DevOps operation, the focus is often solely on the benefits of faster releases (on the short term) the “DevOps” provide over “Dev to Ops”, and many developers do seem to forget the virtues Operations (should) bring to the party.

Have your IT systems joined Social Media?

No, your servers should (probably) not have a facebook profile, nor should your servicebus have a twitter profile, but as the work tools change and evolve, you should probably consider updating the stream of status mails to more modern “social media” used at work.

When you’re in DevOps you probably get a steady stream of emails from various systems checking in. It may be alert emails, health checks or backup completed emails. It’s been more “fun” getting these mails with the rise of unlimited mail storage and powerful email-search tools should you ever need to find something in the endless stream of server-generated mails.

No more pets in IT

Remember the good old days, when IT got a new server. It was a special event - and naturally the naming game. Finding that special name for the server, which neatly fitted into the naming scheme adopted (be it Indian cities, Nordic mythology or cartoon characters).

This ought to be then, but the ceremony still happens regularly in many IT departments, where servers are treated with the same affection as with pets - and with all the bad side effects it may bring along.

Ephemeral Feature Toggles

At its simplest  feature toggle is a small if-statement, which may be toggled without deploying code, and used to enable features (new or changed functionality) in your software.

In an agile setup most developers love  having feature toggles available, as they often allow for a continuous delivery setup  with very little friction and obstacles. While this is true, it often seems developers forget to think of feature toggles as ephemeral, and doesn’t realize what a terrible mess, this can cause - given they don’t remove the toggles once the feature is launched and part of the product.

Baking Audiobooks with m4baker

Building audiobooks on (Debian) Linux in the m4b format is actually possible and doesn’t have to be a pain. I’ve found numerous recipes with shell instructions, but having a nice simple app to handle the building of the books seems much easier.

Most of the apps available for Linux seemed to be in a pre-alpha state, but after a few experiments I’ve settled on m4baker, which - while a bit rough - actually seems to do the job just fine.

Why DevOps works...

I’m digging through a backlog of podcasts and the gem of the day goes to SE-Radio podcast. In episode #247 they talk about DevOps and while I’ve preached and practiced DevOps for years, as mainly as common sense, the podcast did present a more reasonable argument why it works.

Developers are praised and appreciated for short time to market; the number of new features they introduce and changes they make to the running system.