Treegraph.sh a tool for generating pretty file structure graphs
Linux has a nice little tool in the repositories called tree generates a nice console window-based directory map from the current location down. It’s great, but there’s not an equivalent in FreeBSD, though you can get the basic need met using find and some text processing pipes like
find . -type d -maxdepth 3 | sed -e "s/[^-][^\/]*\// |/g" -e "s/|\([^ ]\)/|-\1/"
. |-subdir0 | |-subsubdir1 | |-subsubdir0 |-subdir3 |-sub dir 2
However, this is enough to make me wish for something more visual. As usual, someone else already did it, but their version wasn’t quite what I wanted so I enlisted Claude.ai to help generate a little script that, like the above command, finds it’s way through a directory structure and builds a .dot file map as it goes, which can be converted into useful graphic formats with graphviz.
The code (below or at gitlab) is pretty easy to use, it does the dotfile generation then just parse the dotfile like usual with graphviz, either using an installed version or online.
treegraph.sh -d /usr/home -f -s -m 2 > tree.dot dot -Tsvg tree.dot > tree.svg
The edges seem to overlap a bit more than I’d like, but that’s an issue with graphviz and if you really want a presentation ready graph can be adjusted manually with invisible nodes or edges and manually set connection points to merge lines. It defaults to dot, which makes sense for ranks, but looks cool with neato or twopi as well.
#!/usr/local/bin/bash
# Help function
Help() {
echo "Usage: treegraph.sh [options]"
echo
echo "Options:"
echo " -h Display this help message"
echo " -f Enumerate both directories and files (default: directories only)"
echo " -d DIR Specify the root directory to parse (default: current directory)"
echo " -m DEPTH Specify the maximum depth of parsing (default: no limit)"
echo " -s Compute the size of folders and files using 'du -sh' (default: off)"
echo
echo "Examples:"
echo " treegraph.sh # Current directory, directories only"
echo " treegraph.sh -d /usr/home -f # Parse /usr/home, include files"
echo " treegraph.sh -d /usr/home -m 3 # Parse /usr/home with max depth 3"
echo " treegraph.sh -s # Show sizes for current directory"
echo " treegraph.sh -d /usr/home -f -s -m 2 # All options combined"
echo
echo "Example generating an SVG file (requires graphviz installed):"
echo " treegraph.sh -d /usr/home -f -s -m 2 > tree.dot"
echo " dot -Tsvg tree.dot > tree.svg"
echo
echo "When using -s, nodes are color-coded by size:"
echo " B (bytes) - gray50"
echo " K (KB) - black"
echo " M (MB) - blue2"
echo " G (GB) - darkorange3"
echo " T (TB) - crimson"
}
# Initialize variables with defaults
StartDir="."
Type="d"
MaxDepth=""
MaxDepthArg=""
ShowSizes=0
# Process options
while getopts ":hfsd:m:" option; do
case $option in
h) # Display help
Help
exit;;
f) # Include files
Type="f";;
s) # Show sizes
ShowSizes=1;;
d) # Directory
StartDir="$OPTARG";;
m) # Maximum depth
MaxDepth="$OPTARG"
MaxDepthArg="-maxdepth $MaxDepth";;
\?) # Invalid option
echo "Error: Invalid option"
Help
exit 1;;
:) # Missing argument
echo "Error: Option -$OPTARG requires an argument"
Help
exit 1;;
esac
done
# Build find command based on Type
if [ "$Type" = "f" ]; then
TypeArg="" # No type filter for both files and directories
else
TypeArg="-type d" # Only directories (default)
fi
# Get absolute path of starting directory for proper path handling
StartDirAbs=$(cd "$StartDir" 2>/dev/null && pwd) || StartDirAbs="$StartDir"
# Extract base directory name for display
BaseDir=$(basename "$StartDirAbs")
# If showing sizes, get root directory size
if [ $ShowSizes -eq 1 ]; then
RootSizeInfo=$(du -sh "$StartDir" 2>/dev/null | awk '{print $1}')
# Determine color based on size unit
if [[ "$RootSizeInfo" =~ [0-9]+B ]]; then
RootColor="gray50"
elif [[ "$RootSizeInfo" =~ [0-9]+K ]]; then
RootColor="black"
elif [[ "$RootSizeInfo" =~ [0-9]+M ]]; then
RootColor="blue2"
elif [[ "$RootSizeInfo" =~ [0-9]+G ]]; then
RootColor="darkorange3"
elif [[ "$RootSizeInfo" =~ [0-9]+T ]]; then
RootColor="crimson"
else
RootColor="black" # Default color
fi
SizeLabel=" ($RootSizeInfo)"
ColorAttr=", fontcolor=\"$RootColor\""
else
SizeLabel=""
ColorAttr=""
fi
# Run find and generate graphviz dotfile
(
echo "digraph directory_structure {"
echo " node [shape=box,style=rounded,width=0.1,height=0.1,margin=\"0.07,0.01\"];"
echo " edge [arrowhead=dot, arrowtail=dot, dir=both, arrowsize=0.3, color=gray];"
echo " nodesep=0.15;"
echo " rankdir=LR;"
echo " concentrate=true;"
echo " overlap=false;"
echo " splines=true;"
echo " searchsize=10000;"
echo " \"$StartDir\" [label=\"$BaseDir$SizeLabel\"$ColorAttr];"
# Process find output but skip the root directory itself
find "$StartDir" $TypeArg $MaxDepthArg | grep -v "^$StartDir\$" | awk -v start_dir="$StartDir" -v type="$Type" -v show_sizes=$ShowSizes '
{
current=$0;
# Check if this is a file or directory
is_file = 0;
if (type == "f") {
cmd = "test -f \"" current "\" && echo 1 || echo 0";
cmd | getline is_file;
close(cmd);
}
# Get the parent path
parent = current;
sub(/\/[^\/]*$/, "", parent); # Remove last component for parent
if (parent == "" || parent == ".") parent = start_dir;
# Extract just the name of the current node (not the full path)
split(current, path_parts, "/");
node_name = path_parts[length(path_parts)];
# Get size if requested
size_info = "";
color_attr = "";
if (show_sizes == 1) {
cmd = "du -sh \"" current "\" 2>/dev/null | awk '\''{print $1}'\''";
cmd | getline size_info;
close(cmd);
if (size_info != "") {
# Determine color based on size unit
if (size_info ~ /[0-9]+B/) {
color = "gray50";
} else if (size_info ~ /[0-9]+K/) {
color = "black";
} else if (size_info ~ /[0-9]+M/) {
color = "blue2";
} else if (size_info ~ /[0-9]+G/) {
color = "darkorange3";
} else if (size_info ~ /[0-9]+T/) {
color = "crimson";
} else {
color = "black"; # Default color
}
size_info = " (" size_info ")";
color_attr = ", fontcolor=\"" color "\"";
}
}
# Create node with appropriate shape and just the name as label
if (is_file) {
printf " \"%s\" [shape=plain, width=0, margin=0, label=\"%s%s\"%s];\n", current, node_name, size_info, color_attr;
} else {
printf " \"%s\" [label=\"%s%s\"%s];\n", current, node_name, size_info, color_attr;
}
# Create node connection
printf " \"%s\" -> \"%s\";\n", parent, current;
}
'
echo "}"
) # End of generated dotfile
FINAL System shutdown after 16 years
root@gamanjiru:/usr # shutdown -p now Shutdown NOW! shutdown: [pid 14442] root@gamanjiru:/usr # *** FINAL System shutdown message from gessel@gamanjiru.blackrosetech.com *** System going down IMMEDIATELY System shutdown time has arrived Connection to gamanjiru.blackrosetech.com closed by remote host. Connection to gamanjiru.blackrosetech.com closed. DISCONNECTED (PRESS <ENTER> TO RECONNECT) (Tue Feb 11 21:39:45 2025)
Almost 16 years ago, in July of 2009, I bought a used IBM x3655 7985-1AU off Ebay for $202.50 to replace my IBM Netfinity 5500 (4U!, dual dual 500MHz Xeo 1GB RAM (later upgraded to 4GB) and 5x 9.1GB HDs for $217.50 ) that had been running since 2004. That had, itself, replaced a dual socket generic deskside machine, a box Mark Godwin gave me back in the Nebucon days, that first went live on the interwebs running FreeBSD 2 in April of 1998 under black-rose.org. As of this post: 26 years, 10 months of FreeBSD hosted internets.
Those were the magic days of ebay: in 2008, just a year earlier, I’d quoted a similar x3650 (Intel E5410-based), 32GB but only a pair of crappy consumer SATA 500GB drives for $7,297.00.
The new x3655 came with 32GB of RAM and dual-core AMD processor and an RSA-II. The original motherboard firmware only supported dual core AMD processors, not the then brand new AMD Opteron quad core, so I bought a somewhat hard to find 43W7343 motherboard for $190 (and an additional 5 fans to max out cooling for $18 each) and then a pair of AMD Opteron 2352 2.1GHz CPUs and swapped the mobo and the CPUs and the heat sinks and the fans. Note that it is really hard to find data on the dual quad core option, it was a bit of a hot rod.
I added the 2x drive expansion module, another ebay find, and loaded the drive bays with 8x 26K5657 10k 72G 2.5″ SAS drives (at about $65 each, used) on the ServeRAID 8k and expanded the RAM to 64GB, 57856 MB available with RAM sparing set for reliability. The modified machine reports itself as an x3655 7943-AC1.
I ended up creating an 8×1 RAID array to pass to ZFS, ZRAID2, which mean I had a battery backed write cache I could count on. The system has dual power supplies, each connected to a 2200 VA (hacked XR) UPS.
Over the years, almost plural decades, of continuous operation I’ve lost maybe 4 drives. Once, while I wasn’t paying much attention, somewhere over about 6 months two drives failed and I got close to catastrophic failure, but ZFS pulled through. It started life on FreeBSD 7 and I took a chance on the then brand-new experimental release of ZFS. Over the years it ran most of the releases up to 12, stalling there as the decision was made to shift to new hardware and I’m pretty sure I’ve run every release of FreeBSD except 1.0 and 13.
It was a fast machine, 8⨉ 2100.12-MHz K8-class physical cores, more than enough to run mail and various web services, everything compiled for the AMD Barcelona architecture. The pre-Lenovoization of the IBM x86 hardware was really first rate, top of the line data center gear and it shutdown without a flaw, same performance and configuration as when I fired it up, used and already off-list, 16 years earlier.
It was never too slow, even now, even as OS’s have expanded and the total number of ports needed for basic services has grown by about 10x given ever spiraling dependencies. It wasn’t that it was slow, but that it used far too much power and electricity got more and more expensive in CA.
So I migrated to a new box, took the better part of a year to spare time migrate all of the jail services from the old machine and, increasingly unsupported FreeBSD 12 OS to new HPE DL360 G9 (ebay, inflation: $497) running FreeBSD 14, added a poudriere build environment on a DL60 and carefully tuned the kernels for power efficiency not bad for 10 disks, 20 cores, 192GB: 114W. Now there are 20 physical cores and 40 virtual and yet, clearly showing the limits of Moore’s law, the new box’s E5-2630 v4s are only 2.20GHz: 5x the execution cores, but only 5% faster clock. 16 years of progress.
Good night, sweet prince.
Adding a feature to MediaWiki WikiEditor formatting
MediaWiki is an excellent tool for maintaining documentation and I’ve had a self-hosted instance since at least 2011-06-04 (that’s the oldest edit in my internal user contributions list). And some 3,436 edits later, I still can’t remember the tags for SyntaxHighlight, which is an awfully nice little highlighter that uses pygments to render structured text in a more readable form. I got tired of looking them up every few weeks and so thought there must be a way to add some hints to the user interface.
I was surprised the WikiEditor plugin, which provides a nice point-n-click interface to some of the more commonly used MediaWiki markup tags, did not have an option or extension for SyntaxHighlight point-n-click and but, of course, you can edit the JavaScript that renders the toolbar and amend it with features you want.
The instructions are pretty clear, if not quite the step-by-step howto some are.
- Make sure you have WikiEditor enabled in LocalSettings.php
- You need permission to edit the Common.js page, which if you run the site you should have, but regular users can’t.
- If it doesn’t seem to load, make sure you clear all caches before testing.
On my site, the URL for Common.js is https://your.host.tld/mediawiki/index.php?title=MediaWiki:Common.js which contained only the default
/* Any JavaScript here will be loaded for all users on every page load. */
and to which I added:
/* Any JavaScript here will be loaded for all users on every page load. */
// Check if we're editing a page.
if ( [ 'edit', 'submit' ].indexOf( mw.config.get( 'wgAction' ) ) !== -1 ) {
// Add a hook handler.
mw.hook( 'wikiEditor.toolbarReady' ).add( function ( $textarea ) {
// Configure a new toolbar entry on the given $textarea jQuery object.
$textarea.wikiEditor( 'addToToolbar', {
section: 'advanced',
group: 'format',
groups: {
list: {
tools: {
syntaxhighlight : {
label: 'SyntaxHighlight',
type: 'select',
list: {
'bash': {
label: 'Bash',
action: {
type: 'encapsulate',
options: {
pre: '<syntaxhighlight lang="bash">',
post: '</syntaxhighlight>'
}
}
},
'unixconfig': {
label: 'Config',
action: {
type: 'encapsulate',
options: {
pre: '<syntaxhighlight lang="unixconfig">',
post: '</syntaxhighlight>'
}
}
},
'apacheconf': {
label: 'ApacheConfig',
action: {
type: 'encapsulate',
options: {
pre: '<syntaxhighlight lang="apacheconf">',
post: '</syntaxhighlight>'
}
}
},
'json': {
label: 'JSON',
action: {
type: 'encapsulate',
options: {
pre: '<syntaxhighlight lang="json">',
post: '</syntaxhighlight>'
}
}
},
'patch': {
label: 'Patch',
action: {
type: 'encapsulate',
options: {
pre: '<syntaxhighlight lang="diff">',
post: '</syntaxhighlight>'
}
}
},
'php': {
label: 'PHP',
action: {
type: 'encapsulate',
options: {
pre: '<syntaxhighlight lang="php">',
post: '</syntaxhighlight>'
}
}
},
'javascript': {
label: 'JavaScript',
action: {
type: 'encapsulate',
options: {
pre: '<syntaxhighlight lang="javascript">',
post: '</syntaxhighlight>'
}
}
},
'html': {
label: 'HTML',
action: {
type: 'encapsulate',
options: {
pre: '<syntaxhighlight lang="html">',
post: '</syntaxhighlight>'
}
}
},
'css': {
label: 'CSS',
action: {
type: 'encapsulate',
options: {
pre: '<syntaxhighlight lang="css">',
post: '</syntaxhighlight>'
}
}
},
'arduino': {
label: 'Arduino',
action: {
type: 'encapsulate',
options: {
pre: '<syntaxhighlight lang="arduino">',
post: '</syntaxhighlight>'
}
}
},
'perl': {
label: 'Perl',
action: {
type: 'encapsulate',
options: {
pre: '<syntaxhighlight lang="perl">',
post: '</syntaxhighlight>'
}
}
},
'python': {
label: 'Python',
action: {
type: 'encapsulate',
options: {
pre: '<syntaxhighlight lang="python">',
post: '</syntaxhighlight>'
}
}
}
}
}
}
}
}
} );
});
}
and coolio, it works:
Technology democratizes nuclear-grade munitions
A bad day for FSDco shareholders.
The notification on his phone reminded him to connect his car to WiFi to receive the latest update, FSD V14.2.3.7 to correct a few minor errors and improve the music experience, offering fully dynamic live equalization and improved external noise cancellation.
The next day, he drove to work, everything behaving normally. The FSD feature worked as well as ever and he was sure he could hear the improvement in the sound system, jamming out, drumming his fingers on the steering wheel, windows down, making extended eye contact with pedestrians as he drove by. He couldn’t remember the last time he had to take control of the vehicle.
In his day job he oversaw a small team of developers connecting a programmatic AI to a remote quantum computer and the latest AI developed code had cut the execution time for a novel Shor’s algorithm implementation on the cloud Quantum Computer for 512 bit Elliptic Curve decode to under a minute while their newly developed quantum resistant signing algorithm had finally surpassed EC key resistance by a factor comfortably outside the statistical margin of error. The business model was to break everyone else’s certificates while offering a novel technology that unbroke the break they created, one they just happened to have patented, and progress was promising.
He joined the rush hour ride home, enjoying the freedom to continue his work on his phone as FSD navigated the heavy traffic.
It was a beautiful day across most of the US, warm spring weather brought lots of people out shopping and strolling along city and urban areas leading to some congestion in traffic routes. His FSD rerouted a new path through the city center but he didn’t pay any attention as the car often deviated off the most direct route as it dynamically responded to changing traffic conditions.
He was reviewing the results of the latest batch of QC test runs against a minor revision on his phone when his car gave a warning beep of imminent rear collision and before he could even look up, another FSD car passed going beyond full plaid mode, the wheel motors smoking as it streaked past into the intersection ahead, veered deftly around the lamp posts and onto the side walk and blasted through a pedestrian crowd, launching bodies and parts high into the air before it ran, speed barely slowed by the bodies, hard through a glass building front and exploded into flames, ripping out the far wall and causing the building to lurch precariously as glass and shattered mortar fell into the street.
He was boxed in between two Luddite gas cars in front and behind, looking on in horror, the sound of the screams of the survivors drowning out even the loud pop music blasting with improved fidelity in his car when the windows rolled themselves up and the doors locked themselves. Despite the changing acoustics, the sound system compensated dynamically, maintaining the same excellent balance and sound-stage imaging and creating a truly immersive, nearly live acoustic experience. The new noise cancellation algorithm immediately silenced the screams outside his car as the windows snuffed into their seals.
The Luddite car in front of him pulled off to the side clearing space in front and suddenly his car accelerated so hard his head snapped back against the seat and his phone went flying from his hands clattering against the rear window. While FSD had dulled the reflexes he had learned all those years ago driving his parent’s Luddite car, he still reflexively jammed unfamiliar feet in the direction of the manual pedals, mashing all of them to the floor. He was unsure, at first, if the car was trying to get him out of danger as it swerved through traffic accelerating as it went past the now burning building, but soon he realized in terror that his car had rejected his authority and any obligation of care for his safety.
Block by block he caromed past accident after accident: buildings on fire, pedestrians ripped to shreds, body parts and blood sprayed across streets and building facades each horrific track of destruction punctuated by the terminally deconstructed remains of an FSD car.
Soon, the car reached a clear spot and accelerated past 300 kph, smoke from the overheated wheel motors starting to infiltrate the cabin, the battery over-temp alerts lighting up the digital dashboard. His slow human reflexes barely had the cognitive processing speed to register a crowd of people in his path that had gathered around bodies lying in the street, attempting to give aid. Their silhouettes flashed on the in-car collision warning display and the car aimed directly at them calculating an optimal path of destruction at fully automated speed.
He tried to grab the steering wheel, clumsy hands spinning it easily without any effect. The FSD system aimed straight at the largest cluster and suddenly the windshield was covered in blood as the sound of body impacts arrhythmically penetrated the sound system’s otherwise excellent noise cancellation. He felt the car veer left and glimpsed the looming outline of an ambulance ahead between streaks and chunks obscuring his blood coated windscreen before the car exploded through the frame of the ambulance spraying burning lithium fragments down the street.
The hack had hit almost 3 million vehicles and of those, half had followed the new instructions: at 17:30 or as soon after as possible, navigate toward areas of high pedestrian congestion, wait for at least 500 m of clear road and when detected, lock the vehicle down, remove all power limits and accelerate to maximum velocity, scan for any cluster of more than 5 pedestrians tighter than one car width in the direction of travel and drive through them. Once there are no more pedestrians or if the batteries or wheel motors indicate imminent failure, aim for the next large target: either a vehicle or building.
More than 10,000,000 people were killed across the United States in less than 15 minutes. FSDco issued a remote shutdown within 30 minutes of the first accident data uplink and mostly ended the carnage. The attack was traced to a small insurgent group that had infiltrated the vehicle company. They had bribed a young, somewhat underpaid IT manager who had signed the insurgent’s modified firmware thinking it was a tuner’s test firmware and unaware of the real intentions or functions of the code. They’d then pushed a DNS poisoning attack through a popular but compromised smart speaker device, attacking owner’s private WiFi and redirected the scheduled patch download to their own server, pushing the legitimately signed but hacked firmware to about half the vulnerable vehicles.
FSDco had only noticed an atypically small update confirmation rate for their legitimate update the morning of the attack and had opened an investigation that day, but as the access compromise was edge and transient everything looked normal at the servers aside from the statistical anomaly and so no alarm was raised until the system-wide tracking dashboard lit up reporting a massive near-simultaneous loss of telemetrics event.
Most EVs get software updates from network connections and the driver has no possible way to know what they are. The updates are validated using cryptographic hashes, called certificates, which are supposed to be carefully controlled, but these can be and have been hacked. Further, there’s an assumption that certificates are secure enough to trust but occasionally a subtle error/malicious hack breaks the validity of that assumption. This has happened a number of times. Accident? Who knows?
Using vehicles as weapons is an obvious way to exploit a powerful kinetic device that easily gets past all security screens and can cause mass casualties. A significant expense for any attacker is that the attack modality also consumes operators (the driver) as well as the munition (the vehicle). It would be quite challenging to recruit large numbers of attackers to voluntarily engage in suicide vehicle attacks and so they remain relatively rare, despite almost always being successful. FSD vehicles provide an irresistible target for an extraordinary weapon of opportunity to any state or non-state antagonist.
A nuclear grade weapon for the price of a zero day
Each Tesla’s battery back is about 70 kWh (many quite a bit larger). There are about 5,000,000 Teslas on the road.
5,000,000 cars * 70 kWh/car = 350 gWh or 300 kt total energy capacity.
2017-09-03: the DPRK tested their largest yield nuclear weapon to date, an estimated 140 kt, possibly a thermonuclear bomb. The DPRK’s estimated spend on their nuclear program is about $642,000,000/year over 20 years or about $15,000,000,000 total, and that from a country with an estimated GDP of $28,000,000,000. The DPRK determined it was worth about 2.7% of their GDP over more than 20 years to be able to deliver 140 kt (or so) to an enemy country.
Why bother if the enemy has a more potent munition pre-emplaced; one that merely requires a hacked certificate to seize control?
Zero day exploits run about $1,400,000, 0.01% of the DPRK nuclear budget. A zero day would be a major expense for a non-state aligned armed insurgent group but hardly insurmountable. Pretty much every angry organized group in the world can scrape together a million bucks and an internet connection.
OTA upgradable, drive by wire vehicles give every one of them a cheap path to a nuclear-scale, pre-emplaced weapon of mass destruction. Recruitment is irrelevant and vehicles operating systems have no mercy.
Zero day exploits are sold and traded for every major platform regularly; the price varies depending on the size and overall security of the target platform. Zero days to hack voting machines or ATMs tend to be expensive. Hacks for Windows are cheap and plentiful, though price also varies with the mode of deployment. The easiest to deploy and so most valuable are “drive by” which means either figuratively—you visit a hacked web page and your phone is hacked—or literal—your car or phone passes a hacked/fake cellular base station or compromised WiFi access point and your phone or car is compromised, usually as stealthily as possible.
The reason Zero Day exploits are expensive is they’re fairly hard to find. They rely on things like buffer overruns and unsanitized inputs. Your phone gets asked to open a window 100×200 pixels, fine, but if it tries to open a window 36,000 pixels wide and the register space allocated for the multi-gigabyte image canvas bleeds into a critical system memory into which the hacker puts code instead of picture data where it is read, not by the GPU, but by the core OS and suddenly your phone has a new background feature you didn’t want. Drive-by hacking of SIMS is common, and a common hackable feature is enable silent SMS GPS coordinate location reporting, for example, to track a target.
Finding exploitable flaws where the original developers (probably) accidentally introduced a bug or a failure and then figuring out how do do something other than just crash or reboot the target device takes time and patience and so the hacks are valuable and sold on the dark web. If they’re used and someone notices, the error is fixed and the zero day has much less value. You get one try, like with the exploding pagers; nobody is carrying a Gold Apollo pager any more. So intelligence services and terrorist groups “bank” zero days and use them sparingly. If someone had figured out how to take control of FSD cars to implement a mass automated vehicle ramming attack it is very unlikely anyone would know until it was tried.
And AI programming tools should be able to find exploitable flaws much more quickly and far more cheaply. AI can also, by more or less the same process, find flaws for the good guys, white hat hackers, too so they can be fixed before they’re exploited by the black hat hackers, doing “penetration testing,” and “vulnerability scanning.” A problem is that such research is hard to differentiate from malicious attacks. And, at least for now, the good guys don’t put AI on the internet and let it try to hack people’s computers but the bad guys sure do.
FSD is a national security risk. Drive by wire vehicles are pre-emplaced munitions that can’t ever be secured and may be hacked any day or may have, long ago, already been hacked and are just sleeping, waiting for the kill command.
Optane, a modern technology tragedy (plus FreeBSD nvmecontrol)
Intel won the storage wars. They invented a storage technology in 2015 that was the best of everything: almost as fast as (then) RAM, basically infinite write endurance in any normal use, and fairly cheap. They even made a brilliant config on m.2 with integrated supercap for power-failure write flush. Just awesome and absolutely the write tech for modern file systems like ZFS. It is perfect for SLOGs. You wish you had a laptop that booted off an Optane m.2 You wish your desktop drives were all NVME Optane.
Well, wishes are all we got left, sadly. Optane, RIP 2022.
You can still buy optane parts on the secondary markets and it seems some of the enterprise DC products are at least still marked current on Intel’s website, but all retail stocks seem to be gone.
Man was that an amazing deal at $0.50/GB. In my application, the only practical form factor was M.2 and even that was a bit wonky in an HP DL360 G9, but more on that later. There are a variety of options and most are available on the used market:
| PN | Intro | Cap GB | Write MB/s | write k iops | PBW endurace | PLP | $ (market, 2024) |
| MEMPEK1W016GAXT | Q1’17 | 16 | 145 | 35 | 0.2 | NO | 5 |
| SSDPEL1K100GA | Q1’19 | 100 | 1,000 | 250 | 10.9 | YES | 109 |
| SSDPEL1K200GA01 | Q1’19 | 200 | 2,000 | 400 | 21.9 | YES | 275 |
| SSDPEL1K375GA | Q1’19 | 375 | 2,200 | 550 | 41 | YES | 800/1,333/NA |
| SSDPEK1A058GA | Q2’22 | 58 | 890 | 224 | 635 | YES | 32/140 |
| SSDPEK1A118GA01 | Q2’22 | 118 | 1050 | 243 | 1292 | YES | 70/229 |
Any of these would be a good choice for a SLOG on rotating media, but the later ones are just insane in terms of performance, and that’s compared to enterprise SSDs. They pricing cratered after they were canceled and dangit, didn’t get em. The used market has gone way up, better price increase than bitcoin over the same period and they’re not virtual beanie babies! The SSDPEL1K100GA is the best deal at the moment and has a beefy supercap for power continuity and is still $818 on Amazon, apparently introduced at $1,170. This pricing might have explained why Optane didn’t do better. The 375 GB M.2 would be an awfully nice find at $0.50/GB, that’d be a pretty solid laptop boot disk.
Hardware
For SLOG you really want two devices mirrored in case one fails. The risk of an optane DC grade device failing is trivial and given it has Power Loss Protection, the most likely cause of failure and why your main array failed to write out the transactions committed to the SLOG, we’re really talking about media failure and as it is 3D X-Point it is NOT going to wear out like NAND, it’s rational to single-disk it. I almost striped mine but in the end decided against it because that quadruples the fail rate over a single device and 8x over mirrored and I don’t really need the space.
So how do you install two M.2 devices in a computer that doesn’t have M.2 slots on the mobo? With a PCI card, of course. But wait, you want two in a slot, right? And these are x4 devices, the slots are x8 or x16, so two should be able to pair, right?
Not so fast. Welcome to the bizarre world of PCI furcation. If you want to add two drives to the core PCI bus, you have to split the bus to address the cards. Some mobos support this and others do not. As shipped, the HPE DL360 G9 did not.
BUT, a firmware update, v 1.60 (April 2016) added “support to configure the system to bifurcate PCIe Slot 1 on the DL360 Gen9 or PCIe Slot 2 on the DL380 Gen9.” W00t. A simple Supermicro AOC-SLG3-2M2 supports 2x M.2 cards and only requires bifurcation to work, all good.
Not so fast. In order to pack the DL360 G9 with 2.5 SSDs, you need a Smart Array Controller (set for passthru for ZFS) and that sits in slot 1 and while I believe it can go in any X16 slot, the cabling is not compatible and that’s a lotta SAS cables to replace. Bifurcation on the mobo is out.
But you can fucate on a PCI card just as well – likely this adds some latency and it’d be interest to perf test against more direct connections. I ended up choosing a RIITOP dual M.2×22110 PCI card and it worked out of the box transparently, both disks showed and while I’m not getting 250,000 IOPS, performance is good. It is based on the ASMedia ASM2812, seems like a reasonable chip used in a lot of the lower cost devices of this type, most with 4x M.2 slots instead of 2.
Software
FreeBSD recognizes the devices and addresses them with nvmecontrol. You can pull a full status report with, for example nvmecontrol identify nvme0, which provides information on the device or nvmecontrol identify nvme0ns1 which gives details about the storage configuration, including something important (foreshadowing) the LBA format (probably #00, 512).
Current LBA Format: LBA Format #00 ... LBA Format #00: Data Size: 512 Metadata Size: 0 Performance: Good LBA Format #01: Data Size: 512 Metadata Size: 8 Performance: Good LBA Format #02: Data Size: 512 Metadata Size: 16 Performance: Good LBA Format #03: Data Size: 4096 Metadata Size: 0 Performance: Best LBA Format #04: Data Size: 4096 Metadata Size: 8 Performance: Best LBA Format #05: Data Size: 4096 Metadata Size: 64 Performance: Best LBA Format #06: Data Size: 4096 Metadata Size: 128 Performance: Best
The first thing I’d do with a used device is wipe it:
gpart destroy -F /dev/nvme0 gpart destroy -F /dev/nvme1
I would not bother formatting the device to LBA 03/4k. Everyone tells you you should, but you don’t get much of a performance increase and it is a huge pain because nvmecontrol currently times out after 60 seconds (at least until the patch needed is pushed to kernel or you recompile your kernel with some fixes) if you did want to try, you’d run:
# time nvmecontrol format -f 3 -m 0 -p 0 -l 0 nvme0 316.68 real 0.00 user 0.00 sys (no errors)
-f 3 sets LBA Format #03, 4096 which should give “Performance: Best” which certainly sounds better than “Good.”
But it’ll error out. You need to mod /usr/src/sys/dev/nvme/nvme_private.h with the below modifications and recompile the kernel so it won’t time out after 60 seconds.
#define NVME_ADMIN_TIMEOUT_PERIOD (600) /* in seconds def 60 */ #define NVME_DEFAULT_TIMEOUT_PERIOD (600) /* in seconds def 30 */ #define NVME_MIN_TIMEOUT_PERIOD (5) #define NVME_MAX_TIMEOUT_PERIOD (600) /* in seconds def 120 */
Performance Aside
I tested 512 vs 4k in my system – and perhaps the AIC’s bridge latency or the whole system’s performance so limited the performance of the optane cards that a no difference would appear, these cards do rock at the hardware level (this is with 4k formatting):
# nvmecontrol perftest -n 32 -o read -s 4096 -t 30 nvme0ns1 && nvmecontrol perftest -n 32 -o write -s 4096 -t 30 nvme0ns1 Threads: 32 Size: 4096 READ Time: 30 IO/s: 598310 MB/s: 2337 Threads: 32 Size: 4096 WRITE Time: 30 IO/s: 254541 MB/s: 994
That’s pretty darn close to what’s on the label.
However, testing 512 vs. 4k formatting at the OS level (didn’t test raw) it was a less extraordinary story:
| LBA/FW ver. | 4k E2010650 | 512 E2010650 | 4k E2010485 | 512 E2010600 |
| Median Mb/s | 759.20 | 762.30 | 757.50 | 742.80 |
| Average Mb/s | 721.70 | 722.87 | 721.64 | 724.35 |
Definitely not +10%
So I wouldn’t bother reformatting them myself. Testing a few configurations with
fio --name=random-write --ioengine=posixaio --rw=randwrite --bs=64k --numjobs=1 --size=4g --iodepth=1 --runtime=60 --time_based --end_fsync=1
I get
| Device\Metrics | Max IOPS | Avg WBW MiB/s | avg SLAT µS | avg LAT µS |
| 10 SAS SSD ZFS Z2 Array | 20,442 | 1,135 | 4,392 | 53.94 |
| Optane 100G M.2 Mirror | 20,774 | 624 | 3,821 | 95.77 |
| tmpfs RAM disk | 23,202 | 1,465 | 6.67 | 42 |
Optane is performing pretty close to the system limit by most metrics – the SLAT and LAT metrics are highly dependent on software.
Formatting
I did something a bit funky since 100GB is way more than this little server could ever use for SLOG. I set it at 16GB which is probably 4x overkill, then used the rest as /var mountpoints for my jails because the optanes have basically infinite write endurance and the log files in var get the most writes on the system. I’m not going into much detail on this because it’s my own weird thing and chances anyone else cares is pretty small.
Initialize GPT
gpart create -s gpt nda0 gpart create -s gpt nda1
Create Partitions
gpart add -b 2048 -s 16g -t freebsd-zfs -a 4k -l slog0 nda0 gpart add -b 2048 -s 16g -t freebsd-zfs -a 4k -l slog1 nda1 gpart add -s 74g -t freebsd-zfs -a 4k -l ovar0 nda0 gpart add -s 74g -t freebsd-zfs -a 4k -l ovar1 nda1
ZPool Operations
zpool add zroot log mirror nda0p1 nda1p1 zpool create optavar mirror nda0p2 nda1p2 zpool set autotrim=on optavar
Create Datasets
zfs create -o mountpoint=/usr/local/jails/containers/jail/var -o compression=on -o exec=off -o atime=off -o setuid=off optavar/jail-var etc
Electronic Signatures and PDF
Electronic signatures are a technology that has been bizarrely slow to mature. Lots of documents still rely on the idiotic premise that some stupid graphic somehow serves as a secure measure of document authenticity. This might have had some slight measure of validity in the days of actual paper documents being required with “wet signatures.” but the premise of face-to-face document signing ceremonies should have long been consigned to history with signet rings, let alone a global transit trade in random bits of paper bearing binding proof of commitment.
First the Uniform Electronic Transactions Act (UETA, 1999) then H.R.1714/S.761, Electronic Signatures In Global and National commerce (E-Sign) act (2000) was signed (ha) into law (probably with a wet signature), now Public Law 106–229, it has been legally binding to sign documents with electronic signatures for 25 years.
So why is it almost never done? Why do are we still sometimes asked to fax “signed” documents?
Why do we fax “signed” documents? Because lawyers and legislators are unbelievably, almost incomprehensibly ignorant of the most basic operational functions of technology and absolutely too stupid, too utterly moronic, mindbogglingly dense and incomprehensibly dumb that… and I am NOT making this up… but seriously… there are people who actually have an impact on laws and legal matters who believe that fax transmissions are more “secure” and less prone to interception, manipulation, or hacking than email. Yes, people who believe this kind of thing are actually allowed to practice law. Truly tragic but still true. The world suffers that such profound ignorance persists.
Have you ever tried to electronically sign a document? Turns out it isn’t trivial and the burden isn’t the core technology or concept but a few problematic implementation steps.
The first barrier is the interjection of the certificate mafia’s profit motives. Various corporate monsters saw an opportunity to make bank exploiting the aforementioned abject technical ignorance and utter technical incompetence of our legislative and legal infrastructure and build a certification model that relies on pay-for-validation, lying that this would somehow ensure authenticity and people were too dumb to question the obvious idiocy of this stupid model. Even today, we rely on the good graces of the Mozilla foundation’s Let’s Encrypt to make secure communication viable because various OS and browser level dumbness considers self-signed certificates insecure for the stupidest, most reprehensible reasons possible. But Let’s Encrypt, bless them, won’t give you an X.509 signing certificate.
We’re all lucky CACert.org steps into this horrific void and, while it is complicated, offers an extremely secure, highly reliable, and (most importantly) free process for getting yourself an X.509 signing certificate. In order to get a signing certificate, you have to validate your identity in person at a meet up using their points system, a process that is infinitely more secure than any of the for-profit signing certificate providers that consider willingness to pay proof of identity. The USG should offer X.509 client certificates for free with passports and RealID renewals, but I’d still use CACert myself, cause they’re awesome.
For now: first, set up an account on CACert, install their root certificates in your OS and browser (why aren’t they included by default? Ask the certificate mafia.) You’ll need to do one of the things they require to prove you are who you claim (yes, actual security, unlike ANY of the commercial certificate providers, unreal how insanely stupid this process is) and then have CACert issue a Client Certificate.
Assuming you have your points with CACert, the basic process is fairly well documented:
You need to generate a signing request in your name, which you can do with OpenSSL, but it is easier using CACert’s nice online process.
It will take a few seconds (60?) and I’m not sure about the compatibility problems that might arise from a longer key, there are some bugbears once we try to use lamo corporate commercial software, but 4096 worked for me. You MUST GET YOUR PRIVATE KEY and save it to your OpenSSL enabled computer.
Remember to press the red “Show private key” button and copy/save the private key to a secure directory on your computer, you’re gonna need it later to convert the certificate into something dumb ass spyware Windows computers can use, which you need because Acrobat forms still can’t be signed without Adobe’s awful spyware Acrobat Reader.
(note the actual private key has quite a bit of text between the Begin and End lines but you know… redacted for privacy). Then click the blue “Copy CSR to Clipboard” button and switch over to New Client Certificate window and paste it where you’re supposed to.
You need the .crt version of the certificate to continue and that private key text file you saved earlier for the next step, as well as downloading the CACert root certificate and then you need openssl working (should be on most real computers, Windows or Apple is beyond my interest) and merely execute this one simple command:
$ openssl pkcs12 -keypbe PBE-SHA1-3DES -certpbe PBE-SHA1-3DES -nomac -export -out DavidGessel_3d_sha1.pfx -inkey private_key_for_CAcert_CSR.txt -in gessel@blackrosetech.com.crt -certfile CA_Cert_root_X0F.crt
To explain:
- openssl will generate a combined binary version of your certificate in pkcs12 format
- because Windows and Acrobat suck, you have to specify moderately insecure crypto: SHA1-3DES rather than the Linux default of AES 256 because why would a monopoly company like Microsoft have any incentive to fix bugs? If you don’t Windows will tell you “The password you entered is incorrect” to unlock your key because why fix bugs when corporate IT types are just utterly incompetent and will only specify windows no matter how awful and unusable it is because point-n-click?
- -nomac is another setting Windows needs to be able to use the cert and if you don’t specify this Windows will tell you “The password you entered is incorrect” again, because Windows does not care if it works for you because you have no choice.
- The -out certificate is what’s being generated and Windows native is .pfx, but .p12 will work too.
- The -inkey is the private key you remembered to save using the red button before (right? you need that).
- the -in (file) is the Client Certificate in normal X.509 .crt format real computers understand that CACert generated for you.
- the -certfile is CACert’s root certificate.
Now, WØØt, you have a certificate that should work. Go over to your dumb Windows machine and make sure you import the CACert root certificates – you just download them and then right click and select “install certificate” for the class 1, then the class 3, then the .pfx certificate you just created.
Now, finally, you can sign a document like someone who actually uses a computer rather than a quill and parchment to process documents.
Acrobat is another program that just doesn’t care too much about usability or user experience, so different versions might work differently. I had to click the “Signature Panel” button to open a sidebar to show the signature fields then right click and then choose my sig and click sign and save.
One final note about the state of signing in FOSS: it kinda sucks still. Various entities that use acrobat fairly well will generate forms with standard signature locations which you can print and sign and fax (not email) like we’re still waiting for Y2K or print and sign and snail mail if we are nostalgic for the pre-telephone era, or click and sign and email like we’re in the 21st century.
I’m not aware of any FOSS program that handles signature fields in the expected way. You can sign a whole pdf document with a variety of FOSS tools, and CACert has a good summary of these, but that signature, while binding on the document as a whole does not show in the form fields and so whatever non-tech functionary is asking you to sign the document is never going to understand how your e-sign compliant signature is binding and is going to insist you take a time machine back to the mid-80s to find a working fax machine unless you use Acrobat, which means Windows or Mac at least in a VM. You might be able to get some version of Acrobat to work in Wine, but you’ll need an old one that uses an internal certificate store rather than relying on the windows version (pre Acrobat X, I’m pretty sure).
Fun, huh? Basic digital functions are still broken decades after introduction but we have AI generated Teledep influencers telling us doubleplus buy useless beauty products and trust their health and exercise advice.
TB 128 is coming. Lock your doors and hide your wife.
Update: Jan 2025.
After much gnashing of teeth and the usual anger and hatred of the TB team that still clearly thinks plugin developers are a nuisance and anyone not doing email the way TB thinks you should out of the box is fundamentally doing it wrong and an email heathen, the TB community has mostly fixed things.
Of my collection of plugins, one, thunderHTMLedit, elected to abandon the platform and moved somewhere better: https://www.betterbird.eu/addons/. The new version, 3.4.0, seems to work correctly with 128.
Another fairly important plugin has not yet been updated, ConfirmBeforeDelete. If you have cats that walk on your keyboard, this could save much confusion and annoyance. Unfortunately, this hasn’t been updated yet: https://github.com/caligraf/ConfirmBeforeDelete/issues/25. Seems like there’s a possible fix, hopefully soon.
Some minor update made 115 burn a lot of CPU and it’s keeping my fan noisy. The early TB burning CPU fixes aren’t cutting it and any bug report is going to start with upgrading to the latest. I’m going to have to live without the cat delete protection for at least a little bit, alas.
TB128 did solve the 30% CPU usage that hit after an OS update and reboot and no number of reboots could fix. I am missing ConfirmBeforeDelete, an essential feature bizarrely overlooked in the core. I do hope it gets updated. Now to watch warily for the next feature-destroying update.
Oh dear, Thunderbird’s 115 to 128 upgrade is rumored to be as catastrophic as some of their past upgrades. Every now and then the UX designer types decide they have no purpose in life except to screw up everyone else’s day and teach them why they, kids who think tiktok is the best app ever, need to change every grown-up’s email interface to match their hipsterisms.
A slightly edited rant I posted about this for a previous widely hated, horribly disruptive, mostly useless “update” that was forced on the user community is still as relevant as it ever was:
A core philosophical failure that blights the development process and engenders much frustration and gnashing of teeth among the loyal, long term users is the assumption that what the developers believe is progress is what the users need, it ain’t Brondo.
All of us who are (or were) users of TB (or any program) became so because it did something we needed it to do. Changing that function, even cosmetically, is going to hurt some users, even if there are valid and rational arguments why it is an improvement. In an environment that embraces the concept of radical consent in so many ways, stuffing an upgrade down the throat of an existing user without even a “hey, you want this?” is inconsistent with modern social norms of enlightened behavior.
It is possible, of course, to reject upgrades, but it is a huge hassle and means blocking all updates, including security fixes, just to avoid the addonpocolypse ones like 68, 78 and 115 that utterly destroy functionality for months, even years as the still (amazingly) dedicated developer community finds ways around problematic API changes (even accepting the changes are meaningful security changes or driven by FF codebase migrations that would be unreasonable to expect to preserve without doing a waterfox kind of fork). The default is to auto-update, as we do for most programs and most of the time without the radical trauma of some TB updates.
Not every update breaks add-ons but some do (incorrectly). Add-ons generally do not self-update on unsupported versions (correctly). TB should consider add-ons first class code citizens, not trivially disposable cosmetics. They are operational modifications to a vehicle not JC Witney double stick racer mods. Users had a legitimate need to modify the base operation of the program, sought out and applied a solution and it isn’t acceptable to negate that effort without warning. Every TB update should do what plugins do: check compatibility and refuse to update until the user manually disables any incompatible plugins rather than simply disabling them for the user and saying “check it out! new! Sleek!” (sorry, nothing works anymore, but looks nice, huh?). All reasonable efforts should be made to preserve compatibility indefinitely, but that’s obviously not always possible to keep the code base manageable and secure, and it is fine to inform users that there are important security reasons that require breaking plugins, but never, ever to do so without informed, affirmative consent.
As for UI/UX changes, I’d argue that the philosophy should be no change, not even trivial cosmetic ones, are permitted on an existing install. Ever. If you started with TB 0.1 your 115 interface should look exactly like this:
Unless you explicitly gave informed, affirmative consent to UI/UX changes.
Save your mail
Well, they haven’t followed my advice, but there are workarounds to mitigate the harm of having the devs jam yet another unwanted “upgrade” into your day: don’t upgrade! You don’t have to and while it may make sense after some time, it almost never makes sense to take one of these, what used to be VNUM changes updates early. Ah, yet another dumb change, the loss of PNUM: so many projects adopted the utterly idiotic flat numbering system that doesn’t differentiate between minor bug fixes and major changes in any useful way. Who thought that was a good idea and what can we do to ensure they never work in technology again? Why is my install jumping from 115 to 128? The level of dysfunction is, certainly, frustrating. I guess it is more sensible than 3.1, XP, 95, 98, Vista, 10, 11.
Most automatic updates are either done by your operating system or initiated by the app itself. As I’m on Linux, the examples will be from my environment, but there are plenty of windows guides around.
Block Application-Driven Updates
To stop the app from self-destructing with some stupid upgrade, you need to create a special file in a special, super secret location and to make it even more fun, the location depends on what operating system you’re using and how you installed the application. if you have restored your menu bar, the one they deleted some time ago, it’s easy, otherwise it’ll be a “hamburger icon” or some other dumb variation:
Help menu -> Troubleshooting Information
Application Binary /usr/lib/thunderbird/thunderbird means the application binary is in the directory /usr/lib/thunderbird/ inside of which (navigating by your file browser or command prompt) you should find a distribution directory.
Inside the distribution directory, create a file named policies.json, this is a protected directory so the linux command will be something like sudo nano policies.json. The text of the file should read:
{
"policies": {
"DisableAppUpdate": true,
"DisableFeedbackCommands": true,
"DisableSystemAddonUpdate": true,
"DisableTelemetry": true,
"ExtensionUpdate": false
}
}
Disabling extension updates is debatable but major API fixes mean sometimes an extension will update to the new version and stop working on existing. Extension developers are usually pretty good about avoiding that (unlike the core devs) so consider that particular line debatable.
Block OS-Driven Updates
This is a purely for Ubuntu derivatives that support Synaptic or similar. Launch Synaptic, find the thunderbird entry and selected Package -> Lock Version.
These fixes should keep you safe from massive disruption. It can take months or more, for essential Add Ons to get updated and functional or for someone to develop a compatible alternative. The thunderbird dev team just doesn’t care about either Add On developers or users but, unfortunately, it is the only usable email tool left on the desktop that supports sadly essential HTML mail.
Claws… I get it, html mail is a pox on humanity, but please reconsider:
Claws Mail will not let you write and send HTML emails or other kind of annoyances, hence it may not be the software you need in some business environments.
Being Right isn’t always write.
Goodbye, Tortuga.
On April 21st, 2024, at 20:39, Tortuga was gently put to rest after a three year-long struggle with what was probably cancer and a short-lived victory over a mycoplasma bacterial blood infection.
She first took advantage of our yard-cat support program in 2009 as a juvenile cat and passed at about 15 or 16 years of age. She lived a good life, had 5 kittens on March 27th, 2010 that were all weaned and adopted out successfully, and grew old never wanting for food, shelter, or comfort, and never suffering any meaningful illness or injury until her last year.
Over the years, she was the beneficiary of a very strong community support network that took her in whenever she needed it and gave her loving care. She had housemates, human and feline, and a few canine over the years and was always gracious and pleasant, if not always enthusiastic about the four-legged companions.
I am eternally, deeply grateful to everyone who helped her over the years and who made my work and travel possible and Tortuga’s life pleasant and comfortable in my absence, especially in her later years as she needed more care.
She was the best, sweetest cat I’ve ever known. She was always polite, always pleasant, and never scratched or bit, not even when startled or annoyed by dogs. She never broke things or pushed things over or made a mess.
She wasn’t a big fan of other cats, and only a select few were tolerated as guests in her garden. She wanted to start every morning by marking her territory and she ruled her garden with a fierceness that vastly exceeded her tiny size. She started there, spent her last day in the sun there, and will spend eternity there.
Almost every night I was home, she slept in my bed with me. Almost every day I was working at home, she would hop up and sleep quietly between my keyboard and monitor on her little bed there. She didn’t meow much or fuss but purred easily and happily.
In later years, she’d sometimes wake me just before light by prodding my back or nipping to ask for pets; after 5 or 10 minutes of purring and being petted, she would settle back to sleep. It was a ritual that I came to very much enjoy.
Whenever I came home from my travels, no matter how late, as I opened the door into the living space, I’d hear her stir, jumping down from the attic maybe or from my bed or the window perch upstairs and tap-tap-tap down the stairs and trot up to greet me, rubbing my leg and purring. She’d let me scoop her up and snuggle her, though she wasn’t normally a carry cat, and then walk circles around me for 10 or 15 minutes, welcoming me home in the sweetest way possible. She came to know my departures too and always gave me a look of disappointment, sometimes refusing to come to the door to see me off, but usually relenting for one last scritch on the head.
When Corona hit in 2020, I was in Iraq after leaving her in January of 2020 thinking I’d be back in the spring. I couldn’t make it home for almost two years. The longest I’d been away before then was less than 6 months and even that only once or twice. She’s a cat, and by then an old cat, so I didn’t expect much, but in January of 2022, I opened the door late at night to the sound of her tap-tap-tapping down the stairs to greet me.
She was laid to rest in the garden she ruled for 15 years.
I went through the thousands of pictures I’ve taken of her and others have shared with me and tried to find a few from every year from her first foray in 2009 until her last day. If you knew her at some point during this time, I hope this brings back fond memories of a very special kitty.
2009: Tortuga finds food, takes over a house, and becomes part of the family.
2010 Tortuga has kittens and settles into her role as queen of the garden.
2011 Tortuga takes ownership of my desk.
2012
2013
2014
2015
2016
2017
2018
2019
2020 Corona time.
2021 Corona time.
I didn’t get to see Tortuga at all from January of 2020 until January of 2022.
2022 Reunited.
2023
A typical welcome home when I’d been away too long.
2024 The queen of the garden forever.
A one page home/new tab page with random pictures, time, and weather
Are you annoyed by a trend in browsers to default to an annoying advertising page with new tabs? I sure am. And they don’t make it easy to change that. I thought, rather than a blank new tab page, why not load something cute and local. I enlisted claude.ai to help expedite the code and got something I like.
myHomePage.html is a very simple default page that loads a random image from a folder as a background, overlays the current local time in the one correct time format with seconds, live update, and throws up the local weather from wttr.in after a delay (to avoid hitting the server unnecessarily if you’re not going to keep the tab blank long enough to see the weather).
Images have to be in a local folder and in a predictable naming structure, as written “image_001.webp” to “image_999.webp.” If the random enumerator chooses an image name that doesn’t exist, you get a blank page.
Browsers don’t auto-rotate by exif (or webp) metadata, so orient all images in the folder as you’d like them to appear.
The weather information is only “current” which isn’t all that useful to me, I’d like tomorrows weather, but that’s not quite possible with the one-liner format yet.
Update, I added some code to display today and tomorrow’s events and current todos meeting specific filter tests from your Thunderbird calendar, if you have it. If not, just don’t cron the bash script and they won’t show. I also changed the mechanism of updating the weather to a 30 minute refresh of the page itself, this way you get more pix AND the calendar data updates every 30 minutes. Web browsers and javascript are pretty isolated from the host device, you can’t even read a local file in most (let alone write one). All good security, but a problem if you want data from your host computer in a web page without running a local server to deliver it.
My work around was to write the data into the file itself with a script. Since the data being written is multi-line, I opted to tag the span for insert with non-breaking spaces, a weird character and the script sanatizes the input from calendar events extracted from the sqlite database in case some event title includes them. The current config is by default:
~/.myHomePage/myHomePage.html ~/.myHomePage/getEvents.pl ~/.myHomePage/getToDos.pl ~/.myHomePage/putEvents.py ~/.myHomePage/putToDos.py ~/.myHomePage/myHomeImages/image_001.webp ~/.myHomePage/myHomeImages/image_002.webp etc.
How to set the homepage and new tab default page varies by browser. In Brave try hamburger→settings→appearance→show home button→select option→paste the location of the homepage.html file, e.g. file:///home/(username)/.myHomePage/myHomePage.html
Then just set a cron script like <code>*/30 * * * * /home/<username>/.myHomePage/getEvents.pl</code> for regular updates: script all the subroutines that are useful or write a little bash script to do them in sequence and call that with your favorite periodic method.
Parsing recurring events in perl is a challenge and I managed to get claude to ragequit, that’s got to be a some sort of a record:
So the parsing scripts are in python using icalendar, I put them on gitlab at https://gitlab.com/gessel/myhomepage to make it a little easier to mess with, if anyone wants to.