Treegraph.sh a tool for generating pretty file structure graphs

Friday, February 28, 2025 

Linux has a nice little tool in the repositories called tree generates a nice console window-based directory map from the current location down. It’s great, but there’s not an equivalent in FreeBSD, though you can get the basic need met using find and some text processing pipes like

find . -type d -maxdepth 3 | sed -e "s/[^-][^\/]*\// |/g" -e "s/|\([^ ]\)/|-\1/"
.
 |-subdir0
 | |-subsubdir1
 | |-subsubdir0
 |-subdir3
 |-sub dir 2

However, this is enough to make me wish for something more visual. As usual, someone else already did it, but their version wasn’t quite what I wanted so I enlisted Claude.ai to help generate a little script that, like the above command, finds it’s way through a directory structure and builds a .dot file map as it goes, which can be converted into useful graphic formats with graphviz.

tree graph of a directory structure

The code (below or at gitlab) is pretty easy to use, it does the dotfile generation then just parse the dotfile like usual with graphviz, either using an installed version or online.

treegraph.sh -d /usr/home -f -s -m 2 > tree.dot
dot -Tsvg tree.dot > tree.svg

The edges seem to overlap a bit more than I’d like, but that’s an issue with graphviz and if you really want a presentation ready graph can be adjusted manually with invisible nodes or edges and manually set connection points to merge lines. It defaults to dot, which makes sense for ranks, but looks cool with neato or twopi as well.

#!/usr/local/bin/bash

# Help function
Help() {
   echo "Usage: treegraph.sh [options]"
   echo
   echo "Options:"
   echo "  -h         Display this help message"
   echo "  -f         Enumerate both directories and files (default: directories only)"
   echo "  -d DIR     Specify the root directory to parse (default: current directory)"
   echo "  -m DEPTH   Specify the maximum depth of parsing (default: no limit)"
   echo "  -s         Compute the size of folders and files using 'du -sh' (default: off)"
   echo
   echo "Examples:"
   echo "  treegraph.sh                         # Current directory, directories only"
   echo "  treegraph.sh -d /usr/home -f         # Parse /usr/home, include files"
   echo "  treegraph.sh -d /usr/home -m 3       # Parse /usr/home with max depth 3"
   echo "  treegraph.sh -s                      # Show sizes for current directory"
   echo "  treegraph.sh -d /usr/home -f -s -m 2 # All options combined"
   echo
   echo "Example generating an SVG file (requires graphviz installed):"
   echo "  treegraph.sh -d /usr/home -f -s -m 2 > tree.dot"
   echo "  dot -Tsvg tree.dot > tree.svg"
   echo
   echo "When using -s, nodes are color-coded by size:"
   echo "  B (bytes)  - gray50"
   echo "  K (KB)     - black"
   echo "  M (MB)     - blue2"
   echo "  G (GB)     - darkorange3"
   echo "  T (TB)     - crimson"
}

# Initialize variables with defaults
StartDir="."
Type="d"
MaxDepth=""
MaxDepthArg=""
ShowSizes=0

# Process options
while getopts ":hfsd:m:" option; do
   case $option in
      h) # Display help
         Help
         exit;;
      f) # Include files
         Type="f";;
      s) # Show sizes
         ShowSizes=1;;
      d) # Directory
         StartDir="$OPTARG";;
      m) # Maximum depth
         MaxDepth="$OPTARG"
         MaxDepthArg="-maxdepth $MaxDepth";;
     \?) # Invalid option
         echo "Error: Invalid option"
         Help
         exit 1;;
      :) # Missing argument
         echo "Error: Option -$OPTARG requires an argument"
         Help
         exit 1;;
   esac
done

# Build find command based on Type
if [ "$Type" = "f" ]; then
    TypeArg=""  # No type filter for both files and directories
else
    TypeArg="-type d"  # Only directories (default)
fi

# Get absolute path of starting directory for proper path handling
StartDirAbs=$(cd "$StartDir" 2>/dev/null && pwd) || StartDirAbs="$StartDir"

# Extract base directory name for display
BaseDir=$(basename "$StartDirAbs")

# If showing sizes, get root directory size
if [ $ShowSizes -eq 1 ]; then
    RootSizeInfo=$(du -sh "$StartDir" 2>/dev/null | awk '{print $1}')

    # Determine color based on size unit
    if [[ "$RootSizeInfo" =~ [0-9]+B ]]; then
        RootColor="gray50"
    elif [[ "$RootSizeInfo" =~ [0-9]+K ]]; then
        RootColor="black"
    elif [[ "$RootSizeInfo" =~ [0-9]+M ]]; then
        RootColor="blue2"
    elif [[ "$RootSizeInfo" =~ [0-9]+G ]]; then
        RootColor="darkorange3"
    elif [[ "$RootSizeInfo" =~ [0-9]+T ]]; then
        RootColor="crimson"
    else
        RootColor="black"  # Default color
    fi

    SizeLabel=" ($RootSizeInfo)"
    ColorAttr=", fontcolor=\"$RootColor\""
else
    SizeLabel=""
    ColorAttr=""
fi

# Run find and generate graphviz dotfile
(
echo "digraph directory_structure {"
echo "    node [shape=box,style=rounded,width=0.1,height=0.1,margin=\"0.07,0.01\"];"
echo "    edge [arrowhead=dot, arrowtail=dot, dir=both, arrowsize=0.3, color=gray];"
echo "    nodesep=0.15;"
echo "    rankdir=LR;"
echo "    concentrate=true;"
echo "    overlap=false;"
echo "    splines=true;"
echo "    searchsize=10000;"
echo "    \"$StartDir\" [label=\"$BaseDir$SizeLabel\"$ColorAttr];"

# Process find output but skip the root directory itself
find "$StartDir" $TypeArg $MaxDepthArg | grep -v "^$StartDir\$" | awk -v start_dir="$StartDir" -v type="$Type" -v show_sizes=$ShowSizes '
{
    current=$0;

    # Check if this is a file or directory
    is_file = 0;
    if (type == "f") {
        cmd = "test -f \"" current "\" && echo 1 || echo 0";
        cmd | getline is_file;
        close(cmd);
    }

    # Get the parent path
    parent = current;
    sub(/\/[^\/]*$/, "", parent);  # Remove last component for parent
    if (parent == "" || parent == ".") parent = start_dir;

    # Extract just the name of the current node (not the full path)
    split(current, path_parts, "/");
    node_name = path_parts[length(path_parts)];

    # Get size if requested
    size_info = "";
    color_attr = "";
    if (show_sizes == 1) {
        cmd = "du -sh \"" current "\" 2>/dev/null | awk '\''{print $1}'\''";
        cmd | getline size_info;
        close(cmd);

        if (size_info != "") {
            # Determine color based on size unit
            if (size_info ~ /[0-9]+B/) {
                color = "gray50";
            } else if (size_info ~ /[0-9]+K/) {
                color = "black";
            } else if (size_info ~ /[0-9]+M/) {
                color = "blue2";
            } else if (size_info ~ /[0-9]+G/) {
                color = "darkorange3";
            } else if (size_info ~ /[0-9]+T/) {
                color = "crimson";
            } else {
                color = "black";  # Default color
            }

            size_info = " (" size_info ")";
            color_attr = ", fontcolor=\"" color "\"";
        }
    }

    # Create node with appropriate shape and just the name as label
    if (is_file) {
        printf "    \"%s\" [shape=plain, width=0, margin=0, label=\"%s%s\"%s];\n", current, node_name, size_info, color_attr;
    } else {
        printf "    \"%s\" [label=\"%s%s\"%s];\n", current, node_name, size_info, color_attr;
    }

    # Create node connection
    printf "    \"%s\" -> \"%s\";\n", parent, current;
}
'

echo "}"
) # End of generated dotfile

 

Posted at 10:47:36 GMT-0700

Category: CodeFreeBSDTechnology

FINAL System shutdown after 16 years

Wednesday, February 12, 2025 
root@gamanjiru:/usr # shutdown -p now
Shutdown NOW!
shutdown: [pid 14442]
root@gamanjiru:/usr #                                                                                
*** FINAL System shutdown message from gessel@gamanjiru.blackrosetech.com ***

System going down IMMEDIATELY                                                  

                                                                            
System shutdown time has arrived
Connection to gamanjiru.blackrosetech.com closed by remote host.
Connection to gamanjiru.blackrosetech.com closed.

DISCONNECTED (PRESS <ENTER> TO RECONNECT) (Tue Feb 11 21:39:45 2025)

Almost 16 years ago, in July of 2009, I bought a used IBM x3655 7985-1AU off Ebay for $202.50 to replace my IBM Netfinity 5500 (4U!, dual dual 500MHz Xeo 1GB RAM (later upgraded to 4GB) and 5x 9.1GB HDs for $217.50 ) that had been running since 2004. That had, itself, replaced a dual socket generic deskside machine, a box Mark Godwin gave me back in the Nebucon days, that first went live on the interwebs running FreeBSD 2 in April of 1998 under black-rose.org. As of this post: 26 years, 10 months of FreeBSD hosted internets.

Those were the magic days of ebay: in 2008, just a year earlier, I’d quoted a similar x3650 (Intel E5410-based), 32GB but only a pair of crappy consumer SATA 500GB drives for $7,297.00.

x3655

The new x3655 came with 32GB of RAM and dual-core AMD processor and an RSA-II.  The original motherboard firmware only supported dual core AMD processors, not the then brand new AMD Opteron quad core, so I bought a somewhat hard to find 43W7343 motherboard for $190 (and an additional 5 fans to max out cooling for $18 each) and then a pair of AMD Opteron 2352 2.1GHz CPUs and swapped the mobo and the CPUs and the heat sinks and the fans. Note that it is really hard to find data on the dual quad core option, it was a bit of a hot rod.

I added the 2x drive expansion module, another ebay find, and loaded the drive bays with 8x 26K5657 10k 72G 2.5″ SAS drives (at about $65 each, used) on the ServeRAID 8k and expanded the RAM to 64GB, 57856 MB available with RAM sparing set for reliability. The modified machine reports itself as an x3655 7943-AC1.

7943 AC1

I ended up creating an 8×1 RAID array to pass to ZFS, ZRAID2, which mean I had a battery backed write cache I could count on. The system has dual power supplies, each connected to a 2200 VA (hacked XR) UPS.

Over the years, almost plural decades, of continuous operation I’ve lost maybe 4 drives.  Once, while I wasn’t paying much attention, somewhere over about 6 months two drives failed and I got close to catastrophic failure, but ZFS pulled through. It started life on FreeBSD 7 and I took a chance on the then brand-new experimental release of ZFS. Over the years it ran most of the releases up to 12, stalling there as the decision was made to shift to new hardware and I’m pretty sure I’ve run every release of FreeBSD except 1.0 and 13.

It was a fast machine, 8⨉ 2100.12-MHz K8-class physical cores, more than enough to run mail and various web services, everything compiled for the AMD Barcelona architecture. The pre-Lenovoization of the IBM x86 hardware was really first rate, top of the line data center gear and it shutdown without a flaw, same performance and configuration as when I fired it up, used and already off-list, 16 years earlier.

It was never too slow, even now, even as OS’s have expanded and the total number of ports needed for basic services has grown by about 10x given ever spiraling dependencies. It wasn’t that it was slow, but that it used far too much power and electricity got more and more expensive in CA.

So I migrated to a new box, took the better part of a year to spare time migrate all of the jail services from the old machine and, increasingly unsupported FreeBSD 12 OS to new HPE DL360 G9 (ebay, inflation: $497) running FreeBSD 14, added a poudriere build environment on a DL60 and carefully tuned the kernels for power efficiency not bad for 10 disks, 20 cores, 192GB: 114W.  Now there are 20 physical cores and 40 virtual and yet, clearly showing the limits of Moore’s law, the new box’s E5-2630 v4s are only 2.20GHz: 5x the execution cores, but only 5% faster clock. 16 years of progress.

Good night, sweet prince.

Posted at 19:44:46 GMT-0700

Category: EventsFreeBSDTechnology