Resolving ZFS Error on Alpine Linux

My alpine setup is modest and pretty usual. I installed it back in 3.19 times with the main partition using Ext4 and all data on ZFS. Why didn’t I install the root file system on ZFS? Well, I needed the machine quickly and installing using default settings instead of messing with ZFS was a faster way to do it. And, as it often happens, temporary installation became permanent.

As Alpine Linux is on 3.20.2 now, I felt my system might do well with a bit of upgrading. It’s easy after all. I just changed my /etc/apk/repositories ti point toward the latest stable repositories:

https://dl-cdn.alpinelinux.org/alpine/latest-stable/main
https://dl-cdn.alpinelinux.org/alpine/latest-stable/community

And followed this with standard update-upgrade dance:

apk update
apk upgrade

One short reboot later and my system was upgraded! Oh, yeah, and my data was gone. What gives?

Fortunately for my blood pressure, I quickly determined that my data was not really gone but just hiding as my ZFS modules weren’t loaded and all ZFS commands advised me to do modprobe zfs. I followed the same only to be greeted by an error message:

modprobe: ERROR: could not insert zfs: Invalid argument

I tried removing and readding packages, fixing them, and making many more small adjustments. Pretty much any command that the internet had to offer, I tried. But all those things always brought me back to the same cryptic message.

At the end, I decided to fall forward. If one upgrade broke the system, maybe another upgrade would solve it. And yes, someone smarter would probably go with a downgrade instead, but I don’t roll that way. It was either upgrade or reinstall for that naughty server.

Where do you upgrade from 3.20, you ask? Well, there’s always an “edge”. And upgrade to it was again just a minor change to /etc/apk/repositories followed by update/upgrade:

https://dl-cdn.alpinelinux.org/alpine/edge/main
https://dl-cdn.alpinelinux.org/alpine/edge/community

Wouldn’t you know it, edge was fine and my ZFS was buzzing along once more.

But, while I didn’t mind reinstalling this machine, keeping it on edge long-term (as my “temporary” projects tend to get), didn’t really give me a level of confidence I wanted. So I decided either 3.20 would work or I would reinstall it fully now I knew for sure my data was fine.

How do you downgrade to 3.20? Well, the first step is to change /etc/apk/repositories yet again:

https://dl-cdn.alpinelinux.org/alpine/latest-stable/main
https://dl-cdn.alpinelinux.org/alpine/latest-stable/community

The difference is that this time I wanted to force a downgrade of installed packages thus slightly modified commands:

apk update
apk upgrade -a

After a reboot, my system happily presented itself in all its 3.20 glory with ZFS loaded without any further issues.

What was the problem? I have no idea. However, apk package handling and painless upgrade/downgrade is growing on me.

Grayscale Avalonia Icons

For disabled icons in Avalonia toolbar, you can go two ways. One is just using an external tool to convert your existing color icons into their color-less variant and have them as a completely separate set. The one I prefer is to actually convert images on demand.

As I’m currently playing with this in Avalonia, I decided to share my code. And it’s not as straightforward as I would like. To start with, here is the code: As I’m currently playing with this in Avalonia, I decided to share my code. And it’s not as straightforward as I would like. To start with, here is the code:

public static Bitmap BitmapAsGreyscale(Bitmap bitmap) {
    var width = bitmap.PixelSize.Width;
    var height = bitmap.PixelSize.Height;

    var buffer = new byte[width * height * 4];
    var bufferPtr = GCHandle.Alloc(buffer, GCHandleType.Pinned);
    try {
        var stride = 4 * width;
        bitmap.CopyPixels(default, bufferPtr.AddrOfPinnedObject(), buffer.Length, stride);

        for (var i = 0; i < buffer.Length; i += 4) {
            var b = buffer[i + 0];
            var g = buffer[i + 1];
            var r = buffer[i + 2];

            var grey = byte.CreateSaturating(0.299 * r + 0.587 * g + 0.114 * b);

            buffer[i + 0] = grey;
            buffer[i + 1] = grey;
            buffer[i + 2] = grey;
        }

        var writableBitmap = new WriteableBitmap(new PixelSize(width, height), new Vector(96, 96), Avalonia.Platform.PixelFormat.Bgra8888);
        using (var stream = writableBitmap.Lock());
        Marshal.Copy(buffer, 0, stream.Address, buffer.Length);

        return writableBitmap;
    } finally {
        bufferPtr.Free();
    }
}

Since Avalonia doesn’t really expose pixel-level operations, first we need to obtain values of all the pixels. The easiest approach I found was just using the CopyPixels method to get all the data to our buffer. As this code in Avalonia is quite low-level and requires a pointer, we need to have our buffer pinned. Anything pinned also needs releasing, thus our finally block.

Once we have raw bytes, there is just a matter of figuring out which byte holds which value, and here I suspect that pretty much anybody will use the most common RGBA byte ordering. It’s most common by far, and I would say it will be 99% what you end up with.

To get gray, we can use averages, but I prefer using slightly more complicated BT.601 luma calculation. And yes, this doesn’t take into account gamma correction; nor is it the only way to get a grayscale. However, I found it works well for icons without much calculation needed. You can opt to use any conversion you prefer as long as the result is a nice 8-bit value. Using this value for each of RGB components gives us the gray component. Further, note that in code above, I only modify RGB values, leaving the alpha channel alone.

Once the bytes are in the desired state, just create a WritableBitmap based on that same buffer and with the same overall properties (including 32-bit color).

Dealing with X11 Primary Clipboard under Avalonia

It all started with Avalonia and a discovery that its clipboard handling under Linux doesn’t include the primary buffer. Since I was building a new GUI for my password manager this was an issue for me. I wanted to be able to paste directly into the terminal using Shift+Insert instead of messing with the mouse. And honestly, especially for password prompts, having a different result depending on which paste operation you use is annoying at best. So, I went onto building my own X11 code to deal with it.

The first idea was to see if somebody else had written the necessary code already. Most useful source for this became mono repository. However, its clipboard support was intertwined with other X11 stuff. It was way too much code to deal with for a simple paste operation but I did make use of it for figuring out X11 structures.

What helped me the most was actually seeing how X11 clipboard handling was implemented in C. From that I managed to get my initialization code running:

DisplayPtr = NativeMethods.XOpenDisplay(null);
RootWindowPtr = NativeMethods.XDefaultRootWindow(DisplayPtr);
WindowPtr = NativeMethods.XCreateSimpleWindow(DisplayPtr, RootWindowPtr, -10, -10, 1, 1, 0, 0, 0);
TargetsAtom = NativeMethods.XInternAtom(DisplayPtr, "TARGETS", only_if_exists: false);
ClipboardAtom = NativeMethods.XInternAtom(DisplayPtr, "PRIMARY", only_if_exists: false);
Utf8StringAtom = NativeMethods.XInternAtom(DisplayPtr, "UTF8_STRING", only_if_exists: false);
MetaSelectionAtom = NativeMethods.XInternAtom(DisplayPtr, "META_SELECTION", only_if_exists: false);
EventThread = new Thread(EventLoop) {  // last to initialize so we can use it as detection for successful init
  IsBackground = true,
};
EventThread.Start();

This code creates a window (XCreateSimpleWindow) for an event loop (that we’ll handle in a separate thread) and also specifies a few X11 atoms for clipboard handling.

In order to set clipboard text, we need to tell X11 that we’re the owner of the clipboard and that it should use our event handler to answer any queries. I also opted to prepare UTF-8 string bytes so we don’t need to deal with them in the loop.

private byte[] BytesOut = [];

public void SetText(string text) {
  BytesOut = Encoding.UTF8.GetBytes(text);
  NativeMethods.XSetSelectionOwner(DisplayPtr, ClipboardAtom, WindowPtr, 0);
}

But the real code is actually in our event loop where we wait for SelectionRequest event:

private void Loop() {
  while (true) {
    NativeMethods.XEvent @event = new();
    NativeMethods.XNextEvent(DisplayPtr, ref @event);

    switch (@event.type) {
      case NativeMethods.XEventType.SelectionRequest: {
        var requestEvent = @event.xselectionrequest;
        if (NativeMethods.XGetSelectionOwner(DisplayPtr, requestEvent.selection) != WindowPtr) { continue; }
        if (requestEvent.selection != ClipboardAtom) { continue; }
        if (requestEvent.property == IntPtr.Zero) { continue; }

        // rest of selection handling code
        break;
    }
  }
}

There are two subrequests possible here. The first one for the other application asking for text formats (i.e., TARGETS atom query). Here we can give it UTF8_STRING atom that seems to be universally supported. We could have given it more formats but I honestly saw no point in messing with ANSI support. It’s 2024, for god’s sake.

if (requestEvent.target == TargetsAtom) {
  var formats = new[] { Utf8StringAtom };
  NativeMethods.XChangeProperty(requestEvent.display,
                                requestEvent.requestor,
                                requestEvent.property,
                                4,   // XA_ATOM
                                32,  // 32-bit data
                                0,   // Replace
                                formats,
                                formats.Length);

  var sendEvent = ... // create XSelectionEvent structure with type=SelectionNotify
  NativeMethods.XSendEvent(DisplayPtr,
                           requestEvent.requestor,
                           propagate: false,
                           eventMask: IntPtr.Zero,
                           ref sendEvent);
}

After we told the terminal what formats we support, we can expect a query for that data type next within the same SelectionRequest event type. Here we can finally use previously prepared our UTF-8 bytes. I opted to allocate a new buffer to avoid any issues. As in the previous case, all work is done by setting the property on the destination window (XChangeProperty) with XSendEvent serving to inform the window we’re done.

if (requestEvent.target == Utf8StringAtom) {
  var bufferPtr = IntPtr.Zero;
  int bufferLength;
  try {
    bufferPtr = Marshal.AllocHGlobal(BytesOut.Length);
    bufferLength = BytesOut.Length;
    Marshal.Copy(BytesOut, 0, bufferPtr, BytesOut.Length);
  }

  NativeMethods.XChangeProperty(DisplayPtr,
                                requestEvent.requestor,
                                requestEvent.property,
                                requestEvent.target,
                                8,  // 8-bit data
                                0,  // Replace
                                bufferPtr,
                                bufferLength);
  } finally {
    if (bufferPtr != IntPtr.Zero) { Marshal.FreeHGlobal(bufferPtr); }
  }

  var sendEvent = ... // create XSelectionEvent structure with type=SelectionNotify
  NativeMethods.XSendEvent(DisplayPtr,
                           requestEvent.requestor,
                           propagate: false,
                           eventMask: IntPtr.Zero,
                           ref sendEvent);
}

And that’s all you just need in order to support SetText. However, it seemed like a waste not to implement GetText method too.

The code for retrieving text is a bit more complicated since we cannot retrieve text directly. We must ask for it using our UTF8_STRING atom. However, we cannot read the clipboard text directly but only via our event loop. So, we need to wait for our AutoResetEvent to signal data is ready before returning.

private readonly AutoResetEvent BytesInLock = new(false);
private byte[] BytesIn = [];

public string GetText() {
  NativeMethods.XConvertSelection(DisplayPtr,
                                  ClipboardAtom,
                                  Utf8StringAtom,
                                  MetaSelectionAtom,
                                  WindowPtr,
                                  IntPtr.Zero);
  NativeMethods.XFlush(DisplayPtr);

  if (BytesInLock.WaitOne(100)) {  // 100 ms wait
    return Encoding.UTF8.GetString(BytesIn);
  } else {
    return string.Empty;
  }
}

In the event loop, we need to add an extra case for SelectionNotify event where we can handle reading the data and signaling our AutoResetEvent.

case NativeMethods.XEventType.SelectionNotify: {
  var selectionEvent = @event.xselection;
  if (selectionEvent.target != Utf8StringAtom) { continue; }  // we ignore anything not clipboard

  if (selectionEvent.property == 0) {  // nothing in clipboard
    BytesIn = [];
    BytesInLock.Set();
    continue;
  }

  var data = IntPtr.Zero;
  NativeMethods.XGetWindowProperty(DisplayPtr,
                                   selectionEvent.requestor,
                                   selectionEvent.property,
                                   long_offset: 0,
                                   long_length: int.MaxValue,
                                   delete: false,
                                   0,  // AnyPropertyType
                                   out var type,
                                   out var format,
                                   out var nitems,
                                   out var bytes_after,
                                   ref data);
  BytesIn = new byte[nitems.ToInt32()];
  Marshal.Copy(data, BytesIn, 0, BytesIn.Length);
  BytesInLock.Set();
  NativeMethods.XFree(data);
}
break;

With all this code in, you can now handle primary (aka, middle-click) clipboard just fine. And yes, the code is not fully complete, so you might want to check my X11Clipboard class, which not only provides support for primary but also for a normal clipboard too. E.g.:

X11Clipboard.Primary.SetText("A");
X11Clipboard.Primary.SetText("B");