I’m usually stubborn when it comes to medical matters, refusing to go to the doctors if I’m not in too much inconvenience, but thankfully this time I thought to myself that this wasn’t normal, nothing I had experienced before, and decided to head into A&E to get checked out.

As it turned out, I had a coronary incident, a heart attack in common lingo. Thankfully it was a ‘small big heart attack’ in the words of the consultant. I didn’t need any surgery, or a ‘stent’ placed in my artery, as by the time I got to the Liverpool Heart & Chest Hospital the block had cleared itself. I have also been diagnosed with heart failure; scary big words but all it means is that my heart is operating below expected.

As you can imagine this was the singular most frightening experience of my life. I’m now approaching a month after the event and I think for the first time I can write about this without having some sort of anxiety attack over the thought of it. For a few moments in the hospital, I had thought it was the end, not because I was actually at any risk, but more that my unhealthy lifestyle had finally caught up to me and was presenting real, in your face, issues.

I have several risk factors, I’m obese, Type 2 Diabetic, and have high blood pressure. I didn’t know at the time that I also had high cholesterol. My blasé attitude about my health was down to not seeing any real impacts on my life, I was just a ticking time bomb waiting to pop.

…and it happened.

My old life is over. I can’t do this to myself any more.

In the past month after leaving the hospital, I have started eating a healthy, balanced diet. I’ve dropped nearly 10% of my body weight (which is insane to think of as I type this!), my T2D is under control, and my blood pressure is the best it’s been in years.

Don’t be me. Everything that happened to me was easily preventable if only I cared to change. No doctor is going to dismiss you if you want to get an ‘MOT’ and check yourself out, catching the contributors early can stop it.

Without these wonderful people, I might not be here today. At some point I’ll be doing some fundraising for them, but that will be after I recover a bit more.

• Northwest Ambulance Service
• Liverpool Heart and Chest Hospital

• On the ‘Back To Work’ podcast, Episode 617. Merlin discusses the upcoming tvOS 17 and how they’ve introduced Continuty to allow you to Facetime using your iPhone.

• A modified Honda Monkey motorbike breaks the world record for the distance travelled on a single tank of fuel. Sure, the tank is 30 gallons, but impressive non the less, especially since they didn’t run out of fuel when they got to their target location.

• Cabel Sasser bought and scanned a auction lot of “Backstage Disneyland”, which was a unofficial magazine targetted at Disneyland cast members. The full collection is available on archive.org.

• Linking to the last one, Jason Schultz has created an amazing archive of Disneyland assets called Parkendium. He was also able to provide Cabel with the missing issues of Backstage Disneylad.

• Saurabh made a post about Advanced macOS Commands that you may not know about, which spurred an entire conversation on HackerNews about other useful tools and commands.

Thats it for now, other posts in this series will be tagged with #found-this-week, so check back later.

I’m currently trying to setup a Kiosk-mode Raspberry Pi for Leigh Hackspace to act as a ‘Hackspace Status’ screen, showing some stats and a rotating list of artwork to show upcoming events.

I decided to just try and build something myself, but while I was using the Raspberry Pi Imager I spotted a section for purpose-specific OS installations, and within there was Anthias. “Hey, that looks good, Web UI and does what I need it to do” I thought to myself. Anthias is the rebadged Screenly OSE, as to avoid confusion they renamed it to split it away from the commercial Screenly option. Open source versions of commercial products never really work well, but maybe it’ll do the bare minimum I need.

I got it installed on a Raspberry Pi 3, booted it up and it sat on the boot-screen and did nothing else. Oh, did it need some pre-config or something? I plugged the SD card into another system and it mounted ‘resin-boot’, a sign of using BalenaOS rather than Raspbian. I headed back to the website and checked for documentation and nothing on the site?

So, I found the Github repository, “General Documentation”, great. First port of call in the document: “SSH into the host”. Try to connect, “connection refused”. Oh, ok, do I need to enable SSH? The document links to the official Raspbian docs on how to do that but it isn’t running Raspbian…

All official documentation has been exhausted, time to consult the forums. Many people seem to report the same as me:

• I simply can’t get Anthias to work!
• Installing Anthias (pi3) from the Raspberry Pi Imager v1.7.4 just hangs at the logo
• Only shows the Anthias Logo

But, it quickly became obvious what the issue was: Using the official image that uses BalenaOS.

As it turns out, it straight up doesn’t work, to the point that every response seems to be ‘I used Raspberry Pi OS Lite and installed it via the script, and it works there’, even the developer of the product posts the same.

I understand that Screenly has no real investment in this open-source version of their product, but straight-up shipping a broken version of your product into the official Raspberry Pi imaging tool seems dumb on a new level.

I’ve given up and moved on to trying FullPageOS, it might not have the flashy UI but I can manage that via Ansible.

Yesterday I attempted this. At Leigh Hackspace we wanted a webcam on our out-of-band access box, just so we (as in the infra guys) can look at the status of the rack without visiting the space. After having a dig around our electronics area I found a box of old Trust WB-1200P webcams, they’re from 2009 and barely scrape out at 0.1 megapixel, but they should be good enough to see the blinkenlights in the rack to see if we have an issue (e.g. a server doesn’t have power).

I grab one, plug it into the OOB Raspberry Pi running the latest, Debian 11, Raspian and it is detected and loads the drivers! It works??? Wow.

The Trust WB-1200P is a rebadged ‘Pixart Imaging Inc.’ PAC207, a pre-UVC webcam that is part of the gspca driver set in the kernel. Support for it is included in the current upstream main, and support V4L2. But, getting it to load a driver is only the first step of the battle. Pre-UVC devices have their own special quirks and issues, usually related to the pixel and video formats they output.

In this case, the PAC207 has its own special pixel format that the application you wish to use needs to understand and handle. While backward-compatible kernel drivers are there, application support moves on. Most of the V4L2 tools I attempted to use would either throw an error, crash or cause the driver to spit out some errors into dmesg

[39011.323706] gspca_main: set alt 0 err -32
[39011.323771] pac207 1-1.2:1.0: submit int URB failed with error -2
[39181.529143] Transfer to device 5 endpoint 0x5 frame 1542 failed - FIQ reported NYET. Data may have been lost.
[39191.828144] Transfer to device 5 endpoint 0x5 frame 1601 failed - FIQ reported NYET. Data may have been lost.
[39450.446323] gspca_main: set alt 0 err -22
[39454.058295] gspca_main: pac207-2.14.0 probing 093a:2468

I suspected it may be due to the output format of the camera, most applications expect YUV or Motion JPEG, neither of which this camera supported. I attempted to Google the camera’s model to try and find some more information, then tried Github to see if anything caught my eye.

#define V4L2_PIX_FMT_SPCA508  v4l2_fourcc('S', '5', '0', '8') /* YUVY per line */
#define V4L2_PIX_FMT_SPCA561  v4l2_fourcc('S', '5', '6', '1') /* compressed GBRG bayer */
#define V4L2_PIX_FMT_PAC207   v4l2_fourcc('P', '2', '0', '7') /* compressed BGGR bayer */
#define V4L2_PIX_FMT_MR97310A v4l2_fourcc('M', '3', '1', '0') /* compressed BGGR bayer */
#define V4L2_PIX_FMT_JL2005BCD v4l2_fourcc('J', 'L', '2', '0') /* compressed RGGB bayer */

Hello, V4L2_PIX_FMT_PAC207? That is a special format that so happens to have the same name as the model of the webcam.

Further splunking using this value directed me to a configuration file for Motion, a relatively old application for streaming webcams

# V4L2_PIX_FMT_MJPEG then motion will by default use V4L2_PIX_FMT_MJPEG.
# Setting v4l2_palette to 2 forces motion to use V4L2_PIX_FMT_SBGGR8
# instead.
# V4L2_PIX_FMT_SGRBG8  : 5  'GRBG'
# V4L2_PIX_FMT_PAC207  : 6  'P207'
# V4L2_PIX_FMT_PJPG    : 7  'PJPG'

Motion is still maintained, and still available in most OS repositories, and it did exactly what I needed it to do; export a camera to a URL so we can look at it. Win-win. I installed Motion, added v4l2_palette 6 to the configuration file, started up the daemon and hit the URL, it loaded and…. showed a green image and more dmesg errors.

As it turns out, not everything is still compatible. the pac207 driver is a V4Lv1 type driver and requires a small amount of wrangling to get working correctly. Thankfully, the V4L2 developers provide a compatibility library that can be loaded using LD_PRELOAD to fix the major issues.

All thats needed is the libv4l-0 library on Debian, then using a systemd service override I was able to insert the environment variable into the startup:

/etc/systemd/system/motion.service.d/overrides.conf

[Service]
Environment=LD_PRELOAD=/usr/lib/arm-linux-gnueabihf/libv4l/v4l1compat.so

Restart the service and…

Right, 0.1MP, and probably not focused correctly, autofocus wasn’t a thing on this!

I found out later that Trust still has this device listed on their website’s support section, from 2009! It also has Windows 8 drivers!

In my case, my controller is a pod on my Kubernetes cluster and doesn’t have much of a Layer 2 presence on the network. The USW Mini depends on the L2 features of the controller to enable adoption, but with a little bit of work, you can get them added. The switches support using DHCP to gather their configuration information, but information on how to configure this outside of Ubiquiti’s gateway devices is difficult to find. First of all, they make use of DHCP Option 43, and the value in this option should be an encoded version of your controller’s IP. I use pfSense as my DHCP server, so here is what I did:

• Go to the Services -> DHCP Server in pfSense
• On the interface, your switch is on, scroll to the bottom of the page and click to expand Additional BOOTP/DHCP Options
  • In the Option box, put 43
  • Set the Type value to String
  • Convert the IP address of your controller into Hex format
  • Add 01:04 to the *Value field, followed by the Hex IP address, e.g. 01:04:FF:FF:FF:FF
• Click Save

Now, you can restart your USW-Mini, and it should appear as adoptable in the controller interface.

After trawling Github and some discussions around Mastodon, I came across a useful post by Tero Keski-Valkama detailing how to use the Rails console to run some fixes over your data. I noticed that some were a little heavy-handed and caused a lot of media to be re-downloaded, so I used by very basic Ruby knowledge to try and reduce the workload on the DB.

So here is the snippet I came up with:

MediaAttachment.cached.where.not(remote_url: '').each do |attachment|
  def save = false
  if attachment.file? && !attachment.file.exists?
    attachment.file.destroy 
    save = true
  end
  if attachment.thumbnail? && !attachment.thumbnail.exists?
    attachment.thumbnail.destroy
    save = true
  end
  if save
    puts attachment
    attachment.save
  end
end

This will check every cached media attachment, and if the file or thumbnail doesn’t exist it’ll clear them and save the record. The next time Mastodon looks at that MediaAttachment it should identify that the file needs re-downloading from the original source.

Hope this is useful for someone, it took about 5-10 minutes to run on my small instance with about 50GB of cached files (according to the database).

[Edit - 2023/02/07]

As it turns out, this still isn’t the magic bullet to resolve missing media, some items are still showing as dead images, but it did take care of the vast majority. I should spend more time investigating why they’ve been missed, but by that time my cronjob to cleanup remote media would of already nuked them.

The site at Bring Back Blogging holds a list of everyone involved, the barrier for entry is small, and if you have your own publishing space then I’d suggest you join in!

Thankfully, Evan Philip submitted a PR to clean them up using the existing tootctl command. The change consisted of a single file and it is easy to patch into an existing installation while awaiting 4.0.3 or 4.1. For me, our Mastodon instance is run in a home Kubernetes cluster, so it’s not a simple case of replacing a file on a filesystem. This type of patching can be done in Kubernetes, it’s just not as obvious how to do it.

ConfigMap and Secret objects can be mounted into a Pod much like a PVC. So by creating a ConfigMap and defining it as a volume in a Pod you can override any files within the container itself:

First of all, you want to create a ConfigMap with the new file. The filename here doesn’t matter, but for ease, you can keep it consistent:

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: media-cli-patch
  namespace: web
data:
  media_cli.rb: |
    # frozen_string_literal: true

    require_relative '../../config/boot'
    require_relative '../../config/environment'
    require_relative 'cli_helper'

    module Mastodon
      class MediaCLI < Thor
        include ActionView::Helpers::NumberHelper
        include CLIHelper

    ...    

Then, you need to update your Pod definition to load in the ConfigMap object as a volume. First, add it to the volumes definition and give it a name:

  volumes:
    - name: system
      persistentVolumeClaim:
        claimName: mastodon-system
    - name: media-cli-patch
      configMap:
        name: media-cli-patch

Then, add it to your volumeMounts section:

  volumeMounts:
    - name: system
      mountPath: /opt/mastodon/public/system
    - name: media-cli-patch
      mountPath: /opt/mastodon/lib/mastodon/media_cli.rb
      subPath: media_cli.rb

The mountPath is the full path within the Pod where it is to be mounted, and the subPath value refers to the filename you gave it in the ConfigMap. Push the resource to the cluster and you’ll have a pod running with the modified file. I use a CronJob to run these routine clean-up jobs, the Web and Sidekiq instances don’t need the patched file as it only applies to commands run via tootctl.

Here is my full CronJob for context:

---
apiVersion: batch/v1
kind: CronJob
metadata:
  name: mastodon-cron
  namespace: web
spec:
  schedule: "0 * * * *"
  successfulJobsHistoryLimit: 0
  failedJobsHistoryLimit: 1
  concurrencyPolicy: Forbid
  jobTemplate:
    spec:
      template:
        spec:
          containers:
            - name: mastodon-cleanup
              image: tootsuite/mastodon:v4.0.2
              command: ["/bin/sh"]
              args:
                ["-c", "tootctl media remove --days=1 --prune-profiles && tootctl preview_cards remove --days=14"]
              envFrom:
                - configMapRef:
                    name: mastodon-config
                - secretRef:
                    name: mastodon-secrets
                - secretRef:
                    name: mastodon-postgresql-auth
              volumeMounts:
                - name: system
                  mountPath: /opt/mastodon/public/system
                - name: media-cli-patch
                  mountPath: /opt/mastodon/lib/mastodon/media_cli.rb
                  subPath: media_cli.rb
          restartPolicy: Never
          volumes:
            - name: system
              persistentVolumeClaim:
                claimName: mastodon-system
            - name: media-cli-patch
              configMap:
                name: media-cli-patch

My workshop started as a small hobby hole, I had been gifted a wood turning course in 2016 by my wife, I was unsure if its something i’d be interested in but I did have the drive to try something different from IT. In conversation I had brought up the idea of doing blacksmithing one day, and my wife did look for something but couldn’t find anywhere local that runs a from basics course. Instead, she found a wood turning course run by Cheshire Wood Working. I was initially unsure, but after the two day course I was transfixed.

I had been working on the idea of buying a lathe, pricing up a few options, but was always stuck of where I could put it. I live in a relatively large town house, but I have no garage or space in the garden to place a shed. I’m not sure how I come to look at storage units but I found a local company that offered powered workshop spaces, after a quick conversation I jumped at the opportunity and signed a lease. It was a 150 sqft storage unit at around £320/month, exceedingly expensive but it gave me the space I needed. The same day I headed down to Axminster Tools and purchased my lathe.

After paying a substantial amount of money for the last year for what was essentially a hobby I decided I needed to make at least some income to subsidize the overall cost. One day while I was watching Game Night! on the BoardGameGeek channel I spotted a large meeple in the background of the video. A meeple is a name for a common token used across a lot of modern boardgames, but first came about from Carcassonne. I searched eBay and Etsy and discovered that very few people were making large meeples, as in bigger than three inches. I thought maybe I had found a market.

I produced a selection of solid wood meeples, plywood painted meeples, and some other gaming related items. Meeples became a major income source over the next few years and it subsidised the workshop and my hobbies for three years. With that my tools and workshop expanded, learning new skills to create interesting an different treen. Working in the workshop was something so different from IT, when people asked why I always used this to explain:

In IT you can work hard all week and produce nothing, with wood working you can work hard and you always end up with something, even if it is a pile of scrap wood and sawdust.

Unfortunately COVID-19 started up in early 2020 and kept me locked out of the workshop for a few months, orders dropped off and it became problematic to sustain the workshop. Orders did pickup mid 2020 and it did end up as my most successful year, 2021 had a slight drop off from there, but early 2022 I hit zero, zero orders for three months. COVID had been quite hard on my fiances for numerous reasons, and as long as the workshop contributed some money back into the pot I could take the hit of the remaining rental, but it wasn’t sustainable. At zero sales I had to make the hard decision to close it.

I’m still not 100% sure what has happened, but I assume that 2020/2021 was so successful due to the lockdowns being experienced worldwide, people couldn’t get out to spend their money on other things so gifs and oddities was a easy spend. Now that travel has resumed and the world is opening up again in 2022 I guess that everyone has much better things to spend on. Maybe like me, the pandemic finances finally caught up to people, and with the rising fuel costs and inflation we’ve just hit a breaking point. I don’t begrudge anyone, my customers were subsidizing a hobby and side business after all.

As my wife says, everything is temporary, maybe in a year or so I can start again. For the moment, thank you Unit G/055 - you will be missed.

For me, RSS never really went anywhere. People like to call it dead after Google Reader shut down, if anything it unified the remaining users into generating new tools and applications to consume RSS. On the website front, RSS is still there just not front and centre, and most large websites still publish their RSS feeds. I’ve personally been using Miniflux as my primary RSS feed consumption tool for a couple of years now, its incredibly easy to self host with only Postgresql as a dependency, and it has a lot of tools built in to manage even difficult RSS feeds that mangle the output.

Tom Critchlow popped up on HackerNews with a post about Increasing the surface area of blogging which discusses RSS, OPML, and personal feeds of news and information. Sharing is a key component of this ecosystem, so I thought i’d take a crack at showing my Miniflux feeds out on my Hugo website.

The Script

First of all, I needed to get the OPML data out of Miniflux. This is quite simple to do as it provides an API which is amazingly simple to use. Using Python 3, Requests and no other external modules I put together a small script that would extract out the OPML from the API and write out the raw XML, as well as an optional JSON formatted version.

miniflux2opml.py

To run it, all you need to do is provide your instance URL (or the hosted version at https://reader.miniflux.app) and your API token. It’ll then connect to Miniflux and pull down the OPML to your console. If you want to write it out to a file, use -o, and if you want to write out the JSON formatted version, use -j.

Hugo Shortcode

Now that we’ve got a script extract the data into a JSON format, we need to parse it in Hugo. Thankfully, Hugo supports arbitrary data files stored in data of your site root, these can then be accessed via .Site.Data values within your templates.

Using that method, I created a shortcode under layouts/shortcodes/blogroll.html:

<p><small>Last Updated: <b>{{ dateFormat "2006-01-02 03:04" (.Site.Data.feeds.opml.head.dateCreated | time) }}</b></small></p>

{{ $opmlJson := index .Site.Data.feeds "opml" "body" "outline" }}
{{ range sort $opmlJson "_text" "asc" }}
    <h2>{{ ._text }}</h2>

    {{ if gt (len .outline) 0 }}
    {{ range .outline }}
    <li><a href="{{ ._htmlUrl }}" target="_blank">{{ ._title }}</a></li>
    {{ end }}
    {{ end }}

{{ end }}

When then can be used in a simple post:

---
title: "Blog Roll"
date: 2022-04-30
draft: false
toc: false
images:
tags:
    - blogroll
referral: false
---

My blogroll is a list of sites and feeds I follow using [Miniflux](https://miniflux.app). This list is auto generated on a weekly basis using a small Python tool that wrangles the XML OPML format into JSON that Hugo can use nicely.

{{< blogroll >}}

Can you can see the result here

Automating the updates

I don’t want to manually run this command every so often, ideally it should be automated and left on auto pilot. I use Github for my repository so i’m able to make use of Github Workflows to regularly run the script and commit it to the repository.

name: Pull OPML from Miniflux
on:
  workflow_dispatch:
  schedule:
    # * is a special character in YAML so you have to quote this string
    - cron:  '0 5 * * 6'
jobs:
  pull_opml:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Download and Commit OPML
        run: |
          tools/miniflux2opml.py -u https://rss.doofnet.uk -t '${{ secrets.MINIFLUX_API_KEY }}' -o static/feeds.opml -j data/feeds.json
          git config --global user.name 'Andrew Williams'
          git config --global user.email 'nikdoof@users.noreply.github.com'
          git add static/feeds.opml data/feeds.json
          git commit -am "Update OPML"
          git push          

With this the feed is pulled and updated on a weekly basis, and my OPML file is also put into the static folder to ensure its available to the public. Now people an be subjected to the weird content I read in a automated way.

In my previous post I discussed switching over to Obsidian for my “PKM” with a plan to create a set of tooling around migrating from LogSeq. If anyone had checked out the promised repository then you would of seen that it is empty and has been untouched since the original post.

The issue I had is that I had no easy way to actually migrate my data over, every method was messy and ended up with me having to rewrite any notes into a handy format. I attempted this for a few days but eventually gave up and went back to LogSeq. The problems I had discussed in the previous post are still there, while a mobile application is in the process of being developed we still have no simple way to sync between multiple systems. In the past month, LogSeq finally removed the last parts of the Github integration that allowed for easy pushing of changes into a git repo, for me that killed my last benefit.

So, I made the jump to Supernotes.

Supernotes

I still don’t know how I came across Supernotes, but it was a note-taking application directed more towards research than the daily journaling / PKM method that I use, but it had them few key features that pulled me in:

• Foward and Back linking.
• A mobile application for iOS.
• Markdown writing style.
• A REST API.

Supernotes is a freemium model, you can have 50 notes free, and a few extra for each person you recommend, or you can pay £8/month for unlimited notes and access to the mobile apps on iOS and Android. I’ve stumped up for the premium option out of the gate, while I could of worked with 50 notes the mobile application was the killer feature for me, to which at the moment is still in Test Flight and requires a subscription.

After my migration frustrations with Obsidian I decided to just drop everything and start again, while picking and converting any particularly useful notes over as soon as I need them. This caused a few days of frustration having to switch back to LogSeq every so often, but I got to the point where I didn’t need to open it anymore.

I’ve been working with Supernotes as my primary tool for two weeks now, and here is what i’ve found. I’ll work with the common format of the good, the bad, and the ugly.

NOTE: This list was made 2022-04-29, things may of changed or improved since then.

The Good

Supernotes note card based layout makes it incredibly easy to produce small snippets of useful information, almost directing to a Zettlekasten style of note taking without much active thinking. The notes themselves provide a small guide at the bottom to the number of characters remaining, this isn’t some hard or fast limit, more a Supernotes suggestion of what they think would produce a concise and useful note.

Interlinking notes is easy, with a super useful command palette that is accessed with the / key. From there you can link, add a parent, give the card an icon or colour, and even insert simple templates. The palette is basic but hopefully in the future they’ll introduce a few more advanced features (more of that in the Bad).

By default Supernotes includes two views, ‘Daily’ and ‘Thoughts’. Daily will show you all notes produced on a set day, the top has a small calendar that allows you to browse back and the dates are colour coded indicating the number of notes made on that day. Thoughts is the place for lost notes, ones that don’t have content, a tag, a parent or other incomplete information. If you throw a quick note into the application you don’t have to worry about where it is stored, usually Thoughts will give you quick access to those snippets that need to be organised.

The outline on the left panel allows for quick access to any notes flagged as Priority, each note has a visibility; Priority shows the note on the outline, Visible makes it appear in search results and as child notes, and Invisible does exactly what it says and is only visible to very specific searches for it. Using the visibility you can build a quick reference structure in the outline to allow quick access to groups of notes.

If you wish to export your notes you have a selection of formats, from basic Markdown, PDF, and also a simple JSON format that includes all the metadata you have with your notes. Outstanding, especially when compared to some other services out there.

The Bad

Going back to the outline and those visibility settings. Visibility is a great tool to build a note structure on your outline, but Supernotes also has a ‘Note board’ view, where it’ll show that note and all its child notes in a quick reference view. Say for example you had all your notes for to Ansible as children under a ‘Ansible’ note, the Ansible note itself is set as visible as its also a child of ‘Technology’ which is visible in the outline. By entering note board view on the Ansible note it’ll adjust that notes visibility from Visible to Priority, making it appear on the outline.

Notes can be pinned to a right side panel, making them always visible, but the panel itself is width restricted. In fact, the app itself shows a lot of whitespace. While I understand they’re styling the notes as note cards, it would be nice to allow true fluid width on them as an option.

The screenshot at the top of this post doesn’t really show as a good example due to my Macbook’s screen, but on my 4k screen running at 125% the amount of whitespace real estate on full screen is crazy. I think this could be improved, maybe with a few options to allow some notes to be fully expanded to show the full note content as well rather than the ‘See more’ text.

Supernotes includes an export to PDF feature, select a note then you can export the result as a PDF for easy reference outside of Supernotes, the problem is that it only exports only direct child notes, not the grandchildren. I would like it to export all of them.

The Ugly

A global search is a frustrating omission, the search bar only searches the current context so if you’re in ‘Daily’ it’ll only show notes that would appear there. To search all your (visible) notes you need to click the ‘Home’ tab first, then use the search bar.

Lastly, filters, while they are useful they usually end up causing confusion. At the time of writing i’m hitting 240 notes in my system and i’ve yet to find where they are useful. If you accidentally hit on a filter for one reason or another it’ll affect all views until its disabled, only a small coloured dot on the filter button is your clue that they’re enabled. While I don’t think anything is massively wrong with the idea, I feel the implementation is more a hindrance than a help.

The Future

Supernotes has only just hit version 2.0, and the developers are very active in the community forums. Their last patch to the application had nearly 30 tweaks and fixes, so it shows they really care about this product. For now i’ll be sticking to Supernotes as my primary note taking tool, and i’ll write a further review maybe at a year into it.

I’d be interested in hearing about other Supernotes user’s workflows, what works for you and what doesn’t, so consider sending me a message on Mastodon.

Now approaching its second year I think its time for a little bit of an update on whats going on. In the last year we’ve made some changes:

• Updated from CentOS 8 to CentOS Stream
• Upped the resources from 1 CPU / 1GB to 2 CPU / 2GB
• Reworked the website into a Hugo site
• Setup true mirrors of the HTTP website in Gemini and Gopher (including all the functions!)

In terms of uptime, we’ve had no unexpected downtime outside of scheduled updates and patches. Thankfully, Digital Ocean provides a very stable platform to run a pubnix.

I’ve got a few aims for our second year:

1. Investigate a few extra services - NNTP with NNCP, Matrix, or XMPP.
2. Extend out /home with a new disk.
3. Update the wiki with more details.
4. Grow the community.

As always we’re always taking feedback from members for any improvements. Thank you to the community that has grown around dimension, without you it wouldn’t be worth running.

• 20 Hue devices
• 9 IKEA Tradfri devices.
• 3 Generic Zigbee sensors.
• 6 ESPHome Smart Plugs.
• 1 ESPHome “busylight”.
• 2 ESPHome hacked IKEA air quality sensors.
• 3 Apple Home Pods (recently ousting some Echo devices).

My home is quite automated. While people may think its a “Internet of Shit” for bored techies, it actually keeps us from doing dumb stuff like leaving the 3KW heater in the living room on all night. This simple automation actually pushed me to investigate what other power savings could be had using Home Assistant and a few extra items.

In August 2021 the Home Assistant team added the Energy Dashboard, a tool to give a single dashboard view of energy consumption and generation. The dashboard is primarily designed for those people who have home generation setups, such as a PV installation, but it is useful for tracking energy usage of the average home. After my experience with taming the living room heater I decided to try and get as many items in the house into this dashboard, and then cry at the true energy usage of the household.

The update and Energy dashboard makes use of sensors to track power usage, the key one is the energy device class, and the unit of kwh. If your device or integration produces a sensor with those two attributes then it will be available on the energy dashboard. Unfortunately, very few devices provide this and some adjustments and additional tweaks to Home Assistant are required to make them visible.

ESPHome Devices

I currently use two types of smart plugs in the home, but both types are ESP8266 based Sonoff clones. They include a hlw8012 chipset that allows for power sensing and provide amps, watts, and voltage to ESPHome without much configuration. The ESPHome framework has a software sensor called “Total Daily Energy” that can use these values and output a sum total kWh sensor that Home Assistant can use for the dashboard.

sensor:
  ...
  - platform: total_daily_energy
    name: "${friendly_name} kWh"
    power_id: power_consumption_watt
    filters:
      - multiply: 0.001
    unit_of_measurement: kWh

In this case, power_consumption_watt is a common name I use for a sensor that outputs the current usage in watts. For the plugs it uses the hlw8012, but for other devices that don’t have power sensing I make use of a static sensor:

interval:
  - interval: 1min
    then:
      - sensor.template.publish:
          id: power_consumption_watt
          state: $power_consumption

sensor:
  - platform: template
    name: "${friendly_name} Power Usage"
    id: power_consumption_watt
    device_class: power
    state_class: measurement
    unit_of_measurement: W
    accuracy_decimals: 4
    filters:
      - heartbeat: 60s

Every minute the interval timer publishes the value defined as power_consumption to the ID power consumption_watt, and that ID is defined as a Template sensor providing watts. The value of power_consumption is defined in the per device configuration as a substitution and is based on monitoring the device with a external power monitor (or another smart plug).

substitutions:
  devicename: bedroom_air_quality
  friendly_name: Bedroom Air Quality
  device_description: Bedroom AQ Monitor
  power_consumption: "0.5696"

IPMI Servers

For my sins I run a two cluster vSphere homelab and a TrueNAS storage server, this small stack likes to guzzle power and getting it visible within Home Assistant was a small challenge. I use Prometheus as my metrics tracking and I currently harvest IPMI sensor information into the database already, as part of the sensors it includes a current watt usage of the system. The issue I had is that Home Assistant doesn’t have any in-built component to pull this data from Prometheus.

Thankfully, the community provided one. A user on Github, lfasci, created homeassistant-prometheus-query which gives the ability to run simple PromQL against your Prometheus instance and import it into Home Assistant. The component has a few issues and is very much developed for the “happy path”, but after some trial and error I was able to create a sensor in Home Assistant with the following:

- platform: prometheus_query
  name: Anshar Power Usage
  unique_id: anshar_power_usage
  prometheus_url: http://prometheus-server.monitoring
  prometheus_query: ipmi_sensor_value{name="pwr_consumption",server="anshar-idrac.int.doofnet.uk"}
  unit_of_measurement: W
  state_class: measurement
  device_class: power

- platform: integration
  source: sensor.anshar_power_usage
  name: Anshar Power Usage kWh
  unit_prefix: k
  unit_time: h
  round: 2

The second part of the puzzle is the “integration” platform. This will take the tracking Watt value and covert it to kWh. This works by calculating the total value when source sensor changes, so if your sensor is very stable and doesn’t change too much you could have very spikey results from the platform.

One issue to point out, if your experimenting with your sensor query and it returns null or invalid data, you may have issues using the integration plugin later. Once the data is stored in the recorder integration it’ll base its data off what is available in there, so if for example your unit of measurement is wrong then the resulting sensor will also be wrong.

Hue & IKEA Tradfri

At the time of writing (2022-01-05), the configuration I have (using zigbee2mqtt) doesn’t support providing any sort of power usage stats. I don’t expect this to change in anyway in the near future, but it may be possible to add fake sensors, much like the hacked air quality sensors, to Home Assistant to produce the required kWh output. You could potentially measure a bulb’s wattage using an external tool, then publishing watt values depending if the bulb is on or off in Home Assistant. This feels like it would be best done by a Z2M Extension, or even something that uses MQTT to pull the state and publish the wattage values.

For the moment, that gives some per device statistics and integrates it nicely into the Home Assistant energy panel.

My next project is to look into monitoring the meter itself to get a total household usage into Home Assistant, but that will be another post.

I copied my configs over to the new cluster, and Home Assistant got in a bit of a tizz. Not matter what I did I couldn’t login to the UI. All that was coming up was the prompt of “Login aborted: Your computer is not allowed”

My Home Assistant config was correct, as far as I knew:

homeassistant:
  ...
  auth_providers:
    - type: trusted_networks
      trusted_networks:
        - 10.101.0.0/16

http:
  server_host: 0.0.0.0
  ip_ban_enabled: false
  login_attempts_threshold: 100
  use_x_forwarded_for: true
  trusted_proxies:
    - 10.85.0.0/16
    - 10.101.10.0/24

The nodes are on 10.101.10.0/24, the Pod network is 10.85.0.0/16, and the clients are in 10.101.0.0/16. !?!?

Using the good ol’ inspector within Safari I was able to pull out the response from the request, login_flow returned:

{
    "type": "abort",
    "flow_id": "4c8575ddc18b4d5b83565c08420d3093",
    "handler": [
        "trusted_networks",
        null
    ],
    "reason": "not_allowed",
    "description_placeholders": null
}

Not a massively useful error. Digging into the code shows that its catching an exception and throwing an error. I adjusted the code in the container to throw a bit more details of what is going on, by passing through the exception itself.

        except InvalidAuthError as exc:                                                                                      
            return self.async_abort(reason=str(exc))    

Now, it was complaining of “Can’t allow access from a proxy server”, huh? X-Forwarded-For is switched on in the Home Assistant, but the client IP address was showing as one of the node’s IPs, not the end user IP. Then it suddenly dawned on me.

On my previous cluster, Traefik had been the edge taking connections directly from client systems, but now it was getting requests from HAproxy that knew what the client IPs are. Traefik by default will not trust certain headers it receives from the client and will re-write them to what it understands is the correct value.

The fix? Allowing Traefik to trust the headers. This can be done with the following additions onto the Helm chart:

    additionalArguments:
      - "--entryPoints.web.proxyProtocol.insecure"
      - "--entryPoints.web.forwardedHeaders.insecure"

Further information can be found on the Traefik documentation site

Like all good software tools, over time everything develops competitors. Roam gave birth to multiple tools in its wake, and many other tools gave birth to Roam. All have been missing some sort of killer feature for me, that was until last week.

Obsidian has been on my radar for some time, primarily desktop based and more of a general note taking tool than outliner, I had always played a little with it and then moved back to LogSeq. Obsidian’s big selling point is its modularity and a wealth of community plugins that bring new and exciting features into a normal note taking tool. Since my workflow mostly consisted around daily journals and outlining I was happy to see that Obsidian finally had plugins to support it.

Last week, Obsidian announced the final missing piece of the jigsaw: a mobile application. Combined with their paid-for sync service you can now have access to your notes on the go! I think its finally time for me to move away from LogSeq.

The problem is, i’ve got nearly a year of notes in my LogSeq; 3090 commits to the repo, hundreds of journals, hundreds of pages. Thankfully, a recent LogSeq update allowed the conversion from their “LogSeq Markdown” to a more standard one, but Obsidian doesn’t support Frontmatter and it’ll require some sort of manipulation to convert between the two systems easily.

So today, i’ve decided that i’ll work on a set of tooling to allow for easy migration, quick i’ve dubbed as “ObsSeq”. The repository is created and the license is set to MIT, and hopefully over the next few weeks I can provide some decent tooling to assist with the migration.

ObsSeq

When I was younger, much, much younger, my prized possession was a Sega Game Gear. At the time it was the pinnacle of gaming technology; a colour backlit screen, a ‘powerful’ processor equal to the Master System. The device went along everywhere with me and I chewed through batteries like there was no tomorrow.

One day, I loaned it to a friend of mine, as he wanted to play through Crystal Warriors, and I never saw it again. Turns out that his father had got addicted to Columns and decided to take the Game Gear on his work trips with him. After a while I heard that it was mangled in accident, but I never got to see the remains.

In the last few years i’ve been collecting Gameboy consoles, mostly due to my love of Pokemon, and last week I came upon a extremely cheap Game Gear on eBay, just £30. Game Gears have a terrible reputation for quality, the capacitors don’t survive well over the years, and its well known that if you find a working Game Gear without modifications, then you’ve hit gold. I took my chances and purchased it, with no idea what I was getting into.

Game Gear Capacitor Replacement

First of all, they listed the item as powers on but the screen is “broken”, again, this could be due to many reasons and not something to write off. My first concern was when I opened the box the first thing I saw was Japanese newspaper, and this device had been bought from within the UK. Maybe, they bought it from Japan then released that its either beyond repair, or their expertise, or just too expensive to get fixed.

The outside looks OK, its had some heavy use but nothing too drastic. Then I opened the battery compartments:

Oh no, that doesn’t look good. Thankfully this was the worst damage in the entire device, it seems that it had been left with some rotting batteries for quite some time and they had rusted away the battery connectors. The Game Gear requires six AA batteries to be ran in series to get the 9v required by the console, so these pads are not the main connectors to the power board, phew.

The power board didn’t look bad at all, it has some signs of water marks but no obvious damage. I don’t intend on keeping this board, rather replacing it with a CleanPower GG which is a drop in replacement for the board and has a USB-C port for easy connection up to a power bank, rather than lugging a 9v wall wart around or six AA batteries and spares. I ordered mine from RetroSix which is a UK retro console parts supplier in the UK, the delivery took 2 days which is great for a smaller company.

After I received the new power board I wasted no time getting it in the device, the Game Gear’s boards are all connected via simple connectors so it was a quick swap out. Then, it came time to power the device on for the first time.

I flicked the switch, and the backlight switched on, nothing on the screen. I noticed the power LED hadn’t lit up at all, so after a little bit of poking it sprung into life, the screen was on and the LED, then as quickly as it came on it died again. Capacitors, must be.

So that is where i’m up to at the moment, i’m currently deciding on spending £50 on a repair service to replace the caps, or £90 on a good soldering iron and to try it myself.

As of this post, I’ve added a call into my site build script. Hopefully, this service will present a more active view of the Gemspace community and allow some sites that have slipped through the nets of CAPCOM to get noticed.

I’m a newbie to Go development, but the code for Kiln is simple and easy to understand. I encountered a strange bug on dimension.sh that would cause Kiln to SEGV on a build due to path walking, i’ve still not been able to identify why but I was at least able to identify where the error occurred and how to mitigate it for the meanwhile.

Since i’ve been digging deep into the tool and how to configure it, i’ve been keeping notes of anything useful i’ve discovered. The documentation for Kiln is sparse at the moment but enough to get people working, hopefully over time it’ll improve. I still have to navigate SourceHut’s submission by mailing list patching, something i’ve not done for the best part of 15 years.

Kiln Notes

I started using Kubernetes about a year ago when I needed to learn the platform for some upcoming projects in my day job. As a good test ground, I moved the vast majority of my home workloads over to a small one master and three worker cluster, excluding the apparent issues of persistent storage and ingress. It was relatively easy and is now in a state where it’s stable to use for more critical items. To ease configuration, I make use of Flux v2 to manage Helm deployments and other objects using a single Github repository.

The Hardware

The node I’m going to be using to host the adapter is a relatively low-powered system, a Optiplex 160, its an SFF (small form factor) Atom-based PC. One problem is that the system itself isn’t powerful enough to run Zigbee2MQTT. Thankfully, Z2M supports remote devices over TCP sockets, so I’ll be able to keep a small pod to act as a gateway to the device on the hardware and use the fast VM nodes to run Z2M itself.

First of all, we needed a consistent device name for the adapter. This step isn’t critical, as you could use the full device path, but by giving it a consistent name (as defined by udev) it makes the configuration a little more user-friendly. On the node I created /etc/udev/rules.d/zigbee-controller.rules with the following:

# CC2531
KERNEL=="ttyACM*", ATTRS{idVendor}=="0451", ATTRS{idProduct}=="16a8", MODE="0666", SYMLINK+="zigbee1"
# Zig-a-zig-ah
KERNEL=="ttyUSB*", ATTRS{idVendor}=="1a86", ATTRS{idProduct}=="7523", MODE="0666", SYMLINK+="zigbee2"

I have two controllers of a different type, so I created rules to export them as zigbee1 and zigbee2. A quick reboot and these devices now appear as their correct names:

% ls -1 /dev/zigbee*
/dev/zigbee1
/dev/zigbee2

I am going to be making use of the ser2sock Helm chart from K8s-at-home, this deploys a small application to proxy the serial port to a TCP socket, which Zigbee2MQTT can use. But before I could deploy it, I needed to make sure that the pod can find the hardware and be assigned to the same node every time.

By default with Kubernetes, your application will be scheduled on any worker node that is operational and not cordoned off. This presents a problem when dealing with physical hardware, it is only available on a select node, and you need to ensure that the application is ran on that particular node. Kubernetes supports the concept of labelling, where a node can be tagged with a text string, and a Pod definition can require a set of labels that it expects for the node it’ll be running on. Using kubectl its a quick change to the node:

% kubectl label node k8s-node04 doofnet.uk/device=zigbee-controller
% kubectl get node k8s-node04 --show-labels
NAME         STATUS                     ROLES    AGE   VERSION   LABELS
k8s-node04   Ready,SchedulingDisabled   <none>   40d   v1.20.4   doofnet.uk/device=zigbee-controller

So now I have the elements I need to write a Helm release. The device will be linked as /dev/zigbee2 and available on a node labelled with doofnet.uk/device=zigbee-controller. So I create a HelmRelease object to deploy ser2sock:

---
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: zigbee2
  namespace: ha
spec:
  interval: 5m
  chart:
    spec:
      chart: ser2sock
      version: '2.0.3'
      sourceRef:
        kind: HelmRepository
        name: k8s-at-home
        namespace: flux-system
      interval: 1m
  values:
    device: /dev/zigbee2
    tolerations:
      - effect: NoSchedule
        operator: Exists
      - key: CriticalAddonsOnly
        operator: Exists
      - effect: NoExecute
        operator: Exists
    affinity:
      nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
          - matchExpressions:
            - key: doofnet.uk/device
              operator: In
              values:
              - zigbee-controller

Another option I’ve put in the release is the list of tolerations. My physical node is very underpowered and I don’t want any other workloads on it, so I marked the node as ScheduleDisable. By adding the tolerations I can have this pod ignore the restrictions and still run on the node.

Once pushed to Flux and deployed by Helm you’ll see a new pod appear called zigbee2-ser2sock, and if you check the logs you’ll be able to see if it found the hardware and started correctly:

ser2sock Serial 2 Socket Relay version V1.5.5 starting
ser2sock Listening socket created on port 10000
ser2sock Start wait loop using ser2sock communication mode
ser2sock Opened com port at /dev/ttyUSB0
ser2sock Setting speed 115200
ser2sock Set speed successful

The pod definition, created by the Helm chart, maps the device name you provided (/dev/zigbee2) to /dev/ttyUSB0 for ease of configuration. Initially this confused me and its always worth checking the Chart definitions to ensure what value is being used where.

Zigbee2MQTT

Now that the adapter is available via TCP using ser2sock, I can deploy Zigbee2MQTT. K8s-at-home also has a chart ready to deploy the application, so I make use of it:

---
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: zigbee2mqtt-zigbee2
  namespace: ha
spec:
  interval: 5m
  chart:
    spec:
      chart: zigbee2mqtt
      version: "6.2.1"
      sourceRef:
        kind: HelmRepository
        name: k8s-at-home
        namespace: flux-system
      interval: 1m
  values:
    image:
      tag: 1.18.2
    config:
      homeassistant: true
      permit_join: true
      mqtt:
        base_topic: zigbee2
        server: "mqtt://mosquitto.monitoring"
      serial:
        port: "tcp://zigbee2-ser2sock:10000"
      frontend:
        port: 8080
    persistence:
      data:
        enabled: true
        accessMode: ReadWriteOnce
        size: 1Gi
    ingress:
      enabled: true
      hosts:
        - host: zigbee2-dashboard.apps.doofnet.uk
          paths:
            - path: /
              pathType: Prefix

Under the values section, you can see a key of config; any subkey of this value used to create the Zigbee2MQTT configuration YAML file in a special location on the pod, so on the first boot it’ll use these values as the defaults. I defined URL for our ser2sock instance, and also details of our MQTT server and a base topic to use.

Once done, push it to Flux, wait for it to deploy, and check the pod named zigbee2mqtt-zigbee2:

zigbee2mqtt-zigbee2 Using '/data' as data directory
zigbee2mqtt-zigbee2 > zigbee2mqtt@1.18.2 start /app
zigbee2mqtt-zigbee2 > node index.js
zigbee2mqtt-zigbee2 Logging to console only'
zigbee2mqtt-zigbee2 Starting Zigbee2MQTT version 1.18.2 (commit #abd8a09)
zigbee2mqtt-zigbee2 Starting zigbee-herdsman (0.13.88)
zigbee2mqtt-zigbee2 zigbee-herdsman started
zigbee2mqtt-zigbee2 Coordinator firmware version: '{"meta":{"maintrel":1,"majorrel":2,"minorrel":7,"product":1,"revision":20210120,"transportrev":2},"type":"zStack3x0"}'

And I can check the dashboard ingress endpoint I defined in the configuration:

Success! We have a running Zigbee2MQTT in Kubernetes. This solution was built off the back of hard work done by the Zigbee2MQTT team, K8s-at-home, and a few others, but with relatively little work, you can have a hardware-dependent application running quite quickly within Kubernetes.

The result of a standard weekly AWX run on a system was an error. Failed to manage policy for boolean ...: [Errno 0] Error, not exactly the most helpful error to be spat out by Ansible, but it’s all I had. I had recently completed patching on the VM, so the occurrence of the issue can be attributed to an updated package.

The Issue

The Ansible error is a generic catch-all error within the SEBoolean module. The Python in the module tries to replicate what the management commands are doing under the hood, so while the code is complicated, it should be easily replicated with the CLI commands. Running to set the boolean worked:

# setsebool httpd_enable_homedirs=on
#

But, running to apply the boolean permanently failed.

# setsebool -P httpd_enable_homedirs=on
libsepol.context_from_record: type systemd_sleep_exec_t is not defined
libsepol.context_from_record: could not create context structure
libsepol.context_from_string: could not create context structure
libsepol.sepol_context_to_sid: could not convert system_u:object_r:systemd_sleep_exec_t:s0 to sid
invalid context system_u:object_r:systemd_sleep_exec_t:s0
#

The underlying error is that type systemd_sleep_exec_t isn’t defined in the SELinux policy, a fundamental part in my opinion, which points to something wrong with the policy files. Searching on RedHat’s and CentOS bug trackers directs me to a few other people experiencing similar.

The Resolution

In previous instances, I had an issue where multiple versions of the SELinux policy files were installed, and removing the old instance and re-installing the current version resolved the issue. In this instance, I only had one installed version, so I re-installed:

# dnf re-install "selinux-policy*"

...

Reinstalling:
 selinux-policy             noarch  3.14.3-54.el8_3.2 baseos 622 k
 selinux-policy-targeted    noarch  3.14.3-54.el8_3.2 baseos 15 M

...

Reinstalled:
  selinux-policy-3.14.3-54.el8_3.2.noarch
  selinux-policy-targeted-3.14.3-54.el8_3.2.noarch

Complete!

And now it works as expected.

# setsebool -P httpd_enable_homedirs=on
#

I use my M1 in ‘clamshell’ mode when at my desk, so I had to look around for a webcam. I usually wouldn’t bother, but with the current situation, all of our meetings are remote. Having a webcam available makes the meeting flow a little easier, especially when dealing with new people, which was precisely my situation due to starting a new job. When shopping, options were quite limited online, and the cheap camera I bought in the past had tripled in price over the last couple of months. I decided to suck up the cost and pay for a real webcam, something with decent features and image quality, and in the end, I picked up the Logitech StreamCam for an eye-watering £129.

The camera itself is jam-packed full of features; 1080p 60fps, auto-focus and exposure, USB C, and even a 1/8" screw mounting base to place it on a tripod if needed. As it turned out I picked the wrong camera for my set of devices, and I am dogged with some compatibility problems:

• No Apple Silicon support in the tools to access the advanced features, and the x86 tools just crash under Rosetta.
• Incompatibilities with the StreamCam’s 5gbps/10gbps support with the Caldigit TS3+
• Very hit and miss 60fps support.

M1 support, I couldn’t expect Logitech to have that ready just a month or two after the first actual hardware launch, after all even bigger companies are still trying to support it.

The CalDigit incompatibility was a head-scratcher. I emailed CalDigit’s support, and within 24 hours I had back an extremely detailed response, to which I deleted and can’t share here, but Scott Macpherson received a similar reply. I’ve yet to approach Logitech support, but I expect the result won’t be any real resolution as it looks more and more like a hardware issue. For the moment, the StreamCam works perfectly in one of the Thunderbolt ports, the front USB-C port, or with a C to A adapter in most of the other ports. The only port set that has issues is the 10gbps group on the back of the TS3+.

As for the final item, I don’t have any strict proof, but I suspect this is related to the lack of application support on M1 chips. I can’t get any application to use the promised 60fps mode on the camera, maybe it needs the applications to enable it, but I’ll have to wait and see if Logitech updates their application. Initially, I thought it was due to using a C to A adapter in a USB 3.1 port and that the bandwidth wasn’t wide enough for 60fps (USB 2.0 is 480mbps something something, but I digress), but now its plugged into a Thunderbolt port I don’t see any marked improvement.

In summary, this was a quick warning about using a Logitech StreamCam with a CalDigit TS3+ or an M1 Mac. I’d stay clear if you hit either of these situations at the moment. Maybe a hardware revision or a firmware update could resolve it shortly, but I’m not holding my breath.

[Update 2021/03/17]

I spotted a comment from Swarm2099 on Reddit, version 2.0.200 of Logitech Capture, designed from macOS 10.14, works on M1 Macs. The version is massively out of date, but at least its something.

[Update 2021/05/05]

After a short and grumpy tweet on my account, Logitech replied stating that an M1 version of Logitech Capture is in the works, but with no release date defined.

“Buy it for Life” is the idea that you purchase practical, durable, and quality-made products that could potentially last a lifetime. You can’t ‘buy for life’ with electronics, but a quality item will readily give you ten years of life. I justified my expensive mouse purchase in terms of value for money. My Explorer had lasted ten years, and if the MX Master lasted another ten then that’ll spread the cost to around £9/year.

Keyboards have always been this practical item, a tool to be used, abused, and disposed of when broke. From a £10 PS2 keyboard I got with a new PC, to a Dell Slim 104-key USB keyboard I picked up from eBay as it was the same as the keyboard I was using at work, I never really took any time to think about my primary input device.

Mechanical keyboards have taken off in a big way in the last ten years, driven by what I think is people’s desire for something better. I was exposed quite early, around 1998, to an IBM M mechanical keyboard that was hidden away with an IBM PS2 in a family cupboard. At the time I thought it was far too loud to use daily, and I stuck with my terrible domed keyboard. Rolling on ten years later and what is old is new again, people are demanding more from their keyboard, the rise in popularity of the ‘Das Keyboard’ showed that people still wanted old clicky mechanism.

I didn’t investigate mechanical keyboards until 2016 when popularity was surging, and many pre-built keyboards were available for the average user. Even then, the choice of switches and price drove me away from the idea. Then in early 2020, when COVID-19 pushed most into the home office, I came to the realization I didn’t have a home office desk that I could call my own; the equipment I needed just wasn’t there. Like everyone else, I quickly ordered bits online; monitor, desk, headsets, my company provided the bulk of what I needed, but I was still missing a few key elements. When browsing for a keyboard on Amazon I was dissatisfied with the choices, and I couldn’t find my favorite Dell keyboard on eBay. Only then did I think back to 2015 and my selection of mice; the MX Master is still chugging on without issues, a bit worn, but still working. I thought it’d be nice if I had a keyboard like that.

My initial mech keyboard of choice was an Akko 3068, a 68% keyboard that covered the base keys in a small form-factor, pre-built, and available from China on a reasonable timescale. It was my main workhorse for most of the lockdown, bar the US layout, the loud Cherry MX Brown switches, and the echoing internals, it served me well. Six months later, I had already decided I needed an upgrade. Around that time, Caps Unlocked was running a group buy (essentially, a pre-order) for their new CU65 keyboard. It has a milled aluminum case, with a hot-swappable PCB so you can change your switches without getting the soldering iron out, and a USB-C interface. Perfect for my new home working desk setup.

March 2021, and I’ve finally received my CU65. Like a child with a new toy, I had to get it open, built, and working. It took only 3-4 hours, which I think is good for my first ever build, and I’m seriously impressed with the results.

• Keyboard and switches: £159
• Stabilizers: £14.95
• Key Caps: £39.99

In total, it came to £213.94, which is an eye-watering amount for a keyboard. A well-constructed and repairable keyboard could potentially last ten years. Mechanical switches are rated for upwards of 50 million key presses and cost in the range of 50p to £1 each, so replacing broken switches won’t blow the budget.

Let’s get back on topic; why would an expensive keyboard be a productivity tool?

First big point is comfort. Custom keyboards come in all shapes and sizes, from full-size to 65%, to Dactyl, so depending on your usage style, you can find a keyboard that suits you. You have to remember that your keyboard is your primary interface to your computer, most modern knowledge work is done in front of a computer, so much like investing in a good chair, a good keyboard is a must.

Most custom keyboards are configurable to some degree. QMK is a custom firmware that allows you to configure and remap keys exactly how you want. It allows you to select the features that you want rather than the choices being made by someone who will never meet you. For example, as a developer, I use certain keys and features on the keyboard; Page Up, Page Down, Insert, and Delete are required within easy reach. While they do come on standard on my 65% keyboard, I could easily remap them to a new location. You can modify the keyboard layout to suit you and keep you productive.

Finally, with a longer life-span it means you will not be replacing this keyboard anytime soon, so familiarity will help a lot. People get used to how they use a keyboard, and it was why I previously bought one of those Dell Slim keyboards as I was using it daily at work. Once you are familiar with your new device and layout, you’ll find that your typing speed will increase, shaving time off any tasks. Ali Abdaal did a video purely on his mechanical keyboard, and how he can type fast, so I’m not alone in thinking this.

Anyway, I’m sure I’ve written this post only to justify my purchase in some way. I’ll be able to give a longer review of my CU65 in the near future. *clack clack clack*

Finding the right avenue to post my work has always been a struggle for me. In the past, I took to creating new blogs (much like this one) to express a facet of my work and interests, but as I get older I feel that my interests are merging and the lines of separation are no longer there. For example, technology and my Getting Things Done journey have collided in the past ten years, with my productivity system being a creation from the technical aspects of my life and work.

I have to grab hold of the situation and change something, as my diluted content is slowly slipping in quality and frequency. After a short process of mapping out my choices I’ve cut down my outputs to two; This blog for productivity-related output, and another (at a to be decided domain) for my combined technical writing.

I also need to take control of my time I set aside to write. My writing in the previous six months could be described as ‘scattered’ at best. I’d work for an hour or so when I could to create a post but only if I have an idea of what to write. My new attempt to resolve this problem is to schedule time each week to write, even if it is a few hundred words and half-cooked. A few hours on Sunday morning from now on is going to be dedicated to writing.

Location, time, all that is missing now is what to write about.

I spent every morning reviewing my RSS feeds, and I save anything I find interesting into Instapaper, I also use Readwise to consolidate my highlights into a single view from all my media. Finding an interesting snippet or a post that I bring up in conversation is quite easy with this system, but I don’t produce any output from gathering all this data. I’m going to try something for a week or two to see if it helps:

What did you read that was interesting?

I’ll add this to my Daily Tasks on my Journal. Hopefully when I reach Sunday i’ll have a short list of interesting things I could write about, which could reduce the mental strain of trying to think of a topic.

With my first block of time only three days from now, lets see how we go.

Readwise aims to solve this problem, by giving a single location for all your highlights and snippets to coalesce into a single, reviewable view. Their exporting tools allow for quickly moving your important highlights from Readwise to other PKM, such as Notion, and Roam.

Most people look at the price and wonder if it is really worth the $7.99/month price tag. Initially, when I first encountered Readwise it only had a basic Roam export and the price felt high for what was essentially a fancy Kindle sync. After reviewing the feature set now, the team added more and more features that I can start to see the benefit of the tool.

At the time of writing (2021-02-04), Readwise supports the following sources:

• Kindle
• Instapaper
• Hypothesis
• Goodreads
• Medium
• Airr
• Feedly
• Pocket
• Twitter
• Apple Books
• Their own web highlighter
• Manual imports from Email, CSV, free form text, photos, * PDFs
• Scribd
• O’Reilly Learning
• Google Play Books
• Command

They’re very quickly approaching IFTTT levels of integration for highlights! While I only use 5 of these sources I can’t wait to see what else the team introduces shortly.

Readwise posted on their blog “Why we’re bootstrapping Readwise” which breaks down their decision. In short, they don’t feel the product they want to build will appeal to the VCs, so they’ve decided to let paying customers drive their future progress. Turning away free users will have an impact, but having a small and loyal customer base will push them in the right direction for the market they want to target.

It comes down to a few questions, Would you prefer to pay a premium for a product that fulfills a niche? If so, how much of a premium would you consider paying? By not allowing their product to be driven by a VC, Readwise is concentrating on their core following who helped bootstrap the idea. Too many companies get sucked into the “VC thinking” of mass-market appeal, slowly transforming into something that doesn’t represent the original goals they had.

Is it a good tool? Yes. Is it for you? Maybe, maybe not. Is it worth the price? Totally.

The big selling point of the new Macbook Air is the M1 chipset; a ARM-based SoC (named “Apple Silicon”) that has been derived from Apple’s work on the A series processes used in iPhones and iPad for quite a few years. With switching to ARM, Apple has been able to reduce heat and increase battery life to way beyond what is expected in a productive laptop, while maintaining the processing speed of a good mid-range Intel system.

So, lets talk about performance. I can safely say that the M1 outperforms the Surface Pro 6 by a large margin, and while I expect people to note that the Surface Pro 6 is over 2 years old I can’t really point out how much of a night and day difference it is. My workload is mostly based around Visual Studio Code and other simple development applications, on the SP6 it would take about 3-4 seconds to open my VS Code setup, on the M1 its under a second. Safari opens near-instantly, I can run several Electron apps with no visible slowdown. The seconds here and there saved allow for more frictionless workflow i’ve ever experienced with my SP6. From what i’ve heard a similar jump can be felt even with the ARM-based Surface Pro X and the Surface Pro 7.

With all major architecture changes you do hit some snags, biggest is the lack of support for “Apple Silicon” on some applications, which require using Rosetta 2 with Intel-based binaries. When you initially load a Intel binary on a M1 you can expect a wait of 5-10 seconds depending on the size of the application, but after that its back to near instant. The transcoded Intel applications run slower than native, and its most obvious on CPU chewing applications like video rendering and gaming, but for day to day apps like Office they feel the same.

Adjustment to my workflow has been a bit of a challenge, as it turns out I have switched to a heavily Microsoft Edge based workflow using Memex and other tools, having to rediscover tools for MacOS that work with Safari has been a challenge. While I could switch to using Edge on MacOS or Chrome, I ideally want to keep in sync with my iOS based devices now that I can take advantage of the extra features available to me. I’m no stranger to MacOS, as I used a 2013 MacBook Pro before my Surface Pro 6, but its something to consider if you want to switch to using a Mac full time.

As a relatively low power user, even for my job role, I find the M1 a amazing workhorse for the price, and I’m extremely happy to have this as my daily driver. I’m hoping that I can make use of this device for a good 5-6 years before I have t consider upgrading again, much like my old MacBook Pro. Apple are pushing hard into ARM/AS and new devices are on the horizon. If you’re considering upgrading then I’d wait until the next generation of devices to come out, hopefully with a even more crazy jump of power that the M1s had.

In comes LogSeq. LogSeq is an in-development, local-first, non-liner outliner notebook, much in the same thread as Roam Research, Obsidian, and several other tools that are available. The key difference is that LogSeq will (eventually) be open source. LogSeq is a web application first, and it runs reasonably well on desktop, iPad, and iPhone allowing for access in any location.

So, why did I chose LogSeq for my Second Brain?

As it turned out the fixed formal layout of Notion stifled how I like to browse information, while Notion does now support backlinks, I was essentially looking at a list of notes much like how I had them formatted in Evernote. For years the key tool keeping Evernote useable for me was the fully-featured search, deep inspection into PDFs and other file formats allowed for you to quickly pick the correct note for the job, Notion lacks this feature. LogSeq also lacks “deep searching” but the application itself dissuades you from just dumping files into it, instead you write concise notes which are empowered by backlinks and “unlinked reference” to discover new relevant information.

Notion and Evernote are fundamentally different tools than LogSeq and their ilk, but as it turns out I was always trying to make a non-liner notebook in a tool not designed for it. Once I started using it I realized what I had been missing and it allowed me to finally take notes in the form that I wanted while keeping them useful and searchable.

Of course, like any new application, it doesn’t come without its issues. LogSeq has a lot of quirks and issues which will be resolved over time, but for the moment they can be quite troubling.

• You can’t have it open on multiple devices at the same time, as its essentially using Git under the hood it is very easy for it to get into a situation where a merge is required to save your data, which it doesn’t support at the moment.
• Copy and paste in some information “crashes” the application, as its all browser-based JavaScript it does have a few issues with pasted data, thankfully a quick refresh resolves the issue.
• While custom CSS is supported for theming, the base CSS for the site is very complicated and has a lot of overriding. I’ve started creating a set of base16 themes for LogSeq but its been quite a challenge.
• Markdown isn’t parsed into the LogSeq format, pasting in a list of bullet points won’t create a list blocks, instead create a single block with a list of bullets within it.

These issues exist as of 2020-10-14, and mat have been resolved in the future.

It is still very early days for LogSeq, you can follow the development on GitHub and the Discord community. If you want a more feature-complete product right now I’d suggest using Roam Research.

Notion sells itself as the “all-in-one workspace tool”, while primarily built for business and small teams it has found a home with individuals trying to manage their piles of information. The idea is simple, you have two types of data; pages and databases, and with them you can build complex structures much like a wiki.

I’ve now been using Notion for a few weeks, and I’ve hit on a few issues which I’d like to see resolved. I understand that these issues could be trivial to other users, but these are the issues that are affecting me personally.

Global Search for Properties

Global free text and content search works well. From anywhere in Notion you can hit the shortcut of Ctrl+P to bring up a panel, much like Visual Studio Code, and search for freeform text in your pages and databases. My P.A.R.A. resources are stored in a database with a multi-select indicating the topic, when you attempt to search for a topic you don’t get any hits, unless that text is in the contents of a page. If Notion allows the text search to also look at Select and Multi-Select I feel that’ll open up the usage of these fields. The workaround for this is to use the search on the database view, which does mean a few extra steps of discovering the database you need, then searching the values in there.

Page Locking

Locking is Notion’s way to stop editing on your pages and database, once the lock is enabled you can’t modify the page layout and properties, but sub pages can be created without issue. This works wonderfully for my Areas and Projects which the fields will very rarely change, but for Resources I’ve hit a problem that reduces its efficiency.

Again, Multi-selects, I use them for tagging topics and when I create a new Resource I throw a few topics on the page to allow for easier searching, but when the database is locked you cannot add new items to the Multi-Select. A suggestion would be to have a mechanism where the lock is granular, say only allow additions to selects on selected fields, rather than locking every element in the database.

Offline Mode

Offline mode has been one of the biggest issues for the Notion community, and I am by no means the only one with an issue on this subject. Honestly, this is one of my biggest Notion regrets. I moved from Evernote which has a fully-featured Offline mode, the access this gave me was amazing and I was able to reference documents and notes on the go in any location from my phone. Its a feature I grew to depend on over time and now I don’t have access to similar it is causing issues for my workflow.

Notion are currently working on an offline mode, which I am extremely pleased about, but I think my requirements differ from the majority of Notion users. Rarely do I need to create anything on the go, and I work around this gap by using Drafts on my phone and watch. So in reality I’m looking for an “offline read-only” mode, which I feel would be a lot simpler for Notion to implement.

Opening Multiple Items

If you wish to look at another page while writing one, you have two options;

• Click the link and take it full page, then switching back.
• Ctrl+Click to open a new tab/instance of Notion and flick between the two running instances.

The second option doesn’t sound to bad, but it starts a full instance of the Notion UI again on your desktop. Ideally what I’d be looking for is how Evernote works, when you Ctrl+Click a note in Evernote it opens a new window showing just the note.

Data Portability

Notion can export your data as a collection of Markdown and CSV files, but once you have that data you’ll quickly realize that the Markdown files are not formatted the way you expect. When using a database to store paged you can assign properties that you can use for key information, for example in my “To Read” database I have properties for the URL on Good Reads, the Author, and dates when I discovered, read, and finished the book. When you export this data out of notion it is pasted into the Markdown as just text.

A very common format for Markdown documents, used by a lot of static website generators, is the YAML Front-matter, this allows you to assign metadata in YAML format to your document, and I think this should be the way that Notion exports its Markdown pages. By using the YAML front-matter it would be painless to take a Notion export and copy it into a Hugo setup to publish the pages outside of the Notion offered public hosting.

Conclusion

These may feel like minor issues for Notion, and most of them could be easily resolved, but they can present some real annoyances to someone’s workflow. Notion is a product designed for one usage but is being used for another, and I can understand that their primary goal is not to service the PKM crowed, but with a few modifications the product could be outstanding.

Everyone knows that sound. My phone, watch, tablet, all making noises to say “HEY LOOK AT ME” and rarely is it anything important. On a normal day, I have about 60-100 notifications on my devices; Each one of these pulls my attention away from my current focus, and while I’ve got better at managing these “micro interruptions” it is still just another interruption.

Over the past couple of years Microsoft, Apple, and Google have all worked on bringing distraction-free functions to their OSes and devices. Windows 10 introduced “Focus Assist” and Apple has “Do Not Disturb” settings, but even them only take away the burden for a period of time and those notifications are still building up behind the scenes.

So I thought; What if I just do away with these notifications, prune down the list of applications to a few essentials that I allow to notify me? I already do with my Apple Watch, which was more out of absolute frustration that I’m getting a wrist notification of an upcoming SpaceX launch from the KSC app. Maybe now it is time to do the same with my iPhone.

Notification Permissions

First off I took my iPhone and mass purged notification privileges from as many apps as I could. The first issue I encountered was seemingly important applications with notifications enabled, but with no details about what would be sent.

• Health - Does that mean any heart rate notifications?
• FaceTime - Does that affect when people call me?
• Steam - I’m guessing I can’t use Steam Guard with it off?

Some applications break down the notification types into groups, Photos, for example, breaks down its notification to “Memories”, “Shared Albums” and “Sharing Suggestions”. It’s useful to have that granularity, but I don’t care about any of the options presented to me. If all applications had this option it’d be very useful to reduce the noise.

After the initial cull I’m left with 41 applications that have some level of notifications enabled.

Sounds

The pesky dings and vibrates, the big interrupter themselves. In almost all cases I don’t want to be interrupted by a notification, so I disabled these on all applications except the critical applications I need to hear from:

• Phone - for obvious reasons
• OpsGenie - for those impending call-outs from work
• Health - if it detects something I need to be notified about, I want to know…
• Telegram - my wife’s preferential communication method

This alone cut down my “micro interruptions” a lot, no more detecting every little vibration through the desk and checking to see what it is, if it’s vibrating it’s most likely important (or my wife sharing a Catana comic with me).

Banners

Banner notifications are the next tier of notifications to be checked over. These are the on-screen prompts that you get, the main interrupters. By default notification pop on the screen, hang around on your lock screen, and also sit in the Notification Centre. iOS has an option to control all three of these elements, so the next questions for each of the 41 applications were:

1. If I’m using my phone, do I need to know about it right now?
2. If not, do I need to see it the second I look at my phone?
3. If not, then do I need to know about it at all?

As it turns out, I don’t need most application’s notifications right now, Notification Centre is fine, even the lock screen is a push for most of them. Wallet and banking notifications? Sure I want to know about them as soon as I look at my phone, but I don’t need to see the contents of my email inbox that second.

Badges

Badges are useful, especially for messaging applications. In most cases it’s all I need to see, for example; my emails are not massively important and don’t need instant attention. If the application has a notification then it’ll appear as a badge on the application icon. After removing banners for most applications it seems to be the perfect balance, I have the majority of my important applications on my home screen in groups I can quickly see that I’ve notifications and decide to look at them on my terms. My home screen can look peppered with red “8"s and other numbers but at least I can check them when I feel like it, rather than have the notifications flashed in front of my eyes.

The Week So Far…

It has been a good 48 hours now since I made the change, and I have to say it has been bliss. My phone now sits screen up on my desk, I don’t have it dragging my eyes every minute or so when the screen lights up. Sounds and vibrations are mostly a thing of the past and reserved only for the most important items. Since implementing the changes on my iPhone I’ve been able to switch back to “Mirror my iPhone” on my Apple Watch without any negative consequences.

Of course, it can’t be all positives, I’m missing my critical emails. When you login Netflix and other services online you’ll get an email notification saying that a new device as logged in, with my lockdown of all notifications from Outlook I’m also missing these notifications. It’d be useful if Outlook had the option to mute all notifications except for one folder in your account, while the option exists for “Focused Inbox” it doesn’t for any other folder.

I expect I’ll be writing a follow-up article in a month or so to discuss further issues I’ve found, or just to rave about how much it has changed my life. One thing that stands out is that my battery usually lasts just about the full day, and at the moment at 6 pm and at 72%… wow.

Touch Interface

Windows’ touch interface is amazingly immature and you come across interface issues on numerous apps where it is obvious the developer never thought of a touch interface. Add onto it that application developers never seem to think about the touch interface as the mouse and keyboard are still the primary interface for Windows.

I suppose this is one area where Apple and iPadOS succeeded, they started with Touch only and slowly progressed toward keyboard and mouse interfaces, but it feels incredibly unfair to compare something that was developed from different viewpoints. I don’t think Windows will be ever truly touch native to the level of iPadOS, and now for future devices I may actually avoid touch screen devices and pickup a recent iPad for all my touch and writing needs.

“Soft” Keyboard

While the keyboard does indeed function of a keyboard, the lack of a hard hinge really restricts the usability of the Surface Pro in all situations, if you don’t have enough space for that kickstand then its not worth even trying to use the device. You can get third party options to give it a hard keyboard but this is really a mistake on my part, I never thought I was a lap user of a laptop, and it turns out I am…

Hardware Specs & Quality

Originally when I bought the Surface Pro I was doing a spec for spec comparison between the latest Macbook and the Surface Pro, which I feel was my biggest mistake. Again, Apple works hard on that software and hardware integration and Apple manages to squeeze every little ounce of performance out of a relatively mid spec machine. For the Surface it seems that while Microsoft had done some performance tweaks and modifications to the device firmware for speed it fails terribly with the quality of the software itself.

The Surface range has a lot of firmware issues, one big one for me is that sometime the charger just stops being detected, it wont light up and charge and before you realise it your on 10% battery in the middle of a project worrying that your power brick is dead.

Another issue i’ve experienced is the 400mhz clock issue, the CPU thinks its overheating and the thermal throttle kicks in, totally expected for a system with no fans in it, but it never stops. The only fix is either to run a piece of software to force a flag off on the CPU, or a hard reset of the hardware itself. Thankfully this has been fixed but it took Microsoft over 6 months to acknowledge and fix the firmware.

What Next?

Honest, I may go back to a Mac. I really like my Windows environment, i’d love to have a Linux laptop but my workflow is so dependent on Mac/Win tools that I can’t step away at the moment. Maybe i’ll look at Lenovo or Dell for my next device…

Fifteen years later and i’m swiftly approaching my 39th birthday, time to pickup the book again and refresh myself. This time its with the 2015 Edition of the book, the fundamentals are still there but its had small updates for the ever moving state of technology.

In a nutshell, for newcomers

The idea of Getting Things Done is better living through lists, getting all those pesky tasks out of your head and into a trusted system. By trusted system that could mean one of many things, your pocket notebook, a to-do list app, back of a napkin, or post-it notes on your monitor. The system doesn’t have to be something tried and tested, it only has to be something that you can trust to keep your lists. Once everything is out of your head you can start doing rather than worrying about what you’re forgetting.

Another element is the “Inbox”, a method to store items for review later and potentially add to your task list, a reference file system to keep important information filed away and easily accessible, and how to manage projects from start to finish.

By combining these techniques David hopes that you can attain “Mind Like Water”, in that you can react quickly and efficiently to new issues that arise without having that drowning feeling of hundreds of to-do list items building up on you.

Now, back to my reading.

Projects, reviewed

After my second reading it became obvious I have been doing projects wrong, very wrong. I’m not sure where my initial misunderstandings came from, maybe it was worded differently in an older revision, but this time it clicked. The biggest takeaway was if you look at a next action and it needs more than one step to complete that action then it probably should be a project. It sounds stupid, but take these examples:

• Buy new household contents insurance.
• Pack my suitcase for holiday.
• Clean the house

I can hear you saying “Wait a minute, they’re next actions!”, let me show why they’re projects:

Buy new household contents insurance.

• Create a contents inventory and valuation for the house
• Research which providers have the required cover
• Use a price comparison website to find the cheapest
• Purchase insurance

Pack my suitcase for holiday

• Create a packing list of items
• Wash any clothing that is needed
• Pack items into packing cubes
• Pack toiletries
• Pack suitcase

Clean the house

• Buy replacement bleach
• Vacuum Living Room
• Dust Shelves
• Empty Trash from bathroom bins

While its much easier to bulk all these items into a single next item, you’re not really breaking it down to atomic tasks that can be done individually in a short amount of time, when grouped together “Clean the house” is a two hour job, where as “Dust Shelves” is a ten minute task that can be done in isolation. By breaking up the tasks you start reducing the amount of time required to complete the task, people generally have a few hours of spare time to get things done, and these types of a projects could be chipped away in ten minute chunks through out the week.

The power of the ‘Someday/Maybe’

Another sticking point I had was the use of the Someday Maybe lists.

Make an Inventory of Your Creative Imaginings What are the things you really might want to do someday if you have the time, money, and inclination? Write them on your Someday/Maybe list.

After reading this in the book, I suddenly look over at my to-do list of hobby projects and realise i’ve been creating another Someday Maybe list away from my main list. As for why, I have no idea. My main list seemed to be reserved for the big picture goals, retire, progress my career, complete my bucket list.

Having a separate list is by no means a bad thing, even David recommends it, but for me it had a mental distinction away from a Someday Maybe list, when in reality its exactly the same thing.

The ‘Horizons of Focus’

This was another topic that confused me, maybe it was due to my age, or maybe that i’ve never had “horizon 0” under control, but I never did really see the point of the exercise.

What are your key goals and objectives in your work? What should you have in place a year or three years from now? How is your career going? Is this the lifestyle that is most fulfilling to you? Are you doing what you really want or need to do, from a deeper and longer-term perspective?

The higher levels are not really about day to day work, rather keeping a list of what will drive your future projects, plans, and your someday list. Introducing “horizon 3” or “Areas of Responsibility” has been a key driver for a lot of new actions and projects in my system. it may feel obvious to collect next actions into logical groups, and you may already do this by using contexts, but by having those groupings outside of your system you just have another thing in your mind to keep hold of. My current list of areas are:

• Self
• Finances
• Home
• Family
• Job 1
• Job 2
• Hobby 1
• Hobby 2

Much like a trigger list, those areas drive my mind in the right direction and usually make it easier to identify new actions and items for my inbox.

Don’t be strict with contexts

Before I go on a long trip, I will create “Before Trip” as a temporary category into which I will move everything from any of my action lists that must be handled before I leave. That becomes the only list I need to review, until they’re all done.

Contexts are a tool, I used to have them as a fixed list in my Nirvana; Calls, Internet, Office, and spent time tagging all actions for only that list. That quote from David changed my thinking of this, your context list should be adaptable to what is coming up in the near future, if you come across a task that would work better in a new context, why not?

I thought of a prime example for this, a few years ago I went to a wood turning symposium and I have a specific list of items I wanted to achieve while I was there; buy this, buy that, meet x, pickup a copy of y. I was building a shopping list of actions to complete at the hotel it was based at, when that could of just been a single context I could create and solely focus on for the weekend I was there.

Conclusion

For most people who have read Getting Things Done i’m sure my point are nothing outstanding, but its always good to grow and learn from previous mistakes. GTD is by no means a perfect system, or is it a set of fixed rules to follow, everyone has their own implementations and quirks. Its always useful to re-read the source material from time to time, as looking at it from your current perspective could bring new insights, and maybe fix an issue or two you’re currently having. I can safely say I have a lot to re-implement as my personal system has drifted, for the worse, from the ideal.

For the longest time I ran a modified version of NextAction which did most of the heavy process lifting in Todoist, but when it came down to it the script was just a kludge to make a non-GTD system work with GTD ideals. NirvanaHQ was designed from the ground up with GTD in mind, and with that a lot of the processes and ideas in GTD worked seamlessly out of the box without any modifications. Moving my workload was easy and after taking a day or two to trial the system I jumped in with a lifetime subscription.

My filing system started out as a basic Evernote account, initially spurred by the good reviews from Merlin Mann and other names in the early GTD community. Evernote seemed to cover the essentials, somewhere to store your documents and with a well thought out organisation and search system. It worked, I gathered around 2,000 documents within a short time period and most of my paper documentation was quickly scanned, stored, and shredded.

Within a few years, Evernote started ramping up their subscription fees. While I can’t argue the service was worth paying for the issue I had was that I was paying for Office365 at the same time, which included OneNote and a sizeable amount of storage. During that time I was trying to reduce my overall subscription costs and decided that Evernote was a easy tool to be rid of, and I migrated to OneNote as my filing system.

NirvanaHQ and OneNote have been the cornerstone of my system for a good three years, and yet again i’ve recently been shown a new system: Notion.

Notion came along and really changed my view of what a filing system could be, with Notion you have the concept of Pages and Databases, which is a new type of flexibility that OneNote or Evernote could never offer. While they both have the concept of tables they were never truly databases that could interlink data.

About the same time I came across the PARA Method which works well with a GTD system to provide some structure and management around your filing system. Within the first book of GTD the filing system is referenced as a store that needs reviewing every so often for cleaning it out, with PARA it splits the mass into “Areas” and “Projects’ much like GTD, and assigns the useful information storage to “Resources”.

Notion has allowed me to build my PARA method storage system in a very structured way, allowing me to define a database for areas, projects, and resources but allowing those records within to be free form and formatted how they need to be. PARA is by no means restricted to a certain set of tools, its worth noting that Tiago’s blogs do use Evernote as the example data store.

The only item i’m missing from Notion is something Evernote did really well; storage of actual files. I’ve amassed a large collection of PDFs of scanned payslips, copies of instruction manuals, and receipts from that expensive but fragile item. OneNote had the same issues in that it never really like the concept of storing a file, Notion is the same but i’m sure that over time it’ll resolve itself. For the moment the raw files are filed away on Onedrive, in what seems to be the last bastion of chaos in my system.

My system is now comprised of the following:

• Task/List Management - NirvanaHQ
• Reference System - Notion
• Calendar - Office 365 Calendar
• File Storage - OneDrive

I’ve also added a new item on the header of the site to track my workflow tools, and i’ll be keeping it up to date with what i’m using.

It is not unusual to find many people jaded at best about the value of goal-setting, given the stress created by what are often perceived as artificial expectations decreed from on high. - David Allen

Nothing sums up my experiences with goals better than that quote. I’ve been a recurrent “goal breaker” for many years, in that every goal I set myself I either fail or ignore after a short period of time. Getting the right balance in a goal can be difficult and almost a skill in its own right. Most people struggle with either trying to define the scope of something monumental and endless (e.g. Progress my Career), or keep it too easy (e.g. Don’t dine out for a month) and feel unfulfilled when completed.

A lot of discussion about goal setting has been done by almost every productivity “guru” out there. You can find hundreds of articles talking about the best way to do it, but i’m not here to discuss that, more to explain how I came to the goal i’ve defined for this site.

Starting a project like this site is useless unless you have something to aim for. Writing out into the void is a good cause but if you’ve got nothing to drive you then your willpower to continue will drift away after a short time. Personally I want to write something, be it technical and informational to just airing my mental drool, but I want to do it consistently and with purpose.

Frequency

I want to post frequently, but not place such as strain on my free hours that it causes other obligations to slip, or that the site is pushed aside by other pressing issues. In the past i’ve set myself a goal of posting daily, which was a huge mistake and within two weeks I was three days behind. This time i’ll create a sensible balance with my available time. I’m lucky in that I can enjoy my weekends relatively interruption free, so i’m able to ensure I get at least an hour to put together an article.

Content

I want the articles to be a length worthy of posting, rather than some short snippets. I’ve previously fell into the trap of producing content that was nothing more than a few lines of semi-related information thrown together to form some sort of post. What does that actually achieve? Yes i’m ticking off my goals but essentially I was cheating myself of a satisfying result.

I want the site to have some consistent and meaningful content, which people would like to read, and keeping to the topic for the overall theme of the site. Again, referring back to some old blogs i’ve deviated off-topic too many times, and potentially losing readers due to it. If i’ve decided on a topic I should stick to it.

The Goal

Post at least one new article a week, and the content should be meaningful and keep with the overall topic and theme of the site.

With the goal set, I can start thinking about next actions and projects, defining the scope all with an aim to complete this open-ended goal.

ShopDisney is suffering from a bot problem. With the parks and stores shut, Disney now sell limited edition items through the ShopDisney website, in the past, the stock has been equally split between the stores and online to avoid a single location being the only marketplace for the items. Since lockdown has begun, the technical collectors have worked out the best way to game the ShopDisney website to access limited edition items before everyone else.

Today, I aim to make this information a little more public to allow Disney to consider fixing it.

So we have the following facts of how ShopDisney works:

• Each item has a Product Code. This product code is used everywhere on the ShopDisney website: pages, image references, XHR calls, and so on.
• Visiting shopdisney.co.uk/<product id> redirects you to the correct page for the product, even when not actively for sale.
• The CDN for the ShopDisney website is hosted by Adobe Scene7.
• Scene7 has no rate limiting or banning of frequent requests.
• Images are available on the CDN even when the product is unavailable to purchase.

Interested parties could for example:

• Take a Product ID of the same type of item you’re interested in, say a pair of limited edition Ears.
• Iterate the ID and hit the CDN until you get a 2xx response.
• Hit https://shopdisney.co.uk/<id> and get redirected to the correct URL for the product.
• Log the URL and image.

After a few hours, it is possible to discover a slew of items that will be available shortly on ShopDisney. Others have reported that the XHR-based shopping basket system trusts the client when adding items to the basket. You won’t be able to checkout the basket items, but with popular items, the few seconds you’ll save by having the items in your basket already will give you a significant advantage.

The Format

I spent some time on what format I would ideally like to keep these posts in, text with some simple markup for links and formatting. The obvious chose was Markdown, sticking to the true core of the markup and not using any extensions or plugins available for most parsers these days.

Of course not everything should be embedded in the post, so I took pointers from Hugo and Jeykll and worked the posts into a YAML/Markdown hybrid. YAML’s multidocument feature makes it easy to parse these documents with a few lines of Python:

import yaml

with open(args.file) as fobj:
    raw_doc = fobj.read()

_, header, text = raw_doc.split('---')
docs = [yaml.safe_load(header), text]

So an example file would look like:

---
title: "Post Title"
date: 2020-07-09
tags:
    - tag1
    - tag2
---
Post *content* **goes** [here](#)

Gathering The Corpus

As it turns out, I wrote a lot of posts. While I don’t think i’m at the level of a daily blogger I was a very directed technical person who only took time to write a blog post when I felt the need to broadcast something to the world. So having a collection of over 300 posts feels impressive for me, sure a vast majority of them a small and pointless posts, but good for me.

The issue next is that a lot of these posts were in various formats: B2, B2evolution, Mephisto, Wordpress, and archive.org stored only one type of output: HTML. Converting these posts is all manual labor, find the post on archive.org, copy and paste the formatted text, and correct the formatting in Visual Studio Code.

So far this process is ongoing, i’ve done 3 years of posts and i’ve still got 10 years more…

Reviewing My Old Posts

The next job would be to review the posts and see what is applicable and acceptable for the modern internet. 2003-2007 was a strange and weird time on the internet, most of the big social media wasn’t around and people were generally safe posting far too much information online. Everyone had their guard down and blogs were usually only seen by close friends and family in a person’s “bubble”.

For example I purchased a Apple Powerbook in 2003, for some reason I felt the need to share the tracking number of my deliver online for people to view where my delivery was up to. Today that’d be stupid, really stupid. I think if you did that now on social media someone would call up the courier and re-route your parcel.

Another “elephant in the room” is the direct posting of my inane mental drivel from my angst filled years of my late teens. I was stupid, young, and not really thinking about what I posted and why. In the end it did actually lose my job due to a post I made on that blog, and today some of those posts will be classified as hateful. Thankfully i’ve grown and i’m not the same person as back then, so these posts will be confined to this post archive and archive.org forever and will never be published again, stored there as a reminder to myself.

Epilogue

Gathering my posts has been an interesting exercise, while a lot of up front work i’ll hopefully be able to avoid this in the future by sticking to the Hugo/Jekyll mixed YAML/Markdown format for all future posts. At least now i’ll have a corpus of my output in one place that I can take to any new re-design or site that takes my fancy.

Also from a GTD this feels like one long review of my past, it has allowed me to see some of my actions in new light, see old hobbies i’ve long since forgotten in a new light, and gave me some new aims for the future.

The problem is when you’re trying to setup a pubnix system with a legacy feel, it doesn’t really feel right excluding some tools that people want to use. For example using Maildir restricted the usable mail clients essentially down to mutt, which can be a bit of a beast to use at the best of times. ~gohan asked for alpine to be installed as it was his preferred client and the question came forefront again as to why I’ve setup Maildir.

Thankfully a patched version of alpine is available to support Maildir, so i’ve compiled a version and placed it on the Diemsion RPMs repository as alpine-maildir, for anyone interested.

But, we did encounter a few issues with some other packages, for example efingerd and gophernicus, these are available on most Debian distributions but not for RHEL base ones. So I took the time to create some package SPEC files and create the RPMs for CentOS 8.

These SPECs are now available on GitHub, and the resulting RPMs are available at YUM repository hosted on GitHub pages. To use the YUM repo all you need to do is either manually download the .repo file. or run the following:

yum-config-manager --add-repo http://dimension-sh.github.io/dimension-rpms/dimension-rpms.repo

At the moment I’m not planning on building for RHEL/CentOS 7, but the SPECs should work for them and its relatively easy to build the repository manually.

The error I encountered was “Cannot run upgrade script on host”, a lovely generic error which had me scrabbling around inside the ESXi logs to find the solution: It turns out that one time or another I put the USB stick with the ESXi install into my Mac, which in turn sprayed a collection of “.Spotlight-V100”, “.fseventd’, and various other Mac specific files in the local datastore and various critical folders in the filesystem. Thankfully the host still booted so I was able to resolve it.

• Enable SSH on your ESXi host
• Run find -name ".Spotlight-V100" -type d -exec rm -rf {} \;
• Run find -name ".Trashes" -type d -exec rm -rf {} \;
• Run find -name ".fseventd" -type d -exec rm -rf {} \;
• Re-run the Upgrade

Now this should of all worked, and the logs indicated it wasn’t failing on any silly parts, but again I was hit with the “Unable to run update script” error. Further digging was required.

It turns out that VUM writes out a very detailed log of exactly what it is doing into /var/log/vua.log and this should be your first port of call for debugging any issues. In my log it indicated it was expecting that 6.5 was already installed and when it was comparing the list of VIBs to update it was extremely confused why everything was out of date. It seems that ESXi depends on a system called “locker” to store all its package information and one of the first VIBs that is updated includes the updated locker files. Somehow I had to revert these files back to the 6.0 files. VMWare itself seems to recommend copying over the files from a working host, which wasn’t possible in my case as the other host was already on 6.5. So I held my breath and did the following:

• Remove tools-light using esxcli software via remove -n tools-light
• Install the 6.0 version using update profile esxcli software profile update -p ESXi-6.0.0-20161004001-standard -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml

…and thankfully it seemed to complete without issue. Running VUM from then on updated to 6.5 without issue. Each situation is different, but looking at the logs can really give you some insight into what is going on, or you can run the installer ISO directly and watch it spit out the specific issue its hitting. But, here are some reference links for working out what is wrong:

• VMware KB - Upgrading a VMware ESXI host fails with the error “Cannot run upgrade script on host”
• VMware KB - “The host returns esxupdate error code:15
• viktorious.nl - ESXi 5.1 upgrade fails with the error “Cannot run upgrade script on host”
• virtuallyhyper - ESXi host fails to upgrade to 5.1 with Update Manager

Thankfully, after a while Googling and a few experimentations i’ve found the following magic bullet:

# apt-get install libxml-libxml-perl perl-doc libssl-dev e2fsprogs libarchive-zip-perl libcrypt-ssleay-perl libclass-methodmaker-perl libdata-dump-perl libsoap-lite-perl libdatetime-format-iso8601-perl

# echo "ubuntu" > /etc/tmp-release
# export httpproxy=
# export ftp_proxy=

This works for Debian 8 (Jessie), and its been reported that it works for Debian 7 as well.

Ps. VMware, no, /usr/bin isn’t a sane default for installing your software into!

The “current” generation (and I use that loosely) is a rebadged Dell R710 with the bare minimum fitting options:

• 2 x Intel Xeon E5620
• 48GB ECC RAM
• PERC H700 with 8 x 2.5" SAS drives
• iDRAC Express

Extra niceties have been left out to cut costs, so no internal SD card reader, no CD/DVD drive. The hardware was generally given free with a license so you can find these devices popping up on the market every so often when people’s licenses expire and they don’t want the hardware filling up their racks.

Google actually publishes some quick guidelines on repurposing the hardware after the end of the license which should be good enough for the vast majority of people, but the BIOS is out of date and its still tagged up with the Google Search Appliance boot screen, finding updates for the BIOS is near impossible. If you want to run ESXi with all the fluffy bits it can be a bit of a pain.

But, this is a R710, can’t we just use Dell’s version?

Actually, yes you can. On the Dell R710 support page, grab the latest BIOS package in the “Non-Packaged” format and put it on a bootable USB stick with FreeDOS on it. Then just follow these steps:

1. Pop the lid off your GB-7007 and disable the BIOS password (check on the back of the lid for details)
2. Boot the system and enter the BIOS, change the boot order to USB first.
3. Reboot with the USB stick in one of the ports, wait until you hit the FreeDOS prompt
4. From the prompt run R710-060400C.exe /forcetype

OK, that last item may look a little scary. The update process has a check in it to see if the system you’re trying to flash the BIOS to is the target system, this appliance will identify as a Google Search Appliance so will always fail this check, even though the hardware is identical to the R710 system. The /forcetype option disables this check and forces the BIOS to install.

After a minute or two your system will reboot and you’ll get the normal Dell boot logo and options, congratulations, your Google Search Appliance is now a Dell R710 sporting a lovely yellow case.

First of all, check what firmware and boot ROM your switch is using using the show flash command on the CLI:

sw3# show flash
Image           Size(Bytes)   Date   Version
-----           ----------  -------- -------
Primary Image   : 3003952   12/21/05 I.08.87
Secondary Image : 3003952   12/21/05 I.08.87
Boot Rom Version: I.08.07
Current Boot    : Primary

All firmware versions I.08.07 onwards need the I.08.07 Boot ROM, and you’ll need to flash up to this version first. Thankfully HP provide that specific version on the website to download. Follow the exact same steps as below for I.08.07, then repeat for whatever version you’re upgrading to.

To get the firmware to the switch we use a TFTP server, a little out of scope for this article but you can find a lot of free and open source servers for this, i’m using my local pfSense gateway’s TFTP server for this, i’ve uploaded the I_10_107.swi firmware file to the TFTP and from the switch’s CLI I run the following:

sw3# copy tftp flash 10.1.1.1 I_10_107.swi secondary
The Secondary OS Image will be deleted, continue [y/n]?  y
03261K

After a few seconds you’ll be back at the prompt. To check everything has worked as expected check the show flash command:

sw3# show flash
Image           Size(Bytes)   Date   Version
-----           ----------  -------- -------
Primary Image   : 3003952   12/21/05 I.08.87
Secondary Image : 3428242   08/24/15 I.10.107
Boot Rom Version: I.08.07
Current Boot    : Primary

All you need to do is reboot the switch with the new firmware, check everything works, then flash over the image to the primary flash storage:

sw3# boot system flash secondary
Device will be rebooted, do you want to continue [y/n]?

Once the system is up and working, use show flash again to check its booted to the secondary area.

sw3# show flash
Image           Size(Bytes)   Date   Version
-----           ----------  -------- -------
Primary Image   : 3003952   12/21/05 I.08.87
Secondary Image : 3428242   08/24/15 I.10.107
Boot Rom Version: I.08.07
Current Boot    : Secondary

And if everything is working as expected, flash the firmware over to the primary image exactly the same way as before

sw3# copy tftp flash 10.1.1.1 I_10_107.swi primary
The Primary OS Image will be deleted, continue [y/n]?  y
03261K

For the final (optional) step, switch back to the primary image:

sw3# boot system flash primary
Device will be rebooted, do you want to continue [y/n]?

And you’re all done.

Main problem is that most of these ex-corporate switches come pre-configured with some setup you neither know or care about, thankfully resetting this switch is amazingly easy.

1. With the power on, poke the Reset and Clear buttons at the same time with whatever pokey devices you can find.
2. Release the Reset button
3. Wait until the Test LED starts blinking
4. Release the Clear button

Within a few seconds you’ll have a factory default switch, grab your straight through serial cable and have a play with the CLI.

Its worth taking the time to get the firmware up to date, remember to check the change logs and the documentation as some interim steps may be needed to bring it up to the current version. The software for this switch has progressed quite a bit, its still the same horrible Java based Web UI but little features introduced here and there really help out.

What I thought was a simple configuration change has been puzzling me for the last few days, what I didn’t realise is that after switching on IPv6 in the OpenVPN server all IPv4 traffic hasn’t been correctly routed via the VPN. It turns out a small issue in either the OpenVPN client, iOS or something in-between has broke the configuration, but thankfully it only requires a small fix.

The solution finally came from the OpenVPN bug tracker, ticket 614:

IPv4 routing on iOS 9 is broken if IPv6 is enabled inside the tunnel. The tests were done with tun-ipv6 and redirect-gateway activated and all the IPv4 traffic bypasses VPN gateway, while IPv6 works fine. Works as expected without tun-ipv6. Doesn’t work with tun-ipv6 but no IPv6 address.

Exactly what I was experiencing. Thankfully fkooman came across an entry in the FAQ which mentioned an undocumented option called redirect-gateway ipv6. Injecting this option in the OpenVPN server resolves the routing issues.

On pfSense you just need to add push "redirect-gateway ipv6" into the “Advanced Options” section of the OpenVPN server configuration

With the aim to take matters into my own hands I decided to hunt around for an open source solution that I could self host, I’m already paying for a dedicated server so why not use that to host it?

Thankfully, it seems that a lot of other people had the same issue and a large list of open source solutions had popped up. The interesting one seems to involve the “Fever” API, which is a simple method of exporting these feed readers out to mobile and desktop readers without any quirky reader dependent applications, my favourite RSS application Reeder supported this API so really helped with the decision of what solution I needed.

Miniflux seems to be the perfect balance between function and simplicity, It can be installed damn near anywhere as it only uses PHP and a few standard modules, in addition it supports importing and exporting OPML files and the Fever API to allow my desktop and mobile client to keep in sync with no extra work.

Installation couldn’t be simpler. Checkout the repo, move to a folder of your choice and throw in a Nginx configuration:

server {
  listen 80;
  server_name rss.domain.com;
  root /home/user/www/rss.domain.com/;
  index index.php;

  index index.php index.html index.htm;

  # the following line is responsible for clean URLs
  try_files $uri $uri/ /index.php?$args;

  # serve static files directly
  location ~* ^.+.(jpg|jpeg|gif|css|png|js|ico|html|xml|txt)$ {
    access_log        off;
    expires           max;
  }

  location ^~ /data/ {
    deny all;
  }

  location ~ \.php$ {
    # Security: must set cgi.fixpathinfo to 0 in php.ini!
    fastcgi_split_path_info ^(.+\.php)(/.+)$;
    fastcgi_pass 127.0.0.1:8812;
    fastcgi_index index.php;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    fastcgi_param PATH_INFO $fastcgi_path_info;
    include /etc/nginx/fastcgi_params;
  }
}

Done. Your Fever API endpoint is available at /fever/ and the username and password can be configured in the UI for the application. Everything is stored in Sqlite so easy to backup and move around.

If you’re looking for something thats simple and works, i’d recommend giving it a try!

Puppet Enterprise comes with a 10 free nodes license as stock, for a small homelab its perfect for managing that configuration that applies to all systems, DNS, routing, SSH keys, you get the idea. Also, as my day job runs Puppet Open Source its great to test out the commercial version and get to know it before the inevitable upgrade where a lot more is at stake.

For my installation I went with CentOS 7 and a single node installation, I use Code Manager to automatically deploy my configuration from a git repository I have stored in Gogs, which if you’ve not seen already I highly suggest checking out. Agents are mostly Debian 8 with a sprinkle of CentOS7 and RHEL7 for my learning needs.

Heres some handy hints from my Puppet usage, both in work and home:

Use Puppet Enterprise

10 free nodes! Take advantage of it if you can. While open source Puppet is great, the installer and Console makes Enterprise worth the $100/year/node just for the saved time of fiddling with config.

Use Puppet Forge

It might seem obvious, but a lot of places suffer from NIH when it comes to Puppet and decide to re-write from scratch instead of expanding on an already open source module. While the vast majority of modules I use are public on the Forge I have slipped into a habit of quickly hacking together a profile for an application rather than write a full module to share. In general using the Forge will save you time, so take advantage of it.

Use distro packages

While you can grab x .tar.gz from y website, extract, run, copy files and such, save yourself the pain and use distribution packages whenever possible. Not only does it make for much easier installation and management it saves you a lot of time when it comes to upgrading.

Don’t aim for 100% coverage

Trying to configure every part of a system with Puppet will burn you out quickly, cover the required elements and tick them off first. In my opinion Puppet shouldn’t be handling assigning IPs to devices or managing file systems, but setting DNS, firewall rules and package repositories are right up its street.

Things break, so check your config first

The --noop option is your friend. Make use of it to check that your new shiny config won’t blow a hole in the side of your system due to a dodgy Heira YAML file. In Puppet Enterprise you can even run this from the console.

If you have the infrastructure to spare, get a Jenkins system setup and lint/test that config before it hits a live system. If you want to get really fancy, have Jenkins auto push to your production branch after testing for that continuous deployment feeling.

Puppet Enterprise has Application Orchestration!

While its a recent development I highly suggest you read the documentation and have a play with this. Hand holding multi system deployments is no longer needed!

I’m sure people have a hundred and one other things to say, but i’ll leave that for the experts…

The strange part was how it failed. Usually these SSDs just stopped working in every way and would appear to the BIOS. In this instance it was still there, it still booted, and it got about half way through the boot sequence for Windows XP before dying with IO errors BSOD. At the time I wrote off the disk as a complete failure, trying to plug it into another PC didn’t work, USB to SATA connector didn’t work, even when I did manage to get recognising on a system it said around 95% of the blocks were bad on the device. New SSD was purchased and this one was forgot about on my desk until I picked up a new USB 3.0 to SATA cable from Amazon today.

On a whim I decided to plug it into the drive, then into my Mac. OSX by default doesn’t write to NTFS but can read it, and it turns out it identifies something very weird in this device. When operated in read-only mode with no writes attempted to the device it works perfectly, this also confirms what I was seeing in the PC in that the boot loader and initial stages of Windows XP worked fine, but when it came to actually check the disk and do a write it caused the device to lock solid.

So, if you have a OCZ Petrol that you need to recover data from, try getting a device that supports write blocking and give it a go.

Today i’ve been finally configuring my FreeNAS installation on my HP Gen8 Microserver, and after a good twenty or so minutes researching the issue I found a small post on the FreeNAS forums suggesting the following settings:

ea support = no
store dos attributes = no

Boom, quickly added to the configuration files and browsing now flies. Next is to try and improve the overall transfer speed over 25MiB/sec.

In addition to the new switch I “acquired” Jo’s PC to use as a VM host, with some extra network cards and a bit more memory its now serving as a multifunction machine; pfSense, various lab VMs, and monitoring systems.

From time to time i’ll be posting about my latest experiments, what i’m learning, and how its now presenting a even larger drain on the electricity than it was previously.

As to expect, it all came pre-configured for the last user’s network, and this time I had no helpful IP sticker like the HP 2824 (I have no idea how I managed to reset that). This switch however had a RJ45 console port, which luckily responded to a standard Cisco rollover cable. Once you’re all plugged all you need to do is:

• Set the port to 38400 baud, 8 data bits, No parity, 1 stop bit (8/N/1)
• Reboot the device, when prompted hit CTRL+B
• At the prompt reset then reset again and the device should reboot.

Once alive again the switch should be available on 192.168.0.1 with admin as the username and admin as the password. I’ve yet to really delve into the CLI of this device but i’d expect some nicer features to be tucked away in there, for the moment its doing a fine job of splitting out that dirty Murdoch device (Sky TV box) and the snooping LG TV off the main network.

So, straight out of the box and into my Debian Jessie test system and everything worked, no tweaking or hassling, within seconds I had a working DVB adapter and I used the standard DVB tools to scan and create a channels.conf within a minute or so. My last experience with the LinuxDVB stack was around 2005-2006ish with MythTV, the drivers “sort of” worked and everything was a little rough round the edges, it seems the last 10 years have really cleaned up the stack. With that in mind its not really worth posting about getting the DVB-T tuner to work, because it just did…

SDR required a little extra work. I’ve not spent a large amount on time trying to get the full toolset to work on Debian, the rtl-sdr toolset is available as a package with Jessie and can be easily install, the biggest problem was that because I was using my test system I didn’t have a X session running to run anything on. I got everything installed and spun up a rtl_tcp instance without much incident, the biggest roadblock was that you can’t have the DVB kmod inserted at the same time as using the rtl-sdr package tools, but a quick rmmod and blacklist sorted that out, the tools are very quick to point out exactly what needs to be done.

Instead of working on Linux I got everything up and running on my Macbook Pro running OS X Yosemite, while OSX doesn’t have the full suite of tools available a few good ones have been developed for the platform. I found that CubicSDR was by far the easiest to get rolling with, no messing with MacPorts or any other third party packaging tools, just a DMG and a pre-packaged application. While it isn’t as feature complete as some of the other packages out there it does cover the basics to go poking around. Their todo list does looks interesting, especially with the target of having digital demodulation built in.

Quick overview done and i’m now looking for a better antenna. While not being used as a SDR the stick itself will be happily serving as a DVB-T source for my Plex system using TVHeadEnd and with a quick MCx to Coax adapter you can have it plumbed into the household aerial without much issue.

[Update - 2015/08/31]

Regarding the links earlier on in this post, it turns out that the two tuners are actually different, with the Nooelec it includes an improved tuner chip (the R820T2). As it turns out it is worth investing the few pounds more for this version as its more sensitive and also includes a more stable crystal that won’t require much adjustment.

So with this in mind I put a few hours into the tool today, it now has a proper setup.py and a basic CLI interface, just enough to get it running on my VM host without much issue or changes.

If you’re interested in my branch of the tool, check it out on Github.

My tool is already a solved problem, but as is the way of development and EVE i’ve set re-inventing the wheel for the sake of “security” and “counter-intelligence”, well, I spin it up that way but really I just wanted to try and do it for myself. In the last few years EVE has had a small UI overhaul which now allows almost anything to be copied and pasted outside of the game, the bonus is that once inaccessible scans, inventory lists, and channel member lists are now sources of information to be parsed and worked with. A common tool to come out of all this is a “D-Scan” tool that allows quick parsing and overview of the scan results from your directional scanner, over the last few years a good scan parser has become an essential tool of any FC and scout.

In my app i’m taking a new twist on the tool, trying out a few new views and consolidating some of the loved features from other tools into one that I can use. In the process of developing this i’ve set myself a goal of not having this tool depend on a database in anyway, instead using Redis as a caching backend for the various APIs and data stores needed.

The first big problem you need to work with is the EVE SDE (Static Data Extract) and the “Inventory Types”, this table of around 50,000 rows is something the tool will need to categorize the scan correctly. The positive here is that the SDE doesn’t update that often, only when we see content releases will the SDE be updated by CCP and even then the world isn’t going to end by not having the latest and greatest SDE to work with. So my solution was to have a package data file populated with a JSON extract of the data I need and when the data is needed its loaded into memory, the relative memory increase of 1-2mb of RAM is nothing in the overall scheme of the application.

So what about the actual scans and results? Parsing the d-scan data is relatively quick, as its essentially a tab delimited file of a fixed format, combined with a few quick lookups of reference data which is all held in a dictionary in memory makes even a taxing Jita d-scan get processed in a few milliseconds without any major optimization. Once the initial parse is done then the results are dumped to JSON, compressed with zlib, then dumped to a unique key in Redis with a expiry of an hour. The view to show the scan results does nothing more than to take the key from the URL and attempt to grab the results from Redis, decompress, and pass the resulting parsed JSON to the template.

The deployment target is Heroku, and ideally the Heroku free tier, so this has dictated some of the design, for example the zlib compression of the resulting scan is there to shave off as many bytes as possible to get the maximum use out of the Redis 25mb services you have available, with the requests we’re CPU rich but storage poor, so the trade off works quite well in this case. So how would this work in a DoS issue? If one person keeps spamming large d-scans into the system would the Redis server fill up and stop working for all? Well, no, as the config will be set to expire the oldest keys in the case of low memory which would work perfectly for our tool.

Today is a very different place, Auth was created at the start of a open source revolution for EVE Online applications, and over time more and more have started becoming open with now specific projects being spun up (such as ECM) to create tools, or large alliances (Brave Newbies) opening their backend for everyone to use.

The repository copies of the code I have a quite out of date, and i’m purely the copyright holder of them, which gives me the power to license and open them as I see fit. Now that its been a good amount of time since I left I feel I can safely release these into the public domain without doing any disservice to TEST and the current sysadmins.

So over the next few days i’ll be looking to move the following repositories from my private Bitbucket over onto GitHub:

nikdoof / cynomap nikdoof / django-testauth nikdoof / limetime nikdoof / pacmanager nikdoof / posmaster nikdoof / test-auth

All in various states, but hopefully useful for someone.

[Update - 2014/03/13]

Also i’ll update with links once they’re over.

Thankfully Jeff Knupp posted about open sourcing a python project the right way, which covers getting your project setup right, making it easily testable, and getting it working on TravisCI.

So, my project is live on GitHub, is it useful? Probably not, but at least i’ve put it out there incase people want to use it or improve on it. Next on the todo list is some better documentation and more tests.

The same happened tonight, exact same result but this time I was prepared to some point. After attempting to login in the console and seeing memory allocation errors, then SSH dying on its arse, I checked my Munin install and notice the memory was heavy swapping. This machine has about 8GB of RAM but at any time its using about 600mb, at first I thought it was a memory leak in something but usually OOM Killer does a good job smiting any unruly processes. Then I checked my process list and noticed it was well over 4,000 sleeping processes, something had obviously gone wrong.

On my Deluge setup, due to the instability of a few of the trackers I use, I have a small Python script that checks the current state of the torrent and if they’re “red” it restarts them. Deluge’s API uses the Twisted framework to make everything async and accordingly a lot easier to work with, this was my first venture into the land of Twisted and it seems I made an error; I didn’t catch the “unable to connect” error. So after it was unable to connect the Twisted reactor was sitting there and running constantly, and as this job was running every 5 minutes it stacked up over 24 hours and killed the machine.

So, its always worth checking for errors, and not assuming that it’ll sort itself out. Lesson learned.

This weekend i’ve been thinking of a few things that slow down my workflow, either time tracking, task planning or something to that nature. For a long time i’ve used Dave Seah’s Emergent Task Timer to track my time usage in work, for a while I tried the Flash app but it didn’t really worked how I wanted it. After a while I ended up going offline with Dave’s excellent templates and i’ve been a happy user for well over three months.

So the idea popped into my head to make a persistent, online ETT that allows for some quick exporting into a format I can use for my timesheets in work, but after a while I decided how much time would I have to spend to improve the paper system that Dave has honed over a long time? Especially now that he has expanded onto Amazon and selling pre-printed notepads of his ETP sheet.

The next idea was to create a CLI for Trello, and after about five minutes I realised its already a heavily done solution, fine no Go or Python client, but still done quite a bit. So here I am, twiddling my thumbs, waiting for the next idea to pop into my head.

upstream deluge  {
  server localhost:8112;
}

server {
  server_name  deluge.home;

  location / {
    proxy_pass  http://deluge;
    proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
    proxy_redirect off;
    proxy_buffering off;
    proxy_set_header        Host            $host;
    proxy_set_header        X-Real-IP       $remote_addr;
    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header        X-Deluge-Base   "/";
  }
}

Brother have released a driver set for the device, but it does have quite a few issues that present massive stumbling blocks. The deb packages seem to be setup for Ubuntu only, and die horribly when installed in Debian due to CUPS using a init file of “cups” instead of “cupsys”. A hacky way around it does exist, but honestly this is more poking than should be done for a simple deb package.

It seems that the driver has a few issues regarding configuration settings, define too many in CUPS and processor tool Brother includes segfaults, a fix does exist but a general lack of interest in this issue seems to perpetuate this error, even when a patch is available to resolve the issue. The driver has gone unchanged in Ubuntu and Debian for many years now.

Thankfully, it seems that Linux isn’t the only one affected by this issue. After several hours of frustration trying to get the device to work I gave up and setup the printer in CUPS as a raw printer, allowing my Windows PCs to use it with the official drivers, only for it to happen again…

So, why the post? Well if you’re looking for a Linux compatible label printer i’d advise to stay away from Brother’s offering. Mine works at the moment, but its doing what I really wanted it to do.

In previous versions of the API we handled databases in a very weird way, either by writing our own SQLAlchemy queries or boding the Django ORM to give us a basic cursor to work with, while it was far from perfect it allowed us to edit the databases of other applications without much hassle.

Recently, Django has updated into version 1.2 and with it came the MultiDB functionality which allows you to access multiple databases natively in the ORM. For our database layer this presents some new options not available to us in the old versions.

Using the Django database introspect you are able to generate a Model from a existing database schema, for this example we’re working with Mediawiki’s native database in MySQL. So first of all we need to define the database in our settings.

DATABASES = {
    'dreddit_wiki': {
        'NAME': 'wiki',
        'ENGINE': 'django.db.backends.mysql',
        'USER': 'wiki',
        'PASSWORD': 'passwordgoeshere',
    }
}

Next we fire off the inspect command to produce our database layout

./manage.py inspectdb --database=wiki > wikimodels.py

After a short time you’ll have a fresh Python module with all your database models nearly ready to go, first thing to do would be to edit this file and change any foreign keys to the required Django ForeignKey() field. While inspect does as much as it can it can’t detect foreign keys.

Once your read to rock its a simple case of getting your shell out and giving it a test run.

./manage.py shell
>>> from wiki.wikimodels import User
>>> User.objects.using('wiki').get(user_id=1).user_name
'Matalok'

Simple! No more hassling with db cursors, just simple ORM access without the hassle. The next big leap is defining the database connections at runtime, injecting into the DATABASES variable, by doing this I can remove the problem of having to manage the services’s database connection in the settings.py and instead have them defined on a per service basis.

That is why it pays to have the following settings in your apt preferences if you want to keep things to a minimum:

APT::Install-Recommends "false";
APT::Install-Suggests "false";

I’m now the lucky owner of two amazing little workhorses, a SPARCStation 20 and a UltraSPARC IV, both true workhorses of the Sun/Solaris era. The SPARCStation is a old world SPARC machine, in its pizza-box format and the interesting MBUS and SBUS connection interfaces with a sprinkle of SCSI, the Ultra on the other hand is more modern PCI and IDE so I was able to get fast network cards and big disks without much issue. due to the old nature of the SPARCStation hardware I’ve not really invested much time into it, I’ll have to poke at it sometime this weekend and get the old thing up and running.

For the moment I was far more interested in getting the Ultra running as a OpenBSD firewall / router, the Ultra is relatively small (but large when compared to the SPARCStation) and allowed me to tuck it under a table to run, much to the disappointment of my other half. The existing DD-WRT based Linksys was starting to show its age and was becoming flaky after a few years of running overclocked without added cooling. The SPARC machine presented an excellent opportunity due to the good support of OpenBSD and the amazing pf packet filtering included.

The next few days were spent faffing with the hardware and re-installing OpenBSD 4.5, I had numerous small issues that were all down to a faulty network card, possibly a faulty PCI slot but i’ve not had the time to push it further. The PCI issue was quite difficult to diagnose for someone whose had no experiences of the platform before hand, I’ve now had a days crash course in OpenBoot. I have to say, that OpenBoot is a fantastic platform and it aided me a lot in diagnosing the strange issues.

Unfortunately, there’s always one problem you can never get to the bottom of. I updated the box with a new PCI network card, a old CDROM and a fresh HDD, A day or so passed and no matter what I tried I couldn’t get the machine to boot from CDROM, it refused to detect media and even some times the drive. Thankfully, the previous owner had left a basic OpenBSD install on the machine that allowed me to download the install image and write to the swap partition, allowing for a quick and simple reinstall using the swap partition as the boot media. The CDROM works perfectly in OpenBSD, I don’t have the energy to chase down this bug any further.

Finally, after what feels like a week of work, I have a small footprint firewall that kicks the arse out of my existing DD-WRT box. While it may not be a amazing icon to show off, like Chris as his SGI coffee table, it gives me warm fuzzies that technology that would generally be disposable by modern standards has its use somewhere. Now the box is up and working, it’ll slowly disappear from my radar and the experiences I had with this individual bit of hardware will slip away, that is, until OpenBSD 4.6 or another hardware failure.

Fingers crossed eh?

First of all, grab the software pack from Rapidshare, due to the questionable nature of this copy of the firmware no one has been willing to host it on their own hosting, and I’ll keep to that idea. Extract the files from the RAR and you should have a firmware upgrade, and a installation folder for the connection software. As the existing Three connection software is very limited, the software package includes the Telstra version which allows you to define your own settings. Before you attempt the software upgrade, you need to remove any existing Three software, install the Telstra version and remove your SIM card from the dongle, then simply plug it in and run the firmware upgrade. This process will take around 15-25 minutes and once it’s done it’ll give you a prompt. During the upgrade do not power off your PC or remove the dongle from the USB socket. This will brick your dongle rendering it completely useless. Now, put in your non-Three SIM card and plug it back into your PC, the Telstra software should start-up and try detect the device, you need to configure the software for your provider’s APN settings, but the PDF document included with the software package will give you all the details you need. Remember, I take no responsibility for people bricking their equipment, you have been warned.

Out of Box Experience

I ordered my handset online so I had the joy’s of Royal Mail to contend with, that alone is a separate post and UK followers will know the usual pain everyone goes through in regards to getting anything shipped by them. The first issue that tripped me up was that the SIM card was loose in the packaging and not even slid into the handset box. In a rush I nearly binned the SIM and scupper my chances of actually using the device for another week.

The G1 is very nicely boxed, almost in the same way Apple devices are usually done. Since the release of the iPod and it’s over the top fancy packaging a lot of device developers have been scrabbling to match that “Out of Box” experience, the joy of opening the packaging and having your device presented to you in soft black foam padding. The first major gripe was struck at this early stage, while unpacking the device and it’s related accessories I noted that my white phone has some very nasty black accessories. While the contrast of black and white may work well in some peoples minds, I’d personally like the accessories to be a matching colour.

The box included the standard extras, a quick start guide, a small manual, USB cable, charger and a wired headset. Nothing really to write home about. The manual is the usual introduction spiel, which I refuse to read. I decided to get the phone up and working and to have a play around.

What surprised me next is my first battle with the phone, trying to get the SIM card in the actual device. Turns out you have to use the little pull tab at the top of the phone to remove the back, of course the device or documentation didn’t have any mention of how to do this. Jo can vouch that I spent a frustrated five minutes trying to tear the back off without destroying the phone. A simple plastic pull tab would of sorted this, but I guess it’s the last thing on the manufactures’ mind.

Next came the activation, I thought being a non-tied phone that I wouldn’t have to jump through so many hoops like the iPhone, while this is true the actual procedure can be a little frustrating. Anyone who has visited my house can attest to the near Faraday Cage properties that it has for some networks, unfortunately T-Mobile is one of them, the signal levels in the local area are great, just not in my house. This presented a major issue when I was asked to login to my Google account to sync over various details, the phone then spent the next ten minutes trying to establish a GPRS connection to the outside world. After the tenth or so try it managed to get all the details it needs. I understand that the activation sequence can also do WiFi connection but I didn’t see any mention of this during the set-up, and I think it’s reserved for the people who have “rooted” their phones already, something I’ll want to avoid wherever possible.

So after much hissing, cursing, and a few cups of tea, I was ready to roll.

Applications

Almost everything on Android is wrote in Java, and I’m quite amazed that it runs as well as it does. While Google/OHA are still polishing the edges the OS seems very stable and easy to use, once you’ve worked out the basics of navigating around you’ll be flying through the applications in no time. The standard “tool set” included with Android will cover the 90% of users, the usual host of tools are included; SMS, Email, Web Browser, Call Manager, along with a few others you might not usually see, like IM.

As this is the “Google Phone” the standard software includes the usual Google mobile applications, Gmail, Maps, and Youtube. I’ve recently moved away from using Gmail, so I can’t really comment on how the application works, and the rest of them operate just as you would expect on the N95 or any other Series 60 handset. I’m not going to spend all day digging into specifics as anyone who has had a go of these apps will know what to expect.

The biggest seller is the Market, the Android team broke away from the strict market that you see on the iPhone and went with a more open process.This has allowed developers to create a wide range of apps in a very short period of time, including replacement applications for the built in-clients. One great example is K9Mail, which expands on the existing Email client to enable better interaction with IMAP servers and a few added features. I’m sure over time the market will grow, and with the introduction of the paid market we’ll see some of the big players start developing apps for it.

Overview

Ok, I’ve warbled on for a while about my usage of the phone, it’s still early days and I’m still not 100% up to speed with the handset. It’ll take time and I’m sure i’ll have more posts in the future. I’m starting to get my feet wet in the SDK and I’ve got my first “Hello World” application currently installed on my handset. So, Would I recommend the handset to anyone else? A resounding yes, it’s got a lot of potential and anyone slightly technically inclined will love using it.

I’ve now got CrunchBang installed on my main desktop machine and I’ve been using it for a day, Maybe it’s a short length of time to review a distribution but I feel with my past experiences with numerous distros will help me get to grips with a new one quite quickly. Some of you may know, after being a Ubuntu user for well over a year I decided to move back to Debian and became quite critical of Ubuntu for its rash decisions regarding design and key choices. My dislike is not centred purely on Ubuntu, I remember one time where I had a near fit at using a OpenSUSE KDE 4.0 Live CD as I couldn’t switch off the default sound scheme, but that’s for another post. Back to the review…

CrunchBang Linux promotes itself as a lightweight version of Ubuntu, unlike Xubuntu’s XFCE desktop they’ve decided on using OpenBox and a few key programs from other desktop environments, like Thunar and Lxpanel.

My previous experience of the *box window managers have been with Blackbox during the very early days, when Enlightenment was all the rage and most distros used FVWM95, so checking out Openbox will hopefully be a refreshing blast to the past. My main concern was compatability, a lot of applications out there depend on certain features of the desktop environment. I left all my expectations at the door and decided to grab the Live CD and have a 10-15 minute play to see if everything works as expected and that it actually works on my slightly quirky setup.

The Live CD / Installation media is mirrored on a few sites, as it’s only a “baby” distro it’s not been picked up by the mainstream mirrors, thankfully, a few people in the community had offered some space up to the project and finding a local, fast mirror isn’t that difficult. As with all Ubuntu style Live CDs, it was a simple case of burning the ISO to a disc and rebooting the machine. I’m not sure if this is a feature of all new Ubuntu discs now, but the ISOLINUX menu had a option to check the installation media for errors, this would save you quite a bit of time if you suspect dodgy media.

The boot was quick, quicker than I expected. Usually with Ubuntu CDs I pop the disc into the drive the slip off to make a cup of tea and head back in time to get the last second or so of the desktop booting. This wasn’t the case with CrunchBang, after returning from a delightful brew making trip I noticed that the desktop was loaded and the default conky panel on the right side informed me that it’s been booted for about 5 minutes. So, boot speed, even from the CD it’s nice and quick.

To a user who has been brought up on the GNOME or KDE environments the initial desktop may take a second to sink in, by default it comes with a minimal panel and system information pane on the right side of the screen and nothing more, no desktop icons or fluffy applications menu, just a basic desktop. Right clicking anywhere on the desktop brings up the system menu and the list of applications. The default install gives you a nice range of applications, some you’ll never use, others are dire essentials.The default includes a few keynote applications:

• Firefox 3.0.4
• Pidgin 2.5.2
• Rhythmbox 0.11.3
• Skype 2.0
• Gwibber 0.7.2
• GIMP 2.6

A few more are available, and a full list can be found on the Crunchbang Wiki. Needless to say I was impressed, not only had they selected reasonable defaults but as the distribution is based off Intrepid it had the latest and greatest versions available. Skype is a interesting nugget in my opinion, possibly being the only QT application in the default installation. I do understand that lots of people use Skype for VOIP, but maybe they should consider including another application like Ekiga.

So, I have my desktop running as a Live CD, time to see how it fayred in real world usage. I can happly say, after a good hour or so usage I didn’t feel restricted by the choice of desktop environment, Openbox is low key but quick and powerful. I decided after just a few hours usage to commit to this distro, ditching my current Debian Lenny install.

The installation of CrunchBang was nothing really spectacular, It’s a standard Ubiquity installer which does it’s job very quickly. A few quick selections and the dreaded disc paritioner screens and you on your way. Installation took about 10 minutes on my machine and felt a little quicker than previous Ubuntu installs, but I put this down to a little bias on my part. Rebooting the machine brought up a standard GRUB menu and I happly noticed that it detected my existing Windows installation and put the relative entry in. Again, the boot was quick and my machine boots to the desktop in under a minute.

So, here comes the negatives. A few minor issues have bugged me since i’ve started using CrunchBang, but nothing show stopping. So to save time I’ll just put them down as bullet points:

• xcompmgr seems to have a “dicky-fit” after a few hours use, making all window focus go out of the window. Disable/enable of Compositing fixes that.
• Tray Icons are hit and miss to what actual colour they use for their background. In my case with the “Fawn” gtk theme you get either a brown or beige background, which looks a little messy. Not really a distribution problem but still annoying.
• Restarting Conky seems to paint over the entire desktop for no reason, causing the Windows-esq issue when you have to use an existing window to get the desktop to repaint.
• By default, the xserver won’t detect 1280×1024. Simple fix of modifying the Xorg configuration but initial boot of the Live CD can be annoying with a mishmash resolution.

As I said, the negatives are MINOR. Really, really minor. CrunchBang was designed as a “2nd - 3rd” distribution for users, so it targets the section that are more than happy to have a twiddle with the system configuration and the thought of text only configuration doesn’t phase them. If you fall into this category and you’re looking for a lightweight desktop distribution then i’d suggest you grab a copy of CrunchBang and give it a whirl.

My first talk was on the usage of the Wiimote within Linux, The Wiimote are very simple Bluetooth devices that can be accessed over the standard APIs with an additional library called CWiid. This allows the device to be used as a input device or as a general I/O device.

It’s quite hard to explain it in just text alone, So i’ve put my presentation on the LivLUG wiki everyone to have a look at. I recommend you grab it and give it a try yourself.

EDIT: Yes, It’s on the wiki now, but heres the direct link.

The day starts at 10:00am with a introduction and a simple overview of how to map your data, then we’ll head out and meet back at the Marbella Cafe for lunch, then head out again for the afternoon and concluding in drinks in a local watering hole.

Check out the wiki and Upcoming for more details. Hope to see you all there!

Marbella Café 2nd Floor Sunshine Studios 52-54 Newton St Manchester M1 1ED

Similar functionality can be achieved in Linux, and it’s very useful if your like me and want to drop someone a message when you don’t have your phone around.

For this we’ll be using Gammu, which is a toolset for managing phones via the AT GSM command set. It was originally forked from Gnokii, which was a similar toolset for Nokia handsets. As the E160G opens a serial port with access to the AT command set this is a relatively easy tool to setup.

First of all, we need to grab the packages. As these are standard Debian packages you should have no issues.

# sudo apt-get install gammu

Next, we need to configure Gammu to pickup the correct device. Check your dmesg for the serial port:

$ dmesg|grep tty
[12321.308078] usb 5-3: GSM modem (1-port) converter now attached to ttyUSB0
[12321.308275] usb 5-3: GSM modem (1-port) converter now attached to ttyUSB1

Edit ~/.gammurc, or run gammu-config to change the device settings. Your ~/.gammurc file should look similar to:

[gammu]
port = /dev/ttyUSB0
model =
connection = at19200
synchronizetime = yes
logfile =
logformat = nothing
use_locking =
gammuloc =

Give it a test by getting all the SMS from the device:

# gammu getallsms

This should bring back all the SMS currently stored on the stick, which should include your login details for the Three website (unless you’ve deleted them). To send a SMS use the “sendsms” command:

$ gammu sendsms text 07874454543
Enter message text and press ^D:
Test Message!!!!!1!
Sending SMS 1/1....waiting for network answer..OK, message reference=2

Gammu has a lot more tools and options to explore, now you have the basic config you can setup a SMSD, which can expose the ability to send SMS to a network. Also, Gammu has a python interface so you can possibly build your own frontend client for sending SMS. For more details explore the Gammu Wiki.

Oh boy was I wrong.

First I attempted to install Zeee (Zenwalk for the EeePC), the installation “media” came as a compressed disk image, nothing that unusual as most of the distros come in their own little installation media package. It turns out that this image is a raw dump of a file system, so I had to create the installation media on a USB stick with the various handy tools, mke2fs, grub, you get the idea. After about 45 minutes of fiddling I called it a day, for some reason the GRUB installation wasn’t detecting the ext2 partition on the USB stick, and couldn’t find the menu.lst file. While this is probably a simple issue it’s a bit more than I could be arsed with, the Zeee guys are doing well but the installation method need a little work, maybe a prepackaged ext2 dump

After the kerfuffle with Zeee I moved onto the latest Foresight Linux Mobile Edition, I’ve heard Dan & Fab mention Foresight on the Linux Outlaws podcast and I have downloaded a live CD previously, so I decided to get the image and have a go. This installation went a lot easier, the image was a precompiled usb installation so no hassle there, the installation took time but I put t hat down to the quality of the USB stick I was using. After about 30 minutes I had a working Foresight Linux install, and everything seemed to work out of the box, including the WIFI (which is the usual sticking point for most distros).

Foresight Mobile uses the clutter based launcher you can also find in the Ubuntu Netbook Remix, the mainstream applications are pre-installed and are usable. Within a few minutes I hit my all time pet hate, touchpad clicking, ever since i’ve owned a laptop I’ve never been able to use touchpad click to any degree of success and I don’t see any reason why it should be enabled by default. In previous distribution I know the way to fix this issue is to simply changing the settings in gsynaptics or modify the Xorg config, as I was trying to operate from a user perspective I went the simple route of using gsynaptics. It wasn’t installed. I went digging around in the package manager (conary) and didn’t find a related package. After about ten minutes searching I found the “synaptics” package, which proved useless as I had no idea of what it does.

Three hours in and my experiment with Foresight was over, people may complain that it’s a simple issue but having the option enabled by default and then hiding the configuration in a non standard package doesn’t help matters. I have to give Foresight kudos for being one of the first distributions to have a full “netbook” version, but it still needs a little refinement.

So, now I’m back on Debian, tried and tested. This time I installed using the updated Lenny installation media for the EeePC and it was a breeze, and since I’ve done this “fresh install” a lot more of the features work consistently. In the process of configuring my machine again I’ve noticed that the older guide for the E160G using Network Manager is a little wrong, so I’ll have to update that sometime. For now I’ll be sticking on ol’ faithful. Maybe when the “next big distro” gets released I’ll give it a try.

Over the next few weeks it’s my plan to experiment with a few EeePC tailored distributions, much in the same way Dan did. My first distro of choice is Zeee, which is a customised version of Zenwalk. I’ve heard good things about Zenwalk, so now is my time to experience it.

So, I’ll post a follow-up tomorrow.

dateadd(dd,0, datediff(dd,0,
   dateadd(day,-1*datepart(weekday,getdate())+1,dateadd(week,-1,getdate()))
))
dateadd(dd,0, datediff(dd,0,
   dateadd(day,7,dateadd(day,-1*datepart(weekday,getdate()),dateadd(week,-1,getdate())))
))

Dropbox is one of the new wave of online storage, sort of a cross between WebDAV and SVN, in fact, I’d say its almost exactly like SVN, just with a nice GUI. I guess all it would need to take the “value-added” part of this product is for someone to develop a nice front end for Amazon S3, and by looking at their future prices it could be cheaper.

Anyway, picking out the bits of the service is not what I’m here to do. At the moment I run a Debian Testing/Unstable desktop machine, I was quite disappointed to not see a specific Debian package for their software on the website. I realised after a few dumb minutes that I could use the Ubuntu packages.

In sources.list, I referenced their Gusty archive:

deb http://www.getdropbox.com/static/ubuntu gutsy main

Then in /etc/apt/preferences I set some basic package pinning to make sure that any packages didn’t collide with the existing Debian repository, not likely but you never know.

Package: *
Pin: release a=gutsy
Pin-Priority: 400

Do a “apt-get update” and you should have the “nautilus-dropbox” package available to install. Simple! Remember, you’ll need to restart nautilus by either killing it (killall -9 nautilus) or restarting your Gnome session. [edit: Fixed the first URL]

While hotkeys via ACPI are well supported by acpid and its ilk it is no longer the best way to handle these types of keys. Generally, the drivers for the mainstream laptops (ibm/lenovo, hp) have moved over to the input framework to communicate these key presses, usually displaying as an extra input device under /dev/input. These input devices can be handled by HAL and notifications of key presses send over the dbus allowing for desktop environments such as GNOME to handle these events without any strange hackery and fakekeys calls.

Thanks to the previous work of the asus-laptop developers there’s a patch that exists to disable the existing ACPI events and provide a input device for the extra keys, I’ve managed to hack together a version of the eeepc-acpi module using the Debian 1.01 source to export the “Asus Extra Buttons” input device.

After you have the inputs available, it’s a simple matter of producing a FDI for HAL to identify the device and map the scan codes to the actual keys. After the initial FDI was created I could use the volume keys without any extra software and also map the two application buttons (marked as VGA switch, and AP button) in GNOME to call scripts. The wifi key (Fn+F2) presented more of a problem, while it was mapped to “wifi” HAL didn’t know how to actually switch off the Atheros card. The killswitch for the card would need to be implemented as a program that listens to dbus, something a little outside my skill set.

The other buttons on the keyboard (sleep, brightness) are pure ACPI calls. This presents a problem that the keys produce events via the input layer and the ACPI layer at the same time, so for example you hit the brightness down button and HAL will pickup the notification and display the brightness OSD, but it quickly goes out of sync as what HAL sees and what the ACPI are doing are completely separate. Again, this is outside my skill set but I’d probably approach it by filtering out the keys in the kernel and let the ACPI events do their work.

The guys over at Fedora have a similar idea of moving over to an input based module, but for the moment no source has been produced. Due to the numerous little issues I’ve had I’ve decided to put this little project on the back-burner until I see what the Fedora people have produced, after all they’ll have people that are more experienced in this type of thing, whereas I am not.

I’ll get round to posting the source deb for the modified eee-acpi tonight or tomorrow.

Some people on the EeeUser forums ripped out the existing script from the default Xandros install, a simple acpi script that jumped through some hoops to disable the modules and clear everything down. The script worked as part of the existing acpi-support package and worked when using the acpi suspend options, now i’ve got GNOME and HAL installed it turns out these are no longer used, therefore still causing the issue.

After a little research it seems that the suspend support within Debian is currently in a state of flux, and a few bug tickets have been raised about the various issues. This provided my first hint of how to resolve it, a quick script in pm-utils, much like the acpi one, will fix this for good.

Simply, place this script into your /usr/lib/pm-utils/sleep.d/ folder, I’ve got it as 45eee-wifi that way if the script fails for some reason at least your video will be resumed.

#!/bin/bash

PWR=$(cat /proc/acpi/asus/wlan)

load_modules() {
   modprobe ath_pci
   modprobe wlan_wep
   modprobe wlan_tkip
   modprobe wlan_ccmp
}

unload_modules() {
   rmmod ath_pci
   rmmod wlan_scan_sta
   rmmod wlan_tkip
   rmmod wlan_wep
   rmmod wlan_ccmp
   rmmod ath_rate_sample
   rmmod wlan_acl
   rmmod wlan
   rmmod ath_hal
}

wifi_on() {

   if [ "$PWR" = "0" ]; then
      modprobe pciehp pciehp_force=1
      sleep 3
      echo 1 > /proc/acpi/asus/wlan
      sleep 2
      load_modules
      sleep 1
   fi
}

wifi_off() {
   if [ "$PWR" = "1" ]; then
      unload_modules

      echo 0 > /proc/acpi/asus/wlan
      sleep 1
      rmmod pciehp
      rmmod pci_hotplug
   fi
}

case "$1" in
        hibernate|suspend)
                wifi_off
                ;;
        thaw|resume)
                wifi_on
                ;;
        *)
                ;;
esac

The scripts in the “Arch acpi-eee” package provided the basis for this script, and it also works alot better than the existing scripts provided on eeeuser.com.

First of all, install User Agent Switch for Firefox and setup the iPhone user-agent:

• Description: iPhone
• User Agent: Mozilla/5.0 (iPhone; U; CPU like Mac OS X; en)
• App Name: AppleWebKit/420+ (KHTML, like Gecko)
• App Version: Version/3.0
• Platform: Mobile/1A542a Safari/419.3

Now browse to any BBC iPlayer program page and you’ll notice that it tries to serve up a Quicktime video, the MP4. As the URL isn’t displayed raw in the code, you can use a little Javascript wizardry to redirect you to the raw stream:

javascript:(function(){url = document.getElementById('mip-flash-player').getElementsByTagName("object")[0].childNodes[0].value; window.location = url;})()

Or if you want a simple drag and drop bookmarklet: iPlayer Download

The BBC will either pull the iPhone beta or re-engineer it with the iPhone SDK to develop a full client, either way this will not last long. Initally when I heard the iPhone was supported by iPlayer I was outraged, Why does a device with only around 100,000 users in the UK get priority over a operating system? It almost seems like Karma is against them, but no doubt this will get into the news as “hackers exploiting the system” rubbish. Only time will tell, enjoy it while you can.

First of all I created a bucket on S3 with the name assets.nikdoof.net, from there I setup a CNAME in BIND for assets.nikdoof.net pointing to s3.amazonaws.com to allow direct referencing to the files within the bucket.

assets.nikdoof.net.     IN      CNAME   s3.amazonaws.com.

Then for the relocation of the misc folder, setup a simple Apache mod_rewrite rule to transfer all requests for the misc folder to S3.

RewriteEngine on
RewriteRule ^/misc/(.*)$ http://assets.nikdoof.net/$1 [R,L]

So now it’s all up and working, and to give it a try yourself, here is a fetching wallpaper of Rachel Stevens.

So much for seamless updates eh?

First of all, lets start on the obvious. The N95 is Nokia’s convergence product combining 3G, Wifi, and GPS with the Series60 3rd Edition (SymbianOS 9.1) software. The N95 has been a hit with few due to the ever crowded market place (HTC smartphones, iPhone) and a few minor issues. Nokia have recently addressed the major concerns with the launch of the N95-3 which has 8gb on-board storage, better battery, and larger screen, but it could be a little late with the impending arrival of the iPhone in europe. I’ve got the N95-1 version of the phone which was the main release version for the UK.

First boot, the phone showed the traditional “holding hands” Nokia branding, date/time setup, then dropped into the front screen. For the last year or so I’ve been using a HTC Wizard running Windows Mobile and have had to deal with a fair share of awful interface design from the hands of 3rd party developers, but the only way I could describe the front screen that came with the current firmware is a abomination. Orange had decided, in it’s infinite wisdom, to replace the plain and simple Series60 front screen with a menu driven nightmare that chews RAM and makes the whole phone sluggish. I had to get rid of it. I jumped into the familiar menus and tried in vain to remove the horrid front screen, it seems orange don’t want you to remove it.

I might as well admit, I’ve voided my warranty already, in fact, I voided it on the same day. Theres a few details available on-line to change your product ID to a generic Nokia one, then using the official Nokia Software Update to re-flash the phone with a nice generic firmware. This worked a treat. It might just be my opinions, but the Orange menu was just too much of a hindrance on the phone, enough to warrant trying to get rid of it as fast as I could. This is a carrier issue, not really what everyone would experience with the phone.

Bar the misfortune of the Orange menu, the interface is slick and easy to use. The menus for the phone are a tad over-complicated but you soon get used to the location of the commonly used functions. So, i’m keeping it, flashy menus and features win me over. The battery life could do with some improvement, and hopefully Nokia will release the improved battery for N95-1 edition handsets, but I guess only time will tell.

In Widnes, theres alot of unmapped area, what has been done has been done by Chris, but only around Appleton has been done. In 3-4 days i’ve managed to do a few of my major routes. I’d highly recommend anyone with a GPS to get out there and help the project.