Categories
Stuff

TBD: Nuki Lock and Bluetooth Dev in Python

$ sudo apt install python3-venv
$ python3 -m venv doorctl
$ cd doorctl/
$ . bin/activate
$ sudo apt install pkg-config python3-dev libglib2.0-dev libboost-thread-dev libbluetooth-dev libboost-python-dev
$ pip install gattlib
$ pip install pybluez

scan for BLE devices with: sudo hcitool lescan

pybluez with examples, gattlib

Nuki Bluetooth API, forum

Python lib/tool: nukiPyBridge (does not seem to work with Lock V2 and current BT protocol?)

Categories
Stuff

Rademacher Bridge zerlegt

Es musste ein alter Gurtwickler nach Jahren harter Arbeit ersetzt werden, der Motor r├Âchelte schon nur noch ­čÖé Nachdem ich k├╝rzlich schon einen Rademacher “RolloTron Basis DuoFern 1200” angeschaut und f├╝r ok befunden habe, habe ich fix noch einen gekauft, zusammen mit der Bridge. Und was soll ich sagen, die Bridge ist f├╝r DIY tats├Ąchlich ausgesprochen gut geeignet!

Hardware

Interessanterweise ist n├Ąmlich ein Raspberry Pi Zero verbaut worden, in der Version ohne Wifi – daher kann die Bridge auch nur per Ethernet angebunden werden. Wenn unbedingt notwendig k├Ânnte man Wifi nun vermutlich dranbasteln, entweder indem man einen USB Wifi Stick verwendet, oder gleich durch Austausch des Raspi. Vielleicht ein Projekt f├╝r sp├Ąter ­čÖé

Die Elektronik auf der Zusatzplatine enth├Ąlt kaum ├ťberraschungen, im wesentlichen werkelt hier nur ein KSZ 8851 SNL f├╝r den Ethernetport, und ein NRF 905 f├╝r den Funk. (Der gro├če scheinbar leere Teil der Platine gegen├╝ber vom Raspi ist die Antenne – sehr gut diese m├Âglichst weit vom Raspi entfernt zu positionieren!)

Doch es gibt noch einen dritten Chip, dieser ist nur beschriftet mit “104GKA 742FZ00” (0en k├Ânnten auch Os sein, und 1er k├Ânnten auch gro├če I oder kleine l sein, der Font ist nicht ganz klar). Dazu finde ich keine Informationen oder Datenbl├Ątter, ich vermute dies ist ein kleiner Custom Microcontroller, welcher die DuoFern Verschl├╝sselung handhabt. Er schein jedenfalls in der Schaltung zwischen Raspi und Funkchip zu sitzen, im Detail habe ich das aber nicht untersucht.

Software

Die 16GB SD Karte habe ich nat├╝rlich an einem anderen Rechner ein wenig durchgesehen. Interessanterweise werden nur etwa 6,7GB verwendet, der Rest ist nicht einmal partitioniert. Die vier Partitionen sind:

  1. /boot, verwendet den U-Boot Bootloader
  2. Root-FS / f├╝r normalen Betrieb
  3. Root-FS / f├╝r “Recovery” – einfach ein Minimalsystem das die SD-Karte updaten kann
  4. /data, Ablage von Konfigurations- und Nutzerdaten mittels OverlayFS

Das verwendete Linux ist ein radikal abgespecktes Debian-Derivat, im Endeffekt l├Ąuft einfach nur ein nginx, ein mosquitto, und eine Reihe Java-basierte Server f├╝r die Homepilot Funktionalit├Ąt. Nicht viel anders als z.B. eine Hue Bridge.

Im Prinzip wird auch ein SSH Server (Dropbear) gestartet, doch ist dieser nicht im normalen Betrieb erreichbar, und der root Account hat eh kein Passwort. Etwas merkw├╝rdig ist dies alles aber schon, dazu evtl. sp├Ąter mehr.

Betrieb

Die Bridge holt sich ├╝ber DHCP eine IP und ist nach ein oder zwei Minuten im Netz (ausschlie├člich) per http zu erreichen. Das Web-UI zeigt zu Anfang einige Fehlermeldungen, nach der Registrierung eines Gurtwicklers verschwanden diese aber.

Zuerst dachte ich, man braucht unbedingt die Mobile App um die Bridge zu betreiben, da dies in der Schnellstart-Doku so dargestellt ist. Die Homepilot App verlangt aber eine Anmeldung mit lauter pers├Ânlichen Daten und unter Abzeichnung einer ewig langen AGB, das ist jetzt echt nicht cool von Rademacher, da mache ich nicht mit, ich will deren Cloud doch ├╝berhaupt nicht verwenden. Die Bridge funktioniert auch ohne die App problemlos.

Anmelden der Gurtwickler geht problemlos, der Zustand ist in der Web-UI zu sehen und die Ger├Ąte auch von dort zu steuern. Zu meiner ├ťberraschung und Freude stellt sich das Protokoll der Web-UI zwischen Browser und Server als einfache REST-API ohne Verschl├╝sselung oder andere Komplikationen dar. Sehr gut! \o/

Abfrage des Zustands aller Ger├Ąte, die aktuelle Position des Rolladens ist in “statusesMap.Position”:

GET http://192.168.1.100/v4/devices?devtype=Actuator
->
{"response":"get_visible_devices","devices":[{"description":"Ihre Ger├Ątebeschreibung","deviceGroup":2,"did":1,"hasErrors":0,"iconSetInverted":0,"iconSet":{"k":"iconset15"},"messages":[],"name":"Gurtwickler","properties":{"closingContact":3,"dawn":3,"dusk":3,"motion":3,"rain":3,"smartphone":3,"smoke":3,"sun":3,"temperature":3,"time":3,"trigger":3,"warning":3,"wind":3},"statusValid":true,"statusesMap":{"Manuellbetrieb":0,"Position":100},"visible":true,"deviceNumber":"12345","uid":"12345_1","voiceControlledBy":"Alexa,Google","origin":"HomePilot"}]}

Um einen Gurtwickler z.B. hoch zu fahren wird dieser Befehl verwendet:

{"name":"POS_UP_CMD"}
->
PUT http://192.168.1.100/devices/1

Das l├Ąsst sich alles gut in den Chrome Developer Tools nachvollziehen. Die Web-UI pollt dabei alle 5 Sekunden den Zustand aller Ger├Ąte.

Automatisierung mit Node-Red

Ich bin seit einiger Zeit dabei meine Heimautomatisierung auf Node-Red umzustellen. Aufgrund der einfachen API ist das Einbinden der DuoFern Bridge einfach, z.B.:

Der HTTP Request ist schlicht ein GET auf die o.g. URL, dann wird payload.devices extrahiert, und nach dem Split topic auf die JSONata Expression "node-red/state/shutter/" & payload.name und payload auf payload.statusesMap.Position gesetzt.

Categories
Stuff

Kleine Analyse eines Gurtwicklers

Ich w├╝rde gerne ein (oder vielleicht mehrere) manuelle (Gurt-)Jalousien mit der Heimautomatisierung verbinden. Leider scheint es nichts zu geben das direkt den DIY-Markt anspricht. Ich finde nur irgendwelche geschlossenen “L├Âsungen” wie Rademacher’s DuoFern, oder Leute die einfach Kabel an die Kn├Âpfe eines Gurtwicklers l├Âten. Ich will aber eigentlich nicht noch irgendeine Bridge mit zweifelhafter Zukunft und mehr oder (mehr wahrscheinlich) weniger Funktionen haben.

Also habe ich mir mal ein RolloTron 1200 bestellt, dieser hat DuoFern als Funkl├Âsung eingebaut. Meine Hoffnung war entweder einen “alternativen Zugang” zur Elektronik zu finden, oder zumindest mehr ├╝ber DuoFern zu lernen. Vielleicht ist ja einfach das Gefunke als diskrete kleine Schaltung “drangeflascht” und austauschbar?

Das erhaltene Ger├Ąt nat├╝rlich gleich auseinandergebaut ­čÖé Der innere Aufbau ist recht einfach und die Haupt-Steuerplatine leicht zu entnehmen, hier perspektivkorrigierte Fotos von beiden Seiten (R├╝ckseite spiegelverkehrt!) und ├╝bereinandergelegt, um Durchkontaktierungen besser zu erkennen:

Es f├Ąllt direkt auf das mehrere Bauelemente nicht best├╝ckt sind – vermutlich wird dieselbe Platine f├╝r mehrere verschiedene Ger├Ąte benutzt und nur dann entsprechend best├╝ckt. Ich habe im Netz ein Foto der Platine vom Rollotron 1100 gefunden (selbes Ger├Ąt wie meins, nur ohne DuoFern) und dort sind die beiden Chips ganz oben nicht best├╝ckt, daf├╝r statt dessen der untere Chip in der Mitte. Der obere unbest├╝ckte Chip d├╝rfte f├╝r eine Version mit Display sein, daf├╝r dann auch die horizontale Reihe Kontakte und die Einbuchtungen an den Seiten.

Schaltungsbeschreibung

  • Der unterste “Chip” mit den 2×4 L├Âchern durch die Platine hindurch ist die Kontaktierung f├╝r die Netzteil- und Motorplatine.
    • Jeweils ein Pin f├╝r 10V und Masse vom Netzteil.
    • Ein Pin f├╝hr ein 50Hz Signal bei etwa 4V vom Netzteil, ohne Zweifel aus dem 230V Netz abgeleitet. Ich vermute dies wird verwendet um ├╝ber lange Zeit die innere Uhr des Mikrocontrollers zu stabilisieren, damit bei einer Zeitsteuerung jeden Tag zur gleichen Zeit die Jalousie verf├Ąhrt, und dieser Zeitpunkt nicht “wegwandert”.
    • Zwei Pins f├╝r eine DC-Motor Steuerung: Ein Pin f├╝r die Richtung, einer mit PWM Signal zur Geschwindigkeit.
    • Drei Pins mit irgendeiner Art Steuer- oder Datensignal – ich bin mir nicht sicher was genau. Alle drei Signale zeigen regelm├Ą├čige Ausschl├Ąge mit 62.5Hz, wobei der Ausgangspegel von der Motor-Drehrichtung abh├Ąngt. An der Motorachse sitzt ein Magnet ├╝ber irgendeinem “Chip” (nicht zu erkennen), vielleicht sind diese drei Signale ein Feedback um die Drehrichtung zu ├╝berpr├╝fen und ggf. ein Blockieren des Motors zu erkennen?
  • Links neben dem unteren Schalter ist ein 78L33A Spannungsregler f├╝r 3.3V.
  • Der gro├če Chip unterhalb des oberen Schalters ist ein R5F 100, das ist ein 16bit Microcontroller von Renesas aus der RL78 Serie, Datenblatt hier. (Nicht das ich Lust h├Ątte einen gro├čen Haufen Maschinencode zu entziffern, aber soweit eine kurze Recherche im Netz ergab erlaubt dieser Microcontroller sowieso kein Auslesen des Programm-Flash.)
  • Der kleinere Chip schr├Ąg daneben ist ein NRF 905: 433/868/915MHz Transceiver. (Datenblatt) Der Kleinkram links daneben ist die Antennenabstimmung, die Antenne selber ist die einzelne einsame Leiterbahn ganz am linken Rand. Gut zu erkennen der gro├če Abstand zur restlichen Elektronik, “weil Funk” ­čÖé Interessanterweise ist die Antenne nur etwa 11-12 cm lang – bei den verwendeten 434.5 MHz w├Ąre etwa 17.2 cm (Lambda/4) eigentlich besser, aber passt vermutlich einfach nicht ins Ger├Ąt.
  • Microcontroller und Funkchip sind per SPI-Bus und ein paar weiterer Steuerleitungen verbunden. Soweit ich das sehe kann hat der Transceiver keine “h├Âheren” Features, das Funkprotokoll und evtl. Verschl├╝sselung muss also im Mikrocontroller erfolgen. Man k├Ânnte ├╝berlegen den SPI-Bus abzuh├Âhren, aber das w├╝rde im wesentlichen auch nur dieselben Daten zeigen wie das was ├╝ber den Funk geht.
  • Am oberen Platinenrand ist ein Reed-Kontakt, dieser wird durch einen Magneten in der Gurtrolle getriggert. Ich vermute dies ist Teil der Blockiererkennung – wenn die Jalousie sich nicht weiter abw├Ąrts bewegt wird der Gurt locker und die Rolle dreht sich nicht mehr.
  • Am rechten Rand sind drei nummerierte und zum Teil best├╝ckte Wiederst├Ąnde, diese sind elektrisch nicht mit der restlichen Platine verbunden und nur ├╝ber die Kontaktfl├Ąchen auf der R├╝ckseite erreichbar. ├ťberhaupt hat die R├╝ckseite eine erstaunlich gro├če Anzahl Kontaktfl├Ąchen. Die drei “Code-Wiederst├Ąnde” d├╝rften es erlauben in Testst├Ąnden die jeweils best├╝ckte Platinenversion abzufragen und automatisch z.B. die Programmierung an die jeweils best├╝ckten Chips anzupassen.

Fazit

So wie ich mich das vorgestellt habe wird das nichts – bei diesem Ger├Ąt m├╝sste man den Microcontroller komplett umprogrammieren oder die Elektronik austauschen um ein anderes Funkprotokoll zu bekommen. Und einfach nur Dr├Ąhte an die auf/ab-Kn├Âpfe anzul├Âten mag ich nicht, dann gibt es keine R├╝ckmeldung ├╝ber den aktuellen Zustand der Jalousie.

Categories
Stuff

Raspberry Pi Kiosk Mode

For home automation or such. Directly boot into a graphical screen and allow user interaction through a touchscreen.

Setup 1: Living Room Control Panel

Intention is that this is “off” by default, turns on when the screen is touched, then allows interaction with a browser, and after some time of no use turns off again.

The method I’m using follows what’s lined out here, or on various other pages. Basically configure the raspi to autologin with graphical UI, then configure the user’s X session in ~/.config/lxsession/LXDE/autostart like this:

@xscreensaver
@/usr/bin/chromium-browser --check-for-update-interval=31536000 --noerrdialogs --disable-features=TranslateUI --disable-infobars --incognito --kiosk http://192.168.5.73:1880/ui/

Using xscreensaver because here it’s trivial to include in the LXDE autostart. There’s also an .xscreensaver file for a simple blank screensaver:

timeout: 0:01:00
lock: False
splash: False
dpmsEnabled: True
dpmsQuickOff: True
mode: blank

To make the boot slightly faster and avoiding extra graphics the /boot/cmdline.txt includes logo.nologo and /boot/config.txt includes disable_splash=1

I should consider a regular reboot, but this is running stable enough not to require that. If I remember correctly I also struggled a lot with getting Chromium to properly output sound to a USB-connected soundcard/speaker, but I don’t remember what I did to get it to work – I think it was about running pavucontrol while Chromium was playing some sound, and then fixing devices/volumes in there. Side note: Chromium does not include Google’s Text-to-Speech system, and I could not get some system-side thing (e.g. espeak) to work.

Setup 2: Device Controller

I’m working on a little something that I’ll share more about later. This “needs” (*ahem*) a Raspberry Pi with a small touchscreen as a local interactive controller. I have an (old) Watterott 2.8″ touch display I want to use on a Raspberry 3a.

The setup needs to be a little different than the one described above. This will run a fullscreen Kivy app and the screensaver should be steered by this app and not happen automatically. The system might even be shut down between it’s use, I’ll see about that later.

The basic Kivy installation is described here.

I’m using chilipie-kiosk as the concept for the basic setup, steps:

  • raspi-config: System -> Boot -> Console Autologin for user kapet (that should be automatically the case if sudo from that user).
  • Couple of small tweak/optimizations to make boot faster/cleaner:
    • Empty out /etc/motd, so it’s not shown when user is logged in.
    • Add disable_splash=1 to /boot/config.txt (not 100% sure this helps but it doesn’t hurt either…)
    • Make sure splash screen at boot is disabled, raspi-config: System -> Splash at boot
    • Add After=getty.target in the [Unit] section of /etc/systemd/system/raspi2fb@.service to show fewer of the boot msgs on the TFT (and maybe boot a tiny bit quicker).
  • And remember to enable the GL (not Legacy!) driver in raspi-config or Kivy will be nasty slow. “Fake KMS” GL driver works well on my Raspberry Pi 3a, but unfortunately X11 gets confused of the screen resolution and needs a manual hint by running xrandr in the .xsession file.

Add this at the top of ~/.profile to start X when auto-logged’in:

if [ "$(tty)" == "/dev/tty1" ]; then
    exec startx >/dev/null 2>&1
fi

Create an ~/.xsession file for X setup and starting the app:

#!/bin/bash

export DISPLAY=:0.0

# X11 gets the resolution wrong with the GL driver
xrandr --output HDMI-1 --mode 320x240

# disable automatic screensaver, app will handle this
xset s off
xset -dpms
xset s noblank

cd /home/kapet/ergo
bin/python3 ergo.py >stdout.log 2>stderr.log &

# a window manager is required of VNC doesn't work properly
exec matchbox-window-manager -use_titlebar no
Categories
Stuff

Kivy on Raspberry Pi with a Touchscreen

This took way too long to get right ­čśó Really all I want is writing a simple graphical fullscreen UI in Python that runs on the small touchscreen attached to it.

Fundamental Problems:

  • The touchscreen is connected through the GPIO port and not through the GPU. This means access is through the kernel FB driver only and there is no hardware acceleration of any kind.
  • SDL on Raspbian has not been compiled with proper touch-device support, specifically tslib is not enabled. This breaks pygame in interesting ways (touch input is all over the place wrong) and requires a custom built SDL.
  • SDL also has not been compiled with the kmsdrm backend, which apparently breaks Kivy when run on the console.
  • As far as I can tell Kivy requires GL support of some kind, which is only provided by the GPU and not by the FB device anyway – in other words – Kivy can not output directly to the touchscreen.
  • I’ve played with Kivy before and found it somewhat frustrating – there’s so much magic going on in the background that I found very difficult to grok. Also when things go wrong, error messages are not helpful. (For example, during console testing I had set some SDL environment vars. They were still set when I tried running Kivy under x11 and caused fascinating but completely misleading errors that had me look in all sorts of wrong places for hours… ­čśĺ ) Still, Kivy seems to be the best of breed if I don’t want to hack basic fundamental UI things by hand.
  • My first thought was to run the UI straight on the console because it’s “simpler”, but I eventually realized that because of all the SDL and framebuffer snafu it would require recompiling a bunch. (Which is something I don’t like – I prefer my automation hosts to have rather pristine system setups with as much customization limited to the apps as possible.) And there’s also another issue – I can’t reasonably test console apps on my other systems. So running the apps under X11 would have the advantage of allowing to VNC into the machine for remote testing at least.

Approach Taken:

  • Basic enablement of touchscreen through /boot/config.txt:
    dtoverlay=rpi-display,speed=32000000,rotate=270
  • Use the builtin GPU to get full hardware acceleration support (whatever little the Raspberry Pi really has…) and render video output without a HDMI device connected. For this force HDMI resolution to the same as the touchscreen by adding this to /boot/config.txt:
    hdmi_force_hotplug=1
    hdmi_group=2
    hdmi_mode=87
    hdmi_cvt=320 240 60 1 0 0 0
  • Run raspi2fb as a service under systemd to constantly copy the output of the GPU onto the touchscreen FB device.
    • I also experimented with rpi-fbcp. The code is much simpler but by default runs at ~40hz or so update rate, which eats a lot of CPU time. It has fewer options and no systemd file either.
    • I looked at fbcp-ili9341 too, which is incredibly impressive with a lot of clever tricks to speed up the copy process, achieving incredible FPS and apparently allowing fluent games even. But the speed comes at the cost of missing touch support – that’s a no-go for me.
  • Configure X input for touchscreen, in (new) directory /etc/X11/xorg.conf.d/ create file 99-ads7846-cal.conf. (This is the “270┬░ rotation” setup from here.)
Section "InputClass"
    Identifier "calibration"
    MatchProduct "ADS7846 Touchscreen"
    Option "EmulateThirdButton" "1"
    Option "EmulateThirdButtonButton" "3"
    Option "EmulateThirdButtonTimeout" "1500"
    Option "EmulateThirdButtonMoveThreshold" "30"
    Option "InvertX" "0"
    Option "InvertY" "1"
    Option "SwapAxes" "1"
    Option "TransformationMatrix" "0 1 0 -1 0 1 0 0 1"
EndSection
  • Configure system using raspi-config:
    • System -> Boot -> Desktop autologin
    • Interface -> VNC -> enable
    • Performance -> GPU Memory -> 64M (or something, just not the smallest amount or X won’t start)
    • Advanced -> GL Driver -> GL with Fake KMS

Now after rebooting the touchscreen shows a (tiny) X11 desktop and it should be possible to connect through VNC and see the same. Next step is installing Kivy.

  • Confusingly some Kivy dependencies that need to be installed are only listed after the rest of the process is described, making them easy to miss. ­čśĺ Look them up first, for Kivy2 they’re here right now. Install big list for Buster first, then small list for “Desktop environment”.
  • Now back to the normal Kivy installation. No need to install and use virtualenv, the Python3 venv works fine.
  • Testing with e.g. share/kivy-examples/demo/touchtracer/main.py should work now, again both on the actual touchscreen as well as over VNC, here is the essential bits of the Kivy log:
[INFO ] [Image ] Providers: img_tex, img_dds, img_sdl2, img_pil (img_ffpyplayer ignored)
[INFO ] [Text ] Provider: sdl2(['text_pango'] ignored)
[INFO ] [Window ] Provider: sdl2(['window_egl_rpi'] ignored)
[INFO ] [GL ] Using the "OpenGL" graphics system
[INFO ] [GL ] Backend used
[INFO ] [GL ] OpenGL version
[INFO ] [GL ] OpenGL vendor
[INFO ] [GL ] OpenGL renderer
[INFO ] [GL ] OpenGL parsed version: 3, 1
[INFO ] [GL ] Shading version
[INFO ] [GL ] Texture max size <8192>
[INFO ] [GL ] Texture max units <32>
[INFO ] [Window ] auto add sdl2 input provider
[INFO ] [Window ] virtual keyboard not allowed, single mode, not docked
[INFO ] [GL ] NPOT texture support is available
[INFO ] [ProbeSysfs ] device match: /dev/input/event0
[INFO ] [HIDInput ] Read event from
[INFO ] [Base ] Start application main loop
[INFO ] [HIDMotionEvent] using
[INFO ] [HIDMotionEvent] range ABS X position is 0 - 4095
[INFO ] [HIDMotionEvent] range ABS Y position is 0 - 4095
[INFO ] [HIDMotionEvent] range ABS pressure is 0 - 255

Important to check that it is using sdl2 and the touch input is properly detected.

Followup: setting up kiosk mode

Categories
Stuff

Kleine Spielzeugreparatur

Meine Kinder haben eine “Exost Loop” Autorennbahn, doch leider funktionierte jetzt eins der Autos nicht mehr – am Controller konnte man dr├╝cken wie man wollte, das Auto bewegte sich nicht. ­čÖü

Stellt sich heraus, bei einer der zwei IR-Leuchtdioden ist ein L├Âtpunkt samt Leiterbahn von der Platine abgerissen. Im Mikroskop gut zu erkennen, mit blo├čem Auge kaum. Ich vermute, da ist mal jemand auf die Fernbedienung drauf getreten…

Etwas L├Âtlack abgekratzt, und die defekte Stelle mit einem St├╝ck Draht ├╝berbr├╝ckt.

Es funktioniert wieder! (Aufnahme mit der Kamera des Handys; der Sensor der meisten Digitalkameras kann das IR-Licht der ├╝blichen Fernbedienungen darstellen.)

Categories
Stuff

Raspberry Pi Setup

A few notes on how I set up new raspi systems, of which I am using quite a few for random home automation or toy things.

Prepping the SD card

  • Install “Lite” version of Raspberry Pi OS
  • This should have created one FAT formatted partition that Windows/Mac can access, this is the boot partition
  • Create file “ssh.txt” on boot partition to enable ssh right away, otherwise need to attach a screen for first login
  • If newer raspi and wifi should be used, create file “wpa_supplicant.conf” on boot partition (docs)
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1
country=<Insert 2 letter ISO 3166-1 country code here>

network={
 ssid="<Name of your wireless LAN>"
 psk="<Password for your wireless LAN>"
}

Initial Setup

  • Startup raspi, find IP in DHCP server or try direct ssh to host “raspberrypi” and cross fingers that something somewhere worked right (I think Unifi DHCP server automatically using DHCP provided hostname for DNS? Or is this some MDNS magic? No idea, never looked into it.)
  • Log in with username “pi”, default password “raspberry”
  • sudo raspi-config
    • set system-> hostname
    • set localization -> timezone
    • exit without reboot, it’s not necessary now
  • sudo su -
    • adduser kapet
    • adduser kapet adm
    • adduser kapet sudo
      • and more groups as necessary: dialout, cdrom, audio, video, plugdev, games, input, netdev, spi, i2c, gpio
    • Copy /etc/sudoers.d/010_pi-nopasswd for kapet and update it accordingly
    • reboot
  • Log in as “kapet”
  • sudo su -
    • disable login to the pi account: usermod -e1 -L pi
    • enable longrunning and at-boot started scripts for kapet: loginctl enable-linger kapet
    • apt update
    • apt dist-upgrade
    • apt install lsof tcpdump vim-nox
    • set the NTP server in /etc/systemd/timesyncd.conf
    • reboot

Other Potential Setup

  • set up basic X11: apt install raspberrypi-ui-mods
  • disable bluetooth on pi3: add dtoverlay=pi3-disable-bt to /boot/config.txt
  • when using any GPU to framebuffer magic like raspi2fb for a local LCD display remember to give the GPU enough memory through raspi-config, the minimum amount is not sufficient
  • setup Watterott display: docs

Categories
Stuff

A Surprisingly Useful Tool

20x magnifying stereo microscope, with a common ruler in its fangs.

Based on a recommendation from a coworker I bought a simple stereo microscope for around 30 Euro a while ago. I believe the manufacturers idea is that kids look at insects or little rocks with this (it actually comes with a few tiny splinters of different rocks), but this thing turned out to be extremely useful for several of the electronics stuff I’ve done lately.

For example I’ve used it to check my work after etching a PCB or soldering it. The magnification of 20x turned out to be perfect – it’s significantly better than my eyes but still “real” enough that I can interact with the object using small tools or a soldering bit.

I found having two occulars is not magically improving what I can see, but it certainly makes it less stressful to look through it for a long time. The magnified image shows about 8mm of object width.


In a recent case I was struggling with an Arduino Nano, which was connected to a Raspberry Pi, and just did not want to work if the host rebooted while it was plugged in the USB port. All I got were errors like this in the logfile:

May 23 20:32:28 raspberrypi kernel: [    3.707998] usb 1-1.3: new full-speed USB device number 4 using dwc_otg
May 23 20:32:28 raspberrypi kernel: [    3.807985] usb 1-1.3: device descriptor read/64, error -32
May 23 20:32:28 raspberrypi kernel: [    4.028021] usb 1-1.3: device descriptor read/64, error -32
May 23 20:32:28 raspberrypi kernel: [    4.257989] usb 1-1.3: new full-speed USB device number 5 using dwc_otg         
May 23 20:32:28 raspberrypi kernel: [    4.357989] usb 1-1.3: device descriptor read/64, error -32
May 23 20:32:28 raspberrypi kernel: [    4.577988] usb 1-1.3: device descriptor read/64, error -32
May 23 20:32:28 raspberrypi kernel: [    4.698153] usb 1-1-port3: attempt power cycle
May 23 20:32:28 raspberrypi kernel: [    5.358060] usb 1-1.3: new full-speed USB device number 6 using dwc_otg
May 23 20:32:28 raspberrypi kernel: [    5.818026] usb 1-1.3: device not accepting address 6, error -32
May 23 20:32:28 raspberrypi kernel: [    5.918003] usb 1-1.3: new full-speed USB device number 7 using dwc_otg
May 23 20:32:28 raspberrypi kernel: [    6.357982] usb 1-1.3: device not accepting address 7, error -32
May 23 20:32:28 raspberrypi kernel: [    6.358192] usb 1-1-port3: unable to enumerate USB device

Turns out the problem is not with the USB driver in the Linux kernel, but instead it’s a hardware bug in the Nano: The FTDI serial chip’s “TEST” pin was left unconnected. This way it “floats” after a power cycle and reliably prevented prober USB device enumerating. Ugh. I realized this is the problem after a lengthy search and finally stumbling over this issue in the FHEM wiki.

So here it goes, on the bottom side of the Arduino Nano I need to connect two freaking small SMD pins with each other, so TEST is reliably pulled to ground. I can hardly see those pins properly!

Perfect situation to use the microscope: Put a tiny blob of solder on the very tip of the soldering iron, veeeery slowly direct it to the right pins, and done!

With this the Nano gets connected properly whenever then Raspberry Pi reboots. ­čÖé

Microscope set up to allow viewing the bottom side of the Arduino Nano.
Solder blob connecting TEST and AGND pins.

Categories
Stuff

Hello Noteblog

Started this Blog to have a place to dump random notes that other people might find useful and so I can find it again later. ­čÖé

Categories
Stuff

Kerberos, AFS and Batch Systems

(This is an overview I wrote in 2004. I would suspect that a lot of the technology is no longer relevant, but actually this article got still occasional hits on my old homepage so I’m migrating it just in case it’s still useful for some people.)

Introduction

Kerberos is a system for securely authenticating users in an unsecure network environment. It was developed in the 1980s at the MIT as part of the famous project Athena. During the 1990s Kerberos V5 was standardized in RFC 1510 and became widely used, especially after Microsoft decided to base Windows 2000 security on it. Within Kerberos, each user has a Ticket Granting Ticket (TGT) which can be used to acquire dedicated service-tickets. These service-tickets finally are used to authenticate a user to that service.

AFS (the “Andrew File System”) was started at the Carnegie-Mellon University as a research project. By using a slightly modified Kerberos V4 system they built a secure network filesystem which allows several data-storing servers and complex access control lists. Today AFS is used all over the world, especially by Universities and other research institutions. From the Kerberos point of view, AFS is just a service. AFS itself requires the user to own a valid Kerberos ticket for the service “AFS”, this is often called the “AFS-Token”.

A Batch System is needed for controlling resource usage of special machines like a supercomputer or some compute servers. Users just define what their jobs need (e.g. the number of CPUs), the Batch System decides if and when the user will get these resources. It then takes care of starting (and probably killing) the job, finally some accounting information about the job is saved for accounting or statistics. There are several commercial and non-commercial Batch Systems available these days, two important open source systems are Torque/OpenPBS and Sun’s Grid Engine.

This article is about the problems which arise, when these technologies are used together: Imagine a user submits a job which uses files from his home-directory in AFS. This requires the Batch System to make sure that the job has the users AFS-Token while running or the job would not be able to access the files. Making these files readable by anyone would allow the Batch System to not care about AFS, but this is not an option because of security considerations. Another problem is the limited lifetime of the AFS-Token, it has to be prolonged or renewed to allow the job access AFS continuously.

More information:


Kaserver / Kerberos V4 and AFS-Tokens

The Kaserver is a modified Kerberos V4 server used by the original AFS setup. It has some special features and speaks an additional network protocol (“Rx”) used only by AFS utilities. Its possible to use a Kerberos V4 server instead of the Kaserver (Using MIT’s Kerberos Server with AFS). The normal lifetime of an AFS-Token is configured per user on the server, usually its about 25 hours. The Token is isolated against the rest of the system in a Process Authentication Group (PAG) called structure which is identified by two unique group IDs.

Renewal of an AFS-Token

One possibility to provide a long lasting job with a token is to regain the token from time to time. The process doing this usually needs some knowledge about the users password though, either by asking him during job submission or by requiring him to store it somewhere.

One tool for this task is the “Password Storage and Retrieval System” (PSR), which uses asymmetric cryptography to securely store the users password encrypted in his AFS space. When a job needs to acquire a new token, the password gets decrypted and is used to simply request a new token. The secret key needed to decrypt the password is only stored on the machine that runs the job. If the user changes his password and updates the encrypted storage too, the new password is automatically used on the next renewal.

Another way to store the password is to use a “SRVTAB” file. Such a file is normally used to store a server key but it can also be used to store the key of a user. The stored key is not the plaintext password but some kind of hash. This way the password is not revealed, but be aware: Concerning Kerberos, the hash can be used just like the password. So when a job needs to acquire a new token, the hash can simply be used. You can find a description of this technology here: UMich: “How to run long lived jobs with AFS” Some quick hints to be used with KTH Kerberos V4: Create the SRVTAB like this: ksrvutil -f mysrvtab -c example.com add where example.com is the name of your AFS cell. Enter your username when prompted for “Name:”, the name of your AFS cell in uppercase for “Realm:” and just press enter for “Instance:” and “Version Number:”. It will then ask a password twice, enter your normal AFS password. The created file “mysrvtab” can be used like this: kauth -n myname -f mysrvtab bash where myname is your
username. kauth will run the given command (here: bash) and repeatedly renew the AFS-Token by using the secret from mysrvtab.

Prolongation of an AFS-Token

A completely different approach to extending the lifetime of AFS-Tokens is to prolong them, extend their lifetime without acquiring a new one.

To do this one has to extract the Token from the current environment and decrypt it with the AFS specific Kerberos service key (known only by the Kerberos server and the AFS fileservers). Its now possible to put a new timestamp into the Token, thereby extending its lifetime. After encrypting it with the service key again and putting it back into the users environment, the user has a Token with an extended lifetime. If this process is repeated regularly, the Token never expires.

To my knowledge this way was first gone at CERN in the first half of
the 1990s, they created the programs GetToken, SetToken and forge. These programs became the base of CERN’s “Authenticated Remote Control” (ARC) system, and some time later Codine and LoadLeveler evolved with support for these tools. (Codine is today known as Sun GridEngine, which still contains support for this method.)

A reimplementation of this method can be found in Mike Bechers OpenPBS module extension.

I did just another reimplementation, which does not rely on OpenAFS but uses only Kerberos and “krbafs”, found on any Fedora Core 1 machine.

Kerberos V5 and AFS-Tokens

Kerberos V5 brought some new features, renewable TGT-tickets being the most notable with regard to this article. Such a ticket can be renewed via kinit -R without the need to enter the password again. During a renewal, the kinit command contacts the Kerberos server and asks if the renewal is acceptable. This makes it possible to inhibit further usage of a stolen TGT by e.g. disabling the account on the server. As another security measure, a renewable ticket not only contains the usual (short) lifetime specification, but also features a (long) “renewable lifetime” that declares an upper limit for ticket renewal. Usually the normal lifetime is about a day, while the renewable lifetime can last for months.

Creating AFS-Tokens out of Thin Air

A completely different way to provide jobs with AFS-Tokens is to fake them. This is easy if you know the service key of the AFS service. (Remember: AFS is a Kerberos service, the AFS-Token is just a Kerberos V4 service ticket. Therefore it has a well known structure and is encrypted with the AFS’ service key.) An implementation of this method is available as GSSKLOG, a tool which uses the GSS-API to authenticate a client to the server, eventually giving back a faked AFS-Token if authentication succeeded.


Batch Systems: Required Steps for AFS-Integration

A complete solution would be to include full Kerberos and AFS support into the Batch System. But this would probably require considerable changes in network communication and internal structures, so some simpler way would be better.

When a job is submitted (e.g. qsub)

  • Kerberos V4 with Renewal: Ask the user for his password or check for some prepared storage. Probably attach some information on it to the job.
  • Kerberos V4 with Prolongation: Extract the users Token from the current PAG on the submit host and attach it to the job.
  • Kerberos V5: Forward the TGT along with the job.
  • Fake Tokens: The Batch System must become 100% sure about the users identity.

While the job is queued

  • Kerberos V4: Do nothing.
  • Kerberos V5: Renew the TGT repeatedly.
  • Fake Tokens: Do nothing.

When the job is started

  • Kerberos V4 with Renewal: Instantiate a PAG. Let Kerberos create a new TGT and a new AFS-Token.
  • Kerberos V4 with Prolongation: Instantiate a PAG. Prolong the AFS-Token and insert it into the PAG.
  • Kerberos V5: Instantiate a PAG. Let Kerberos create an AFS-Token (based on the TGT).
  • Fake Tokens: Instantiate a PAG. Create a fake token and insert it into the PAG.

While the job is running

  • Kerberos V4 with Renewal: Create a new TGT and a new AFS-Token repeatedly inside the PAG.
  • Kerberos V4 with Prolongation: Repeatedly prolong the AFS-Token and insert it into the PAG.
  • Kerberos V5: Renew the TGT repeatedly inside the PAG, let Kerberos create an AFS-Token each time.
  • Fake Tokens: Repeatedly create a new fake token and insert it into the PAG.

When the job has finished

  • Kerberos V4: Destroy all local knowledge about the users password and close the PAG.
  • Kerberos V5: Close the PAG.
  • Fake Tokens: Close the PAG.

To sum it up, a Batch System that wishes to support AFS has – at least be able to:

  • call an external program on the submitting host when a job is submitted
  • call an external program regularly while a job is queued
  • allow to start processes by using an external program (a PAG-shell)
  • run an external “shepher” program together with the users process OR call an external program regularly from an own “shepher” process
  • pass some information along with the job and make this information available to the external programs

Not all described mechanisms need all these hooks though.


Links

  • MIT Longjobs: A patched version of OpenPBS which includes some Kerberos support to allow Kerberos authentication and long running job which access AFS.
  • Kerberos on Wall Street: Paper about the Kerberos V4 to V5 migration of Morgan Stanley, includes some remarks about AFS.
  • Globus and AFS

(C)opyright by Karsten Petersen <kapet@hrz.tu-chemnitz.de> in 2004