When .local Stops Being Local

I have a Raspberry Pi server called bigbrain. Normally I can reach this machine on my local network using:

bigbrain.local

This works using mDNS (Multicast DNS). Addresses ending with .local are handled specially. Rather than asking an outside world name server to resolve a name to an address, when a device sees an address ending .local it will use a local service that listens for devices that broadcast addressing information on the local network. On a Raspberry Pi, mDNS is normally provided by a service called Avahi. Avahi announces the name of the machine on the local network so that other devices can find it without needing an entry in a conventional DNS server.

This morning I noticed that bigbrain.local didn’t work any more. This was a bit worrying, it’s the machine that hosts the custard cream server, among other things. I logged in using the Raspberry Pi Connect service and was pleased to discover that the machine was still working. The first thing to I needed to check was whether or not the Avahi service was running. I opened a terminal window and typed:

systemctl status avahi-daemon

Avahi was running, but the output contained an important clue:

Host name conflict, retrying with bigbrain-2

and:

Server startup complete. Host name is bigbrain-2.local.

This explained why bigbrain.local wasn't working. Avahi thought that another machine on the network was already using that name. Rather than create a conflict, it had renamed its mDNS identity to:

bigbrain-2.local

Was there another Bigbrain?

My first thought was that I might have booted another Raspberry Pi containing an old copy of the bigbrain system. If two machines both have the hostname bigbrain, they will both attempt to advertise themselves as bigbrain.local.

But I wasn't certain that I'd actually done this, which raised a slightly less comfortable question: was an unknown device appearing on my network?

Fortunately, the system log provided some more information.

The conflict had occurred at 02:55 on 22 September:

Sep 22 02:55:17 bigbrain avahi-daemon[585]:
Host name conflict, retrying with bigbrain-2

Looking at the Avahi journal around this time showed something interesting. For several minutes before the conflict, Avahi had been repeatedly registering and withdrawing network addresses.

At 02:55:17 it withdrew the addresses associated with both network interfaces and then registered them again. Five seconds later NetworkManager reported:

dhcp4 (eth0): state changed new lease, address=192.168.7.107

So something was definitely happening to the network at the same time that Avahi decided there was another bigbrain.

The missing piece

I use a system called eero to provide Wi-Fi access at home. It turned out that the eero network had automatically updated its software earlier that morning. The update caused the network infrastructure to restart. At 02:19 on 22nd September. That provided a much more convincing explanation.

As the network came back, DHCP leases, IPv6 addresses, multicast groups and mDNS advertisements all had to be re-established. The Avahi logs showed exactly this kind of activity.

There was another complication. bigbrain was connected to the same local network in two different ways:

              bigbrain
             /        \
          WiFi       Ethernet
           |             |
    192.168.7.22   192.168.7.107
           \             /
            local network

Both interfaces were therefore participating in mDNS on the same network and both represented the same machine with the same hostname.

Normally this works. But during the disruption caused by the network restart, Avahi apparently saw enough conflicting information to conclude that somebody else was already advertising bigbrain.local.

It did the safe thing and changed its advertised name to bigbrain-2.local.

Once the network had settled, restarting Avahi:

sudo systemctl restart avahi-daemon

allowed it to register bigbrain.local normally again.There was no evidence here of an unknown machine or a security compromise (which was nice). The hostname conflict occurred during a period of substantial network reconfiguration following an eero software update. But the incident did expose an unnecessary complication in the server configuration.

The Raspberry Pi was connected to the same LAN using both Ethernet and Wi-Fi.

There are situations where having multiple network interfaces is useful, but a fixed server generally isn't one of them. Two connections to the same network mean two IP addresses, two routes onto the LAN and two interfaces participating in multicast protocols such as mDNS. Most of the time the networking software sorts everything out. When the network is being restarted or reconfigured, however, things can get more interesting.

For a Raspberry Pi being used as a server, the simplest rule is therefore:

If you have a wired Ethernet connection, use Ethernet and turn WiFi off. If you need WiFi, use WiFi and leave Ethernet disconnected. Don't routinely use both to connect the server to the same local network.

I’m going to disable the Wi-Fi on bigbrain. Networking is complicated enough without giving the packets a choice of doors.

Friday Widescreen

Friday finds us at a presentation at the National Science and Media Museum in Bradford. It was all about widescreen movies. Back in the days when movies were things you went to, not things you made, a plucky band of souls formed the Amateur Widescreen Association. Impressed by the widescreen movies of the day they tried to create the same experience using films and equipment available for home use. At the time cameras and projectors were entirely mechanical, so the task was to use different lenses, film gate shapes and film sizes make the biggest and most impressive wide screen effect. Every year they held an event where they all gathered to show off their latest creations. They built some amazing hardware, including one presentation that used 3 synchronised projectors. Each year they had a competition to pick the best films, and we got to see a bunch of test films, along with some of the winners. It was great fun.

Of course, nowadays widescreen is an option you click on your phone camera and image quality is just something you take for granted. But the presentation showed that with determination and ingenuity you can make tiny negatives really sing on the big screen. Some of the movies were projected as film, in one case it was the original celluloid that went to Afghanistan, was exposed and mixed there in camera and then brought back for development. Lots of material ends up as being of historical interest, as we saw scenes of places that no longer exist. Thanks to Guy Edmonds and the rest of the team for doing such a great job of bringing the past back to life.

The presentation was part of a larger Widescreen Weekend in Bradford which runs once a year. There are lots of movies on show. Maybe next year I’ll sign up and spend an entire weekend eating popcorn…

Hardware Meetup with added Claude

This is the device. Check out that welcome deal

I took everything with me to the Hardware Meetup tonight. Everything, that is, except the Raspberry Pi PICO I was supposed to be working on. Ross took pity on me and lent me an ESP32 powered display that he had brought along to play with. A while ago I’d seen a video of someone who just plugged a random device into his PC and then asked Claude to figure out what it was and how to program it. I figured I’d have a go at that. The results were deeply scary.

Claude was able to identify the device and even pull out the demonstration program inside the device memory. Then it had a look at the code and told me what the demo program did. I asked Claude to re-write the demo in Python. Claude thought for a while and then told me that Python would be too slow for the graphical update. So I suggested Claude use Rust instead. Claude then figured out that I didn’t have rust on my laptop and offered to instal it. Then Claude figured out that I was using Windows on ARM, found the correct version of the SDK, installed it, tried to build something, failed, updated some elements and rebuilt others. Then it managed to get a test program running on the device. This was in around an hour or so and is officially amazing.

It was great to see so many folks turn up and just set to building stuff. And we kind of managed to make the local pizza place look busy when we went for dinner.

I took this picture when I got back home. I now have Rust code running on the device, and Claude is decoding the original binary and writing a Rust version of it. We live in very strange times.

Use thick wires for your power signals

The black and the red wires are nice and chunky

The “Bourbon Classic Camera” is very nearly finished. It’s been in this state for the last few weeks as I add a range of “finishing touches” including things like a screen that works and a power supply. Today I was wiring in the UPS unit that I’m using to power the Pi directly and it didn’t work. That is, it would get into the boot process, reach the point where the screen is supposed to light up and then stop. Oh well. As I’ve said before, power supplies are always a problem.

I’m doing something mildly naughty, in that I’m putting the power into the Pi via the 40 pin “hat” connector. This works, but it bypasses all the funky power negotiation that you get when you use the usb c connector. It means that you can provide five volts, and only five volts, and they had better be stable and able to deliver the current the Pi needs.

Mine weren’t. Mainly because I’d used dPont cables. These are fine for moving data around (although personally I prefer to wire wrap) but they just don’t have the copper thickness for power. The wire ends up being a resistor in series with the power supply. This means that the more power the Pi wants, the lower the supply voltage gets.

In the end I used some speaker cable from way back (another reason never to throw anything away). I also soldered pin connectors directly onto the wire and made sure that they gave a good positive connection. All is well now, and the Pi gets through the boot process and works a treat. So, if the device in your build is running slow or not running at all, you might want to think about using thicker wires for the power.

What to do when your Raspberry Pi memory card wears out

My AI Picture Ink Display has been running for a few months and it’s been nice to see a different crazy picture appear every now and then. But recently it has taken to failing. I, of course, instantly assumed that my software was broken, but it turns out to be a hardware issue with the memory card in the Raspberry Pi. The Stable Diffusion program that makes the pictures works the file store very hard, repeatedly reading and writing data, and this can result in the storage device failing. I diagnosed the problem by taking a look at recent Linux kernel messages:

dmesg -T | grep -Ei 'error|ext4|mmc|i/o|corrupt'

The statement above filters the error messages for ones about storage issues. In this case, the important message was:

I/O error, dev mmcblk0, sector 22491848 ... (READ)

mmcblk0 is normally the Raspberry Pi's microSD card. An I/O error at this level means Linux attempted to read data from the storage device and the operation failed. The most precious thing on any computer is the data, in this case the pictures that had been generated, so while the system was still working I needed to get those files off the machine. I used a USB memory stick. I plugged it in, waited a few seconds for the Pi to notice it, and then checked to see what devices were available.

lsblk -f

I got this:

That told me that the usb memory key was working and had the path of “/media/rob/HP v100w” (note that this contains a space). You will get a slightly different path. Now I could copy the contents of the generated_images folder onto the USB device.

cp -av /home/rob/generated_images/ "/media/rob/HP v100w/"

The quotes around the destination are necessary because the USB drive's name, HP v100w, contains a space. The trailing / on /home/rob/generated_images/ means copy the contents of the directory into the destination directory. Once the copy was complete, the next thing to do was make sure that any cached writes to the usb drive were complete:

sync

Then then I could unmount the USB drive:

sudo umount "/media/rob/HP v100w"

Now I had a copy of the folder on the drive. If my Pi was running with the desktop this would have been much easier, I could have done it all in the File Explorer application. But the image generator runs “headless”. One tip: if the USB drive is not picked up when you plug it in, try using a different USB port. You should see blue and black usb sockets. I’ve found that for usb drives the black ones work best.

As for the suspect memory card; once I’ve got all my files off it the card is going in the bin. And I’m going to look into running the program on a device with a “proper” solid state disk. This will go faster and should not be susceptible to wear like this.

INTER/ACTIVE 26

Charles told me about INTER/ACTIVE 26 a while back and I went along today to take a look. I took the Custard Cream Camera to grab some snaps and have some fun with them.

It was great. There were folks showing off all kinds of tech, from vintage radios, military laptops, amateur radio, 3D printing and lots of hardware you could play with. Charles had brought along Penelope, his AI powered robot work in progress. You can see a slightly tampered image above. In fact, I used the camera to make a lot of slightly tampered images. Converting people to Lego was the most popular transformation.

..although watercolour and line drawings were also in the mix. If you want to see a slideshow of the pictures that I made, take a look here. Kudos to all the folks who set this up. It is exactly the kind of thing that we need to be doing to tell people about tech.

Go and See "Pressure"

Pressure is not a superhero film. Except that it is. It’s a story of how the largest enterprises can pivot on a single person; in this case James Stagg (played by Andrew Scott) who was Britain’s chief meteorological officer in 1944. He had the unenviable job of telling General Eisenhower whether the troops taking part on the largest seaborn invasion in history would be able to get off their boats and onto the beaches. Basically, if Stagg gets the weather wrong and the invasion arrives in the middle of a storm, lots of people will die and the war could be lost.

That’s pressure. Everybody in the film is played wonderfully. I’d forgotten what a good actor Brendan Fraser (Eisenhower) is too. You should see this film. No car chases, alien invasions or gymnastic fighting. Just a true story with the highest stakes imaginable.

Lightroom Love-Hate

I spent a big chunk of today getting all my holiday photos out of the camera and loaded onto a real, spinning, magnetic disk. I know that flash storage is the future, with super speedy reads and writes, but it is also super expensive. Particularly now. So, I put all 1,669 shots on a local hard disk on my machine. I was worried that this would slow down the editing process as files were moved on and off the rusty platters, but I’ve not really noticed a difference in performance.

I’ve used Lightroom for photo editing for so long now that I think it is a habit I may never break. Even though it is consistently annoying and still won’t let you open more than one catalogue at once. I don’t consider myself as much of a pro with the tool, so I don’t have many pro-tips to offer. One thing it is worth remembering though is that you can copy and paste image edit settings very easily and apply the settings to lots of images at once. This doesn’t work too well for things like cropping, but it is a great way to apply the same style to a group of pictures. The AI object selection and layers are also well worth exploring. You can create export settings so that you can easily create downsized images from your originals. It is very easy to mark specific images (using rating numbers 1-5 or colours from 6 upwards) and then select multiple images by rating or colour. And you can attach keywords to images and then search for them, as well as find images from a particular camera or even lens.

So, actually quite useful then I suppose…

Last day in Bergen

Well, this is sad. Our flight leaves at 7:30 pm, and we are spending our last day in Bergen. Fortunately the check out is at 12 noon, so we’ll have a room to nip back to for the morning. We had another walk round and I found a fantastic camera shop. Something of a payback for all the time I’ve spent in wool shops over the last few days.

Anyhoo, I did buy a camera, but it is kind of a joke device, shaped like a film cassette and with a tiny viewfinder sticking out of it. It does work, and you can get pictures out of it.

I was a bit upset to discover that it was sold without a memory card but since we were having our lunchtime sandwich opposite a tech shop I was able to head across the road and solve that problem easily. The camera is a blind box deal, with six different film simulations. So they say. The pictures are quite low resolution, massively over-sharpened and rather charming.

The pictures below have been made deliberately small so that they don’t look too bad.

After lunch we chilled in the hotel lobby before walking to the station and taking a tram to the airport. The tram is not the fastest way to get there but it is the cheapest, at around four quids for the pair of us. Plus you get to visit all the places where normal people hang out and see a bit more scenery.

The airport was nice until we got to the “cattle class” gate for the plane. They must have considered the probable passenger count for each plane as a few hundred and then gone ahead with the plans for only sixty or so seats at the gate. I’d like to have been at that meeting.

The flight home was smooth, and at around nine o’clock in the evening we started the two hour journey home in the car. Unfortunately, we were now back in England using English roads. Which they like to close at night so they can play with them. So, we got the diversion experience of a lifetime. As in “it seemed to take a lifetime to get home” and “it will take a sizeable chunk of a lifetime to forget it”. One part of the diversion planning process seemed to involve sending enormous lorries down tiny roads past rows of parked cars outside neat little houses that probably weren’t expecting motorway traffic levels.

We also need to mention the signage. What there was of it. You might think that if you were planning a road diversion, you’d get an appropriate number of signs in. Not so. Signs were rare and confusing. At one point we spotted one with the word “Diversion” on it being displayed upside down, presumably so that the arrow pointed the correct way. It felt like we got home in spite of this “help”. In the end we let the Satnav sort it out and just hoped it would get us back on the motorway after the blockage. Which it did. If you ever wonder if you should take a trip to Norway you totally should. We had a great time.

Putting the fun into funicular

We used our special power of getting up early to ensure that we got to the funicular railway before the queues started. Side note: Nothing pleases us more than walking past a big queue for something we walked straight into an hour ago. Anyhoo, the railway was great, the views tremendous and they had a splendid wildlife park at the top of the railway. With goats.

They were also running some kind of, er, run, at the top with figures in track suits tightening shoelaces and donning ankle tracking brackets to precisely time their entries. We just had a hot chocolate and watched them all.

Then it was back down for a quick sandwich lunch and a walk around Bergen, which is lovely. Getting lost is not a problem if you know that you can always walk downhill to the water’s edge. Then it was out for another nice meal. Italian this time. By the time we’d finished – walking out past a huge queue waiting to get in – yay! It was close to sunset and we’d enjoyed the funicular so much we thought we’d do it again. So we did. Slight snag. The sun sets right behind the city, so the pictures had to be taken right into the light. But as it got lower this became less of a problem. And it was great to see the city slowly light up below us.

Yet another cruise

Time for another cruise this morning. This one was not electric but was pretty high-speed diesel. Going out on the deck to grab pictures was braver and more exciting than I would have liked and the light wasn’t that amazing – although the scenery was. A member of the crew was saying that if the captain is in a good mood he’ll take the boat right into a waterfall cascading down the side of the fjord. He was and he did. I wondered about asking what the captain would do if he was in a bad mood but decided against it.

We found a lovely place for lunch by mistake, looking for a different (and as it turned out probably lessor) one. Then, in our quest to find the missing B&O hifi store we found a really nice wool shop. Norway folks take their knitting and crochet really seriously. In fact, they seem to take everything in life properly seriously. Until you make them laugh, at which point they really go for it.  

We did plan to go back to the same place for dinner (a mistake we’ve made many times) but it was full so we were forced to strike out and find somewhere new. Which we did. The owner told us he had only opened the day before. It was awesome. Just the right amount of food for us. On a stick. Although I’m told that the proper name for this form of serving is skewer.

Boat, Bus, Train

The day started with a trip on a boat. Which was very nice. Same kind of setup as the first day, but with slightly less pleasing weather. That dropped us off to pick up a bus ride past even more nice scenery to Voss where we caught the train to Bergen.

Tunnel vision on the bus

Not our train

Everything fitted together nicely, including a filling station purchased meal, eaten in the Voss station waiting room. Then on to Bergen. A quick walk from the station to the hotel. Past a Bang and Olufsen HiFi shop. Not seen one of those in a while. We must be in the posh part of town.

Down in Flam

Flam is lovely. Based at the end of a fjord it is a tiny village with just hotels, museums, souvenir shops and a general store to its name. Business looks to be good though, what with a succession of cruise ships dropping of interested folks with cash to spend and time to kill.

We took a walk around and found a little Farm Café with superb garden and even a swing. In the afternoon we did a bit more wandering, then headed for our poshest lunch of the tour (it turned out).

We managed to inadvertently shoplift a bag from the museum (the assistant forgot to charge us), but having decided that we wanted good rather than larcenous memories of the place we took it back and offered to pay. They were pleased to see us and we got a discount. So honestly is not a bad policy after all.

Six hours to Flam

The title of this post is a bit of a fiction. It’s actually six hours to Myrdal, with another hour or so down the side of the fjord to Flam. The trip was billed as one of the most beautiful railway journeys in the world and I quite agree with this. Although it turned out to be spectacularly difficult to capture the scenery through the thick, double glazed, train windows.

The lines in the sky are the reflections of the carriage lights. I need to find a way to get rid of them.

We got on the train first thing, got off it a couple of hours into the journey to stretch our legs for ten minutes, and then climbed up above the tree line to Myrdal. Which was quite chilly and totally vindicated our choice of cosy clothing. The train ride down on the Flam Railway (which is a thing) was very nice, with a stop halfway to look at a waterfall.

We arrived starving and found a lovely place to eat which also served good beer.

..although some of the staff were a bit wooden. Ho ho.

Electric Cruising

Lots of boats

Best way to see Oslo? From boat. Best kind of boat? An electric one. We were up bright and early to get on the first of our pre-arranged “trips around the bay”. This was on a super modern, carbon fibre, all electric boat which glided silently out into the bay and around a bunch of pretty islands and inlets. You could wander around the boat taking pictures, so I did. The weather started super-grey – a bit like being at home – but it got nicer and nicer until we arrived back in something approaching sunshine.

Private island

Walked back from the boat past an amazing bakery where we had a couple of amazing sandwiches. No dry ice , but it did have a really good view of the harbour.

After a quick freshen up at the hotel we headed for the opera house. They let you walk on the roof, and this is totally a good thing. The views were superb and the lighting rather special.

I even managed to find a really nice camera store where I didn’t buy a camera. Or a lens. But did get a rather nice bag.

A change of pace for tea, where we headed for “Proud Mary’s”, a pub/venue which served up a bunch of nice nibbles which, considering the huge breakfast and lunch that we had enjoyed, fitted the bill perfectly.  Then to bed. Big journey tomorrow.

Hello Oslo

Today finds us in Oslo as part of our one-week world tour of Norway. We were up at 6:00 am and off on the road by 7:00am. The journey to the airport – always a giggle – was enlivened by plenty of queues and jeopardy. But we got to Manchester in good time and after a quick trip in and out of the car park (got to love their signage) we got parked up and hot footed it over to Terminal 2 in time to have a coffee in a place with thousands of people and no seats.

Then through passport control, a quick lunch, a half hour delay in the plane and off we went. I think short haul flights are allowed to have their seats closer together than long haul ones. All I can say is ouch.

We got to Oslo in good time and had a very smooth train journey to the hotel. Checked in, a quick walk around and then tea at The Duck and Dumplings, a place I spotted on the map while making plans on the plane after landing. Very nice food and deserts with dry ice.

I asked if all meals in Norway are accompanied by billowing white smoke and I was told that they are. Rather looking forward to breakfast tomorrow.

You may have noticed some interruption to the service at robmiles.com today. It turns out that the day you leave the country is probably not the best day to transfer your domain name. Particularly if the domain you are moving is used in the email address used for 2 factor authentication with your DNS provider. All of which added up to an interesting conversation with a NameCheap support person who was super helpful and managed to give me enough access to able to sort it out with the help of the hotel wi-fi. Very glad I brought the laptop now…..