Arduino - 5x8 ISO 8859-2 font

A while ago I took an existing 3x8 font, converted it by hand for use with AdaFruit's graphics library and subsequently modified it to use ISO 8859-2 characters (in a presumably innocent coincidence this popped up in the Adafruit GFX Library a few months later). Anyway, I recently received a request to perform the same modification to the standard 5x8 fonts and only recently got round to looking at this. As a quick reminder, ISO 8859-2 covers ~128 extra characters for alphabets used by Central and Eastern and Southern European languages - these characters look like this:

After a little bit of experimentation I realised that the extra 2 columns aren't really all that useful for adding the extra ligatures (čarky, hačky, umlauts etc) - we're really constrained by vertical space. Many of the existing letters need to be modified or reworked entirely, since they consume the almost entire 8 rows and we need 2 rows for some ligatures. For example Ä is currently implemented as the following:

    0x7D, 0x12, 0x11, 0x12, 0x7D

Which looks like this:

This is visually a little confusing, but more importantly we cannot really re-use it for ISO 8859-2 since some of the ligatures we need to add to the "A" require at least two rows. Instead of having an "A" which jumps around depending on the ligature, I've created a single A for when a ligature is used and left the ungarnished original letter alone.

Just as another example of why this can be tricky the existing ä looks really weird to me, the umlauts are skewed off to the left and look like they're joined to the letter itself. 

I've moved this up into a central position which is the same on all letters involving umlauts. This is purely based on personal taste, but I think it looks better - below is the original style compared to my modified version:

There are similar considerations in some of the other letters that are left as an exercise for the reader - see if you can devise a neat system to fit all the letters below into 3x8 grid in a way that is consistent and legible, it's pretty tricky. I've made an initial stab at this (see GitHub gist below), but after revisiting this I've realised how flakey and error-prone this process of creating fonts is. 



Arduino - 8 Bit Graphics with SSD1306

After my initial foray into graphics I wanted to see what was possible with the Atmega328p and the SSD1306 screen. Lots of applications which use this little microcontroller involve pretty mundane, uninspiring things like home automation and I like the idea of giving it a wee chance to shine and do something fun. The constraints of a relatively slow processor, extremely low memory, as well as a tiny and monochrome display were pretty enticing to me, especially after reading some of the early stuff on folklore.org on the hacks Bill Atkinson, Andy Hertzfeld et al employed to get most of out the Macintosh's 68000 processor.

FPS counter

Firstly we need to know the limits of the hardware. I wasn't sure whether the processor, the SPI bus or something inside the display itself would limit the frame rate to something unacceptable and put the kibosh on any sort of fast/responsive visualisations. So I whipped up a quick test which sent a bunch of empty frames to the display and every so often output the average frame rate.

Mercifully it seems that we're able to push as much as 207fps to the SSD1306, which means that we've got a fair bit of room to play with and that if anything the Atmega328p is the bottleneck.

Code: ssd1306_spi_maxfps.ino

Breakout

My first attempt was a little clone of breakout. I am apparently hopeless at it - but it's actually the first "game" I'd ever written, so that's fun.

Code: https://github.com/smcl/breakoutduino.git

3D cube

Since the Breakout clone worked smoothly, wanted to see if it was possible to produce simple 3D graphics - eventually settling on creating a spinning cube bouncing around the screen.

Initially I got a little too ambitious and attempted to roll my own general purpose library for 3D graphics - with a set of functions to manipulate a stack of transformation matrices. However I kept on blowing through my stack space  and clobbering a load of program state. This makes sense, since if we have a matrix implementation of 4x4 floats, we'd need to create one for the camera, a rotation and translation then that's already 3 matrices of 64 bytes each. Then we have to multiply the matrices and apply this to 8 vertices you can easily see how a naive implementation could at some point go a bit haywire and chew through more stack than it should.

Instead of taking the time to trim this down using the limited debug tools available to me (no stepping, watch, gdb through my simple USB cable) I just manually calculated the transformations I needed for my rotating/bouncing cube ahead of time and implemented them so they could be parameterised. So starting with the following (with Tx, Ty & Tz representing the center position of the cube, θ being angle of rotation and x, y and z being the co-ordinates of each cube)

Which means that each point can be represented by the following vector

Since I just wanted a rotating cube with wireframe lines I didn't need to do much other than use this to calculate the expected x/y positions of each point on screen and then use existing drawLine() function to connect each. Which resulted in this slightly clumsy function:

The final video involved two rotating cubes, one of which is flying around the screen.

It's quite satisfying result as it's the first time I'd produced a from-scratch 3D projection since university, and was actually surprisingly fast - the video looks a little shakey but in person it's as smooth as butter.

Code3D_cubes.ino

Seed Cathedral


Known as the Seed Cathedral the UK pavilion at Expo 2010 in Shanghai caused quite a stir and rightly earned itself an award from the organising committee for its outstanding design. It's a simple cube,with thousands of transparent plastic rods sticking out of it - each one containing a different type of seed. When you viewed it from a dozen steps back a subtle Union Flag pattern appears, which I must admit is is very neat even if I'm not a proud/patriotic Brit myself.

When I was trying to think of weird or interesting things that could be modelled quickly and easily in 3D which would have an interesting effect this jumped into my mind. The implementation is pretty simple - use similar code to create the projection, and shuffle the camera around a little to create a shimmering monochrome Union Flag. 

... later that day

And after all that hard work, I saw some stuff that left me pretty deflated - two excellent implementations of graphics libraries for AVR microcontrollers. The first one is u8g - a slightly more heavy duty library than the one Adafruit provided. It also provides some nicer font handling ... including a tiny font similar to the one I did. Here's an example of someone using it to draw a rotating cube at a reasonable clip, ~40 fps no less:

The second one is even more impressive and needs to be seen to be believed. Someone managed to create a library to render a 16 bit colour scene in 3D with texturing and lighting.

Be sure to check out the video at the bottom of this link, it's a little humbling after you've just thrown together a couple of simple monochrome visualisations - http://hackaday.com/2016/01/02/better-3d-graphics-on-the-arduino


Arduino - Hacking a Canon DSLR shutter release

The Canon 450D supports external shutter release mechanisms using a simple 2.5mm TRS stereo plug - closing Tip or Ring to Sleeve will cause the camera to either take a picture or autofocus:

I created a quick circuit to test out the shutter (I've left out autofocus for now) and take a picture every few seconds using my Arduino, with the idea that I'd put together a nice sunrise timelapse. More on this later. The circuit itself is pretty simple, but the tough part for me was splicing my cable in a way that I could connect it to the breadboard.

In practice my setup is actually a little more complex as I wanted to use my SSD1306 display to show a little indicator of how many photos I'd taken without having to fiddle with the camera - note the very sketchy wiring :-)

The first thing I did was set it up to create a picture every 30 seconds to create a little timelapse of my puppy - Alfie - sleeping on a pile of my jumpers:


I used ffmpeg to produce a nicely sized mp4 video from the generated images.


Satisfied that I could get a decent result, I prepared a couple of timelapses from my living room window:

Timelapse #1 - overcast/snow

The first timelapse is a little boring - a grey sky turning dark blue then being illuminated by street lights. I pointed the camera roughly SSE/120° let the Arduino trigger a picture every 30 seconds for a few hours leading up to 5.30pm. Each frame was originally taken on the lowest-res setting available on my camera - 2256x1504 - at ISO 800 (since we'd also need to take evening pics) using a Canon 20mm f/2.8 lens. I actually started this one off at 1pm but there was a lot of boring grey sky so I trimmed it.

Timelapse #2 - sun/sunset

The second timelapse is a little more interesting. I used the same basic timing and setup but used a lower ISO since the evening pictures ended up a little darker and blurrier anyway, and this time pointed directly towards the sunset (SSW/220°). The main thing I learned from this video is that my window is filthy

Again I used ffmpeg to produce an h264 video:

Code for the Arduino sketch is available HERE but really it's just writing a 1 to a GPIO every few seconds so not particularly tough. 

Some AVR memory questions

I had a few Arduino itches I wanted to scratch, bits and pieces which aren't particularly complex or interesting enough to warrant a standalone blog post, but which are nonetheless worth spending a few paragrapsh on.

Why does every Arduino build include "-R .eeprom" when producing the ELF binary?

I had previously wondered whether it was necessary to pass in "-R .eeprom" when building the ELF binary, but I didn't really explore it much at the time. Essentially all articles or guides about using the GNU toolchain to compile programs for Arduino/AVR (and indeed the Arduino IDE itself) specify this switch so it's really worth clarifying what it's all about.

The first thing is that some articles are just plain wrong in their documentation about this option, for example the Arduino Playground docs for FreeBSD state that it's required to upload our binary to the Arduino board:

This is pretty wrong, since in avr-objcopy the -R switch actually specifies sections to be removed from the binary - per the manpage below:

So let's investigate what's up - here's an avr-objdump on the binary produced by blink.c:

There's actually no .eeprom section here to be removed - so nothing in binary would end up there anyway. If we define a variable with the necessary attribute "__section__" we can force something to be placed here, like the following string:

char *foo __attribute__((__section__(".eeprom"))) = "bar";

and now if we rebuild our ELF binary and inspect using avr-objdump:

All of a sudden we've got an .eeprom section in our executable, which would be removed if we followed the default instructions.

So why is this removed by default when uploading an Arduino sketch? I can only guess, but presumably it's just a common courtesy to anyone who uses .eeprom as persistent runtime storage - so they don't accidentally hose their data when programming their board. Hopefully if you know enough to place a variable in .eeprom, then you'd have the know-how to be able to alter the build configuration to omit these flags. It could also be possible that the eeprom wears out after fewer writes than internal flash so a bit of extra care is taken to preserve it - though I really cannot say why.

How is Arduino memory laid out and what controls it?

I was curious about how the memory layout is defined in the avr-gcc toolchain used in building programs for the Arduino, and ow it can be controlled. I've played around with LDF files for Analog Devices' Blackfin and SHARC chips (which look like ADSP-BF533.ldf and ADSP-21160.ldf respectively), but have never used GCC's linker scripts before.

On OS X using CrossPack these linker scripts reside in /usr/local/CrossPack-AVR/avr/lib/ldscripts, and there are an astonishing 95 of them in total:

These linker scripts aren't defined per microprocessor (i.e. atmega328, atmega168), but are written per architecture - taking a quick look at the documentation at nongnu.org we can see that the AVR chips used in current Arduino boards are all in the avr5 family (except for the atmega2560, which is avr6) so we can narrow things down slightly, but there are still five linker scripts to choose from:

Which one of these is used depends on the arguments passed to the linker, and each of them should be different and tailored for a certain purpose - although in practice this is not strictly true:

  • avr5.x: default linker script, this is used for linking straight C programs
  • avr5.xn: linker script used when the "-n" flag is passed to the linker. According the the ld manpage this flag is used to turn off page alignment and mark the output as "NMAGIC" however diffing this and avr5.x shows no difference in practice.
  • avr5.xbn: used when the "-N" flag is passed to ld. It's meant to mark the .text and .data sections to be readable & writeable (as well as not page-aligning the data segment apparently) however again this is script is identical to avr5.x.
  • avr5.xr: used when "-r" flag passed to ld. This is to "generate relocatable output" - but essentially it's used to pre-link C++ files I think?
  • avr5.xu: used when "-Ur" is passed to ld. This is used for the final link on C++. It's roughly the same as avr5.xr but includes something about ctor table

To see what the memory layout the basic linker script produces, we can take a pretty boring program like blink.c and take it apart using avr-objdump. The GCC docs have a nice wee visualisation of the SRAM usage of an atmega128, which won't be a million miles away from the 328p:

So the SRAM contains Data, BSS (a segment containing variables initialised to zero) and then some space for the heap (which starts at the address __heap_start and continues to a handful of bytes before the stack base) and the stack begins at RAMEND (address 0x8FF on Atmega328p) and grows down as necessary.

This is why you get that little "low memory" warning in the Arduino IDE - if there's only a couple hundred bytes spare in SRAM it's not too hard to accidentally write a program that uses too much stack and ends up overwriting some of heap variables. There's no memory protection (that I'm aware of) on these chips so there's no way to prevent this other than being careful.

How quickly can we access the Arduino's SRAM, Program Memory and EEPROM?

I wrote a couple of simple benchmarks to see how quickly we can access the different memory types available on the Arduino:

  • read: read and sum a 256 byte array
  • write: write the values 0...255 into a 256 byte array

This is not massively scientific but I just wanted ballpark figures to see how things stacked up. The results I got were:

memory read (μs)
write (μs)
data 148 148
program 196 n/a
eeprom 584 859580

I was pretty surprised by how fast reads from program memory could be, and even more so how quick eeprom reads were - but slow eeprom writes aren't particularly surprising. It's worth noting that for program memory at runtime you can only write to certain sections reserved for the bootloader and I want to risk messing that up so I left it out.

A tiny font ... Eastern European edition

In a previous post I created a little replacement font for the Adafruit Graphics Library, but left the non-ASCII characters (values > 0x7F, 128 in total) completely blank. This is because this area is used to implement the additional symbols necessary for languages other than English, and I had initially just intended to create an English language font for my own purposes.

Obviously you can't represent all of the necessary symbols for all the world's languages in just 128 slots, and this is where ISO 8859 comes in*. There are 16 ISO/IEC 8859 character sets which implement the symbols for loosely related (and sometimes unrelated) languages in the upper 128 bytes.

The most common one is probably ISO 8859-1 which can represent most of the EU's main languages, so it should be a pretty sensible one to implement. However since I live in the Czech Republic and I love Central\Eastern Europe I decided to implement ISO 8859-2 which covers Bosnian, Polish, Croatian, Czech, Slovak, Slovene, Serbian, and Hungarian and looks like the below

Note: the above rendering I picked up someplace on the internet is actually incorrect - when the letters t and d have a caron/haček it actually looks more like an apostrophe - ď and ť.

I only implemented this for my slimmer 3x8 font, but if there's any interest I can quickly put together something for the Adafruit 5x8 one or attempt a different character set so long as it's based based on a latin script. Sadly I'm not sure if my skills are up to the task of creating something tougher like Chinese, Tamil or Thai.

I'm a novice at font design, but looking at the required diacritics it seemed to make sense to reserve the top two rows for things like Haček (e.g. č, ř and ž) and Čarka (e.g. á, ý and é) and the bottom row for anything below the letters - which leaves a 3x5 space in the middle to implement the root of each letter - which I managed to do with a couple of exceptions. Here's a quick visualisation of this:

So for example if we want to create the letter "č" it would look something like this:

With only three columns available there are a number of characters which will look a little weird - the worst of which were Đ, § and ď - and the Albanian characters with the tail (ç, ţ and the like). Here's what the font looks like:

It's a little cramped - with some characters appearing to be joined to those above/below - which is due to the fact that the top rows were previously blank and served as line spacing.

I've pushed this into my fork of the Adafruit GFX Library repo on github - check it out and copy the whole thing into your Arduino libraries folder to install it. Note that the Arduino IDE saves files using UTF-8 encoding, so you can't just throw string literals in and assume they'll work - the simplest way to output a string using this character set is to manually enter the hex values. 

So if we want to have the string representation of a nice little Czech sentence, we'd encode it as follows:

When we pass this to display.write() - not display.print(), which will just output the numeric values - we'll see this:

* = Yes I know that we have unicode to properly address this problem, but it's quite complex and is mercifully not used by the Adafruit Graphics library for simplicity's sake.

A tiny new font for the SSD1306 128x64 OLED screen

The default font for the Adafruit graphics library is a 5x7 font, it looks a little something like this:

It's readable, but I thought it'd be interesting to use a more compact font like "Tom Thumb" by Robey Pointer:

The font currently bundled with the library is in a file called glcdfont.c and is defined as a big byte array:

Each line of that array is 5 bytes and represents a simple monochrome bitmap of a single ascii character, with each byte representing a column. For example example the letter "a" is represented by the following line:

0x20, 0x54, 0x54, 0x78, 0x40,

In binary each of these is:

You can almost see the shape if you twist your head 90 degrees to the right, but to make things a little clearer here's what this actually represents:

If we were to use the Tom Thumb font we could save ourselves two bytes per character, since it is a little slimmer, meaning that "a" could be represented by the following three bytes:

0x68, 0x58, 0x70

Or, in binary:

Which looks like this:

Since there's a bit of wasted space at the top (the top two bits won't get used at all) we could technically save ourselves a little more space, but for simplicity's sake I'm going to sacrifice those two bits. Currently the font can be easily retrieved and manipulated as each column fits neatly into a single byte, however we'd introduce a good deal more complexity by squeezing those additional bits - which would mean substantial changes to the driver itself to compute the correct address to read the bitmap from, and shift/mask the necessary bits.

However even with this tradeoff we can still save 2 bytes per character, which is a pretty impressive 512 bytes over the whole ascii space which is pretty substantial.

The main challenge is creating a separate .c file we can swap glcdfont.c out for. I had trouble dealing with the BDF file Robey shared, but since there's a little bitmap representation of the whole ascii space it wasn't too tough recreating the font by hand ... just extremely tedious! I hacked together a simple little app (source is here) using Processing that let me paint each character by clicking boxes and which spat out the hex values when I hit a key. I'd then copy and paste this into the glcdfont.c. As I said, extremely tedious.

Once I'd produce my file it was time to test it - the beauty of having the 3x6 font means that the entire ascii space could be displayed on a 128x64 screen:

It's actually pretty surprising how much text you can fit on screen.

And just a little reminder of how small this screen is, with a €2 coin for scale

You can see the amount of memory saved in the .text section (where program code lives) by examining binaries GCC has produced - exactly 512 bytes.

My fork of the Adafruit gfx library with the Tom Thumb font is at https://github.com/smcl/Adafruit-GFX-Library and is a drop-in replacement for the existing library (replace the entire library, though, since I made some changes in Adafruit_GFX.cpp. You can switch between the fonts by toggling the #if ... #else condition in glcdfont.h below:

Apes and Embedded Systems (part two)

In part one I just hooked up a little monkey keychain called Chuggzilla to my Arduino. A day or so prior to my 90 Kč spending spree on Chuggzilla I'd just received a 128x64 Adafruit OLED display and realised this was a nice opportunity to play a bit with it.

The idea was that Chuggzilla would still scream his weird morse code madness, but the screen would display the actual text behind the message, with the current letter highlighted. I was just wanting to quickly get something working, so instead of integrating with the OLED library using C I just ported my morse code stuff to the Arduino IDE so I could easily re-use the existing API. The final result is pretty fun - though I suspect the current drain from the screen is too high because the previously manic ape noises were almost whisper quiet.

I am not sure exactly who would want to reproduce this, but if you're interested hook the OLED screen up as per the Adafruit wiring instructions then connect up whatever animal or thing you want to control per my original schematic (connecting digital 2 instead of digital13 to the transistor's base, since we need that for the screen), create a sketch with the code below and add the Adafruit libraries.

Code is on this gist on github. 

Apes and Embedded Systems (part one)

I woke up today and decided to do some neurosurgery. I set off into the town and sought out a willing test subject - ladies and gentlemen, meet Chuggzilla (credit goes to Miriam for the name):

Chuggzilla is a keychain monkey I bought in Tiger for around 90 Kč. Pressing a little button on the back of his head causes his LED eyes to light up and a little speaker inside to play a monkey noise:

This is what he looks like on the inside

I soldered a couple of wires to either side of the button element, so that I could control him electronically:

Testing him out with a button on a breadboard

Finally being controlled (via a transistor) by an output pin on the Arduino using the morse code program I previously wrote - the message is "hello world" or ".... . .-.. .-.. --- / .-- --- .-. .-.. -.."

The circuit driving this is below. I used a little LED for debugging when I was checking Chuggzilla was connected fine, and just kept it in the circuit - you can omit it if you're certain your animal is wired up fine :)

The code behind this is at https://github.com/smcl/arduino_morse - it's straight C, so if you want to load it to your Arduino you'll need to have the AVR GCC tools installed, and change the AVRDUDE_PROGRAMMING_DEVICE variable in the Makefile, then run make flash.

Arduino primer/refresher - OS X perspective

I've been wanting to get back into some interesting embedded systems and electronics work - so I figured I'd dust off my old Arduino. In the process I ended up having to re-learn how to program it using C without using the Arduino IDE (it's not my favourite thing), which is what I'm going to go into here.

I downloaded CrossPack which contains GCC cross compiler for the AVR architecture, as well as a handful of other useful tools (like avrdude, which we'll use to flash the Arduino). Once that's setup we can start off with a pretty simple program which rapidly blinks the arduino's onboard LED, which is sort of the Hello World of the Arduino community:


To build this using GCC we just need to call avr-gcc, making sure to specify the processor (using -mmcu), define the processor frequency (the F_CPU macro)


This creates an ELF binary. One odd thing is that pretty much everyone building C code for Arduino seems to include the "-R .eeprom" switch which removes the ".eeprom" section from the output binary. However I've not ever seen this section does not exist in my ELF binary - see below:


Anyway, perhaps there's something else at play that I've not understood so I've kept this switch in (I'm making a mental note to explore this later). Again many other places seem to insist on using the avr-objcopy utility to produce intel hex file before uploading to the Arduino - so often you'll see the below:


However again this is not a necessary step it seems, since the utility we use to upload the binary to the Arduino - avrdude - supports ELF so you can skip it. When we want to upload the binary to the Arduino we'll use avrdude, but first we need to know the programming device. For me this is just the USB type A to type B cable that comes with the arduino, which in the example below is /dev/tty.usbmodem1421:


I've found it useful to create a little Makefile to tie these steps all together:


Finally to tie this together in a slightly more existing example than blink.c, I've created a little program that will flash the LED attached to pin 13 with a message encoded in morse - "hello world" or ".... . .-.. .-.. --- / .-- --- .-. .-.. -..":


The code is on github at https://github.com/smcl/arduino_morse and only requires a bare Arduino and USB cable.