I enjoy technical challenges...if and when I succeed eventually. This posting may be a technical challenge for you, but watch me not care. Some of you may appreciate the information and that's good enough for me. So, if you're interested in homegrown MP3 music boxes, Linux on Netvistas, PIC microprocessors, RS-232, infrared remote controls, and what an obtuse idiot I occasionally am, read on!

(As always I also hand out the involved source code, which might come handy if you want to build something similar.)

In my bedroom I have an ancient, 70s-vintage Sanyo DCX8000k receiver (which I really like) and a headless Netvista with a wireless net interface serving as juke box. Until recently I used an ancient Palm III with a crappy homegrown app to control the MPD player on the netvista remotely (by cable). The problem with that was that a) the app is lousy because I never got around to finish programming it out fully (Palm OS is not exactly loads of fun to develop for, all quite tedious GUI crap in C), and b) that running the serial transceiver in the old Palm III meant draining its (rechargable) batteries very quickly, which sucked.

Unfortunately the Netvista offers only two usb ports for connectivity (one of which is used by the wireless adapter, the other connected to the Palm via a usb-to-serial converter) and I couldn't create anything as simple as I did on the main jukebox in the living room (20x4 LCD on the parallel port, IR receiver on the serial, LCDproc and lirc, MPD, homegrown app on top).

So after restoring the Palm for the umpteeth time when it ran out of juice unexpectedly I started Hatching Plans.

Cunning Plans

First idea: the usb converter has +5V, the palm wants 2..3V, why not plop in two diodes to drop the 5V to something palatable and run the palm without batteries? In the end I nixed that plan, because it would still leave me with the boring Palm programming and if I had a power outage or something I would still have to restore the palm. Also that's still a cabled setup. Too brittle, too much palm-gutting for too little benefit.

Second and Final idea: About a month ago I decided to build something cooler but more extensive, a Serial-LCD-IRremote transceiver/decoder/controller/mangler/magic module, based on a PIC microcontroller.

I called it "dervish", because dervishes are meant to be ascetic, unflappable and sources of wisdom and witticism (says wikipedia); furthermore there's the whirling dervishes' dance which resembles the whirlwind shuffling of tasks the PIC has to do to provide the functionality I want it to.

The idea behind the dervish module was to provide a set of services similar to the main jukebox but multiplexed across only one interface: bi-directional serial via the usb-to-serial adapter.

So the plan was: get a cheap 20x4 LCD module, dig up one of the remote controls from the junk pile, a PIC and sundry small bits and build a module that

• receives and deciphers the infrared remote signals,
• sends the decoded remote control key info via serial to the Netvista,
• receives commands and data for the LCD via serial and forwards them to the LCD,
• and gets its power from the usb converter to be self-contained.

I had a bunch of PIC16F628's around, which have an (TTL-level) USART onboard, and no electronics grab bag is complete without a few MAX232's. The 20x4 LCD module is one of the ubiquitous HD44780-compatible ones, which can be fed their stuff in either 8 bit or 4 bit mode: in 4 bit mode, one needs 2 control lines and only 4 data lines and that is what I used.

I decided to also add a computer-controllable backlight and a piezo buzzer (as an annoying alarm clock).

Occam and Me

Now why the allusion to my not-so-sharp deductional razor? Well, it's been...interesting to make things work, especially the LCD part. The interesting part is: how do you determine the cause of problems if you have no or few I/O capabilities and if everything of interest happens within microseconds on a standalone microcontroller, and often involves said microcontroller talking to yet another microcontroller (the HD44780 in the LCD)?

So you rely on the datasheets. The problem with that is that apparently (or at least in my experience) the HD44780(-compatibles?) commonly used are often a tad...odd. For example, whatever chips my LCD module (a JDM-204A) has under those epoxy blobs would not talk to me in 4 bit mode unless I added a delay of about 50µs to the specified trigger events, and would work reliably with just one of about three or four different ways of setting up the trigger. Of course it took me quite some time to figure this out (three days of debugging, IIRC).

However, when I had coerced the LCD into cooperation, debugging became a lot easier; the axe had acquired a fairly sharp edge. The remaining components (infrared remote control reception and decoding, serial reception and transmission) were implemented very quickly and all seemed good. This is where my obtuseness came into play.

One of the interesting aspects of dervish is that a number of concurrent activities take place, all of which need different amounts of time: sending a byte to the LCD takes about 100µs, receiving or sending one byte from the serial takes 521µs (at 19200bps), and receiving a key press on the remote control consists of 34 separate events which are spaced between 1.2ms and 13.6ms.

Often the PIC would be busy with one of the slower operations when one of the other events was about to interfere. What do you do? You use interrupts and some queuing of tasks so that noting is lost. What did I do? I used interrupts and some queueing, ensuring that nothing is lost. My mistake? Stupidly using a stack or LIFO as queue, instead of a ring buffer or FIFO. This reversed the order of elements in the queue, thus confusing everything when the queue would be emptied.

The problem was that queuing wasn't required all the time but only if the PIC was especially busy, which wasn't as common as I had thought. This means that my mistake didn't immediately or obviously wreck operations but rather showed up as corruption and confusion at some random time...the classic case of an intermittent fault.

In the end I spent another day or so of carefully conducted experiments trying to reproduce and isolate the problem. The revelation of my silly mistake was...powerful, the fix quite quick; after all a ring buffer isn't that hard to program even in assembler and on a platform that has a total of 224 bytes (not K, M or G!) of memory.

The lesson I learned again:

"For every complex problem there is a solution that is simple, neat and wrong." -- H. L. Mencken

The rest of this post explains the technical functioning of dervish, the hardware and the code; interesting if you want to build your own but likely over the top otherwise.

How and Why Dervish Works - Infrared

The new thing (for me) was figuring out how to deal with an infrared remote control with the PIC. On the other box I used lirc, but I wanted to avoid that mess here: I'd need the PIC anyway to transfer from serial to LCD, so why not rise to the challenge and do all the remote control processing there as well? On the Netvista, I'd be using lcdproc (which has various lcd-via-serial-converter drivers, see below) and my homegrown music box app.

Digging through the lirc code and reading this great explanation showed me that it would be dead simple to perform this decoding, at least for the remotes I have lying around: they are all following the NEC protocol more or less. The NEC protocol is a length-encoded mark-space scheme, a bit like asynchronous RS-232.

If you look at the description closely, you see that the easiest way to deal with these signals is to measure the delay between rising edges of the signal:

• puls 9100µs space 4500µs: is a header
• puls 9100µs space 2250µs: is the repeat command
• puls 560 space 560: a zero
• puls 560 space 1690: a one
• a final trailing puls 560 makes a stopbit

The logic for this is super-trivial: start the clock on the first edge, measure elapsed time at the next edge, repeat until you have 32 bits (and the final stoppie). A delay of about 13.6ms is a header, 11.35ms is a repeat, 2.25ms is a one and 1.12ms is a zero. The nice benefit of this kind of decoding is that between edges the PIC is free for other tasks; all we need is a small and fast interrupt handler and a timer.

I run the 16f628 with the internal oscillator (4MHz, 1µs cycle time) and use the 8-bit tmr0 timer/counter with a 1:128 prescaler (otherwise the timer would wrap every 256µs). The infrared receiver I had lying around (desoldered from some decrepit junk) is a ZD1952 (similar to the common SFH5110) which is active-low, so I need to trigger on falling edges. The 16f628 has a very convenient interrupt feature on RB0 pin, with selectable edges to trigger an interrupt.

I decided to have the infrared decoder get a whole keypress (32 bits) and then transmit that via serial. Most remotes that I've seen have a constant part at the beginning of the signals, usually 16 bits that never change. The decoder receives all 32 bits, but the subsequent serial transmitter routine only sends the hexified 16 bits that change.

I cheated a tad, and ran the remote I planned to use through irrecord on another machine to pre-determine the precise timing values (not all remotes I have around use the same variation of the NEC protocol timings).

To deal with the asynchronous infrared events, the PIC runs two main sequences of code. One is the main loop which does the non-timing-critical things (a chugging, plodding workhorse), and the other threads are two interrupt handlers, one for infrared and one for serial input.

The infrared decoding is done from an interrupt handler, which keeps track of how many more bits to expect and resets to a sane state if the signal is garbled. Note that most NEC protocol remotes that I've seen use "constant length" signals, where (at least) the variable part consists of complementary bits. One could add a check for this aspect, too, but I was too lazy for that.

Upon successful reading the last stop bit of a remote key (32 bits for my remotes), the 4 bytes making up the code are flagged valid to the workhorse. the interrupt handler start looking for new keypresses, but does only record them once the workhorse has consumed the key bytes.

Serial Comms

The serial comms uses RB1 and RB2, I need no handshaking, and the level conversion is done by a (underused and half-bored) MAX232. With the onboard 4MHz oscillator I found the fastest baud rate that worked reliably to be 19200 (this being due to the limited accuracy of the actual baudrates the PIC USART generator can produce). 38400 worked somewhat but not reliably, but 1920cps is easily good enough for driving the LCD module.

One of the reasons for using the PIC at all is that the LCD needs 9 bits of information: 8 data bits and one RS (register select or cmd-vs-data) bit, and these fed on a parallel bus. On the serial side we get only 8 bits per read (well, without some ugly code and modifying a lot of the lcdproc code). The intended Netvista-side code, lcdproc, has a number of drivers for serial-to-parallel-LCD interfaces, and all of these use a simple scheme with an "instruction escape byte" which I adopted for simplicity as well.

Dervish handles four byte values specially: 0x80 means the next byte is a command, 0x81 means the next byte is the desired backlight on/off state, 0x82 means the next byte indicates the desired buzzer state and 0x83 is for sending any of the special bytes as data.

Receiving from the serial is done in an int handler, which plops the received value into the receive ring buffer (sigh), from where it is handled later on by the workhorse loop.

Transmissions are done by the workhorse and using a second, separate ring buffer which is filled by the IR receiver code. The transmitting code sends stuff from the ring buffer whenever the USART transmit register is ready/empty.

LCD Comms

The communication with the LCD is done with the upper half of RB for data and two RA pins for the RS and Enable signals.

Getting the LCD to initialize in 8-bit mode was not too much of a problem, but convincing this particular one (a JDM-204A) to cooperate in 4-bit mode took a number of agonizing days.

None of the hd44780 specs and notes I found out there mention any timing restrictions for the nibbles in 4-bit mode or for the delays between RS and E. Also all the datasheets are stupidly inconsistent about the "initialize-by-instruction" sequence, which is required for 4-bit mode. The only way I could get this module working reliably, was to raise RS (regardless of need) for 2-4µs (then lower it for cmds), then raise E, put out the first nibble, lower E (which is the trigger for the hd44780), then wait 50µs, raise E, put the second nibble on the lines, lower E and again wait 50µs. According to the specs, neither of the 50µs should have been required. I tried dozens (literally) of "correct" ordering and timing variations, but nothing (or at least nothing faster) would work consistently. Also, the "dummy" raise/lower of RS seemed to help the synchronisation; without that I'd need to wait a lot longer between triggers to get it working. 8-bit mode is simpler, but even there this particular module would require 50µs between triggers.

That being out of the way, I had reliable output! printf-debugging, here we go (well, dump-hex-as-ascii, and at the expense of 100µs per cmd/char and the resulting screwup of any timing-sensitive ops).

Building Dervish

Here is the schematic; not complicated at all. One PIC16F628, one MAX232, one NPN transistor, one 10k pot and a few resistors, a buzzer, an infrared receiver IC, some machined-pin header strips and sockets and a bunch of cabling salvaged from old floppy ribbon cables.

A picture of the late stages of the testbed. As you can see there is already some output, we have a few diagnostic leds as well, and the infrared receiver was just about to be added.

I used simple protoyping board to build the circuit, and made good use of the PICs in-circuit programming capability (that's what the extra sockets in the last photo are for).

The housing is an old floppy box, and the easy-open nature of that box was really useful during the long debugging sessions. The power for dervish I simply vampired off the usb-to-serial converter. It certainly does not conform to the USB standard as the converter doesn't know or signal the computer that it sucks about 300mA when the backlight is on but the Netvista doesn't care.

Software

The PC/Netvista-side of things uses LCDproc to drive the LCD, with a small patch to add support for dervish as a serial interface driver. Here's the patch adding dervish support. It's against 0.5.1-3 but should apply cleanly to any 0.5.1 version.

Music is produced by MPD, and the user interface is provided by my home-grown perl application called "dwim" (which stands for "do what i mean!". Yes, the app is a tad idiosyncratic). Dwim can either read from irw (part of the lirc infrared control app) or straight from dervish. The irw mode is cruder but more reliable than any of the perl-to-lirc interfaces.

Dwim needs a few perl modules, most of which are in debian. Audio::MPD is not but it's available from CPAN. Here's the dwim tarball. You'll need to adjust one of the config files to match your own remote. Irrecord is useful for conveniently examining a remote.

Here's the dervish tarball; it consists of the main dervish.asm which you will need to adjust to match your remote's timing (look for ZEROTMR), and two support include files: one for multiple FIFOs and one for delay routines. If your remote does not send 16 fixed bits and 16 variable ones, then you'll need to adjust the code following the handle_ir label to save and send all four bytes. (If your remote does not use the NEC protocol, dervish will need major modifications and you're on your own.)

And that's it. Have fun sharing, mangling, modifying my code! Here are some final photos of dervish and dwim in action.