Daily Archives: March 12, 2010

A short history of the Midi filer project from floppy disk to 8GB SD

This is why I never blog. I get so involved with projects, there is never anytime for anything anywhere. Recently I set about to salvage a project from 2003. If I had been smart I would have blogged the whole thing. Who was to know the future.

Many years ago, or is it really decades ago, I started a project to read MIDI files from floppy disks. As luck would have it, someone actually wanted to pay me to do this. So off to Kentucky I went. The project was a disaster. Many people claimed ownership and the funding failed. After three to five years of development, the shipping date was always 6 months away. In the end they shot the horse.

In the middle of these near fist fights I was able to get the prototype unit working, which read Floppy disks in the popular MIDI formats, Sandard type 0 as well as the Yamaha and Pianodisc variations. This technology I was given as the partner I direclty dealt with wanted to have nothing more to do with it.

One of the other partners suggested I continue with the design. Several clients indicated an interest in another source of midi filers. I was given the specs for the Yamaha MDF-3 as a guideline. I had envisioned a system that would read the compact flash cards. This was actually suggested by my dad.

When I first took the project back to California, I set about to extend it to read Compact flash cards. The main controller was a DS80C320 chip. This was attached to a PSD and 128KB of external memory. Much of the issue the prior developers had, was getting enough data pins out of the system. To fix this a separate Atmel processor was used to scan a button matrix.

My first effort was to improve the button matrix. This I moved to a slightly larger processor (Still Atmel.) This was actually made. I was not given any budget for new main boards. These were large with a lot of dead space on them. I managed to make these things work. The main code was in C, with the button scan in ASM. While the 80C320 has two serial ports, these have to use the same baud rate, which was determined to be the MIDI baud rate.

The little mega 4433 ran circles around the 20mhz 80C320. It looked like a full midi parser would fit. I could greatly reduce the cost by 2/3 if I went only with the atmel chip doing everything. Atmel even made one with the same pin layout as the 80C320. The choice was further defined, when the first partner demanded all his hardware back. The second partner was still supportive, So I returned the hardware to the first and re-designed the whole thing using only Atmel. Only the 128KB memory remained. The second partner continued the funding.

I set a goal to have a dozen units ready for the next MBSI meeting. It was a mad dash. The boards were ready in time, but the program still had issues. I put some photographs from this into the Gallery here. I was surprised to see the corner of the old Overture monster behind my then state of art unit. I was over confident and did not include a debug port. I also made a mistake on the LCD controls, which caused a glitch every time the display was written. I also learned a lot about PC board layout. Fortunatly there are some good online resources such as the EDA tech forum and the conferences and daily updates I get from the UP media group. Check out the Update Newsletter I need to give them a plug every now and then.

Three months after the MBSI meeting a few units were shipped. These worked well. Over time I extended the system to read SMF1 files as well as SMF0 files. By the time I had this working floppy disks were dead. I was also deep into rebuilding the Caliola. The project was such a disaster I wound up out of cash and back living in my parents basement. No more trips to Swizerland.

I still wanted a compact flash reader. I figured to use the file parsing code written for floppy disks. Reading the CF flash card was easy. I quickly reduced the design to fit on a small proto board. The dozen or so unsold floppy units still a bit of a bother. I am a backroom engineer. The LCD display was too big for the case, so it sat on top. At this point, I did not want to spend any more on the floppy box. It worked, but did not look pretty. I have no idea what ever happened to the 5 units I sold. The users never complained, and indicated that the boxes were functional and worked.

I was working with Spencer Chase on improvments to the e-valves. Spencer assisted with the development and the 3 units, of the CF card versions sold quickly. One is in regular use, and has been for a number of years. I took back the Spencer’s to upgrade and it has taken some years to get back to it. I have no idea what happened to the second unit.

In the image gallery, one of these “Toaster” units is shown. I found some old photographs, one which shows the large floppy board next to the “mini” unit for reading Compact flash cards. I could not afford a FAT32 card and the small FAT12 cards worked funny. The solution was easy, to add one more address line and read the disk sector as a memory map. Easy to debug. set the registers and the AVR studio debugger would show the sector. Bit dicey though as the bytes have to be read in order

Other projects intervened, and I pretty much decided I had done with MIDI filers. As with all dead end projects, I figured to revive them somehow and sell off what assets I could in time. The little CF readers are cute, and can do a lot. I was more than happy to revive and make a new batch for Larry Doe and the Keystone. The catch was that MIDI in was needed.

So the code was completely re-written. After some extensive testing, a unit was shipped. This had a problem with the system resetting during the perforation. While updating the companion MIDI “primary” controller board, I discovered and issue where input collisions happen in the interrupt even though the flag is technically clear.

Now for the pea soup course. In early 2009 I learned about the reprap project, which is a 3D printer. As I was installing a Theater organ and building a 20 note busker, I did not have the time to look into 3D printing. A desirable device, when one wants pretty cases for electronics. Then an old friend wanted a “primary” board. Sell the rest on eBay, he suggested. Even that was two years ago.

Back to the Reprap. The hardware on this is almost identical to my Midi filer, and the MIDI “Primary” boards. Same processor. There is also someone in Germany selling a system using SD/MMC cards. The boards are full of SD “How to” postings.

Is SD really that simple? Compact flash was trivial, but it needs a lot of I/O pins and a large layout on the board. SD on the other hand promises to only use the same pins as the system program interface. On the Floppy midi filer, I used some of these pins for the key matrix. Could one be modified to use SD instead of a floppy?

I built a reprap motherboard with SD. Even started to convert my 8K fat reader from MIDI to G-code. Yes, Sophie, I am actually getting to things that happened this month! So I went back to the old unsold floppy filer boards. That lack of a debug port really hurts. If I had 17 seconds to do over, I would have put that footprint down. The CF boards have a debug port, But the Ice200 broke, and the mega162 put the JTAG on the same pins I use for the CF DMA. Another dead project revived. Using stuff about the house I made a USB adapter. Ice200 comes back to life.

Many time I have attempted to fix the issue with the LCD on the filer board not being memory mapped. Backing up a bit, I went over the logic again. On the CF version I used a 74LS138 as I should have in the first place. With only two selects on the filer and one a select high, I went all cutesy and did it with logic gates. The problem was I was always one inverter short.

I fixed the filer, by adding in gate to OR and invert the readwrite line. This was all and well, apart from that the memory mapped all over the lower 32K with duplicate windows everywhere. I never liked this fix. it did not work with the mega162, which is needed if one wants to write MIDI as well as read MIDI.

A few days ago, I figured it out. By combining the read write invert with the address invert, I am able to fit the needed logic into the two existing chips on the filer. I also decided to move the selection back down to the lower memory. In debug mode the m162 only has a tiny window for memory mapping. Even with the current fix, the DMA takes 256 bytes per address. I really do not like switching on the latched memory buss pins. The 128K memory is sensitive as it is.

Now it is possible to write fat code that fits into 8K and in C too. At least for FAT 16 cards. On the other hand I have the 128K boards and I really need to sell them off to make the world better and people happy. So I set out to rework the filer boards. The first was to make the CF filer board the same as the floppy filer. This to test the LCD logic.

A new issue arose, The button matrix, which dates back to the old 4433 processor, used the system program pins. The LED indicators were always an option. So these could be done away with. This left two lines to be used by the matrix. Two lines also needed to detect Card insertion and write protect. As the chip is full, no writing is possible, so that can be ignored. Detecting the card is done at power on, so this can be a poll on the card ready bit. — Then an epiphany. Play is often combined with pause, so that frees a location in the key matrix. This is perfect for placing the write switch such an input can be shared with the record logic. It is also possible to add the card detect to the key matrix. A problem, that has bothered for some time suddenly disappears like an illusionist trick. The problem never existed in the first place.

So the lashup was complete. The logic worked on the mega 162 board. Fixing the Ice200 and connecting the SD card to the system program pins. The magician stands ready. Nothing up the sleeves he says. And the directory loads into the external Sram. Changes are made, The 8GB card comes ready. The directory loads. One single cluster, 32K fills one bank of the 128K external memory.

What a sight. The entire contents of the first developments Hard drive was only two gigabytes. Here it is loadable into what was originally a system for reading tiny little floppy disks. Every midi file I have takes a small fraction of this card. It is a lot of button pushes to get to any one of them, at least I implemented hierarchical directories.

Of course it is only a magicians trick, The code sees the 32K sector wrap to zero and the system crashes. Sill the 8GB are readable. As all ways there is more to do. Sectoring logic to be written so that one can step through hundreds, if not thousands of files. The SD code is also a bit more complex than CF, I had to switch off FAT12 and Piano disk support to add the init code in. The floppy and CF access was DMA, so a few bytes to set a register and then read the DMA window. SD requires almost 64 bytes of tables to be saved which detail the commands Used for card type detection. While a lot of these are zeros, it takes two bytes to load them into a register, where as if stored as a table, the read loop takes only a handful of bytes.

There is still much more to do, But the main issues are resolved, The old boards awakened from sleep. Not Zombies or Gouhls, these boards were never finished, the unmade, or unborne fairies of the aiether. Soon to come forth, with new dresses, and fly on the desires and dreams of all who want a dependable and reliable way of saving more MIDI music than can be had at any cost.