How To Geek article on floppy data recovery
There are a number of newer tools, for imaging of disks, for preservation.
I've been closely watching this one [called Fluxengine] http://cowlark.com/fluxengine/index.html
The author is working on support for a lot of common formats, and additionally, a number of the 80s-90s dedicated word processor formats. It's a simple and rather nifty solution.
Also, of course, it's 100% opens source hardware and software, which always makes me happy! - Alex
This reply is long, because I'm arguing in opposition, and that means I have to make a case about, and explain about. But I'll save some people, some time. I"m going to fuss about these microcontrollers becoming obsolete. If you don't care about that, save time and stop reading here. ----------- Well, I read some of the support information on the Fluxengine. Also I looked recently, at the Greaseweasel. Both these methodologies use some kind of $10 - $20 commodity microcontroller; and that's their problem in my opinion. Those interested can look up both and size them up, so I won't talk about details and features. The Greaseweasel uses a STM32 microcontroller chip on a board called "a Blue Pill". Problem with that (in my opinion) is, that chip isn't just one chip produced identically. There's knockoffs and fakes, that is less-featured chip versions remarked or not remarked. And since that "product" is more than five years old apparently, it's somewhat out of favor (that's my conclusion, it seems hard to find except on eBay). Bottom line, it's not now produced by bigger named hobby or industrial companies; they all seem to come from ebay/Amazon small producers. At literally a few dollars each, the only way to make more money from these, is to use cheaper components or reduced feature components. For instance: the 3.3V regulators fail because they are undersized for power. When I looked around Amazon for these, the reviews per seller were all "these are fakes because". This is not just Herb Johnson's problem. The Greaseweasel support page on github, has a section on how to recognize fakes with performance and visual tests. It does not have, a section on where to buy these. Apparently one has to join a Facebook group to get answers and further support. I won't do that, but someone offered to look there for me, to see if they discuss suppliers. No word yet on that. "Why is this a problem?" Specifically: some of the knockoff boards fail, apparently, because their 3.3V regulators burn up. Others, lack some useful features or even critical features (enough memory for instance). And in the longer term, the ability to obtain these items at all, may end. In between, the means to (say) assemble the code or download the code into the device, may become obsolete or unavailable. In general, the problem with end-of-production items is this: Because you and I are dealing with *vintage computing*. We deal with technology that's *decades* old, not last month. When products fall out of production, how do you get more? When things break, how do you fix them? When you can't get the programming debugging tools from the manufacturer (or producers), you have to find old archives of them. In the real world, this stuff is normal. Things don't persist. You just move on to what's current and throw out the old. We in vintage computing, can't do that by definition. OK? If you disagree, stop reading here, sorry I wasted your time fussing about stuff getting old. ----------------------- I described at length the problem with Greaseweasel hardware, because I'll make the same argument as a *prediction* for Fluxengine. It uses a "Cypress PSoC5LP CY8CKIT-059 development board" for $10. That's apparently some ARM processor with FPGA (programmable logic). At this point, Cypress (a leading chip producer) is selling them direct. Let's check... yep, $10 from cypress.com or $20 from major distributors, plus shipping. One expects delays due to Coronavirus disrupting manufacture and shipping except from USA stocks. I have another objection, to programmable logic. All those PLD's are unique by model and brand. They simply don't stay in production for many years. Worse, the software tools needed to program those specific devices, become unsupported in several years. There's no standards for such tools and software, other than some data formats, so you are stuck with that brand's tool-set. This is, of course, an intended consequence. Figure that out. I checked the datasheets for the product: they date from early 2015. That's about the same age as the "Blue Pill". But this is still produced (or at least offered as current) by Cypress. Also: since this has a very particular programmable-logic set-of-features, it may be hard to knock-off the chip by an unlicensed company. That's my guess. So: looking at Amazon, they only sell a book. Looking at eBay, There's a couple units from the UK for $28-$33 with shipping. I'd not buy from Europe right now... a few are US sold but essentially it's left-over stocks. So: again in my opinion: while this product is still in manufacturer's production, it suffers from being too proprietary - Cypress will lose interest someday soon. Also, there's that unique-programming problem, figuring out how-to-program those PLA's. If you aren't gonna reprogram one of these puppies, I guess that doesn't matter, until there's a Fluxengine update you can't update. And in closing .... The reason I'm familiar with this kind of "these go obsolete" arguments, is because I've seen this show before. Over the last few decades, various modern means to read ancient floppy diskettes have been produced. They have become in-favor and discussed and distributed. Then the developers lose interest; then the users lose interest; and they go out of production and use. Over geologic time (several years), these things simply come and go. So the vintage computers remain, but these modern devices do not. That's about as simple as I can put the argument. Regards, Herb Johnson -- Herbert R. Johnson, New Jersey in the USA http://www.retrotechnology.com OR .net preserve, recover, restore 1970's computing email: hjohnson AT retrotechnology DOT com or try later herbjohnson AT retrotechnology DOT info
However, consider that, over time, it may matter less and less as a larger volume of old media gets archived. Maybe it’s ok that these devices are somewhat ephemeral. Just food for thought. On Fri, May 8, 2020 at 11:45 AM Herb Johnson via vcf-midatlantic < vcf-midatlantic@lists.vcfed.org> wrote:
There are a number of newer tools, for imaging of disks, for preservation.
I've been closely watching this one [called Fluxengine] http://cowlark.com/fluxengine/index.html
The author is working on support for a lot of common formats, and additionally, a number of the 80s-90s dedicated word processor formats. It's a simple and rather nifty solution.
Also, of course, it's 100% opens source hardware and software, which always makes me happy! - Alex
This reply is long, because I'm arguing in opposition, and that means I have to make a case about, and explain about. But I'll save some people, some time. I"m going to fuss about these microcontrollers becoming obsolete. If you don't care about that, save time and stop reading here.
-----------
Well, I read some of the support information on the Fluxengine. Also I looked recently, at the Greaseweasel. Both these methodologies use some kind of $10 - $20 commodity microcontroller; and that's their problem in my opinion. Those interested can look up both and size them up, so I won't talk about details and features.
The Greaseweasel uses a STM32 microcontroller chip on a board called "a Blue Pill". Problem with that (in my opinion) is, that chip isn't just one chip produced identically. There's knockoffs and fakes, that is less-featured chip versions remarked or not remarked. And since that "product" is more than five years old apparently, it's somewhat out of favor (that's my conclusion, it seems hard to find except on eBay).
Bottom line, it's not now produced by bigger named hobby or industrial companies; they all seem to come from ebay/Amazon small producers. At literally a few dollars each, the only way to make more money from these, is to use cheaper components or reduced feature components. For instance: the 3.3V regulators fail because they are undersized for power. When I looked around Amazon for these, the reviews per seller were all "these are fakes because".
This is not just Herb Johnson's problem. The Greaseweasel support page on github, has a section on how to recognize fakes with performance and visual tests. It does not have, a section on where to buy these. Apparently one has to join a Facebook group to get answers and further support. I won't do that, but someone offered to look there for me, to see if they discuss suppliers. No word yet on that.
"Why is this a problem?" Specifically: some of the knockoff boards fail, apparently, because their 3.3V regulators burn up. Others, lack some useful features or even critical features (enough memory for instance). And in the longer term, the ability to obtain these items at all, may end. In between, the means to (say) assemble the code or download the code into the device, may become obsolete or unavailable.
In general, the problem with end-of-production items is this: Because you and I are dealing with *vintage computing*. We deal with technology that's *decades* old, not last month. When products fall out of production, how do you get more? When things break, how do you fix them? When you can't get the programming debugging tools from the manufacturer (or producers), you have to find old archives of them.
In the real world, this stuff is normal. Things don't persist. You just move on to what's current and throw out the old. We in vintage computing, can't do that by definition. OK? If you disagree, stop reading here, sorry I wasted your time fussing about stuff getting old.
-----------------------
I described at length the problem with Greaseweasel hardware, because I'll make the same argument as a *prediction* for Fluxengine. It uses a "Cypress PSoC5LP CY8CKIT-059 development board" for $10. That's apparently some ARM processor with FPGA (programmable logic). At this point, Cypress (a leading chip producer) is selling them direct. Let's check... yep, $10 from cypress.com or $20 from major distributors, plus shipping. One expects delays due to Coronavirus disrupting manufacture and shipping except from USA stocks.
I have another objection, to programmable logic. All those PLD's are unique by model and brand. They simply don't stay in production for many years. Worse, the software tools needed to program those specific devices, become unsupported in several years. There's no standards for such tools and software, other than some data formats, so you are stuck with that brand's tool-set.
This is, of course, an intended consequence. Figure that out.
I checked the datasheets for the product: they date from early 2015. That's about the same age as the "Blue Pill". But this is still produced (or at least offered as current) by Cypress. Also: since this has a very particular programmable-logic set-of-features, it may be hard to knock-off the chip by an unlicensed company. That's my guess.
So: looking at Amazon, they only sell a book. Looking at eBay, There's a couple units from the UK for $28-$33 with shipping. I'd not buy from Europe right now... a few are US sold but essentially it's left-over stocks.
So: again in my opinion: while this product is still in manufacturer's production, it suffers from being too proprietary - Cypress will lose interest someday soon. Also, there's that unique-programming problem, figuring out how-to-program those PLA's. If you aren't gonna reprogram one of these puppies, I guess that doesn't matter, until there's a Fluxengine update you can't update.
And in closing ....
The reason I'm familiar with this kind of "these go obsolete" arguments, is because I've seen this show before. Over the last few decades, various modern means to read ancient floppy diskettes have been produced. They have become in-favor and discussed and distributed. Then the developers lose interest; then the users lose interest; and they go out of production and use. Over geologic time (several years), these things simply come and go.
So the vintage computers remain, but these modern devices do not. That's about as simple as I can put the argument.
Regards, Herb Johnson
-- Herbert R. Johnson, New Jersey in the USA http://www.retrotechnology.com OR .net preserve, recover, restore 1970's computing email: hjohnson AT retrotechnology DOT com or try later herbjohnson AT retrotechnology DOT info
On 5/8/2020 12:04 PM, Dean Notarnicola wrote:
However, consider that, over time, it may matter less and less as a larger volume of old media gets archived. Maybe it’s ok that these devices are somewhat ephemeral. Just food for thought.
If I accept that proposition, Dean: vintage computing may matter less and less, as a larger volume of information and actual computers get archived in collections and on Web pages. Maybe it's OK that everything is ephemeral - including you and me. That's called "carrying an argument to its logical conclusions". Or to extreme conclusions, you decide. The weaker response, is that many people have decided that floppy diskettes are so obsolete, one should simply archive their contents and avoid their use. In that case, the fact that archival mechanisms come and go - these various microcontrollers - doesn't matter either. One finds a means to archive or recover; one does the recovery; one moves on. I have a few Web pages, about the efforts needed to "archive", and then recover and put to use, various inconvenient floppy disk formats; such as M2FM or MMFM and Intel Multibus system disks. Eventually, the recovery was performed by *actual period hardware and Intel systems*. Your mileage may vary, regarding your favorite vintage systems. Again, I call out the difference between vintage computing as acts of preservation; and modern computing as an "ephemeral" activity where only the data persists (if that). So let's run emulators and go home, job done. (shrug) It's a matter of choices and consequences. Regards, Herb Johnson
On Fri, May 8, 2020 at 11:45 AM Herb Johnson wrote:
This reply is long, because I'm arguing in opposition, and that means I have to make a case about, and explain about. But I'll save some people, some time. I"m going to fuss about these microcontrollers becoming obsolete. If you don't care about that, save time and stop reading here.
-- Herbert R. Johnson, New Jersey in the USA http://www.retrotechnology.com OR .net preserve, recover, restore 1970's computing email: hjohnson AT retrotechnology DOT com or try later herbjohnson AT retrotechnology DOT info
I agree that these are two different things; preservation of data and storage formats lives in a separate domain from preserving vintage hardware, but you may argue that one helps the other and keeps these systems usable after mechanical components have failed beyond reasonable repair. On Fri, May 8, 2020 at 12:38 PM Herb Johnson <hjohnson@retrotechnology.info> wrote:
On 5/8/2020 12:04 PM, Dean Notarnicola wrote:
However, consider that, over time, it may matter less and less as a larger volume of old media gets archived. Maybe it’s ok that these devices are somewhat ephemeral. Just food for thought.
If I accept that proposition, Dean: vintage computing may matter less and less, as a larger volume of information and actual computers get archived in collections and on Web pages. Maybe it's OK that everything is ephemeral - including you and me. That's called "carrying an argument to its logical conclusions". Or to extreme conclusions, you decide.
The weaker response, is that many people have decided that floppy diskettes are so obsolete, one should simply archive their contents and avoid their use. In that case, the fact that archival mechanisms come and go - these various microcontrollers - doesn't matter either. One finds a means to archive or recover; one does the recovery; one moves on.
I have a few Web pages, about the efforts needed to "archive", and then recover and put to use, various inconvenient floppy disk formats; such as M2FM or MMFM and Intel Multibus system disks. Eventually, the recovery was performed by *actual period hardware and Intel systems*. Your mileage may vary, regarding your favorite vintage systems.
Again, I call out the difference between vintage computing as acts of preservation; and modern computing as an "ephemeral" activity where only the data persists (if that). So let's run emulators and go home, job done. (shrug) It's a matter of choices and consequences.
Regards, Herb Johnson
On Fri, May 8, 2020 at 11:45 AM Herb Johnson wrote:
This reply is long, because I'm arguing in opposition, and that
means I
have to make a case about, and explain about. But I'll save some people, some time. I"m going to fuss about these microcontrollers becoming obsolete. If you don't care about that, save time and stop reading
here.
-- Herbert R. Johnson, New Jersey in the USA http://www.retrotechnology.com OR .net preserve, recover, restore 1970's computing email: hjohnson AT retrotechnology DOT com or try later herbjohnson AT retrotechnology DOT info
The subject of floppy drives and diskettes, is of great interest to me. One can Web search my domain and find many Web pages about them, and about vintage floppy controllers too. Some people find my site useful in that way. So I had something to say. I think I mis-interpreted Dean's comments, as some kind of established opinion, and not just his expression of his particular interests and preferences. Sorry if I over-responded as if to make a serious argument out of them. I'm sure many would agree with his considerations. But I'm not taking a vote, to establish my interests or to draw my conclusions. Thanks, Dean, for your responses and your opinions. You need not share my particular interests of course. If you find information that's contrary to facts and propositions I've made, I'd be interested in hearing about them. That applies to anyone else of course. And I welcome a strong counter argument, I'd be informed, maybe others would be. But I don't insist on consensus or agreement; or pretend my views are established in that way. I hope my facts are correct, the tech I mention is correctly stated. The rest are my considerations and others may have different ones. Regards, Herb Johnson On 5/8/2020 12:43 PM, Dean Notarnicola wrote:
I agree that these are two different things; preservation of data and storage formats lives in a separate domain from preserving vintage hardware, but you may argue that one helps the other and keeps these systems usable after mechanical components have failed beyond reasonable repair.
On Fri, May 8, 2020 at 12:38 PM Herb Johnson wrote:
On 5/8/2020 12:04 PM, Dean Notarnicola wrote: > However, consider that, over time, it may matter less and less as a > larger volume of old media gets archived. Maybe it’s ok that these > devices are somewhat ephemeral. Just food for thought.
If I accept that proposition, Dean: vintage computing may matter less and less, as a larger volume of information and actual computers get archived in collections and on Web pages. Maybe it's OK that everything is ephemeral - including you and me. That's called "carrying an argument to its logical conclusions". Or to extreme conclusions, you decide.
The weaker response, is that many people have decided that floppy diskettes are so obsolete, one should simply archive their contents and avoid their use. In that case, the fact that archival mechanisms come and go - these various microcontrollers - doesn't matter either. One finds a means to archive or recover; one does the recovery; one moves on.
I have a few Web pages, about the efforts needed to "archive", and then recover and put to use, various inconvenient floppy disk formats; such as M2FM or MMFM and Intel Multibus system disks. Eventually, the recovery was performed by *actual period hardware and Intel systems*. Your mileage may vary, regarding your favorite vintage systems.
Again, I call out the difference between vintage computing as acts of preservation; and modern computing as an "ephemeral" activity where only the data persists (if that). So let's run emulators and go home, job done. (shrug) It's a matter of choices and consequences.
Regards, Herb Johnson
> > On Fri, May 8, 2020 at 11:45 AM Herb Johnson wrote: > > This reply is long, because I'm arguing in opposition, and that means I > have to make a case about, and explain about. But I'll save some > people, > some time. I"m going to fuss about these microcontrollers becoming > obsolete. If you don't care about that, save time and stop reading here.
-- Herbert R. Johnson, New Jersey in the USA http://www.retrotechnology.com OR .net preserve, recover, restore 1970's computing email: hjohnson AT retrotechnology DOT com or try later herbjohnson AT retrotechnology DOT info
Herb, Your comments were taken in the spirit in which you intended. And you correctly surmised that my statement was merely my opinion. All aspects of vintage computing are of interest to me, as they enlighten us about how tech arrived to its current state, and so the instinct to support all efforts to preserve what was is strong in me. Gordon’s comment about the interface to older computers being invariable is spot on. There will hopefully always be those who look for new abs better ways to preserve the functionality of mechanical units that degrade with time, transparent to the original computers. I don’t think that’s mutually exclusive to the desire to preserve what was or that it will necessarily cause a mad rush to abandon hardware for emulators. - Dean On Fri, May 8, 2020 at 2:00 PM Herb Johnson <hjohnson@retrotechnology.info> wrote:
The subject of floppy drives and diskettes, is of great interest to me. One can Web search my domain and find many Web pages about them, and about vintage floppy controllers too. Some people find my site useful in that way. So I had something to say.
I think I mis-interpreted Dean's comments, as some kind of established opinion, and not just his expression of his particular interests and preferences. Sorry if I over-responded as if to make a serious argument out of them. I'm sure many would agree with his considerations. But I'm not taking a vote, to establish my interests or to draw my conclusions.
Thanks, Dean, for your responses and your opinions. You need not share my particular interests of course. If you find information that's contrary to facts and propositions I've made, I'd be interested in hearing about them. That applies to anyone else of course.
And I welcome a strong counter argument, I'd be informed, maybe others would be. But I don't insist on consensus or agreement; or pretend my views are established in that way. I hope my facts are correct, the tech I mention is correctly stated. The rest are my considerations and others may have different ones.
Regards, Herb Johnson
On 5/8/2020 12:43 PM, Dean Notarnicola wrote:
I agree that these are two different things; preservation of data and storage formats lives in a separate domain from preserving vintage hardware, but you may argue that one helps the other and keeps these systems usable after mechanical components have failed beyond reasonable repair.
On Fri, May 8, 2020 at 12:38 PM Herb Johnson wrote:
On 5/8/2020 12:04 PM, Dean Notarnicola wrote: > However, consider that, over time, it may matter less and less as a > larger volume of old media gets archived. Maybe it’s ok that these > devices are somewhat ephemeral. Just food for thought.
If I accept that proposition, Dean: vintage computing may matter less and less, as a larger volume of information and actual computers get archived in collections and on Web pages. Maybe it's OK that everything is ephemeral - including you and me. That's called "carrying an argument to its logical conclusions". Or to extreme conclusions, you decide.
The weaker response, is that many people have decided that floppy diskettes are so obsolete, one should simply archive their contents and avoid their use. In that case, the fact that archival mechanisms come and go - these various microcontrollers - doesn't matter either. One finds a means to archive or recover; one does the recovery; one moves on.
I have a few Web pages, about the efforts needed to "archive", and then recover and put to use, various inconvenient floppy disk formats; such as M2FM or MMFM and Intel Multibus system disks. Eventually, the recovery was performed by *actual period hardware and Intel systems*. Your mileage may vary, regarding your favorite vintage systems.
Again, I call out the difference between vintage computing as acts of preservation; and modern computing as an "ephemeral" activity where only the data persists (if that). So let's run emulators and go home, job done. (shrug) It's a matter of choices and consequences.
Regards, Herb Johnson
> > On Fri, May 8, 2020 at 11:45 AM Herb Johnson wrote: > > This reply is long, because I'm arguing in opposition, and that means I > have to make a case about, and explain about. But I'll save some > people, > some time. I"m going to fuss about these microcontrollers becoming > obsolete. If you don't care about that, save time and stop reading here.
-- Herbert R. Johnson, New Jersey in the USA http://www.retrotechnology.com OR .net preserve, recover, restore 1970's computing email: hjohnson AT retrotechnology DOT com or try later herbjohnson AT retrotechnology DOT info
So, Gordon S. writes, something like this. He buys-in, that these floppy-recover solutions, are made from currently available hardware, but that hardware goes away over time. So, he says, why not implement on widely available and well-sourced hardware? The challenge as he put it is: "The only real irritant, then, becomes working out how to cheaply do whatever it is these other projects are using FPGAs for. Those are the parts that get updated whenever someone works out a better technique, but... as a general rule, the interface to whatever vintage hardware is more or less invariant, correct? That would mean the alterable functions can be carried out in software rather than in (redefinable) hardware. Flash updates are still flash updates, regardless of whether they contain computer programs or FPGA configuration files." "Can anyone poke holes in this? - Gordon S". Well, I reject the limitation "cheaply". That constrains any modern solutions to the hell of rapid obsolescence and proprietary solutions. That's the current strategy. I'm being blunt but brief. At $10, $20 single-board single-chip solutions, how does any hobbyist outdesign *that* with more persistent hardware? The vintage hardware interface, is either a floppy drive for data recovery; or a floppy drive controller, for diskette emulators (not discussed but the complimentary device). Well, po-TAE-toe, po-TAUGHT-oe, to a first order. That's not the hard part. On FPGAs, and the hard part. Turns out the support page for the Fluxengine, is pretty informative about this and other arguments for alternative technologies. After five years, that's already been defended (but I only read the gloss). The defense is, it comes down to the design practice, of massive oversampling of floppy data streams, to extract arbitrary-rates of binary data relatively fast. Go to the site for details. So: one ether borrows cheep ARM-class devices that do that, or cheep FPLA devices which do that, or both. Because of that "cheap" constraint. Shrug. In what used to be the real world, floppy disk controllers were dedicated microprocessors with fast bit-serial/parallel hardware. Read the early floppy controller FDC chip data sheets about that. They ran at defined data rates - so write hardware was easier. Reading hardware involved some skew, so one needed some tricks to sync-sample the read data. We call that data, single/double density, FM/MFM. Or the Apple mind-trick of the Apple II (later Mac) "Woz machine", a programmed-logic device (!) to perform 3-of-5 encoding and decoding. Or, some custom hardware and software, on other computers of vintage. At one time, ALL floppy controllers were custom logic. (Search retrotechnology.com ) Now, in some sense, the custom logic is in the FPGA. This might make a case for preserving vintage computers (he he ...). To replace all that, the cheat is to sample like hell and let post-1990's computers sort out the results in software. Fair enough. I appreciate Gordon's point, that functionality in modern devices is a matter of programming, and reprogramming. That's fine, for those who enjoy modern embedded programming. But when the hardware to be programmed changes "too much"; the old software becomes obsolete. Yesterday's FPLAs can't be programmed with today's FPLA tools. The codes are changed too. The binaries are useless, even proprietary. That was my point! Subtle comment by Gordon: "We already know what stays around, or at least remains well-documented and well-supported." Trick question. The trick answer #1 is "that's the same thing". Answer #2 is "well, stuff persists until it breaks; if you can't fix it you replace it". Sort of the short version of the argument. Now, the software to RUN the platform around the FPLA, could in principle be standardized. For big-enough embedded systems, that's fine, they are application-specific not implementation-specific (kinda). But this is just a *floppy controller*. There *is no application*. Nobody is running MS-DOS or Linux on these puppies. They are driven-by something, the something steers it around, and that's all. And by the way? The 1990's solution - the Catweasel, an ISA or PCI bit-sampling controller on PC's? Last I checked, there were five-six "installation solutions" for running that puppy. Surprisingly, each developer came up with their own software to drive it. Go figure. I don't know how much that happens, with 21st century $20 microcontroller solutions; the Arduino platform is pretty established; the RaspPi, somewhat the same. Then each other device-family, has their manufacturer's development engine software (to keep you from using another product, he he). I made that point before. So: walk through this with me. What floppy control hardware is pretty much standardized, available, and stable; year after year and even into decades? and what software to support what software to drive it, has those qualities? Answer: XT/AT/386 class PC's and MS-DOS. And that's why solutions like "Imagedsk" / MS-DOS / ISA floppy controllers / IMG files just keep on workin'. But they don't work for 68K Mac disk formats, Apple II disk formats, etc. etc. That's why bit-sampling general solutions emerged in the 1990's, from Compaticard to Catweasel. And you know, maybe I'll think of a 1985 solution to the problem for those reasons. But it's harder to design PCI cards; many people don't have computers with PCI slots; USB is the "bus" of choice; and there's those dammed $10, $20, $30 stick-puters. They are the black-holes of design-your-own-devices. But the question of the day, was about a $10 solution. It's an amazing bit of application, that FluxEngine; as is/was the GreaseWeasel. But it's like other $10 embedded solutions, with all the faults I mentioned and alternatives Gordon suggests. One can work around these, because in the end, nobody is gonna die because a $10 embedded stick coughs and sputters, or can't be found. There's no mission-critical task in play that matters. They will come and go, as do supporters of some application like read-a-floppy. Sorry for the long lecture, I got a real question and it took time to answer it. Hope this did that. regards, Herb Johnson -- Herbert R. Johnson, New Jersey in the USA http://www.retrotechnology.com OR .net preserve, recover, restore 1970's computing email: hjohnson AT retrotechnology DOT com or try later herbjohnson AT retrotechnology DOT info
I'm watching the development of the AppleSauce controller, and it's accompanying software, very closely. I've had a chance to use the device, at KansasFest, and it's a very well-engineered device. Since they have expanded the imaging capabilities beyond just the Apple II, it's of much more interest to me. - Alex On Fri, May 8, 2020 at 6:45 PM Herb Johnson via vcf-midatlantic < vcf-midatlantic@lists.vcfed.org> wrote:
So, Gordon S. writes, something like this. He buys-in, that these floppy-recover solutions, are made from currently available hardware, but that hardware goes away over time. So, he says, why not implement on widely available and well-sourced hardware? The challenge as he put it is:
"The only real irritant, then, becomes working out how to cheaply do whatever it is these other projects are using FPGAs for. Those are the parts that get updated whenever someone works out a better technique, but... as a general rule, the interface to whatever vintage hardware is more or less invariant, correct? That would mean the alterable functions can be carried out in software rather than in (redefinable) hardware. Flash updates are still flash updates, regardless of whether they contain computer programs or FPGA configuration files."
"Can anyone poke holes in this? - Gordon S".
Well, I reject the limitation "cheaply". That constrains any modern solutions to the hell of rapid obsolescence and proprietary solutions. That's the current strategy. I'm being blunt but brief. At $10, $20 single-board single-chip solutions, how does any hobbyist outdesign *that* with more persistent hardware?
The vintage hardware interface, is either a floppy drive for data recovery; or a floppy drive controller, for diskette emulators (not discussed but the complimentary device). Well, po-TAE-toe, po-TAUGHT-oe, to a first order. That's not the hard part.
On FPGAs, and the hard part. Turns out the support page for the Fluxengine, is pretty informative about this and other arguments for alternative technologies. After five years, that's already been defended (but I only read the gloss).
The defense is, it comes down to the design practice, of massive oversampling of floppy data streams, to extract arbitrary-rates of binary data relatively fast. Go to the site for details. So: one ether borrows cheep ARM-class devices that do that, or cheep FPLA devices which do that, or both. Because of that "cheap" constraint. Shrug.
In what used to be the real world, floppy disk controllers were dedicated microprocessors with fast bit-serial/parallel hardware. Read the early floppy controller FDC chip data sheets about that. They ran at defined data rates - so write hardware was easier. Reading hardware involved some skew, so one needed some tricks to sync-sample the read data. We call that data, single/double density, FM/MFM. Or the Apple mind-trick of the Apple II (later Mac) "Woz machine", a programmed-logic device (!) to perform 3-of-5 encoding and decoding. Or, some custom hardware and software, on other computers of vintage.
At one time, ALL floppy controllers were custom logic. (Search retrotechnology.com ) Now, in some sense, the custom logic is in the FPGA. This might make a case for preserving vintage computers (he he ...).
To replace all that, the cheat is to sample like hell and let post-1990's computers sort out the results in software. Fair enough.
I appreciate Gordon's point, that functionality in modern devices is a matter of programming, and reprogramming. That's fine, for those who enjoy modern embedded programming. But when the hardware to be programmed changes "too much"; the old software becomes obsolete. Yesterday's FPLAs can't be programmed with today's FPLA tools. The codes are changed too. The binaries are useless, even proprietary. That was my point!
Subtle comment by Gordon: "We already know what stays around, or at least remains well-documented and well-supported." Trick question. The trick answer #1 is "that's the same thing". Answer #2 is "well, stuff persists until it breaks; if you can't fix it you replace it". Sort of the short version of the argument.
Now, the software to RUN the platform around the FPLA, could in principle be standardized. For big-enough embedded systems, that's fine, they are application-specific not implementation-specific (kinda). But this is just a *floppy controller*. There *is no application*. Nobody is running MS-DOS or Linux on these puppies. They are driven-by something, the something steers it around, and that's all.
And by the way? The 1990's solution - the Catweasel, an ISA or PCI bit-sampling controller on PC's? Last I checked, there were five-six "installation solutions" for running that puppy. Surprisingly, each developer came up with their own software to drive it. Go figure.
I don't know how much that happens, with 21st century $20 microcontroller solutions; the Arduino platform is pretty established; the RaspPi, somewhat the same. Then each other device-family, has their manufacturer's development engine software (to keep you from using another product, he he). I made that point before.
So: walk through this with me. What floppy control hardware is pretty much standardized, available, and stable; year after year and even into decades? and what software to support what software to drive it, has those qualities? Answer: XT/AT/386 class PC's and MS-DOS. And that's why solutions like "Imagedsk" / MS-DOS / ISA floppy controllers / IMG files just keep on workin'.
But they don't work for 68K Mac disk formats, Apple II disk formats, etc. etc. That's why bit-sampling general solutions emerged in the 1990's, from Compaticard to Catweasel. And you know, maybe I'll think of a 1985 solution to the problem for those reasons. But it's harder to design PCI cards; many people don't have computers with PCI slots; USB is the "bus" of choice; and there's those dammed $10, $20, $30 stick-puters. They are the black-holes of design-your-own-devices.
But the question of the day, was about a $10 solution. It's an amazing bit of application, that FluxEngine; as is/was the GreaseWeasel. But it's like other $10 embedded solutions, with all the faults I mentioned and alternatives Gordon suggests. One can work around these, because in the end, nobody is gonna die because a $10 embedded stick coughs and sputters, or can't be found. There's no mission-critical task in play that matters. They will come and go, as do supporters of some application like read-a-floppy.
Sorry for the long lecture, I got a real question and it took time to answer it. Hope this did that.
regards, Herb Johnson
-- Herbert R. Johnson, New Jersey in the USA http://www.retrotechnology.com OR .net preserve, recover, restore 1970's computing email: hjohnson AT retrotechnology DOT com or try later herbjohnson AT retrotechnology DOT info
John has continued to push the software and hardware to its limits. He hopes to eventually get a windows client as well. It can currently do Flux images (a2r) of Apple II 5.25”, 3.5”, macintosh 3.5” 800k and 1.44mb disks. We have one for VCF and I have a couple including one of the prototype units since I was an alpha and beta tester. I’m not sure if the VCF one ever had the upgrade done, I’ll have to check it. Eric Rangell set it up originally. I know it should be able to do some other formats but I don’t recall if that’s been built into the software yet. I haven’t had an imaging station set up for awhile though I continue to follow his updates. Tony Sent from my iPhone
On May 9, 2020, at 6:12 PM, Alexander Jacocks via vcf-midatlantic <vcf-midatlantic@lists.vcfed.org> wrote:
I'm watching the development of the AppleSauce controller, and it's accompanying software, very closely. I've had a chance to use the device, at KansasFest, and it's a very well-engineered device. Since they have expanded the imaging capabilities beyond just the Apple II, it's of much more interest to me.
- Alex
On Fri, May 8, 2020 at 6:45 PM Herb Johnson via vcf-midatlantic < vcf-midatlantic@lists.vcfed.org> wrote:
So, Gordon S. writes, something like this. He buys-in, that these floppy-recover solutions, are made from currently available hardware, but that hardware goes away over time. So, he says, why not implement on widely available and well-sourced hardware? The challenge as he put it is:
"The only real irritant, then, becomes working out how to cheaply do whatever it is these other projects are using FPGAs for. Those are the parts that get updated whenever someone works out a better technique, but... as a general rule, the interface to whatever vintage hardware is more or less invariant, correct? That would mean the alterable functions can be carried out in software rather than in (redefinable) hardware. Flash updates are still flash updates, regardless of whether they contain computer programs or FPGA configuration files."
"Can anyone poke holes in this? - Gordon S".
Well, I reject the limitation "cheaply". That constrains any modern solutions to the hell of rapid obsolescence and proprietary solutions. That's the current strategy. I'm being blunt but brief. At $10, $20 single-board single-chip solutions, how does any hobbyist outdesign *that* with more persistent hardware?
The vintage hardware interface, is either a floppy drive for data recovery; or a floppy drive controller, for diskette emulators (not discussed but the complimentary device). Well, po-TAE-toe, po-TAUGHT-oe, to a first order. That's not the hard part.
On FPGAs, and the hard part. Turns out the support page for the Fluxengine, is pretty informative about this and other arguments for alternative technologies. After five years, that's already been defended (but I only read the gloss).
The defense is, it comes down to the design practice, of massive oversampling of floppy data streams, to extract arbitrary-rates of binary data relatively fast. Go to the site for details. So: one ether borrows cheep ARM-class devices that do that, or cheep FPLA devices which do that, or both. Because of that "cheap" constraint. Shrug.
In what used to be the real world, floppy disk controllers were dedicated microprocessors with fast bit-serial/parallel hardware. Read the early floppy controller FDC chip data sheets about that. They ran at defined data rates - so write hardware was easier. Reading hardware involved some skew, so one needed some tricks to sync-sample the read data. We call that data, single/double density, FM/MFM. Or the Apple mind-trick of the Apple II (later Mac) "Woz machine", a programmed-logic device (!) to perform 3-of-5 encoding and decoding. Or, some custom hardware and software, on other computers of vintage.
At one time, ALL floppy controllers were custom logic. (Search retrotechnology.com ) Now, in some sense, the custom logic is in the FPGA. This might make a case for preserving vintage computers (he he ...).
To replace all that, the cheat is to sample like hell and let post-1990's computers sort out the results in software. Fair enough.
I appreciate Gordon's point, that functionality in modern devices is a matter of programming, and reprogramming. That's fine, for those who enjoy modern embedded programming. But when the hardware to be programmed changes "too much"; the old software becomes obsolete. Yesterday's FPLAs can't be programmed with today's FPLA tools. The codes are changed too. The binaries are useless, even proprietary. That was my point!
Subtle comment by Gordon: "We already know what stays around, or at least remains well-documented and well-supported." Trick question. The trick answer #1 is "that's the same thing". Answer #2 is "well, stuff persists until it breaks; if you can't fix it you replace it". Sort of the short version of the argument.
Now, the software to RUN the platform around the FPLA, could in principle be standardized. For big-enough embedded systems, that's fine, they are application-specific not implementation-specific (kinda). But this is just a *floppy controller*. There *is no application*. Nobody is running MS-DOS or Linux on these puppies. They are driven-by something, the something steers it around, and that's all.
And by the way? The 1990's solution - the Catweasel, an ISA or PCI bit-sampling controller on PC's? Last I checked, there were five-six "installation solutions" for running that puppy. Surprisingly, each developer came up with their own software to drive it. Go figure.
I don't know how much that happens, with 21st century $20 microcontroller solutions; the Arduino platform is pretty established; the RaspPi, somewhat the same. Then each other device-family, has their manufacturer's development engine software (to keep you from using another product, he he). I made that point before.
So: walk through this with me. What floppy control hardware is pretty much standardized, available, and stable; year after year and even into decades? and what software to support what software to drive it, has those qualities? Answer: XT/AT/386 class PC's and MS-DOS. And that's why solutions like "Imagedsk" / MS-DOS / ISA floppy controllers / IMG files just keep on workin'.
But they don't work for 68K Mac disk formats, Apple II disk formats, etc. etc. That's why bit-sampling general solutions emerged in the 1990's, from Compaticard to Catweasel. And you know, maybe I'll think of a 1985 solution to the problem for those reasons. But it's harder to design PCI cards; many people don't have computers with PCI slots; USB is the "bus" of choice; and there's those dammed $10, $20, $30 stick-puters. They are the black-holes of design-your-own-devices.
But the question of the day, was about a $10 solution. It's an amazing bit of application, that FluxEngine; as is/was the GreaseWeasel. But it's like other $10 embedded solutions, with all the faults I mentioned and alternatives Gordon suggests. One can work around these, because in the end, nobody is gonna die because a $10 embedded stick coughs and sputters, or can't be found. There's no mission-critical task in play that matters. They will come and go, as do supporters of some application like read-a-floppy.
Sorry for the long lecture, I got a real question and it took time to answer it. Hope this did that.
regards, Herb Johnson
-- Herbert R. Johnson, New Jersey in the USA http://www.retrotechnology.com OR .net preserve, recover, restore 1970's computing email: hjohnson AT retrotechnology DOT com or try later herbjohnson AT retrotechnology DOT info
Jack Rubin > Sat May 9 09:41:14 EDT 2020
Hard to reply directly to your post since I only get the vcf digest, but a couple of points re new modes of disc (I'm an HP fan) imaging.
1) The flavor-of-the-day MCU or whatever isn't as important as maintaining accessible disk images and metadata. Flux level transcriptions weren't really practical when Dave Dunfield was building his ImageDisk system, so we should be thankful for all the extra CPU cycles available to us today, especially in a $15 package.
2) The approach is a little different but the philosophy is the same - for ImageDisk stuff, I still have a Dell 486 with all the right stuff, including an FDADAP and a Thinker Toys 8" floppy, but I also have a couple virtual machines with the proper toolchains installed to control a few Arduinos and PSoCs for the new stuff that demands a full software ecosystem beyond the hardware. More important, today's cutting edge developers can still read (and write?) .imd files.
I was thinking about this last night as I was installing and configuring all the stuff necessary to start working on my first Verilog project. It took me a lot longer to get the point where I could blink a light on the demo board than it would have to reconfigure a CP/M bios for a new serial card and sysgen a new system.
Be well and stay safe, Jack
Always good to hear from Jack Rubin, an old friend. I'm going to reply back in the previous thread, and include Jacks' post there. And I've not decided I'm being a curmudgeon, yet. My responses focus on the tech, not my attitudes. Unless of course, being technical is old-school now. ;) Again, this argument takes awhile. For the impatient: Jack has not convinced me of the merits of short-term disk-imaging solutions. I'm focused on the delivery hardware; Jack on the imaged disk results. In effect, he's conceding to the modern reality of what I call "popsicle stick computers". But I think he misses something, in not focusing on the problem of non-persistent tech, and ignores problems with flux-imaged file formats. I end by pointing to the irony that interest in keeping old stuff around is called "curmudgeon" in a vintage computing discussion. -------------------- Jack's point 1 says, accessible disk images and metadata matter. That is correct. But that's not all that matters. Of course, if we are voting on what matters: then a million for-free emulator users, outnumber the hundreds who create the disc images. But whose role is critical? The problem I point at, is disk imaging hardware and software which becomes obsolete. The fact it's $15 technology makes it more LIKELY to become obsolete. And when the hardware is obsolete, you can't add more imaged discs, and at some point the already-imaged content become less accessible as the software falls out of favor, off-line. And developers of new-tech, often enough start from scratch. This consideration has occurred with a more expensive tech - the Catweasel. Only persons who bought those decades ago, can exercise the hardware; the software has persisted but many people are unfamiliar with the device or its flux image formats. That's what obsolete looks like. An aside: Dunfield didn't need to flux-image diskettes, because almost all the 1980's and 70's diskettes in question, were accessible with many available ISA based PC-compatible floppy controllers. Dunfield's genius, was to identify appropriate floppy controllers!(Some stuff got left behind: Intel Multibus M2FM format disks took decades to be recovered for reuse. That history I know.) And: he came up with means to native floppy controllers which were incompatible. Namely, one injects into the native machine a downloader, which can then download a native program to format and fill a diskette-image - COLD. And: Dave stored his .IMG images as BYTE data in sector/track order, in a specific format. The .IMD format became a defacto emulated-disk format for many emulators of 1970's and 80's computers in the PC-compatible enviroment (of the 1990's forward). Those emulators would have gone nuts, decoding fluxes. As Jack points out in his point 2, many developers today still support .IMD format. It's hard to avoid: it's essentially as-recorded byte/sector data. I dwell on those points, because all those features of Dunfield's work, have been reinvented by other people - or forgotten by other people - as they reinvent disk imaging technology. Jack's point 2. He says "the philosophy is the same" between using a 1990's PC and IMAGEDSK and physical drives, versus use of a modern desktop computer with "the proper toolchains" and "software ecosystems" for arduinos and PSoCs. Well, that's *almost* the problem I call out. What Jack neglects to mention explicitly, is that for *each* Arduino or programmable system on a chip, one needs *another ecosystem*. My point is: if one wants to maintain or recreate an "old" (five years ago) Arduino or PSOC, one has to have or find that old ecosystem, and find that old piece of hardware. Jack appears to be saying, one doesn't have to "maintain". One ignores obsolescence, simply keeps the data, and moves along to the next "flavor of the day MCU". He doesn't say how flavor A's flux-images are reinterpreted by flavor B's flux-imager device. Or how computer-emulators might be obliged to support several flux-imaged formats. In fact he leans on the obvious choice for emulators, byte-correct formats including .IMG. I"ve all but explained why that is. I'm aware, Jack, these popsicle-stick computers don't and can't stay around long. They aren't built for that purpose. The world most of those MCUs are produced for, is a short-life world. Think cell phones. But: the Arduino world, is a meta-environment created and supported to solve that problem. It EXACTLY allows Arduino-class devices to come and go: one keeps the Arduino environment but changes the libraries to match the popsicle flavor desired AND available. There's consequences that are off-topic to discuss, but Arduino has persisted as a working solution to obsolescence and transient technology. But for this topic: not all Arduinos or MCUs or PSOCs have sufficient hardware to perform rapid bit sampling of 100K-bit data streams. So, some of the "flavored" disc-imaging hardware solutions, are not in the Arduino universe. That makes hardware obsolescence a *worse* and more-likely problem. To Jack's point as I see it. To developers of these of these flux-sampling disc-samplers and producer devices, they just hop to current products and platforms for them. No problem - of course they self select, we only pay attention to the successful results. To the "customers", all my considerations are techno-babble. They aren't developing anything. They buy what is available. The disc libraries are there, for free. They are consumers and outnumber the producers, if it comes down to a popularity contest. In between, is the tough part: the people who image found diskettes. They don't likely have technical skills, much less PSOC developer skills. They will buy and run, whatever disk-imaging hardware they can get today; or pass their discs along to whomever has such stuff. Or the disks are lost. And, contrary to prior discussion, disks are still found, and still lost. Now: there's people who want to restore and run the vintage hardware. Like me, like many that's part of the audience in this email list. Are they all required to be PSOC developers? No. Are they all required to be 1980's 90's digital engineers? No. They make choices about what old-tech to have and what new-tech to have. But they choose between a $500 CP/M machine (or Amiga, or Mac) and a zero-cost emulator on their laptop - who wins that? And so they choose USB devices for $10, $20, over some clunky floppy drive that needs $100 of additional hardware for either domain. Again - who wins the vote for modern over vintage, even among vintage-computing enthusiasts? In closing: It's funny to use the word "curmudgeon", in an email list exclusive to those interested in 1980's and 90's (and earlier, maybe later) computers. Because, it references age, and the subject is old-stuff. But the actual reference, is to old *people*, and the consequences of having accumulated knowledge and experience, versus (pardon me) a larger world where those are lost in normal practice. My regards to Jack and all, Herb Johnson -- Herbert R. Johnson, New Jersey in the USA http://www.retrotechnology.com OR .net preserve, recover, restore 1970's computing email: hjohnson AT retrotechnology DOT com or try later herbjohnson AT retrotechnology DOT info
Continuing the long reply tradition: I wanted to join this discussion earlier but had another distraction. This is my take on this as actual maker of a flux imager except for MFM hard drives and a user of floppy imager. I don't see the issue as being the inexpensive hardware I see that as an advantage. I've seen a number of people say they can't justify getting my reader because its too expensive to for reading one drive. I suspect some people aren't going to shell out 99 euro for the Kryoflux to image some floppies but might get a $10-$20 board. The Kryoflux is the long term supported imager (around at least 10 years). It has some licensing issues that encouraged people to make alternatives. The blue pill problem you discussed is a separate issue. My time is too important to save a couple $ if I'm going to waste time on poor hardware. Simple to use would be nice but its a lot of effort for the creator and the real world variability makes it hard. I'm sure users of my tool mutter what was I thinking at times. I liked the FluxEngine simplicity of only needing a connector soldered to it. No custom board needed. I got a KryoFlux since I had a bunch of M2FM disks to read and had 8" drives but no Intel MDS to image them. Still processing them but they will get to Bitsavers and anywhere else that wants them. I see the real issue as that the projects don't cooperate. Each one implements their own set of decoding tools and image formats with limited ability to convert between image formats. I got the Kryoflux because I found a decoder for it for Intel M2FM. It had some issues and I fixed some then found out that their was another project that I didn't know about that had a decoder with the issues I spend time fixing already fixed. I have only looked at them a little but it looks like the Fluxengine and Greaseweasel are doing the same separate development with limited ability to capture flux files on one and use the decoder from the other if it has the format you need. I think most of the value is in the decoding code. If it was properly isolated from the hardware specifics then when the one inexpensive board goes away and the next person creates an imager around the next board it wouldn't matter. I've spend a lot of time adding support for the 50+ disk formats my tool can deal with. And that's only the low level disk format. I'm not touching the file system formats. So far I'm the only tool that reads MFM hard drives so no ability to cooperate with others. I did make some attempt to allow the format to be useful for other boards such as putting the sample rate in the header of the flux transition file and not just assuming the rate my board uses. I will admit my decoder will just print an error if the file rate isn't what I use but that could be fixed if needed. I also image with real hardware when I have it and I distribute software for the PDP-8 to allow people to image various media on the PDP-8. A selection of tools allows people to pick the one that is best for them. Its only a problem when the accessories (decoders & file formats) are tied to the tool that you care if one particular tool is discontinued.
participants (5)
-
Alexander Jacocks -
David Gesswein -
Dean Notarnicola -
Herb Johnson -
Tony Bogan