 Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me. |
Forum Index : Microcontroller and PC projects : CMM2: Introducing MiteWAN a shared disk drive for the CMM2
| Author | Message | ||||
PeteCotton Guru  Joined: 13/08/2020 Location: CanadaPosts: 368 |
One of the few barriers for users of the CMM2 is the requirement to have another computer of some sort to download and install programs on the CMM2 (and a suitable SD card to USB adapter for that computer). It's a bit like getting a Commodore 64 on Christmas morning, only to discover that your parents didn't buy the tape deck -or any games. I've been working on a simple solution to this over the Christmas break... and here it is: MiteWAN. The hardware is very very simple, it's an ESP 32 with just one UART wired to the COM2 port on the CMM2. 4 wires in total (TX, RX, GND, +5V). I've mounted the prototype on a plug in board for the back panel (Gen 1 CMM2 - so we'd need to flip the connector for Gen 2), but ideally it would be soldered onto the COM2 pinouts on the Gen 2 motherboard as part of the permanent hardware.  There's a corresponding MiteWAN program that could be supplied on the SD card of new machines, and now the user can share programs directly to the cloud. This is not a WiFi expansion or web browser, nor does it give you access to the web (though we could add that later). Yes, it does use your WiFi, but it is uses it specifically to tunnel into the cloud storage. Think of this as a shared disk drive on the cloud. I wanted it to be as simple as possible for any new users. You can select any program from your local SD Card and upload it. MiteWAN will assign your program a unique 5 letter code. You can share that code with anyone you want (or post it publicly on the forums) and then people can use it to download your program directly on to their own SD card without needing to use any other computers. Here's a quick explanation video where I share my Defender game which has been given the code Njp76. The idea is that I could come on the forum here and say - hey guys - try my new game out, it's MiteWAN code is Njp76, and you guys could try it out directly from your CMM2. https://youtu.be/lmqAr8MDy00 Similarly, if I update the program, I can re-upload it (it retains the same code) and users can download the update with ease. We could add a web front end with screenshots etc. for people to scroll through, but for the time being I'm just happy to have the technology bits working. I did hit quite a few barriers in development, but it's very stable now. I've tested it with a bunch of large programs, including ones with huge sound/graphics files and multiple sub-directories and it works flawlessly. The ESP32 does require 8MB PSRam, as it uses this to cache the downloaded files. But you can get these chips for about $15 USD. The CMM2 uses it's graphics memory to cache the program (to ensure a completed file transfer) before writing it to the SD Card. |
||||
| lizby Guru Joined: 17/05/2016 Location: United StatesPosts: 3150 |
Neat idea. Can you share your code? My favorite ESP32 is the $2.18 S2Mini with 2MB PSRAM: S2Mini Other than for super-large files, any reason why 8MB PSRAM would be required? With 8MB RAM, why does the CMM2 need to use graphics memory for the file transfer? I've done multi-megabyte downloads to the CMM2 for test purposes (via serial). PicoMite, Armmite F4, SensorKits, MMBasic Hardware, Games, etc. on fruitoftheshed |
||||
Grogster Admin Group  Joined: 31/12/2012 Location: New ZealandPosts: 9307 |
Very clever!  Smoke makes things work. When the smoke gets out, it stops! |
||||
| PeteCotton Guru Joined: 13/08/2020 Location: CanadaPosts: 368 |
To an extent - yes. The CMM2 code is of course fully open. But the ESP32 code contains the SAS key for the server. While this in itself isn't a massive issue, I'd rather keep it out of the hands of any malicious parties. Therefore, I'd be happy to share it with long term forum members such as yourself, but won't be publishing it publicly. The 8MB was arbitrary, we could make it work with 2MB, assuming that there aren't any individual files larger than that in CMM2 programs. Just change the size of the ps_malloc command. I originally had it downloading - and saving in 16K packets, but this created partially written files if the download timed out. Not a huge issue, but I went with caching in memory to get around that. I also kept hitting a bottle neck of 20k/second. This appears to be limited by the WiFi on my ESP32. Every other aspect (My internet, the serial link between the ESP32/CMM2 and the speed of the SD Card) are many factors of 10 faster than this. I don't know if other ESP32's are faster. I have a bunch of Teensy 4.1 chips, and I'm wondering if the WiFi is faster on that - but haven't tested it. However they are quite expensive. I might go back to that scheme - but for now I'm just getting all of the parts in the chain (the cloud server - the tunnel - the ESP32 and the CMM2 code) all working well. I wanted each file in memory so that I can check that it has downloaded correctly before writing it to the disk. It saves having to tidy up corrupted files. To be honest - there isn't really a reason not to use the graphics memory like this. We can peek and poke it directly - and it's a huge bank of contiguous storage. Very convenient. Thanks! Edited 2024-01-05 15:12 by PeteCotton |
||||
Bryan1 Guru Joined: 22/02/2006 Location: AustraliaPosts: 1344 |
I'm wondering if that microbridge we used on the micromite would work to connect a com port to the CMM2. A cheap proven solution for the micromite and in theory it should just work. Then MMEdit would be able to see the CMM2 and transfer the files over. |
||||
| Volhout Guru Joined: 05/03/2018 Location: NetherlandsPosts: 4234 |
Hi Pete, If I understand right, there will be 1 cloud server that we (all) can use a distribution point for our programs. Currently you are hosting that data ? The CMM2 communicate serial to a "black box ESP32" that does the communication to that server. The "black box" needs to connect to your local WIFI to establish the link to the server. On the CMM2 there is a program to open the serial port, and move the data between SD card and the black box (and verifies the data before doing so). In essence this could also work with a PicoMite, using the same "black box" and a suitible MMBasic program for the platform. Is this correct ? Volhout PicomiteVGA PETSCII ROBOTS |
||||
| lizby Guru Joined: 17/05/2016 Location: United StatesPosts: 3150 |
That was the direction I was tending towards--something like a forum "dropbox" for internet-less mites. PicoMite, Armmite F4, SensorKits, MMBasic Hardware, Games, etc. on fruitoftheshed |
||||
Mixtel90 Guru Joined: 05/10/2019 Location: United KingdomPosts: 6792 |
As it appears to the user device as a COM port I don't see why it couldn't be used with anything. You may have to have a suitably sized buffer, which may need introduction of a "block" system (possibly with CRC) to allow for smaller RAM devices. Mick Zilog Inside! nascom.info for Nascom & Gemini Preliminary MMBasic docs & my PCB designs |
||||
| PeteCotton Guru Joined: 13/08/2020 Location: CanadaPosts: 368 |
Yes. I'm very happy to pay for the cost of this going forward, however this does raise the issue of what happens if I get hit by a bus. Currently it's hosted on my company cloud servers. When it's ready for release I will set up a completely independent cloud contract (which I'll still pay for), and then I would like to form a small admin group of senior members from this forum - all of whom will have full access to the cloud services. We can set these admins up, so that the accounts cannot be disabled by other admins - thus ensuring that control of the hosting is always accessible. 100% correct. All correct. To be honest, I did initially think of the other mites, but then when I was making it all work I lost sight of that a bit (by using the graphics memory as a buffer). I think this makes a very good argument for getting rid of the graphic memory caching and reverting to the 16K packet method. That way, this would be usable by all "mites" with a COM port. Edited 2024-01-06 02:50 by PeteCotton |
||||
 Print this page |
