Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me.
|
Forum Index : Microcontroller and PC projects : Web Interrupts and WatchDogs
Author | Message | ||||
Malibu Senior Member Joined: 07/07/2018 Location: AustraliaPosts: 228 |
G'day all I've had a niggling problem for a while when using a WebMite connected to my local WiFi network (and, WWW in general). I think I've fixed the problem now, but I'm trying to get my wee-brain around what the cause has been... Someone more in the know with Web traffic than me could have the answer. All versions of my code have been written with a simple WebInterrupt Sub based on Geoff's Retic.bas code In general, everything seemed to work ok, but I was getting a lot of Watchdog timeouts, plus even more of the: "Internal error in tcp_server_recv - Attempting recovery" errors... which was mostly recoverable... Process of elimination, I pinned the faults down to this sort of traffic coming down the network (with the gibberish ASCII characters that made no sense to me) (This is just a sample of the web traffic I've logged, there's many more similar) The ONLY thing that made sense to me is that the 'Mite receives web traffic, processes it (in this case, would send out a 404 code), but the originator of the request is no longer on line (A bot maybe?) and so a jamb-up occurs and fires the watchdog. Not being great with web 'hand-shaking', I'm only guessing what it could be... As a solution, I rewrote the Interrupt code to this (Sorry, lots of comments) - WEB TCP INTERRUPT WebInterrupt 'Catch the web traffic and deal with it settick 1000, ShowTicks, 1 'Do something every 1 second do 'Main code loop here timer = 0 'Reset the timer for cycle times Watchdog 20000 'WatchDog time-out value loop 'Keep looping sub WebInterrupt 'Web data has arrived local integer ConnNum 'Var for connection number local integer ConnData(512) 'Var for web data ? timer 'Display the timer val for time stamp for ConnNum = 1 to mm.info(max connections) 'Loop through all the connections longstring clear ConnData() 'Clear any previous data web tcp read ConnNum, ConnData() 'Get the web traffic if llen(ConnData()) > 0 then 'If the data has anything in it, do this ? "Web Interrupt ---------" 'Display the page break ? if linstr(ConnData(),MM.Info(ip address)) > 0 then 'Is the WebMite address included in the data? ? "Format OK" 'Display that things are good LongString print ConnData() 'Display the data that came in web transmit code ConnNum, 404 'Move along, there's nothing to see here Else 'No WebMite address? ? "*** Error in Format ***" 'Display that we're not interested LongString print ConnData() 'Display the data that came in web tcp close ConnNum 'Close the connection to avoid a possible WatchDog error end if ? "-----------------------" 'Page break end if next ConnNum 'Do the next connection ? timer 'Display the timer val for time stamp ? end Sub Now, ONLY web traffic with the WebMite address is accepted and processed - anything else closes the connection. So far, so good... It's been running for 36 hours without any hiccups, even with the massive amounts of dribble that comes through and gets processed. First question: is this a normal amount and normal string format that comes through a WWW connection? Second: Is my coding bad in that I filter the web request that only have the 'Mite IP Address in it and take action on that? John |
||||
TassyJim Guru Joined: 07/08/2011 Location: AustraliaPosts: 6098 |
My guess is it's a broadcast packet of some kind. I know there a plenty of them on any network but I have not tried to work out what type is causing the problem. I have had similar issues here. Your cure looks like a good one and I will look forward to a report in a few days. Jim VK7JH MMedit MMBasic Help |
||||
Ossi Newbie Joined: 22/06/2023 Location: GermanyPosts: 6 |
Hello John, in a "normal home environment" behind a router without open ports (e.g. FritzBox) there should be no such wild telegrams. Basically an html request should consist of readable ASCII characters. I have tried whether the WEB SERVER reacts to a ping. The ping response from the WEB Mite comes back, but my WEB SERVER interrupt gets nothing from it. Try using something like Wireshark to determine the IP address of the request. Show your OPTION LIST. What does your WEB CONNECT look like. Good luck with the further analysis. Ossi |
||||
matherp Guru Joined: 11/12/2012 Location: United KingdomPosts: 9112 |
I've implemented this fix into Geoff's watering controller code and so far at 1 full day with no glitches If this proves to be the issue all along I'll see if I can integrate the fix into the firmware |
||||
Malibu Senior Member Joined: 07/07/2018 Location: AustraliaPosts: 228 |
G'day guys, thanks for the feedback on my curiosity... Up until this point, it has been running for around 4 days with no unexplained timeouts. There's been massive amounts of ASCII gibberish come through, and all of it has been rejected without firing the WD timer. Of course, it's a little limiting in usability being only able to accept traffic directly targeting the Mite, so I twiddled the code to this: sub WebInterrupt 'Web data has arrived local integer ConnNum 'Var for connection number local integer ConnData(512) 'Var for web data local String Host1, Host2 'Vars for hosts Host1 = MM.Info(ip address) 'The local IP Address Host2 = "210.50." 'The primary numbers for my ISP address for ConnNum = 1 to mm.info(max connections) 'Loop through all the connections longstring clear ConnData() 'Clear any previous data web tcp read ConnNum, ConnData() 'Get the web traffic if llen(ConnData()) > 0 then 'If the data has anything in it, do this ? "Web Interrupt -", Time$, date$ 'Display the page break, so we know web traffic has occured ? "Start time stamp:", timer 'Display the timer val for time stamp if linstr(ConnData(), "Host:") > 0 then 'Is the traffic from a legitimate host? ? "Host string found." 'Yes, we've found a 'host: ' string, all good so far... if (linstr(ConnData(), Host1) > 0) or (linstr(ConnData(),Host2) > 0) then 'Address of local WebMite, or ISP? ? "Valid address found." 'We have a match! ? LongString print ConnData() 'Display the data that came in if linstr(ConnData(), "GET /test.html HTTP") then 'Is the pointless web page requested? ? "Sending test.html" WEB TRANSMIT PAGE ConnNum, "/test.html" 'Send the pointless page else if linstr(ConnData(), "GET /lizard.jpg") then 'Picture requested? ? "Sending lizard.jpg" web transmit file ConnNum, "lizard.jpg", "image/jpeg" 'Yeah, send it too... else ? "Sending 404 Code" web transmit code ConnNum, 404 'So far, so bad... move along, there's nothing to see here end if else ? "Error in Address." 'The address is a dud, deal with it from here ? LongString print ConnData() 'Display the data that came in ? "Closing connection" web tcp close ConnNum 'Close the connection to avoid a possible WatchDog error end if Else 'No WebMite address? ? "*** Error in Format ***" 'Display that we're not interested LongString print ConnData() 'Display the data that came in ? "Closing connection" web tcp close ConnNum 'Close the connection to avoid a possible WatchDog error end if end if next ConnNum 'Do the next connection ? "End time stamp:", timer 'Display the timer val for time stamp ? "-----------------------" 'Page break ? end Sub I also added a useless web page to test the functionality on that side of it. The new code now accepts ONLY the correct Mite address, and/or anything that comes from my ISP. So far, seems to be working good, but I'll get into work next week and try to access the web page from the CNC computer and give it a solid test. I'll keep you posted (Apologies for the cluttering-mess of my comments... it helps keep my train of thought on track!) John |
||||
Malibu Senior Member Joined: 07/07/2018 Location: AustraliaPosts: 228 |
Oh, and just for a giggle, this is my favorite gibberish traffic I've logged... Really? What does that even mean!!? John |
||||
TassyJim Guru Joined: 07/08/2011 Location: AustraliaPosts: 6098 |
The only traffic that should be able to get to you from the WWW are packets that have been port forwarded. Broadcasts packets shouldn't be forwarded. That is why I suspect that they might be locally generated. What ports have you forwarded to the webmite? If you can save the suss packets to file instead of just printing them, it would be easier for me (and others) to examine them and possibly work out the source. It is not easy to retype the screendumps as images. Jim VK7JH MMedit MMBasic Help |
||||
Malibu Senior Member Joined: 07/07/2018 Location: AustraliaPosts: 228 |
G'day Jim, It's been a cut/paste from the MMCC screen to Programmers Notepad for a week or so, but I had all the data saved to a local file. Attached file is any relevant info for this, including the basic source plus the web-page and graphics - plus of course the text file of the errors (Edited with a lot of the good HTML requests taken out for a smaller file size) Web Test 2.zip The last entry in the error file was from 3:00 this morning... the first untrapped fault in more than a few days of running John |
||||
TassyJim Guru Joined: 07/08/2011 Location: AustraliaPosts: 6098 |
Thanks for that. I was wrong, the packets are not local but coming from the WWW and are probably some sort of port scan hitting your port 80 (or whichever external port you have forwarded). This is normal and one of the curses we have to deal with. My code closely follows Peters original example and looks for "GET /" rather than "HOST:" in the data. Either would work as a basic filter. I can see some repetition in the packets but nothing I recognise. I will set up a system to capture external hits on my system but it might be a few days. Jim VK7JH MMedit MMBasic Help |
||||
Malibu Senior Member Joined: 07/07/2018 Location: AustraliaPosts: 228 |
Thanks Jim, appreciate you taking a look and glad to hear I'm not quite going crazy yet... It seems like a lot of 'noise' for something that's normal, but as I say, I've no real idea on WWW protocols, so there were a few guesses on my part. There's probably a few filter words that could be used ("GET /", as you say) but I selected "HOST" because I also Tx/Rx from a VB program for changing code parameters, so I wanted to keep GET, POST (etc, etc) as reserved words and not muddy the waters with HTML code within VB. It was quick and easy to append the TxData string Private Sub Winsock1_Connect() Dim tmp As Variant tmp = "Host: " + NodeData(NodeNum).Address + vbCrLf Winsock1.SendData tmp + TxData End Sub In the VB code through WinSock It would be interesting to see what you can come up during your tests; I look forward to your results John |
||||
Malibu Senior Member Joined: 07/07/2018 Location: AustraliaPosts: 228 |
'Morning all I've been looking a bit more into my problem because every now and then I still get a watchdog timeout after a web hit (There seems to be no pattern to the timing, nor patterns in the web hit 'language', even seemingly good web requests) I've tried to monitor with WireShark, but as a first time user of it, I can't seem to set it up properly. My basic networking is: Local IP range is 192.168.0.XXX Router is 192.168.0.1 WebMite is 192.168.0.114 (Forwarded from the router) My desktop PC is 192.168.0.107 My public IP is 210.50.XX.XX So, WireShark picks up all traffic from my local PC no matter if I enter WebMite address, or access via the public IP, and I get the test webpage back as expected. If I try to get the webpage from my phone, yes, the webpage loads onto the phone, but there's no capture in WireShark. Setting a display filter to HTTP, or tcp/udp:80 port, it's always my desktop PC address that is listed, so I can't find what traffic is happening with the WebMite unless it comes from my desktop PC... Very frustrating! Any WireShark experts have a clue where I'm going wrong? John |
||||
TassyJim Guru Joined: 07/08/2011 Location: AustraliaPosts: 6098 |
Apart from broadcast traffic, packets go to the destination device and other devices (your PC) can't see them. Your router 'might' be able to capture data which is either saved to the router memory for later retrieval or forwarded to another device such as your PC. Some have the ability but hide it from casual users. One other alternative is (yet to be tested). If your PC is wired rather than wireless, set up the PC as a mobile hotspot and connect the WEBmite to the hotspot. That way the traffic to and from the WEBmite is going via your Ethernet port which WireShark can capture. I also have a managed switch which can echo traffic on on port to another port. If I am desperate that is the way I go but it needs an access point for the WEBmite so the managed switch is between the router and the access point. I have been monitoring the WEBmite with your sub, hoping for an interesting packet to appear but no luck so far. My Internet must be cleaner than yours! Jim Edited 2024-03-17 11:45 by TassyJim VK7JH MMedit MMBasic Help |
||||
Malibu Senior Member Joined: 07/07/2018 Location: AustraliaPosts: 228 |
Thanks Jim, much appreciated I looked at that a few weeks ago, but didn't really find anything useful there. A bit limited in knowledge on networking, I could only see that network traffic was happening, but couldn't get any real info from it. A good suggestion that I have running now, so I'll see what comes of that. Unfortunately, it's given a completely new IP range with the server IP as 192.168.137.1 so it's off the usual network range (I'll have a play and see what I can figure out on that part of it) I'll run it as-is for a few days and see what comes of it Hmmmm... that's a bit frustrating for my side of it. You might be right and I might just have a crappy ISP, so I'm chasing my tail here. (Then again, maybe it's some state-sponsored hacking agency trying to get all the top-secret files from my computer! [KGB, XYZ, CIA, FBI, ABC, PRC... they're all after me!] ) John |
||||
TassyJim Guru Joined: 07/08/2011 Location: AustraliaPosts: 6098 |
first, you forward from the Internet to your PC port 80 then you forward that to the WEBmite with another port forward, this time on your PC You will have to configure port forwarding to the hotspot. You will also have to open the firewall to allow port 80 in. This is a good step by step: https://redfishiaven.medium.com/port-forwarding-in-windows-and-ways-to-set-it-up-c337e171086f The only thing that needs changing is the line that does the forwarding listenport=80 listenaddress=yourPCaddress connectport=80 connectaddress=192.168.137.xxx (the WEBmite) Networking is fun! Jim Edit. This will let you capture any odd connects but the WEBmite will still have hissy fits occasionally and you do need the hardware watchdog set up to restart. Edited 2024-03-17 16:30 by TassyJim VK7JH MMedit MMBasic Help |
||||
Malibu Senior Member Joined: 07/07/2018 Location: AustraliaPosts: 228 |
Thanks Jim, I think I've done it correct... I'm getting to log the web hits that show in MMCC with WireShark, but I'll leave it run for a few days and see what comes out of it. It was a little bit of head-scratching on my part, but here's fingers crossed! As they say in the classics... 'No Comment' Ok, I'll admit - I'm learning! Can't ask for much more than that I guess (My comms experience goes back to 422 & 485 many (many!) moons ago, then only a few moons ago with ASi Bus) I'll keep you posted on the results John |
||||
Malibu Senior Member Joined: 07/07/2018 Location: AustraliaPosts: 228 |
Morning all Here's the latest of my investigations for my problem (It's been a while but I've been a bit bust lately) Here's the only part of the WireShark logs that I can guarantee is the 'fault', because I was watching the WS screen scrolling at the exact time - so I know I got a good capture. This is a screen grab from an Excel file of the log, because I could highlight various parts of it to (hopefully) make a little more sense to me... Down to line 10, it was a good conversation. On line 11, a SYN was sent, but not answered - so it seems the handshaking was never completed (SYN : SYN,ACK : ACK), with 4 more tried attempts before I manually removed power and restarted. Here's the console print to MMCC at that moment Interestingly, the WMite watchdog didn't fire and the heartbeat continued to flash. For some reason, that seems to be the usual case running from a Hot Spot (that's what I've noticed, anyway) I'm still at a loss to why it's happening - it 'seems' the WebMite is missing the interrupt and I thought this morning that maybe the 1-second timer tick is being served when a web interrupt arrives. Is that possible? John |
||||
Print this page |