 Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me. |
Forum Index : Electronics : Just what to log
  Page 3 of 7   |
|||||
| Author | Message | ||||
fillm Guru  Joined: 10/02/2007 Location: AustraliaPosts: 730 |
Hi Gordon , Well done , I know how much work you have done with doing some refinements to this program , especially the problem with the USB - serial data that causes the watt/hrs to count backwards with my dual log. I hope a lot more poeple with windmills start to use logging to compare and record data . PhillM ...Oz Wind Engineering..Wind Turbine Kits 500W - 5000W ~ F&P Dual Kits ~ GOE222Blades- Voltage Control Parts ------- Tower kits |
||||
GWatPE Senior Member  Joined: 01/09/2006 Location: AustraliaPosts: 2127 |
Hi Phill, The count back should not be happen now, and I kept the whour file logging. Logging will be very telling for most systems. The record is more objective, than quoting recorded maximums. I have been able to get a printout, but only part of the plot window graphs are printed, yet all of the rest of the form is OK. I may need to refresh the plot window within the print routine. I have a big job ahead, adding my new found fixes to all my other logging apps. Gordon. become more energy aware |
||||
Downwind Guru Joined: 09/09/2009 Location: AustraliaPosts: 2333 |
Since i loaded the latest version of the logger i have no hangups with errors where it has stopped logging. Although when i checked the days log i had no file for the 25th...something wrong here!.... Hmmm the idiot here forgot to click Start after he was last in having a play around in calibration. I do like the little feature of being able to clear the error messages by open/close the comm's button. So far it has all run stable here on Vista 32 and i have not been able to break it yet. Nice work Gordon (other then it not being idiot proof) Pete. Sometimes it just works |
||||
| GWatPE Senior Member Joined: 01/09/2006 Location: AustraliaPosts: 2127 |
For those readers who may have been able to give me assistance with printing, it is no longer needed. I have a working print routine now. I have found that VB6, has limits on the size of plot windows that can be printed. I rearranged the graphing window layout and I can get a full size A4 landscape plot, as well as retaining all the other components. Just have to get the right mix of colours, and possibly a reduction in unnecessary printed info, and a slightly better page layout with possibly some header and footer info now. I suppose the expression- "where there is a will, there is a way" comes to mind. The printing of graphs to a hard copy will allow the rapid comparing of day to day, or any other times data. Sheet paper is easier to manipulate when doing comparisons of graphs. The report looks a bit like something you would get from a hospital after tests are done now. Gordon. become more energy aware |
||||
| domwild Guru Joined: 16/12/2005 Location: AustraliaPosts: 873 |
Gordon, "Where there is a will, there is a family!" Taxation as a means of achieving prosperity is like a man standing inside a bucket trying to lift himself up. Winston Churchill |
||||
| GWatPE Senior Member Joined: 01/09/2006 Location: AustraliaPosts: 2127 |
It doesn't look like much useful feedback will be coming, so there can't be much wrong with the program, or no one is using the piclog routinely. Maybe there really are better programs that readers use but are not prepared to share with others. I could probably keep adding more to the app. I have finalized a way to collate all my own logged data. without a database as such. I have made a start to collate all my house power data. I could add this feature to the piclog, but it may be flogging a dead horse. I do see evidence of the piclog being used, surfacing from time to time and hope the users do get a chance to try the piclog2010, and leave their comments. I thank those who have made the effort and hope many more will realize the usefullness of reliable logging. Gordon. become more energy aware |
||||
powerednut Senior Member Joined: 09/12/2009 Location: AustraliaPosts: 221 |
Gordon & Downwind, Looks like you guys have been doing a fantastic job. I'm in 100% agreement with you on the usefulness of logging and I would be using your system if I had the energy budget to run a machine 24/7 to gather data. Hopefully down the track when my system grows that will change. |
||||
Bryan1 Guru Joined: 22/02/2006 Location: AustraliaPosts: 1344 |
Hi Gordon, I did download the program but without a piclog it doesn't look like it would work. As I don't have and don't intend to get a picaxe when I get time (next joke) I'll do a pic version say using a 12f683 which is the picaxe chip before 90% gets stripped to make it a picaxe. Cheers Bryan |
||||
| GWatPE Senior Member Joined: 01/09/2006 Location: AustraliaPosts: 2127 |
Hi Bryan, All you need is a PIC, with a serial data stream output. I have made provisions for either 2400,4800,9600, or 57600Baud, with 8data and 1stop bit. The data has to have a format like this :- Voltage, Current, RPM, and Windspeed, and a checksum[optional]. The data should be in the form of the decimal number as ASCII, not the binary word. The checksum is just the mathematical sum of the 4 data values, and if present, will be checked to confirm the data transmission integrity. The <XML> type encoding is important, and if the format is different, then it will be rejected, and not logged. The encoding can be in any order, but must include the [ at the start, and the ] at the end. The data string format will look something like this. [<V>XXX</V><I>XXX</I><R>XXX</R><S>XXX</S><C>xxxxx</C>]crlf This is basically the same as Gizmo's, with the addition of the <C>xxxxx</C> checksum and the crlf. These are optional, and my code will work without, but some corrupt data may not be picked up without. I am sure this would not be any problem to code in a PIC. The data stream should only be sent a maximum of 2Hz. The data will be averaged for the whole of the data logging interval, for all valid data recieved during the interval, so a higher transmission rate serves no useful purpose. The program Vasi wrote in PIC, that emulated Gizmo's original piclog should work as well. Gordon. become more energy aware |
||||
| GWatPE Senior Member Joined: 01/09/2006 Location: AustraliaPosts: 2127 |
Hi powerednut, My logging system has 6 apps running, and logging a total of about 25 parameters. My computer doing the work is an ATOM board, consuming approx 12-14W. I am sure an E-PC would be a winner as well. I do not have one to test. A normal PC is out of the question, but many laptops would be OK. Downwind [Pete] has a small DC, PC that would be suitable. I don't know how small your system is, but if you don't have enough power to run a logger like this, you don't have much power to do much else. As far as this app goes, the power is not really in the actual logging, but in the the ability to graphically visualize the data history, comparing different data parameters, looking at daily data in detail, storing portions for manipulation in a spreadsheet, and maybe even printing hard copy, or collating readings to compare over longer time periods. In my professional career, I started out with liquid ink chart recorders, and strip printers, and then digital charts. If we had what is now available we could have spent a lot more time doing more interesting work. Gordon. become more energy aware |
||||
| Downwind Guru Joined: 09/09/2009 Location: AustraliaPosts: 2333 |
Hi Bryan, If you care to write your program as a replacement for the 08M picaxe to suit the same pinouts that i have used, for the circuit design of the Power Logger, soon to be published on the forum, then we can add your pic program to the project information if you care to share it. It would make sense to have one universal board design for both pic and picaxe, unless you have your own ideas for the circuit. I can give you all the pinouts and the picaxe code used for you to translate to pic if you would like. (all of about 6 lines of code) Let me know if you are interested. Pete. Sometimes it just works |
||||
| powerednut Senior Member Joined: 09/12/2009 Location: AustraliaPosts: 221 |
Hi Gwatt, My system currently is a single 100 Watt solar panel charging a lead acid bank whos capacity I don't remeber off the top of my head. Its designed to handle about 320 Watt hours of usage a day with 3 days of no-sun capacity - basically a laptop and a couple of compact fluros for about 4 hours every evening, plus a few smaller bits and pieces 24/7. For larger, short run loads I run my generator (normally less than an hour a week). In the short term the system will be "upgraded" by a cheap-nasty 300Watt chinese wind turbine. Longer term I'm planning a couple of axial flux turbines and another 100-200Watts of solar. In about 2 years when we have built a house to live in I expect to have around 2-3Kw of solar, and about the same for wind. In the meantime, with a setup this small there isn't a lot of point in burning 280-330 Watt hours a day keeping a data logger running. That said I'm pretty sure I've got the SD card serial data logger circuit figured out, and when the parts (rtc chip) arrive I'll test it out. If it works I'll post the schematic and source code to the forum so others with tinyRenewable setups can make use of dataloggers like yours and downwind. I expect that energy usage for it will be around 40-80 Watt hours a day, but I'm hoping to do better. |
||||
| Downwind Guru Joined: 09/09/2009 Location: AustraliaPosts: 2333 |
Hi powerednut I am wondering in what format are you going to store the data to sd as there is a lot of number crunching that the pc dose prior to writing it to file. I find if the data rate is to high via the serial port the piclog program has trouble processing the data fast enough. For avarage logging with no more than several data packets a second its fine, but to dump a sd card of data and expect the piclog program to handle it, could be a tall ask. To do the number crunching in the micro is not the best either as its slow and no integers can be used so it gets messy with low resolution. The other problem is when you swap sd cards over you will end up with 2 files for the same day of the card swap. How do you intergrate this to one file for the day so when viewing a month or years data it all flows correctly. I do like the thought of data storage without a pc but these are just a few of the problems i came across when considering remote data storage without a pc needed. Pete. Sometimes it just works |
||||
| GWatPE Senior Member Joined: 01/09/2006 Location: AustraliaPosts: 2127 |
Hi Powerednut, There are some additional problems I see, to what Pete has just mentioned. To store data say every 10 seconds, generates about 8640 data sets per day. This is about the minimum for meaningful data viewing and averaging on a wide ranging data stream. The rate at which data is downloaded or from SD changeovers will determine the number of days data that will need to be stored, and if the data is to be in a single file, or multiple files. The only way I see this SD card working is in combination with a true database program, and the individual data packets are stored in a database, with the date and time stamp used to parse the data. This will avoid the SD changeover problems. The parsing program will need to be able to handle the data conversions as well. SQL programs could be written to do pretty well anything with the data once in the database. Remember the database is only as good as the data in it, and if you get the format wrong, it can be hard to fix. I would think that an SD micro combo could be in the 1-2W power range per logger. In my case I would have at least 6, so I would be no better OFF. Gordon. become more energy aware |
||||
| powerednut Senior Member Joined: 09/12/2009 Location: AustraliaPosts: 221 |
let me know if I'm hijacking and I'll split this off into a seperate thread. The plan at the moment for the sd card logger is to take whatever serial data comes in, add a time date combo and write it to a file, or possibly raw to the card. Personally I like the raw to the card option, but I expect others would prefer to deal with files. So if it was plugged into gordon's logger the line written to the file would be: <YYYYMMDDHHMMSS>[<V>XXX</V><I>XXX</I><R>XXX</R><S>XXX</S><C>xxxxx</C>]crlf if it was plugged into something that only spat out hello world the line would be: <YYYYMMDDHHMMSS>HELLO WORLDcrlf Because each entry gets a time date stamp I don't think card change overs would be a problem. Processing would be done later, possibly using piclog though I havn't got that far thinking it through to be honest. Worse case I'll roll my own. |
||||
| GWatPE Senior Member Joined: 01/09/2006 Location: AustraliaPosts: 2127 |
Hi powerednut, The string would be OK only if it came in through the COMM port. The piclog uses the computer date and time, to create its files. The data that is stored in the compuuter is a different format to the COMM data. It is really a matter of records. Any more than 10000 records per file and the plotting becomes a waiting problem. You would need to write your own database program to handle the data you intend storing. The piclog1010, or piclog cannot use the date and time stamps, but if you go this way, then use a normal database, and SQL type programs to view, print, export the data. Gordon. become more energy aware |
||||
| Downwind Guru Joined: 09/09/2009 Location: AustraliaPosts: 2333 |
Hi powerednut It should be said we are not tring to discourage you from doing the sd card as there is well a use for it. Gordon and myself have kicked this cat around some, to the point the poor puss has bald spots now. We keep coming up agains problems that the solutions are not so straight forward. It would require a compleet redesign of software to make it easy and useable for the average person. We have spent much time in our design of circuit and software to make it easy to construct and user friendly in software operations, as not every one is fluient with electronics and software as some of us are. If it was not for a common interest and pooling our resources this current project would not have come as far as it has. As you say there is a lot to think through and our comments are only intended to offer advice on areas that you may not have considerd yet and problems that you are sure to run into. There is a few who claim to have done simiular to this but are not prepared to share with the rest of us, what and how they have gone about it. I commend you on your implyed willingness to share you project and the discussion on it for us all to learn and benefit from your work. To those who love to tell us what they have done and not share the information, i say, shutup or put it up for us all to see. Pete. Sometimes it just works |
||||
| powerednut Senior Member Joined: 09/12/2009 Location: AustraliaPosts: 221 |
Thanks Pete and Gordon, I wasn't taking it as critism, just helpful suggestions. I'm a bit concerned about how to get it to talk to piclog. Thats something that would take the circuit from being usable by just me, to being usable to the community. When I get a little further down the road on this I'll download Gordon's PicLog software and see how it ticks. It might be easiest if I write a program that takes the files from the SD card and performs the necessary calculations on them before outputting PicLog files. Alternatively I'll just dump it all into an SQL database and handle the graphing with an app I write myself. I'd prefer to use piclog though - Gordon's done some really nice work there by the look of it. |
||||
| powerednut Senior Member Joined: 09/12/2009 Location: AustraliaPosts: 221 |
sorry for the double post Missed some comments from Gordon - sorry about that. Yeah, the 1-2 Watts is about what I was expecting. My watt hour/day estimate was based on expected (1.5-2) * 24 (hrs running per day). The higher figure was obtained by multiplying by 2, which I figure will probably cover newbie design mistooks. The database is probably inevitable, but I have a fair bit of experience with them. I'm a programmer really, so I'm more worried about the electronics side of things than the software. I'm a complete electronics-newbie really, so it'll be an interesting learning experience.  |
||||
Steve9R Regular Member  Joined: 24/01/2006 Location: AustraliaPosts: 72 |
Im looking forward to the circuit diagram for the logger so i can build one to test it.. any idea on eta ? Cheers Steve |
||||
| Page 3 of 7 |
|||||
 Print this page |
