Datalogger

More
26 Jun 2013 03:52 #11364 by PhracturedBlue
Datalogger was created by PhracturedBlue
Since the new gui layout is just about finished, and Rbe has redone the Devo12 pages, I thought I'd tackle the next item on my list.

So I've started work on a data-logger.

Here is an example of the configuration screen:



My plan is that the configuration is done per-model. There will be a directory 'datalog' which contains a bunch of files (datalog1.txt, datalog2.txt, ...) Each file will be 4kB and they will be filled in series (write 4k of data to datalog1.txt then switch to datalog2.txt)
The logs will be written periodically in 'append' mode (so restarting the Tx or switching models will not overwrite previous data.
There will be a button to reset the log files and start writing from the beginning.

The idea is that you can select which data you want to capture from the above config page.

I'm not yet sure how to write the data. If I write it in csv format, it'll be easy to work with, but will potentially use up a lot of space.
I can write binary data in a much more compressed manner, but then you'll need some application to extract it into a usable format.

We are again limited by the filesystem code in that I cannot create or resize files on the Tx, so all files need to be pre-created. i could use a single big file instead of many small ones, however then resizing the file needs special toos.
Attachments:

Please Log in or Create an account to join the conversation.

More
26 Jun 2013 04:58 #11370 by Kdean
Replied by Kdean on topic Datalogger
After thinking about a data logger in the tx and what practical uses it would have, i can only see a program that can do a virtual playback of a flight.

What are some of the other uses for such a logger?

Please Log in or Create an account to join the conversation.

More
26 Jun 2013 05:49 - 26 Jun 2013 05:53 #11375 by vlad_vy
Replied by vlad_vy on topic Datalogger
Will be nice to have datalogger view screen, there we can select (by one or several items) and display data as graphs with different colors.

Is it possible to export data from binary to csv format by any means (over USB or anything else)?
Last edit: 26 Jun 2013 05:53 by vlad_vy.

Please Log in or Create an account to join the conversation.

More
26 Jun 2013 12:00 #11401 by Kdean
Replied by Kdean on topic Datalogger
After more thinking.........

It can also function as a "Black Box" like real aircraft have. After a crash you can look at the data to see what happened before impact. Would still need a simulator like play back to see attitude of aircraft with given inputs. But at least the telemetry data could be useful to see if it was a power issue or burn out of esc or motor.

Please Log in or Create an account to join the conversation.

More
26 Jun 2013 12:29 - 26 Jun 2013 12:31 #11405 by RandMental
Replied by RandMental on topic Datalogger
I see an option to export / replay this file via ppm-out into
a simulator (Phoenix) to see you flight and the reason for that "crash"!
Last edit: 26 Jun 2013 12:31 by RandMental.

Please Log in or Create an account to join the conversation.

More
26 Jun 2013 12:36 #11406 by FDR
Replied by FDR on topic Datalogger

RandMental wrote: I see an option to export / replay this file via ppm-out into
a simulator (Phoenix) to see you flight and the reason for that "crash"!


I think you couldn't use for that, since it would be sampled with limited frequency, so it wouldn't play back the command changes between those samples...

Please Log in or Create an account to join the conversation.

More
26 Jun 2013 12:45 #11407 by RandMental
Replied by RandMental on topic Datalogger

FDR wrote:

RandMental wrote: I see an option to export / replay this file via ppm-out into
a simulator (Phoenix) to see you flight and the reason for that "crash"!


I think you couldn't use for that, since it would be sampled with limited frequency, so it wouldn't play back the command changes between those samples...


Yip, your right of course, a case of typing rather than thinking it through. Would have been nice, though.

Please Log in or Create an account to join the conversation.

More
26 Jun 2013 14:31 #11413 by PhracturedBlue
Replied by PhracturedBlue on topic Datalogger
I think the most likely use for datalogging is in conjunction with telemetry. For the FPV folks, it could provide maps of the flight. For others, it would contain things like battery state.

I'm not sure the Tx is really a useful place to look at the data, but it could be done.
You can't convert to csv on the fly, but it would be easy to have an app that could do so.

My guess is that the value of more data outweighs the readability of it. Also If we want to view it on the Tx, binary will require a lot less code.

Please Log in or Create an account to join the conversation.

More
26 Jun 2013 14:31 - 26 Jun 2013 14:32 #11414 by RandMental
Replied by RandMental on topic Datalogger

RandMental wrote:

FDR wrote:

RandMental wrote: I see an option to export / replay this file via ppm-out into
a simulator (Phoenix) to see you flight and the reason for that "crash"!


I think you couldn't use for that, since it would be sampled with limited frequency, so it wouldn't play back the command changes between those samples...


Yip, your right of course, a case of typing rather than thinking it through. Would have been nice, though.


Hi FDR, PB,

May be it is possible, using differential encoding decoding and a fast enough clock such a differential bit stream may be able to give a good Simulator experience on 6 channels.

Do you know what is the typical or required rate of change of the PPM signals when flying? If we work on a 5% PPM resolution (steps), updated at the TX 8 channel frame rate. it might just work to log a 5-6 minute flight in 4kbyet files.

Will do some PPM measurements tonight...
Last edit: 26 Jun 2013 14:32 by RandMental.

Please Log in or Create an account to join the conversation.

More
26 Jun 2013 14:47 #11417 by FDR
Replied by FDR on topic Datalogger
I'm a bit skeptical about the possibility of playing back a log on the sim whatever fine the sampling is.
The physics are different and the conditions like wind differ too, so the same commands would result in a very different behaviour. The model in the sim would probably crash very soon...

Please Log in or Create an account to join the conversation.

More
26 Jun 2013 14:56 #11418 by PhracturedBlue
Replied by PhracturedBlue on topic Datalogger
for the joystick, I currently use 8-bits per channel and it seems to be sufficient. So that provides 680 samples per 4kb file for 6channels. At 100msec sampling rate, that provides ~1 minute of flight time. But any session won't be limited to a single 4k file, so this may be workable. It wasn't really my purpose though. Initially the sample rate will likely be limited to 1sec.

Please Log in or Create an account to join the conversation.

More
26 Jun 2013 14:58 #11420 by PhracturedBlue
Replied by PhracturedBlue on topic Datalogger

FDR wrote: I'm a bit skeptical about the possibility of playing back a log on the sim whatever fine the sampling is.
The physics are different and the conditions like wind differ too, so the same commands would result in a very different behaviour. The model in the sim would probably crash very soon...

Yes I agree, as a playback it wouldn't make much sense other than to analyze your stick movements to see what if anything you were doing wrong.

Please Log in or Create an account to join the conversation.

More
26 Jun 2013 15:06 #11421 by Kdean
Replied by Kdean on topic Datalogger
I dont think the sim idea that you guys are thinking of is what im seeing in my head. I know it would be nice to see the flight in a 3d enviorment, i was thinking of a static model moveing on 3 axis (pitch, roll, and yaw) but even calibrating these with what the actual model is doing would be a challange. I know skookum has a 3 axis gyro that can play back a simple display of the flight, but the gyro is attached to the model itself. HelisimRC (free download sim that i use) has a training function that locks the axis. Take a look at it if you dont know what i'm rambeling on about.

Please Log in or Create an account to join the conversation.

More
26 Jun 2013 16:53 #11424 by RandMental
Replied by RandMental on topic Datalogger

FDR wrote: I'm a bit skeptical about the possibility of playing back a log on the sim whatever fine the sampling is.
The physics are different and the conditions like wind differ too, so the same commands would result in a very different behaviour. The model in the sim would probably crash very soon...


Hi FDR, I have to agree to a certain extend, but the converse is also true - by practising on a SIM you have the perfect helicopter, and while it helps training for orientation and muscle memory, in real life you have to content with a non-perfect heli which depending on the mechanical setup and off centre COB, etc, can be a very different animal. (i know you can change the heli config, weather, etc, on most SIMs, but that is not the point here)

So perhaps there is still a case to be made for such a TX output logger, BUT then it is probably easier just to record the PPM-out signals and find a way to replay them into the PPM-in HID dongle. Having said that, PB can we add the option to output PPM even when flying with other protocols?
Then we can verify and test my suggestion without affecting your plans for the data logger.

Please Log in or Create an account to join the conversation.

More
29 Jun 2013 21:45 - 29 Jun 2013 21:47 #11594 by PhracturedBlue
Replied by PhracturedBlue on topic Datalogger
I've checked in support for the data-logger for the Devo6/Devo8/Devo12

I thought I'd get it out there for testing and feedback while I work on the Devo10 interface (this is another feature that won't be available on the 7e)

Notes:
You can only change the selected elements when the logger is not running.

There is currently no way to toggle it from the main screen

The data is written in a binary format, and does not include the header information. This means if you run a custom firmware with different capabilities, you'll need to also alter the parsing script to be able to decode the log.

The log is in datalog.bin. If this file isn't available, logging will be disabled. The default is 16kb but it can be as large as you like.

There is a script in utils/datalog2csv.py that can be used to convert the binary data into a csv.

I will supply a compiled python (exe) version with the release so users don't need python to use it
Last edit: 29 Jun 2013 21:47 by PhracturedBlue.

Please Log in or Create an account to join the conversation.

More
30 Jun 2013 06:09 - 30 Jun 2013 06:13 #11617 by vlad_vy
Replied by vlad_vy on topic Datalogger
Empty place after Timer4


TelemRPM1 only (TelemRPM2???)


What is the Time? Other GPS options (Altitude, Speed)?


Wiil be nice to group all telemetry options at the top of the list.
Attachments:
Last edit: 30 Jun 2013 06:13 by vlad_vy.

Please Log in or Create an account to join the conversation.

More
30 Jun 2013 13:07 - 30 Jun 2013 13:10 #11627 by PhracturedBlue
Replied by PhracturedBlue on topic Datalogger
Oops. I noticed the space before, but forgot to address it (fixed). the Time is RTC, I've renamed it. The GPSLoc previously included the altitude and speed. now it is separate. I also added loading/saving as well as paging on the scrollbar.

Also, I chose not to put the GPS stuff at the top because very few folks will actually use it. The RTC is at the end to keep the code simple due to it being conditionally available.
Last edit: 30 Jun 2013 13:10 by PhracturedBlue.

Please Log in or Create an account to join the conversation.

More
01 Jul 2013 06:36 #11663 by vlad_vy
Replied by vlad_vy on topic Datalogger
Sample rate??? After 60sec empty space.
Attachments:

Please Log in or Create an account to join the conversation.

  • rbe2012
  • rbe2012's Avatar
  • Offline
  • So much to do, so little time...
More
01 Jul 2013 06:38 #11664 by rbe2012
Replied by rbe2012 on topic Datalogger
I want to play with the datalogger. First I looked into the sources and this line
"MIX0", "MIX1", "MIX2", "FMODE1", "FMODE2", "FMODE3"]
in the utils/datalog2csv.py (4 times) looks strange: I expected the FMODE also from 0 to 2. For Devo7e it seems correct (FMODE0 and 1).

Please Log in or Create an account to join the conversation.

  • rbe2012
  • rbe2012's Avatar
  • Offline
  • So much to do, so little time...
More
01 Jul 2013 06:54 #11665 by rbe2012
Replied by rbe2012 on topic Datalogger
Sample rate on config page:
If you have reached 60s you can just go further right. Text disappears.
Look here in pages/common/_datalog_page.c line 28++:
static const char *ratesel_cb(guiObject_t *obj, int dir, void *data)
{
    (void)obj;
    (void)data;
    dlog.rate = GUI_TextSelectHelper(dlog.rate, 0, DLOG_RATE_LAST, dir, 1, 1, NULL);
    return DATALOG_RateString(dlog.rate);
}
DLOG_RATE_LAST should be (DLOG_RATE_LAST-1).

Please Log in or Create an account to join the conversation.

Time to create page: 0.130 seconds
Powered by Kunena Forum