Simple Logger

Developer topics relating to software that logs data from ECUs

Moderator: Freon

Postby west_minist » Sun May 29, 2005 8:42 am

Hi Guys!

I knowt that linux have ability to draw graph with data provided to them.

How easy is it to logged from the screen to a file to do this? can it be done real time?
west_minist
 
Posts: 515
Joined: Fri Jan 07, 2005 1:31 pm
Location: Barbados

Postby realwomble » Sun May 29, 2005 10:54 am

Guys,

I've ported Mikes code to native windows APIS, anc have it talking to the ECU now.

I'm working on generic handling of *all* supported parameters, and a generic way of describing different ECU versions (where locations can change).

It'll be written in such a way to make it easy to add "dashboard" like function as well.

Once that is done, I'll post it all up here.

For graphing, I was planning to investigate using PERL to parse the log and drive EXCEL. Years back I know it was simple ot use PERL and one of the included modules to "drive" excel - build a sheet, configure chart etc. I've had no tgime to investigate this, so is someone else has time... :)

Thanks,
Tom
realwomble
 
Posts: 21
Joined: Thu May 26, 2005 4:07 pm
Location: Seattle, WA

Postby west_minist » Sun May 29, 2005 11:02 am

There are graphing functions like png and libpng that does graphing.

I guess with my very limited knowledge of programming, I can look into this area. I would also look for a windows program that can read a logg and draw a graph.
west_minist
 
Posts: 515
Joined: Fri Jan 07, 2005 1:31 pm
Location: Barbados

Postby cdvma » Sun May 29, 2005 2:59 pm

realwomble wrote:Guys,

I've ported Mikes code to native windows APIS, anc have it talking to the ECU now.

I'm working on generic handling of *all* supported parameters, and a generic way of describing different ECU versions (where locations can change).

It'll be written in such a way to make it easy to add "dashboard" like function as well.

Once that is done, I'll post it all up here.

For graphing, I was planning to investigate using PERL to parse the log and drive EXCEL. Years back I know it was simple ot use PERL and one of the included modules to "drive" excel - build a sheet, configure chart etc. I've had no tgime to investigate this, so is someone else has time... :)

Thanks,
Tom


Windows APIs = BAD

We should keep this as much ANSI C as possible and for each system, if required, make an abstraction layer for the serial interface so that the communications are platform specific but the logic and data handling is not. This will be much more benneficial. I say first do the abstraction layer and then worry about getting the data.

In order for this system to be pretty, a GUI is going to be needed as well. Pick, again, a x-platform graphics API. There are a few out there...
cdvma
 
Posts: 86
Joined: Tue Jan 04, 2005 9:18 pm
Location: Roch. Inst. of Tech.

Postby west_minist » Sun May 29, 2005 3:30 pm

I work in the IT Field. I am a Network Engineer.

I can tell you that modularity really works. I agree w/ cdvma.

I think we should have 3 mods. Interface, Logger & GUI. It would be like protocol analyser in computer networks. Eg, Ethereal http://www.ethereal.com

I personal think the logger with need more work due to different subaru CPU's out there.

Thats my 2 cents worth. :)
west_minist
 
Posts: 515
Joined: Fri Jan 07, 2005 1:31 pm
Location: Barbados

Postby realwomble » Sun May 29, 2005 4:20 pm

cdvma wrote:
Windows APIs = BAD

We should keep this as much ANSI C as possible and for each system, if required, make an abstraction layer for the serial interface so that the communications are platform specific but the logic and data handling is not. This will be much more benneficial. I say first do the abstraction layer and then worry about getting the data.

In order for this system to be pretty, a GUI is going to be needed as well. Pick, again, a x-platform graphics API. There are a few out there...


You're entitled to your opinion ;)

Personally, I don't really want to waste time right now with clunky x-platform stuff.

I'm not really concerned with "pretty".

My priorities are

1. get the data collection core working well, in a way that can cope with the many different ECU revisions and differing field locations etc.

2. initially just produce logs

3. use existing software (e.g. PERL +excel or similiar) to produce logs and charts. Why reinvent the wheel, again.

4. maybe throw on a UI later.
Last edited by realwomble on Sun May 29, 2005 4:26 pm, edited 1 time in total.
realwomble
 
Posts: 21
Joined: Thu May 26, 2005 4:07 pm
Location: Seattle, WA

Postby realwomble » Sun May 29, 2005 4:22 pm

west_minist wrote:I work in the IT Field. I am a Network Engineer.

I can tell you that modularity really works. I agree w/ cdvma.

I think we should have 3 mods. Interface, Logger & GUI. It would be like protocol analyser in computer networks. Eg, Ethereal http://www.ethereal.com

I personal think the logger with need more work due to different subaru CPU's out there.

Thats my 2 cents worth. :)


Ok... I'm familiar with ethereal.... I used it regularly.

I'm also familiar with the concept of modularity ...

I write OS kernel software as my day job...
realwomble
 
Posts: 21
Joined: Thu May 26, 2005 4:07 pm
Location: Seattle, WA

Postby west_minist » Sun May 29, 2005 5:05 pm

Thats good.

That my view. Get the hard sorted for different platforms, but I think 90% of the work will be on the logger.
west_minist
 
Posts: 515
Joined: Fri Jan 07, 2005 1:31 pm
Location: Barbados

Postby bofh » Wed Jun 01, 2005 12:54 pm

Can I agree with both sides? The windows API is easy, so it makes sense to use it. It is also on most peoples computers already, which helps. For example, try and talk my mother through installing cgywin... :shock:

On the other hand, the code should be modular enough that the OS specific APIs can be replaced with Windows, Linux or palm as needed.
bofh
 
Posts: 50
Joined: Thu Dec 30, 2004 1:59 pm
Location: Houston, TX

Another implementation

Postby ll » Wed Jun 01, 2005 1:19 pm

I've taken the sample code posted here and re-written it as a platform independent Java application (although this has yet to be entirely tested).

So far this is complete:
- Communication with the ECU using SSP with dynamic queries. (the byte addresses for parameters are not hard-coded into the code).
- ECU Parameters are completely declarative. You give them a name, declare a class for handling the conversion, and specifiy a byte address. This will allow anyone to re-configure the application for any ECU which understands SSP or add parameters I have left out.

I plan to have a simple console and log renderer done by this weekend. At that point I'll probably post my source here and offer some rudamentary installation/usage instructions.

Here's my short-term To-Do list. If you want to tackle any of this send me a PM. I'd be willing to send you my current code if you're itching to get started, but we'd probably have some merge issues as we don't have any kind of source control.

-Build a generic, declarative interface for "renderers" (console, dashbord, log, graphical charting, ect). I will have this done for the first "release".
-Come up with some simple command line arguments for run time configuration. Might look something like this for linux:
Code: Select all
./openECU.sh -render console, csvlog -param RPM MAP EGT KNK -file ~/logs/myLog.csv

-Build a configurable "dashboard" UI in swing. This will probably be digital at first but I plan to make it all pretty with analogue gauges and skins eventually.

I'm more of an enthusiast than a turner so my need is to verify my car is operating within reasonable parameters while I'm driving it. That's where my work will take me initially. I'm not very interested in logging tons of data and pouring over it for hours.

As a group we need to do a few things in order to make this a successful collaborative effort. Generally in this order:

1. Decide on a platform (I'm Java biased, of course, but I think QT/C++ would be a viable alternative). We need platform independence for sure.
2. Set up the underpinnings of a successful open source project.
A. Make somebody the "leader" and come up with a process.
B. Assign roles and build a development roadmap.
C. Set up a website for hosting releases and bug-tracking, cvs/svn ect. Sourceforge would be an excellent solution for this at least initially.


I don't have time to be this "leader" guy right now. I'm involved in another project and I'm getting married in two weeks, but I'm sure there's somebody smart here who'd love to step up to the plate. You'll get to put "Organized an open source development project" on your resume at the very least.

cheers, ll
ll
 
Posts: 6
Joined: Wed Jun 01, 2005 12:43 pm

Postby cboles » Wed Jun 01, 2005 1:22 pm

I know I haven't shared my code yet, but this is what I'm shooting for. Right now I have a bunch of GUI-less C++ console apps in Windows to do various things like reflash the ECU. The only code you would have to change for *nix would be some of the serial port code and a delay function. Very easy to put a conditional compile in for it. Eventually I will replace the logging / messages going to stdout with some better form of callback messaging so that this core code can be easily harnessed inside of a GUI without it becoming GUI platform specific.
cboles
Site Admin
 
Posts: 1233
Joined: Wed Dec 29, 2004 5:45 pm
Location: Seattle, WA

Postby realwomble » Wed Jun 01, 2005 4:05 pm

Unfortunately the total time I have available to spend on this is measurable in hours, so I won't be much help.

I have a simple C data collector / logger near complete which will pretty much satisfy my aims (given that I'm not bothered about UI at all - I'll use perl / excel).

I'll post all the code here when done (couple days) if its useful to anyone else then thats just a bonus.

- built as win32 console app

- platform specific parts are (1) COM port stuff (uses Windows API, easy to swap out) and (2) threading / synchronisation (uses a couple of threads during data collection so the data consumers do not impact the collection process).

- ecu info compiled in, but structures could easily be filled from (e.g.) XML file or whatever.

- Supports completely different memory layouts for different ECU revisions.

The other important part of all this that nobody has really mentioned is the need to increase the volume of ECU knowledge (different revisions / differences etc) and keep it up to date and public here on openecu.

Llack of that knowledge has been the real barrier to people writing software.
realwomble
 
Posts: 21
Joined: Thu May 26, 2005 4:07 pm
Location: Seattle, WA

This is a problem

Postby ll » Thu Jun 02, 2005 7:36 am

We're all duplicating one another's work here, which isn't really helping anyone out. Between the origional post, my work and the work of others I'm sure we probably have five or six applications which generally do the *exact* same thing. I think we'd all be a lot further down the road to a comprehensive and useful tool if we'd figured out how to work together first. But hindsight is always 20/20, right? So where do we go from here?

First, we *really* need some source control. The defacto route to this end would be a sourceforge project. That way we can at least get everyone's current work up someplace (even if it isn't finished). At least we can save the late stragglers the pain of re-discovering all our hard-won knowledge. Additionally this will allow people to contribute to the existing efforts versus spinning off their own tool.

For the source forge project we need two things:

1. A comprehensive description of our project (I can come up with this), or cboles? He started this whole deal.
2. The license under which our project will be released. GPL? need to discuss.

Let's figure this out quickly and I'll create the account.
ll
 
Posts: 6
Joined: Wed Jun 01, 2005 12:43 pm

Postby realwomble » Sat Jun 04, 2005 1:50 pm

Posted what I have so far, source to follow shortly

http://forums.openecu.org/viewtopic.php?p=491#491
realwomble
 
Posts: 21
Joined: Thu May 26, 2005 4:07 pm
Location: Seattle, WA

Re: This is a problem

Postby cdvma » Sun Jun 05, 2005 6:41 am

ll wrote:We're all duplicating one another's work here


Yea.

Code production needs to stop. It will make more work in the end, trust me. A single codebase needs to be selected and used with very clear guidlines for design. A few things need to be established first.

What are the target platforms?

I'd imagine everything popular so Windows, Mac X, Linux (2.4 onward).

So we have a problem. They all use different serial APIs. Oh no wait...maybee not because it depends on the language used. Java will use the same library but Java isn't the best language for this task. Its really easy to make Java code HUGE, and really slow (aka wrong), especially with multiple developers.

Assuming we have a de-facto language, what needs to be done?

As cboles has been working on or outlined, there are several aspects that need to be done. A serial communication layer needs to be created so that all aspects of the program can communicate with the ECU in a standard manner. All the code that is out there is hard coded for logging. Not good.

We need a serial abstraction layer for each OS, a communications API and mini apps on top of that for data interaction such as logging, flashing ect.

If we want to make progress, we don't need more logging code, we don't need a GUI. What we need is the foundation - a communications API definition. I think C++ would be a fantastic language for what we need. Cross platform and quick, its class based which screams modularity. Someone should make a comm API and we should also get input on which graphics library to eventually use. GUIs take a long time thats why I bring it up so soon as food for thought.

We have proved that logging is easy. Now make it good :)
cdvma
 
Posts: 86
Joined: Tue Jan 04, 2005 9:18 pm
Location: Roch. Inst. of Tech.

PreviousNext

Return to Data Logging Software

Who is online

Users browsing this forum: No registered users and 4 guests