ASE Home Page Products Download Purchase Support About ASE
ChartDirector Support
Forum HomeForum Home   SearchSearch

Message ListMessage List     Post MessagePost Message

  High resolution (<1ms) chart data acquisition - VS2019 C#
Posted by William on Jan-20-2021 18:43
Hello,

Well, I have given a glance at the previous posts but it seems that nothing clear appears so far about how to manage high resolution DAQ. Unfortunately, when it comes to timers, Windows doesn't want us to enjoy high speed signals. So here is the deal. I intend to create a datalogger that displays in a realtime chart data coming from an external application via TCP port. In a first place, I thought that something went wrong with the TCP link but with a homemade test app, I have tested the transmission and visualized it on Wireshark and the 5ms interval looks OK for Wireshark. In a second time, I thought that something was wrong with the code and there will probably be but even if the timer is on, I can see that if I plot a math buffer like this :
                double dataB = 3.2 + 0.01 * Math.Cos(p / 6.7) * Math.Cos(p / 11.9);
my plot is just perfect but my double dataA is my data from TCP and it does not handle streams faster than 15~16ms which is windows thread time. Now, I know that oscilloscopes can handle sampling frequencies up to 1GSPS and even higher but my app does not need such a fast sampling frequency. I would kindly settle on a 1 sample/millisecond, 10 samples should be the dream!

My config is a code in C# on Visual Studio 2019, .NET Framework 4.5,
I acquire TCP frames with the simpleTCP nuget,
all my acquisition process is made in the class private void Server_DataReceived(object sender, SimpleTCP.Message e)
and I parse my TCP message in txtStatus.Invoke((MethodInvoker)delegate ().
By the way, I declare all my parsing variables and my dataX buffers in this Server_DataReceived class. I don't know whether it is a big deal for the application.
I have tried it in RealTimeMeasure and in RealTimeViewPort. I was thinking about using RealTimeMultithread but I don't know if it would make a big difference and if so, how I could reach high "bandwith".

Best regards

  Re: High resolution (<1ms) chart data acquisition - VS2019 C#
Posted by Peter Kwan on Jan-21-2021 12:36
Hi William,

You do not need to worry about the clock in your computer. For your case, the clock that matter is in your datalogger. The PC does not need to sample the high speed signal. Your datalogger will sample the high speed signal. Your datalogger should have an accurate clock for that purpose.

Your datalogger then transfer the data to your computer. You can image that it is like a file transfer. In your message, you mention the transfer occurs every 5ms via TCP. If your datalogger gets a sample every 1ms, then each transfer must contain an average of 5 samples, otherwise your datalogger or network is losing data. You can see that the 5ms is not important. It can transfer every 20ms and the system will still work normally. It just includes 20 samples in each transfer. The important thing is no data is lost.

In a typical computer, when it receives network data, it may not process is immediately as the computer has many other things to do. The data will be stored in a buffer. Eventually, the computer will process the data. If the computer process the data after 20ms, it should see 4 packets with a total of 20 samples in the buffer. Again the timing is not critical. The important thing is no data is lost.

The above is similar to how youtube or zoom video conference streams data to your computer. As long as your network and computer is fast enough so that no data is lost, you can see the video normally.

In your case, the transfer is done by simpleTCP. As long as this system does not lose data, the chart should be fine. In Server_DataReceived, you should receive your samples and no samples are lost.

You can start with the Multi-Threading Real-Time Chart sample code if you like. You do not need to multi-threading random number generator, so please disable that part of the code.

In the original sample code, there is an onData subroutine:

"private void onData(double elapsedTime, double series0, double series1)"

So in your "private void Server_DataReceived(object sender, SimpleTCP.Message e)", you just need to obtain the data value and elapsedTime and pass the value to the chart by call onData. I would imagine it is like:


private double elapsedTime = 0;
private void Server_DataReceived(object sender, SimpleTCP.Message e)
{
      // As explained above, there should be multiple data values per message,
      // so some kind of loop may be need to get all the values
      while (.. still have more values ...)
      {
           double v = ... get one value ...;

           // pass the data to the chart
           onData(elapsedTime, v, 0);

           // As the datalogger samples the data every 1ms, so we just increment the
           // time by 0.001 second.
           elapsedTime += 0.001;
      }
}


With the above code, if no data are lost, the timeStamp must be correct and the chart will reflect the data.

Hope this can help.

Regards
Peter Kwan

  Re: High resolution (<1ms) chart data acquisition - VS2019 C#
Posted by William on Feb-05-2021 11:55
Attachments:
Thank you Peter,

Very interesting what you said. Sorry for the absence but I was trying to figure out what you said and how to embed this in the solution.

Well, when I used the word datalogger, there is nothing external, all is processed by my computer (app that sends the data and chartdirector based app that receives the data all over TCP) so there is no external device, PCB, whatsoever. I just use a client application which feeds data continuously. I guess it won't make a big difference except the fact that the other application uses Windows timers as well for which granularity ~15ms. This is if I send a "plain" timestamp, date hour, etc... . Maybe if I send the time in ms, I could skip these windows timers. I could give a glance at it. However, I should be very careful with that because, I can open many of them, something like 1 application per data channel and there should be a little delay from an application to the other for which the information elapsedTime will probably be erroneous. So the solution feed the timestamp or better the elapsed time in ms directly in TCP messages is interesting but there could be errors due to the fact that each app client will have its own starting time.

The only thing left I could do is to receive everything from these separate applications in my Chart Director-based program and manage the time directly in this last one.

As I said before, I tried to figure out what you recommended but RealTime Multithread is an overall module and the RandomWalk.cs doesn't help.
The elapsedTime +=0.001; is inaccurate because my Server_DataReceived() function is called only when needed so interval of time is random. My elapsedTime should be increased by an external timer interrupt (the MicroLibrary.cs library), so this library will provide to increase the elapsedTime variable. The point I can't solve is how to handle my data whereas I still have RandomWalk and don't know how to skip it.
My Server_DataReceived is like that :
        private void Server_DataReceived(object sender, SimpleTCP.Message e)
        {
            pos_1st_comma = e.MessageString.IndexOf(",", 0, e.MessageString.Length) + 1;
            pos_2nd_comma = e.MessageString.IndexOf(",", pos_1st_comma, e.MessageString.Length - pos_1st_comma) + 1;
            delta_1_commas = pos_2nd_comma - (pos_1st_comma + 1);
            string_ch_unit = e.MessageString.Substring(0, pos_1st_comma - 1);
            string_ch_name = e.MessageString.Substring(pos_1st_comma, delta_1_commas)
            string_value = e.MessageString.Substring(pos_2nd_comma, delta_2_commas);
            // pass the data to the chart
            v = Convert.ToDouble(string_value);
            onData(???, v, 0); //if I put elapsedTime instead of ??? the compiler tells me that elapsedTime does not exist in the current context
        }
So the message as you can understand is "string_ch_unit,string_ch_name,string_value". For example (not a concrete case but just to illustrate), I can send messages like these :
"V,Ch1,3.341" //V3.3
"mA,Ch2,0.02"  //V3.3 consumption
"MHz,Ch3,16.04" //Clk frequency

To remove RandomWalk, what should I do? Remove this bit in FrmRealtimeMultiThread_Load

dataSource = new RandomWalk(onData);
dataSource.start();

? It doesn't work. There is no display at all.

I could also keep RandomWalk.cs but modify the section while(!stopThread){} putting the values coming from my Server_DataReceived section but being part of the main cs file, I don't know how to handle this.

Anyway, I can send you the files as they are for the moment. It will be shorter than to write the whole code on this post. Here is the RealTimeMultiThread module of the project. And in order to help you better, I also attach a simulationTCP client that sends messages with identical format. Please pay attention to the fact that the timer interval textbox is in microseconds. But as long as it is a tool to help me debug better, I only made a functional tool, not a flawless program, for which if you enter intervals below 1000µs, the CPU will get mad. Other point, being a client, if the corresponding TCP Port is not opened by a server ready to listen to this very port, the message generator will crash. Except these two cases, everything should work fine.

Thank you for your contribution Peter. Your help is really appreciated.
TCP Message Generator.rar
TCP Message Generator.rar

6.81 Kb
RealTimeMultiThread.rar
RealTimeMultiThread.rar

25.66 Kb

  Re: High resolution (<1ms) chart data acquisition - VS2019 C#
Posted by William on Feb-10-2021 01:19
Hello Peter,

I have kept looking to the issue and I have some news.
I used a textbox to display every single sample and the relative timestamp to get better idea of the communication feed. For some reason, if I put all my feed parsing in this bit :
txtStatus.Invoke((MethodInvoker)delegate ()
{
//My code
if(trigger conditions == true)
//Enable timer
});

I will face my 15ms resolution issue. On the other hand, I will be allowed to use my timer for the trigger (I use a timer to freeze the chart - to stop the chart update - on event, for the moment, only when there is a jump of more than x volts between a sample and the previous one, maybe in the future, I could add a trigg to stop the chart update when there is a pulse of less than y ms).
Conversely, if I remove this txtStatus.Invoke delegate thing, the 15ms resolution issue disappears and the trigger does not work anymore and the chart goes on and on even if my trigger condition is verified.

What I did to tackle this is remove txtStatus.Invoke in my parsing code and so we have
//My code
if(trigger conditions == true)
   txtStatus.Invoke((MethodInvoker)delegate ()
   {//Enable timer
   };

I have tried this both in RealTimeMeasure and RealTimeViewport and it is confirmed for both. In the meantime, I am still out of ideas for RealTimeMultiThread :/ .

Regards,

  Re: High resolution (<1ms) chart data acquisition - VS2019 C#
Posted by Peter Kwan on Feb-10-2021 14:45
Attachments:
Hi William,

I have tested your code. After I remove the MicroTimer. It is not necessary as the data time comes from your data, not from the timer. I also remove the random number generator. I obtain the data from your TCP code by adding 4 lines to your code, and it works normally.

The modified Server_DataReceived becomes:

double currentTime = 0;
private void Server_DataReceived(object sender, SimpleTCP.Message e)
{
pos_1st_comma = e.MessageString.IndexOf(",", 0, e.MessageString.Length) + 1;
pos_2nd_comma = e.MessageString.IndexOf(",", pos_1st_comma, e.MessageString.Length - pos_1st_comma) + 1;
pos_3rd_comma = e.MessageString.IndexOf(",", pos_2nd_comma, e.MessageString.Length - pos_2nd_comma) + 1;
pos_4th_comma = e.MessageString.IndexOf(",", pos_3rd_comma, e.MessageString.Length - pos_3rd_comma);
delta_3_commas = pos_4th_comma - pos_3rd_comma;
delta_2_commas = pos_3rd_comma - (pos_2nd_comma + 1);
delta_1_commas = pos_2nd_comma - (pos_1st_comma + 1);
string_1 = e.MessageString.Substring(pos_2nd_comma, delta_2_commas);
string_2 = e.MessageString.Substring(pos_3rd_comma, delta_3_commas);
v = Convert.ToDouble(string_1);
double v2 = Convert.ToDouble(string_2);
onData(currentTime, v, v2);
currentTime += 0.01;
}

The first line "double  currentTime = 0;" and the last 3 lines are added by me to pass the data to ChartDirector.

You can zoom in the chart and verify that each data point is 0.01 second apart.

If your data points are 1ms apart, you can change the increment from 0.01 to 0.001.

Attached please find the modified code.

For your "TCP Message Generator", I am not sure how it works. It should send the data to the server may be every 50ms. If it is collecting data every 1ms, then each packet should should contain 50 data points. (I see that in your code, there are 2 data points per line. If you have 3 data channels, each collecting data every 1ms, you can send 150 data points per line.)

In summary, the timer on the server is not important. The important thing is the timer in your data source (the TCP Message Generator), and that it can send all the data to the server without data loss.

Regards
Peter Kwan
frmrealtimemultithread.cs
using System;
using System.Collections.Generic;
using System.Text;
using System.Windows.Forms;
using ChartDirector;
using SimpleTCP;

namespace ChartDirectorSampleCode
{
    public partial class FrmRealTimeMultiThread : Form
    {
        float[] Level1 = new float[6];
        float[] Level1_prev = new float[6];
        float[] Level2 = new float[6];
        float[] Level2_prev = new float[6];
        string channel_name;
        int index_channel = 0;
        int max_channel = 0;
        int j = 0;
        bool start_npause;
        int i = 0;
        double v = 0;

        int pos_1st_comma, pos_2nd_comma, pos_3rd_comma, pos_4th_comma;
        int delta_1_commas, delta_2_commas, delta_3_commas;
        string string_1, string_2;
        //int test_channel = 0;

        bool starting_time = true;
        int hour, minute;
        double second;
        int chart_timer_interval = 100;


        // The random data source
        // private RandomWalk dataSource;

        // Declare MicroTimer
        //private readonly MicroLibrary.MicroTimer _microTimer;

        double ElapsedSeconds = 0;

        // A thread-safe queue with minimal read/write contention
        private class DataPacket
        {
            public double elapsedTime;
            public double series0;
            public double series1;
        };
        private DoubleBufferedQueue<DataPacket> buffer = new DoubleBufferedQueue<DataPacket>();

        // The data arrays that store the realtime data. The data arrays are updated in realtime. 
        // In this demo, we store at most 10000 values. 
        private const int sampleSize = 10000;
        private double[] timeStamps = new double[sampleSize];
        private double[] dataSeriesA = new double[sampleSize];
        private double[] dataSeriesB = new double[sampleSize];
/*        private double[] dataSeriesC = new double[sampleSize];
        private double[] dataSeriesD = new double[sampleSize];
        private double[] dataSeriesE = new double[sampleSize];
        private double[] dataSeriesF = new double[sampleSize];*/
        private string[] Channel = new string[6];

        // The index of the array position to which new data values are added.
        private int currentIndex = 0;

        // The full range is initialized to 60 seconds of data. It can be extended when more data
        // are available.
        private int initialFullRange = 60;

        // The maximum zoom in is 5 seconds.
        private double zoomInLimit = .1;

        // If the track cursor is at the end of the data series, we will automatic move the track
        // line when new data arrives.
        private double trackLineEndPos;
        private bool trackLineIsAtEnd;

        System.ComponentModel.ComponentResourceManager resources = new System.ComponentModel.ComponentResourceManager(typeof(FrmRealTimeMultiThread));

        public FrmRealTimeMultiThread()
        {
            InitializeComponent();
            // Instantiate new MicroTimer and add event handler
            //_microTimer = new MicroLibrary.MicroTimer();
            //_microTimer.MicroTimerElapsed +=
            //    new MicroLibrary.MicroTimer.MicroTimerElapsedEventHandler(OnTimedEvent);
        }

        SimpleTcpServer server;

        //private void OnTimedEvent(object sender,
        //                          MicroLibrary.MicroTimerEventArgs timerEventArgs)
        //{
            // Do something small that takes significantly less time than Interval.
            // BeginInvoke executes on the UI thread but this calling thread does not
            //  wait for completion before continuing (i.e. it executes asynchronously)
            //ElapsedSeconds = timerEventArgs.ElapsedMicroseconds;
            //ElapsedSeconds = ElapsedSeconds / 1000000;
            //if (InvokeRequired)
            //{
                //BeginInvoke((MethodInvoker)delegate
                //{
                    //client.WriteLine(DateTime.UtcNow.ToString("HH:mm:ss.fff") + " " + "\\r\\n");
                    //TextBoxElapsedTime.Text =  ElapsedSeconds.ToString();
                    //TextBoxElapsedTime.Text =  i.ToString();
                //});
            //}
        //}

        private void FrmRealtimeMultiThread_Load(object sender, EventArgs e)
        {
            // Initialize the WinChartViewer
            initChartViewer(winChartViewer1);

            // Start the random data generator
            //dataSource = new RandomWalk(onData);
            //dataSource.start();

            // Now can start the timers for data collection and chart update
            chartUpdateTimer.Interval = 100;
            chartUpdateTimer.Start();

            //long interval;

            // Read interval from form
            //if (!long.TryParse(TextBoxInterval.Text, out interval))
            //{
            //    return;
            //}

            // Set timer interval
            //_microTimer.Interval = interval;

            // Ignore event if late by half the interval
            //_microTimer.IgnoreEventIfLateBy = interval / 2;

            // Start timer
            //_microTimer.Start();


            server = new SimpleTcpServer();
            //server.Delimiter = 0x59;
            server.Delimiter = 0x13;
            server.StringEncoder = Encoding.UTF8;
            server.DataReceived += Server_DataReceived;
            
        }

        private void FrmRealTimeMultiThread_FormClosing(object sender, FormClosingEventArgs e)
        {
            //if (null != dataSource)
            //    dataSource.stop();
            // Stop the timer, wait for up to 1 sec for current event to finish,
            //  if it does not finish within this time abort the timer thread
            //if (!_microTimer.StopAndWait(1000))
            //{
            //    _microTimer.Abort();
            //}
        }

        //
        // Initialize the WinChartViewer
        //
        private void initChartViewer(WinChartViewer viewer)
        {
            viewer.MouseWheelZoomRatio = 1.1;

            // Initially set the mouse usage to "Pointer" mode (Drag to Scroll mode)
            pointerPB.Checked = true;
        }

        //
        // Handles realtime data from RandomWalk. The RandomWalk will call this method from its own thread.
        //
        private void onData(double elapsedTime, double series0, double series1)
        {
            DataPacket p = new DataPacket();
            p.elapsedTime = elapsedTime;
            p.series0 = series0;
            p.series1 = series1;
            buffer.put(p);
        }

        double currentTime = 0;
        private void Server_DataReceived(object sender, SimpleTCP.Message e)
        {
            pos_1st_comma = e.MessageString.IndexOf(",", 0, e.MessageString.Length) + 1;
            pos_2nd_comma = e.MessageString.IndexOf(",", pos_1st_comma, e.MessageString.Length - pos_1st_comma) + 1;
            pos_3rd_comma = e.MessageString.IndexOf(",", pos_2nd_comma, e.MessageString.Length - pos_2nd_comma) + 1;
            pos_4th_comma = e.MessageString.IndexOf(",", pos_3rd_comma, e.MessageString.Length - pos_3rd_comma);
            delta_3_commas = pos_4th_comma - pos_3rd_comma;
            delta_2_commas = pos_3rd_comma - (pos_2nd_comma + 1);
            delta_1_commas = pos_2nd_comma - (pos_1st_comma + 1);
            string_1 = e.MessageString.Substring(pos_2nd_comma, delta_2_commas);
            string_2 = e.MessageString.Substring(pos_3rd_comma, delta_3_commas);
            v = Convert.ToDouble(string_1);
            double v2 = Convert.ToDouble(string_2);
            onData(currentTime, v, v2);
            currentTime += 0.01;
        }

            //
            // Update the chart and the viewport periodically
            //
        private void chartUpdateTimer_Tick(object sender, EventArgs e)
        {
            WinChartViewer viewer = winChartViewer1;
            
            // Enables auto scroll if the viewport is showing the latest data before the update
            bool autoScroll = (currentIndex > 0) && (0.01 + viewer.getValueAtViewPort("x",
                viewer.ViewPortLeft + viewer.ViewPortWidth) >= timeStamps[currentIndex - 1]);

            // Get new data from the queue and append them to the data arrays
            var packets = buffer.get();
            if (packets.Count <= 0)
                return;
            
            // if data arrays have insufficient space, we need to remove some old data.
            if (currentIndex + packets.Count >= sampleSize)
            {
                // For safety, we check if the queue contains too much data than the entire data arrays. If
                // this is the case, we only use the latest data to completely fill the data arrays.
                if (packets.Count > sampleSize)
                    packets = new ArraySegment<DataPacket>(packets.Array, packets.Count - sampleSize, sampleSize);

                // Remove oldest data to leave space for new data. To avoid frequent removal, we ensure at
                // least 5% empty space available after removal.
                int originalIndex = currentIndex;
                currentIndex = sampleSize * 95 / 100 - 1;
                if (currentIndex > sampleSize - packets.Count)
                    currentIndex = sampleSize - packets.Count;

                for (int i = 0; i < currentIndex; ++i)
                {
                    int srcIndex = i + originalIndex - currentIndex;
                    timeStamps[i] = timeStamps[srcIndex];
                    dataSeriesA[i] = dataSeriesA[srcIndex];
                    dataSeriesB[i] = dataSeriesB[srcIndex];
                }
            }

            // Append the data from the queue to the data arrays
            for (int n = packets.Offset; n < packets.Offset + packets.Count; ++n)
            {
                DataPacket p = packets.Array[n];
                timeStamps[currentIndex] = p.elapsedTime;
                dataSeriesA[currentIndex] = p.series0;
                dataSeriesB[currentIndex] = p.series1;
                ++currentIndex;
            }

            //
            // As we added more data, we may need to update the full range. 
            //

            double startDate = timeStamps[0];
            double endDate = timeStamps[currentIndex - 1];
            
            // Use the initialFullRange (which is 60 seconds in this demo) if this is sufficient.
            double duration = endDate - startDate;
            if (duration < initialFullRange)
                endDate = startDate + initialFullRange;

            // Update the new full data range to include the latest data
            bool axisScaleHasChanged = viewer.updateFullRangeH("x", startDate, endDate,
                Chart.KeepVisibleRange);

            if (autoScroll)
            {
                // Scroll the viewport if necessary to display the latest data
                double viewPortEndPos = viewer.getViewPortAtValue("x", timeStamps[currentIndex - 1]);
                if (viewPortEndPos > viewer.ViewPortLeft + viewer.ViewPortWidth)
                {
                    viewer.ViewPortLeft = viewPortEndPos - viewer.ViewPortWidth;
                    axisScaleHasChanged = true;
                }
            }

            // Set the zoom in limit as a ratio to the full range
            viewer.ZoomInWidthLimit = zoomInLimit / (viewer.getValueAtViewPort("x", 1) -
                viewer.getValueAtViewPort("x", 0));

            // Trigger the viewPortChanged event. Updates the chart if the axis scale has changed
            // (scrolling or zooming) or if new data are added to the existing axis scale.
            viewer.updateViewPort(axisScaleHasChanged || (duration < initialFullRange), false);
        }

        //
        // The ViewPortChanged event handler. This event occurs if the user scrolls or zooms in
        // or out the chart by dragging or clicking on the chart. It can also be triggered by
        // calling WinChartViewer.updateViewPort.
        //
        private void winChartViewer1_ViewPortChanged(object sender, WinViewPortEventArgs e)
        {
            // In addition to updating the chart, we may also need to update other controls that
            // changes based on the view port.
            updateControls(winChartViewer1);

            // Update the chart if necessary
            if (e.NeedUpdateChart)
                drawChart(winChartViewer1);
        }

        //
        // Update other controls when the view port changed
        //
        private void updateControls(WinChartViewer viewer)
        {
            // Update the scroll bar to reflect the view port position and width.           
            hScrollBar1.Enabled = viewer.ViewPortWidth < 1;
            hScrollBar1.LargeChange = (int)Math.Ceiling(viewer.ViewPortWidth * 
                (hScrollBar1.Maximum - hScrollBar1.Minimum));
            hScrollBar1.SmallChange = (int)Math.Ceiling(hScrollBar1.LargeChange * 0.1);
            hScrollBar1.Value = (int)Math.Round(viewer.ViewPortLeft *
                (hScrollBar1.Maximum - hScrollBar1.Minimum)) + hScrollBar1.Minimum;
        }

        //
        // Draw the chart.
        //
        private void drawChart(WinChartViewer viewer)
        {
            // Get the start date and end date that are visible on the chart.
            double viewPortStartDate = viewer.getValueAtViewPort("x", viewer.ViewPortLeft);
            double viewPortEndDate = viewer.getValueAtViewPort("x", viewer.ViewPortLeft + viewer.ViewPortWidth);

            // Extract the part of the data arrays that are visible.
            double[] viewPortTimeStamps = null;
            double[] viewPortDataSeriesA = null;
            double[] viewPortDataSeriesB = null;
            double[] viewPortDataSeriesC = null;
            double[] viewPortDataSeriesD = null;
            double[] viewPortDataSeriesE = null;
            double[] viewPortDataSeriesF = null;

            if (currentIndex > 0)
            {
                // Get the array indexes that corresponds to the visible start and end dates
                int startIndex = (int)Math.Floor(Chart.bSearch2(timeStamps, 0, currentIndex, viewPortStartDate));
                int endIndex = (int)Math.Ceiling(Chart.bSearch2(timeStamps, 0, currentIndex, viewPortEndDate));
                int noOfPoints = endIndex - startIndex + 1;
                
                // Extract the visible data
                viewPortTimeStamps = (double[])Chart.arraySlice(timeStamps, startIndex, noOfPoints);
                viewPortDataSeriesA = (double[])Chart.arraySlice(dataSeriesA, startIndex, noOfPoints);
                viewPortDataSeriesB = (double[])Chart.arraySlice(dataSeriesB, startIndex, noOfPoints);

                // Keep track of the latest available data at chart plotting time
                trackLineEndPos = timeStamps[currentIndex - 1];
            }

            //
            // At this stage, we have extracted the visible data. We can use those data to plot the chart.
            //

            //================================================================================
            // Configure overall chart appearance.
            //================================================================================

            // Create an XYChart object of size 640 x 350 pixels
            XYChart c = new XYChart(640, 350);
            c.setSize(this.Size.Width - 120, this.Size.Height - 80);
            //c.setSize(this.Size.Width - 550, this.Size.Height - 80);
            // Set the plotarea at (55, 50) with width 85 pixels less than chart width, and height 80 pixels
            // less than chart height. Use a vertical gradient from light blue (f0f6ff) to sky blue (a0c0ff)
            // as background. Set border to transparent and grid lines to white (ffffff).
            //            c.setPlotArea(55, 50, c.getWidth() - 85, c.getHeight() - 80, c.linearGradientColor(0, 50, 0,
            //                c.getHeight() - 30, 0xf0f6ff, 0xa0c0ff), -1, Chart.Transparent, 0xffffff, 0xffffff);
            c.setPlotArea(20, 30, c.getWidth() - 41, c.getHeight() - 50, c.linearGradientColor(0, 30, 0,
                c.getHeight() - 20, 0xf0f6ff, 0xa0c0ff), -1, Chart.Transparent, 0xffffff, 0xffffff);

            // As the data can lie outside the plotarea in a zoomed chart, we need enable clipping.
            c.setClipping();

            // Add a title to the chart using 18 pts Times New Roman Bold Italic font
            c.addTitle("   Multithreading Real-Time Chart", "Arial", 18);

            // Add a legend box at (55, 25) using horizontal layout. Use 8pts Arial Bold as font. Set the
            // background and border color to Transparent and use line style legend key.
            LegendBox b = c.addLegend(55, 25, false, "Arial Bold", 10);
            b.setBackground(Chart.Transparent);
            b.setLineStyleKey();

            // Set the x and y axis stems to transparent and the label font to 10pt Arial
            c.xAxis().setColors(Chart.Transparent);
            c.yAxis().setColors(Chart.Transparent);
            c.xAxis().setLabelStyle("Arial", 10);
            c.yAxis().setLabelStyle("Arial", 10);

            // Add axis title using 10pts Arial Bold Italic font
            c.yAxis().setTitle("Temperature (C)", "Arial Bold", 10);

            //================================================================================
            // Add data to chart
            //================================================================================

            //
            // In this example, we represent the data by lines. You may modify the code below to use other
            // representations (areas, scatter plot, etc).
            //

            // Add a line layer for the lines, using a line width of 2 pixels
            LineLayer layer = c.addLineLayer2();
            layer.setLineWidth(2);
            layer.setFastLineMode();

            // Now we add the 3 data series to a line layer, using the color red (ff0000), green (00cc00)
            // and blue (0000ff)
            layer.setXData(viewPortTimeStamps);
//            if (Channel[0] != null && chkChannel1.Checked)
                layer.addDataSet(viewPortDataSeriesA, 0xff0000, Channel[0]);
//            if (Channel[1] != null && chkChannel2.Checked)
                layer.addDataSet(viewPortDataSeriesB, 0x00cc00, Channel[1]);
//            if (Channel[2] != null && chkChannel3.Checked)
                layer.addDataSet(viewPortDataSeriesC, 0x0000ff, Channel[2]);
//            if (Channel[3] != null && chkChannel4.Checked)
                layer.addDataSet(viewPortDataSeriesD, 0xffcc00, Channel[3]);
//            if (Channel[4] != null && chkChannel5.Checked)
                layer.addDataSet(viewPortDataSeriesE, 0x00ccff, Channel[4]);
//            if (Channel[5] != null && chkChannel6.Checked)
                layer.addDataSet(viewPortDataSeriesF, 0xff00ff, Channel[5]);
            layer.addDataSet(viewPortDataSeriesA, 0xff0000, "Alpha");
            layer.addDataSet(viewPortDataSeriesB, 0x00cc00, "Beta");

            //================================================================================
            // Configure axis scale and labelling
            //================================================================================

            if (currentIndex > 0)
                c.xAxis().setDateScale(viewPortStartDate, viewPortEndDate);

            // For the automatic axis labels, set the minimum spacing to 75/30 pixels for the x/y axis.
            c.xAxis().setTickDensity(75);
            c.yAxis().setTickDensity(30);

            // We use "hh:nn:ss" as the axis label format.
            c.xAxis().setLabelFormat("{value|hh:nn:ss.fff}");

            // We make sure the tick increment must be at least 1 second.
            c.xAxis().setMinTickInc(0.05);

            // Set the auto-scale margin to 0.05, and the zero affinity to 0.6
            c.yAxis().setAutoScale(0.05, 0.05, 0.6);

            //================================================================================
            // Output the chart
            //================================================================================

            // We need to update the track line too. If the mouse is moving on the chart (eg. if 
            // the user drags the mouse on the chart to scroll it), the track line will be updated
            // in the MouseMovePlotArea event. Otherwise, we need to update the track line here.
            if (!winChartViewer1.IsInMouseMoveEvent)
                trackLineLabel(c, trackLineIsAtEnd ? c.getWidth() : viewer.PlotAreaMouseX);
 
            viewer.Chart = c;
        }

        //
        // Draw track line with data labels
        //
        private double trackLineLabel(XYChart c, int mouseX)
        {
            // Clear the current dynamic layer and get the DrawArea object to draw on it.
            DrawArea d = c.initDynamicLayer();

            // The plot area object
            PlotArea plotArea = c.getPlotArea();

            // Get the data x-value that is nearest to the mouse, and find its pixel coordinate.
            double xValue = c.getNearestXValue(mouseX);
            int xCoor = c.getXCoor(xValue);
            if (xCoor < plotArea.getLeftX())
                return xValue;

            // Draw a vertical track line at the x-position
            d.vline(plotArea.getTopY(), plotArea.getBottomY(), xCoor, 0x888888);

            // Draw a label on the x-axis to show the track line position.
            string xlabel = "<*font,bgColor=000000*> " + c.xAxis().getFormattedLabel(xValue, "hh:nn:ss.fff") +
                " <*/font*>";
            TTFText t = d.text(xlabel, "Arial Bold", 10);

            // Restrict the x-pixel position of the label to make sure it stays inside the chart image.
            int xLabelPos = Math.Max(0, Math.Min(xCoor - t.getWidth() / 2, c.getWidth() - t.getWidth()));
            t.draw(xLabelPos, plotArea.getBottomY() + 6, 0xffffff);

            // Iterate through all layers to draw the data labels
            for (int i = 0; i < c.getLayerCount(); ++i)
            {
                Layer layer = c.getLayerByZ(i);

                // The data array index of the x-value
                int xIndex = layer.getXIndexOf(xValue);

                // Iterate through all the data sets in the layer
                for (int j = 0; j < layer.getDataSetCount(); ++j)
                {
                    ChartDirector.DataSet dataSet = layer.getDataSetByZ(j);

                    // Get the color and position of the data label
                    int color = dataSet.getDataColor();
                    int yCoor = c.getYCoor(dataSet.getPosition(xIndex), dataSet.getUseYAxis());

                    // Draw a track dot with a label next to it for visible data points in the plot area
                    if ((yCoor >= plotArea.getTopY()) && (yCoor <= plotArea.getBottomY()) && (color !=
                        Chart.Transparent) && (!string.IsNullOrEmpty(dataSet.getDataName())))
                    {
                        d.circle(xCoor, yCoor, 4, 4, color, color);

                        string label = "<*font,bgColor=" + color.ToString("x") + "*> " + c.formatValue(
                            dataSet.getValue(xIndex), "{value|P4}") + " <*/font*>";
                        t = d.text(label, "Arial Bold", 10);

                        // Draw the label on the right side of the dot if the mouse is on the left side the
                        // chart, and vice versa. This ensures the label will not go outside the chart image.
                        if (xCoor <= (plotArea.getLeftX() + plotArea.getRightX()) / 2)
                            t.draw(xCoor + 5, yCoor, 0xffffff, Chart.Left);
                        else
                            t.draw(xCoor - 5, yCoor, 0xffffff, Chart.Right);
                    }
                }
            }
            return xValue;
        }

        //
        // The scroll bar event handler
        //
        private void hScrollBar1_ValueChanged(object sender, EventArgs e)
        {
            // When the view port is changed (user drags on the chart to scroll), the scroll bar will get
            // updated. When the scroll bar changes (eg. user drags on the scroll bar), the view port will
            // get updated. This creates an infinite loop. To avoid this, the scroll bar can update the 
            // view port only if the view port is not updating the scroll bar.
            if (!winChartViewer1.IsInViewPortChangedEvent)
            {
                winChartViewer1.ViewPortLeft = ((double)(hScrollBar1.Value - hScrollBar1.Minimum))
                    / (hScrollBar1.Maximum - hScrollBar1.Minimum);

                // Trigger a view port changed event to update the chart
                winChartViewer1.updateViewPort(true, false);
            }
        }

        //
        // Draw track cursor when mouse is moving over plotarea
        //
        private void winChartViewer1_MouseMovePlotArea(object sender, MouseEventArgs e)
        {
            WinChartViewer viewer = (WinChartViewer)sender;
            double trackLinePos = trackLineLabel((XYChart)viewer.Chart, viewer.PlotAreaMouseX);
            trackLineIsAtEnd = (currentIndex <= 0) || (trackLinePos == trackLineEndPos);
            viewer.updateDisplay();
        }

        //
        // Pointer (Drag to Scroll) button event handler
        //
        private void pointerPB_CheckedChanged(object sender, EventArgs e)
        {
            if (((RadioButton)sender).Checked)
                winChartViewer1.MouseUsage = WinChartMouseUsage.ScrollOnDrag;
        }

        //
        // Zoom In button event handler
        //
        private void zoomInPB_CheckedChanged(object sender, EventArgs e)
        {
            if (((RadioButton)sender).Checked)
                winChartViewer1.MouseUsage = WinChartMouseUsage.ZoomIn;
        }

        //
        // Zoom Out button event handler
        //
        private void zoomOutPB_CheckedChanged(object sender, EventArgs e)
        {
            if (((RadioButton)sender).Checked)
                winChartViewer1.MouseUsage = WinChartMouseUsage.ZoomOut;
        }

        //
        // Save button event handler
        //
        private void savePB_Click(object sender, EventArgs e)
        {
            // The standard Save File dialog
            SaveFileDialog fileDlg = new SaveFileDialog();
            fileDlg.Filter = "PNG (*.png)|*.png|JPG (*.jpg)|*.jpg|GIF (*.gif)|*.gif|BMP (*.bmp)|*.bmp|" +
                "SVG (*.svg)|*.svg|PDF (*.pdf)|*.pdf";
            fileDlg.FileName = "chartdirector_demo";
            if (fileDlg.ShowDialog() != DialogResult.OK)
                return;

            // Save the chart
            if (null != winChartViewer1.Chart)
                winChartViewer1.Chart.makeChart(fileDlg.FileName);
        }

		private void btnStart_Click(object sender, EventArgs e)
		{
            long interval;

            // Read interval from form
            if (!long.TryParse(TextBoxInterval.Text, out interval))
            {
                return;
            }

            // Set timer interval
            //_microTimer.Interval = interval;

            // Ignore event if late by half the interval
            //_microTimer.IgnoreEventIfLateBy = interval / 2;

            // Start timer
            //_microTimer.Start();

            System.Net.IPAddress ip = System.Net.IPAddress.Parse("127.0.0.1");// txtHost.Text);
            if (!(server.IsStarted))
                server.Start(ip, Convert.ToInt32(txtPort.Text));
            if (!start_npause)
            {
                chartUpdateTimer.Start();
                btnStart.Text = "      Pause";
                btnStart.Image = ((System.Drawing.Image)(resources.GetObject("btnPause.Image")));
            }
            if (start_npause)
            {
                chartUpdateTimer.Stop();
                btnStart.Text = "      Start";
                btnStart.Image = ((System.Drawing.Image)(resources.GetObject("btnStart.Image")));
            }
//            start_npause = !start_npause;
        }

		private void btnStop_Click(object sender, EventArgs e)
		{
            // Stop the timer
            //_microTimer.Stop();
        }
    }
}

  Re: High resolution (<1ms) chart data acquisition - VS2019 C#
Posted by William on Feb-11-2021 10:14
Attachments:
Thank you Peter for the detailed answer.

Peter Kwan wrote:
After I remove the MicroTimer. It is not necessary as the data time comes from your data, not from the timer.
(...)
You can zoom in the chart and verify that each data point is 0.01 second apart.

If your data points are 1ms apart, you can change the increment from 0.01 to 0.001.

For your "TCP Message Generator", I am not sure how it works. It should send the data to the server may be every 50ms. If it is collecting data every 1ms, then each packet should contain 50 data points.

Sorry but just to be clear about these TCP messages, they are irregularly sent and you can have an interval between 2 messages of 100us as well as you can have even minutes or hour. This all depends on the activity of the sensors. If there is no variation of their values, there is no point to repeat the last value over and over again.

Other info regarding the structure of my TCP message : I don't have [channel1_value,channel2_value,channel3_value,...]
I have [Channel_name,measurement_unit,channel_value].
This means that I have to parse every single TCP message to extract the channel name, the unit in which the value is measured and the value from which the signal comes.

In order to get better what I try to do, I send you a picture that explains better.

Peter Kwan wrote:

I obtain the data from your TCP code by adding 4 lines to your code, and it works normally.

Attached please find the modified code.

Thank you for the time you spent trying to fix the issue and writing the new code but I have tried repeatedly the file you sent me and so far nothing. When I run the program in debug mode, I have no chart zone at all. I can't tell why. Actually, I discovered that if RandomWalk.cs module was deactivated, the graph was frozen.
One thing I tried to tackle this issue was to let RandomWalk.cs active but code OnData as following :
private void onData(double elapsedTime, double series0, double series1)
{
  DataPacket p = new DataPacket();
  p.elapsedTime = elapsedTime;
//  p.series0 = series0;
//  p.series1 = series1;
  p.series0 = Level2[0];
  p.series1 = Level2[1];

  buffer.put(p);
}

This way, the values I receive are displayed instead of the randomwalk values. Yes, this should be fine, nevertheless, the resolution is still 15~20ms. The best results I have so far are with RealTimeMeasure project. I still have some little stuff to correct on it. I am sad to see that you got very good results with your file and I didn't. I would really love to increase the resolution of the datalogger. I will go on for a little while scratching my head on this and then, if go nowhere, I don't know what else to do. That is too bad because you solution is very extensive and it would be a shame for me to start all over again. Do you think the best solution for me is still the Multithread or the Measure is good?

Best regards,

William
ChartDirector message handling.png

  Re: High resolution (<1ms) chart data acquisition - VS2019 C#
Posted by Peter Kwan on Feb-11-2021 23:13
Hi William,

The code I provided are based on the code provided by you in your previous message:

https://www.chartdir.com/forum/download_thread.php?site=chartdir&bn=chartdir_support&thread=1611139382&new_thread=1#N1612497353

In your original code, the Level1 and Level2 are never used. The "TCP Message Generator" generate messages at regular intervals like an oscilloscope, and the message is using a different format with an embedded timestamp. This is different from what you mentioned in your last post.

If your message are arriving at random intervals, and does not include the timestamp, you can only derive the timestamp on the server. The most accurate way is to obtain the timestamp as fast as possible when the message reaches the server. From my understanding, you use SimpleTCP to handle the TCP connection, and that it will notify you of the message in Server_DataReceived. That is where you should generate the timestamp:

Stopwatch timer = new Stopwatch();
private void Server_DataReceived(object sender, SimpleTCP.Message e)
{
  if (!timer.IsRunning)
  timer.Start();

  double elapsedTime = timer.ElapsedMilliseconds/1000.0;

  ..... now we have the timestamp, your code can read the data .....
          ..... code to read data from message .....

onData(elapsedTime, myDataValue1, myDataValue2);
}

In the above method, the timing accuracy depends on SimpleTCP. From my understanding, the SimpleTCP events are not multi-threaded. It just runs in the GUI thread, so it will compete with the GUI for CPU resources. If the GUI is running (such as when something in the GUI is being updated, or when you resize or move the window, etc), it may affect the SimpleTCP.

From experience, the timing can be more accurate if you use multi-threading. That means you may need to use another method to obtain the TCP message (such as to write your own networking code and run it in another thread).

Best regards,
William