|
 Regression/Unit Testing |
| Posted by TN on Aug-20-2022 01:24 |
|
| Our use case is to generate SVG charts to embed in reports with batch jobs. Is the svg string that gets generated independent/consistent regardless the computer it gets run on (for example, different OS/ display resolution)? How about the other binary chart formats? |
| Re: Regression/Unit Testing |
| Posted by Peter Kwan on Aug-20-2022 05:11 |
|
Hi TN,
Internally, we also use automatic SVG and image comparison for automatic testing and validation. However, there are several complications:
(a) The fonts in different OS or OS versions are generally inconsistent. (You may aware the text in Windows, macOS and Linux can look different.) This may cause the SVG to have slightly different text positioning (usually at the order of 0.01 pixels - since SVG can scale indefinitely, ChartDirector needs to use high precision to ensure the SVG looks correct when scale up).
(b) SVG requires the "id" attribute of the tags to be unique within the SVG. Considering people can combine multiple SVGs into one SVG (eg. to create something like a "multi-chart"), an ChartDirector instance will never reuse the same "id", even across different SVGs. That means the text representation of every SVGs are different. To compare the SVG, it is necessary to parse the SVG to check if they contain the same structure except the "id".
(c) Due to the non-deterministic nature of floating point computations, around 3% of the charts may have two possible images, very slightly different and both correct.
(d) Some SVG can contain bitmap shapes (such as custom icons provided by the user). The icons will be compressed in the SVG. It is possible different OS produces different compression result. So a byte by byte comparison will not work. The icon needs to be decompressed and then a pixel by pixel comparison performed.
The (a) and (c) above also applies to PNG/JPEG/GIF/BMP outputs.
Best Regards
Peter Kwan |
| Re: Regression/Unit Testing |
| Posted by TN on Aug-20-2022 05:27 |
|
Hi Peter,
If we stick with Windows 10 do you expect differences in SVG generation on different computers given the same chart code due to machine precision? Is the SVG string generated from a rasterized representation of the chart or from internal chart data structures? Has to be the latter...
Thanks! |
| Re: Regression/Unit Testing |
| Posted by TN on Aug-20-2022 05:29 |
|
| As for the ID issue, if a single chart is generated per executable instance, would that solve that problem? |
| Re: Regression/Unit Testing |
| Posted by Peter Kwan on Aug-20-2022 19:09 |
|
Hi TN,
Yes, this would solve the "id" problem.
Regards
Peter Kwan |
| Re: Regression/Unit Testing |
| Posted by Peter Kwan on Aug-20-2022 19:08 |
|
Hi TN,
The SVG is generated from internal data structures.
In our own testing, we found that on Windows 10, the SVG generated on different machines are the same (except for the "id" attribute).
Note that if you update the ChartDirector DLL, the SVG may change.
Regards
Peter Kwan |
| Re: Regression/Unit Testing |
| Posted by Steve on Nov-02-2022 02:28 |
|
Hi Peter,
TN suggested rendering one chart per executable instance. If I sequentially generate multiple charts per executable, is there any way to make the "id" attribute values independent of the number of charts generated? I'm envisioning somehow resetting a seed value for the ID sequence after each chart is generated.
Thanks,
Steve |
| Re: Regression/Unit Testing |
| Posted by Peter Kwan on Nov-02-2022 13:31 |
|
Hi Steve,
We do a lot of regression and unit testing too. After creating the SVG, out "matching code" simply renames the "id" attributes to some deterministic values. The steps are:
(a) Get all the id attributes by searching for the pattern " id='XXX'" (without the double quotes - the XXX can contain any number of characters). Create a table that maps the XXX to a deterministic value.
(b) Apply the map to the following patterns, which are the id attribute itself, and its references.
id='XXX'
'#XXX'
(#XXX)
If your scripting language of your regression test system supports regular expressions, the above can probably be done in a few lines of code.
Best Regards
Peter Kwan |
|
Forum Home
Search
Message List
Post Message