ScpcPGN - analysis of PGN files annotated by Scid vs. PC

external link:


Stockfish
development versions

the latest versions of Stockfish

internal links:


evaluation relevance reduction

move and position evaluations together with NAG and Informator symbols

keyboard layout for chess annotation

article links:


article start

what can ScpcPGN do?

contents of the programme package

Automatic game annotation by remote control of Scid vs. PC

Processing and analysis of engine variations and annotations

Insertion of game move comments into the PGN file

Insertion of variation move comments into the PGN file

ASCII/Unicode characters in PGN files

position evaluation symbols

move evaluation symbols

move-square colours

excursus: move-square colours and coloured move styles in Aquarium especially for users of the Aquarium GUI

removal of irrelevant game move variations

reduction of variation moves

replacement of NAG values and annotation modules

printing Scid vs. PC PGN files via LibreOffice Writer

replacement of NAG values

replacement of engine related modules

replacement of mate announcements

replacement of chess piece text modules

clipboard mode

HTML mode

the mate announcement debacle

evaluation relevance reduction

programme selection for PGN files

parameters and checkboxes

GUI background colour

ScpcPGN programming


ScpcPGN - download:


zip-Icon

program ScpcPGN - development version 11th November 2022
ZIP file [2.45 MB]: ScpcPGN.zip
in the ScpcPGN GUI click on the button 'English'



Preliminary note:
This article was written in the original in German. The English translation comes from the author too. A sufficient quality of the translation cannot be guaranteed.

Update from 11th November 2022:
Due to a change in the program code of Stockfish, the calculation of the win, draw and loss percentages by ScpcPGN has been adapted to the percentages recently calculated by this engine.


Update from 18th October 2022:
The automatic game analysis for Scid vs. PC can now be
accelerated by changing the parameter file (the parameter files were supplemented accordingly and must be changed) and
cancelled by pressing the Esc key.


Update from 31st August 2022:
The automatic game analysis for Scid vs. PC has been extended by null move variations for the representation of opponent threats (the parameter files were supplemented accordingly and must be changed);
optimized variation evaluation comparisons, which should now also cope with normal and null move variations in chaotic order;
the formula for the optimum rate has been improved.


Update from 11th June 2022:
minor programme modifications;
the following explanations have been partially adapted to version 4.23 of Scid vs. PC.


Update from 2nd April 2022:
All previous information categories are supplemented using the engine-WDL-statistics (win/draw/loss estimate ratios).



ScpcPGN - a Windows program that the world doesn't need, at most some adventurous users of the Chess GUI Scid vs. PC



What can ScpcPGN do?

Intelligent processing of PGN files annotated with Scid vs. PC along with generation of browser and spreadsheet files with game analysis. It appropriates thereby the knowledges described in the article 'Evaluation Relevance Reduction'.

1) The program package contains:



2) Automatic game annotation by remote control of Scid vs. PC:

ScpcPGN relies for its analysis function on games annotated by Scid vs. PC and saved in PGN files. However, Scid vs. PC has only limited automatic game annotation, as it does not offer a multi-variation mode ('MultiPV'), which inserts several variations per game move into the notation. Reference is made to this fruitless discussion with the programme author Steven Atkinson. While ScpcPGN is able to process only one annotated variation per game move (single-variation mode), but the analysis function is considerably limited as a result.

However, there is still the option of manual move annotation by clicking on the button '++V' with the tooltip 'Add All Variations' in the engine analysis window. To the right, the number of variations can be set. The only problem is the work involved in the perpetual clicks, not to mention the adherence to a fixed evaluation time for all the moves to be annotated.

Now ScpcPGN comes into play. The ScpcPGN GUI section 'Scpc game analysis' can be used to automate the addition of engine variations and annotations in Scid vs. PC by remotely controlling Scid vs. PC from ScpcPGN. If you should dare to do this, you should, in your own interest, carefully read and observe the notes on preparation that appear after starting this function. After all the author excludes any liability for any damage due to malfunctions of ScpcPGN and in particular due to malfunctions of this special program feature (see the attached file 'COPYING'). Mystically speaking: After starting this program function, the user's computer is in the power of fate!

A goodness proposal for jumpy souls:
In Scid vs. PC, position the PGN window at the top to keep an eye on the progress of the variation entry,
modestly start the game analysis with only a few half-moves,
in case of a malfunction, press the Esc key to stop the variation entries before the next half-move and
then check what all has crashed …

On the one hand, the 'analysis time/move in sec. nor. variations' can be set by which is meant the usual variations as opposed to null move variations (more on this later). Analysis times of less than one second are theoretically possible, but are slowed down in practice by the graphic programme flow.

When choosing the 'half moves forward (+x) / backward (-x)' to be commented on, it should be noted that the half-moves available in the relevant game direction do not exceed the value entered.

'sides':
'1', if variation(s) should be formed only for the side that was granted variation(s) at the beginning, the other side is skipped;
'2' if variation(s) should be formed for both sides.

You will not find a choice of single- or multi-variation mode here. As described above, this must be done in the engine analysis window.

Now we come to the most delicate point. The '++V' button in the engine analysis window must be found and activated by ScpcPGN by means of an identification number. This number must be entered in the ScpcPGN GUI under 'Instance'. The problem is, however, that it changes depending on the window arrangement in Scid vs. PC.

The following window arrangement is therefore recommended:
selection of the menu Options – Windows – Dock windows – not absolutely necessary, but then the exact wording of the title of the engine window must be entered in the field after "engine:" so that ScpcPGN can find this window for its mouse clicks; board top left; board top left;
'Game info' window on the left below the board;
'PGN Window' top right;
engine analysis window on the right below the 'PGN Window'.

With this window arrangement, experience shows that the 'Instance' is 9 or 10 - with emphasis on 10 for docked windows. This setup also has 2 advantages. It only contains what is necessary for the annotation and one can directly follow the progress of the variation insertion in the 'PGN Window'.

Since it is ultimately up to the user to find out the correct setting on his system, here is a tip: To determine the appropriate 'instance' value, one can use the tool 'Au3Info.exe', which is included in the downloadable program package of AutoIt. It can also be downloaded individually from this website. After the start the 'Finder Tool' must be dragged on the button '++V' with the tooltip 'Add All Variations' in the engine analysis window. The following can then be read under 'Basic Control Info':

Class: Button
Instance: x
where the 'x' stands for the instance you are looking for.

During the analysis by the chess engine you can switch to another window or program. If then Scid vs. PC has to insert an engine evaluation, ScpcPGN will hopefully automatically switch to Scid vs. PC in the background and back to the previously used window or program after the work is done.

And in the case of pressing the Esc key, the variation entry is terminated before the next half-move.

Since the update of August 2022, even null move variations can optionally be inserted automatically. For this purpose
in the menu Options – Windows – Dock windows is to be voted out;
under 'Instance' the correct value as described above is to be entered - now probably '9' due to the undocked windows;
in the GUI the checkbox in front of "Zero moves:" is to be selected;
subsequently the analysis time in seconds especially for these zero move variations is to be entered and
the exact wording of the title of the engine window is to be entered in the field after 'engine:' so that ScpcPGN can find this window for its mouse clicks.

Null move variations matter in the analysis in order to become aware of the opponent's threats in the case of inaction and to derive position optimisations from them. They are processed by ScpcPGN in the course of move annotation for analysis PGN files, but not in the context of the creation of move evaluation symbols for game and regular variation moves, HTML files and clipboard data.

Normally, when selecting null move variations, ScpcPGN adds a single null move variation after one or more regular variations have been entered. Scid vs. PC does not allow several null move variations in this way. However, it is even possible to pave a game fully with null move variations. To do this, only either "0" or nothing at all must be entered in the field next to 'analysis time/move in sec. nor. variations'. It should even be possible to add only null move variations to an already with regular variations equipped game.

Before exporting games with null move variations, 'Export Scid Flags' must be affirmed in order to preserve the symbols for the null moves ("--").

Apropos 'export': In principle, when saving games, the export via the menu Tools - Export Current Game - Export Game to PGN should be preferred to the simple saving via the menu File - Save PGN.

On the one hand, formatting options are available during export, whereby the following selections have proven useful:
Space after Move Numbers: Yes
Export comments: Yes
Strip square/arrow mark codes from comments: No
Export Scid Flags: Yes
Indent comments: Yes
Export variations: Yes
Indent variations: Yes
Insert newlines every 80 chars: Yes
Column style (one move per line): No
Symbolic annotation style: $2 $14
Convert null moves to comments: No
Character encoding: Utf-8.

Furthermore, it is possible to export several games into a PGN file. To do this, the games or the file with several games must be moved to the clipbase (accessed via the menu Windows - Game List (bottom left)) and then exported via the menu Tools - Export All Filter Games - Export Filter to PGN.

While the engine analysis time can even be reduced to less than a second, the graphical flow of the automatic game analysis is running quite slowly. This is due to the default settings of the programming language AutoIt, which was used to create Scid vs PC. However, there are two hidden adjusting screws that can speed up the programme considerably. By default, the last two lines of the parameter files each contain the values 250 as given by AutoIt. If these values are reduced up to 0, a graphic acceleration takes place. This is explicitly not recommended because it is not possible to predict what this change will do to the user's computer. Turning these adjusting screws is therefore at your own risk!

3) Processing and analysis of engine variations and annotations:

ScpcPGN uses engine variations and comments, which are inserted by Scid vs. PC into the game notation with a manual or automatic click on the '++V' button in the engine analysis window while the engine analysis is running.

These engine comments are evaluated by ScpcPGN and the analysis results optionally
A PGN file prepared in this way by Scid vs. PC is loaded by clicking on the button "start of PGN file".

4) Insertion of game move comments into the PGN file:

ScpcPGN inserts comments after those game moves for which Scid vs. PC provides it with data. To this, it replaces the text modules utilized by the user in the five-digit format #.....# in the ScpcPGN GUI in the text input field under the heading 'insertion of game move comments', if this option is selected. But the text definition in the supplied 'Scpc.par' file using any editor in line 18 is much more convenient.

The following text modules are available – in case of duplicity, the first module starting with ‘#W-’ represents a WDL ERR value and the second a user ERR value (cf. chapter 18)):

only in game move comments:



in game and variation move comments:


5) Insertion of variation move comments into the PGN file:

ScpcPGN can also modify and extend the variation move comments coming from Scid vs PC. To this, it replaces the text modules utilized by the user in the format #.....# in the ScpcPGN GUI in the text input field under the heading 'modification of engine comments at variation start', if this option is selected. But the text definition in the supplied '*.par' files using any editor in line 21 is much more convenient. Of course, ScpcPGN adopts again the user-selectable explanatory texts between the modules.

Not all of the text modules listed in chapter 4) above make sense here. Only the text modules listed there under the heading "in game and variation move comments" can be used for variations.

Furthermore, the following can only be used for variation move comments:

#var-n#
number of the variation in the ranking of all variations of the respective game move given by Scid vs. PC (disregarding null move variations); recommended if the checkbox 'removal of irrelevant game move variations' is selected in the ScpcPGN GUI; null move variations are marked with 'N'; if 'insertion of game move comments' has been selected, game move variations are not commented in order to avoid double annotations.

6) ASCII/Unicode characters in PGN files:

Scid vs. PC and Aquarium know how to handle Unicode characters in PGN files. However nitial tests with Arena, Kvetka and Shredder Classic 4 were negative. If one of these programmes should be selected as output programme in the ScpcPGN GUI, ScpcPGN will therefore try to convert some used Unicode characters into ASCII characters. At present, this still has plenty of experimental character.

The AqChessUnicode font has chess Unicode characters, especially the chess figurines. If AqChessUnicode is not available, the following download link may be helpful: AqChessUnicode font. This font is also included in the chess GUI Aquarium.

For the explanation of suitable text modules in the '*.par' files '∑' (sum) and 'ø' (average) characters could be used. They can be found under hexadecimal 2211 or decimal 8721 and hexadecimal F8 or decimal 248.

In the text outside the actual text modules, you should use the fixed, non-breaking space character (hexadecimal A0 or decimal 160) in suitable places. Otherwise the chess programs could mercilessly break such text in the annotation on the right side.

If there is a selection of Aquarium according to chapter 19), the normal hyphen should be avoided under certain circumstances and replaced by the protected or non-breaking hyphen (hexadecimal 2011 or decimal 8209). However, if you intend to load the PGN files into Arena, Kvetka or Shredder Classic 4, you should refrain from the non-breaking hyphen due to lack of Unicode character recognition in these programs (as well as from the fixed space character, by the way); you could then use the underscore '_'.

7) Position evaluation symbols:

These are inserted by ScpcPGN automatically in the course of the annotation optionally with the following text modules:


In ASCII mode they are output as follows:


In 1-character-mode, instead of +/=, -/=, +/- and -/+, the following special characters common in chess annotations are output as single character in each case:
⩲, ⩱, ±, ∓.

Only '±' is an ASCII character, the other 3 are Unicode characters. Scid vs. PC and Aquarium will display these 3 special characters correctly if a suitable Unicode font such as AqChessUnicode or Cambria is used. The clipboard output (see chapter 15) also automatically falls back on these special characters, so the installation of such a font is necessary for their correct display.

Information about the position evaluation symbols in the context of user ERR is provided on the left side in the ScpcPGN GUI after clicking the 'info' button under the headline 'position evaluation symbols ...' and 'scheme:'. The user ERR position evaluation symbols inserted by ScpcPGN depend on the adjusted parameters of the evaluation relevance reduction and the position evaluation scheme selected in the ScpcPGN GUI.

In the window after clicking on the button 'complete analysis for one or two special engine evaluations' of course the result of the parameter evaluations for the entered values with regard to the position evaluation symbols can be shown.

8) Move evaluation symbols:

These are inserted by ScpcPGN in the notation behind game and variation moves, provided that the checkbox 'move evaluation symbols !! / ! / !? / ?! / ? / ??' on the right side of the GUI is selected. They are determined
in the case of optimal game and variation moves by evaluation comparison with the second-best variation move (if available): !! / ! / !?;
in the case of suboptimal game and variation moves by evaluation comparison with the best variation move: ?! / ? / ??.
Without multi-variation mode due to lack of comparison possibilities no positive move evaluation symbols!

Their meaning is:


Limited information about the move evaluation symbols in the context of user ERR for an initial evaluation of 0.00 is provided after clicking the 'information' button under the headline 'move evaluation symbols !! / ! / !? / ?! / ? / ??:' on the left side in the ScpcPGN GUI. The move user ERR evaluation symbols inserted by ScpcPGN depend on the adjusted parameters of the evaluation relevance reduction and the move evaluation scheme selected in the ScpcPGN GUI.

After clicking the button '!! / ! / !? / ?! / ? / ?? - matrix for 2 engine evaluations from +10 to -10' the relevant evaluation differences and move evaluation symbols each according to WDL ERR / user ERR for a variety of engine evaluation pairs are displayed. If you are looking for results for White, select an evaluation from the column on the left margin and a comparative evaluation from the row on the top margin. The intersection field contains the result. The opposite procedure is followed for Black. The diagonal cells from top left to bottom right naturally contain relevant evaluation differences of 0.00, since the two evaluations are identical. Above this diagonal, the values are positive, below negative. The colors used should be self-explanatory.

And if you want information about special evaluations, for example because you have variation evaluations during a manual annotation, but you are not sure which move or position evaluation symbol you should assign, you can click on the button 'complete analysis for one or two special engine evaluations' to see the result of the parameter analysis.

Example for the case of an 'evaluation at 0.75-game-result-probability' of 1.50, an 'evaluation at 0.75-plus-game-result-probability' of 3.00, a '0.75-plus-game-result-probability' of 0.875 each in the ScpcPGN GUI and – after clicking on the button 'complete analysis for one or two special engine evaluations' – entering 1 as a better evaluation and then 0 for the worse:
In both schemes, the move evaluation symbol for the better evaluation is "!?", for the worse "?!".
The move evaluation symbol thresholds for the better evaluation reveal that neither of the two schemes allows a '!!' for a better alternative move in relation to the move with the evaluation 1.00 and that a '!' can only be granted if, depending on the scheme, the alternative move evaluation is at least 3.45 or 4.42. For all other move evaluation symbols, especially the negative ones, however, there are thresholds so that they can be distributed if the alternative move is only good or bad enough to reach or exceed the respective threshold.

Trial by example: Test the evaluation pair 3.44 and 1 with the button 'complete analysis for one or two special engine evaluations' and you will find the move evaluation symbols '!?' and '?!'. So far correct, because the value 3.45 was undercut. New attempt: this time 3.45 and 1 - voila: '!' and '?' in the scheme with 7 sectors. And for the evaluation pair 4.42 and 1 both schemes agree again.

This does not work with the WDL-BRR, because with each new evaluation, new integral equations are also formed, which prevent comparability with previous results.

Move evaluation symbols without further annotations allow a quick overview of the move quality in the course of a game. To do this, select only the ‘move evaluation symbols’ in the right-hand area of the ScpcPGN GUI, possibly also the ‘move-square colours’ for Scid vs. PC and Aquarium and – if you attach importance to the evaluation diagram – the ‘insertion of game move comments’ with a single text module: #e-c-e#. The evaluation inserted for this by ScpcPGN can remain invisible in Scid vs. PC, but (probably?) not in Aquarium. In contrast to Scid vs. PC, Aquarium compensates for this with a sea of colours.

In both programmes, the variation move comments not touched by ScpcPGN in this constellation can be clicked away, so that only the game moves with the move evaluation symbols, possibly with the colours and possibly with the evaluations in Aquarium remain.

If you want to switch comfortably between different selection constellations or text modules for the comments, the choice and storage of ‘parameters’ with the "load" and "save" buttons on the left in the GUI is recommended.

9) Move-square colours:

ScpcPGN colours the chessboard square on which a piece has settled during a move, provided that the checkbox 'move-square colours for !! / ! / !? / ?! / ? / ??' on the right of the GUI has been selected. The colour selection is made according to the user specifications in the 6 colour code lines under the mentioned checkbox. To determine the colour codes, the HTML Color Picker is recommended.

In Scid vs. PC nothing has to be changed for the colouring of the move-squares. Of course, it is only visible when a game or variation move is selected and involves an evaluation win or loss that corresponds to one of the 6 move evaluation symbols. The move colour codes are invisible in the annotation of Scid vs. PC if 'PGN – Options – Hide Square/Arrow Codes' is checked, but not in the comment editor.

If one should get the idea to take over the move-square colours used by ScpcPGN for markings in the comment editor of Scid vs. PC or to simply use other colours there, one would first be fighting a losing battle. As far as can be seen, Scid vs. PC does not allow changing the colour palette in the comment editor, especially not via the menu Options – Colors. However, if you dare to manipulate Scid vs. PC quasi in the machine room, you might be helped:

Load the file .../bin/scid.gui in the program directory of Scid vs. PC into an editor and search for the line
'variable  colorList {}  markTypeList {}'.
The line 125499 that follows reads, for example:
'set colorList {cornsilk2 wheat2 burlyWood3 grey50 gray10 rosyBrown mediumPurple1 royalBlue3 skyBlue cadetBlue2 paleGreen2 yellow3 darkOrange1 indianRed3 }'.
The colour codes in this line need to be changed, for example as follows:
'set colorList {#00FF84 #80FFC1 #B3FFDA #FFB3FF #FF80FF #FF4DFF blue #00CC69 #CC00CC white black yellow3 darkOrange1 indianRed3 }'.

The colour codes preceded by a "#" and immediately followed by a 6-digit HEX number can be changed as desired. After finishing Scid vs. PC, the scid.gui file must be saved. After restarting Scid vs. PC, the new colour palette should be available.

10) Excursus: move-square colours and coloured move styles in Aquarium especially for users of the Aquarium GUI:

Aquarium is cheap and sophisticated, hence perhaps the most interesting alternative to Scid vs. PC as a programme for displaying the PGN files created by ScpcPGN. Therefore this digression.

Move-square colours from ScpcPGN are also displayed in Aquarium, but limited to the two colours red and green, as this programme only offers only 4 colours for square marks at all.

On the other hand, Aquarium shines in the colouring of moves in the notation. To this it uses all the colours that can be set in the ScpcPGN GUI and are used for the colouring of the move-squares in Scid vs. PC.

This feature, which is unfortunately not available in Scid vs. PC, takes advantage of the Aquarium comment styles that are normally used during manual annotation using the Aquarium 'Comments' tab and click on the left 'Annotate' button and following click on 'Styles'. The AquaPGN GUI holds a checkbox 'move styles' in the upper right corner, which can be used to assign special styles, especially colors, to good and bad moves in the generated PGN files. The 6 move qualities discussed in the chapter 'move evaluation symbols' are affected, whereby such moves do not necessarily have to be provided with a move evaluation symbol in the annotation in order to be able to be colour formatted.

These styles are defined in Aquarium by selecting the tab 'Comments' and click on the arrow symbol under the left button 'Annotate' or simply by using the key combination Shift-S and following click on 'Edit styles'. You will then find yourself in the GUI 'Edit special styles...'. The list there under 'Style name' already contains style categories called
'Good move' (in ScpcPGN synonymous with 'impressive move' - '!'),
'Interesting move' (in ScpcPGN synonymous with 'attractive move' - '!?'),
'Dubious move' (in ScpcPGN synonymous with 'questionable move' - '?!'),
'Bad move' (in ScpcPGN synonymous with 'weak move' - '?').

AquaPGN uses these as well as the two following style categories:
'Special Style 22' for brilliant moves ('!!'),
'Special style 23' for miserable moves ("??").

The detailed definition of these 6 styles in Aquarium is in principle up to the user. Nevertheless some tips should be mentioned here:

Mouse click on the button 'Edit styles...'.
Selection of a style to be edited in the 'Style name' list, whereby only the 6 styles between 'Good move' and 'Special style 23' are relevant here.
In the 'Substyle' list, 'Move' must be selected.
In the upper right corner the 'Priority' is shown which should not be changed because AquaPGN uses it.
With selection of the 4 check boxes 'Override global' you can select font, font size, text color and background color, furthermore you can manipulate the font style. At least one of these checkboxes should be selected or at least the font style be changed, otherwise the selected style would not differ from the general style.

The selected styles should contrast clearly the general styles for avoiding confusion. These are set in the Aquarium GUI after clicking on the fish symbol in the upper left corner under 'Options' - 'Display Options' - 'Notation styles...'. The cryptic categories there have the following meaning:
'Main line': game moves;
'Main branch': variation moves;
'Simple branch': subvariation moves;
'Text comment': user annotations;
'Complex branch': first move of a variation - since ScpcPGN marks these moves with move styles, if they are to be praised as good or denounced or bad moves, a clever distinctive style choice should be made here.

Again: all mentioned 6 + 5 = 11 styles should be able to be differentiated (color-wise) reasonably.

11) Removal of irrelevant game move variations:

A game move variation is a variation whose 1st move is identical to the game move. It is irrelevant if the subsequent game move contains an engine-commented variation (which automatically comments on the previous move) and the depth of the engine evaluation of the game move variation (game move depth) is less than or equal to the depth of the engine evaluation in the variation of the subsequent move (subsequent move depth). This is because in this case the subsequent move depth in the variation evaluation extends further than the game move depth, making the game move variation superfluous.

By selecting the checkbox 'removal of irrelevant game move variations', ScpcPGN will remove them completely from the annotation, which serves for clarity. They still exist program-internally and can even play a role in the annotation, for example in the text module #va-no#.

12) Reduction of variation moves:

The variations delivered by Scid vs. PC are sometimes incredibly long. ScpcPGN can therefore reduce the variations to a maximum number of variation moves by selecting the checkbox "reduction of variation moves to maximum" and entering the number of variation moves in the text field. This goes hand in hand with the idea that the significance of the engine evaluation evaporates as the number of variation moves increases.

The mate display problem of Scid vs. PC mentioned in chapter 17) is not aggravated by this reduction of the v ariation moves. Even if ScpcPGN removes the last variation move with a hash sign as mate indication, ScpcPGN still recognises mate and corrects the evaluation at the start of the variation accordingly.

13) Printing Scid vs. PC PGN files via LibreOffice Writer:

Immediate printing of PGN files generated in Scid vs. PC via export in any editor is unconvincing. A two-column set allows smoother reading of a more compact text. Accordingly, an empty text document with normal page size could be created or loaded in LibreOffice Writer and transformed into a two-column text document via Format - Page - Columns: 2. Then the PGN file must be integrated into the text document via Insert - Insert text from file. If text modules with hyphens are separated unnecessarily despite hyphenation being switched off, the replace function could be helpful, whereby the normal hyphen is to be replaced with the non-breaking hyphen (DEC 8209 – HEX 2011 – for example available in the font Cambria).

When exporting the PGN file in Scid vs. PC, the following options would be useful:

Space after Move Numbers: Yes
Export comments: Yes
Strip square/arrow mark codes from comments: Yes
Export Scid Flags: Yes
Indent comments: Yes
Export variations: Yes
Indent variations: Yes
Insert newlines every 80 chars: No
Column style (one move per line): No
Symbolic annotation style: ! += $14
Convert null moves to comments: No
Character encoding: Utf-8.

14) Replacement of NAG values and annotation modules:

This function again does not require engine-generated and annotated variations, but simply any PGN file with such text modules. It removes these modules from the entire PGN file and replaces them with new comments.

A selection of the checkbox "replacement of NAG values and annotation modules" automatically deselects all checkboxes for move annotation and HTML/clipboard. If a selection of the checkbox 'removal of move-square/move colours for Scid vs. PC/Aquarium' should also have taken place, for which in practice there should be a need in the rarest cases, however a restriction to Scid vs. PC or Aquarium as an output programme is undesirable, this checkbox would have to be deselected.

a) Replacement of NAG values:

NAG text modules are replaced which are introduced with the "$" character and continued with a one to three-digit number up to a maximum of 215. The replacement modules are supplied in the file ScpcPGN_english.nag and are automatically loaded and used by ScpcPGN as soon as it encounters a NAG text module. The replacement modules were taken from the help in Scid vs. PC (Index – NAG annotation values).

b) Replacement of engine related modules:

#gxv → game move = x[st/nd/rd/th] variation (x from 1 to 9)
#sdp → search depth
#ela → evaluation
#egm → evaluation game move
#eom → evaluation optimal move
#pes → position evaluation symbol
#pgr → probabilistic game result
#aeg → absolute evaluation gain
#ael → absolute evaluation loss
#reg → relevant evaluation gain
#rel → relevant evaluation loss
#omr → optimum rate

c) Replacement of mate announcements:

#Mx → mate in x moves
#M1 → mate in one move

#+Mx → White mates in x moves
#+M1 → White mates in one move

#-Mx → Black mates in x moves
#-M1 → Black mates in one move

where x from 1 to 999.

d) Replacement of chess piece text modules:

Generally, the minimum 2-digit and maximum 3-digit input is:

[#][w/b][K/k/Q/q/R/r/B/b/N/n/P/p/C/O/o]

where
the '[...]' brackets are for illustration purposes only and must be omitted during input,
'/' stands for 'or' and must also be omitted during input and
[w/b] may be omitted.

Converted are:

w → white
b → black

K → king
k → ♔

Q → queen
q → ♕

R → Rook
r → ♖

B → Bishop
b → ♗

N → Knight
n → ♘

P → Pawn
p → ♙

C → Castling
O → O-O-O
o → O-O

Example:

attack on the #bKside → attack on the black kingside

15) Clipboard mode:

The checkbox "data → clipboard" makes it possible to transfer a variety of data and analyses from a PGN file provided with variation annotations (i.e. possibly even from several games) as well as additionally calculated data, as described above in connection with the text modules for game and variation move commentary, to the clipboard. From there, they can be transferred to other programmes via Ctrl-V. In particular, a spreadsheet program that evaluates these data for diagrams may be appropriate. Such a spreadsheet document called ScpcPGN.ods, which was created as an ods file with LibreOffice, version 6.1.5.2, can be found in the programme package.

To enter the clipboard data there,
immediately after the work was done by ScpcPGN and return to the ScpcPGN GUI (i.e. after filling the clipboard) select the 1st table 'data',
if necessary, delete all rows in the data sheet from row 5,
select the yellow cell A4 and
then enter Ctrl-V.

You could choose in the dialogue ‘Text import’ that thereupon appears:
Character Set: Unicode (UTF-16)
Language: English (USA)
From row: 1
Separated by
Tab
Return/Enter
Voila.

If 'English' was selected as the language in ScpcPGN, the dot is used for decimal numbers in the clipboard. In this respect, it might be advisable to make sure in LibreOffice that under 'Tools – Options - Language Settings – Language' is entered:
Locale setting: English (USA)
Decimal separator key: Same as locale setting
Decimal separator key: [x] Same as locale setting.

Somewhat hidden to the right of the last data line of a game are found the game tags present in the PGN file as well as various evaluations. Particular mention should be made in the case of the presence of Elo values of both players of the calculation of the expected values (between 0 and 1) as well as furthermore of the calculation of the new Elo values in the event of the presence of a meaningful game result. The calculation was thereby based on the formulas of Jeff Sonas, „The Sonas Rating Formula – Better than Elo?“

The diagrams in the 7 diagram tables of ScpcPGN_english.ods are created automatically from the clipboard data inserted in the first table without the need to rely on a macro. This automatic function works either for all games in the range up to data line 1000, which can of course be extended, or for a range of moves from it to be selected by the user (see the green cells in table 'dia data', line 1). If the diagrams should become confusing due to high numerical values, the diagram representation can be restricted to the chosen move area by selecting lower numerical values in the 5 green cells in row 4 of the "dia data" table.

In the 'consistency' spreadsheet it is checked whether various clipboard data supplied by ScpcPGN really correspond to the intended logic of the program code. The data taken over from ScpcPGN and calculated in the table (yellow) are located in adjacent columns. Should there a contradiction arise, the calculated yellow cell will be colored red. In LibreOffice Calc the cell references of the yellow cells can be clearly displayed via Tools - Detective - Trace Precedents / Dependants.

This spreadsheet 'consistency' is not very helpful for the normal user, but it consumes a lot of storage space or CPU performance. It can be deleted in case of dissatisfaction.

If only clipboard mode is selected in the ScpcPGN GUI, possibly in conjunction with HTML mode (chapter 16)), no modified PGN file will be saved.

16) HTML mode:

The checkbox 'data → HTML' causes the output of the clipboard data mentioned in chapter 15) to a HTML file inclusive PGN game header, PGN tags of the PGN file if available, various game evaluations as well as diagrams with the probabilistic game results for each single game. Only those game moves are listed for which engine evaluations entered by Scid vs. PC can be found in the loaded PGN file or for which ScpcPGN has exceptionally formed phantom evaluations (see chapter 4), text module #eva-s#).

ScpcPGN uses the ScpcPGN.html file cited in chapter 1), which contains the necessary CSS instructions. ScpcPGN contributes the basic framework and the analysis data.

The newly formed HTML file must be saved in a folder that also contains the files
jquery.flot.js,
jquery.js,
jquery.flot.barlabels.js,
LICENSE.txt
located in the HTML folder. Otherwise, the charts are not displayed.

The file can be scrolled up and down using the orange-colored arrow buttons.

The moves are color-coded in accordance with the move evaluation symbols. If there is none, it remains with the basic colors white or grey. In the supplied file ScpcPGN.html which is modified during the programme run and then has to e saved under a new name, the move evaluation colours are defined with 6-digit hexadecimal numbers (0-9, A-F, a-f) as follows:

.pos3 {background-color: #00E677;} (!!)
.pos2 {background-color: #4DFFA9;} (!)
.pos1 {background-color: #B3FFDA;} (!?)
.neg1 {background-color: #FFB3FF;} (?!)
.neg2 {background-color: #FF4DFF;} (?)
.neg3 {background-color: #E600E6;} (??)

These definitions as well as all other CSS instructions can be modified by the user.

Also in HTML mode the Elo evaluations mentioned above in the clipboard chapter are displayed if the PGN file provides the corresponding data.

If only HTML mode is selected in the ScpcPGN GUI, possibly in conjunction with clipboard mode, no modified PGN file will be saved.

17) The mate announcement debacle:

Also in version 4.23 Scid vs. PC displays mate announcements partly incorrectly, namely always in the form "Mx", where "x" stands for any number. This applies to mate announcements in favour of White and Black. It is therefore impossible to ascertain from this information which side could mate.

At most, a look at the last move of a variation could help if the mate sign "#" is there. Then it would be possible to determine from this move which side could be the winner. However, there is no guarantee for this, as the help in Scid vs. PC also confirms:

'Additionally, the added line may not be complete - ie. it may not end in mate. This is due to the engine reporting "Mate in n", but not actually giving the whole line/PV, which is a common UCI engine oddity.'

Reference is made to this fruitless discussion.

Scid vs. PC permits itself another mistake in the evaluation of the final mate move, because there, just as with the previous move, it announces an "M1" instead of commenting "mate" or consequently "M0".

In the case of mate announcements, ScpcPGN examines the last variation move and a possible mate sign "#". If it is present, ScpcPGN determines whether the move was executed by White or Black and gives an evaluation corresponding to this colour.

On the other hand, all variations that contain an "Mx" but no mate sign at their end are eliminated by ScpcPGN, since no move evaluation, the heart of the programme, is possible. In addition, it removes all variation blocks (several variations for one game move) if the first variation has such a deficiency. To eliminate only this first best variation, but to leave a worse variation standing and thus upgrade it as a pseudo-best, would be humbug.

It would be a shame to waste valuable engine information and CPU power with such a radical measure. Therefore, in this discussion a simple concept was presented how to get Scid vs. PC away from this misbehaviour:

Load the program file \bin\scid.gui into an editor and search for the following passage (in line 140465 - 140466):

'if {$analysis(logName)} {
set text [format "\[%s\] %d:%s" $analysis(name$n) [lindex $i 0] $tmp_scoremate]'

Change for single-variations: Replace the line

'set text [format "\[%s\] %d:%s" $analysis(name$n) [lindex $i 0] $tmp_scoremate]'

with the following passage:

'if { [lindex $i 1] > 320 } {
set text [format "\[%s\] %d:+%s" $analysis(name$n) [lindex $i 0] $tmp_scoremate]
} elseif { [lindex $i 1] < -320 } {
set text [format "\[%s\] %d:-%s" $analysis(name$n) [lindex $i 0] $tmp_scoremate]
} else {
set text [format "\[%s\] %d:%s" $analysis(name$n) [lindex $i 0] $tmp_scoremate]
}'

Then search for the following passage (in line 141260 – 141261 of the scid.gui modified according to the previous recommendation):

'if {$analysis(logName)} {
set text [format "\[%s\] %d:%s" $analysis(name$n) $analysis(depth$n) $tmp_scoremate]'

Change for multi-variations: Replace the line

'set text [format "\[%s\] %d:%s" $analysis(name$n) $analysis(depth$n) $tmp_scoremate]'

with the following passage:

'if { $analysis(score$n) > 320 } {
set text [format "\[%s\] %d:+%s" $analysis(name$n) $analysis(depth$n) $tmp_scoremate]
} elseif { $analysis(score$n) < -320 } {
set text [format "\[%s\] %d:-%s" $analysis(name$n) $analysis(depth$n) $tmp_scoremate]
} else {
set text [format "\[%s\] %d:%s" $analysis(name$n) $analysis(depth$n) $tmp_scoremate]
}'

Whenever Scid vs. PC finds an internal evaluation of > 320 or < ‑320, it will output a '+Mx' or a '‑Mx' with these changes, where the ‘x’ represents the distance to mate in full moves. ScpcPGN converts this expression into half moves and outputs a rating of ±300.00 ∓ number of half moves to mate – with white advantage without sign, with black with preceding minus sign. An immediate mate move is evaluated at (‑)300.00. The last variation move is then no longer important for ScpcPGN.

If the text module # mate# is used for the game or variation move comments, ScpcPGN uses a mate announcement in the form ' Mx' in favour of White or ' ‑Mx' in favour of Black, where 'x' stands for the distance to mate in full moves. If, as is usually the case, no mate is to be announced, the text module is removed. # mate# indicates that ScpcPGN automatically places a fixed space before this mate announcement. It is therefore advisable to insert the text module immediately after one of the 3 evaluation text modules, whereby #evalu## mate# is generally suitable, but #e-c-e## mate# is also appropriate for the game move comments in Aquarium. A blank space between both modules is omitted so that the removal of the # mate# module does not leave an unnecessary space.

18) Evaluation relevance reduction:

For many of the programme features discussed above, especially the annotating by move evaluation symbols, the move colouring, the position evaluation symbols, the relevant wins and losses, the optimum rates and the probabilistic game results, the probably worldwide unique evaluation relevance reduction (ERR) was the inspiration. It relativizes engine evaluations and reduces absolute engine evaluation differences to relevant evaluation differences by means of integral functions.

0.00-evaluations are taken into account at 100%, extreme positive or negative evaluations at 0% (WDL ERR) or close to 0% (user ERR) and all other evaluations in the range between these extreme values.

Since the programme update of February 2022, the engine/calculation WDL ERR inspired by the engine Stockfish is available as an alternative to the previous user ERR. It is based on win/remis/loss estimation quotas, which are written for each move into the game commentary by Scid vs. PC for the first time with version 4.22 when using a suitable engine, such as Stockfish. However, Stockfish does not show these WDL values automatically, but only after clicking on the checkbox ‘UCI_ShowWDL’ in the configuration window (Tools – Analysis – Engines – Edit – Configure).

More details on this topic can be found here.

Unfortunately, Scid vs. PC only notes two-digit percentages in the move commentary. To have Scid vs. PC add at least one additional decimal place, for example, the programme code of Scid vs. PC must be modified as follows:

Load the program file \bin\scid.gui into an editor and modify lines 173550, 173552 and 173554 of the scid.gui changed according to the previous recommendations respectively so that
'[expr {[lindex $data $i] / 10}]'
is replaced by
'[format "%.1f" [expr {[lindex $data $i] / 10.0}]]'.

Scid vs. PC does another manipulation of the original engine WDL data by exchanging the win and loss data for Black's moves, thus favouring the White view - highly confusing! The remedy is the radical removal of lines 173616 to 173620 as well as 173623 to 173637 from the scid.gui changed according to the previous recommendations.

For the PGN move annotation, either the WDL or user ERR text modules are available. For the HTML and spreadsheet files, the move evaluation symbols and move field colours, the ERR mode can be selected by choosing one of the 3 following alternatives at the bottom right of the ScidvsPC:
'as per original engine WDLs or calculation' (recommended if moves should not have been equipped with WDL values by Scid vs. PC); 'only as per original engine WDLs' (recommended if one wants to avoid calculation of WDL values by an algorithm used by ScpcPGN); 'never - only user evaluation relevan. reduction'.

The relevant parameters are to be entered on the left in the ScpcPGN GUI under the heading 'evaluation relevance reduction':


Firstly the TCEC engine statistics may be helpful in answering the question of which 3 values make sense. Because not least the evaluation behaviour of an engine that was used for the game annotation in Scid vs PC could be decisive for the measurement of these parameters. In addition, the WDL ERR opens up a new source of information. In the generated HTML and spreadsheet files, average theoretical WDL evaluations for the probabilistic game results 0.75 as well as 0.75+ (to be entered by the user in the ScpcPGN GUI) can be found separately for White and Black after each game, although they can differ considerably depending on the ratings occurring in the game and are therefore not very practical.

ScpcPGN is thus a programme in which the parameters that are decisive for such essential chess assessments as move and position evaluation symbols, relevant wins and losses, optimum quotes and probabilistic game results are transparently available to the user for personal experiments.

19) Programme selection for PGN files:

On the right side of the ScpcPGN GUI under the headline 'PGN file → chess programme' there is a selection list for the following 7 chess programmes:

Aquarium,
Arena,
Kvetka,
Lucas Chess,
pgn4web,
Scid vs. PC,
Shredder Classic 4.

Some modifications are made to the new PGN file to be saved, depending on which programme was selected to load the file. Affected are


Lucas Chess displays PGN games including variations and annotations via Tools - PGN - Read PGN - Edit, if under Options - Configuration - Appearance - PGN table – Enable information panel by default is selected. Otherwise, you must also click on Config - View - PGN information. Lucas Chess can colour moves that are provided with move evaluation symbols, but the colors are set there under Options - Colors - General – PGN.

The 2 games in the PGN file "ScpcPGN_Analyses_english.pgn" can be viewed with the help of pgn4web on the Internet.

20) Parameters and checkboxes:

All settings relevant for the ScpcPGN GUI, including the selection of checkboxes, can be loaded and saved individually as parameters.

Not every click on any checkbox or in the list of chess programmes is rewarded with the expected reaction. In some cases, the programme knows how to avoid counterproductive orders and deselects checkboxes on the spot. This is a programme feature, not a bug.

If the button labelled 'English' at the bottom left of the ScpcPGN GUI was clicked to switch to English-language programme texts, only the English-language text modules would be converted into data when creating an analysis PGN file, provided they were loaded with a suitable parameter file. If the button designation is 'German', such a click would cause German to be selected as the GUI official language and loaded German-language text modules would again be given their due.

21) GUI background colour:

The input field between the sections 'data → clipboard' and 'data → HTML' allows the choice of the GUI background colour with a colour code analogous to the 6 colour code lines already mentioned in chapter 9) on the right side of the GUI under the checkbox 'move-square colours for !! / ! / !? / ?! / ? / ??'.

22) ScpcPGN programming:

ScpcPGN was created with AutoIt3 v3.3.14.5. When making changes to the source code in the ScpcPGN.au3 file under other AutoIt versions, 'script breaking changes' should be considered.

ScpcPGN has been tested with the latest version 4.23 of Scid vs. PC.

The PGN files modified by ScpcPGN were tested with Scid vs. PC itself, of course, as well as with Aquarium, then a little with Arena, Kvetka, Lucas Chess, Shredder Classic 4, and with the internet PGN viewer pgn4web (versions 2.90 and 3.05).

If the user lets analyze several games in a PGN file with the aim of saving a new PGN file, he will inevitably be confronted with the snail's pace of ScpcPGN. With more than one game per PGN file, the programme very quickly will be brought to its knees. AutoIt3 is an interpreter language and therefore sometimes painfully slow, but on the other hand it allows extremely imaginative applications without immense workload. It is a compromise!

No software is error-free, ScpcPGN is certainly not. The program does complicated things and as a lonely individual programmer you can't devote yourself to endless tests before publication. In this sense the error frustrated users are abused as involuntary beta testers and encouraged to make use of the following e-mail address in order to give the opportunity to eliminate the shameful missteps of ScpcPGN with a new program version.

Contact: mail@konrod.info


zip-Icon

program ScpcPGN - development version 11th November 2022
ZIP file [2.45 MB]: ScpcPGN.zip
in the ScpcPGN GUI click on the button 'English'








 

Finito ♦ Aus die Maus ♦ Schicht im Schacht ♦ Klappe zu - Affe tot

So long ♦ See You Later, Alligator - In A While, Crocodile ♦ Over And Out