Category Archives: Tips and Tricks

openSSL vs. vcpkg – some strange experiences

One of the projects I was working in the last period uses CURL that uses OpenSSL version 1.1.1.1 in foreground. At some moment in time, for some technical reasons I was tempted to try using the CPPRESTSDK library. But because I had some local NuGet miss-configuration issues in order to have the CPPRESTSDK library build I realized I need to build it myself. But in order to achieve this, I needed the vcpkg tool.

And, because with a quick search I haven’t found the download and the Azure DevOps of this tool for built was down and no Artifacts available, I said: what if I build it myself?

So, I did it… It lasted some time on my laptop but at the end was fine. I had a brand new vcpkg package and I built the cpprestsdk.

Later, returning to my project and integrating the new cpprestsdk library I realized I was started getting new linking errors.

2>Generating   Code...   
2>libcrypto.lib(comp_lib.obj)   : error LNK2005: COMP_CTX_free already defined in libeay32.lib(LIBEAY32.dll)  
2>libcrypto.lib(comp_lib.obj)   : error LNK2005: COMP_CTX_new already defined in libeay32.lib(LIBEAY32.dll)
2>libcrypto.lib(comp_lib.obj)   : error LNK2005: COMP_compress_block already defined in   libeay32.lib(LIBEAY32.dll)  
2>libcrypto.lib(comp_lib.obj)   : error LNK2005: COMP_expand_block already defined in   libeay32.lib(LIBEAY32.dll)   
2>Creating library C:\dev\ my-project\Build\x64\Debug\myproject.lib and object   C:\dev\my-project\Build\x64\Debug\myproject.exp   
2>C:\dev\my-project\Build\x64\Debug\   myproject.dll : fatal error LNK1169: one or more multiply defined symbols   found

Within the sample solution, the situation is even worth. I was able to find out even more compile errors while the vcpkg is installed.

Vcpkg uses OpenSSL to insure secure connection and googling I found out that starting with the OpenSSL 1.1.x Windows version the static LIB and DLL filenames have changed, so depending on local files, it may affect, the former DLL’s libeay32.dll and ssleay32.dll are no longer available.

I double-checked the environment variables, trying to identify paths containing the old libeay32.dll (that was renamed to openssl.dll) and to remove it. I did such investigation, but I found nothing “suspicions”.
The goal was to avoid the names’ collision in case the linker was looking to link “our” openSSL together with LIBEAY32.dll.

The salvation came from GauthamBanasandra’s suggestion. I just ran the next command within the vcpkg repository local folder.

.\vcpkg.exe integrate remove

After the execution of this command, I was able to build back the project as before. And it was fine for me… I don’t need the vcpkg for other proposes right now.
By the way, this command has to be executed within the vcpkg cloning repository folder. Otherwise, you’ll get an error: Could not detect vcpkg-root in case you don’t have an environment variable VCPKG_ROOT pointing to your vcpkg binary repository.

Off-course, in case you still need vcpkg.exe you can reinstall it temporary with vcpkg.exe integrate install. But then, until there is a fix for this situation or a better workaround, you might face the initial problem I had within OpenSSL based projects. So, most probably you will have to call again vcpkg.exe integrate remove.

The issues are created by the fact the actual vcpkg.exe manager is still using the old openSSL library, libeay32.dll.

Until the Vcpkg team didn’t upgrade to a newer OpenSSL library (1.1.x+), we will have to find such workarounds in order to compile different projects. If someone knows other workaround and is open to share, I would not mind mentioning here her/his idea.

The good news is that an upgrade to the OpenSSL 1.1.x is ongoing. The upgrade issue is already open and the pull request #8566 seems advanced in progress. Let’s stay tuned!

psa.exe partial tree sample

Process Status Analysis – the first steps

I am pleased to announce my first cross-platform and open source project, the Process Status Analysis tool, available on GitHub.

The Process Status Analysis (psa) version 0.2 is available for Windows and Linux (Debian derived / Ubuntu tested) Operating Systems.
Download: psa for Ubuntu Linux x64 (3582 downloads)
Download: psa for Ubuntu Linux x86 (2833 downloads)
Download: psa.exe for Windows x64 (2811 downloads)
Download: psa.exe for Win32 (2829 downloads)

You may wonder why I did it or what it brings new. Well, I did it for fun, in my spare time and I will continue improving it when I’ll find a time to do it.

The project is written in modern C++ using idioms from the C++ 1x standards. Even if initially was done as a C++ for Windows only, during the past days I managed the port of it for Linux using Visual Studio 2017’s project templates and a connection via SSH.
In general, the source code base is similar, differing just by OS specific stuff.

In case you want to find out more about how to develop C++ Linux projects from the best development tool (imho), Visual Studio, you can find more information on Visual Studio development team blog.

Related to this psa project, the Linux version requires libprocps4-dev library in order to build.

The main reason for starting this project was that I wanted to know what’s the total memory amount used by my Chrome browser. I know it uses a lot of resources, but not that much… 🙂

Even if my preferred processes analysis tool, the Process Hacker offers a lot of processes administration possibilities, but it didn’t provide what I want, so I decided to enjoy a bit.

Chrome processes in Process Hacker

Sample – Google Chrome processes in Process Hacker tool

So, what I achieved by psa.exe was something like:

C:\Windows\system32> psa -o chrome
PID [896] chrome.exe 237.5234 MB
PID [1496] chrome.exe 87.6875 MB
PID [2388] chrome.exe 166.5000 MB
PID [5860] chrome.exe 3.1211 MB
PID [7336] chrome.exe 273.2188 MB
PID [8444] chrome.exe 68.0508 MB
PID [8624] chrome.exe 63.3945 MB
PID [9180] chrome.exe 296.9766 MB
PID [10600] chrome.exe 292.6289 MB
PID [12352] chrome.exe 182.5977 MB
PID [13688] chrome.exe 2.3555 MB
PID [14052] chrome.exe 73.1875 MB
PID [16200] chrome.exe 211.9805 MB
PID [17284] chrome.exe 55.7148 MB
PID [18036] chrome.exe 208.6680 MB
PID [19012] chrome.exe 457.2305 MB
PID [19312] chrome.exe 143.4766 MB
-----------------------------------
Total used memory: 2824.31 MB

A bit too much in my humble opinion…
The features this tool offers includes:

Get all processes loaded in memory information

I case you want to have a snapshot of all the processes loaded in the OS’s memory you can have it with.

psa -a

Get process only used memory

With psa.exe you can reach the used memory by a specific parameter -o after the process name or at least a part of its name.

psa -o chrome             // find how much memory uses your Chrome!   o_O

Currently, there is no string replace ‘*’ but it’s ongoing.


Print processes tree

Storing the processes’ data within a generic tree done by me, I took the decision to print the processes’ tree output, similarly there is in Windows with tree.exe tool or on Linux in pstree or even htop.

./psa -t
./psa -t 1034

psa.exe partial tree sample

Process Status Analysis partial tree of Windows process

Top most “expensive” processes

In case you want to see what are the most expensive processes within your operating system, you can achieve it with:

psa -e 20

or simpler psa -e in case you’re sure you want top 10 expensive processes (the default value).

silviu@ubuntu-dev-server:~/projects/psa-lin/bin/x64/Release$ ./psa -e
Top 10 consuming memory processes
-------------------------------------------
PID        Process Name         RAM Usage
-------------------------------------------
[924]    /usr/lib/policykit-1/polkitd   270.68 MB
[906]    /usr/lib/accountsservice/accounts-daemon       269.43 MB
[842]    /usr/sbin/rsyslogd     250.39 MB
[878]    /usr/lib/snapd/snapd   197.29 MB
[531]    /lib/systemd/systemd-timesyncd         97.97 MB
[1614]    sshd: silviu@pts/0    93.16 MB
[1576]    sshd: silviu [priv]   93.16 MB
[863]    /usr/bin/lxcfs         93.13 MB
[427]    /sbin/lvmetad          92.55 MB
[1034]    /usr/sbin/sshd        63.98 MB
-------------------------------------------


C:\Windows\system32> psa -e 10
Top 10 consuming memory processes
-------------------------------------------
PID        Process Name         RAM Usage
-------------------------------------------
[19012]    chrome.exe           435.67 MB
[17684]    chrome.exe           329.51 MB
[7336]    chrome.exe            259.61 MB
[15576]    devenv.exe           248.20 MB
[896]    chrome.exe             222.85 MB
[2388]    chrome.exe            188.21 MB
[18428]    chrome.exe           184.48 MB
[15760]    chrome.exe           173.82 MB
[488]    Dropbox.exe            162.15 MB
[5488]    googledrivesync.exe   161.11 MB
-------------------------------------------
Total used memory: 2365.61 MB

Redirect output to a file

From the standard output the information can be easily redirected to a file.

c:\> psa -t > windows_processes_tree.txt
# ./psa -t > linux_processes_tree.txt        

Kill process feature

This feature was not implemented yet but in case we need it we can be done it easily with the existing tools on the target OS (ex. Task Manager, Process Exporer/Hacker, pskill.exe for Windows or the combination ps + kill on LInux).

Feedbacks and improvements
Any constructive feedback, suggestions, contributions to improvements are appreciated.
Feel free to add any issue you find, wish or suggestion you have in the GitHub repository, the 
Issues section or here as a comment.

Finally, I got Windows 10

Finally, I got the Windows 10 for my laptop… Even if this task is trivial, it was a surprising experience this time. But let me tell you the story of Windows 10 installation on my Samsung RC 710 laptop.

Back in August, I tweeted “That’s all I have within #Windows10 X64 Ent setup on Samsung RC710 with SSD”. By that, I meant that my setup process was stuck in the boot phase using the installation of this Windows brand new version. That was reproduced within Windows 8.1 x64 upgrade tentative or with clean Windows 10 installation.
I tried few Windows 10 .ISOs and a friend’s DVD but no chance. The setup has started and stuck within few seconds.

win10_setup_stockes

Because the laptop got some hardware upgrades from the original configuration, I tried to restore to the initial configuration but nothing new. Reading this article where is specifying “For 64-bit installations, a small number of older PCs may be blocked from installation because they do not support CMPXCHG16b, PrefetchW, and LAHF/SAHF“, I tried an x86 .ISO but I got the same situation. Also, I tried some BIOS changes without any improvements.

Getting online contact with Microsoft it offered me no new things. I got only typical support trivial answers.

During the time on the same laptop, I was running fine Win XP, Win 7 and Win 8.1 OSs without any such bad experiences. To me it was clear that Windows 10 has some backward compatibilities issues.

So, I took it on my own googling for my situation. Reading different forums I realized this is a common issue for Samsung old laptops and it is generated by the WiFi card.

The solution in my case was buying a brand new Atheros AR5B22 WiFi card and replacing the old WiFi card with that.

If you’re in a similar situation and you’re looking for instructions how to disassemble your laptop here it is a brief presentation.

By the way, you’re doing on your own risk. If you’re not confident, please contact a specialist.samsung_laptop_to_win10_3samsung_laptop_to_win10_4samsung_laptop_to_win10_6samsung_laptop_to_win10_5upgrading_windows_on_samsung_laptop

With this new WiFi card plugged-in, the upgrade from Window 8.1 x64 to Windows 10 x64 became a trivial task.

It would be nice if Microsoft would get more in count such behaviors and would improve the Windows 10’s backward compatibility, especially because the old computers are included within their OS target.

Update 02.06.2016: Samsung admits they are lame: “Don’t Install Windows 10 Because We Suck At Making Drivers”. Sad…

You can find additional information here. That’s why, most probably I will never buy a Samsung phone or any other gadget made by them, anymore.

Experiences with Adobe Acrobat/Reader Plug-ins

box_adobe_150x150I wrote this document after a challenging experience I had recently within an Adobe Acrobat/Reader plugin creation. Even if the Adobe’s SDK it’s nicely documented within PDF files, one of the major reason that determined me to write this article was the frustration I got sometimes when, for instance, trying to see “why the plugin was not loading into Acrobat/Reader” and the Google’s engine provided me a lot of references such “why the Reader plugin is not loading into a browser”. Also, the search functionality from Adobe’s forum didn’t help me too much. I hope to help others by clarifying some challenges might meet a developer at begin of creation such kind of plugin.
Adobe has two products for .PDFs file handling: the freeware Reader capable of reading only and the Acrobat for reading and one that supports effective .PDFs creation, Adobe Acrobat. Both Acrobat and Reader use the same SDK but the Reader APIs is a subset of those available for Acrobat (obvious).
There are three types of plugins: regular plugins, reader-enabled plug-ins and certified plug-ins.

General considerations

Plug-ins for Acrobat can be developed and distributed freely and no license is required from Adobe. The payment exception appears in case of DRM agreement which includes a $50,000 annual fee and a 5.5% revenue royalty. Adobe considers to apply digital rights management (DRM) in case the developed plugins functionality invoke “encrypting a PDF file or controlling access to a PDF file, then it is DRM. Also, if you add any functionality to the security settings of Adobe Acrobat (…). If your plug-in is required for someone to access the PDF file, then we would consider it to have DRM functionality”.

Only plug-ins that are shipped as part of Acrobat and Reader can be ‘certified’. This is so that if users wish to run Acrobat or Reader without any 3rd party plug-ins, they can do this easily by using the ‘Certified plug-ins only’ checkbox in the preferences.

Adobe maintains a registry of four-character prefixes for each company that develops extensions for its own products. The new companies that intend to develop such plugins should contact Acrobat Developer Support to obtain a four-character prefix to be used. Adobe’s prefix is ADBE or ACRO. This prefix is needed to be used with various elements as well as private data that it writes into PDF documents.

CropMenu = AVMenuNew(SuperCrop, "ADBE:SuperCropMenu", gExtensionID);

For Adobe Reader the plugin needs a special macro to be defined by project settings READER_PLUGIN. By defining it it’s easy to identify in case you’re calling an Acrobat only specific SDK function because it causes compiler errors.

The First challenges

After downloading the SDK the first instinct was trying the project samples. Once with this step appeared the first annoying situation: I loaded all.sln solution into Visual Studio I have been observed that whatever project I built and deployed into the Reader “plug_ins” subfolder I was not able to see them in Adobe Reader. The “plug_ins” subfolder or one more subfolder level down is the place you have to deploy the built plugins. These plugins are DLLs with an .API extension. The confusion has amplified because by downloading and installing the FileOpen WebPublisher Client plugin I observed that the plugin was running perfectly and I saw it even into Help – Adobe Third Part Plug-ins menu.

But deploying such plugins in the Adobe Acrobat “plug_ins” folder was up and running. I started reading the Developing Plug-ins and Application for Adobe Reader I followed the “why a plug-in might not load” founded steps but no solution for Acrobat Reader. Just in case I unchecked the “Use only certified plug-ins” Reader’s setting and nothing (‘Certified plug-ins only’ = Edit > Preference: Application Startup: Use only certified plug-ins (unchecked)).

Trying to debug over the plugin source code by attaching to Reader project or by starting the debug with Reader application didn’t help me more.

Later, after some challenges, I found out that the key point of understanding why the SDK sample DLLs were not loading into Adobe Reader it was that the plugins for Reader need to be signed before being deployed into “\Program Files\Adobe\Acrobat\plug_ins” directory. Such information is not present into that “why is not loadingmanual list.

How to sign the plug-in for Adobe Reader

As I mentioned earlier the plug-in in Reader must be signed by using a certificate provided by Adobe. It is highly recommended that you make the application for a key at the beginning of the development process since the application can be denied in case the plugin functionality is not in accordance with Adobe’s business goals. Also, this ensures that your agreement is in place when you are ready to build the Reader version of the plug-in. If the key is approved, the developer must build public key and pair key files using a tool in the Acrobat SDK.

In the moment somebody wants to develop a plug-in for Acrobat Reader has to fulfill an integration form, not before creating an Adobe ID. According to Adobe the approval process might take some time (up to two weeks). The application should be filled out completely and your responses will be used to determine your eligibility. If you are building a DRM-based Adobe Reader plug-in, we recommend you send an email to with details of your request so that we can guide you through the application process.

Generates the public and private pair keys by the Makekeys tool:

MakeKeys -kp MyPlugIn.Pair.key -pk MyPlugIn.Public.Key YourGenericKey

The size of the public key should be 98 bytes. The size of the public and private key pair should be 451 bytes. The size of the returned encrypted key should be 554 bytes. Save this .key generated files into a proper location cause later might be useful including it into your project.This tool is located into your SDK: ex. sdk110_v1\PluginSupport\Tools\Reader-enabling Tools\win.

Submit the new created Public Key file and the fulfilled form document to and wait to get the digital certificate. This will be a RIKLA-DigCert.rc file.  In case you will receive approval from Adobe there are several more steps you need to follow to receive your Reader Integrated Key for your plug-in. The Key arrives as a digital certificate. Once this is done the plug-in will load into Reader. Note that if the plug-in is recompiled the plug-in must again be signed (the same certificate and key-pair files can be used).

Once you get the digital certificate file, you should sign the fresh built plugin, before deploying it into Reader’s plug_ins folder.

Here, because of using SDK 11 I got some confusion because of the steps described in the “enabling the plug-in for Acrobat Reader” section, according to apps documentation guide. They are talking about Makecmd32.exe tool, some API_ENCRYPTED_GIGEST and API_DIGITAL_CERTIFATE IDs, etc. But the SDK 11 has no Makemd32.exe tool coming with. This tool can be downloaded within other RIKLATools.zip file but I preferred following the actual SDK 11 documentation especially because it has another signing approach. Instead of Makemd32.exe I had to use SignPlugin.exe (into SDK docs: Plug-ins and Applications > Developing Plug-ins and Applications > Creating an Adobe Reader Plug-In > Developing and enabling an Adobe Reader plug-in > Enabling the plug-in for Adobe Reader).

..\MySources\MyPlugIn\keys>SignPlugin -kp MyPlugIn.Pair.key -cf RIKLA-DigCert.rc
"C:\Program Files (x86)\Adobe\Reader 10.0\Reader\plug_ins"\MyPlugIn.api

Plug-In Structure

The Acrobat/Reader applications have few steps approach for plugins: initialization + plugin, handshaking, exporting and importing HFTs and unloading the plugin, implemented as callbacks. A minimum operation that a plug-in must implement is PluginInit() callback function.

The plugin life cycle into Adobe Acrobat/Reader invokes next steps:

  • At startup search into its plugin directory (plug_ins). It looks in the .API files for the exported PlugInMain, it loads the plugin by invoking the LoadLibrary and call the function pointed by the symbol of PlugInMain.
  • For each detected plugin (.API) it attempts loading the file. If the plugin is successfully loaded the Reader/Acrobat invokes routines from PIMain.c and completes the handshaking process.
  • Invokes callbacks in the next order:

PIHandshake

PluginExportHFTs

PluginImportReplaceAndRegister

PluginInit

  • Before closing Reader/Acrobat the PluginUnload callback function it’s executed. That’s the proper place to release the allocated resources.

In the initialization phase, the plugin hooks into Acrobat’s user interface by adding menu items, toolbars, etc. The unload procedure should free any memory the plug-in allocated and remove any user interface changes it made.

Handshaking is also one of most important step. The application performs checking with each plugin before opening it. It is the step where a plugin for Adobe Reader it is tested before loading. During this operation the plug-in specifies the name, some initialization procedures, signature test and optional an unload procedure if is needed. In case the signed test fails the loading process of that plugin is stopping.

How to create a plugin

Even if the Acrobat SDK allows creating plug-ins for OS platforms out of Windows (MAC, Unix/Linux) without too many differences (most because of configuration and used tools), I will describe down some details for plugin development on Windows platform.

Download the latest Acrobat SDK and unzip in a preferred location. Create an environment variable for AcroSDKPIDir that contained the SDK content.

$(AcroSDKPIDir) = D:\work\adobe\acrobat\SDK110_v1

Running Visual Studio “as administrator” it’s a good idea in order to be able to succeed the write into Adobe’s plug_ins folder. In order to establish an easier debug and deploy process, I preferred to add two additional environment variable AcroPluginsDir containing the Acrobat plugins files and ReaderPluginsDir for Reader plugins files.

AcroPluginsDir = C:\Program Files (x86)\Adobe\Acrobat 11.0\Acrobat\plug_ins
ReaderPluginsDir = C:\Program Files (x86)\Adobe\Reader 10.0\Reader\plug_ins

Having these environment variable set into your OS you can start the effective plugin creation.

According to Acrobat SDK you can start from an existing sample so-called Starter project or you can start from an empty DLL project. The first version allows you having a fast up and running own plugin by just adjusting the files name and starting to apply the business logic.

In case you choose the clean approach you need to add paths to the SDK header files into C/C++ > General > Additional Include Directories as for instance:

..\..\..\Arch\Adobe\Acrobat\SDK110_v1\API
..\..\..\Arch\Adobe\Acrobat\SDK110_v1\SDK

This will be needed for instance to easily include “PIHeaders.h” file.
Add next preprocessors definitions into project settings: WIN_PLATFORM, WIN_ENV and READER_PLUGIN (C/C++ > Preprocessor > Preprocessor Definitions).
Include PIMain.c file into your project. This file is located into your Acrobat SDK path. In my case it is:

..\..\..\Arch\Adobe\Acrobat\SDK110_v1\API\PIMain.c

Add the standard Acrobat callbacks functions prototype into other .cpp file (functions invoked into plugin structure topic) and start the business logic implementation. Here you can inspire from the content of StarterInit.cpp file (Starter sample project). In case you want to add some menu, toolbar or other UI items these should be added into PluginInit() function.

The PlugInMain() function is the entry point into such plugins and it’s needed to add the export flag to PlugInMain() function via project settings:

Linker > All Options > Additional Options the next command: /EXPORT:PlugInMain %(AdditionalOptions).

Without this setting, you will get a big surprise even if at the very first point of view the built plugin is signed and the DllMain() is accessed into a debugging flow. But none of the callbacks functions without this export.

In order to automatize the process for plugin build and deployment you might be added some Post Build Event commands:

copy $(Configuration)\MyPlugIn.api "$(AcroPluginsDir)"
copy $(Configuration)\MyPlugIn.api "$(ReaderPluginsDir)"
SignPlugin -kp keys\MyPlugin.Pair.key -cf keys\RIKLA-DigCert.rc "%ReaderPluginsDir%"\MyPlugIn.api

Conclusions: In my opinion, the Acrobat SDK it’s nicely designed but even if there are a lot of PDF references, somehow it doesn’t have the best online structured content, causing users to waste enough time to match all the pieces. Maybe because of complexity and flexibility that exposes it’s not very easy to find complete clean references.

Getting Table’s indexes experiences – workaround

Trying to get table indexes information in SQL Server 2012 I identified a strange situation within a specific method that I was using so long but it was not acting as expected in one situation.

The way of getting indexes information using the ODBC C API into that old and inherited method looks like:

nRetCode = ::SQLStatistics(hstmtAux,
                                    NULL,
                                    0,
                                    NULL,
                                    0,
                                    (TCHAR*)(LPCTSTR)strTempTable,
                                    SQL_NTS,
                                    SQL_INDEX_ALL,
                                    SQL_ENSURE);
if (nRetCode == SQL_SUCCESS || nRetCode == SQL_SUCCESS_WITH_INFO) {
  nRetCode = ::SQLBindCol(hstmtAux, 4, SQL_C_SHORT, &swNonUnique, sizeof(SWORD),
                          &cbNonUnique);
  nRetCode = ::SQLBindCol(hstmtAux, 5, SQL_CHAR, szIdxQualif,
                          sizeof(CHAR) * 130, &cbIdxQualif);
  nRetCode = ::SQLBindCol(hstmtAux, 6, SQL_C_CHAR, szIdxName,
                          sizeof(CHAR) * 130, &cbIdxName);
  nRetCode =
      ::SQLBindCol(hstmtAux, 7, SQL_C_SHORT, &swType, sizeof(SWORD), &cbType);
  nRetCode = ::SQLBindCol(hstmtAux, 8, SQL_C_SHORT, &swSeqInIdx, sizeof(SWORD),
                          &cbSeqInIdx);
  nRetCode = ::SQLBindCol(hstmtAux, 9, SQL_C_CHAR, szIdxColName,
                          sizeof(CHAR) * 130, &cbIdxColName);

  while (bNoFetch || nRetCode == SQL_SUCCESS_WITH_INFO ||
         (nRetCode = ::SQLExtendedFetch(hstmtAux, SQL_FETCH_NEXT, 1, &crow,
                                        &rgfRowStatus)) == SQL_SUCCESS) {
    if (cbIdxName != SQL_NULL_DATA &&
        _tcslen((TCHAR)szIdxName) < 0) {  
        // rest of the code 
    } 
  } // rest of the code 
}

Usually, I got the right information about indexes but in one situation I encounter a strange behavior. It’s about having a clustered index into a scenario. I have a table that contains two indexes referenced to some fields: IndexField_1 and IndexField_3 mapped over int, NULL fields. When IndexField_1 is Non-Unique, Non-Clustered and IndexField_3 is Clustered index I get the right information.
But if the index IndexField_1 is Clustered and the IndexField_3 is Non-Unique, Non-Clustered I get no information about IndexField_1 index (eg. szIdxName and szIdxColName are “” and their length is -1 that means SQL_NULL_DATA). Within while loop, with the next iteration, I get correct information about the second index IndexField_3.

Because SQLExtendedFetch() is deprecated I tried using SQLFetchScroll() but the behavior is the same from my interest point of view.

I was not sure whether the problem is with SQLStatistics, the bindings or SQLFetchScroll (they all always return SQL_SUCCESS). It looks such a problem with the driver when the first index is clustered.
According to SQLStatistics documentation if my swType parameter is SQL_TABLE_STAT I have no information for index or field. But for this scenario, I had no indexes of combined fields.
For the good scenario I observed that my while loop had 3 iterations including one of having swType = SQL_TABLE_STAT without information in szIdxName. But for the bad scenario, the loop had only 2 iterations. So it looks like SQLExtendedFetch() is not getting the last one index.

After some googling and research without very significant solutions, I decided to apply a workaround by avoiding the old API and I rewrite my method.

So, in order to get table indexes information, I have chosen a direct SQL query into SYS tables: sys.tables, sys.indexes, sys.schema.

SELECT DISTINCT I.[name] AS IndexName, I.is_unique AS IsUnique,
    I.is_primary_key AS IsPrimaryKey, I.type AS IndexType,
    I.type_desc AS IndexDesc,
    T.[object_id] AS ObjectID FROM sys.tables AS T INNER JOIN
        sys.indexes AS I ON T.[object_id] = I.[object_id]and T.name =
        'myTABLE' and
        I.type_desc<> 'HEAP' INNER JOIN sys.schemas AS S ON T.schema_id =
            S.schema_id and s.name = 'myTABLE'

Because I preferred getting also information about the index’s composed fields, I applied a second additional SQL query:

SELECT AC.Name as ColumnName FROM sys.tables as T INNER JOIN
    sys.indexes as I on T.[object_id] =
    I.[object_id] INNER JOIN sys.index_columns as IC on IC.[object_id] =
        I.[object_id] and IC.[index_id] =
            I.[index_id] INNER JOIN sys.all_columns as AC on IC.[object_id] =
                AC.[object_id] and IC.[column_id] =
                    AC.[column_id] WHERE I.object_id =
                        'NumericObjectID' and T.name = 'TableName' and I.name =
                            'IndexName' order by T.name,
                        I.name

and I have collected data into a container of defined structure according to my SQL Indexes interest information:

struct SQL_INDEX {
  CString index_name;
  bool is_unique;
  long object_id;
  bool is_primary;
  short index_type;
  CString index_desc;
  std::vector vectColumns;
};

The last member vectColumns stores information about the columns that are used for a specific index.

Finally, the new method that collects table indexes information looks like:

// collect table indexes information
void CFoo::GetIndexesInformation(const CString& strSchema,
                                 const CString& strTable,
                                 std::vector& vectKeys,
                                 CDatabase* pDBdata) {
  HSTMT hstmt = SQL_NULL_HSTMT;
  SQLRETURN nRetCode = 0;
  CString sqlQuery;

  sqlQuery.Format(
      _T("SELECT DISTINCT I.[name] as IndexName, I.is_unique as IsUnique, I.is_primary_key as IsPrimaryKey, I.type as IndexType, I.type_desc as IndexDesc, T.[object_id] as ObjectID \
FROM sys.tables as T \
INNER JOIN sys.indexes as I on T.[object_id] = I.[object_id] and T.name = '%s' and I.type_desc <> 'HEAP' \
INNER JOIN sys.schemas as S on T.schema_id = S.schema_id and s.name = '%s'"),
      strTable, strSchema);

  if ((nRetCode = ::SQLAllocStmt(pDBdata->m_hdbc, &hstmt)) == SQL_SUCCESS) {
    if (SQL_NULL_HSTMT != hstmt) {
      pDBdata->OnSetOptions(hstmt);

      nRetCode = ::SQLExecDirect(hstmt, (CHAR*)(LPCTSTR)sqlQuery,
                                 sqlQuery.GetLength());
      if (SQL_SUCCESS == nRetCode) {
        CHAR buffer[128] = {0};
        SQLLEN iLen = 0;
        short int val = 0;
        long obj_id = 0;

        while (::SQLFetch(hstmt) == SQL_SUCCESS) {
          SQL_INDEX ob;

          if (::SQLGetData(hstmt, 1, SQL_C_CHAR, buffer, 128, &iLen) ==
              SQL_SUCCESS)
            ob.index_name = buffer;

          if (::SQLGetData(hstmt, 2, SQL_C_SHORT, &val, sizeof(short int),
                           &iLen) == SQL_SUCCESS)
            ob.is_unique = (1 == val);

          if (::SQLGetData(hstmt, 3, SQL_C_SHORT, &val, sizeof(short int),
                           &iLen) == SQL_SUCCESS)
            ob.is_primary = (1 == val);

          if (::SQLGetData(hstmt, 4, SQL_C_SHORT, &val, sizeof(short int),
                           &iLen) == SQL_SUCCESS)
            ob.index_type = val;

          if (::SQLGetData(hstmt, 5, SQL_C_CHAR, buffer, 128, &iLen) ==
              SQL_SUCCESS)
            ob.index_desc = buffer;

          if (::SQLGetData(hstmt, 6, SQL_C_LONG, &obj_id, sizeof(long),
                           &iLen) == SQL_SUCCESS)
            ob.object_id = obj_id;

          vectKeys.push_back(ob);
        }
      }
    }
  }

  // collect index’s columns/fields information
  for (auto it = vectKeys.begin(); it != vectKeys.end(); ++it) {
    HSTMT hstmt2 = SQL_NULL_HSTMT;
    if ((nRetCode = ::SQLAllocStmt(pDBdata->m_hdbc, &hstmt2)) == SQL_SUCCESS) {
      if (hstmt2 != SQL_NULL_HSTMT) {
        pDBdata->OnSetOptions(hstmt2);

        CString sSQL;
        SQLLEN iLen = 0;

        sSQL.Format(_T("SELECT AC.Name as ColumnName \
FROM sys.tables as T inner join sys.indexes as I on T.[object_id] = I.[object_id] \
INNER JOIN sys.index_columns as IC on IC.[object_id] = I.[object_id] and IC.[index_id] = I.[index_id] \
INNER JOIN sys.all_columns as AC on IC.[object_id] = AC.[object_id] and IC.[column_id] = AC.[column_id] \
WHERE I.object_id = %ld and T.name = '%s' and I.name = '%s' order by T.name, I.name"),
                    it->object_id, strTable, it->index_name);

        nRetCode =
            ::SQLExecDirect(hstmt2, (UCHAR*)(LPCTSTR)sSQL, sSQL.GetLength());
        if (SQL_SUCCESS == nRetCode) {
          CHAR buffer[128] = {0};
          while (::SQLFetch(hstmt2) == SQL_SUCCESS) {
            if (::SQLGetData(hstmt2, 1, SQL_C_CHAR, buffer, 128, &iLen) ==
                SQL_SUCCESS) {
              it->vectColumns.push_back(buffer);
            }
          }
        }
      }
    }
  }
}

In this way I have complete information about the indexes of my tables.

std::vector vectIndexesSQL;
pFoo->GetIndexesInformation(strSchema, pDBTable->strTblName, vectIndexesSQL, pDataDB);

Conclusion: When the C/C++ API doesn’t give you any hopes don’t forget that SQL saves you.

HTML files generation using XML and XSLT with Microsoft XML DOM API

This short tutorial shows how easy it’s to generate reports in HTML pages using Microsoft XML DOM API together XML and XSLT.

XML (Extensible Markup Language) became a universal standard of encoding data in a format that is both human-readable and machine-readable. It’s widely used in business applications and even Microsoft Office uses it into internal file formats.
XSLT is used for XML documents decoration. Once we have data into a XML files, using the XSLT (Extensible Stylesheet Language Transformations) we can easily generate HTML and xHTML files. XSLT is a W3C recommendation still from 16. November 1999 and in the meantime, it was extended with a new version XSLT v2.0.

XSLT uses XPATH to get the XML’s tags information, complete the predefined temples and transform results into a .html document.
Each decent browser has support for XML and XSLT. All we have to do it’s to link two such files (.xml and .xslt) and once we execute the XML file the browser will generate and render our XHTML content.

// content of XML file

<!--?xml version="1.0" encoding="utf-8"?-->
<!--?xml-stylesheet href="sample_1.xslt" type="text/xsl"?-->

// the rest of the XML file

But in case we are writing non-browser applications the HTML generation becomes a bit complicated in case you are not satisfied with a hard-coded solution and want a flexible solution.

Using the Microsoft’s XML Core Services (MSXML) our job became a piece of cake. We focus once over the HTML generator and later in case we want to change something into our look and content we have to deal only with the .xml and .xslt files.

bool CHTMLGen::Generate(const std::wstring& sXmlFile,
                        const std::wstring& sXsltFile,
                        const std::wstring& sHTMLFile) {
  if (!PathFileExists(sXmlFile.c_str()) || !PathFileExists(sXsltFile.c_str())) {
    return false;
  }

  HRESULT hr;
  CComPtr pXml, pXslt;

  hr = CoCreateInstance(CLSID_DOMDocument, NULL, CLSCTX_INPROC_SERVER,
                        IID_IXMLDOMDocument, (void**)&pXml);
  if (FAILED(hr))
    return false;

  VARIANT_BOOL bOkLoad;
  CComVariant varFile;

  varFile = sXmlFile.c_str();
  pXml->put_async(VARIANT_FALSE);
  hr = pXml->load(varFile, &bOkLoad);
  if (FAILED(hr) && (bOkLoad == VARIANT_FALSE))
    return false;

  hr = CoCreateInstance(CLSID_DOMDocument, NULL, CLSCTX_INPROC_SERVER,
                        IID_IXMLDOMDocument, (void**)&pXslt);
  if (FAILED(hr))
    return false;

  varFile = sXsltFile.c_str();
  pXslt->put_async(VARIANT_FALSE);
  hr = pXslt->load(varFile, &bOkLoad);
  if (FAILED(hr) && (bOkLoad == VARIANT_FALSE))
    return false;

  CComBSTR bsHtmlRes;
  std::wstring sHTMLRes = _T("");
  hr = pXml->transformNode(pXslt, &bsHtmlRes);
  if (SUCCEEDED(hr)) {
    CAtlString sRes(bsHtmlRes);
    if (!_tcsnicmp(sRes, _T(""));
      sHTMLRes += szEnd ? (szEnd + 2) : sRes;
  } else
    sHTMLRes += sRes;
 }

 return (_T("") != sHTMLRes) ? SaveFileContent(sHTMLFile, sHTMLRes) : false;
}

Because of using COM don’t forget proper the calls of CoInitialize() and CoUninitialize().
Here are two samples files generated with the test application using the upper method: sample_1, sample_2.

The combination of XML, XSLT and XPATH offers a very flexible way to generate HTML files. With such an approach even native application does not need to change in case we change the HTMLs look. Within the presented case the hard-coded solutions are avoided and most probably a new recompilation is not needed in case we want to change data content (XML) or the look (XSLT).
In case you want to add sophisticated HTML code (ex. colored, formatted, images, etc.) you need to convert that code into XHTML format before adding data into the .XSLT file.

demo application (3005 downloads)

Some experiences with the last world-wide WordPress brute force attack

As you most probably know, this website uses WordPress. Last Saturday, trying to access the site admin area I was faced with an error generated by too many redirects.
error_redirects
Having other things to do, I ignored it for that moment. Later a friend of mine published on his website some information about an existing world-wide brute force attack over WordPress websites and then I started reading more information about this issue.

My Firefox’s Web Console has thrown such messages:

[13:38:30.162] GET http://my_site.ro/wp-admin/ [HTTP/1.1 302 Moved Temporarily 834ms]
[13:38:30.998] GET http://my_site.ro/wp-admin/ [HTTP/1.1 302 Moved Temporarily 403ms]
[13:38:31.405] GET http://my_site.ro/wp-admin/ [HTTP/1.1 302 Moved Temporarily 580ms]
[13:38:31.990] GET http://my_site.ro/wp-admin/ [HTTP/1.1 302 Moved Temporarily 558ms]
[13:38:32.558] GET http://my_site.ro/wp-admin/ [HTTP/1.1 302 Moved Temporarily 553ms]

Most probably I was also one target for that attack.

Having a strong password and not using admin user the effective website was not affected otherwise than in wp-admin area. I contacted my website hosting provider and after some emails exchange, I was able to login into wp-admin area. First thing I did was to install and activate Limit Login Attempts plugin and the results didn’t expect too much to appear.
Today morning, this plugin sent me an interesting email.
results_of_limiting_login_attempts

So an attempt of brute force attack is cut.

In order to avoid any such unpleased issues it is strongly recommended following few basic steps:

  • Avoid using default users (ex. admin) having high privileges.
  • Use strong passwords that contain special characters also in order avoid dictionary attacks used by brute force methods. You can check if the password it’s strong enough using online free tools such passwordmeter.com or Password Checker.
  • Install and activate a tool such Limit Login Attempts.
  • Enjoy your life. 🙂

SubclassWindow() method issues in projects base on MFC Feature Pack

The Problem
Trying to paint a background image into client area of a MDI application build in VC++ 6.0 to VC++ 2005 IDE it’s not a difficult task.
In case you need, you can find easily good references. For instance, there are two references from Microsoft (KB129471 and KB103786) and one I prefer: a FAQ wrote by a friend of mine.

Unfortunately things are changing radically in case you’re following the same steps in a Visual C++ IDE that has MFC Feature Pack support. If you’re building from the scratch a VC++ 2008/VC++ 2010 a MDI project that has MFC Feature Pack support and you’re trying to apply sub-classing steps, you will have a big surprise in the moment you’re starting your application in debug mode. Effectively your application will crash in the moment you are trying to call SubclassWindow() in CMainFrame::OnCreate().

int CMainFrame::OnCreate(LPCREATESTRUCT lpCreateStruct) {
  if (CMDIFrameWndEx::OnCreate(lpCreateStruct) == -1)
    return -1;

  // ---- code ---

  // BANG! IN VS 2010 or VS 2008 with MFC Feature Pack
  // m_wndMDIClient.SubclassWindow(m_hWndMDIClient);
  // where m_wndMDIClient is an instance of CMDIClientWnd
  // (http://support.microsoft.com/kb/129471)

  // ------ code ----

  return 0;
}

Problem details
Starting with MFC Feature Pack CMDIFrameWndEx is the new CMainFrame’s parent class instead of CMDIFrameWnd and the problem acts inside of Attach() method:
because CWnd::FromHandlePermanent(HWND hWnd) looks up into a permanent handle map and in returns existing CWnd pointer.

Wnd* PASCAL CWnd::FromHandlePermanent(HWND hWnd) {
  CHandleMap* pMap = afxMapHWND();
  CWnd* pWnd = NULL;
  if (pMap != NULL) {
    // only look in the permanent map - does no allocations
    pWnd = (CWnd*)pMap->LookupPermanent(hWnd);
    ASSERT(pWnd == NULL || pWnd->m_hWnd == hWnd);
  }
  return pWnd;
}

CHandleMap is the wrapper that implements the mapping mechanism between the pointers of MFC wrapped classes and the Windows object handles. Internally, this class has to dictionaries (m_permanentMap and m_temporaryMap) implemented as CMapPtrToPtr, m_nHandles – the number of handles, m_nOffset – the offset of handles in the object and it has a m_pClass pointer of CRuntimeClass (a run time class associated with all MFC classes).
In case you’re interest in more details, you can find more information here.

We have a pointer to a CHandleMap instance that is assigned with the returned pointer of a handle map returned by afxMapHWND(). The returned pointer pWnd it’s assigned with the result returned by pMap->LookupPermanent(hWnd). LookupPermanet() effectively search into a the permanent hash map for exiting HANDLEs and in our case it find it.

inline CObject* CHandleMap::LookupPermanent(HANDLE h) {
  return (CObject*)m_permanentMap.GetValueAt((LPVOID)h);
}

where

void* CMapPtrToPtr::GetValueAt(void* key) const {
  // find value (or return NULL -- NULL values not different as a result)

  ENSURE(this);

  if (m_pHashTable == NULL)
    return NULL;

  UINT nHash = HashKey(key) % m_nHashTableSize;

  // see if it exists
  CAssoc* pAssoc;
  for (pAssoc = m_pHashTable[nHash]; pAssoc != NULL; pAssoc = pAssoc->pNext) {
    if (pAssoc->key == key)
      return pAssoc->value;
  }
  return NULL;
}

If the item having nHash key was found into m_pHashTable then the condition if (pAssoc->key == key) is TRUE because the attribute m_hWndMDIClient of CMDIFrameWnd is used yet.
So, effectively what LookupPermanent() has found in m_permanentMap map is m_hWndMDIClient. And because pMap->SetPermanent(m_hWnd = hWndNew, this) is one of the next call into Attach() method those ASSERTs are a must.
Even if those ASSERT() calls from Attach() are available only in debug mode (because of ASSERT() macro behavior) a release build would not save the situation. Soon or later you’ll get conflicts and the application will crash.

Trying to find where this has happened is not so complicated as long as we take in consider our CMainFrame class it’s derived from CMDIFrameWndEx a class that extends CMDIFrameWnd. If we are looking into CMDIFrameWndEx class implementation (AfxMDIClientAreaWnd.cpp) we will see that into this class SubclassWindow() method it’s called jet:

BOOL CMDIFrameWndEx::OnCreateClient(LPCREATESTRUCT lpcs,
                                    CCreateContext* pContext) {
  if (!CMDIFrameWnd::OnCreateClient(lpcs, pContext)) {
    return FALSE;
  }

  if (m_bDoSubclass) {
    m_wndClientArea.SubclassWindow(m_hWndMDIClient);  // this is it!
  }

  return TRUE;
}

Subclassing a CWnd derived instance that has already a mapped HWND item is an error and these ASSERTs try to avoid this from development moment. Having two different CWnd-derived objects with the same HWND is not possible – the only exception is CDC instances that have 2 HWNDs (m_hDC and m_hAttribDC).
Related to my issue, according to Steve Horne from Microsoft, “anything that uses the MFC Feature Pack will be using CMDIFrameWndEx which is a very different beast. It has this feature built it as you’ve found out”.
The worst part is that “If you were able to subclass the Ex client area, you’d probably end up breaking a lot of the FluentUI features.”
The VS 2008 / VS 2010 wizard generates and use a lot of Feature Pack FluendUI items.

A bad solution
An approach might be trying to adapt sub-classing idea directly into CMainFrame class. So, the steps might be:

  • No CMDIClientWnd instance is needed (as in existing tutorials). So no more SubclassWindow() call in CMainFrame::OnCreate().
  • Handle WM_ERASEBKGND, WM_SIZE and WM_PAINT on CMainFrame.
void CMainFrame::OnPaint() {
  CWnd* pWnd =
      CWnd::FromHandle(m_hWndMDIClient);  // returns a MFC object pointer
  // for the given handle if it is valid
  CPaintDC dc(pWnd);

  // the rest of the code
}

CWnd::FromHandle() acquires a pointer to an MFC object pointer from CHandleMap via afxMapHWND().

BOOL CMainFrame::OnEraseBkgnd(CDC* pDC) {
  return FALSE;
}

void CMainFrame::OnSize(UINT nType, int cx, int cy) {
  CMDIFrameWndEx::OnSize(nType, cx, cy);

  Invalidate();
}

At the very first time everything looked nice. But unfortunately I have to admit Steve Horne’s observations. In different situations (most on resizing or moving messages) some of the FluentUI items were not correctly painted (some Ribbon items painting issues – different cases).

So, a better solution is needed.

A good but not perfect solution
In my research, for projects base on MFC Feature Pack, there is no perfect solution for this issue. I mean something similarly with the good solutions that I mentioned in the beginning of this article but acts fine until the first IDE that use MFC Feature Pack.
As we have seen on top trying to subclass a window with an already mapped is not a good idea.
The solution is based on Joseph M. Newcomery’s idea, a well-known book writer and Microsoft Visual C++ MVP. Joe proposes “temporary” remapping only for the case we need – in my case painting actions. For the rest of the action the mapping process inside of framework continues in the classic way. It’s a “gross and ugly” solution but until having a better solution from Microsoft or others I consider it fine for my needs.

  • First step is to define a class CMDIClientWnd derived from CWnd and add WM_PAINT and WM_ERASEBKGND handle methods.
BOOL CMDIClientWnd::OnEraseBkgnd(CDC* pDC) {
  return FALSE;  // let OnPaint() to paint, only
}

void CMDIClientWnd::OnPaint() {
  CPaintDC dc(this);

  // effective painting stuff
}
  • Catch the WM_PAINT message in CMainFrame via PreTranslateMessage() before the message is dispatched for execution and calling our redraw method.
BOOL CMDIClientWnd::OnEraseBkgnd(CDC* pDC) {
  return FALSE;  // let OnPaint() to paint, only
}

void CMDIClientWnd::OnPaint() {
  CPaintDC dc(this);

  // effective painting stuff
}

Here is the RedrawClientArea() public method.

void CMainFrame::RedrawClientArea() {
  CMDIClientWnd wnd_cl;

  wnd_cl.Attach(m_wndClientArea.Detach());
  wnd_cl.Invalidate();
  wnd_cl.UpdateWindow();
  m_wndClientArea.Attach(wnd_cl.Detach());
}

So we create locally an instance of CMDIClientWnd and we attach it internally to ChandleMap::m_permanetMap via Attach(), not before detaching m_wndClientArea (an CMDIClientAreaWnd instance, attribute in CMDIFrameWndEx and as we have seen before it subclass the CMDIFrameWndEx in CMDIFrameWndEx::OnCreateClient()).

The idea is that our CMDIClientWnd instance temporary replace m_wndClientArea instance of CMDIClientAreaWnd right before effective WM_PAINT message is dispatched via PreTranslateMessage().

  • Include your new class header (ex. MDIClientWnd.h) in MainFrm.cpp and call RedrawClientArea() in CMainFrame::OnSize().
void CMainFrame::OnSize(UINT nType, int cx, int cy) {
  CMDIFrameWndEx::OnSize(nType, cx, cy);

  RedrawClientArea();  // repaint on WM_SIZE
}
  • If the child frames window is not tabbed style (when all client area is hidden) and the client area is still visible than we have to call RedrawClientArea() method from WM_MOVE and WM_SIZE handler of CChildFrame and we have to include MainFrm.h into ChildFrame.cpp.
void CChildFrame::OnMove(int x, int y) {
  CMDIChildWndEx::OnMove(x, y);

  CMainFrame* pMainFrame = (CMainFrame*)GetParentFrame();
  ASSERT(pMainFrame);

  pMainFrame->RedrawClientArea();
}

void CChildFrame::OnSize(UINT nType, int cx, int cy) {
  CMDIChildWndEx::OnSize(nType, cx, cy);

  CMainFrame* pMainFrame = (CMainFrame*)GetParentFrame();
  ASSERT(pMainFrame);

  pMainFrame->RedrawClientArea();
}
  • Additionally, in order to make sure the painting message is received by main frame at application’s starting moment and your image is correctly painted from the beginning, please call pMainFrame->Invalidate() after pMainFrame->UpdateWindow() in InitInstance() method of your application class. Otherwise, if your application it’s starting with no opened document (for instance new document), your picture will appear only in the moment a WM_PAINT message is generated in CMainFrame (for instance when you resize your application, select the menu, etc).

A disadvantage of this approach is that the interest message (WM_PAINT) is not handled inside the class of m_wndClientArea, but the good point is that the rest of the messages are left at the correct class of the framework and will work correctly.
Demo application (3862 downloads)

Several C++ singleton implementations

This article offers some insight into singleton design-pattern.
The singleton pattern is a design pattern used to implement the mathematical concept of a singleton, by restricting the instantiation of a class to one object. The GoF book describes the singleton as: “Ensure a class only has one instance, and provide a global point of access to it.
The Singleton design pattern is not as simple as it appears at a first look and this is proven by the abundance of Singleton discussions and implementations. That’s way I’m trying to figure a few implementations, some base on C++ 11 features (smart pointers and locking primitives as mutexs). I am starting from, maybe, the most basic singleton implementation trying to figure different weaknesses and tried to add gradually better implementations.
The basic idea of a singleton class implies using a static private instance, a private constructor and an interface method that returns the static instance.

Version 1
Maybe, the most common and simpler approach looks like this:

class simpleSingleton {
  simpleSingleton();
  static simpleSingleton* _pInstance;

public:
  ~simpleSingleton() {}
  static simpleSingleton* getInstance() {
    if (!_pInstance) {
      _pInstance = new simpleSingleton();
    }
    return _pInstance;
  }
  void demo() {
    std::cout << "simple singleton # next - your code ..." << std::endl;
  }
};

simpleSingleton* simpleSingleton::_pInstance = nullptr;

Unfortunately this approach has many issues. Even if the default constructor is private, because the copy constructor and the assignment operator are not defined as private the compiler generates them and the next calls are valid:

// Version 1
simpleSingleton * p = simpleSingleton::getInstance(); // cache instance pointer p->demo();

// Version 2
simpleSingleton::getInstance()->demo();

simpleSingleton ob2(*p); // copy constructor
ob2.demo();

simpleSingleton ob3 = ob2; // copy constructor
ob2.demo();

So we have to define the copy constructor and the assignment operator having private visibility.

Version 2 – Scott Meyers version
Scott Meyers in his Effective C++ book adds a slightly improved version and in the getInstance() method returns a reference instead of a pointer. So the pointer final deleting problem disappears.
One advantage of this solution is that the function-static object is initialized when the control flow is first passing its definition.

class otherSingleton {
  static otherSingleton* pInstance;

  otherSingleton();

  otherSingleton(const otherSingleton& rs) { pInstance = rs.pInstance; }

  otherSingleton& operator=(const otherSingleton& rs) {
    if (this != &rs) {
      pInstance = rs.pInstance;
    }

    return *this;
  }

  ~otherSingleton();

 public:
  static otherSingleton& getInstance() {
    static otherSingleton theInstance;
    pInstance = &theInstance;

    return *pInstance;
  }

  void demo() {
    std::cout << "other singleton # next - your code ..." << std::endl;
  }
};

otherSingleton * otherSingleton::pInstance = nullptr;

The destructor is private in order to prevent clients that hold a pointer to the Singleton object from deleting it accidentally. So, this time a copy object creation is not allowed:

otherSingleton ob = *p;
ob.demo();


error C2248: otherSingleton::otherSingleton ' : cannot access private member declared in class 'otherSingleton'
error C2248: 'otherSingleton::~otherSingleton' : cannot access private member declared in class 'otherSingleton'

but we can still use:

// Version 1
otherSingleton *p = &otherSingleton::getInstance(); // cache instance pointer p->demo();
// Version 2
otherSingleton::getInstance().demo();

This singleton implementation was not thread-safe until the C++ 11 standard. In C++11 the thread-safety initialization and destruction is enforced in the standard.

If you’re sure that your compiler is 100% C++11 compliant then this approach is thread-safe. If you’re not such sure, please use the approach version 4.

Multi-threaded environment
Both implementations are fine in a single-threaded application but in the multi-threaded world things are not as simple as they look. Raymond Chen explains here why C++ statics are not thread safe by default and this behavior is required by the C++ 99 standard.
The shared global resource and normally it is open for race conditions and threading issues. So, the singleton object is not immune to this issue.
Let’s imagine the next situation in a multithreaded application:

static simpleSingleton* getInstance() {
  if (!pInstance)  // 1
  {
    pInstance = new simpleSingleton();  // 2
  }

  return pInstance;  // 3
}

At the very first access a thread call getInstance() and pInstance is null. The thread reaches the second line (2) and is ready to invoke the new operator. It might just happen that the OS scheduler unwittingly interrupts the first thread at this point and passes control to the other thread.
That thread follows the same steps: calls the new operator, assigns pInstance in place, and gets away with it.
After that the first thread resumes, it continues the execution of line 2, so it reassigns pInstance and gets away with it, too.
So now we have two singleton objects instead of one, and one of them will leak for sure. Each thread holds a distinct instance.

An improvement to this situation might be a thread locking mechanism and we have it in the new C++ standard C++ 11. So we don’t need using POSIX or OS threading stuff and now locking getInstance() from Meyers’s implementation looks like:

static otherSingleton& getInstance() {
  std::lock_guard lock(_mutex);
  static otherSingleton theInstance;
  pInstance = &theInstance;
  return *pInstance;
}

The constructor of class std::lock_guard (C++11) locks the mutex, and its destructor unlocks the mutex. While _mutex is locked, other threads that try to lock the same mutex are blocked.
But in this implementation we’re paying for synchronization overhead for each getInstance() call and this is not what we need. Each access of the singleton requires the acquisition of a lock, but in reality we need a lock only when initializing pInstance. If pInstance is called n times during the course of a program run, we need the lock only for the first time.
Writing a C++ singleton 100% thread safe implementation it’s not as simple as it appears as long as for many years C++ had no threading standard support. In order to implement a thread-safe singleton we have to apply the double-checked locking (DCLP) pattern.
The pattern consists of checking before entering the synchronized code, and then check the condition again.
So the first singleton implementation would be rewritten using a temporary object:

static simpleSingleton* getInstance() {
  if (!pInstance) {
    std::lock_guard lock(_mutex);

    if (!pInstance) {
      simpleSingleton* temp = new simpleSingleton;
      pInstance = temp;
    }
  }

  return pInstance;
}

This pattern involves testing pInstance for nullness before trying to acquire a lock and only if the test succeeds the lock is acquired and after that, the test is performed again. The second test is needed for avoiding race conditions in case other thread happens to initialize pInstance between the time pInstance was tested and the time the lock was acquired.
Theoretically, this pattern is correct, but in practice is not always true, especially in multiprocessor environments.
Due to this rearranging of writes, the memory as seen by one processor at a time might look as if the operations are not performed in the correct order by another processor. In our case, the assignment to pInstance performed by a processor might occur before the Singleton object has been fully initialized.
After the first call of getInstance() the implementation with pointers (non-smart) needs pointer to that instance in order to avoid memory leaks.

Version 3 – Singleton with smart pointers
Until C++ 11, the C++ standard didn’t have a threading model and developers needed to use external threading APIs (POSIX or OS dependent primitives). But finally C++ 11 standard has threading support.
Unfortunately, the first C++ new standard implementation in Visual C++ 2010 is incomplete and threading support is available only starting with beta version of VS 2011 or the VS 2012 release preview version.

class smartSingleton {
 private:
  static std::mutex _mutex;

  smartSingleton();
  smartSingleton(const smartSingleton& rs);
  smartSingleton& operator=(const smartSingleton& rs);

 public:
  ~smartSingleton();

  static std::shared_ptr& getInstance() {
    static std::shared_ptr instance = nullptr;

    if (!instance) {
      std::lock_guard lock(_mutex);

      if (!instance) {
        instance.reset(new smartSingleton());
      }
    }

    return instance;
  }

  void demo() {
    std::cout << "smart pointers # next - your code ..." << std::endl;
  }
};

As we know, in C++ by default the class members are private. So, our default constructor is private too. I added here in order to avoid misunderstanding and explicitly adding to public / protected.
Finally, feel free to use your special instance (singleton):

// Version 1
std::shared_ptr p = smartSingleton::getInstance(); // cache instance pointer
p->demo();

// Version 2
std::weak_ptr pw = smartSingleton::getInstance(); // cache instance pointer
pw.lock()->demo();

// Version 3
smartSingleton::getInstance()->demo();

And no memory leaks emotion… 🙂
Multiple threads can simultaneously read and write different std::shared_ptr objects, even when the objects are copies that share ownership.
But even this implementation using double checking pattern but is not optimal to double check each time.


Version 4 – Thread safe singleton C++ 11
To have a thread safe implementation we need to make sure that the class single instance is locked and created only once in a multi-threaded environment.
Fortunately, C++ 11 comes in our help with two new entities: std::call_once and std::once_flag. Using them with a standard compiler we have the guaranty that our singleton is thread safely and no memory leak.
Invocations of std::call_once on the same std::once_flag object are serialized.
Instances of std::once_flag are used with std::call_once to ensure that a particular function is called exactly once, even if multiple threads invoke the call concurrently.
Instances of std::once_flag are neither CopyConstructible, CopyAssignable, MoveConstructible nor MoveAssignable.

Here it is my proposal for a singleton thread safe implementation in C++ 11:

class safeSingleton {
  static std::shared_ptr<safeSingleton> instance_;
  static std::once_flag only_one;

  safeSingleton(int id) {
    std::cout << "safeSingleton::Singleton()" << id << std::endl;
  }

  safeSingleton(const safeSingleton& rs) { instance_ = rs.instance_; }

  safeSingleton& operator=(const safeSingleton& rs) {
    if (this != &rs) {
      instance_ = rs.instance_;
    }

    return *this;
  }

 public:
  ~safeSingleton() { std::cout << "Singleton::~Singleton" << std::endl; }

  static safeSingleton& getInstance(int id) {
    std::call_once(
        safeSingleton::only_one,
        [](int idx) {
          safeSingleton::instance_.reset(new safeSingleton(idx));

          std::cout << "safeSingleton::create_singleton_() | thread id " + idx
                    << std::endl;
        },
        id);

    return *safeSingleton::instance_;
  }

  void demo(int id) {
    std::cout << "demo stuff from thread id " << id << std::endl;
  }
};

std::once_flag safeSingleton::only_one;
std::shared_ptr<safeSingleton> safeSingleton::instance_ = nullptr;

The parameter to getInstance() was added for demo reasons only and should be passed to a new proper constructor. As you can see, I am using a lambda instead normal method.
This is how I tested my safeSingleton and smartSingleton classes.

std::vector v;
int num = 20;

for (int n = 0; n < num; ++n) {
  v.push_back(
      std::thread([](int id) { safeSingleton::getInstance(id).demo(id); }, n));
}

std::for_each(v.begin(), v.end(), std::mem_fn(&std::thread::join));

// Version 1
std::shared_ptr<smartSingleton> p = smartSingleton::getInstance(
    1);  // cache instance pointer p->demo("demo 1");

// Version 2
std::weak_ptr<smartSingleton> pw = smartSingleton::getInstance(2);  // cache instance pointer
pw.lock()->demo(2);

// Version 3
smartSingleton::getInstance(3)->demo(3);

So I create 20 threads and I launch them in parallel (std::thread::join) and each thread accesses getInstance() (with a demo id parameter). Only one of the threads that is trying to create the instance succeeds.
Additionally, if you’re using a C++11 100% compiler you could also delete the copy constructor and assignment operator. This will allow you to obtain an error while trying to use such deleted members.

Other comments
I tested this implementation on a machine with Intel i5 processor (4 cores). If you see some concurrent issues in this implementation please fell free to share here. I am open to other good implementations, too.
An alternative to this approach is creating the singleton instance of a class in the main thread and pass it to the objects which require it. In case we have many singleton objects this approach is not so nice because the objects discrepancies can be bundled into a single ‘Context’ object which is then passed around where necessary.

Update: According to Boris’s observation I removed std::mutex instance from safeSingleton class. This is not necessary anymore because std::call_once is enough to have thread safe behavior for this class.

Update2: According to Ervin and Remus’s observation, in order to make things clear I simplified the implementation version 3 and this is not using std::weak_ptr anymore.

References:
just::thread – Anthony Williams – Just Software Solutions Ltd
C++ and the Perils of Double-Checked Locking by Scott Meyers and Andrei Alexandrescu
Modern C++ Design: Generic Programming and Design Patterns Applied by Andrei Alexandrescu

Ubuntu – How to fix update errors for a server behind of a proxy

If you’re behind of a proxy server and the IP address has changed (or proxy it’s a new node in your network topology) then probably when you’re trying to update your binaries then you’ll get a list of errors.

W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/oneiric-security/multiverse/i18n/Translation-en_US Unable to connect to 192.168.120.240:3128:
W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/oneiric-security/multiverse/i18n/Translation-en Unable to connect to 192.168.120.240:3128: 
W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/oneiric-security/restricted/i18n/Translation-en_US Unable to connect to 192.168.120.240:3128: 
W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/oneiric-security/restricted/i18n/Translation-en Unable to connect to 192.168.120.240:3128:

This happens because you’re unable to connect to Ubuntu mirrors. In order to fix this issue, you have to edit apt.conf file settings.
– Check the proxy settings in apt.conf file:

my-user@my-server:/#sudo -s
root@my-server:/# view /etc/apt/apt.conf

– Update the new proxy IP and port (ex. http://192.168.230.99:3128):

root@my-server:/# vi /etc/apt/apt.conf
Acquire::http::Proxy "http://192.168.230.99:3128";

If you need user and password for logging to the Proxy server this can be achieved in most cases by using the following layout:

Acquire::http::Proxy "http://username:password@your_proxy_address:proxy_port";

– Finally, you may enjoy running the update commands:

root@my-server:/# apt-get update
root@my-server:/# apt-get upgrade

Adventures with _chkstk

Called by the compiler when you have more than one page of local variables in your function.
_chkstk Routine is a helper routine for the C compiler. For x86 compilers, _chkstk Routine is called when the local variables exceed 4K bytes; for x64 compilers, it is 8K.

That’s all that you get from _chkstk()’s msdn web page. Nothing more…

Overview
A process starts with a fixed stack space. The top of a stack is pointed to by the ESP register (Extended Stack Pointer) and this is a decrementing pointer. Every function calls results in a stack created for the function inside this Process Stack. Every thread function has its own stack. The stack is a downward growing array. When a function starts, the default stack reservation size is 1 MB.
This is contrasting with the heap’s size, whether theoretically increases to a limit of 4 GB on 32bits OS. See more information here.

Every thread under Windows gets its own block of contiguous memory, and while function calls are made, the stack pointer is increasing and decreasing. In contrast, a different thread within the same process might get a different block of contiguous memory – its own stack. When a context switch occurs, the current thread’s ESP (along with the IP and other registers) are saved in the thread’s context structure, and restored when the thread is activated the next time.
To specify a different default stack reservation size for all threads and fibers, use the STACKSIZE statement in the module definition (.def) file. To change the initially committed stack space, use the dwStackSize parameter of the CreateThread, CreateRemoteThread, or CreateFiber function.
Most stack problems occur in overflows of existing stacks, as their sizes are fixed and they cannot be expanded.

_chkstk() increases the stack when needed by committing some of the pages previously reserved for the stack. If there is no more physical memory available for committed pages, _chkstk fails. When you enter a function (VC++ with the stack checking enabled), it will call the _chkstk located in CHKSTK.ASM. This function does a stack page probing and causes the necessary pages of memory to be allocated using the guard page scheme, if possible. In this function, it is stated that when it encounters _XCPT_GUARD_PAGE_VIOLATION, the OS will attempt to allocate another guarded page and if it encounters _XCPT_UNABLE_TO_GROW_STACK then it’s a stack overflow error. When _XCPT_UNABLE_TO_GROW_STACK is encountered, the stack is not yet set up properly, that is why, that it will not call the catch because calling it will use invalid stack variables which will again cause another exception.

Case – Too many or too big variables on stack
As I said on top, the function stack size is 1 MB. If you miss that and you’re trying to define and use internally an array like this:

int arr[4000][200];

When you’ll compile with VC++ compiler in debug mode, you will have a big surprise: the application is crashing on _chkstk() at the moment the _chkstk() tries to create new memory page on stack and fails.
The output window shows next message:
First-chance exception at 0x004116e7 in testApp.exe: 0xC00000FD: Stack overflow.
Unhandled exception at 0x004116e7 in testApp.exe: 0xC00000FD: Stack overflow.

This happens because the 1MB limit is overloaded even on a win32 OS: 4000*200*4 = 3.2MB (approx.).
Same story if you define many local variables and their stack usage overloads the 1MB limit. Off-course the thread stack size can be changed but think once again if it is really needed to do that.
If you really need this big array, then the best solution to avoid this crash is using the heap.

int **arr = new int*[4000];
for(int i=0; i<4000; ++i) {
  arr[i] = new int[200];
}

// and finally don't forget to delete
for(int i=0;i<4000; ++i) {
  delete[] arr[i];
}
delete[] arr;

Case – Recursive functions
If you have an infinite recursion, then you will gate the same stack overflow error and the application crashes in _chkstk.asm. Recursive function is not the subject of this article so I don’t go in deep… Here it is a good example of what happens with recursive functions.
The solution is to avoid using recursive functions as much as possible and try to implement an iterative function.

Case – A stack corruption
I have started looking over _chkstk() function at the moment when I got few bugs with crashes with some similarly details. I had to analyze some .dump files and solve a few bugs that contained a call stack with _chkstk() on top.
Most of the .dump files call stack contained a second similarly thing: the call of a thread function (so called ThreadFoo()) that was running in a thread’s pool.
At that moment I started to research why _chkstk() fails and my first track was to debug the stack overflows. I followed a MSDN debugging tutorial and unfortunately I didn’t find something strange. I checked if the local stack variables are not so big in order to fill the ThreadFoo() function’s stack, and it did not.
Then a new study of ThreadFoo() function has followed in order to detect the internal functions calls that can fail in some circumstances. I stopped to some trace function calls and I studied deeply. Those trace functions were defined in an external debug class and each time when a new trace file was added it used an internal buffer (TCHAR szBuff[2048] = _T(“”);).
The writing of this buffer was done using: swprintf(). As we know, this function is unsafe and is not recommended to use. As long as the content of these trace lines was dynamically build (in some cases those line may contain even dynamically build SQL queries that failed) then the length of these trace lines could be higher than 2048 bytes and then guess what: a stack corruption appears! UPS! The stack pointer will be corrupted (the classic stack overflow case).

So I have implemented and used the next macros:

#if defined(UNICODE) || defined(_UNICODE)
#define usprintf(x, ...) \
   _snwprintf(x, _countof(x)-1, ##__VA_ARGS__); \
   x[_countof(x)-1] = 0
#else
#define usprintf(x, ...) \
   _snprintf(x, _countof(x)-1, ##__VA_ARGS__); \
   x[_countof(x)-1] = 0
#endif

Now, if we’re using the safe macro we will have no issues.

int x = 23;
TCHAR szMsg[2048] = {0};
usprintf(szMsg, L"Error: %d", x);

A safety alternative way to that buffer was the heap using but the heap access is not fast as the stack access, so I preferred this approach (in a business application every millisecond’s matters for the log system).
After that fixed, I met no other stack corruptions in ThreadFoo() and other code areas.

Even if the top of the call stack was _chkstk() this was not the function that failed. The error appeared because of that stack corruption and _chkstk() has just detected.

Conclusion
If your code produces a stack overflow, then you have to rethink your design in right away:

  • If you see _chkstk() on the top of call stack, check if you have no stack corruptions – stack overflow.
  • Don’t try to manipulate the stack by yourself. The default 1MB stack size is basically enough
  • Allocate dynamically if you’re using big arrays
  • If you have recursive functions producing a stack overflow, re-write them using loops (a tip: it is a proven fact that any recursive functions can be programmed non-recursive)

References
Set stack size
Thread Stack Size
_chkstk Routine
Stack (data structure)
Debugging a Stack Overflow – tutorial
Visual C++ apps crashing in _chkstk() under load
Optimization Note (C++) 1: push, pop, call _chkstk
What is Recursion?

Flexible changes for product version properties – Visual C++ binaries

Manually editing of binary files version in the resource editor of Visual Studio IDE is not a viable solution. If we have dozens of projects in our solution, then for each kit building we should need manual resources file edit. Otherwise, we can use a special tool that does this thing for us.
Unfortunately this approach is not the most flexible and could fail.

For our flexible binaries properties changes and in order to avoid manual edit for each rebuild we can create and include a header file (version.h) that contains some constants of product version and file version of our project (.rc files).

#define PRODUCT_VERSION 4.3.2.198
#define PRODUCT_VERSION_STR "4.3.2.198"

Then, for each .rc file wherever we have FileVersion and ProductVersion we have to use this constants.
When we will build a new kit, we have to change only these constants and then to start the kit building process. Everything is fine until we add new controls in our projects resource files. Then, because of Visual Studio IDE automation we can get an unlikely surprise: the FileVersion and the ProductVersion properties could be reset to 1.0.0.0.

In order to avoid this issue and edit the version only in a single place I propose the following workaround:

  • Create a version.h header file that have to contain these constants only (as on top). I should recommend to create it into the root solution in order of being easy included in all the projects.
  • Include this file in the project you need to use.
  • Use a text editor (ex. notepad.exe) and include next code section at the end of .rc2 resource file of your project (res\your_project.rc2) – this section contains the include version.h file section, too.
// Neutral resources
#if !defined(AFX_RESOURCE_DLL) || defined(AFX_TARG_NEU)
#ifdef _WIN32
LANGUAGE LANG_NEUTRAL, SUBLANG_NEUTRAL
#pragma code_page(1252)
#endif //_WIN32
/////////////////////////////////////////////////////////////////////////////
// Versio
//
#include "..\version.h"
VS_VERSION_INFO VERSIONINFO
FILEVERSION PRODUCT_VERSION
PRODUCTVERSION PRODUCT_VERSION
FILEFLAGSMASK 0x3fL
#ifdef _DEBUG
FILEFLAGS 0x1L
#else
FILEFLAGS 0x0L
#endif
FILEOS 0x4L
FILETYPE 0x1L
FILESUBTYPE 0x0
BEGIN
BLOCK "StringFileInfo"
BEGIN
BLOCK "040904e4"
BEGIN
VALUE "CompanyName", "TODO: "
VALUE "FileDescription", "TODO: "
VALUE "FileVersion", PRODUCT_VERSION_STR
VALUE "InternalName", "testResources.exe"
VALUE "LegalCopyright", "TODO: (c) . All rights reserved."
VALUE "OriginalFilename", "your_project.exe"
VALUE "ProductName", "TODO: "
VALUE "ProductVersion", PRODUCT_VERSION_STR
END
END
BLOCK "VarFileInfo"
BEGIN
VALUE "Translation", 0x409, 1252
END
END
#endif // Neutral resources
/////////////////////////////////////////////////////////////////////////////
  • Edit “040904e4” block code with same data as if we should edit in a resources editor and use the version.h‘s file defined constants. As you can see in my example, for the FileVersion and ProductVersion properties I use my version.h constants. These properties will not be edited anymore.
  • Delete “// Version” section from default resource file your_project.rc (including comments – recommended).
  • Insert next lines into your_project.rc file after “3 TEXTINCLUDE BEGIN” and before “#define _AFX_NO_SPLITTER_RESOURCES\r\n“:
\r\n""#include ""res\\your_project.rc2""\r\n""\0"
3 TEXTINCLUDE
BEGIN
"\r\n"
"#include ""res\\your_project.rc2""\r\n"
"\0"
"#define _AFX_NO_SPLITTER_RESOURCES\r\n"
"#define _AFX_NO_OLE_RESOURCES\r\n"
"#define _AFX_NO_TRACKER_RESOURCES\r\n"
"#define _AFX_NO_PROPERTY_RESOURCES\r\n"
"\r\n"
"#if !defined(AFX_RESOURCE_DLL) || defined(AFX_TARG_ENU)\r\n"
"LANGUAGE 9, 1\r\n"
"#pragma code_page(1252)\r\n"
"#include ""res\\your_project.rc2"" // non-Microsoft Visual C++ edited resources\r\n"
"#include ""afxres.rc"" // Standard components\r\n"
"#endif\r\n"
"\0"
END

Don’t forget to edit .rc2 file name with the right file name of your project.

  • In your_project.rc file the section “// Generated from the TEXTINCLUDE 3 resource.” have to contain only next declaration:

#include "res\your_project.rc2"

The rest of the section’s lines have to be deleted.

  • We save both resources files: your_project.rc and your_project.rc2.
  • Rebuild the project and check the new generated binary properties. In the FileVersion we will have the major version (in my case 4.0.0.0) and in ProductVersion we have the current build version (4.3.2.198).

Demo application - AutoProductVersion (2147 downloads)