Versionable Object’s Serialization using MFC in non Document View applications

Most existing applications operate with data that must be stored and loaded in different times and different locations. The data is stored in text or binary files with a well defined format.
The Problem
Initially, in the first version 1.0, an application operates with data structures that can be stored and loaded. But, next version (2.0) these data structures suffers changes. Some structure’s attributes are added and other could be removed. These things change files format and structure when a new file version is saved.
Question: What happens when you are using an application version 2.0 and you are trying to load old files format (version 1.0)?
Answer: Most cases should cause incompatibility troubles between the current new application and files in old format. This could throw exceptions and the application could have undefined behavior.
That’s why the application must be written in order to be able to open both file versions.
Solution
In order to solve the compatibility issue exist many solutions, more or less professional. The recommended solution is the serialization.
The serialization is a write / read object process to/from a persistent storage. Serialization it’s a good choice in order to maintain a good data structure. Many different frameworks offer serialization support. One of these is Microsoft Foundation Classes – MFC.

If we want to use MFC serialization support, we can use a CArchive instance. This object, combined with a CFile instance provides a strong mechanism for objects serialization.

Because, in different applications version, the file suffers significant structure changes we have to use MFC’s serialization concept called Versionable Schema.
Versionable Schema means the using of CArchive class methods GetObjectSchema() and SetObjectSchema() and a constant VERSIONABLE_SCHEMA (that you can find it in afx.h file and has 0x80000000 value) combined with a OR LOGIC operator and the last application version number as a parameter of IMPLEMENT_SERIAL macro.
The GetObjectSchema() method is used in order to detect stored objects version from a file that is loaded in our application. The complement of this method, SetObjectSchema() method, allow us to save the objects version.
Different by the C++ I/O standard streams, the CArchive class is special designed only for objects serialization in binary files.

In order to serialize a class’s objects we have to follow next steps:
1. The class that we want to serialize has to be derived from the abstract class CObject (or other classes derived from CObject).
2. Overwrite CObject’s Serialize() method.
3. Use DECLARE_SERIAL macro in your class declaration.
4. The serializable class has to have a default constructor, without arguments.
5. Use IMPLEMENT_SERIAL macro in the implementation file of serializable class.

More information you can find it here and in links of this page.

But, from these steps until to a complete serialization and versionable application there are few significant steps to follow.
Next, I will present to you a Dialog base sample application that supports serialization and is versionable.

    Sample application – SerAddressBook

Next, I will present to you how you can create an address book application (based on a MFC Dialog application architecture).

Suppose that initially our client requested us an address book that contaions: name, prename, address and phone number. But, once with the mobile phone and Internet area extensions our client needs two new fields for mobile phone number and for email address.
File versions structure

Our application with a file version 2 looks like this:
Application window

Because this is a demo application I kept on my window the possibility to save both version, using two radio buttons.
A good application design helps us if we have new requirements and we have to change the application structure. The code changes have to be done without too many code interactions. Ideally, with add code only.
That’s why, my application classes design looks like this:
Classes Hierarchy

Although Contact class and CAddressBook class are serializable, the objects serialization is implemented into Contact class.

    Contact class

From Contact’s interface class you can observe:
• I derived this class from the abstract CObject;
DECLARE_SERIAL, macro calling;
Serialize()‘s method declaration in order to overwrite the parent class;
• Our class attributes.

[cpp]class Contact : public CObject
{
public:
DECLARE_SERIAL(Contact);

Contact();
Contact(const Contact& rhs);
Contact(const CString& strFirstName, const CString& strLastName, const CString& strAddress, const CString& strPhone,
// these two must have a default value because they are not used in the first version
const CString& strMobilePhone = _T(“”), const CString& strEmail = _T(“”));
Contact& operator=(const Contact& rhs);

virtual ~Contact();

void Serialize( CArchive& ar );

CString GetFirstName() const {return m_strFirstName;}
CString GetLastName() const {return m_strLastName;}
CString GetAddress() const {return m_strAddress;}
CString GetPhoneNumber() const {return m_strPhone;}
CString GetMobileNumber() const {return m_strMobilePhone;}
CString GetEmail() const {return m_strEmail;}

void SetFirstName(const CString& name) {m_strFirstName = name;}
void SetLastName(const CString& name) {m_strLastName = name;}
void SetAddress(const CString& addr) {m_strAddress = addr;}
void SetPhoneNumber(const CString& phone) {m_strPhone = phone;}
void SetMobileNumber(const CString& mobile) {m_strMobilePhone = mobile;}
void SetEmail(const CString& email) {m_strEmail = email;}

static void SetCurrentFileVersion(int nCV) { CURRENT_VERSION = nCV; }
static int GetCurrentFileVersion() { return CURRENT_VERSION; }

private:
static int CURRENT_VERSION;
CString m_strFirstName;
CString m_strLastName;
CString m_strAddress;
CString m_strPhone;
CString m_strMobilePhone;
CString m_strEmail;
};

typedef CList ContactList;
[/cpp]

The last line represents an “alias” definition for a MFC list definition, used in order to store displayed data. This list is using for Contact object administration.
In the implementation file we declare the IMPLEMENT_SERIAL macro and we are initializing the static variable with our current application version.

[cpp]IMPLEMENT_SERIAL( Contact, CObject, VERSIONABLE_SCHEMA | 2);
int Contact::CURRENT_VERSION = 2;[/cpp]

Into this declaration you can observe the VERSIOABLE_SCHEMA constant combined on OR logic with 2 (my demo application last version). This, the third macro argument is essential for objects versioning, combined with CArchive::GetObjectSchema() and CArchive::SetObjectSchema().
More details about this constant and it using process or about these methods you can find here.
The implementation of Contacte::Serialize() looks like this:
[cpp]
void Contact::Serialize( CArchive& ar )
{
if (ar.IsStoring())
{
CRuntimeClass* pruntime = Contact::GetRuntimeClass();
int oldnr = pruntime->m_wSchema;
pruntime->m_wSchema = CURRENT_VERSION;

ar.SerializeClass(pruntime);

switch (CURRENT_VERSION)
{
case 1:
ar << m_strFirstName << m_strLastName << m_strAddress << m_strPhone; break; case 2: ar << m_strFirstName << m_strLastName << m_strAddress << m_strPhone << m_strMobilePhone << m_strEmail; break; default: // unknown version for this object AfxMessageBox(_T("Unknown file version."), MB_ICONSTOP); break; } pruntime->m_wSchema = oldnr;
}
else // loading code
{
ar.SerializeClass(RUNTIME_CLASS(Contact));

UINT nVersion = ar.GetObjectSchema();

switch (nVersion)
{
case 1:
ar >> m_strFirstName >> m_strLastName >> m_strAddress >> m_strPhone;
m_strMobilePhone = _T(“”);
m_strEmail = _T(“”);
break;

case 2:
ar >> m_strFirstName >> m_strLastName >> m_strAddress >> m_strPhone >> m_strMobilePhone >> m_strEmail;
break;

default:
// unknown version for this object
AfxThrowArchiveException(CArchiveException::badSchema);
break;
}
}
}[/cpp]

If CArchive constructor sets store-load flag on CArchive::store (save to file) then the code flow will follow true if’s block and objects data is sending to archive and stored in the file (including file version, too).
When we want to open an existing file, our CArchive’s constructor receives CArcuhive::load flag and enter to else if’s block of Serialize(). Is extracting file version, and after that is loading all Contact objects.

    CAddressBook class

CAddressBook class make the link between interface dialog class (CSerAddressBookDlg) and the serialized class Contact. This class contains a Contact objects list. CAddressBook class is administrating this contact list and is realizing load/store object.

The interface of this class is looking like this:
[cpp]class CAddressBook : public CObject
{
DECLARE_SERIAL(CAddressBook);

ContactList m_cContactsList;

public:
CAddressBook();
virtual ~CAddressBook();

void Serialize( CArchive& ar );

bool AddContact(const Contact& contact);
bool RemoveContact(const CString& firstname,
const CString& lastname);
POSITION FindContact(const CString& firstname) const;
bool FindContact(const CString& firstname, Contact& contact) const;

const ContactList& GetContacts() const {return m_cContactsList;}

void SetFileVersion(int nFV) { m_uiFileVersion = nFV; }
int GetFileVersion() { return m_uiFileVersion; }

private:
int m_uiFileVersion;
};[/cpp]

Into this declaration you can see the existence of a contact list instance (m_cContactsList). This class contains add, update or remove contact methods.

Because our class has to be a serialized method we have to overwrite Serialize() method, the method has to me used by the client class ( in our case the interface class – CSerAddressBookDlg).
[cpp]
void CAddressBook::Serialize( CArchive& ar )
{
ar.SerializeClass(RUNTIME_CLASS(CAddressBook));

// storing
if (ar.IsStoring())
{
ar << m_uiFileVersion; // write the number of contacts ar << (int)m_cContactsList.GetCount(); Contact::SetCurrentFileVersion(m_uiFileVersion); // write all the contacts POSITION pos = m_cContactsList.GetHeadPosition(); while(pos != NULL) { Contact contact = m_cContactsList.GetNext(pos); contact.Serialize(ar); } } else // loading { ar >> m_uiFileVersion;

m_cContactsList.RemoveAll();

int count = 0;
ar >> count;

// read the number of contacts
for(INT_PTR i = 0; i < count; ++i) { Contact contact; contact.Serialize(ar); m_cContactsList.AddTail(contact); } } } [/cpp] Because CArchive class doesn’t provides any method or attribute in order to obtain the objects (I’m counting when I’m loading or storing), I decided to save the objects count into my files. That’s why, if I’m storing, I call next line:
[cpp]ar << m_cContactsList.GetCount();[/cpp] Same story, for file version, before starting Contact objects serialization:
[cpp]ar << m_uiFileVersion;[/cpp] Then, into a while loop I’m iterating over the contact list. I am serializing the data and I’m storing to my new file. If I load a file from disk (else branch) then I follow next steps:
• I’m cleaning contact list;
• Get the objects count;
• Get file version and serialize all objects for load;
• Add all data to my Contact object list;

    CSerAddressBookDlg – The application interface class

Once we have implemented this serialization mechanism the using of this one into our application became very easy.

For instance, when the user wants to save into a file all he’s new data, he will call next method:
[cpp]
void CSerAddressBookDlg::SaveDataContentToFile(CString strSaveFile)
{
CFile wFile(strSaveFile, CFile::modeCreate | CFile::modeWrite);

// Create a storing archive
CArchive arStore(&wFile, CArchive::store);
m_c_AddressBook.Serialize(arStore);

// Close the storing archive
arStore.Close();

PopulateList();
}
[/cpp]

As you can see, I have a CFile object that I’m using it, combined with a CArchive instance, for data storing to a file. Although my local CArchive instance receive as a first parameter the address of the file handler and the store flag CArchive::store.
Next I call CAddressBook::Serialize() method and I’m closing the store operation.

Loading file method, based on my Contact serialization mechanism looks like this:
[cpp]
void CSerAddressBookDlg::LoadDataContentFromFile(CString strLoadedFile)
{
CFile rFile(strLoadedFile, CFile::modeRead);

// Create a loading archive
CArchive arLoad(&rFile, CArchive::load);
m_c_AddressBook.Serialize(arLoad);

// Close the loading archive
arLoad.Close();

switch (m_c_AddressBook.GetFileVersion())
{
case 1:
((CButton*)GetDlgItem(IDC_RADIO_VERSION_1))->SetCheck(BST_CHECKED);
((CButton*)GetDlgItem(IDC_RADIO_VERSION_2))->SetCheck(BST_UNCHECKED);
OnBnClickedRadioVersion1();
break;
case 2:
((CButton*)GetDlgItem(IDC_RADIO_VERSION_1))->SetCheck(BST_UNCHECKED);
((CButton*)GetDlgItem(IDC_RADIO_VERSION_2))->SetCheck(BST_CHECKED);
OnBnClickedRadioVersion2();
break;
default:
break;
}

// repopulate the list
PopulateList();
}
[/cpp]

As you can see, I am creating a local CFile object, needed for reading operation. Although, I’m creating a local CArchive instance that received as constructor parameter the file handler address with CArchive::load flag.
Then, I’m calling CAddressBook::Serialize() method. Is entering on else branch and finally we are disconnected the object from file.
The last line contains PopulateList() call and is my populate list method. It populates my list control (a CListCtrl instance) with the file loaded data in order to display it into our dialog.

[cpp]
void CSerAddressBookDlg::PopulateList()
{
// delete all current members
m_cList.DeleteAllItems();

// get a reference to the contacts list
const ContactList& contacts = m_c_AddressBook.GetContacts();

// iterate over all contacts add add them to the list
int nCurrentItem = 0;
POSITION pos = contacts.GetHeadPosition();
while(pos != NULL)
{
const Contact contact = contacts.GetNext(pos);

nCurrentItem = m_cList.InsertItem(nCurrentItem, contact.GetFirstName());
m_cList.SetItemText(nCurrentItem, 1, contact.GetLastName());
m_cList.SetItemText(nCurrentItem, 2, contact.GetAddress());
m_cList.SetItemText(nCurrentItem, 3, contact.GetPhoneNumber());

switch(m_c_AddressBook.GetFileVersion())
{
case 1:
break;
case 2:
m_cList.SetItemText(nCurrentItem, 4, contact.GetMobileNumber());
m_cList.SetItemText(nCurrentItem, 5, contact.GetEmail());
break;
}
}
}
[/cpp]

Conclusions:
The MFC’s Document View architecture offers complete serialization support. Each MDI/SDI application contains default serialization support. My demo solution presented is an adapted serialization version for dialog base applications.

Download demo application: SerAddressBook (Visual C++ 2005 project)

Progress database operations

Preliminary remarks

Application path (Sun Solaris Unix OS): /myApp/myapp111a/
Database location: /myApp/db/test_db

Usually, our workstations have Windows OS and we need to connect to Solaris Unix OS on SSH. That’s why we are using Putty application.
User: root
Password: xxxxxxxxxx

The Samba daemon must be available and activated (in order to support Windows share and map drive).
In this sample, the database is in /myApp/db/test_db path and it’s called, total. Basically, database files name doesn’t change, only the folder name (instead of test_db). In my example, the application port for this database it’s 2540 (might be changed).

Creating empty database and access rights
– Go to the folder that contains the database and create the folder tat contains the new database:
[bash]# cd /myApp/db/
# mkdir test_db/
# cd test_db/[/bash]

– Create the new database called total:
[bash]# /progress/dlc/bin/prodb total /progress/dlc/emty8[/bash]

– Rights assignment for database folder (test_db):
[bash]# cd ..
# chown -R myApp:staff test_db/
# cd test_db/
# chmod 644 ./*[/bash]

– Additionally, we can check database permissions:
[bash]# ls -al[/bash]

Database starting
– Check the open ports, especially database opened ports
[bash]# ps -Af | grep mpr [/bash]
– Start the database using the established port, but an unused port value:
[bash]# /progress/dlc/bin/proserve /myApp/baze/test_db/total -S 2540 -N TCP -L 300000[/bash]

Loading database content
The database and dumps loading process it’s realizing using Database Administration tool, available in Progress’s Windows suite. The suite path it’s: START -> Programs ->PROGRESS -> Data Administration.

Launch this application and select Database menu, Connect option. It will appear Connect Database window and we will complete it as in next image. After that we select OK button.

Connect Database window

From this capture you can observe the database name, the network protocol (TCP), Progress server’s IP (192.168.42.10) and our opened database port (2540).
Once successfully connected to this database, all the application functionalities will be available for us.

If we want to load the database we access the menu: Admin -> Load Data and Definitions -> Data Definitions (*.df file). We select the table’s definition file from the local disk and select OK button.

Dumps files stores database information. If we want to load data content then we have to follow Admin -> Load Data and Definitions -> Table Contents (*.d file) path. Select Select Some… button and then select all tables (*), then press OK.

Select Tables window

If we press OK button, then we will get a new window where we have to introduce the folder path that contains the dumps data files. Attention! This operation takes time.

Progress database link to myApp

In this step, following last step, we have already the database with structure and content.
– Create a configuration file with .pf extension (ex. mydb.pf) in /myApp/myApp111/Total/Pf/ path:
[bash]# cd /myApp/myApp111/Total/Pf/
# cp -p ron.pf mydb.pf[/bash]

– Edit the new created file using and text editor (ex. vi, pico, mc), changing the first line:
[bash]- db total -H 192.168.42.10 -N TCP -S 2521 [/bash]
like this:
[bash]- db total -H 192.168.42.10 -N TCP -S 2540 [/bash]
2540 it’s the new port that we will use it for our database.

– The last step it’s the link creating to our workstation. We create a new desktop shortcut to prowin32 application (\Program Files\PROGRESS\bin\ path) and we change shortcut properties: map a disc drive to the folder that stores total server folder: right click over My Network Places and select Map Network Drive…; select a free drive ( ex. N: ) and in Folder area introduce \\192.168.42.10\myApp111 (my myApp111 folder it’s the path where our application it’s stored) and finally we select Finish button.

Change shortcut properties in order to use our application configuration file.
"\Program Files\PROGRESS\bin\prowin32" -pf \total\pf\mydb.pf"

As you can see the server Sun path stars from total folder and contains mydb.df file. First file path it’s not full completed because it uses map drive setting that was applied.

Unix server’s folder myApp111 is the folder that contains that use Progress database.

Create a database backup

In my system the Unix’s Cron daemon runs every evening a backup script. But, sometimes it’s necessary to save backup in the middle of the day before an admin’s specific task
The steps that you have to follow are:
– Stop current database:
[bash]# /progress/dlc/bin/proshut /myApp/baze/test_db/total /by [/bash]
– Build the backup:
[bash] # /progress/dlc/bin/probkup /myApp/baze/test_db/total /myApp/baze/bkupuri/ testdb_10.12.bk [/bash]

– Restart the database with the specific port (ex. 2540):
# /progress/dlc/bin/proserve /myApp/baze/test_db/total -S 2540 -N TCP -L 300000

Restoring a database backup
My back-ups location on my server is to /hitachi/backup/ path. Cron’s daemon script generates backups in this location.
The files backup could exist in two ways:
1. .bk extension (ex. ronb2005.12.05.bk) – it means uncompressed backup
2. .gz extension (ex. ronb2005.11.30.bk.gz) – it means compressed backup.

In order to restore a database backup we follow next steps:
– Create a new folder (ex. myrest) in /myApp/baze/ path and copy here the backup that we want to restore (ex. ronb2005.11.30.bk.gz):
[bash] #cd /myApp/baze/
#mkdir myrest/
#cd /hitachi/backup/
#ls
#cp ronb2005.11.30.bk.gz /myApp/baze/myrest/
#cd /myApp/baze/myrest/ [/bash]

– If the backup it’s compressed, run restore commands:
[bash]# gzip -d ronb2005.11.30.bk.gz ./*
# /progress/dlc/bin/prorest total ronb2005.11.30.bk
[/bash]
– If the backup file it’s uncompressed (1st backup version) then we don’t need gzip command:
[bash]# /progress/dlc/bin/prorest total ronb2005.12.05.bk [/bash]

– Assign rights to database:
[bash]# chown -R myApp:staff ./*
# chmod -R 644 ./*
# cd ..
# chmod -R 755 myrest/
# chown myApp:staff myrest/ [/bash]

Then, follow Starting a database procedure.

Stop the database
In order to stop a database we have to follow next steps:
– Check witch it the port used by current started database:
[bash]# /ps -Af | grep mpr [/bash]

This it’s an additional step.
– Execute the stop command:
[bash] # /progress/dlc/bin/proshut /myApp/baze/test_db/total -S 2540 -N TCP -L 300000[/bash]

Restart a database
First, when we want to restart a database we have to check that all the database clients are disconnected.

[bash] # /progress/dlc/bin/proshut /myApp/baze/test_db/total [/bash]

This command shows us if there are connected users and if yes show the users name. We have the possibility to force client’s disconnection. We renounce to this action pressing X key and contact the users in order to disconnect. After few minutes we check again that all clients have been disconnected.

– When all the clients are disconnected we start the brute force command:
[bash] # /progress/dlc/bin/proshut /myApp/baze/test_db/total -by [/bash]

– Run restart command:
[bash]# /progress/dlc/bin/proserve /myApp/baze/test_db/total -S 2540 -N TCP -L 300000[/bash]

Compress/Decompress database

In order to compress a database we have to follow next steps:
– We go to the folder that where the database is saved (ex. /hitachi/backup/):
[bash] # cd /hitachi/backup/ [/bash]

– Execute compress command:
[bash] #gzip -9 saved_back_up.bk [/bash]
Then, we wait the process in order to finish. At the end of this command we will get a
saved_back_up.bk.gz file with a smaller size.

In order to decompress a compressed database we have to run next command:
[bash] # gzip -d saved_back_up.bk.gz [/bash]
After a successful result of this command we have the original database backup file saved_back_up.bk. Then follow restoring database procedure.