Category Archives: Linux / Unix

psa.exe partial tree sample

Process Status Analysis – the first steps

I am pleased to announce my first cross-platform and open source project, the Process Status Analysis tool, available on GitHub.

The Process Status Analysis (psa) version 0.2 is available for Windows and Linux (Debian derived / Ubuntu tested) Operating Systems.
Download: psa for Ubuntu Linux x64 (3582 downloads)
Download: psa for Ubuntu Linux x86 (2833 downloads)
Download: psa.exe for Windows x64 (2812 downloads)
Download: psa.exe for Win32 (2829 downloads)

You may wonder why I did it or what it brings new. Well, I did it for fun, in my spare time and I will continue improving it when I’ll find a time to do it.

The project is written in modern C++ using idioms from the C++ 1x standards. Even if initially was done as a C++ for Windows only, during the past days I managed the port of it for Linux using Visual Studio 2017’s project templates and a connection via SSH.
In general, the source code base is similar, differing just by OS specific stuff.

In case you want to find out more about how to develop C++ Linux projects from the best development tool (imho), Visual Studio, you can find more information on Visual Studio development team blog.

Related to this psa project, the Linux version requires libprocps4-dev library in order to build.

The main reason for starting this project was that I wanted to know what’s the total memory amount used by my Chrome browser. I know it uses a lot of resources, but not that much… 🙂

Even if my preferred processes analysis tool, the Process Hacker offers a lot of processes administration possibilities, but it didn’t provide what I want, so I decided to enjoy a bit.

Chrome processes in Process Hacker

Sample – Google Chrome processes in Process Hacker tool

So, what I achieved by psa.exe was something like:

C:\Windows\system32> psa -o chrome
PID [896] chrome.exe 237.5234 MB
PID [1496] chrome.exe 87.6875 MB
PID [2388] chrome.exe 166.5000 MB
PID [5860] chrome.exe 3.1211 MB
PID [7336] chrome.exe 273.2188 MB
PID [8444] chrome.exe 68.0508 MB
PID [8624] chrome.exe 63.3945 MB
PID [9180] chrome.exe 296.9766 MB
PID [10600] chrome.exe 292.6289 MB
PID [12352] chrome.exe 182.5977 MB
PID [13688] chrome.exe 2.3555 MB
PID [14052] chrome.exe 73.1875 MB
PID [16200] chrome.exe 211.9805 MB
PID [17284] chrome.exe 55.7148 MB
PID [18036] chrome.exe 208.6680 MB
PID [19012] chrome.exe 457.2305 MB
PID [19312] chrome.exe 143.4766 MB
-----------------------------------
Total used memory: 2824.31 MB

A bit too much in my humble opinion…
The features this tool offers includes:

Get all processes loaded in memory information

I case you want to have a snapshot of all the processes loaded in the OS’s memory you can have it with.

psa -a

Get process only used memory

With psa.exe you can reach the used memory by a specific parameter -o after the process name or at least a part of its name.

psa -o chrome             // find how much memory uses your Chrome!   o_O

Currently, there is no string replace ‘*’ but it’s ongoing.


Print processes tree

Storing the processes’ data within a generic tree done by me, I took the decision to print the processes’ tree output, similarly there is in Windows with tree.exe tool or on Linux in pstree or even htop.

./psa -t
./psa -t 1034

psa.exe partial tree sample

Process Status Analysis partial tree of Windows process

Top most “expensive” processes

In case you want to see what are the most expensive processes within your operating system, you can achieve it with:

psa -e 20

or simpler psa -e in case you’re sure you want top 10 expensive processes (the default value).

silviu@ubuntu-dev-server:~/projects/psa-lin/bin/x64/Release$ ./psa -e
Top 10 consuming memory processes
-------------------------------------------
PID        Process Name         RAM Usage
-------------------------------------------
[924]    /usr/lib/policykit-1/polkitd   270.68 MB
[906]    /usr/lib/accountsservice/accounts-daemon       269.43 MB
[842]    /usr/sbin/rsyslogd     250.39 MB
[878]    /usr/lib/snapd/snapd   197.29 MB
[531]    /lib/systemd/systemd-timesyncd         97.97 MB
[1614]    sshd: silviu@pts/0    93.16 MB
[1576]    sshd: silviu [priv]   93.16 MB
[863]    /usr/bin/lxcfs         93.13 MB
[427]    /sbin/lvmetad          92.55 MB
[1034]    /usr/sbin/sshd        63.98 MB
-------------------------------------------


C:\Windows\system32> psa -e 10
Top 10 consuming memory processes
-------------------------------------------
PID        Process Name         RAM Usage
-------------------------------------------
[19012]    chrome.exe           435.67 MB
[17684]    chrome.exe           329.51 MB
[7336]    chrome.exe            259.61 MB
[15576]    devenv.exe           248.20 MB
[896]    chrome.exe             222.85 MB
[2388]    chrome.exe            188.21 MB
[18428]    chrome.exe           184.48 MB
[15760]    chrome.exe           173.82 MB
[488]    Dropbox.exe            162.15 MB
[5488]    googledrivesync.exe   161.11 MB
-------------------------------------------
Total used memory: 2365.61 MB

Redirect output to a file

From the standard output the information can be easily redirected to a file.

c:\> psa -t > windows_processes_tree.txt
# ./psa -t > linux_processes_tree.txt        

Kill process feature

This feature was not implemented yet but in case we need it we can be done it easily with the existing tools on the target OS (ex. Task Manager, Process Exporer/Hacker, pskill.exe for Windows or the combination ps + kill on LInux).

Feedbacks and improvements
Any constructive feedback, suggestions, contributions to improvements are appreciated.
Feel free to add any issue you find, wish or suggestion you have in the GitHub repository, the 
Issues section or here as a comment.

apt-get and the static IP challenges within Raspbian OS

This is the story of my first experiences with Raspberry PI 2 card computer. Back in March, I bought a new toy, a Raspberry PI 2. I have downloaded the Raspbian OS ISO image, wrote it on the microSD card and the toy was up and running. Raspbian OS Boot

As most probably you already know, the default package manager tool for Debian Linux derived distribution is apt-get. The apt-get command line tool that allows handling packages for new applications installation, updates, uninstall etc. The existence of this tool is one important thing that convinced me to use Linux Mint and Ubuntu instead of Red Hat fork distributions. The Raspbian OS is, also a Debian derived distribution, special design for small card-computer devices such Raspberry PI 2.

As you know the static IP address assignation facilitates easier access to any device (ex. via the SSH protocol) and I prefer to use it. I assigned a fix IP address to my Raspberry device into my router and in addition, I set the network setting to the proper static IP address (editing /etc/network/interfaces).

auto lo
iface lo inet loopback
#iface eth0 inet dhcp

iface eth0 inet static
address 192.168.0.10
netmask 255.255.255.0
network 192.168.0.0
broadcast 192.168.0.255
gateway 192.168.0.1

allow-hotplug wlan0
iface wlan0 inet manual
wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf
iface default inet dhcp

After a device reboot, I started the first update check via apt-get. Having a long experience with Ubuntu and Linux Mint, I had the surprise of getting errors in the moment of “apt-get update” execution.

Err http://raspberrypi.collabora.com wheezy Release.gpg
Temporary failure resolving 'raspberrypi.collabora.com'
Err http://archive.raspberrypi.org wheezy Release.gpg
Temporary failure resolving 'archive.raspberrypi.org'
Err http://mirrordirector.raspbian.org wheezy Release.gpg
Temporary failure resolving 'mirrordirector.raspbian.org'
Reading package lists... Done
W: Failed to fetch http://mirrordirector.raspbian.org/raspbian/dists/wheezy/Release.gpg Temporary failure resolving 'mirrordirector.raspbian.org'
W: Failed to fetch http://raspberrypi.collabora.com/dists/wheezy/Release.gpg Temporary failure resolving 'raspberrypi.collabora.com'
W: Failed to fetch http://archive.raspberrypi.org/debian/dists/wheezy/Release.gpg Temporary failure resolving 'archive.raspberrypi.org'
W: Some index files failed to download. They have been ignored, or old ones used instead.

apt-get cache cleaning provided no improvements. The ping to an explicit IP from Internet was getting an answer but not for domains, so the DNS was not acting as expected. Usually, the DNS settings modification is not needed in case we are using a DHCP server.
I preferred to use the Google’s DNS public DNS service by editing the /etc/resolv.conf file.

nameserver 8.8.8.8
name server 8.8.4.4

But even if I was logged with the root privileges each time I tried to save the /etc/resolv.conf I got “No space left on device” error message. I was surprised to realize that the disk was “100% full” and my microSD card size was less than a half I know (8GB).

pi@raspberrypi ~ $ df
Filesystem 1K-blocks Used Available Use% Mounted on
rootfs 3036242 3036242 3036242 100% /
/dev/root 3036242 3036242 3036242 100% /

In order to repair such disk space situations, we have to run the raspi-config application, where we have to select the first option Expand Filelsystem.

raspi-config tool

In case you get an error (i got twice), please insist on this option until you’ll get the message “Root partition has been resized“.
After the root partition has been recovered, the proper disk space was displayed. That allowed me the manual define the DNS settings I mentioned earlier within /etc/resolv.conf file.
Off-course you need to restart the network settings. In case you’re connected like me via SSH reboot the toy (sudo reboot). Tring to restart the networking daemon (pi@raspberrypi ~ $ sudo /etc/init.d/networking restart) might make the device unavailable into the network.

With these “improvements” the apt-get application was running perfect with the static IP address and I was able to install whatever official packages I needed.

References:
Raspberry Pi Static IP Address
Raspberry Pi’s “No space left on device” error message

Ubuntu – How to fix update errors for a server behind of a proxy

If you’re behind of a proxy server and the IP address has changed (or proxy it’s a new node in your network topology) then probably when you’re trying to update your binaries then you’ll get a list of errors.

W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/oneiric-security/multiverse/i18n/Translation-en_US Unable to connect to 192.168.120.240:3128:
W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/oneiric-security/multiverse/i18n/Translation-en Unable to connect to 192.168.120.240:3128: 
W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/oneiric-security/restricted/i18n/Translation-en_US Unable to connect to 192.168.120.240:3128: 
W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/oneiric-security/restricted/i18n/Translation-en Unable to connect to 192.168.120.240:3128:

This happens because you’re unable to connect to Ubuntu mirrors. In order to fix this issue, you have to edit apt.conf file settings.
– Check the proxy settings in apt.conf file:

my-user@my-server:/#sudo -s
root@my-server:/# view /etc/apt/apt.conf

– Update the new proxy IP and port (ex. http://192.168.230.99:3128):

root@my-server:/# vi /etc/apt/apt.conf
Acquire::http::Proxy "http://192.168.230.99:3128";

If you need user and password for logging to the Proxy server this can be achieved in most cases by using the following layout:

Acquire::http::Proxy "http://username:password@your_proxy_address:proxy_port";

– Finally, you may enjoy running the update commands:

root@my-server:/# apt-get update
root@my-server:/# apt-get upgrade

Progress database operations

Preliminary remarks

Application path (Sun Solaris Unix OS): /myApp/myapp111a/
Database location: /myApp/db/test_db

Usually, our workstations have Windows OS and we need to connect to Solaris Unix OS on SSH. That’s why we are using Putty application.
User: root
Password: xxxxxxxxxx

The Samba daemon must be available and activated (in order to support Windows share and map drive).
In this sample, the database is in /myApp/db/test_db path and it’s called, total. Basically, database files name doesn’t change, only the folder name (instead of test_db). In my example, the application port for this database it’s 2540 (might be changed).

Creating empty database and access rights
– Go to the folder that contains the database and create the folder tat contains the new database:

# cd /myApp/db/
# mkdir test_db/
# cd test_db/

– Create the new database called total:

# /progress/dlc/bin/prodb total /progress/dlc/emty8

– Rights assignment for database folder (test_db):

# cd ..
# chown -R myApp:staff test_db/
# cd test_db/
# chmod 644 ./*

– Additionally, we can check database permissions:

# ls -al

Database starting
– Check the open ports, especially database opened ports

# ps -Af | grep mpr

– Start the database using the established port, but an unused port value:

# /progress/dlc/bin/proserve /myApp/baze/test_db/total -S 2540 -N TCP -L 300000

Loading database content
The database and dumps loading process it’s realizing using Database Administration tool, available in Progress’s Windows suite. The suite path it’s: START -> Programs ->PROGRESS -> Data Administration.

Launch this application and select Database menu, Connect option. It will appear Connect Database window and we will complete it as in next image. After that we select OK button.

Connect Database window

From this capture you can observe the database name, the network protocol (TCP), Progress server’s IP (192.168.42.10) and our opened database port (2540).
Once successfully connected to this database, all the application functionalities will be available for us.

If we want to load the database we access the menu: Admin -> Load Data and Definitions -> Data Definitions (*.df file). We select the table’s definition file from the local disk and select OK button.

Dumps files stores database information. If we want to load data content then we have to follow Admin -> Load Data and Definitions -> Table Contents (*.d file) path. Select Select Some… button and then select all tables (*), then press OK.

Select Tables window

If we press OK button, then we will get a new window where we have to introduce the folder path that contains the dumps data files. Attention! This operation takes time.

Progress database link to myApp

In this step, following last step, we have already the database with structure and content.
– Create a configuration file with .pf extension (ex. mydb.pf) in /myApp/myApp111/Total/Pf/ path:

# cd /myApp/myApp111/Total/Pf/
# cp -p ron.pf mydb.pf

– Edit the new created file using and text editor (ex. vi, pico, mc), changing the first line:

- db total -H 192.168.42.10 -N TCP -S 2521

like this:

- db total -H 192.168.42.10 -N TCP -S 2540

2540 it’s the new port that we will use it for our database.

– The last step, it’s the link creating to our workstation. We create a new desktop shortcut to prowin32 application (\Program Files\PROGRESS\bin\ path) and we change shortcut properties: map a disc drive to the folder that stores total server folder: right click over My Network Places and select Map Network Drive…; select a free drive ( ex. N: ) and in Folder area introduce \\192.168.42.10\myApp111 (my myApp111 folder it’s the path where our application it’s stored) and finally we select Finish button.

Change shortcut properties in order to use our application configuration file.
"\Program Files\PROGRESS\bin\prowin32" -pf \total\pf\mydb.pf"

As you can see, the server Sun path stars from total folder and contains mydb.df file. First file path, it’s not full completed because it uses map drive setting that was applied.

Unix server’s folder myApp111 is the folder that contains that use Progress database.

Create a database backup

In my system, the Unix’s Cron daemon runs every evening a backup script. But, sometimes it’s necessary to save backup in the middle of the day before an admin’s specific task
The steps that you have to follow are:
– Stop current database:

# /progress/dlc/bin/proshut /myApp/baze/test_db/total /by

– Build the backup:

# /progress/dlc/bin/probkup /myApp/baze/test_db/total /myApp/baze/bkupuri/ testdb_10.12.bk

– Restart the database with the specific port (ex. 2540):

# /progress/dlc/bin/proserve /myApp/baze/test_db/total -S 2540 -N TCP -L 300000

Restoring a database backup
My backups location on my server is to /hitachi/backup/ path. Cron’s daemon script generates backups in this location.
The files backup could exist in two ways:
1. .bk extension (ex. ronb2005.12.05.bk) – it means uncompressed backup
2. .gz extension (ex. ronb2005.11.30.bk.gz) – it means compressed backup.

In order to restore a database backup we follow next steps:
– Create a new folder (ex. myrest) in /myApp/baze/ path and copy here the backup that we want to restore (ex. ronb2005.11.30.bk.gz):

#cd /myApp/baze/
#mkdir myrest/
#cd /hitachi/backup/
#ls
#cp ronb2005.11.30.bk.gz /myApp/baze/myrest/
#cd /myApp/baze/myrest/

– If the backup it’s compressed, run restore commands:

# gzip -d ronb2005.11.30.bk.gz ./*
# /progress/dlc/bin/prorest total ronb2005.11.30.bk

– If the backup file it’s uncompressed (1st backup version) then we don’t need gzip command:

# /progress/dlc/bin/prorest total ronb2005.12.05.bk

– Assign rights to database:

# chown -R myApp:staff ./*
# chmod -R 644 ./*
# cd ..
# chmod -R 755 myrest/
# chown myApp:staff myrest/

Then, follow Starting a database procedure.

Stop the database
In order to stop a database we have to follow next steps:
– Check which it the port used by current started database:

# /ps -Af | grep mpr

This it’s an additional step.
– Execute the stop command:

# /progress/dlc/bin/proshut /myApp/baze/test_db/total -S 2540 -N TCP -L 300000

Restart a database
First, when we want to restart a database, we have to check that all the database clients are disconnected.

# /progress/dlc/bin/proshut /myApp/baze/test_db/total

This command shows us if there are connected users and if yes show the user’s name. We have the possibility to force client’s disconnection. We renounce to this action, pressing X key and contact the users in order to disconnect. After a few minutes, we check again that all clients have been disconnected.

– When all the clients are disconnected, we start the brute force command:

# /progress/dlc/bin/proshut /myApp/baze/test_db/total -by

– Run restart command:

# /progress/dlc/bin/proserve /myApp/baze/test_db/total -S 2540 -N TCP -L 300000

Compress/Decompress database

In order to compress a database we have to follow next steps:
– We go to the folder that where the database is saved (ex. /hitachi/backup/):

# cd /hitachi/backup/

– Execute compress command:

#gzip -9 saved_back_up.bk

Then, we wait the process in order to finish. At the end of this command, we will get a
saved_back_up.bk.gz file with a smaller size.

In order to decompress a compressed database, we have to run next command:

# gzip -d saved_back_up.bk.gz

After a successful result of this command, we have the original database backup file saved_back_up.bk. Then follow restoring database procedure.