Silviu-Marius Ardelean's blog

a software engineer's web log

The First Hackathon Experience #HackTM2016

It have passed approximately two months from my first hackathon experience, the #HackTM2016 from Timisoara. This delay I’m publishing this article is mostly because of the release period to the job and other personal stuff I had to do.

This experience was a reminder of my high school programming competitions where I have attended with different applications written in the already ancient Turbo Pascal 7.0 or Borland C++ 3.1. At that moment, probably because of my high school informatics great teachers, the competition spirit between colleagues was so intense and we have competed each other in creating applications within the local competitions and in other cities from the country.

This year, the hackathon competition from Timisoara was between 20 – 22 May 2016 to the UPT Restaurant, Timisoara and from my point of view it was a great event. The event had categories such eHEALTH, Robots, Smart City, GameDev, IoT, Education, Fintech and Open but no Automotive category as I expected according to previous pitching sessions.

Before the effective event I have attended in two pitching sessions seeing different challenging proposals in few domains. The most appealing were some ideas of the ADAS team from Continental, a team I was apart between 2006-2008 (Siemens VDO department, video camera projects at that time).

Even if I had no team, because of curiosity, I bought the ticket and I went to the hackathon pitching session, trying to figure a team and to decide a project to deal with.

The Continental ADAS team came with hardware and software support how to hack their platform.

We were able to find a small team, first by two persons and later for a period five persons. Unfortunately those last three persons left us while realizing that the project is not what they have imagined initially, that other projects look more challenging or that their knowledge were not matching with what it was required for our project.
So, I installed on my laptop some software used internally by the ADAS team and we took a ride to record real traffic data. Once we had this data, the effective programming for grabbing it can be done in office conditions.

Driving to grab ADAS data

The idea

Having the radar and camera information, GPS data and vehicle dynamics, we were thinking is that in the near future, even before self-driving cars on the common roads, these information might be sent in cloud to be used by the traffic management solutions in order to be optimized the traffic. Even these days, in my city, Timisoara, such traffic management solution is implemented, but is not based on cars internal information. Maybe, this idea will not be applied.

What we have done – the effective experience

So, facing with this challenge, me and my teammate Nikola Kolevski, a Serbian nice guy, have started the work on Friday evening. We have met on that pitching meeting and we had a great cooperation. I was the back-end guy and he the man from the cloud. Because we “spoke” different languages, me C++ and he Python, we have decided to speak the REST way.

During our job we have improvised the “project management” with a Trello board. Of course, we used GitHub for source control.

What I effectively had to do it was to inject some code within a .DLL that was loaded into a Continental application and sent the ADAS information in cloud. Nicola was the guy that received and collected the data. Unfortunately, even if we tried few times, we found no front-end available teammate, just some with slices of time in terms of availability.

ADAS AE-RO HackTM concept

I decided to use the benefits of asynchronous programming and I used the Casablanca REST API. But because of the Continental’s Visual Studio 2012 project constraints (!!!), during the Friday evening I faced up with the challenge to find and adapt a Casablanca library older version to the project. The latest Casablanca’s versions are available for VS 2013 and VS 2015 only. Thanks to NuGet tool, I finally managed to get and use the 1.10 version.

On Saturday we managed the effective work, facing with some challenges related to the TCP/IP communication between our applications because of some Python server configuration, but finally our applications were able to talk each other via REST services, in the night and I have tweeted.

After a sleep break, next morning we did some last code adjustments and being time constrained we tried to improvised a frontend. Also, we had a second trip with the Continental’s Mercedes car to test what we did, using a 4G network.

As usual for a hackathon, everything was on the run with adrenaline, so quite nice! At the end of the hackathon we had to prepare for the hackathon jury’s visit and later for presentation because we have qualified in the first three teams in the Robots section. Yeah, we were included there because the Automotive category did not exist, but it was fine. 🙂

The truth is that the Continental was the only automotive represented company even if there are many such companies in Timisoara, but it seems they are not interest in such events.

Other interesting things from #HackTM2016

Attending to #HackTM2016 was a great experience, I have seen many interesting projects but from far the most exciting one was the Symme 3D Printer, a local start-up.

Conclusions

In a internet of things world our based idea might connect the cars ADAS information with intelligent management future systems to improve the traffic flow in big cities.

It is obvious but I want to underline: if you want to have success in hackathon, try having a core team before event. Otherwise, you might just have fun coding but not ending the prototype.  Strategies of being efficient would be a great asset.

Meeting new people and trying to do something from the scratch in a limited time is a very cool thing even if you don’t have time to write optimized and tested code. Also, you might learn a lot of new things.

Definitely, I will repeat this experience in the future!

Share

Finally, I got Windows 10

Finally, I got the Windows 10 for my laptop… Even if this task is trivial, it was a surprisingly experience this time. But let me tell you the story of Windows 10 installation on my Samsung RC 710 laptop.

Back in August I tweeted “That’s all I have within #Windows10 X64 Ent setup on Samsung RC710 with SSD”. By that, I meant that my setup process was stuck in the boot phase using the installation of this Windows brand new version. That was reproduced within Windows 8.1 x64 upgrade tentative or with clean Windows 10 installation.
I tried few Windows 10 .ISOs and a friend’s DVD but no chance. The setup has started and stuck within few seconds.

win10_setup_stockes

Because the laptop got some hardware upgrades from the original configuration, I tried to restore to the initial configuration but nothing new. Reading this article where is specifying “For 64-bit installations, a small number of older PCs may be blocked from installation because they do not support CMPXCHG16b, PrefetchW, and LAHF/SAHF“, I tried an x86 .ISO but I got the same situation. Also, I tried some BIOS changes without any improvements.

Getting online contact with Microsoft it offered me no new things. I got only typical support trivial answers.

During the time on the same laptop I was running fine Win XP, Win 7 and Win 8.1 OSs without any such bad experiences. To me it was clear that Windows 10 has some backward compatibilities issues.

So, I took it on my own googling for my situation. Reading different forums I realized this is a common issue for Samsung old laptops and it is generated by the WiFi card.

The solution in my case was buying a brand new Atheros AR5B22 WiFi card and replacing the old WiFi card with that.

If you’re in a similarly situation and you you’re looking for instructions how to disassemble your laptop here it is a brief presentation.

By the way, you’re doing on your own risk. If you’re not confident, please contact a specialist.


samsung_laptop_to_win10_3samsung_laptop_to_win10_4samsung_laptop_to_win10_6samsung_laptop_to_win10_5upgrading_windows_on_samsung_laptop

With this new WiFi card plugged-in, the upgrade from Window 8.1 x64 to Windows 10 x64 became a trivial task.

It would be nice if Microsoft would get more in count such behaviors and would improve the Windows 10’s backward compatibility, especially because the old computers are included within their OS target.

Update 02.06.2016: Samsung admits they are lame: “Don’t Install Windows 10 Because We Suck At Making Drivers”. Sad…

You can find additional information here. That’s why, most probably I will never buy a Samsung phone or any other gadget made by them, anymore.

Share

apt-get and the static IP challenges within Raspbian OS

This is the story of my first experiences with Raspberry PI 2 card computer. Back in March I bought a new toy, a Raspberry PI 2. I have downloaded the Raspbian OS ISO image, wrote it on the microSD card and the toy was up and running. Raspbian OS Boot

As most probably you already know, the default package manager tool for Debian Linux derived distribution is apt-get. The apt-get command line tool that allows handling packages for new applications installation, updates, uninstall etc. The existence of this tool is one important things that convinced me to use Linux Mint and Ubuntu instead of Red Hat fork distributions. The Raspbian OS is, also a Debian derived distribution, special design for small card-computer devices such Raspberry PI 2.

As you known the static IP address assignation facilitates easier access to any device (ex. via the SSH protocol) and I prefer to use it. I assigned a fix IP address to my Raspberry device into my router and in addition I set the network setting to the proper static IP address (editing /etc/network/interfaces).

After a device reboot, I started the first update check via apt-get. Having a long experience with Ubuntu and Linux Mint, I had the surprise of getting errors in the moment of “apt-get update” execution.

apt-get cache cleaning provided no improvements. The ping to an explicit IP from Internet was getting answer but not for domains, so the DNS was not acting as expected. Usually, the DNS settings modification is not needed in case we are using a DHCP server.
I preferred to use the Google’s DNS public DNS service by editing the /etc/resolv.conf file.

But even if I was logged with the root privileges each time I tried to save the /etc/resolv.conf I got “No space left on device” error message. I was surprised to realize that the disk was “100% full” and my microSD card size was less than a half I know (8GB).

In order to repair such disk space situations we have to run the raspi-config application, where we have to select the first option Expand Filelsystem.

raspi-config tool

In case you get error (i got twice), please insist over this option until you’ll get the message “Root partition has been resized“.
After the root partition has been recovered, the proper disk space was displayed. That allowed me the manual define the DNS settings I mentioned earlier within /etc/resolv.conf file.
Off-course you need to restart the network settings. In case you’re connected like me via SSH reboot the toy (sudo reboot). Tring to restart the networking daemon (pi@raspberrypi ~ $ sudo /etc/init.d/networking restart) might make the device unavailable into the network.

With these “improvements” the apt-get application was running perfect with the static IP address and I was able to install whatever official packages I needed.

References:
Raspberry Pi Static IP Address
Raspberry Pi’s “No space left on device” error message

Share

Some experiences with the last world-wide WordPress brute force attack

As you most probably know, this website uses WordPress. Last Saturday, trying to access the site admin area I was facing with an error generated by too many redirects.
error_redirects
Having other things to do, I ignored it for that moment. Later a friend of mine published on his website some information about an existing world-wide brute force attack over WordPress websites and then I started reading more information about this issue.

My Firefox’s Web Console has thrown such messages:

[13:38:30.162] GET http://my_site.ro/wp-admin/ [HTTP/1.1 302 Moved Temporarily 834ms]
[13:38:30.998] GET http://my_site.ro/wp-admin/ [HTTP/1.1 302 Moved Temporarily 403ms]
[13:38:31.405] GET http://my_site.ro/wp-admin/ [HTTP/1.1 302 Moved Temporarily 580ms]
[13:38:31.990] GET http://my_site.ro/wp-admin/ [HTTP/1.1 302 Moved Temporarily 558ms]
[13:38:32.558] GET http://my_site.ro/wp-admin/ [HTTP/1.1 302 Moved Temporarily 553ms]

Most probably I was also one target for that attack.

Having a strong password and not using admin user the effective website was not affected otherwise than in wp-admin area. I contacted my website hosting provider and after some emails exchange I was able to login into wp-admin area. First thing I did was to install and activate Limit Login Attempts plugin and the results didn’t expect too much to appear.
Today morning, this plugin sent me an interesting email.
results_of_limiting_login_attempts

So an attempt of brute force attack is cut.

In order to avoid any such unpleased issues it is strongly recommended following few basic steps:

  • Avoid using default users (ex. admin) having high privileges.
  • Use strong passwords that contain special characters also in order avoid dictionary attacks used by brute force methods. You can check if the password it’s strong enough using online free tools such passwordmeter.com or Password Checker.
  • Install and activate a tool such Limit Login Attempts.
  • Enjoy your life. 🙂
  • Share

    Ubuntu – How to fix update errors for a server behind of a proxy

    If you’re behind of a proxy server and the IP address has changed (or proxy it’s a new node in your network topology) then probably when you’re trying to update your binaries then you’ll get a list of errors.

    This happens because you’re unable to connect to Ubuntu mirrors. In order to fix this issue you have to edit apt.conf file settings.
    – Check the proxy settings in in apt.conf file:

    – Update the new proxy IP and port (ex. http://192.168.230.99:3128):

    If you need user and password for logging to the Proxy server this can be achieved in most cases by using the following layout.
    Acquire::http::Proxy "http://username:password@your_proxy_address:proxy_port";

    – Finally you may enjoy running the update commands:

    Share

    Adventures with _chkstk

    Called by the compiler when you have more than one page of local variables in your function.
    _chkstk Routine is a helper routine for the C compiler. For x86 compilers, _chkstk Routine is called when the local variables exceed 4K bytes; for x64 compilers it is 8K.

    That’s all that you get from _chkstk()’s msdn web page. Nothing more…

    Overview
    A process starts with a fixed stack space. The top of a stack is pointed to by the ESP register (Extended Stack Pointer) and this is a decrementing pointer. Every function calls results in a stack created for the function inside this Process Stack. Every thread function has its own stack. The stack is a downward growing array. When a function starts, the default stack reservation size is 1 MB.
    This is contrasting with the heap’s size whether theoretically increases to a limit of 4 GB on 32bits OS. See more information here.

    Every thread under Windows gets its own block of contiguous memory, and while function calls are made, the stack pointer is increasing and decreasing. In contrast, a different thread within the same process might get a different block of contiguous memory – its own stack. When a context switch occurs, the current thread’s ESP (along with the IP and other registers) are saved in the thread’s context structure, and restored when the thread is activated the next time.
    To specify a different default stack reservation size for all threads and fibers, use the STACKSIZE statement in the module definition (.def) file. To change the initially committed stack space, use the dwStackSize parameter of the CreateThread, CreateRemoteThread, or CreateFiber function.
    Most stack problems occur in overflows of existing stacks, as their sizes are fixed and they cannot be expanded.

    _chkstk() increases the stack when needed by committing some of the pages previously reserved for the stack. If there is no more physical memory available for committed pages, _chkstk fails. When you enter a function (VC++ with the stack checking enabled), it will call the _chkstk located in CHKSTK.ASM. This function does a stack page probing and causes the necessary pages of memory to be allocated using the guard page scheme, if possible. In this function, it is stated that when it encounters _XCPT_GUARD_PAGE_VIOLATION, the OS will attempt to allocate another guarded page and if it encounters _XCPT_UNABLE_TO_GROW_STACK then it’s a stack overflow error. When _XCPT_UNABLE_TO_GROW_STACK is encountered, the stack is not yet set up properly, that is why, that it will not call the catch because calling it will use invalid stack variables which will again cause another exception.

    Case – Too many or too big variables on stack
    As I said on top, the function stack size is 1 MB. If you miss that and you’re trying to define and use internally an array like this:

    When you’ll compile with VC++ compiler in debug mode you will have a big surprise: the application is crashing on _chkstk() in the moment the _chkstk() tries to create new memory page on stack and fails.
    The output window shows next message:
    First-chance exception at 0x004116e7 in testApp.exe: 0xC00000FD: Stack overflow.
    Unhandled exception at 0x004116e7 in testApp.exe: 0xC00000FD: Stack overflow.

    This happens because the 1MB limit is overloaded even on a win32 OS: 4000*200*4 = 3.2MB (approx.).
    Same story if you define many local variables and their stack usage overloads the 1MB limit. Off-course the thread stack size can be changed but think once again if it is really needed to do that.
    If you really need this big array then the best solution to avoid this crash is using the heap.

    Case – Recursive functions
    If you have an infinite recursion then you will gate same stack overflow error and the application crashes in _chkstk.asm. Recursive function is not the subject of this article so I don’t go in deep… Here it is a good example of what happens with recursive functions.
    The solution is to avoid using recursive functions as much as possible and try to implement an iterative function.

    Case – A stack corruption
    I have started looking over _chkstk() function in the moment when I got few bugs with crashes with some similarly details. I had to analyze some .dump files and solve few bugs that contained a call stack with _chkstk() on top.
    Most of the .dump files call stack contained a second similarly thing: the call of a thread function (so called ThreadFoo()) that was running in a threads pool.
    In that moment I started to research why _chkstk() fails and my first track was to debug the stack overflows. I followed a MSDN debugging tutorial and unfortunately I didn’t find something strange. I checked if the local stack variables are not so big in order to fill the ThreadFoo() function’s stack and it did not.
    Then a new study of ThreadFoo() function has followed in order to detect the internal functions calls that can fail in some circumstances. I stopped to some trace function calls and I studied deeply. Those trace functions where defined in an external debug class and each time when a new trace file was added it used an internal buffer (TCHAR szBuff[2048] = _T(“”);).
    The writing of this buffer was done using: swprintf(). As we know this function is unsafe and is not recommended to use. As long as the content of these trace lines was dynamically build (in some cases those line may contain even dynamically build SQL queries that failed) then the length of these trace lines could be higher than 2048 bytes and then guess what: a stack corruption appears! UPS! The stack pointer will be corrupted (the classic stack overflow case).

    So I have implemented and used the next macros:

    Now, if we’re using the safe macro we will have no issues.

    A safety alternative way to that buffer was the heap using but the heap access is not fast as the stack access so I preferred this approach (in a business application every milliseconds matters for the log system).
    After that fixed I met no other stack corruptions in ThreadFoo() and other code areas.

    Even if the top of the call stack was _chkstk() this was not the function that failed. The error appeared because of that stack corruption and _chkstk() has just detected.

    Conclusion
    If your code produces a stack overflow, then you have to rethink your design in right away:

    • If you see _chkstk() on the top of call stack, check if you have no stack corruptions – stack overflow.
    • Don’t try to manipulate the stack by yourself. The default 1MB stack size is basically enough
    • Allocate dynamically if you’re using big arrays
    • If you have recursive functions producing a stack overflow, re-write them using loops (a tip: it is a proven fact that any recursive functions can be programmed non-recursive)

    References
    Set stack size
    Thread Stack Size
    _chkstk Routine
    Stack (data structure)
    Debugging a Stack Overflow – tutorial
    Visual C++ apps crashing in _chkstk() under load
    Optimization Note (C++) 1: push, pop, call _chkstk
    What is Recursion?

    Share

    Hungry boy – Google Chrome 5

    My currently default bowser is Google Chrome. I use it more then 90% of my browsing time. I like it because it’s launching so fast, is a secure broswer, respects major W3C standards and has an interesting application architecture. Each tab is an independent process and if appears some troubles in one process, you can stop that process only, without loosing other Chrome’s tabs (processes) data.

    Unfortunately, two weeks ago, watching few slides presentations over slideshare.net I was shocked by a Windows message on my laptop (Windows 7 Ultimate x64 OS): “Your computer is low on memory. Save your files and close these programs: Google Chrome.
    What the hack?!

    Windows special MessageBox

    I closed that window and I investigated Task Manager in order to detect witch process was using my memory.
    I was socked again when I saw that one Chrome process was using even 701 MB!

    huge used memory

    Briliant!

    Memory graph

    The main Google Chrome process had 13 tabs: 8 tabs contained slideshare.net presentations, one with twitter.com and the rest normal web pages.

    Share

    Progress database operations

    Preliminary remarks

    Application path (Sun Solaris Unix OS): /myApp/myapp111a/
    Database location: /myApp/db/test_db

    Usually, our workstations have Windows OS and we need to connect to Solaris Unix OS on SSH. That’s why we are using Putty application.
    User: root
    Password: xxxxxxxxxx

    The Samba daemon must be available and activated (in order to support Windows share and map drive).
    In this sample, the database is in /myApp/db/test_db path and it’s called, total. Basically, database files name doesn’t change, only the folder name (instead of test_db). In my example, the application port for this database it’s 2540 (might be changed).

    Creating empty database and access rights
    – Go to the folder that contains the database and create the folder tat contains the new database:

    – Create the new database called total:

    – Rights assignment for database folder (test_db):

    – Additionally, we can check database permissions:

    Database starting
    – Check the open ports, especially database opened ports

    – Start the database using the established port, but an unused port value:

    Loading database content
    The database and dumps loading process it’s realizing using Database Administration tool, available in Progress’s Windows suite. The suite path it’s: START -> Programs ->PROGRESS -> Data Administration.

    Launch this application and select Database menu, Connect option. It will appear Connect Database window and we will complete it as in next image. After that we select OK button.

    Connect Database window

    From this capture you can observe the database name, the network protocol (TCP), Progress server’s IP (192.168.42.10) and our opened database port (2540).
    Once successfully connected to this database, all the application functionalities will be available for us.

    If we want to load the database we access the menu: Admin -> Load Data and Definitions -> Data Definitions (*.df file). We select the table’s definition file from the local disk and select OK button.

    Dumps files stores database information. If we want to load data content then we have to follow Admin -> Load Data and Definitions -> Table Contents (*.d file) path. Select Select Some… button and then select all tables (*), then press OK.

    Select Tables window

    If we press OK button, then we will get a new window where we have to introduce the folder path that contains the dumps data files. Attention! This operation takes time.

    Progress database link to myApp

    In this step, following last step, we have already the database with structure and content.
    – Create a configuration file with .pf extension (ex. mydb.pf) in /myApp/myApp111/Total/Pf/ path:

    – Edit the new created file using and text editor (ex. vi, pico, mc), changing the first line:

    like this:

    2540 it’s the new port that we will use it for our database.

    – The last step it’s the link creating to our workstation. We create a new desktop shortcut to prowin32 application (\Program Files\PROGRESS\bin\ path) and we change shortcut properties: map a disc drive to the folder that stores total server folder: right click over My Network Places and select Map Network Drive…; select a free drive ( ex. N: ) and in Folder area introduce \\192.168.42.10\myApp111 (my myApp111 folder it’s the path where our application it’s stored) and finally we select Finish button.

    Change shortcut properties in order to use our application configuration file.
    "\Program Files\PROGRESS\bin\prowin32" -pf \total\pf\mydb.pf"

    As you can see the server Sun path stars from total folder and contains mydb.df file. First file path it’s not full completed because it uses map drive setting that was applied.

    Unix server’s folder myApp111 is the folder that contains that use Progress database.

    Create a database backup

    In my system the Unix’s Cron daemon runs every evening a backup script. But, sometimes it’s necessary to save backup in the middle of the day before an admin’s specific task
    The steps that you have to follow are:
    – Stop current database:

    – Build the backup:

    – Restart the database with the specific port (ex. 2540):
    # /progress/dlc/bin/proserve /myApp/baze/test_db/total -S 2540 -N TCP -L 300000

    Restoring a database backup
    My back-ups location on my server is to /hitachi/backup/ path. Cron’s daemon script generates backups in this location.
    The files backup could exist in two ways:
    1. .bk extension (ex. ronb2005.12.05.bk) – it means uncompressed backup
    2. .gz extension (ex. ronb2005.11.30.bk.gz) – it means compressed backup.

    In order to restore a database backup we follow next steps:
    – Create a new folder (ex. myrest) in /myApp/baze/ path and copy here the backup that we want to restore (ex. ronb2005.11.30.bk.gz):

    – If the backup it’s compressed, run restore commands:

    – If the backup file it’s uncompressed (1st backup version) then we don’t need gzip command:

    – Assign rights to database:

    Then, follow Starting a database procedure.

    Stop the database
    In order to stop a database we have to follow next steps:
    – Check witch it the port used by current started database:

    This it’s an additional step.
    – Execute the stop command:

    Restart a database
    First, when we want to restart a database we have to check that all the database clients are disconnected.

    This command shows us if there are connected users and if yes show the users name. We have the possibility to force client’s disconnection. We renounce to this action pressing X key and contact the users in order to disconnect. After few minutes we check again that all clients have been disconnected.

    – When all the clients are disconnected we start the brute force command:

    – Run restart command:

    Compress/Decompress database

    In order to compress a database we have to follow next steps:
    – We go to the folder that where the database is saved (ex. /hitachi/backup/):

    – Execute compress command:

    Then, we wait the process in order to finish. At the end of this command we will get a
    saved_back_up.bk.gz file with a smaller size.

    In order to decompress a compressed database we have to run next command:

    After a successful result of this command we have the original database backup file saved_back_up.bk. Then follow restoring database procedure.

    Share