Monthly Archives: October 2013

C++ || Snippet – Custom Template “ToString” Function For Primitive Data Types

The following is a custom template “ToString” function which converts the value of a primitive data type (i.e: int, double, long, unsigned, char, etc.) to an std::string. So for example, if you wanted to convert an integer value to an std::string, the following function can do that for you.

This function works by accepting a primitive datatype as a function parameter, then using the std::stringstream class to convert and return an std::string object containing the representation of that incoming function parameter as a sequence of characters.

The code demonstrated on this page is different from the std::to_string function in that this implementation does not require a compiler to have >= C++11 support. This function should work on any compiler.


QUICK NOTES:
The highlighted lines are sections of interest to look out for.

The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.

Once compiled, you should get this as your output


-1987, 7.28, 12345678910, k, This is a c-string, This is a std::string

C++ || Multi Process Server & Client Hash Table Using Thread Pools, Message Queues, & Signal Handlers

The following is another homework assignment which was presented in an Operating Systems Concepts class. Using commandline arguments, the following is a program which implements a multi threaded hash table, utilizing message queues to pass text from a client program to a server program and vice versa. This program makes use of multiple interprocess communication function calls provided on Unix based systems.

REQUIRED KNOWLEDGE FOR THIS PROGRAM

How To Override The Default Signal Handler (CTRL-C)
How To Create And Use Pthreads For Interprocess Communication
How To Use Message Queues For Interprocess Communication
Multi Process Synchronization Producer Consumer Problem Using Pthreads
Sample Input Server Records File - Download Here

==== 1. OVERVIEW ====

Hash tables are widely used for efficient storage and retrieval of data. When using hash tables in multi threaded applications, you must ensure that concurrent accesses into the hash table is free from race conditions, dead locks, and other problems that arise in program synchronization.

One solution to overcome this problem is to prevent concurrent accesses into the hash table altogether; i.e. prior to accessing the hash table, a thread acquires a lock, and then releases the lock after the access is complete. Such approach, although simple, is inefficient. The program demonstrated on this page implements an alternative solution, one which permits safe concurrent accesses into the hash table. In the approach implemented on this page, each hash location within a hash table is protected with a separate lock. Hence, multiple threads access the hash table concurrently as long as they are accessing different hash locations. For greater efficiency, this program also makes use of a thread pool.

==== 2. TECHNICAL DETAILS ====

This program was implemented in two parts; a server program and a client program. The server side of the program maintains a hash table of records and a pool of threads. The client program requests from the server program search records by sending record ids over a message queue. The server program then retrieves a request from the message queue, and wakes up a thread in the thread pool. That awakened thread then uses the id (sent from the client program) to retrieve the corresponding record from the hash table, and sends the found record from the server program to the client program over the message queue.

The server also reads a specified file from the commandline, which stores initial user data that is to be inserted and stored into the hash table. The incoming text file has the following format:

a unique numerical id 1 (an integer)
the first name 1 (a string)
the last name 1 (a string)
.
.
.
a unique numerical id N (an integer)
the first name N (a string)
the last name N (a string)

These three fields make up one single record for one individual. More than one record may be present in the incoming text file.

==== 3. SERVER ====

The server has the following structure and function:

Spoiler Inside: Multi Threaded Process Server Flow Control SelectShow



The server program is invoked with the following commandline structure:

./Server [FILE NAME] [NUMBER OF THREADS] (e.g. ./Server database.dat 10)

The server is implemented below.

The client program has a much easier flow of control. It is implemented below.

==== 4. CLIENT ====

The client has the following structure and function:

Spoiler Inside: Multi Threaded Process Client Flow Control SelectShow


The client program was designed to sleep for 1 second every time a new record is obtained from the server. This makes it so its easier for the user to see what is being displayed on the screen.

The client program is invoked with the following commandline structure:

./client

The client is implemented below.


QUICK NOTES:
The highlighted lines are sections of interest to look out for.

The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.

See sample files named server.cpp and client.cpp which illustrate interprocess communications using message queues. See file condvar.cpp which illustrates the use of condition variables. Finally, see file signal.cpp which illustrates the overriding of default signal handlers.

The following is sample output:
(Note: remember to include the initial records input file!)

SERVER OUTPUT:

./Server INPUT_Records_programmingnotes_freeweq_com.txt 26

** SERVER ID #1015808 SUCCESSFULLY ESTABLISHED

^C

Caught the CTRL-C
Shutting down the server connection..

CLIENT OUTPUT:

./Client

** CONNECTION TO SERVER ID #1015808 SUCCESS

ID = 243 Record = Graham Basil
ID = 7943 Record = Tobias Arie
ID = 3607 Record = Claire Amina
ID = 849 Record = Jetta Victoria
ID = 126 Record = Jeramy Tod
ID = 7483 Record = Vivan Krystal
ID = 8036 Record = Lilliam Harley
ID = 1901 Record = Kati Basil
ID = 3524 Record = Kenneth Perkins
ID = 5256 Record = Jodee Albertina
ID = 7065 Record = Marylou Donn
ID = 3951 Record = Ula Domitila
ID = 395 Record = Jaime Lilliam
ID = 9234 Record = Nigel Gene
ID = 4148 Record = Carmella Evelia
ID = 9340 Record = Sang Cherilyn
ID = 3834 Record = Jessica Freddy

** SERVER CONNECTION CLOSED..

C++ || File Copier Using Memory Mapped Files

The following is another homework assignment which was presented in an Operating Systems Concepts class. Using commandline arguments, the following is a program which implements a file copier using memory mapped files. This program makes use of the “mmap” function call provided on Unix based systems.

REQUIRED KNOWLEDGE FOR THIS PROGRAM

How To Use Memory Mapped Files
How To Get The Size Of A File
How To Create A File Of Any Size

==== 1. OVERVIEW ====

A memory mapped file is a segment of virtual memory which has been assigned a direct byte for byte correlation with some portion of a file or file like resource. The primary benefit of memory mapping a file is increasing I/O performance, especially when used on large files. Accessing memory mapped files is faster than using direct read and write operations. Memory mapped files are designed to simplify and optimize file access.

The program demonstrated on this page works with any file type (i.e: txt, cpp, jpg, png, mp4, flv, etc.) and copies the contents of one file, and saves it into another separate output file.

==== 2. TECHNICAL DETAILS ====

This program has the following flow of control:

1. The program is invoked with the source file name and the destination file name as commandline arguments.
2. The program uses the mmap() system call to map both files into memory.
3. The program uses the memory-mapped file memory to to copy the source file to the destination file.

This program is also modified in such a way that you cannot do an mmap on the same file. For example, if you have an input file “A” as the source file, the destination file “B” needs to have a different filename. If both the source and the destination files have the same filename, an error message is displayed, and the program exits. In order for this program to work, the source and the destination files must have different names.


QUICK NOTES:
The highlighted lines are sections of interest to look out for.

The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.

The following is sample output:

./mcp 1.png 2.png

The source file "1.png" is 42004 bytes

The destination file "2.png" has been created and is 42004 bytes

C++ || Snippet – How To Create A File Of Any Size

The following is sample code which demonstrates how to create a file of any size using the fseek and fwrite function calls.


QUICK NOTES:
The highlighted lines are sections of interest to look out for.

The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.

The following is sample output:

./createfileN createfile.txt 12

[FILE "createfile.txt" IS CREATED AND IS 12 BYTES IN SIZE]

C++ || Snippet – How To Use Memory Mapped Files

The following is sample code which demonstrates the use of the “mmap”, “munmap” and “getpagesize” function calls on Unix based systems.

A memory mapped file is a segment of virtual memory which has been assigned a direct byte for byte correlation with some portion of a file or file like resource. This resource is typically a file that is physically present on disk, but can also be a device, shared memory object, or other resource that the operating system can reference through a file descriptor.

The primary benefit of memory mapping a file is increasing I/O performance, especially when used on large files. Accessing memory mapped files is faster than using direct read and write operations for two reasons. Firstly, a system call is orders of magnitude slower than a simple change to a program’s local memory. Secondly, in most operating systems the memory region mapped actually is the kernel’s page cache (file cache), meaning that no copies need to be created in user space.

The following example demonstrates the use of the “mmap” to map a file into memory, and display its contents to the screen.


QUICK NOTES:
The highlighted lines are sections of interest to look out for.

The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.

The following is sample output:

./mmap mmap.cpp

[THE CONTENTS OF "MMAP.CPP" IS DISPLAYED TO THE SCREEN]

C++ || Snippet – How To Display The Size Of A File Using Stat

The following is sample code which demonstrates the use of the “stat” function calls on Unix based systems.

The stat function returns information about a file. No permissions are required on the file itself, but-in the case of stat() and lstat() – execute (search) permission is required on all of the directories in path that lead to the file.

The “stat” system call returns a stat structure, which contains the following fields:

The following example demonstrates the use of the “st_size” struct parameter to display the size of a file.


QUICK NOTES:
The highlighted lines are sections of interest to look out for.

The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.

The following is sample output:

./Stat Stat.cpp

The file "Stat.cpp" is 1170 bytes

C++ || Snippet – How To Use Message Queues For Interprocess Communication

The following is sample code which demonstrates the use of the msgsnd, msgrcv, and msgget function calls for use with message queues on Unix based systems.

Message queues are used for interprocess communication. They provide two (or more) separate programs (i.e: sender and receiver) with the ability to interact with eachother. This is ideal because the sender and receiver do not need to interact with the message queue at the same time. Messages can be sent at one point in time, be placed on the queue until the receiver is ready for it, and then be removed at another point in time when the program(s) that service the queue are finally ready to receive a message.

This concept is different from threaded and forked process IPC procedures, which often times process data in a more streamlined manner. Rather than following a strict pattern as to when data is to be sent and received, queuing is a mechanism in which messages are held until the receiving application is ready to process them.

The example on this page demonstrates the above use of a message queue to transferrer data between two separate programs. The sending program (client.cpp) sends an integer and string variable to the receiving program (server.cpp), which then displays that received data to the screen.

NOTE: The server program must be ran before the client program!

=== 1. Server.cpp (Receiver) ===

=== 2. Client.cpp (Sender) ===


QUICK NOTES:
The highlighted lines are sections of interest to look out for.

You can view all allocated message queues using the ipcs command. You can delete a message queue from command line using ipcrm -q [KEY SHOWN BY IPCS]. Message queues are a finite resource. If something goes wrong during the execution of your program, you must manually delete all your queues.

The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.

The following is sample output.

SERVER OUTPUT:

The server has started!

Waiting for someone to connect to server id #753664 with the key 1258295474

someNumber = 0 buff = Message queues are awesome!
someNumber = 1 buff = Message queues are awesome!
someNumber = 2 buff = Message queues are awesome!
someNumber = 3 buff = Message queues are awesome!
someNumber = 4 buff = Message queues are awesome!
someNumber = 5 buff = Message queues are awesome!
someNumber = 6 buff = Message queues are awesome!
someNumber = 7 buff = Message queues are awesome!
someNumber = 8 buff = Message queues are awesome!
someNumber = 9 buff = Message queues are awesome!

Server is now shutting down!

CLIENT OUTPUT:

Successfully connected to server id #753664 with the key 1258295474

Now sending messages....Sending complete!

C++ || Snippet – Multi-Process Synchronization Producer Consumer Problem Using Threads

The following is sample code which demonstrates the use of POSIX threads (pthreads), aswell as pthread mutex and condition variables for use on Unix based systems.

The use of mutual exclusion (mutex) variables refers to the requirement of ensuring that no two processes or threads are in their critical section at the same time. Here, a critical section refers to the period of time in which a process accesses a shared resource, such as shared memory. This procedure highlights a classic example of a multi-process synchronization problem known as the producer–consumer problem.

The producer–consumer problem describes a scenario in which two processes (the producer and the consumer) share a common resource (i.e: a string buffer). In this scenario, the producer’s job is to generate a piece of data, update that data with the shared resource (the buffer), and repeat. At the same time, the consumer is consuming that shared resource (i.e: removing something from that same buffer) one piece at a time. The problem is to make sure that the producer wont try to manipulate that shared resource (i.e: add data into the buffer) while the consumer is accessing it; and that the consumer wont try to remove data from that shared resource (the buffer) while the producer is updating it.

In this instance, a solution for the producer can be to go to sleep if a synchronization conflict occurs, and the next time the consumer removes an item from the buffer, it can notify the producer to wake up and start to fill the buffer again. In the same way, the consumer can go to sleep when the producer is accessing it. The next time the producer puts data into the buffer, it wakes up the sleeping consumer and the process continues.

The example on this page demonstrates the use of the above solution using the “pthread_mutex_lock()” function call to limit access to the critical section, and the “pthread_cond_wait()” function call to simulate the sleeping process.


QUICK NOTES:
The highlighted lines are sections of interest to look out for.

To use pthreads, the compiler flag “-lpthread” must be set.

The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.

The following is sample output:

The Parent is creating the producer and consumer threads..

Produced: 1
Consumed: 1

Produced: 2
Consumed: 2

Produced: 3
Consumed: 3

Produced: 4
Consumed: 4

Produced: 5
Consumed: 5

Produced: 6
Consumed: 6

Both threads have completed and have terminated!

The Parent is now exiting...

C++ || Snippet – How To Create And Use Threads For Interprocess Communication

The following is sample code which demonstrates the use of POSIX threads (pthreads), aswell as the pthread_create, and pthread_join function calls on Unix based systems.

Much like the fork() function call which is used to create new processes, threads are similar in that they too are used for interprocess communication. Threads allow for multi-threading, which is a widespread programming and execution model that allows for multiple threads to exist within the same context of a single program.

Threads share the calling parent process’ resources. Each process has it’s own address space, but the threads within the same process share that address space. Threads also share any other resources within that process. This means that it’s very easy to share data amongst threads, but it’s also easy for the threads to step on each other, which can lead to bad things.

The example on this page demonstrates the use of multiple pthreads to display shared data (an integer variable) to the screen.


QUICK NOTES:
The highlighted lines are sections of interest to look out for.

So, what is the difference between fork() processes and thread processes? Threads differ from traditional multitasking operating system processes in that:

• processes are typically independent, while threads exist as subsets of a process
• processes carry considerable state information, whereas multiple threads within a process share state as well as memory and other resources
• processes have separate address spaces, whereas threads share their address space
• processes interact only through system-provided inter-process communication mechanisms.
• Context switching between threads in the same process is typically faster than context switching between processes.

To use pthreads, the compiler flag “-lpthread” must be set.

The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.

The following is sample output:

./Pthread 5

The Parent is creating 5 threads!
The Parent is now waiting for the thread(s) to complete...

Hi, Im thread #1 and this is my id number: 1664468736
Hi, Im thread #2 and this is my id number: 1647683328
Hi, Im thread #3 and this is my id number: 1656076032
Hi, Im thread #4 and this is my id number: 1672861440
Hi, Im thread #5 and this is my id number: 1681254144

All thread(s) are complete and have terminated!

The Parent is now exiting...

C++ || Snippet – How To Override The Default Signal Handler (CTRL-C)

The following is sample code which demonstrates the use of the “signal” function call on Unix based systems.

Signals are interrupts delivered to a process by the operating system which can terminate a program prematurely. You can generate interrupts by pressing Ctrl+C. The “signal” function call receives two arguments. The first argument is an integer which represents the signal number, and the second argument is a pointer to the user defined signal handling function.

The following program catches the “SIGINT” signal number using the signal() function.


QUICK NOTES:
The highlighted lines are sections of interest to look out for.

The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.

The following is sample output:

Please press CTRL-C

^C Haha I have 10 lives!
^C Haha I have 9 lives!
^C Haha I have 8 lives!
^C Haha I have 7 lives!
^C Haha I have 6 lives!
^C Haha I have 5 lives!
^C Haha I have 4 lives!
^C Haha I have 3 lives!
^C Haha I have 2 lives!
^C Haha I have 1 lives!
^C ** Ahh you got me...