unix – My Programming Notes

Tag Archives: unix

C++ || Multi Process Server & Client Hash Table Using Thread Pools, Message Queues, & Signal Handlers

October 05, 2013 admin No comments

The following is another homework assignment which was presented in an Operating Systems Concepts class. Using commandline arguments, the following is a program which implements a multi threaded hash table, utilizing message queues to pass text from a client program to a server program and vice versa. This program makes use of multiple interprocess communication function calls provided on Unix based systems.

REQUIRED KNOWLEDGE FOR THIS PROGRAM

How To Override The Default Signal Handler (CTRL-C) How To Create And Use Pthreads For Interprocess Communication How To Use Message Queues For Interprocess Communication Multi Process Synchronization Producer Consumer Problem Using Pthreads Sample Input Server Records File - Download Here

==== 1. OVERVIEW ====

Hash tables are widely used for efficient storage and retrieval of data. When using hash tables in multi threaded applications, you must ensure that concurrent accesses into the hash table is free from race conditions, dead locks, and other problems that arise in program synchronization.

One solution to overcome this problem is to prevent concurrent accesses into the hash table altogether; i.e. prior to accessing the hash table, a thread acquires a lock, and then releases the lock after the access is complete. Such approach, although simple, is inefficient. The program demonstrated on this page implements an alternative solution, one which permits safe concurrent accesses into the hash table. In the approach implemented on this page, each hash location within a hash table is protected with a separate lock. Hence, multiple threads access the hash table concurrently as long as they are accessing different hash locations. For greater efficiency, this program also makes use of a thread pool.

==== 2. TECHNICAL DETAILS ====

This program was implemented in two parts; a server program and a client program. The server side of the program maintains a hash table of records and a pool of threads. The client program requests from the server program search records by sending record ids over a message queue. The server program then retrieves a request from the message queue, and wakes up a thread in the thread pool. That awakened thread then uses the id (sent from the client program) to retrieve the corresponding record from the hash table, and sends the found record from the server program to the client program over the message queue.

The server also reads a specified file from the commandline, which stores initial user data that is to be inserted and stored into the hash table. The incoming text file has the following format:

a unique numerical id 1 (an integer) the first name 1 (a string) the last name 1 (a string) . . . a unique numerical id N (an integer) the first name N (a string) the last name N (a string)

These three fields make up one single record for one individual. More than one record may be present in the incoming text file.

==== 3. SERVER ====

The server has the following structure and function:

Multi Threaded Process Server Flow Control	Select Show
1. The server is invoked with two command line arguments specifying: • The name of the file storing a list of records. • The number of threads. 2. The server reads the specified file, then stores the records in the hash table. The hash table has the following structure and function: • The hash table is an array of 100 cells. • Each cell contains two items (1) a lock (mutex) protecting the cell; and (2) a linked-list of records hashing to the cell. • The hash table supports methods for inserting and retrieving records. • The hash keys of records are computed as record id mod hash table size where record id is the id associated with each record. 3. Creates a message queue using msgget() system call. 4. Creates the specified number of threads. 5. Each thread joins a thread pool. 6. The parent thread then checks the message queue for new messages using msgrcv() system call. 7. When a new message arrives the parent thread retrieves the message, adds it to a list of received messages, and wakes up a thread from the thread pool. 8. The awakened thread removes a message from the list and checks the id contained in the message. The id indicates that the client who sent the message wants to know the record associated with the respective id. 9. The thread then checks if the record with a given id exists in the table and if so retrieves it. The retrieved record is then sent to the client over the message queue. If the record does not exist, then the server sends back a record with id field set to -1. 10. The thread then removes the next message from the list and repeats the same process. If the list is empty, then the thread goes back to the thread pool. 11. The server also simulates a scenario where the hash table is constantly being updated with new records. You will simulate this by using 5 separate threads that periodically (e.g. every 1 second) randomly generate records and insert them into the hash table. 12. When the user presses Ctrl-c, the server catches the SIGINT signal, removes the message queue using msgctl(), deallocates any other resources (e.g. mutexes, condition variables, etc) and exits. To intercept the signal you will need to define a custom signal handler.

Multi Threaded Process Server Flow Control

Select Show

1. The server is invoked with two command line arguments specifying:

2. The server reads the specified file, then stores the records in the hash table. The hash table has the following structure and function:

record id

hash table size

record id

3. Creates a message queue using msgget() system call.

4. Creates the specified number of threads.

5. Each thread joins a thread pool.

6. The parent thread then checks the message queue for new messages using msgrcv() system call.

7. When a new message arrives the parent thread retrieves the message, adds it to a list of received messages, and wakes up a thread from the thread pool.

8. The awakened thread removes a message from the list and checks the id contained in the message. The id indicates that the client who sent the message wants to know the record associated with the respective id.

9. The thread then checks if the record with a given id exists in the table and if so retrieves it. The retrieved record is then sent to the client over the message queue. If the record does not exist, then the server sends back a record with id field set to -1.

10. The thread then removes the next message from the list and repeats the same process. If the list is empty, then the thread goes back to the thread pool.

11. The server also simulates a scenario where the hash table is constantly being updated with new records. You will simulate this by using 5 separate threads that periodically (e.g. every 1 second) randomly generate records and insert them into the hash table.

12. When the user presses Ctrl-c, the server catches the SIGINT signal, removes the message queue using msgctl(), deallocates any other resources (e.g. mutexes, condition variables, etc) and exits. To intercept the signal you will need to define a custom signal handler.

The server program is invoked with the following commandline structure:

./Server [FILE NAME] [NUMBER OF THREADS] (e.g. ./Server database.dat 10)

The server is implemented below.





		
		
			Multi Threaded Process Server With Hash Table
			
C++
			
			// =============================================================================
//   Author: K Perkins
//   Date:   Oct 5, 2013
//   Taken From: http://programmingnotes.org/
//   File:  Server.cpp
//   Description: Using a Hash Table, this program simulates a server and client 
//     application. The server program maintains a hash table of records and 
//     a pool of threads. The client requests records from the server by sending 
//     record ID's over the message queue. The server retrieves a request from 
//     the message queue and wakes up a thread in the thread pool. The awakened 
//     thread then uses the ID to retrieve the corresponding record from the 
//     hash table and sends the record to the client over the message queue.
//     
//     This is the server program which creates a message queue using the msgget() 
//     system call, checks the message queue for new messages using the msgrcv() 
//     system call, and uses the msgsnd system call to send messages back to 
//     the client
// =============================================================================
#include <iostream>
#include <vector>
#include <list>
#include <cstdlib>
#include <cstring>
#include <fstream>
#include <csignal>
#include <pthread.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/msg.h>
#include <unistd.h>
using namespace std;

// Compile & Run
// g++ Server.cpp -o Server -lpthread
// ./Server <FILE NAME> <NUMBER OF THREADS>

// function prototypes
void SignalHandler(int arg);
void InsertInitialDataIntoTable(ifstream& infile);
void* SearchRecords(void* args);
void SendRecord(int id, const char* firstName, const char* lastName);
void* GenerateNewRecord(void* args);

// items which are placed inside the hash table cell
struct Record
{
    int id;
    char firstName[50];
    char lastName[50];
};

// cell for the hash table
struct Cell
{   
    list<Record> cell;
    pthread_mutex_t mutex;
};

// message queue structure
struct MsgQueue
{
    // the channel the connection will be on
    long messageType;
    int recordID;
    char recordName[100];	
};

// constant variables
const int HASH_TABLE_SIZE = 100;
const int NUM_NEW_RECORDS_THREADS = 5;
const int CLIENT_TO_SERVER_CHANNEL = 2;
const int SERVER_TO_CLIENT_CHANNEL = 4;	
const int MSG_Q_ID = 0664;

// global variables
int msqid = 0; // message queue ID
vector<Cell> hashTable(HASH_TABLE_SIZE);
list<int> recievedMsg;

int main(int argc, char* argv[])
{		
    // declare variables
    srand(time(NULL));
    signal(SIGINT, SignalHandler);	// override the CTRL-C terminal singal 
    ifstream infile; // used to read in the file
    int searchThreadNum = 0; // keep track of how many threads are being processed
    int newRecThreadNum = 0;
    int numSearchThreads = 0; // how many threads search the table for records   
    key_t key = 0; // generate key to a file
    pthread_t* searchRecordThreads = NULL; // array of search records threads
    pthread_t* getNewRecordThreads = NULL; // array of new records threads  
    MsgQueue msg;  // instantiate a message queue   
        
    // check if theres enough commandline args
    if(argc < 3)
    {
        cerr<<"n** ERROR NOT ENOUGH ARGS!nn"
            <<"USAGE: "<<argv[0]<<" <FILE NAME> <NUMBER OF THREADS>nn";
        exit(1);
    }

    // open the input records file
    infile.open(argv[1]);

    // check to see if file exists
    if(infile.fail())
    {
        cerr<<"n** ERROR: ""<<argv[1]<<"" could not be found!n";
        exit(1);
    }

    // insert data into the hash table from the source file
    InsertInitialDataIntoTable(infile);

    // close the file after we are done using it
    infile.close();
        
    // get the number of threads from the command line 
    numSearchThreads = atoi(argv[2]);

    // initialize the search thread array with the number 
    // specified from the commandline
    searchRecordThreads = new pthread_t[numSearchThreads];

    // initialize the new records thread array with the
    // specified number
    getNewRecordThreads = new pthread_t[NUM_NEW_RECORDS_THREADS];
        

    // get a key for our msg queue
    key = ftok("/bin/ls", 'O');

    // check if we got a valid key
    if(key < 0)
    {
        perror("ftok error");
        exit(1);
    }	

    // get a msg ID
    msqid = msgget(key, MSG_Q_ID | IPC_CREAT);	
        
    // check if we got a valid ID
    if(msqid < 0)
    {
        perror("msgget error");
        exit(1);	
    }

    cout<<"n** SERVER ID #"<<msqid<<" SUCCESSFULLY ESTABLISHEDn"<<endl;			

    // infinite loop to send/recieve messages to/from client			
    do{	
        // recieve record from client
        if(msgrcv(msqid, &msg, sizeof(msg) - sizeof(long), CLIENT_TO_SERVER_CHANNEL, 0) < 0)
        {
            perror("msgsnd error");
            exit(1);
        }            

        // save the recieved record from the client into a linked list
        // this is used in the "SearchRecords" function
        recievedMsg.push_back(msg.recordID);
	    
    // --- SEARCH RECORDS THREADS        
        // keep track of how many threads are being used 	        			    
        if(searchThreadNum > numSearchThreads-1)
        {
            searchThreadNum = 0;        
        }	

        // create threads & send them to work
        if(pthread_create(&searchRecordThreads[searchThreadNum], NULL, &SearchRecords, NULL) < 0)
        {
            perror("pthread_create error");
            exit(1);
        }       	         

        // wait for threads to finish
        pthread_join(searchRecordThreads[searchThreadNum], NULL);        
    
    // --- GENERATE NEW RECORDS THREADS
        // keep track of how many threads are being used 	        			    
        if(newRecThreadNum > NUM_NEW_RECORDS_THREADS-1)
        {
            newRecThreadNum = 0;        
        }	

        // create threads & send them to work
        if(pthread_create(&getNewRecordThreads[newRecThreadNum], NULL, &GenerateNewRecord, NULL) < 0)
        {
            perror("pthread_create error");
            exit(1);
        }       	         

        // wait for threads to finish
        pthread_join(getNewRecordThreads[newRecThreadNum], NULL); 
        
    // --- INCREMENT NUMBER OF THREADS BEING USED       
        //cout<<searchThreadNum<<endl;
        ++searchThreadNum;  
        ++newRecThreadNum;     
    }while(true);							 
            
    return 0;
}// end of main

void* GenerateNewRecord(void* args)
{	
    const char* names[]={"Alva","Edda","Hiram","Lemuel","Della","Roseann","Sang",
        "Evelia","Claire","Marylou","Magda","Irvin","Reagan","Deb","Hillary",
        "Tuyetm","Cherilyn","Amina","Justin","Neville","Jessica","Demi",
        "Graham","Cinderella","Freddy","Vivan","Marjorie","Krystal","Liza",
        "Spencer","Jordon","Bernie","Geraldine","Kati","Jetta","Carmella",
        "Chery","Earlene","Gene","Lorri","Albertina","Ula","Karena","Johanna",
        "Alex","Tobias","Lashawna","Domitila","Chantel","Deneen","Nigel",
        "Lashanda","Donn","Theda","Many","Jeramy","Jodee","Tamra","Dessie",
        "Lawrence","Jaime","Basil","Roger","Cythia","Homer","Lilliam","Victoria",
        "Tod","Harley","Meghann","Jacquelyne","Arie","Rosemarie","Lyndon","Blanch",
        "Kenneth","Perkins","Kaleena"};
        
    Record temp;
    int record = rand()%10000;        
    int nameLen = sizeof(names)/sizeof(names[0]);  

    // lock the current hash table cell
    if(pthread_mutex_lock(&hashTable[record%HASH_TABLE_SIZE].mutex) < 0)
    {
        perror("pthread_mutex_lock error");
        exit(1);
    }    

    // put the new data into the hash table
    temp.id = record;
    strncpy(temp.firstName, names[rand()%(nameLen-1)], sizeof(temp.firstName));
    strncpy(temp.lastName, names[rand()%(nameLen-1)], sizeof(temp.lastName));

    hashTable[record%HASH_TABLE_SIZE].cell.push_back(temp);
    pthread_mutex_init(&hashTable[temp.id%HASH_TABLE_SIZE].mutex, NULL);
        
    // unlock the current hash table cell
    if(pthread_mutex_unlock(&hashTable[record%HASH_TABLE_SIZE].mutex) < 0)
    {
        perror("pthread_mutex_lock error");
        exit(1);
    }
    return 0;	
}// end of GenerateNewRecord

void* SearchRecords(void* args)
{	
    int record = 0;
    bool found = false;

    // get the record from the client to search the hash table
    if(!recievedMsg.empty())
    {
        record = recievedMsg.front();
        recievedMsg.pop_front();
    }
    //cout<<endl<<record<<endl;

    // if the current hash table cell isnt empty, search for the record id
    if(!hashTable[record%HASH_TABLE_SIZE].cell.empty())
    {   
        // lock the current hash table cell
        if(pthread_mutex_lock(&hashTable[record%HASH_TABLE_SIZE].mutex) < 0)
        {
            perror("pthread_mutex_lock error");
            exit(1);
        }
        
        // search hash table cell to see if given record is there
        for(list<Record>::iterator iter = hashTable[record%HASH_TABLE_SIZE].cell.begin(); 
            iter != hashTable[record%HASH_TABLE_SIZE].cell.end(); ++iter)
        {
            // we found a match in the hash table
            if(iter->id == record)
            {
                // set the data fields of the found record 
                // to prepare it to be sent over the message queue                
                SendRecord(iter->id, iter->firstName, iter->lastName);
                found = true;
                break;
            }
        }

        // unlock the current hash table cell	
        if(pthread_mutex_unlock(&hashTable[record%HASH_TABLE_SIZE].mutex) < 0)
        {
            perror("pthread_mutex_lock error");
            exit(1);
        }
    }

    // if we didnt find a record, tell that to the client
    if(!found)
    {
        SendRecord(-1, "Record Not Found!", "");
    }
    return 0;	
}// end of SearchRecords

void SendRecord(int id, const char* firstName, const char* lastName)
{
    MsgQueue msg;  // instantiate a message queue

    // set the message queue channel
    msg.messageType = SERVER_TO_CLIENT_CHANNEL;	    

    // set the record id
    msg.recordID = id;                

    // set the record name
    strncpy(msg.recordName, firstName, sizeof(msg.recordName));
    strncat(msg.recordName, " ", sizeof(msg.recordName));
    strncat(msg.recordName, lastName, sizeof(msg.recordName));        	

    // send record to client
    if(msgsnd(msqid, &msg, sizeof(msg) - sizeof(long), 0) < 0)
    {
        perror("msgsnd error");
        exit(1);
    }    
}// end of SendRecord   

void InsertInitialDataIntoTable(ifstream& infile)
{  
    Record temp;
    while(infile>>temp.id >>temp.firstName >>temp.lastName)
    {
        hashTable[temp.id%HASH_TABLE_SIZE].cell.push_back(temp);
        pthread_mutex_init(&hashTable[temp.id%HASH_TABLE_SIZE].mutex, NULL);
    }   
}// end of InsertInitialDataIntoTable

void SignalHandler(int arg)
{
    cerr<<"nnCaught the CTRL-Cn"
        <<"Shutting down the server connection..n";			

    // deallocate the queue 
    if(msgctl(msqid, IPC_RMID, NULL) < 0)
    {
        perror("msgctl error");
    }	 		
    exit(0);
}// http://programmingnotes.org/
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
				
						// =============================================================================
//   Author: K Perkins
//   Date:   Oct 5, 2013
//   Taken From: http://programmingnotes.org/
//   File:  Server.cpp
//   Description: Using a Hash Table, this program simulates a server and client 
//     application. The server program maintains a hash table of records and 
//     a pool of threads. The client requests records from the server by sending 
//     record ID's over the message queue. The server retrieves a request from 
//     the message queue and wakes up a thread in the thread pool. The awakened 
//     thread then uses the ID to retrieve the corresponding record from the 
//     hash table and sends the record to the client over the message queue.
//     
//     This is the server program which creates a message queue using the msgget() 
//     system call, checks the message queue for new messages using the msgrcv() 
//     system call, and uses the msgsnd system call to send messages back to 
//     the client
// =============================================================================
#include <iostream>
#include <vector>
#include <list>
#include <cstdlib>
#include <cstring>
#include <fstream>
#include <csignal>
#include <pthread.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/msg.h>
#include <unistd.h>
using namespace std;
 
// Compile & Run
// g++ Server.cpp -o Server -lpthread
// ./Server <FILE NAME> <NUMBER OF THREADS>
 
// function prototypes
void SignalHandler(int arg);
void InsertInitialDataIntoTable(ifstream& infile);
void* SearchRecords(void* args);
void SendRecord(int id, const char* firstName, const char* lastName);
void* GenerateNewRecord(void* args);
 
// items which are placed inside the hash table cell
struct Record
{
    int id;
    char firstName[50];
    char lastName[50];
};
 
// cell for the hash table
struct Cell
{   
    list<Record> cell;
    pthread_mutex_t mutex;
};
 
// message queue structure
struct MsgQueue
{
    // the channel the connection will be on
    long messageType;
    int recordID;
    char recordName[100];	
};
 
// constant variables
const int HASH_TABLE_SIZE = 100;
const int NUM_NEW_RECORDS_THREADS = 5;
const int CLIENT_TO_SERVER_CHANNEL = 2;
const int SERVER_TO_CLIENT_CHANNEL = 4;	
const int MSG_Q_ID = 0664;
 
// global variables
int msqid = 0; // message queue ID
vector<Cell> hashTable(HASH_TABLE_SIZE);
list<int> recievedMsg;
 
int main(int argc, char* argv[])
{		
    // declare variables
    srand(time(NULL));
    signal(SIGINT, SignalHandler);	// override the CTRL-C terminal singal 
    ifstream infile; // used to read in the file
    int searchThreadNum = 0; // keep track of how many threads are being processed
    int newRecThreadNum = 0;
    int numSearchThreads = 0; // how many threads search the table for records   
    key_t key = 0; // generate key to a file
    pthread_t* searchRecordThreads = NULL; // array of search records threads
    pthread_t* getNewRecordThreads = NULL; // array of new records threads  
    MsgQueue msg;  // instantiate a message queue   
        
    // check if theres enough commandline args
    if(argc < 3)
    {
        cerr<<"n** ERROR NOT ENOUGH ARGS!nn"
            <<"USAGE: "<<argv[0]<<" <FILE NAME> <NUMBER OF THREADS>nn";
        exit(1);
    }
 
    // open the input records file
    infile.open(argv[1]);
 
    // check to see if file exists
    if(infile.fail())
    {
        cerr<<"n** ERROR: ""<<argv[1]<<"" could not be found!n";
        exit(1);
    }
 
    // insert data into the hash table from the source file
    InsertInitialDataIntoTable(infile);
 
    // close the file after we are done using it
    infile.close();
        
    // get the number of threads from the command line 
    numSearchThreads = atoi(argv[2]);
 
    // initialize the search thread array with the number 
    // specified from the commandline
    searchRecordThreads = new pthread_t[numSearchThreads];
 
    // initialize the new records thread array with the
    // specified number
    getNewRecordThreads = new pthread_t[NUM_NEW_RECORDS_THREADS];
        
 
    // get a key for our msg queue
    key = ftok("/bin/ls", 'O');
 
    // check if we got a valid key
    if(key < 0)
    {
        perror("ftok error");
        exit(1);
    }	
 
    // get a msg ID
    msqid = msgget(key, MSG_Q_ID | IPC_CREAT);	
        
    // check if we got a valid ID
    if(msqid < 0)
    {
        perror("msgget error");
        exit(1);	
    }
 
    cout<<"n** SERVER ID #"<<msqid<<" SUCCESSFULLY ESTABLISHEDn"<<endl;			
 
    // infinite loop to send/recieve messages to/from client			
    do{	
        // recieve record from client
        if(msgrcv(msqid, &msg, sizeof(msg) - sizeof(long), CLIENT_TO_SERVER_CHANNEL, 0) < 0)
        {
            perror("msgsnd error");
            exit(1);
        }            
 
        // save the recieved record from the client into a linked list
        // this is used in the "SearchRecords" function
        recievedMsg.push_back(msg.recordID);
	    
    // --- SEARCH RECORDS THREADS        
        // keep track of how many threads are being used 	        			    
        if(searchThreadNum > numSearchThreads-1)
        {
            searchThreadNum = 0;        
        }	
 
        // create threads & send them to work
        if(pthread_create(&searchRecordThreads[searchThreadNum], NULL, &SearchRecords, NULL) < 0)
        {
            perror("pthread_create error");
            exit(1);
        }       	         
 
        // wait for threads to finish
        pthread_join(searchRecordThreads[searchThreadNum], NULL);        
    
    // --- GENERATE NEW RECORDS THREADS
        // keep track of how many threads are being used 	        			    
        if(newRecThreadNum > NUM_NEW_RECORDS_THREADS-1)
        {
            newRecThreadNum = 0;        
        }	
 
        // create threads & send them to work
        if(pthread_create(&getNewRecordThreads[newRecThreadNum], NULL, &GenerateNewRecord, NULL) < 0)
        {
            perror("pthread_create error");
            exit(1);
        }       	         
 
        // wait for threads to finish
        pthread_join(getNewRecordThreads[newRecThreadNum], NULL); 
        
    // --- INCREMENT NUMBER OF THREADS BEING USED       
        //cout<<searchThreadNum<<endl;
        ++searchThreadNum;  
        ++newRecThreadNum;     
    }while(true);							 
            
    return 0;
}// end of main
 
void* GenerateNewRecord(void* args)
{	
    const char* names[]={"Alva","Edda","Hiram","Lemuel","Della","Roseann","Sang",
        "Evelia","Claire","Marylou","Magda","Irvin","Reagan","Deb","Hillary",
        "Tuyetm","Cherilyn","Amina","Justin","Neville","Jessica","Demi",
        "Graham","Cinderella","Freddy","Vivan","Marjorie","Krystal","Liza",
        "Spencer","Jordon","Bernie","Geraldine","Kati","Jetta","Carmella",
        "Chery","Earlene","Gene","Lorri","Albertina","Ula","Karena","Johanna",
        "Alex","Tobias","Lashawna","Domitila","Chantel","Deneen","Nigel",
        "Lashanda","Donn","Theda","Many","Jeramy","Jodee","Tamra","Dessie",
        "Lawrence","Jaime","Basil","Roger","Cythia","Homer","Lilliam","Victoria",
        "Tod","Harley","Meghann","Jacquelyne","Arie","Rosemarie","Lyndon","Blanch",
        "Kenneth","Perkins","Kaleena"};
        
    Record temp;
    int record = rand()%10000;        
    int nameLen = sizeof(names)/sizeof(names[0]);  
 
    // lock the current hash table cell
    if(pthread_mutex_lock(&hashTable[record%HASH_TABLE_SIZE].mutex) < 0)
    {
        perror("pthread_mutex_lock error");
        exit(1);
    }    
 
    // put the new data into the hash table
    temp.id = record;
    strncpy(temp.firstName, names[rand()%(nameLen-1)], sizeof(temp.firstName));
    strncpy(temp.lastName, names[rand()%(nameLen-1)], sizeof(temp.lastName));
 
    hashTable[record%HASH_TABLE_SIZE].cell.push_back(temp);
    pthread_mutex_init(&hashTable[temp.id%HASH_TABLE_SIZE].mutex, NULL);
        
    // unlock the current hash table cell
    if(pthread_mutex_unlock(&hashTable[record%HASH_TABLE_SIZE].mutex) < 0)
    {
        perror("pthread_mutex_lock error");
        exit(1);
    }
    return 0;	
}// end of GenerateNewRecord
 
void* SearchRecords(void* args)
{	
    int record = 0;
    bool found = false;
 
    // get the record from the client to search the hash table
    if(!recievedMsg.empty())
    {
        record = recievedMsg.front();
        recievedMsg.pop_front();
    }
    //cout<<endl<<record<<endl;
 
    // if the current hash table cell isnt empty, search for the record id
    if(!hashTable[record%HASH_TABLE_SIZE].cell.empty())
    {   
        // lock the current hash table cell
        if(pthread_mutex_lock(&hashTable[record%HASH_TABLE_SIZE].mutex) < 0)
        {
            perror("pthread_mutex_lock error");
            exit(1);
        }
        
        // search hash table cell to see if given record is there
        for(list<Record>::iterator iter = hashTable[record%HASH_TABLE_SIZE].cell.begin(); 
            iter != hashTable[record%HASH_TABLE_SIZE].cell.end(); ++iter)
        {
            // we found a match in the hash table
            if(iter->id == record)
            {
                // set the data fields of the found record 
                // to prepare it to be sent over the message queue                
                SendRecord(iter->id, iter->firstName, iter->lastName);
                found = true;
                break;
            }
        }
 
        // unlock the current hash table cell	
        if(pthread_mutex_unlock(&hashTable[record%HASH_TABLE_SIZE].mutex) < 0)
        {
            perror("pthread_mutex_lock error");
            exit(1);
        }
    }
 
    // if we didnt find a record, tell that to the client
    if(!found)
    {
        SendRecord(-1, "Record Not Found!", "");
    }
    return 0;	
}// end of SearchRecords
 
void SendRecord(int id, const char* firstName, const char* lastName)
{
    MsgQueue msg;  // instantiate a message queue
 
    // set the message queue channel
    msg.messageType = SERVER_TO_CLIENT_CHANNEL;	    
 
    // set the record id
    msg.recordID = id;                
 
    // set the record name
    strncpy(msg.recordName, firstName, sizeof(msg.recordName));
    strncat(msg.recordName, " ", sizeof(msg.recordName));
    strncat(msg.recordName, lastName, sizeof(msg.recordName));        	
 
    // send record to client
    if(msgsnd(msqid, &msg, sizeof(msg) - sizeof(long), 0) < 0)
    {
        perror("msgsnd error");
        exit(1);
    }    
}// end of SendRecord   
 
void InsertInitialDataIntoTable(ifstream& infile)
{  
    Record temp;
    while(infile>>temp.id >>temp.firstName >>temp.lastName)
    {
        hashTable[temp.id%HASH_TABLE_SIZE].cell.push_back(temp);
        pthread_mutex_init(&hashTable[temp.id%HASH_TABLE_SIZE].mutex, NULL);
    }   
}// end of InsertInitialDataIntoTable
 
void SignalHandler(int arg)
{
    cerr<<"nnCaught the CTRL-Cn"
        <<"Shutting down the server connection..n";			
 
    // deallocate the queue 
    if(msgctl(msqid, IPC_RMID, NULL) < 0)
    {
        perror("msgctl error");
    }	 		
    exit(0);
}// http://programmingnotes.org/

The client program has a much easier flow of control. It is implemented below.

==== 4. CLIENT ====

The client has the following structure and function:

Multi Threaded Process Client Flow Control	Select Show
1. The client program connects to the message queue previously established by the server. 2. If the connection is successful, then the client goes into an infinite loop where it: • Generates a random id between 0 and 10000. • Sends the id to the server via the message queue. • Waits for the server to reply with the requested record. • Displays the record received. • Repeats the process.

Multi Threaded Process Client Flow Control

Select Show

The client program was designed to sleep for 1 second every time a new record is obtained from the server. This makes it so its easier for the user to see what is being displayed on the screen.

The client program is invoked with the following commandline structure:

./client

The client is implemented below.





		
		
			Multi Threaded Process Server With Hash Table
			
C++
			
			// =============================================================================
//   Author: K Perkins
//   Date:   Oct 5, 2013
//   Taken From: http://programmingnotes.org/
//   File:  Client.cpp
//   Description: Using a Hash Table, this program simulates a server and client 
//     application. The server program maintains a hash table of records and 
//     a pool of threads. The client requests records from the server by sending 
//     record ID's over the message queue. The server retrieves a request from 
//     the message queue and wakes up a thread in the thread pool. The awakened 
//     thread then uses the ID to retrieve the corresponding record from the 
//     hash table and sends the record to the client over the message queue.
//     
//     This is the client program which connects to the message queue that the 
//     server previously establishes using the msgget() system call. It checks 
//     the message queue for sucessfully found records using the msgrcv() system 
//     call, and uses the msgsnd system call to send new search records back to 
//     the server, where it searches for new records
// 
//     NOTE: This file is set to display records on a 1 second delay
// =============================================================================
#include <iostream>
#include <cstdlib>
#include <ctime>
#include <unistd.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/msg.h>
using namespace std;

// Compile & Run
// g++ Client.cpp -o Client
// ./Client

// message queue structure
struct MsgQueue
{
    // the channel the connection will be on
	long messageType;
	int recordID;
	char recordName[100];	
};

// constant variables
const int HOW_LONG_TO_SLEEP = 1;
const int CLIENT_TO_SERVER_CHANNEL = 2;
const int SERVER_TO_CLIENT_CHANNEL = 4;
const int MSG_Q_ID = 0664;

int main()
{
    // declare variables
    MsgQueue msg;
    srand(time(NULL));
    key_t key = 0;
    int msqid = 0; // message queue ID
        
    // get a key for the queue
    key = ftok("/bin/ls", 'O');

    // check if we got a valid key
    if(key < 0)
    {
        cerr<<"ftok";
        exit(1);
    }	

    // get the ID of the queue	
    msqid = msgget(key,MSG_Q_ID);	

    // check if we got a valid ID
    if(msqid < 0)
    {
        cerr<<"n** ERROR - CONNECTION TO SERVER NOT FOUNDnn";
        exit(1);	
    }	

    cerr<<"n** CONNECTION TO SERVER ID #"<<msqid<<" SUCCESSn"<<endl;	

    // infinite loop to send/recieve messages from server			
    do{	    
        // connect to the server
        msg.messageType = CLIENT_TO_SERVER_CHANNEL;

        // generate a random search key
        msg.recordID = rand()%10000;
        
        // send record search to the server
        if(msgsnd(msqid, &msg, sizeof(msg) - sizeof(long), 0) < 0)
        {
            cerr<<"n** SERVER CONNECTION CLOSED..nn";
            exit(1);
        }
        
        // recieve a record from the server
        if(msgrcv(msqid, &msg, sizeof(msg) - sizeof(long), SERVER_TO_CLIENT_CHANNEL, 0) < 0)
        {
            cerr<<"n** SERVER CONNECTION CLOSED..nn";
            exit(1);
        }
        
        // if we found a valid record, display it to the screen
        if(msg.recordID > 0)
        {
            cerr<< "ID = "<<msg.recordID<<"  tRecord = "<<msg.recordName<<endl;
            sleep(HOW_LONG_TO_SLEEP);
        }        			    		    
    }while(true);    

    return 0;
}// http://programmingnotes.org/
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
				
						// =============================================================================
//   Author: K Perkins
//   Date:   Oct 5, 2013
//   Taken From: http://programmingnotes.org/
//   File:  Client.cpp
//   Description: Using a Hash Table, this program simulates a server and client 
//     application. The server program maintains a hash table of records and 
//     a pool of threads. The client requests records from the server by sending 
//     record ID's over the message queue. The server retrieves a request from 
//     the message queue and wakes up a thread in the thread pool. The awakened 
//     thread then uses the ID to retrieve the corresponding record from the 
//     hash table and sends the record to the client over the message queue.
//     
//     This is the client program which connects to the message queue that the 
//     server previously establishes using the msgget() system call. It checks 
//     the message queue for sucessfully found records using the msgrcv() system 
//     call, and uses the msgsnd system call to send new search records back to 
//     the server, where it searches for new records
// 
//     NOTE: This file is set to display records on a 1 second delay
// =============================================================================
#include <iostream>
#include <cstdlib>
#include <ctime>
#include <unistd.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/msg.h>
using namespace std;
 
// Compile & Run
// g++ Client.cpp -o Client
// ./Client
 
// message queue structure
struct MsgQueue
{
    // the channel the connection will be on
	long messageType;
	int recordID;
	char recordName[100];	
};
 
// constant variables
const int HOW_LONG_TO_SLEEP = 1;
const int CLIENT_TO_SERVER_CHANNEL = 2;
const int SERVER_TO_CLIENT_CHANNEL = 4;
const int MSG_Q_ID = 0664;
 
int main()
{
    // declare variables
    MsgQueue msg;
    srand(time(NULL));
    key_t key = 0;
    int msqid = 0; // message queue ID
        
    // get a key for the queue
    key = ftok("/bin/ls", 'O');
 
    // check if we got a valid key
    if(key < 0)
    {
        cerr<<"ftok";
        exit(1);
    }	
 
    // get the ID of the queue	
    msqid = msgget(key,MSG_Q_ID);	
 
    // check if we got a valid ID
    if(msqid < 0)
    {
        cerr<<"n** ERROR - CONNECTION TO SERVER NOT FOUNDnn";
        exit(1);	
    }	
 
    cerr<<"n** CONNECTION TO SERVER ID #"<<msqid<<" SUCCESSn"<<endl;	
 
    // infinite loop to send/recieve messages from server			
    do{	    
        // connect to the server
        msg.messageType = CLIENT_TO_SERVER_CHANNEL;
 
        // generate a random search key
        msg.recordID = rand()%10000;
        
        // send record search to the server
        if(msgsnd(msqid, &msg, sizeof(msg) - sizeof(long), 0) < 0)
        {
            cerr<<"n** SERVER CONNECTION CLOSED..nn";
            exit(1);
        }
        
        // recieve a record from the server
        if(msgrcv(msqid, &msg, sizeof(msg) - sizeof(long), SERVER_TO_CLIENT_CHANNEL, 0) < 0)
        {
            cerr<<"n** SERVER CONNECTION CLOSED..nn";
            exit(1);
        }
        
        // if we found a valid record, display it to the screen
        if(msg.recordID > 0)
        {
            cerr<< "ID = "<<msg.recordID<<"  tRecord = "<<msg.recordName<<endl;
            sleep(HOW_LONG_TO_SLEEP);
        }        			    		    
    }while(true);    
 
    return 0;
}// http://programmingnotes.org/

QUICK NOTES:
The highlighted lines are sections of interest to look out for.

The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.

See sample files named server.cpp and client.cpp which illustrate interprocess communications using message queues. See file condvar.cpp which illustrates the use of condition variables. Finally, see file signal.cpp which illustrates the overriding of default signal handlers.

The following is sample output:
(Note: remember to include the initial records input file!)

SERVER OUTPUT:

./Server INPUT_Records_programmingnotes_freeweq_com.txt 26
** SERVER ID #1015808 SUCCESSFULLY ESTABLISHED ^C
Caught the CTRL-C Shutting down the server connection..

CLIENT OUTPUT:

./Client
** CONNECTION TO SERVER ID #1015808 SUCCESS ID = 243 Record = Graham Basil ID = 7943 Record = Tobias Arie ID = 3607 Record = Claire Amina ID = 849 Record = Jetta Victoria ID = 126 Record = Jeramy Tod ID = 7483 Record = Vivan Krystal ID = 8036 Record = Lilliam Harley ID = 1901 Record = Kati Basil ID = 3524 Record = Kenneth Perkins ID = 5256 Record = Jodee Albertina ID = 7065 Record = Marylou Donn ID = 3951 Record = Ula Domitila ID = 395 Record = Jaime Lilliam ID = 9234 Record = Nigel Gene ID = 4148 Record = Carmella Evelia ID = 9340 Record = Sang Cherilyn ID = 3834 Record = Jessica Freddy
** SERVER CONNECTION CLOSED..

C++ || File Copier Using Memory Mapped Files

October 05, 2013 admin No comments

The following is another homework assignment which was presented in an Operating Systems Concepts class. Using commandline arguments, the following is a program which implements a file copier using memory mapped files. This program makes use of the “mmap” function call provided on Unix based systems.

REQUIRED KNOWLEDGE FOR THIS PROGRAM

How To Use Memory Mapped Files How To Get The Size Of A File How To Create A File Of Any Size

==== 1. OVERVIEW ====

A memory mapped file is a segment of virtual memory which has been assigned a direct byte for byte correlation with some portion of a file or file like resource. The primary benefit of memory mapping a file is increasing I/O performance, especially when used on large files. Accessing memory mapped files is faster than using direct read and write operations. Memory mapped files are designed to simplify and optimize file access.

The program demonstrated on this page works with any file type (i.e: txt, cpp, jpg, png, mp4, flv, etc.) and copies the contents of one file, and saves it into another separate output file.

==== 2. TECHNICAL DETAILS ====

This program has the following flow of control:

1. The program is invoked with the source file name and the destination file name as commandline arguments. 2. The program uses the mmap() system call to map both files into memory. 3. The program uses the memory-mapped file memory to to copy the source file to the destination file.

This program is also modified in such a way that you cannot do an mmap on the same file. For example, if you have an input file “A” as the source file, the destination file “B” needs to have a different filename. If both the source and the destination files have the same filename, an error message is displayed, and the program exits. In order for this program to work, the source and the destination files must have different names.





		
		
			File Copier Using Memory Mapped Files
			
C++
			
			// ============================================================================
//    Author: K Perkins
//    Date:   Oct 5, 2013
//    Taken From: http://programmingnotes.org/
//    File:  Mcp.cpp
//    Description: This program implements a file copier based on memory
//       mapped files. 
//                 This program has the following flow of control:
//        (1) The program is invoked with the source file name and the 
//           destination file name as parameters.
//        (2) The program uses the mmap() system call to map both files into
//           memory.
//        (3) The program uses the memory-mapped file memory to copy the 
//           source file to the destination file.
// ============================================================================
#include <iostream>
#include <cstdio>
#include <cstdlib>
#include <cstring>
#include <unistd.h>
#include <fcntl.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/mman.h>
using namespace std;

int main(int argc, char* argv[])
{
    // declare variables
    struct stat statInfo;
    int srcFd = -1;
    int destFd = -1;
    int pagesize = -1;
    int mappingSize = -1;
    int leftToWrite = -1;
    off_t offset = 0;
    char* srcBuff = NULL;
    char* destBuff = NULL;    
    const char* srcFileName = NULL;
    const char* destFileName = NULL;
    FILE* destFp = NULL; 

    // check if theres enough commandline args
    if(argc < 3)
    {
        cerr<<"n** ERROR NOT ENOUGH ARGS!n"
          <<"nUSAGE: "<<argv[0]<<" <SOURCE FILE NAME> <DESTINATION FILE NAME>nn";
        exit(1);
    }

    // set the pointers to the source and destination files
    srcFileName = argv[1];
    destFileName = argv[2];

    // check if the source and the destination file have the same name
    if(strncmp(srcFileName, destFileName, sizeof(srcFileName)) == 0)
    {
        cerr<<"n** ERROR - The source and destination files both have the "
            <<"same name..nnPlease select a name other than ""
            <<srcFileName<<"" for the destination file!nn";
        exit(1);	
    }

    // get the file information
    if(stat(srcFileName, &statInfo) < 0)
    {
        perror("stat error");
        exit(1);
    }

    // open the destination file
    destFp = fopen(destFileName, "w+");

    // make sure it opened ok
    if(!destFp)
    {
        perror("open error");	
        exit(1);
    }

    // display info to the screen
    cerr<<"nThe source file ""<<srcFileName<<"" is "
        <<statInfo.st_size<<" bytesn"; 

    // set the pointer to the end of the file
    if(fseek(destFp, statInfo.st_size - 1, SEEK_SET) < 0)
    {
        perror("fseek error");
        exit(1);
    }
        
    // write a single byte to make the destination file 
    // the same size as the "source" file
    if(fwrite("", 1,  sizeof(""), destFp) < 0)
    {
        perror("write error");
        exit(1);
    }

    // close the destination file
    fclose(destFp);

    // open the source file for reading
    srcFd = open(srcFileName, O_RDONLY);	

    // make sure the file was opened successfully
    if(srcFd < 0) 
    { 
        perror("open error"); 
        exit(1); 
    }

    // open the destination file again, this time for actual writing
    destFd = open(destFileName, O_RDWR); 

    // make sure the destination file was opened successfully
    if(destFd < 0) 
    { 
        perror("open error"); 
        exit(1); 
    }

    // get the size of the page
    pagesize = getpagesize();	

    // how many bytes left to write
    leftToWrite = statInfo.st_size;

    // copy the bytes of the source file to the destination file
    while(leftToWrite)
    {
        // if there is more than pagesize bytes,
        // then map pagesize bytes. Otherwise,
        // map the remaining bytes        
        if(leftToWrite > pagesize)
        {
            mappingSize = pagesize;
        }
        else 
        {
            mappingSize = leftToWrite;
        }
        
        //cerr<<"mappingSize = "<<mappingSize<<endl;
        
        // map the pagesized source file
        if((srcBuff =  (char*)mmap(NULL, mappingSize, 
            PROT_READ, MAP_SHARED, srcFd, offset)) == (void*)-1)
        {
            perror("mmap1 error");
            exit(1);
        }
                
        // map the pagesized destination file
        if((destBuff =  (char*)mmap(NULL, mappingSize, 
            PROT_READ|PROT_WRITE, MAP_SHARED, destFd, offset)) == (void*)-1)

        {
            perror("mmap2 error");
            exit(1);
        }

        // copy the source to destination
        memcpy(destBuff, srcBuff, mappingSize);

        // update the number of bytes left to write
        leftToWrite -= mappingSize;

        // update the offset
        offset += mappingSize;	

        // unmap both buffers
        if(munmap((void*)srcBuff, mappingSize) < 0) 
        { 
            perror("munmap1 error"); 
            exit(1); 
        }

        if(munmap((void*)destBuff, mappingSize) < 0)
        {
            perror("munmap2 error"); 
            exit(1);
        }

    }
    
    // display info to the screen
    cerr<<"nThe destination file ""
        <<destFileName<<"" has been created and is "<<offset<<" bytesnn"; 

    // close both file descriptors
    close(srcFd);
    close(destFd);
        
    return 0;
}// http://programmingnotes.org/
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
				
						// ============================================================================
//    Author: K Perkins
//    Date:   Oct 5, 2013
//    Taken From: http://programmingnotes.org/
//    File:  Mcp.cpp
//    Description: This program implements a file copier based on memory
//       mapped files. 
//                 This program has the following flow of control:
//        (1) The program is invoked with the source file name and the 
//           destination file name as parameters.
//        (2) The program uses the mmap() system call to map both files into
//           memory.
//        (3) The program uses the memory-mapped file memory to copy the 
//           source file to the destination file.
// ============================================================================
#include <iostream>
#include <cstdio>
#include <cstdlib>
#include <cstring>
#include <unistd.h>
#include <fcntl.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/mman.h>
using namespace std;
 
int main(int argc, char* argv[])
{
    // declare variables
    struct stat statInfo;
    int srcFd = -1;
    int destFd = -1;
    int pagesize = -1;
    int mappingSize = -1;
    int leftToWrite = -1;
    off_t offset = 0;
    char* srcBuff = NULL;
    char* destBuff = NULL;    
    const char* srcFileName = NULL;
    const char* destFileName = NULL;
    FILE* destFp = NULL; 
 
    // check if theres enough commandline args
    if(argc < 3)
    {
        cerr<<"n** ERROR NOT ENOUGH ARGS!n"
          <<"nUSAGE: "<<argv[0]<<" <SOURCE FILE NAME> <DESTINATION FILE NAME>nn";
        exit(1);
    }
 
    // set the pointers to the source and destination files
    srcFileName = argv[1];
    destFileName = argv[2];
 
    // check if the source and the destination file have the same name
    if(strncmp(srcFileName, destFileName, sizeof(srcFileName)) == 0)
    {
        cerr<<"n** ERROR - The source and destination files both have the "
            <<"same name..nnPlease select a name other than ""
            <<srcFileName<<"" for the destination file!nn";
        exit(1);	
    }
 
    // get the file information
    if(stat(srcFileName, &statInfo) < 0)
    {
        perror("stat error");
        exit(1);
    }
 
    // open the destination file
    destFp = fopen(destFileName, "w+");
 
    // make sure it opened ok
    if(!destFp)
    {
        perror("open error");	
        exit(1);
    }
 
    // display info to the screen
    cerr<<"nThe source file ""<<srcFileName<<"" is "
        <<statInfo.st_size<<" bytesn"; 
 
    // set the pointer to the end of the file
    if(fseek(destFp, statInfo.st_size - 1, SEEK_SET) < 0)
    {
        perror("fseek error");
        exit(1);
    }
        
    // write a single byte to make the destination file 
    // the same size as the "source" file
    if(fwrite("", 1,  sizeof(""), destFp) < 0)
    {
        perror("write error");
        exit(1);
    }
 
    // close the destination file
    fclose(destFp);
 
    // open the source file for reading
    srcFd = open(srcFileName, O_RDONLY);	
 
    // make sure the file was opened successfully
    if(srcFd < 0) 
    { 
        perror("open error"); 
        exit(1); 
    }
 
    // open the destination file again, this time for actual writing
    destFd = open(destFileName, O_RDWR); 
 
    // make sure the destination file was opened successfully
    if(destFd < 0) 
    { 
        perror("open error"); 
        exit(1); 
    }
 
    // get the size of the page
    pagesize = getpagesize();	
 
    // how many bytes left to write
    leftToWrite = statInfo.st_size;
 
    // copy the bytes of the source file to the destination file
    while(leftToWrite)
    {
        // if there is more than pagesize bytes,
        // then map pagesize bytes. Otherwise,
        // map the remaining bytes        
        if(leftToWrite > pagesize)
        {
            mappingSize = pagesize;
        }
        else 
        {
            mappingSize = leftToWrite;
        }
        
        //cerr<<"mappingSize = "<<mappingSize<<endl;
        
        // map the pagesized source file
        if((srcBuff =  (char*)mmap(NULL, mappingSize, 
            PROT_READ, MAP_SHARED, srcFd, offset)) == (void*)-1)
        {
            perror("mmap1 error");
            exit(1);
        }
                
        // map the pagesized destination file
        if((destBuff =  (char*)mmap(NULL, mappingSize, 
            PROT_READ|PROT_WRITE, MAP_SHARED, destFd, offset)) == (void*)-1)
 
        {
            perror("mmap2 error");
            exit(1);
        }
 
        // copy the source to destination
        memcpy(destBuff, srcBuff, mappingSize);
 
        // update the number of bytes left to write
        leftToWrite -= mappingSize;
 
        // update the offset
        offset += mappingSize;	
 
        // unmap both buffers
        if(munmap((void*)srcBuff, mappingSize) < 0) 
        { 
            perror("munmap1 error"); 
            exit(1); 
        }
 
        if(munmap((void*)destBuff, mappingSize) < 0)
        {
            perror("munmap2 error"); 
            exit(1);
        }
 
    }
    
    // display info to the screen
    cerr<<"nThe destination file ""
        <<destFileName<<"" has been created and is "<<offset<<" bytesnn"; 
 
    // close both file descriptors
    close(srcFd);
    close(destFd);
        
    return 0;
}// http://programmingnotes.org/

QUICK NOTES:
The highlighted lines are sections of interest to look out for.

The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.

The following is sample output:

./mcp 1.png 2.png
The source file "1.png" is 42004 bytes
The destination file "2.png" has been created and is 42004 bytes

C++ || Snippet – How To Create A File Of Any Size

October 04, 2013 admin No comments

The following is sample code which demonstrates how to create a file of any size using the fseek and fwrite function calls.





		
		
			Create A File Of Any Size
			
C++
			
			// ============================================================================
//    Author: K Perkins
//    Date:   Oct 4, 2013
//    Taken From: http://programmingnotes.org/
//    File:  createfileN.cpp
//    Description: Demonstrate how to create a file of any size
// ============================================================================
#include <iostream>
#include <cstdio>
#include <cstdlib>
#include <cstring>
using namespace std;

int main(int argc, char* argv[])
{
    // declare variables
    int fileSize = 0;
    FILE* fp = NULL;

    // check the command line arguments
    if(argc < 3)
    {
        cerr<<"n** ERROR NOT ENOUGH ARGS!n"
            <<"nUSAGE: "<<argv[0]<<" <FILE NAME> <FILE SIZE>nn";	
        exit(1);
    }   
        
    // convert the file size from string to integer
    fileSize = atoi(argv[2]);	

    // open the file to be created
    fp = fopen(argv[1], "w+");

    // make sure the file was created ok
    if(!fp)
    {
        perror("open error");
        exit(1);
    }

    // set the file marker to position N in the file (where N = file size)
    if(fseek(fp, fileSize - 1, SEEK_SET) < 0)
    {
        perror("fseek error");
        exit(1);
    }
        
    // write a single byte to the file
    if(fwrite(" ", sizeof(char), strlen(" "), fp) < 0)
    {
        perror("write error");
        exit(1);
    }

    // close the file
    fclose(fp);
        
    return 0;
}// http://programmingnotes.org/
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
				
						// ============================================================================
//    Author: K Perkins
//    Date:   Oct 4, 2013
//    Taken From: http://programmingnotes.org/
//    File:  createfileN.cpp
//    Description: Demonstrate how to create a file of any size
// ============================================================================
#include <iostream>
#include <cstdio>
#include <cstdlib>
#include <cstring>
using namespace std;
 
int main(int argc, char* argv[])
{
    // declare variables
    int fileSize = 0;
    FILE* fp = NULL;
 
    // check the command line arguments
    if(argc < 3)
    {
        cerr<<"n** ERROR NOT ENOUGH ARGS!n"
            <<"nUSAGE: "<<argv[0]<<" <FILE NAME> <FILE SIZE>nn";	
        exit(1);
    }   
        
    // convert the file size from string to integer
    fileSize = atoi(argv[2]);	
 
    // open the file to be created
    fp = fopen(argv[1], "w+");
 
    // make sure the file was created ok
    if(!fp)
    {
        perror("open error");
        exit(1);
    }
 
    // set the file marker to position N in the file (where N = file size)
    if(fseek(fp, fileSize - 1, SEEK_SET) < 0)
    {
        perror("fseek error");
        exit(1);
    }
        
    // write a single byte to the file
    if(fwrite(" ", sizeof(char), strlen(" "), fp) < 0)
    {
        perror("write error");
        exit(1);
    }
 
    // close the file
    fclose(fp);
        
    return 0;
}// http://programmingnotes.org/

QUICK NOTES:
The highlighted lines are sections of interest to look out for.

The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.

The following is sample output:

./createfileN createfile.txt 12
[FILE "createfile.txt" IS CREATED AND IS 12 BYTES IN SIZE]

C++ || Snippet – How To Display The Size Of A File Using Stat

October 04, 2013 admin No comments

The following is sample code which demonstrates the use of the “stat” function calls on Unix based systems.

The stat function returns information about a file. No permissions are required on the file itself, but-in the case of stat() and lstat() – execute (search) permission is required on all of the directories in path that lead to the file.

The “stat” system call returns a stat structure, which contains the following fields:





		
		
			struct stat 
{
    dev_t     st_dev;     /* ID of device containing file */
    ino_t     st_ino;     /* inode number */
    mode_t    st_mode;    /* protection */
    nlink_t   st_nlink;   /* number of hard links */
    uid_t     st_uid;     /* user ID of owner */
    gid_t     st_gid;     /* group ID of owner */
    dev_t     st_rdev;    /* device ID (if special file) */
    off_t     st_size;    /* total size, in bytes */
    blksize_t st_blksize; /* blocksize for file system I/O */
    blkcnt_t  st_blocks;  /* number of 512B blocks allocated */
    time_t    st_atime;   /* time of last access */
    time_t    st_mtime;   /* time of last modification */
    time_t    st_ctime;   /* time of last status change */
};
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
				
						struct stat 
{
    dev_t     st_dev;     /* ID of device containing file */
    ino_t     st_ino;     /* inode number */
    mode_t    st_mode;    /* protection */
    nlink_t   st_nlink;   /* number of hard links */
    uid_t     st_uid;     /* user ID of owner */
    gid_t     st_gid;     /* group ID of owner */
    dev_t     st_rdev;    /* device ID (if special file) */
    off_t     st_size;    /* total size, in bytes */
    blksize_t st_blksize; /* blocksize for file system I/O */
    blkcnt_t  st_blocks;  /* number of 512B blocks allocated */
    time_t    st_atime;   /* time of last access */
    time_t    st_mtime;   /* time of last modification */
    time_t    st_ctime;   /* time of last status change */
};

The following example demonstrates the use of the “st_size” struct parameter to display the size of a file.





		
		
			How To Display The Size Of A File
			
C++
			
			// ============================================================================
//    Author: K Perkins
//    Date:   Oct 4, 2013
//    Taken From: http://programmingnotes.org/
//    File:  Stat.cpp
//    Description: Demonstrate the use of the "stat" function to display the
//       size of a file
// ============================================================================
#include <iostream>
#include <cstdlib>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
using namespace std;

int main(int argc, char* argv[])
{
    // declare variables
    struct stat fileInfo; // the buffer to store file information

    // check to see if theres enough commandline args 
    if(argc < 2)
    {
        cerr<<"n** ERROR - NOT ENOUGH ARGUMENTS!n"
            <<"nUSAGE: "<<argv[0]<<" <FILE NAME>n";
        exit(1);
    }

    // get the file information
    if(stat(argv[1], &fileInfo) < 0)
    {
        cerr<<"nstat failed (file not found)n";
        exit(1);
    }

    // display info to the screen
    cerr<<"nThe file ""<<argv[1]<<"" is "<<(int)fileInfo.st_size<<" bytesnn"; 
            
    return 0;
}// http://programmingnotes.org/
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
				
						// ============================================================================
//    Author: K Perkins
//    Date:   Oct 4, 2013
//    Taken From: http://programmingnotes.org/
//    File:  Stat.cpp
//    Description: Demonstrate the use of the "stat" function to display the
//       size of a file
// ============================================================================
#include <iostream>
#include <cstdlib>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
using namespace std;
 
int main(int argc, char* argv[])
{
    // declare variables
    struct stat fileInfo; // the buffer to store file information
 
    // check to see if theres enough commandline args 
    if(argc < 2)
    {
        cerr<<"n** ERROR - NOT ENOUGH ARGUMENTS!n"
            <<"nUSAGE: "<<argv[0]<<" <FILE NAME>n";
        exit(1);
    }
 
    // get the file information
    if(stat(argv[1], &fileInfo) < 0)
    {
        cerr<<"nstat failed (file not found)n";
        exit(1);
    }
 
    // display info to the screen
    cerr<<"nThe file ""<<argv[1]<<"" is "<<(int)fileInfo.st_size<<" bytesnn"; 
            
    return 0;
}// http://programmingnotes.org/

QUICK NOTES:
The highlighted lines are sections of interest to look out for.

The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.

The following is sample output:

./Stat Stat.cpp
The file "Stat.cpp" is 1170 bytes

C++ || Snippet – How To Use Message Queues For Interprocess Communication

October 04, 2013 admin No comments

The following is sample code which demonstrates the use of the msgsnd, msgrcv, and msgget function calls for use with message queues on Unix based systems.

Message queues are used for interprocess communication. They provide two (or more) separate programs (i.e: sender and receiver) with the ability to interact with eachother. This is ideal because the sender and receiver do not need to interact with the message queue at the same time. Messages can be sent at one point in time, be placed on the queue until the receiver is ready for it, and then be removed at another point in time when the program(s) that service the queue are finally ready to receive a message.

This concept is different from threaded and forked process IPC procedures, which often times process data in a more streamlined manner. Rather than following a strict pattern as to when data is to be sent and received, queuing is a mechanism in which messages are held until the receiving application is ready to process them.

The example on this page demonstrates the above use of a message queue to transfer data between two separate programs. The sending program (client.cpp) sends an integer and string variable to the receiving program (server.cpp), which then displays that received data to the screen.

NOTE: The server program must be ran before the client program!

=== 1. Server.cpp (Receiver) ===





		
		
			#1 - How To Use Message Queues - Receiver
			
C++
			
			// ============================================================================
//    Author: Kenneth Perkins
//    Date:   Oct 4, 2013
//    Taken From: http://programmingnotes.org/
//    File:  Server.cpp
//    Description: Demonstrate the use of a message queue for interprocess
//       communication. This program illustrates the server (receiving) end of
//       the message queue.
// ============================================================================
#include <iostream>
#include <cstdlib>
#include <unistd.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/msg.h>
using namespace std;

// Compile & Run
// g++ Server.cpp -o Server
// ./Server

// the structure representing the message queue
// & must match the same layout as in the client.cpp
struct MsgQueue
{
    // IMPORTANT: every message structure must start with this
    long messageType;

    // these variables are optional & you can add 
    // more or less if you wish
    int someNumber;
    char buff[100];	

};

// message queue flag
const int MSG_Q_KEY_FLAG = 0664;

// message queue data transfer channel
const int MSG_Q_CHANNEL = 26;

int main()
{
    // declare variables
    key_t key = -1;
    int msqid = -1;
    MsgQueue msg;

    // use a random file and a random character to generate
    // a unique key. The same parameters to this function will 
    // always generate the same value. This is how multiple
    // processes can connect to the same queue.
    key = ftok("/bin/ls", 'K');

    // was the key allocation successful ?
    if(key < 0)
    {
        perror("ftok error");
        exit(1);
    }	

    // allocate the message queue if it does not already exist.
    // this function returns the id of the queue.
    msqid = msgget(key, MSG_Q_KEY_FLAG | IPC_CREAT);

    // was the allocation a success ?
    if(msqid < 0)
    {
        perror("msgget");
        exit(1);
    }

    // display info to the screen
    cout <<"\nThe server has started!\n"
        <<"\nWaiting for someone to connect to server id #"<<msqid<<" with "
        <<"the key "<<key<<endl<<endl;
                
    // recieve 10 messages from the client
    for(int x = 0; x < 10; ++x)
    {
        // this is where we receive messages:
        // @param: msqid - the id of the message queue
        // @param: msg - the message structure which stores the
        //   received message
        // @param: sizeof(msg) - sizeof(long) - size of the message
        //   excluding the required first member (messageType) which is
        //   required.
        // @param: MSG_Q_CHANNEL - receive all messages whose type parameter
        //   is set equal to "MSG_Q_CHANNEL"
        // @param: 0 - flag values (not useful for this example).
        if(msgrcv(msqid, &msg, sizeof(msg) - sizeof(long), MSG_Q_CHANNEL, 0) < 0)
        {
            perror("msgrcv");
            exit(1);
        }
        
        // print the received message from the client
        cout << "someNumber = "<<msg.someNumber<<" buff = "<<msg.buff<<endl;
    }

    // finally, deallocate the message queue
    if(msgctl(msqid, IPC_RMID, NULL) < 0)
    {
        perror("msgctl");
        exit(1);
    }
    
    cout << "\nServer is now shutting down!\n";

    return 0;
}// http://programmingnotes.org/
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
				
						// ============================================================================
//    Author: Kenneth Perkins
//    Date:   Oct 4, 2013
//    Taken From: http://programmingnotes.org/
//    File:  Server.cpp
//    Description: Demonstrate the use of a message queue for interprocess
//       communication. This program illustrates the server (receiving) end of
//       the message queue.
// ============================================================================
#include <iostream>
#include <cstdlib>
#include <unistd.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/msg.h>
using namespace std;
 
// Compile & Run
// g++ Server.cpp -o Server
// ./Server
 
// the structure representing the message queue
// & must match the same layout as in the client.cpp
struct MsgQueue
{
    // IMPORTANT: every message structure must start with this
    long messageType;
 
    // these variables are optional & you can add 
    // more or less if you wish
    int someNumber;
    char buff[100];	
 
};
 
// message queue flag
const int MSG_Q_KEY_FLAG = 0664;
 
// message queue data transfer channel
const int MSG_Q_CHANNEL = 26;
 
int main()
{
    // declare variables
    key_t key = -1;
    int msqid = -1;
    MsgQueue msg;
 
    // use a random file and a random character to generate
    // a unique key. The same parameters to this function will 
    // always generate the same value. This is how multiple
    // processes can connect to the same queue.
    key = ftok("/bin/ls", 'K');
 
    // was the key allocation successful ?
    if(key < 0)
    {
        perror("ftok error");
        exit(1);
    }	
 
    // allocate the message queue if it does not already exist.
    // this function returns the id of the queue.
    msqid = msgget(key, MSG_Q_KEY_FLAG | IPC_CREAT);
 
    // was the allocation a success ?
    if(msqid < 0)
    {
        perror("msgget");
        exit(1);
    }
 
    // display info to the screen
    cout <<"\nThe server has started!\n"
        <<"\nWaiting for someone to connect to server id #"<<msqid<<" with "
        <<"the key "<<key<<endl<<endl;
                
    // recieve 10 messages from the client
    for(int x = 0; x < 10; ++x)
    {
        // this is where we receive messages:
        // @param: msqid - the id of the message queue
        // @param: msg - the message structure which stores the
        //   received message
        // @param: sizeof(msg) - sizeof(long) - size of the message
        //   excluding the required first member (messageType) which is
        //   required.
        // @param: MSG_Q_CHANNEL - receive all messages whose type parameter
        //   is set equal to "MSG_Q_CHANNEL"
        // @param: 0 - flag values (not useful for this example).
        if(msgrcv(msqid, &msg, sizeof(msg) - sizeof(long), MSG_Q_CHANNEL, 0) < 0)
        {
            perror("msgrcv");
            exit(1);
        }
        
        // print the received message from the client
        cout << "someNumber = "<<msg.someNumber<<" buff = "<<msg.buff<<endl;
    }
 
    // finally, deallocate the message queue
    if(msgctl(msqid, IPC_RMID, NULL) < 0)
    {
        perror("msgctl");
        exit(1);
    }
    
    cout << "\nServer is now shutting down!\n";
 
    return 0;
}// http://programmingnotes.org/

=== 2. Client.cpp (Sender) ===





		
		
			#2 - How To Use Message Queues - Sender
			
C++
			
			// ============================================================================
//    Author: Kenneth Perkins
//    Date:   Oct 4, 2013
//    Taken From: http://programmingnotes.org/
//    File:  Client.cpp
//    Description: Demonstrate the use of a message queue for interprocess
//       communication. This program illustrates the client (sending) end of
//       the message queue
// ============================================================================
#include <iostream>
#include <cstdlib>
#include <cstring>
#include <unistd.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/msg.h>
using namespace std;

// Compile & Run
// g++ Client.cpp -o Client
// ./Client

// the structure representing the message queue
// & must match the same layout as in the server.cpp
struct MsgQueue
{
    // IMPORTANT: every message structure must start with this
    long messageType;

    // these variables are optional & you can add 
    // more or less if you wish
    int someNumber;
    char buff[100];	

};

// message queue flag
const int MSG_Q_KEY_FLAG = 0664;

// message queue type
const int MSG_Q_CHANNEL = 26;

int main()
{
    // declare variables
    key_t key = -1;
    int msqid = -1;
    MsgQueue msg;
        
    // use a random file and a random character to generate
    // a unique key. The same parameters to this function will 
    // always generate the same value. This is how multiple
    // processes can connect to the same queue.    
    key = ftok("/bin/ls", 'K');

    // was the key allocation successful ?
    if(key < 0)
    {
        perror("ftok");
        exit(1);
    }	

    // connect to the message queue; fail if the	
    // there is no message queue associated with
    // this key. This function returns the id of 
    // the queue.
    msqid = msgget(key, MSG_Q_KEY_FLAG);	

    // was the allocation a success ?
    if(msqid < 0)
    {
        perror("msgget");
        exit(1);	
    }	

    // display info to the screen
    cout <<"\nSuccessfully connected to server id #"<<msqid<<" with "
        <<"the key "<<key
        <<"\n\nNow sending messages....";
                
    // send 10 messages to the server
    for(int x = 0; x < 10; ++x)
    {
        // set the message type - this must match
        // the 4th parameter of msgrcv() in the server.cpp code
        msg.messageType = MSG_Q_CHANNEL;
        
        // place data into the message queue structure to send to the server
        msg.someNumber = x;
        strncpy(msg.buff, "Message queues are awesome!", sizeof(msg.buff));
        
        // this is where we send messages:
        // @param: msqid - the id of the message queue
        // @param: msg - the message structure which stores the
        //   message to send
        // @param: sizeof(msg) - sizeof(long) - size of the message
        //   excluding the required first member (messageType) which is
        //   required.
        // @param: 0 - flag values (not useful for this example).
        if(msgsnd(msqid, &msg, sizeof(msg) - sizeof(long), 0) < 0)
        {
            perror("msgsnd");
            exit(1);
        }
    }
    
    cout << "Sending complete!\n";

    return 0;
}// http://programmingnotes.org/
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
				
						// ============================================================================
//    Author: Kenneth Perkins
//    Date:   Oct 4, 2013
//    Taken From: http://programmingnotes.org/
//    File:  Client.cpp
//    Description: Demonstrate the use of a message queue for interprocess
//       communication. This program illustrates the client (sending) end of
//       the message queue
// ============================================================================
#include <iostream>
#include <cstdlib>
#include <cstring>
#include <unistd.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/msg.h>
using namespace std;
 
// Compile & Run
// g++ Client.cpp -o Client
// ./Client
 
// the structure representing the message queue
// & must match the same layout as in the server.cpp
struct MsgQueue
{
    // IMPORTANT: every message structure must start with this
    long messageType;
 
    // these variables are optional & you can add 
    // more or less if you wish
    int someNumber;
    char buff[100];	
 
};
 
// message queue flag
const int MSG_Q_KEY_FLAG = 0664;
 
// message queue type
const int MSG_Q_CHANNEL = 26;
 
int main()
{
    // declare variables
    key_t key = -1;
    int msqid = -1;
    MsgQueue msg;
        
    // use a random file and a random character to generate
    // a unique key. The same parameters to this function will 
    // always generate the same value. This is how multiple
    // processes can connect to the same queue.    
    key = ftok("/bin/ls", 'K');
 
    // was the key allocation successful ?
    if(key < 0)
    {
        perror("ftok");
        exit(1);
    }	
 
    // connect to the message queue; fail if the	
    // there is no message queue associated with
    // this key. This function returns the id of 
    // the queue.
    msqid = msgget(key, MSG_Q_KEY_FLAG);	
 
    // was the allocation a success ?
    if(msqid < 0)
    {
        perror("msgget");
        exit(1);	
    }	
 
    // display info to the screen
    cout <<"\nSuccessfully connected to server id #"<<msqid<<" with "
        <<"the key "<<key
        <<"\n\nNow sending messages....";
                
    // send 10 messages to the server
    for(int x = 0; x < 10; ++x)
    {
        // set the message type - this must match
        // the 4th parameter of msgrcv() in the server.cpp code
        msg.messageType = MSG_Q_CHANNEL;
        
        // place data into the message queue structure to send to the server
        msg.someNumber = x;
        strncpy(msg.buff, "Message queues are awesome!", sizeof(msg.buff));
        
        // this is where we send messages:
        // @param: msqid - the id of the message queue
        // @param: msg - the message structure which stores the
        //   message to send
        // @param: sizeof(msg) - sizeof(long) - size of the message
        //   excluding the required first member (messageType) which is
        //   required.
        // @param: 0 - flag values (not useful for this example).
        if(msgsnd(msqid, &msg, sizeof(msg) - sizeof(long), 0) < 0)
        {
            perror("msgsnd");
            exit(1);
        }
    }
    
    cout << "Sending complete!\n";
 
    return 0;
}// http://programmingnotes.org/

QUICK NOTES:
The highlighted lines are sections of interest to look out for.

You can view all allocated message queues using the ipcs command. You can delete a message queue from command line using ipcrm -q [KEY SHOWN BY IPCS]. Message queues are a finite resource. If something goes wrong during the execution of your program, you must manually delete all your queues.

The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.

The following is sample output.

SERVER OUTPUT:

The server has started!
Waiting for someone to connect to server id #753664 with the key 1258295474 someNumber = 0 buff = Message queues are awesome! someNumber = 1 buff = Message queues are awesome! someNumber = 2 buff = Message queues are awesome! someNumber = 3 buff = Message queues are awesome! someNumber = 4 buff = Message queues are awesome! someNumber = 5 buff = Message queues are awesome! someNumber = 6 buff = Message queues are awesome! someNumber = 7 buff = Message queues are awesome! someNumber = 8 buff = Message queues are awesome! someNumber = 9 buff = Message queues are awesome!
Server is now shutting down!

CLIENT OUTPUT:

Successfully connected to server id #753664 with the key 1258295474
Now sending messages....Sending complete!

C++ || Snippet – Multi-Process Synchronization Producer Consumer Problem Using Threads

October 03, 2013 admin 2 Comments

The following is sample code which demonstrates the use of POSIX threads (pthreads), aswell as pthread mutex and condition variables for use on Unix based systems.

The use of mutual exclusion (mutex) variables refers to the requirement of ensuring that no two processes or threads are in their critical section at the same time. Here, a critical section refers to the period of time in which a process accesses a shared resource, such as shared memory. This procedure highlights a classic example of a multi-process synchronization problem known as the producer–consumer problem.

The producer–consumer problem describes a scenario in which two processes (the producer and the consumer) share a common resource (i.e: a string buffer). In this scenario, the producer’s job is to generate a piece of data, update that data with the shared resource (the buffer), and repeat. At the same time, the consumer is consuming that shared resource (i.e: removing something from that same buffer) one piece at a time. The problem is to make sure that the producer wont try to manipulate that shared resource (i.e: add data into the buffer) while the consumer is accessing it; and that the consumer wont try to remove data from that shared resource (the buffer) while the producer is updating it.

In this instance, a solution for the producer can be to go to sleep if a synchronization conflict occurs, and the next time the consumer removes an item from the buffer, it can notify the producer to wake up and start to fill the buffer again. In the same way, the consumer can go to sleep when the producer is accessing it. The next time the producer puts data into the buffer, it wakes up the sleeping consumer and the process continues.

The example on this page demonstrates the use of the above solution using the “pthread_mutex_lock()” function call to limit access to the critical section, and the “pthread_cond_wait()” function call to simulate the sleeping process.





		
		
			Producer Consumer Problem Using Pthreads
			
C++
			
			// ============================================================================
//    Author: Kenneth Perkins
//    Date:   Oct 3, 2013
//    Taken From: http://programmingnotes.org/
//    File:  Condvar.cpp
//    Description: Demonstrate the use of the producer consumer problem
//       using mutex and condition variables
// ============================================================================
#include <iostream>
#include <cstdlib>
#include <pthread.h>
using namespace std;

// Compile & Run
// g++ Condvar.cpp -lpthread -o Condvar
// ./Condvar

// function prototypes
void* Produce(void* arg);
void* Consume(void* arg);

// global variables
// the mutex and the condition variable
pthread_mutex_t mutex;
pthread_cond_t cond;

// the condition flag
bool condition = false;

// the item produced
int count = 0;

// how much to produce
const int NUM_TO_PRODUCE = 6;

int main()
{
    // declare variables
    pthread_t producerThread;
    pthread_t consumerThread;

    // initialize the mutex and the condition variable
    pthread_mutex_init(&mutex, NULL);
    pthread_cond_init(&cond, NULL);

    cout <<"\nThe Parent is creating the producer and consumer threads..\n" << endl;

    // create a producer thread
    if(pthread_create(&producerThread, NULL, Produce, NULL) < 0)
    {
        perror("pthread_create");
        exit(1);
    }

    // create a consumer thread
    if(pthread_create(&consumerThread, NULL, Consume, NULL) < 0)
    {
        perror("pthread_create");
        exit(1);
    }

    // wait for the threads to complete
    // similar to the wait() system call for fork() processes
    if(pthread_join(producerThread, NULL) < 0)
    {
        perror("pthread_join");
        exit(1);
    }

    if(pthread_join(consumerThread, NULL) < 0)
    {
        perror("pthread_join");
        exit(1);
    }

    cout <<"\nBoth threads have completed and have terminated!\n"
        <<"\nThe Parent is now exiting...\n";

    return 0;
}// end of main

/**
 * The producer thread function
 * @param arg - pointer to the thread local data - unused
 */
void* Produce(void* arg)
{
    // produce things until the loop condition is met
    while(count < NUM_TO_PRODUCE)
    {
        // lock the mutex to protect the condition variable
        if(pthread_mutex_lock(&mutex) < 0)
        {
            perror("pthread_mutex_lock");
            exit(1);
        }

        // we have produced something that has not been
        // consumed yet, so we sleep (wait) until the consumer
        // wakes us up.
        while(condition)
        {
            // sleep (wait) on a condition variable until the
            // the consumer wakes the producer up.
            if(pthread_cond_wait(&cond, &mutex) < 0)
            {
                perror("pthread_cond_wait");
                exit(1);
            }
        }

        // produce an item
        cout <<"Produced: "<<++count<<endl;

        // we have produced something
        condition = true;

        // wake up the sleeping consumer
        if(pthread_cond_signal(&cond) < 0)
        {
            perror("pthread_cond_signal");
            exit(1);
        }

        // release the mutex lock
        if(pthread_mutex_unlock(&mutex) < 0)
        {
            perror("pthread_mutex_unlock");
            exit(1);
        }
    }
    return 0;
}// end of Produce

/**
 * The consumer function
 * @param arg - pointer to the thread local data - unused
 */
void* Consume(void* arg)
{
    // consume things until the loop condition is met
    while(count < NUM_TO_PRODUCE)
    {
        // lock the mutex to protect the condition variable
        if(pthread_mutex_lock(&mutex) < 0)
        {
            perror("pthread_mutex_lock");
            exit(1);
        }

        // if there is nothing to consume, then sleep
        while(!condition)
        {
            // sleep (wait) on a condition variable until the
            // producer wakes the consumer up.
            if(pthread_cond_wait(&cond, &mutex) < 0)
            {
                perror("pthread_cond_wait");
                exit(1);
            }
        }

        cout <<"Consumed: "<<count<<endl<<endl;

        // consume the item
        condition = false;

        // wake up the sleeping producer
        if(pthread_cond_signal(&cond) < 0)
        {
            perror("pthread_cond_signal");
            exit(1);
        }

        // unlock the mutex
        if(pthread_mutex_unlock(&mutex) < 0)
        {
            perror("pthread_mutex_unlock");
            exit(1);
        }
    }
    return 0;
}// http://programmingnotes.org/
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
				
						// ============================================================================
//    Author: Kenneth Perkins
//    Date:   Oct 3, 2013
//    Taken From: http://programmingnotes.org/
//    File:  Condvar.cpp
//    Description: Demonstrate the use of the producer consumer problem
//       using mutex and condition variables
// ============================================================================
#include <iostream>
#include <cstdlib>
#include <pthread.h>
using namespace std;
 
// Compile & Run
// g++ Condvar.cpp -lpthread -o Condvar
// ./Condvar
 
// function prototypes
void* Produce(void* arg);
void* Consume(void* arg);
 
// global variables
// the mutex and the condition variable
pthread_mutex_t mutex;
pthread_cond_t cond;
 
// the condition flag
bool condition = false;
 
// the item produced
int count = 0;
 
// how much to produce
const int NUM_TO_PRODUCE = 6;
 
int main()
{
    // declare variables
    pthread_t producerThread;
    pthread_t consumerThread;
 
    // initialize the mutex and the condition variable
    pthread_mutex_init(&mutex, NULL);
    pthread_cond_init(&cond, NULL);
 
    cout <<"\nThe Parent is creating the producer and consumer threads..\n" << endl;
 
    // create a producer thread
    if(pthread_create(&producerThread, NULL, Produce, NULL) < 0)
    {
        perror("pthread_create");
        exit(1);
    }
 
    // create a consumer thread
    if(pthread_create(&consumerThread, NULL, Consume, NULL) < 0)
    {
        perror("pthread_create");
        exit(1);
    }
 
    // wait for the threads to complete
    // similar to the wait() system call for fork() processes
    if(pthread_join(producerThread, NULL) < 0)
    {
        perror("pthread_join");
        exit(1);
    }
 
    if(pthread_join(consumerThread, NULL) < 0)
    {
        perror("pthread_join");
        exit(1);
    }
 
    cout <<"\nBoth threads have completed and have terminated!\n"
        <<"\nThe Parent is now exiting...\n";
 
    return 0;
}// end of main
 
/**
 * The producer thread function
 * @param arg - pointer to the thread local data - unused
 */
void* Produce(void* arg)
{
    // produce things until the loop condition is met
    while(count < NUM_TO_PRODUCE)
    {
        // lock the mutex to protect the condition variable
        if(pthread_mutex_lock(&mutex) < 0)
        {
            perror("pthread_mutex_lock");
            exit(1);
        }
 
        // we have produced something that has not been
        // consumed yet, so we sleep (wait) until the consumer
        // wakes us up.
        while(condition)
        {
            // sleep (wait) on a condition variable until the
            // the consumer wakes the producer up.
            if(pthread_cond_wait(&cond, &mutex) < 0)
            {
                perror("pthread_cond_wait");
                exit(1);
            }
        }
 
        // produce an item
        cout <<"Produced: "<<++count<<endl;
 
        // we have produced something
        condition = true;
 
        // wake up the sleeping consumer
        if(pthread_cond_signal(&cond) < 0)
        {
            perror("pthread_cond_signal");
            exit(1);
        }
 
        // release the mutex lock
        if(pthread_mutex_unlock(&mutex) < 0)
        {
            perror("pthread_mutex_unlock");
            exit(1);
        }
    }
    return 0;
}// end of Produce
 
/**
 * The consumer function
 * @param arg - pointer to the thread local data - unused
 */
void* Consume(void* arg)
{
    // consume things until the loop condition is met
    while(count < NUM_TO_PRODUCE)
    {
        // lock the mutex to protect the condition variable
        if(pthread_mutex_lock(&mutex) < 0)
        {
            perror("pthread_mutex_lock");
            exit(1);
        }
 
        // if there is nothing to consume, then sleep
        while(!condition)
        {
            // sleep (wait) on a condition variable until the
            // producer wakes the consumer up.
            if(pthread_cond_wait(&cond, &mutex) < 0)
            {
                perror("pthread_cond_wait");
                exit(1);
            }
        }
 
        cout <<"Consumed: "<<count<<endl<<endl;
 
        // consume the item
        condition = false;
 
        // wake up the sleeping producer
        if(pthread_cond_signal(&cond) < 0)
        {
            perror("pthread_cond_signal");
            exit(1);
        }
 
        // unlock the mutex
        if(pthread_mutex_unlock(&mutex) < 0)
        {
            perror("pthread_mutex_unlock");
            exit(1);
        }
    }
    return 0;
}// http://programmingnotes.org/

QUICK NOTES:
The highlighted lines are sections of interest to look out for.

To use pthreads, the compiler flag “-lpthread” must be set.

The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.

The following is sample output:

The Parent is creating the producer and consumer threads..
Produced: 1 Consumed: 1 Produced: 2 Consumed: 2 Produced: 3 Consumed: 3 Produced: 4 Consumed: 4 Produced: 5 Consumed: 5 Produced: 6 Consumed: 6 Both threads have completed and have terminated!
The Parent is now exiting...

C++ || Snippet – How To Create And Use Threads For Interprocess Communication

October 03, 2013 admin No comments

The following is sample code which demonstrates the use of POSIX threads (pthreads), aswell as the pthread_create, and pthread_join function calls on Unix based systems.

Much like the fork() function call which is used to create new processes, threads are similar in that they too are used for interprocess communication. Threads allow for multi-threading, which is a widespread programming and execution model that allows for multiple threads to exist within the same context of a single program.

Threads share the calling parent process’ resources. Each process has it’s own address space, but the threads within the same process share that address space. Threads also share any other resources within that process. This means that it’s very easy to share data amongst threads, but it’s also easy for the threads to step on each other, which can lead to bad things.

The example on this page demonstrates the use of multiple pthreads to display shared data (an integer variable) to the screen.





		
		
			How To Use Multiple Pthreads
			
C++
			
			// ============================================================================
//    Author: Kenneth Perkins
//    Date:   Oct 3, 2013
//    Taken From: http://programmingnotes.org/
//    File:  Pthread.cpp
//    Description: Demonstrate the use of using multiple threads using the 
//       pthread_create() function
// ============================================================================
#include <iostream>
#include <cstdlib>
#include <pthread.h>
using namespace std;

// Compile & Run
// g++ Pthread.cpp -lpthread -o Pthread
// ./Pthread <NUMBER OF THREADS TO CREATE>

// function  prototype
void* ThreadFunc(void* args);

// global variable
int threadNum = 0;

int main(int argc, char* argv[])
{
    // declare variables
    int numberOfThreads = 0;
    pthread_t* threads;

    // make sure theres enough commandline arguments
    // and EXIT if theres not
    if(argc < 2)
    {
        cout <<"** ERROR - NOT ENOUGH ARGS!\n"
            <<"\nUSAGE:"<<argv[0]<<" <NUMBER OF THREADS TO CREATE>\n";
        exit(1);
    }

    // convert the number of threads from the command line
    // from string to int value
    numberOfThreads = atoi(argv[1]);

    // declare the array of thread variables 
    threads = new pthread_t[numberOfThreads];
    
    cout <<"\nThe Parent is creating "<<numberOfThreads<<" threads!" << endl;

    // use a loop to create the threads
    for(int x = 0; x < numberOfThreads; ++x)
    {
        // create the thread and call the "ThreadFunc"
        if(pthread_create(&threads[x], NULL, ThreadFunc, NULL) < 0)
        {
            perror("pthread_create error");
            exit(1);
        }
    }
    
    cout <<"The Parent is now waiting for the thread(s) to complete...\n\n";

    // wait for all threads to finish
    for(int x = 0; x < numberOfThreads; ++x)
    {
        pthread_join(threads[x], NULL);
    }
    
    cout <<"\nAll thread(s) are complete and have terminated!\n"
        <<"\nThe Parent is now exiting...\n";

    return 0;
}// end of main

/**
 * The function called by the thread
 * @param args - pointer to the thread local data
 */
void* ThreadFunc(void* args)
{	
    // display the thread info
    cout <<"Hi, Im thread #"<<++threadNum
        <<" and this is my id number: "<<(unsigned)pthread_self()<<endl;
    return 0;
}// http://programmingnotes.org/
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
				
						// ============================================================================
//    Author: Kenneth Perkins
//    Date:   Oct 3, 2013
//    Taken From: http://programmingnotes.org/
//    File:  Pthread.cpp
//    Description: Demonstrate the use of using multiple threads using the 
//       pthread_create() function
// ============================================================================
#include <iostream>
#include <cstdlib>
#include <pthread.h>
using namespace std;
 
// Compile & Run
// g++ Pthread.cpp -lpthread -o Pthread
// ./Pthread <NUMBER OF THREADS TO CREATE>
 
// function  prototype
void* ThreadFunc(void* args);
 
// global variable
int threadNum = 0;
 
int main(int argc, char* argv[])
{
    // declare variables
    int numberOfThreads = 0;
    pthread_t* threads;
 
    // make sure theres enough commandline arguments
    // and EXIT if theres not
    if(argc < 2)
    {
        cout <<"** ERROR - NOT ENOUGH ARGS!\n"
            <<"\nUSAGE:"<<argv[0]<<" <NUMBER OF THREADS TO CREATE>\n";
        exit(1);
    }
 
    // convert the number of threads from the command line
    // from string to int value
    numberOfThreads = atoi(argv[1]);
 
    // declare the array of thread variables 
    threads = new pthread_t[numberOfThreads];
    
    cout <<"\nThe Parent is creating "<<numberOfThreads<<" threads!" << endl;
 
    // use a loop to create the threads
    for(int x = 0; x < numberOfThreads; ++x)
    {
        // create the thread and call the "ThreadFunc"
        if(pthread_create(&threads[x], NULL, ThreadFunc, NULL) < 0)
        {
            perror("pthread_create error");
            exit(1);
        }
    }
    
    cout <<"The Parent is now waiting for the thread(s) to complete...\n\n";
 
    // wait for all threads to finish
    for(int x = 0; x < numberOfThreads; ++x)
    {
        pthread_join(threads[x], NULL);
    }
    
    cout <<"\nAll thread(s) are complete and have terminated!\n"
        <<"\nThe Parent is now exiting...\n";
 
    return 0;
}// end of main
 
/**
 * The function called by the thread
 * @param args - pointer to the thread local data
 */
void* ThreadFunc(void* args)
{	
    // display the thread info
    cout <<"Hi, Im thread #"<<++threadNum
        <<" and this is my id number: "<<(unsigned)pthread_self()<<endl;
    return 0;
}// http://programmingnotes.org/

QUICK NOTES:
The highlighted lines are sections of interest to look out for.

So, what is the difference between fork() processes and thread processes? Threads differ from traditional multitasking operating system processes in that:

• processes are typically independent, while threads exist as subsets of a process • processes carry considerable state information, whereas multiple threads within a process share state as well as memory and other resources • processes have separate address spaces, whereas threads share their address space • processes interact only through system-provided inter-process communication mechanisms. • Context switching between threads in the same process is typically faster than context switching between processes.

To use pthreads, the compiler flag “-lpthread” must be set.

The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.

The following is sample output:

./Pthread 5
The Parent is creating 5 threads! The Parent is now waiting for the thread(s) to complete... Hi, Im thread #1 and this is my id number: 1664468736 Hi, Im thread #2 and this is my id number: 1647683328 Hi, Im thread #3 and this is my id number: 1656076032 Hi, Im thread #4 and this is my id number: 1672861440 Hi, Im thread #5 and this is my id number: 1681254144 All thread(s) are complete and have terminated!
The Parent is now exiting...

C++ || Snippet – How To Override The Default Signal Handler (CTRL-C)

October 03, 2013 admin No comments

The following is sample code which demonstrates the use of the “signal” function call on Unix based systems.

Signals are interrupts delivered to a process by the operating system which can terminate a program prematurely. You can generate interrupts by pressing Ctrl+C. The “signal” function call receives two arguments. The first argument is an integer which represents the signal number, and the second argument is a pointer to the user defined signal handling function.

The following program catches the “SIGINT” signal number using the signal() function.





		
		
			How To Override CTRL-C
			
C++
			
			// ============================================================================
//    Author: K Perkins
//    Date:   Oct 3, 2013
//    Taken From: http://programmingnotes.org/
//    File:  Signal.cpp
//    Description: Demonstrate the use of overriding the default signal 
//       handler for the case when the user presses Ctrl-C. Test it by 
//       running and pressing Ctrl-C
// ============================================================================
#include <iostream>
#include <csignal>
#include <unistd.h>
using namespace std;

// function prototype
void SignalHandlerFunc(int arg);

// global variable
int count = 10;

int main()
{
    // overide the default signal handler (CTRL-C)
    // with our own "SignalHandlerFunc" function
    signal(SIGINT, SignalHandlerFunc);

    cerr<<"nPlease press CTRL-Cnn";

    // loop until condition is met
    do{
        sleep(1);
    }while(count > 0);

    // press ENTER on the keyboard to end
    // the program once all "lives" are lost
    cin.get();

    return 0;
}// end of main

/**
 * This function handles the signal
 * @param arg - the signal number
 */
void SignalHandlerFunc(int arg)
{
    // display text when user presses CTRL-C
    if(count > 0)
    {
        cerr<<" Haha I have "<<count<<" lives!n";
    }
    else
    {
        cerr<<" ** Ahh you got me...n";
    }
    --count;
}// http://programmingnotes.org/
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
				
						// ============================================================================
//    Author: K Perkins
//    Date:   Oct 3, 2013
//    Taken From: http://programmingnotes.org/
//    File:  Signal.cpp
//    Description: Demonstrate the use of overriding the default signal 
//       handler for the case when the user presses Ctrl-C. Test it by 
//       running and pressing Ctrl-C
// ============================================================================
#include <iostream>
#include <csignal>
#include <unistd.h>
using namespace std;
 
// function prototype
void SignalHandlerFunc(int arg);
 
// global variable
int count = 10;
 
int main()
{
    // overide the default signal handler (CTRL-C)
    // with our own "SignalHandlerFunc" function
    signal(SIGINT, SignalHandlerFunc);
 
    cerr<<"nPlease press CTRL-Cnn";
 
    // loop until condition is met
    do{
        sleep(1);
    }while(count > 0);
 
    // press ENTER on the keyboard to end
    // the program once all "lives" are lost
    cin.get();
 
    return 0;
}// end of main
 
/**
 * This function handles the signal
 * @param arg - the signal number
 */
void SignalHandlerFunc(int arg)
{
    // display text when user presses CTRL-C
    if(count > 0)
    {
        cerr<<" Haha I have "<<count<<" lives!n";
    }
    else
    {
        cerr<<" ** Ahh you got me...n";
    }
    --count;
}// http://programmingnotes.org/

QUICK NOTES:
The highlighted lines are sections of interest to look out for.

The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.

The following is sample output:

Please press CTRL-C
^C Haha I have 10 lives! ^C Haha I have 9 lives! ^C Haha I have 8 lives! ^C Haha I have 7 lives! ^C Haha I have 6 lives! ^C Haha I have 5 lives! ^C Haha I have 4 lives! ^C Haha I have 3 lives! ^C Haha I have 2 lives! ^C Haha I have 1 lives! ^C ** Ahh you got me...

C++ || Multi-Hash Interprocess Communication Using Fork, Popen, & Pipes

August 20, 2013 admin No comments

The following is another homework assignment which was presented in an Operating Systems Concepts class. Using two pipes, the following is a program which implements the computing of hash values on a file using the MD5, SHA1, SHA224, SHA256, SHA384, and SHA512 hashing algorithms provided on Unix based systems.

REQUIRED KNOWLEDGE FOR THIS PROGRAM

How To Use Fork How To Use Pipe How To Use Popen

==== 1. OVERVIEW ====

Hash algorithms map large data sets of variable length (e.g. files), to data sets of a fixed length. For example, the contents of a 1GB file may be hashed into a single 128-bit integer. Many hash algorithms exhibit an important property called an avalanche effect – slight changes in the input data trigger significant changes in the hash value.

Hash algorithms are often used for verifying the integrity of files downloaded from the WEB. For example, websites hosting a file usually post the hash value of the file using the MD5 hash algorithm. By doing this, the user can then verify the integrity of the downloaded file by computing the MD5 algorithm on their own, and compare their hash value against the hash value posted on the website. The user will know if the download was valid only if the two hash values match.

==== 2. TECHNICAL DETAILS ====

The following implements a program for computing the hash value of a file using the MD5, SHA1, SHA224, SHA256, SHA384, and SHA512 hashing algorithms provided on Unix based systems.

This program takes the name of the target file being analyzed as a command line argument, and does the following:

1. Check to make sure the file exists. 2. Create two pipes. 3. Create a child process. 4. The parent transmits the name of the file to the child (over the first pipe). 5. The child receives the name of the file and computes the hash of the file using the MD5 algorithm (using Linux program md5sum). 6. The child transmits the computed hash to the parent (over the second pipe) and terminates. 7. The parent receives the hash, prints it, and calls wait(). 8. Repeat the same process starting with step 3, but using algorithms SHA1...SHA512. 9. The parent terminates after all hashes have been computed.

The use of the popen function is used in order to launch the above programs and capture their output into a character array buffer.

This program also uses two pipes. The two pipes created are the following:

(1) Parent to child pipe: Used by the parent to transfer the name of the file to the child. The parent writes to this pipe and the child reads it.
(2) Child to parent pipe: Used by the child to transfer the computed hashes to the parent. The child writes to this pipe and the parent reads it.





		
		
			Multi-Hash Using Fork, Popen, & Pipes
			
C++
			
			// =============================================================================
//    Author: K Perkins
//    Date:   Aug 20, 2013
//    Taken From: http://programmingnotes.org/
//    File: Multihash.cpp
//    Description: Hash algorithms map large data sets of variable length (e.g. 
//       files), to data sets of a fixed length. For example, contents of a 
//       1GB file may be hashed into a single 128-bit integer. Using 2 pipes, 
//       this program implements the computing of hash values on a file using 
//       the MD5, SHA1, SHA224, SHA256, SHA384, and SHA512 hashing algorithms.
//
//                   The two pipes created are the following:
//       (1) Parent to child pipe: Used by the parent to transfer the name of 
//             the file to the child. The parent writes to this pipe and the 
//             child reads it.
//       (2) Child to parent pipe: Used by the child to transfer the computed 
//             hashes to the parent. The child writes to this pipe and the 
//             parent reads it.
// =============================================================================
#include <iostream>
#include <fstream>
#include <cstdlib>
#include <cstring>
#include <unistd.h>
#include <sys/wait.h>
using namespace std;

// compile: g++ Multihash.cpp -o Multihash
// run: ./Multihash <file name>

// parentToChild = pipe for parent to child communication
// childToParent = pipe for child to parent communication
int parentToChild[2];
int childToParent[2];

// names of the hash programs
const char hashAlgs[6][10] = {"md5sum", "sha1sum", "sha224sum", 
            "sha256sum", "sha384sum", "sha512sum"};

// read end of pipe
const int READ_END = 0;

// write end of pipe 
const int WRITE_END = 1;

// number of hash programs 
const int NUM_HASH_ALGS = sizeof(hashAlgs)/sizeof(hashAlgs[0]);

// maximum length of hash value 
const int HASH_LENGTH = 1000;

// maximum length of file name 
const int FILENAME_LENGTH = 100;

// child function which computes the hash of a file and 
// returns the value back to the parent
// @param 'hashAlg' == the name of the hash program
void ComputeHash(const char hashAlg[]);

int main(int argc, char* argv[])
{  
    // declare variables
    ifstream infile;  // used to see if file exists 	
    char hashValue[HASH_LENGTH]; // hash value which child passes to parent   

    // check to see if theres enough commandline args 
    if(argc < 2)
    {
        cerr<<"n** ERROR - NOT ENOUGH ARGUMENTS!n"
            <<"nUSAGE: "<<argv[0]<<" <FILE NAME>n";
        exit(1);
    } 

    // try to open file
    infile.open(argv[1]);
   
    // if file doesnt exist, then exit
    if(infile.fail())
    {
        cerr<<"n** ERROR!n"        
            <<"Cant find the file ""<<argv[1]<<""!nn";
        exit(1);
    }
    infile.close();
    
    // start the piping process
    for(int currentAlg = 0; currentAlg < NUM_HASH_ALGS; ++currentAlg)
    {
        // create pipes
        if((pipe(parentToChild) < 0) || (pipe(childToParent) < 0))
        {
            cerr << "npipe failedn";
            exit(1);
        }	
        // create & fork a child
        pid_t pid = fork();

        // make sure fork succeeded 
        if(pid < 0)
        {
            cerr << "nfork failedn";
            exit(1);
        }		
        // child process	     
        else if(pid == 0) 
        { 
            ComputeHash(hashAlgs[currentAlg]);
        }	
        // parent process 
        else 
        {  
            // close selected pipes ends        	    	    
            close(parentToChild[READ_END]);
            close(childToParent[WRITE_END]);
            
            // pass the filename to the child	    
            write(parentToChild[WRITE_END], argv[1], strlen(argv[1])+1);
            close(parentToChild[WRITE_END]);
            
            // read the incoming hash value from the child
            read(childToParent[READ_END], hashValue, sizeof(hashValue));
            close(childToParent[READ_END]);
            
            // wait for child to complete
            wait(NULL);
            
            // display current hash to the screen
            cout<<"Hash Algorithm #"<<currentAlg+1<<":n"<<hashAlgs[currentAlg]
                    <<" - HASH VALUE: "<<hashValue<<endl;
                    
            // reset hash buffer
            memset(hashValue, (char)NULL, sizeof(hashValue));		            	     
        }  
    }	

    cerr<<"The parent process is now exiting...n";		
    return 0;	  
}// end of main

void ComputeHash(const char hashAlg[])
{
    // declare variables
    char recievedFileName[FILENAME_LENGTH]; // saves recieved filename
    char cmdLine[FILENAME_LENGTH]; // saves command line for popen
    char hashOutput[HASH_LENGTH]; // saves final hash output
    FILE* popenOutput; // popen file pointer
    
    // close selected pipes ends
    close(parentToChild[WRITE_END]);
    close(childToParent[READ_END]);

    // get filename from parent
    read(parentToChild[READ_END], recievedFileName, sizeof(recievedFileName));
    close(parentToChild[READ_END]);    
    
    // construct command line argument to pass to popen
    strncpy(cmdLine, hashAlg, sizeof(cmdLine));
    strncat(cmdLine, " ", sizeof(cmdLine));
    strncat(cmdLine, recievedFileName, sizeof(cmdLine));
    
    // get has value for current hash prog
    popenOutput = popen(cmdLine, "r");
    
    // make sure that popen succeeded 
    if(!popenOutput)
    {
        cerr<<"npopen failedn";
        exit(1);
    } 

    // set hash buffer to all NULLS 
    memset(hashOutput, (char)NULL, sizeof(hashOutput));	

    // read program output into buffer
    fread(hashOutput, sizeof(char), sizeof(char)*sizeof(hashOutput), popenOutput);

    // close popen buffer
    pclose(popenOutput);

    // pass hash value back to parent
    write(childToParent[WRITE_END], hashOutput, strlen(hashOutput)+1);
    close(childToParent[WRITE_END]); 
    
    // exit 'AKA' kill child   
    exit(0);
}// http://programmingnotes.org/
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
				
						// =============================================================================
//    Author: K Perkins
//    Date:   Aug 20, 2013
//    Taken From: http://programmingnotes.org/
//    File: Multihash.cpp
//    Description: Hash algorithms map large data sets of variable length (e.g. 
//       files), to data sets of a fixed length. For example, contents of a 
//       1GB file may be hashed into a single 128-bit integer. Using 2 pipes, 
//       this program implements the computing of hash values on a file using 
//       the MD5, SHA1, SHA224, SHA256, SHA384, and SHA512 hashing algorithms.
//
//                   The two pipes created are the following:
//       (1) Parent to child pipe: Used by the parent to transfer the name of 
//             the file to the child. The parent writes to this pipe and the 
//             child reads it.
//       (2) Child to parent pipe: Used by the child to transfer the computed 
//             hashes to the parent. The child writes to this pipe and the 
//             parent reads it.
// =============================================================================
#include <iostream>
#include <fstream>
#include <cstdlib>
#include <cstring>
#include <unistd.h>
#include <sys/wait.h>
using namespace std;
 
// compile: g++ Multihash.cpp -o Multihash
// run: ./Multihash <file name>
 
// parentToChild = pipe for parent to child communication
// childToParent = pipe for child to parent communication
int parentToChild[2];
int childToParent[2];
 
// names of the hash programs
const char hashAlgs[6][10] = {"md5sum", "sha1sum", "sha224sum", 
            "sha256sum", "sha384sum", "sha512sum"};
 
// read end of pipe
const int READ_END = 0;
 
// write end of pipe 
const int WRITE_END = 1;
 
// number of hash programs 
const int NUM_HASH_ALGS = sizeof(hashAlgs)/sizeof(hashAlgs[0]);
 
// maximum length of hash value 
const int HASH_LENGTH = 1000;
 
// maximum length of file name 
const int FILENAME_LENGTH = 100;
 
// child function which computes the hash of a file and 
// returns the value back to the parent
// @param 'hashAlg' == the name of the hash program
void ComputeHash(const char hashAlg[]);
 
int main(int argc, char* argv[])
{  
    // declare variables
    ifstream infile;  // used to see if file exists 	
    char hashValue[HASH_LENGTH]; // hash value which child passes to parent   
 
    // check to see if theres enough commandline args 
    if(argc < 2)
    {
        cerr<<"n** ERROR - NOT ENOUGH ARGUMENTS!n"
            <<"nUSAGE: "<<argv[0]<<" <FILE NAME>n";
        exit(1);
    } 
 
    // try to open file
    infile.open(argv[1]);
   
    // if file doesnt exist, then exit
    if(infile.fail())
    {
        cerr<<"n** ERROR!n"        
            <<"Cant find the file ""<<argv[1]<<""!nn";
        exit(1);
    }
    infile.close();
    
    // start the piping process
    for(int currentAlg = 0; currentAlg < NUM_HASH_ALGS; ++currentAlg)
    {
        // create pipes
        if((pipe(parentToChild) < 0) || (pipe(childToParent) < 0))
        {
            cerr << "npipe failedn";
            exit(1);
        }	
        // create & fork a child
        pid_t pid = fork();
 
        // make sure fork succeeded 
        if(pid < 0)
        {
            cerr << "nfork failedn";
            exit(1);
        }		
        // child process	     
        else if(pid == 0) 
        { 
            ComputeHash(hashAlgs[currentAlg]);
        }	
        // parent process 
        else 
        {  
            // close selected pipes ends        	    	    
            close(parentToChild[READ_END]);
            close(childToParent[WRITE_END]);
            
            // pass the filename to the child	    
            write(parentToChild[WRITE_END], argv[1], strlen(argv[1])+1);
            close(parentToChild[WRITE_END]);
            
            // read the incoming hash value from the child
            read(childToParent[READ_END], hashValue, sizeof(hashValue));
            close(childToParent[READ_END]);
            
            // wait for child to complete
            wait(NULL);
            
            // display current hash to the screen
            cout<<"Hash Algorithm #"<<currentAlg+1<<":n"<<hashAlgs[currentAlg]
                    <<" - HASH VALUE: "<<hashValue<<endl;
                    
            // reset hash buffer
            memset(hashValue, (char)NULL, sizeof(hashValue));		            	     
        }  
    }	
 
    cerr<<"The parent process is now exiting...n";		
    return 0;	  
}// end of main
 
void ComputeHash(const char hashAlg[])
{
    // declare variables
    char recievedFileName[FILENAME_LENGTH]; // saves recieved filename
    char cmdLine[FILENAME_LENGTH]; // saves command line for popen
    char hashOutput[HASH_LENGTH]; // saves final hash output
    FILE* popenOutput; // popen file pointer
    
    // close selected pipes ends
    close(parentToChild[WRITE_END]);
    close(childToParent[READ_END]);
 
    // get filename from parent
    read(parentToChild[READ_END], recievedFileName, sizeof(recievedFileName));
    close(parentToChild[READ_END]);    
    
    // construct command line argument to pass to popen
    strncpy(cmdLine, hashAlg, sizeof(cmdLine));
    strncat(cmdLine, " ", sizeof(cmdLine));
    strncat(cmdLine, recievedFileName, sizeof(cmdLine));
    
    // get has value for current hash prog
    popenOutput = popen(cmdLine, "r");
    
    // make sure that popen succeeded 
    if(!popenOutput)
    {
        cerr<<"npopen failedn";
        exit(1);
    } 
 
    // set hash buffer to all NULLS 
    memset(hashOutput, (char)NULL, sizeof(hashOutput));	
 
    // read program output into buffer
    fread(hashOutput, sizeof(char), sizeof(char)*sizeof(hashOutput), popenOutput);
 
    // close popen buffer
    pclose(popenOutput);
 
    // pass hash value back to parent
    write(childToParent[WRITE_END], hashOutput, strlen(hashOutput)+1);
    close(childToParent[WRITE_END]); 
    
    // exit 'AKA' kill child   
    exit(0);
}// http://programmingnotes.org/

QUICK NOTES:
The highlighted lines are sections of interest to look out for.

The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.

Using the following example input file located here, the following is sample output:





		
		
			Hash Algorithm #1:
md5sum - HASH VALUE: cd2f02cc5e50bd08d83cef630a32d7d6  INPUT_Dictionary_programmingnotes_freeweq_com.txt

Hash Algorithm #2:
sha1sum - HASH VALUE: 1e1d2fd77b331fc2b2f24822fdf2133f3678d662  INPUT_Dictionary_programmingnotes_freeweq_com.txt

Hash Algorithm #3:
sha224sum - HASH VALUE: bf69da0f9f9990cece8dd6800a25a9faca5381c0c62e0667c258e8d5  INPUT_Dictionary_programmingnotes_freeweq_com.txt

Hash Algorithm #4:
sha256sum - HASH VALUE: a8cd7a082ce571f7e66bc4d5eea8f71e7e455735922c90c060580b51d43593ec  INPUT_Dictionary_programmingnotes_freeweq_com.txt

Hash Algorithm #5:
sha384sum - HASH VALUE: a564df501470a43eebb51c28f83ab07090e39c434fc40bf496622f536c525140301633d6513dec83ea512bcb38c4e2e6  INPUT_Dictionary_programmingnotes_freeweq_com.txt

Hash Algorithm #6:
sha512sum - HASH VALUE: e686a709b3025f8b21fd6d40ce92f741dd8644335e935baced86098ff3e7278b5443e28f02fa886e7c0cd391ce6e8aa91842a6de29535a40453d41862a0bc1c7  INPUT_Dictionary_programmingnotes_freeweq_com.txt

The parent process is now exiting...
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
				
						Hash Algorithm #1:
md5sum - HASH VALUE: cd2f02cc5e50bd08d83cef630a32d7d6  INPUT_Dictionary_programmingnotes_freeweq_com.txt
 
Hash Algorithm #2:
sha1sum - HASH VALUE: 1e1d2fd77b331fc2b2f24822fdf2133f3678d662  INPUT_Dictionary_programmingnotes_freeweq_com.txt
 
Hash Algorithm #3:
sha224sum - HASH VALUE: bf69da0f9f9990cece8dd6800a25a9faca5381c0c62e0667c258e8d5  INPUT_Dictionary_programmingnotes_freeweq_com.txt
 
Hash Algorithm #4:
sha256sum - HASH VALUE: a8cd7a082ce571f7e66bc4d5eea8f71e7e455735922c90c060580b51d43593ec  INPUT_Dictionary_programmingnotes_freeweq_com.txt
 
Hash Algorithm #5:
sha384sum - HASH VALUE: a564df501470a43eebb51c28f83ab07090e39c434fc40bf496622f536c525140301633d6513dec83ea512bcb38c4e2e6  INPUT_Dictionary_programmingnotes_freeweq_com.txt
 
Hash Algorithm #6:
sha512sum - HASH VALUE: e686a709b3025f8b21fd6d40ce92f741dd8644335e935baced86098ff3e7278b5443e28f02fa886e7c0cd391ce6e8aa91842a6de29535a40453d41862a0bc1c7  INPUT_Dictionary_programmingnotes_freeweq_com.txt
 
The parent process is now exiting...

C++ || Snippet – How To Use Popen & Save Results Into A Character Array

August 20, 2013 admin No comments

The following is sample code which demonstrates the use of the “popen” function call on Unix based systems.

The “popen” function call opens a process by creating a pipe, forking, and invoking the shell. It does this by executing the command specified by the incoming string function parameter. It creates a pipe between the calling program and the executed command, and returns a pointer to a stream that can be used to either read from or write to the pipe.

The following example demonstrates how to save the results of the popen command into a char array.





		
		
			How To Use Popen & Save Output Into An Array
			
C++
			
			// ============================================================================
//    Author: Kenneth Perkins
//    Date:   Aug 20, 2013
//    Taken From: http://programmingnotes.org/
//    File:  PopenExample.cpp
//    Description: Demonstrate the use of the popen() command and demonstrate
//      how to save the results into a char array.
// ============================================================================
#include <iostream>
#include <cstdio>
#include <cstring>
#include <cstdlib>
using namespace std;

// the maximum output size
const int MAX_OUTPUT_SIZE = 1000;

int main(int argc, char* argv[])
{
    // declare variables
    char buffer[MAX_OUTPUT_SIZE]; // a char array to display the popen output
    FILE* progOutput; // a file pointer representing the popen output

    // launch the "md5sum" program to compute the MD5 hash of
    // the file "/bin/ls" and save it into the file pointer
    progOutput = popen("md5sum /bin/ls", "r");

    // make sure that popen succeeded
    if (!progOutput)
    {
        perror("popen");
        exit(1);
    }

    // reset buffer to all NULLS
    memset(buffer, (char)NULL, sizeof(buffer));

    // read the popen output into the char array buffer
    if (fread(buffer, sizeof(char), sizeof(char) * sizeof(buffer), progOutput) < 0)
    {
        perror("fread");
        exit(1);
    }

    // close the file pointer representing the popen output
    if (pclose(progOutput) < 0)
    {
        perror("pclose");
        exit(1);
    }

    cout << "Program output: " << buffer << endl;

    return 0;
}// http://programmingnotes.org/
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
				
						// ============================================================================
//    Author: Kenneth Perkins
//    Date:   Aug 20, 2013
//    Taken From: http://programmingnotes.org/
//    File:  PopenExample.cpp
//    Description: Demonstrate the use of the popen() command and demonstrate
//      how to save the results into a char array.
// ============================================================================
#include <iostream>
#include <cstdio>
#include <cstring>
#include <cstdlib>
using namespace std;
 
// the maximum output size
const int MAX_OUTPUT_SIZE = 1000;
 
int main(int argc, char* argv[])
{
    // declare variables
    char buffer[MAX_OUTPUT_SIZE]; // a char array to display the popen output
    FILE* progOutput; // a file pointer representing the popen output
 
    // launch the "md5sum" program to compute the MD5 hash of
    // the file "/bin/ls" and save it into the file pointer
    progOutput = popen("md5sum /bin/ls", "r");
 
    // make sure that popen succeeded
    if (!progOutput)
    {
        perror("popen");
        exit(1);
    }
 
    // reset buffer to all NULLS
    memset(buffer, (char)NULL, sizeof(buffer));
 
    // read the popen output into the char array buffer
    if (fread(buffer, sizeof(char), sizeof(char) * sizeof(buffer), progOutput) < 0)
    {
        perror("fread");
        exit(1);
    }
 
    // close the file pointer representing the popen output
    if (pclose(progOutput) < 0)
    {
        perror("pclose");
        exit(1);
    }
 
    cout << "Program output: " << buffer << endl;
 
    return 0;
}// http://programmingnotes.org/

QUICK NOTES:
The highlighted lines are sections of interest to look out for.

The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.

The following is sample output:

Program output: fa97c59cc414e42d4e0e853ddf5b4745 /bin/ls

C++ || Serial & Parallel Multi Process File Downloader Using Fork & Execlp

August 19, 2013 admin 3 Comments

The following is another homework assignment which was presented in an Operating Systems Concepts class. The following are two multi-process programs using commandline arguments, which demonstrates more practice using the fork() and execlp() system calls on Unix based systems.

==== 1. OVERVIEW ====

File downloaders are programs used for downloading ﬁles from the Internet. The following programs listed on this page implement two distinct type of multi-process downloaders:

1. a serial ﬁle downloader which downloads ﬁles one by one. 2. a parallel ﬁle downloader which dowloads multiple ﬁles in parallel.

In both programs, the parent process first reads a ﬁle via the commandline. This file which is read is the ﬁle that contains the list of URLs of the ﬁles to be downloaded. The incoming url file that is read has the following format:

[URL1] [URL2] . . . [URLN]

Where [URL] is an http internet link with a valid absolute file path extension.
(i.e: http://newsimg.ngfiles.com/270000/270173_0204618900-cc-asmbash.jpg)

After the url file is parsed, next the parent process forks a child process. Each created child process uses the execlp() system call to replace its executable image with that of the “wget” program. The use of the wget program performs the actual file downloading.

==== 2. SERIAL DOWNLOADER ====

The serial downloader downloads files one at a time. After the parent process has read and parsed the incoming url file from the commandline, the serial downloader proceeds as follows:

1. The parent forks off a child process. 2. The child uses execlp("/usr/bin/wget", "wget", [URL STRING1], NULL) system call in order to replace its program with wget program that will download the first file in urls.txt (i.e. the file at URL ). 3. The parent executes a wait() system call until the child exits. 4. The parent forks off another child which downloads the next file specified in url.txt. 5. Repeat the same process until all files are downloaded.

The following is implemented below:





		
		
			Serial File Downloader Using fork() & execlp('wget')
			
C++
			
			// ============================================================================
//    Author: Kenneth Perkins
//    Date:   Aug 19, 2013
//    Taken From: http://programmingnotes.org/
//    File:  Serial.cpp
//    Description: File downloaders are programs used for downloading files
//       from the Internet. Using the fork() & execlp("wget") command, the
//       following is a multi-process serial file downloader which reads an
//       input file containing url file download links as a commandline
//       argument and downloads the files located on the internet one by one.
// ============================================================================
#include <iostream>
#include <cstring>
#include <fstream>
#include <cstdlib>
#include <unistd.h>
#include <sys/wait.h>
using namespace std;

// compile & run
// g++ Serial.cpp -o Serial
// ./Serial urls.txt

int main(int argc, char* argv[])
{
    // declare variables
    pid_t pid = -1;
    int urlNumber = 0;
    char urlName[256];
    ifstream infile;

    // check if theres enough command line args
    if(argc < 2)
    {
        cout <<"\nERROR -- NOT ENOUGH ARGS!"
            <<"\n\nUSAGE: "<<argv[0]<<" <file containing url downloads>\n\n";
        exit(1);
    }

    // try to open the file containing the download url links
    // exit if the url file is not found
    infile.open(argv[1]);
    if(infile.fail())
    {
        cout <<"\nERROR -- "<<argv[1]<<" NOT FOUND!\n\n";
        exit(1);
    }

    // get download url links from the file
    while(infile.getline(urlName, sizeof(urlName)))
    {
        ++urlNumber;

        // fork another process
        pid = fork();

        if(pid < 0)
        {   // ** error occurred
            perror("fork");
            exit(1);
        }
        else if(pid == 0)
        {   // ** child process
            cout <<endl<<"** URL #"<<urlNumber
                    <<" is currently downloading... **\n\n";
            execlp("/usr/bin/wget", "wget", urlName, NULL);
        }
        else
        {   // ** parent process
            // parent will wait for the child to complete
            wait(NULL);
            cout <<endl<<"-- URL #"<<urlNumber<<" is complete! --\n";
        }
    }
    infile.close();

    cout <<endl<<"The parent process is now exiting...\n";

    return 0;
}// http://programmingnotes.org/
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
				
						// ============================================================================
//    Author: Kenneth Perkins
//    Date:   Aug 19, 2013
//    Taken From: http://programmingnotes.org/
//    File:  Serial.cpp
//    Description: File downloaders are programs used for downloading files
//       from the Internet. Using the fork() & execlp("wget") command, the
//       following is a multi-process serial file downloader which reads an
//       input file containing url file download links as a commandline
//       argument and downloads the files located on the internet one by one.
// ============================================================================
#include <iostream>
#include <cstring>
#include <fstream>
#include <cstdlib>
#include <unistd.h>
#include <sys/wait.h>
using namespace std;
 
// compile & run
// g++ Serial.cpp -o Serial
// ./Serial urls.txt
 
int main(int argc, char* argv[])
{
    // declare variables
    pid_t pid = -1;
    int urlNumber = 0;
    char urlName[256];
    ifstream infile;
 
    // check if theres enough command line args
    if(argc < 2)
    {
        cout <<"\nERROR -- NOT ENOUGH ARGS!"
            <<"\n\nUSAGE: "<<argv[0]<<" <file containing url downloads>\n\n";
        exit(1);
    }
 
    // try to open the file containing the download url links
    // exit if the url file is not found
    infile.open(argv[1]);
    if(infile.fail())
    {
        cout <<"\nERROR -- "<<argv[1]<<" NOT FOUND!\n\n";
        exit(1);
    }
 
    // get download url links from the file
    while(infile.getline(urlName, sizeof(urlName)))
    {
        ++urlNumber;
 
        // fork another process
        pid = fork();
 
        if(pid < 0)
        {   // ** error occurred
            perror("fork");
            exit(1);
        }
        else if(pid == 0)
        {   // ** child process
            cout <<endl<<"** URL #"<<urlNumber
                    <<" is currently downloading... **\n\n";
            execlp("/usr/bin/wget", "wget", urlName, NULL);
        }
        else
        {   // ** parent process
            // parent will wait for the child to complete
            wait(NULL);
            cout <<endl<<"-- URL #"<<urlNumber<<" is complete! --\n";
        }
    }
    infile.close();
 
    cout <<endl<<"The parent process is now exiting...\n";
 
    return 0;
}// http://programmingnotes.org/

Since the serial downloader downloads files one at a time, that can become very slow. That is where the parallel downloader comes in handy!

==== 3. PARALLEL DOWNLOADER ====

The parallel downloader downloads files all at once and is implemented much like the serial downloader. The parallel downloader proceeds as follows:

1. The parent forks off n children, where n is the number of URLs in url.txt. 2. Each child executes execlp("/usr/bin/wget", "wget", [URL STRING], NULL) system call where each is a distinct URL in url.txt. 3. The parent calls wait() (n times in a row) and waits for all children to terminate. 4. The parent exits.

The following is implemented below:





		
		
			Parallel File Downloader Using fork() & execlp('wget')
			
C++
			
			// ============================================================================
//    Author: Kenneth Perkins
//    Date:   Aug 19, 2013
//    Taken From: http://programmingnotes.org/
//    File:  Parallel.cpp
//    Description: File downloaders are programs used for downloading files
//       from the Internet. Using the fork() & execlp("wget") command, the
//       following is a multi-process parallel file downloader which reads an
//       input file containing url file download links as a commandline
//       argument and downloads the files located on the internet all at once.
// ============================================================================
#include <iostream>
#include <cstring>
#include <fstream>
#include <cstdlib>
#include <unistd.h>
#include <sys/wait.h>
using namespace std;

// compile & run
// g++ Parallel.cpp -o Parallel
// ./Parallel urls.txt

int main(int argc, char* argv[])
{
    // declare variables
    pid_t pid = -1;
    int urlNumber = 0;
    char urlName[256];
    ifstream infile;

    // check if theres enough command line args
    if(argc < 2)
    {
        cout <<"\nERROR -- NOT ENOUGH ARGS!"
            <<"\n\nUSAGE: "<<argv[0]<<" <file containing url downloads>\n\n";
        exit(1);
    }

    // try to open the file containing the download url links
    // exit if the url file is not found
    infile.open(argv[1]);
    if(infile.fail())
    {
        cout <<"\nERROR -- "<<argv[1]<<" NOT FOUND!\n\n";
        exit(1);
    }

    // get download url links from the file
    while(infile.getline(urlName, sizeof(urlName)))
    {
        ++urlNumber;

        // fork another process
        pid = fork();

        if(pid < 0)
        {   // ** error occurred
            perror("fork");
            exit(1);
        }
        else if(pid == 0)
        {   // ** child process
            cout <<endl<<"** URL #"<<urlNumber
                    <<" is currently downloading... **\n\n";
            execlp("/usr/bin/wget", "wget", urlName, NULL);
        }
    }
    infile.close();

    while(urlNumber > 0)
    {   // ** parent process
        // parent will wait for the child to complete
        wait(NULL);
        cout <<endl<<"-- URL #"<<urlNumber<<" is complete! --\n";
        --urlNumber;
    }

    cout <<endl<<"The parent process is now exiting...\n";

    return 0;
}// http://programmingnotes.org/
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
				
						// ============================================================================
//    Author: Kenneth Perkins
//    Date:   Aug 19, 2013
//    Taken From: http://programmingnotes.org/
//    File:  Parallel.cpp
//    Description: File downloaders are programs used for downloading files
//       from the Internet. Using the fork() & execlp("wget") command, the
//       following is a multi-process parallel file downloader which reads an
//       input file containing url file download links as a commandline
//       argument and downloads the files located on the internet all at once.
// ============================================================================
#include <iostream>
#include <cstring>
#include <fstream>
#include <cstdlib>
#include <unistd.h>
#include <sys/wait.h>
using namespace std;
 
// compile & run
// g++ Parallel.cpp -o Parallel
// ./Parallel urls.txt
 
int main(int argc, char* argv[])
{
    // declare variables
    pid_t pid = -1;
    int urlNumber = 0;
    char urlName[256];
    ifstream infile;
 
    // check if theres enough command line args
    if(argc < 2)
    {
        cout <<"\nERROR -- NOT ENOUGH ARGS!"
            <<"\n\nUSAGE: "<<argv[0]<<" <file containing url downloads>\n\n";
        exit(1);
    }
 
    // try to open the file containing the download url links
    // exit if the url file is not found
    infile.open(argv[1]);
    if(infile.fail())
    {
        cout <<"\nERROR -- "<<argv[1]<<" NOT FOUND!\n\n";
        exit(1);
    }
 
    // get download url links from the file
    while(infile.getline(urlName, sizeof(urlName)))
    {
        ++urlNumber;
 
        // fork another process
        pid = fork();
 
        if(pid < 0)
        {   // ** error occurred
            perror("fork");
            exit(1);
        }
        else if(pid == 0)
        {   // ** child process
            cout <<endl<<"** URL #"<<urlNumber
                    <<" is currently downloading... **\n\n";
            execlp("/usr/bin/wget", "wget", urlName, NULL);
        }
    }
    infile.close();
 
    while(urlNumber > 0)
    {   // ** parent process
        // parent will wait for the child to complete
        wait(NULL);
        cout <<endl<<"-- URL #"<<urlNumber<<" is complete! --\n";
        --urlNumber;
    }
 
    cout <<endl<<"The parent process is now exiting...\n";
 
    return 0;
}// http://programmingnotes.org/

QUICK NOTES:
The highlighted lines are sections of interest to look out for.

The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.

Also note, while the parallel downloader executes, the outputs from different children may intermingle.

C++ || Snippet – How To Use Fork & Pipe For Interprocess Communication

August 19, 2013 admin No comments

The following is sample code which demonstrates the use of the fork, read, and write function calls for use with pipes on Unix based systems.

A pipe is a mechanism for interprocess communication. Data written to a pipe by one process can be read by another process. Creating a pipe is achieved by using the pipe function, which creates both the reading and writing ends of the pipe file descriptor.

In typical use, a parent process creates a pipe just before it forks one or more child processes. The pipe is then used for communication between either the parent or child processes, or between two sibling processes.

A real world example of this kind of communication can be seen in all operating system terminal shells. When you type a command in a shell, it will spawn the executable represented by that command with a call to fork. A pipe is opened to the new child process, and its output is read and printed by the terminal.





		
		
			Fork & Pipe Interprocess Communication
			
C++
			
			// ============================================================================
//    Author: Kenneth Perkins
//    Date:   Aug 19, 2013
//    Taken From: http://programmingnotes.org/
//    File:  ForkPipeExample.cpp
//    Description: Demonstrate the use of the fork() and pipe() command for
//        parent and child interprocess communication. 
// ============================================================================
#include <iostream>
#include <cstdlib>
#include <cstring>
#include <sys/wait.h>
#include <unistd.h>
using namespace std;

// global variables
const int BUFFER_SIZE = 256;
const int READ_END = 0;
const int WRITE_END = 1; 

int main() 
{
    // declare variables
    int fd[2]; // pipe file descriptor
    pid_t pid = -1;     
    char writeMsg[BUFFER_SIZE] = "Greetings From Your Parent!";
    char readMsg[BUFFER_SIZE];
    
    // create a pipe
    if(pipe(fd) < 0) 
    {
        perror("pipe");
        exit(1);
    }
    
    cout <<"\nParent is forking a child." << endl;

    // create a duplicate process of this current program
    pid = fork();
    
    // exit if something went wrong with the fork
    if(pid < 0) 
    {
        perror("fork");
        exit(1);
    }
    // this code only gets executed by the child process
    else if(pid == 0)
    {
        cout <<"\nStarting the child process..\n\n";
        // close the "write" end b/c we arent using it
        close(fd[WRITE_END]);
        // get data from the pipe using the read() function
        read(fd[READ_END], readMsg, sizeof(readMsg));
        // close the "read" end
        close(fd[READ_END]);
        cout <<"Message from the parent via the pipe: "<<readMsg<<endl;
    }
    // the parent process executes here 
    else 
    {
        cout <<"\nParent is now waiting for child id #"<<pid<<" to complete..\n";
        // close the "read" end b/c we arent using it
        close(fd[READ_END]);
        // write data on pipe
        write(fd[WRITE_END], writeMsg, strlen(writeMsg)+1);
        // close the "write" end
        close(fd[WRITE_END]); 
        // use the "wait" function to wait for the child to complete
        wait(NULL);   
        cout <<"\nThe child process is complete and has terminated!\n";
    }
    
    // NOTE: this message gets displayed twice - Why?!
    cout <<"\nProgram is now exiting...\n";
    
    return 0;
}// http://programmingnotes.org/
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
				
						// ============================================================================
//    Author: Kenneth Perkins
//    Date:   Aug 19, 2013
//    Taken From: http://programmingnotes.org/
//    File:  ForkPipeExample.cpp
//    Description: Demonstrate the use of the fork() and pipe() command for
//        parent and child interprocess communication. 
// ============================================================================
#include <iostream>
#include <cstdlib>
#include <cstring>
#include <sys/wait.h>
#include <unistd.h>
using namespace std;
 
// global variables
const int BUFFER_SIZE = 256;
const int READ_END = 0;
const int WRITE_END = 1; 
 
int main() 
{
    // declare variables
    int fd[2]; // pipe file descriptor
    pid_t pid = -1;     
    char writeMsg[BUFFER_SIZE] = "Greetings From Your Parent!";
    char readMsg[BUFFER_SIZE];
    
    // create a pipe
    if(pipe(fd) < 0) 
    {
        perror("pipe");
        exit(1);
    }
    
    cout <<"\nParent is forking a child." << endl;
 
    // create a duplicate process of this current program
    pid = fork();
    
    // exit if something went wrong with the fork
    if(pid < 0) 
    {
        perror("fork");
        exit(1);
    }
    // this code only gets executed by the child process
    else if(pid == 0)
    {
        cout <<"\nStarting the child process..\n\n";
        // close the "write" end b/c we arent using it
        close(fd[WRITE_END]);
        // get data from the pipe using the read() function
        read(fd[READ_END], readMsg, sizeof(readMsg));
        // close the "read" end
        close(fd[READ_END]);
        cout <<"Message from the parent via the pipe: "<<readMsg<<endl;
    }
    // the parent process executes here 
    else 
    {
        cout <<"\nParent is now waiting for child id #"<<pid<<" to complete..\n";
        // close the "read" end b/c we arent using it
        close(fd[READ_END]);
        // write data on pipe
        write(fd[WRITE_END], writeMsg, strlen(writeMsg)+1);
        // close the "write" end
        close(fd[WRITE_END]); 
        // use the "wait" function to wait for the child to complete
        wait(NULL);   
        cout <<"\nThe child process is complete and has terminated!\n";
    }
    
    // NOTE: this message gets displayed twice - Why?!
    cout <<"\nProgram is now exiting...\n";
    
    return 0;
}// http://programmingnotes.org/

QUICK NOTES:
The highlighted lines are sections of interest to look out for.

The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.

The following is sample output:

Parent is forking a child. Parent is now waiting for child id #12776 to complete..
Starting the child process.. Message from the parent via the pipe: Greetings From Your Parent! Program is now exiting... The child process is complete and has terminated!
Program is now exiting...

C++ || Snippet – How To Use Fork & Execlp For Interprocess Communication

August 19, 2013 admin No comments

The following is sample code which demonstrates the use of the “fork” and “execlp” function calls on Unix based systems.

The “fork” function call creates a new process by duplicating the calling process; or in more simpler terms, it creates a duplicate process (a child) of the calling (parent) process.

This new process, referred to as the child, is an exact duplicate of the calling process, referred to as the parent.

The “execlp” function call is part of a family of functions which replaces a current running process image with a new process image. That means by using the “execlp” function call, you are basically replacing the entire current running process with a new user defined program.

Though fork and execlp are not required to be used together, they are often used in conjunction with one another as a way of creating a new program running as a child of another process.





		
		
			How To Use Fork & Execlp
			
C++
			
			// ============================================================================
//    Author: Kenneth Perkins
//    Date:   Aug 19, 2013
//    Taken From: http://programmingnotes.org/
//    File:  ForkExample.cpp
//    Description: Demonstrate the use of the fork() command for parent and
//       child process manipulation.
// ============================================================================
#include <iostream>
#include <cstdlib>
#include <sys/wait.h>
#include <unistd.h>
using namespace std;

int main()
{
    // declare variable
    pid_t pid = -1;

    cout <<"\nParent is forking a child." << endl;

    // create a duplicate process of this current program
    pid = fork();

    // exit if something went wrong with the fork
    if(pid < 0)
    {
        perror("fork");
        exit(1);
    }
    // this code only gets executed by the child process
    else if(pid == 0)
    {
        cout <<"\nStarting the child process..\n" << endl;
        // NOTE: the "execlp" function replaces the current
        // child process with the terminal "ls" command.
        // the child process finally dies after the "execlp"
        // function is complete
        execlp("/bin/ls", "ls", "-l", NULL); // "ls -l" command
        cout <<"\nNOTE: This never gets executed - Why?!";
    }
    // the parent process executes here
    else
    {
        cout <<"\nParent is now waiting for child id #"<<pid<<" to complete.." << endl;
        // use the "wait" function to wait for the child to complete
        wait(NULL);
        cout <<"\nThe child process is complete and has terminated!\n";
    }

    cout <<"\nParent is now exiting...\n";

    return 0;
}// http://programmingnotes.org/
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
				
						// ============================================================================
//    Author: Kenneth Perkins
//    Date:   Aug 19, 2013
//    Taken From: http://programmingnotes.org/
//    File:  ForkExample.cpp
//    Description: Demonstrate the use of the fork() command for parent and
//       child process manipulation.
// ============================================================================
#include <iostream>
#include <cstdlib>
#include <sys/wait.h>
#include <unistd.h>
using namespace std;
 
int main()
{
    // declare variable
    pid_t pid = -1;
 
    cout <<"\nParent is forking a child." << endl;
 
    // create a duplicate process of this current program
    pid = fork();
 
    // exit if something went wrong with the fork
    if(pid < 0)
    {
        perror("fork");
        exit(1);
    }
    // this code only gets executed by the child process
    else if(pid == 0)
    {
        cout <<"\nStarting the child process..\n" << endl;
        // NOTE: the "execlp" function replaces the current
        // child process with the terminal "ls" command.
        // the child process finally dies after the "execlp"
        // function is complete
        execlp("/bin/ls", "ls", "-l", NULL); // "ls -l" command
        cout <<"\nNOTE: This never gets executed - Why?!";
    }
    // the parent process executes here
    else
    {
        cout <<"\nParent is now waiting for child id #"<<pid<<" to complete.." << endl;
        // use the "wait" function to wait for the child to complete
        wait(NULL);
        cout <<"\nThe child process is complete and has terminated!\n";
    }
 
    cout <<"\nParent is now exiting...\n";
 
    return 0;
}// http://programmingnotes.org/

QUICK NOTES:
The highlighted lines are sections of interest to look out for.

The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.

The following is sample output:

Parent is forking a child. Parent is now waiting for child id #10872 to complete..
Starting the child process.. total 1466 -rw-r--r-- 1 admin admin 468 Apr 27 2012 nautilus-computer.desktop -rwxrwxr-x 1 admin admin 9190 Aug 19 15:17 ForkExample -rw-rw-r-- 1 admin admin 1640 Aug 19 15:17 ForkExample.cpp The child process is complete and has terminated!
Parent is now exiting...

Tag Archives: unix

C++ || Multi Process Server & Client Hash Table Using Thread Pools, Message Queues, & Signal Handlers

C++ || File Copier Using Memory Mapped Files

C++ || Snippet – How To Create A File Of Any Size

C++ || Snippet – How To Display The Size Of A File Using Stat

C++ || Snippet – How To Use Message Queues For Interprocess Communication

C++ || Snippet – Multi-Process Synchronization Producer Consumer Problem Using Threads

C++ || Snippet – How To Create And Use Threads For Interprocess Communication

C++ || Snippet – How To Override The Default Signal Handler (CTRL-C)

C++ || Multi-Hash Interprocess Communication Using Fork, Popen, & Pipes

C++ || Snippet – How To Use Popen & Save Results Into A Character Array

C++ || Serial & Parallel Multi Process File Downloader Using Fork & Execlp

C++ || Snippet – How To Use Fork & Pipe For Interprocess Communication

C++ || Snippet – How To Use Fork & Execlp For Interprocess Communication

Search

Compile Your Code Now

Translate Page

Popular Pages

Recent

Archives

Categories