Tag Archives: linux
C++ || Snippet – How To Use Message Queues For Interprocess Communication
The following is sample code which demonstrates the use of the msgsnd, msgrcv, and msgget function calls for use with message queues on Unix based systems.
Message queues are used for interprocess communication. They provide two (or more) separate programs (i.e: sender and receiver) with the ability to interact with eachother. This is ideal because the sender and receiver do not need to interact with the message queue at the same time. Messages can be sent at one point in time, be placed on the queue until the receiver is ready for it, and then be removed at another point in time when the program(s) that service the queue are finally ready to receive a message.
This concept is different from threaded and forked process IPC procedures, which often times process data in a more streamlined manner. Rather than following a strict pattern as to when data is to be sent and received, queuing is a mechanism in which messages are held until the receiving application is ready to process them.
The example on this page demonstrates the above use of a message queue to transfer data between two separate programs. The sending program (client.cpp) sends an integer and string variable to the receiving program (server.cpp), which then displays that received data to the screen.
NOTE: The server program must be ran before the client program!
=== 1. Server.cpp (Receiver) ===
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 |
// ============================================================================ // Author: Kenneth Perkins // Date: Oct 4, 2013 // Taken From: http://programmingnotes.org/ // File: Server.cpp // Description: Demonstrate the use of a message queue for interprocess // communication. This program illustrates the server (receiving) end of // the message queue. // ============================================================================ #include <iostream> #include <cstdlib> #include <unistd.h> #include <sys/types.h> #include <sys/ipc.h> #include <sys/msg.h> using namespace std; // Compile & Run // g++ Server.cpp -o Server // ./Server // the structure representing the message queue // & must match the same layout as in the client.cpp struct MsgQueue { // IMPORTANT: every message structure must start with this long messageType; // these variables are optional & you can add // more or less if you wish int someNumber; char buff[100]; }; // message queue flag const int MSG_Q_KEY_FLAG = 0664; // message queue data transfer channel const int MSG_Q_CHANNEL = 26; int main() { // declare variables key_t key = -1; int msqid = -1; MsgQueue msg; // use a random file and a random character to generate // a unique key. The same parameters to this function will // always generate the same value. This is how multiple // processes can connect to the same queue. key = ftok("/bin/ls", 'K'); // was the key allocation successful ? if(key < 0) { perror("ftok error"); exit(1); } // allocate the message queue if it does not already exist. // this function returns the id of the queue. msqid = msgget(key, MSG_Q_KEY_FLAG | IPC_CREAT); // was the allocation a success ? if(msqid < 0) { perror("msgget"); exit(1); } // display info to the screen cout <<"\nThe server has started!\n" <<"\nWaiting for someone to connect to server id #"<<msqid<<" with " <<"the key "<<key<<endl<<endl; // recieve 10 messages from the client for(int x = 0; x < 10; ++x) { // this is where we receive messages: // @param: msqid - the id of the message queue // @param: msg - the message structure which stores the // received message // @param: sizeof(msg) - sizeof(long) - size of the message // excluding the required first member (messageType) which is // required. // @param: MSG_Q_CHANNEL - receive all messages whose type parameter // is set equal to "MSG_Q_CHANNEL" // @param: 0 - flag values (not useful for this example). if(msgrcv(msqid, &msg, sizeof(msg) - sizeof(long), MSG_Q_CHANNEL, 0) < 0) { perror("msgrcv"); exit(1); } // print the received message from the client cout << "someNumber = "<<msg.someNumber<<" buff = "<<msg.buff<<endl; } // finally, deallocate the message queue if(msgctl(msqid, IPC_RMID, NULL) < 0) { perror("msgctl"); exit(1); } cout << "\nServer is now shutting down!\n"; return 0; }// http://programmingnotes.org/ |
=== 2. Client.cpp (Sender) ===
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 |
// ============================================================================ // Author: Kenneth Perkins // Date: Oct 4, 2013 // Taken From: http://programmingnotes.org/ // File: Client.cpp // Description: Demonstrate the use of a message queue for interprocess // communication. This program illustrates the client (sending) end of // the message queue // ============================================================================ #include <iostream> #include <cstdlib> #include <cstring> #include <unistd.h> #include <sys/types.h> #include <sys/ipc.h> #include <sys/msg.h> using namespace std; // Compile & Run // g++ Client.cpp -o Client // ./Client // the structure representing the message queue // & must match the same layout as in the server.cpp struct MsgQueue { // IMPORTANT: every message structure must start with this long messageType; // these variables are optional & you can add // more or less if you wish int someNumber; char buff[100]; }; // message queue flag const int MSG_Q_KEY_FLAG = 0664; // message queue type const int MSG_Q_CHANNEL = 26; int main() { // declare variables key_t key = -1; int msqid = -1; MsgQueue msg; // use a random file and a random character to generate // a unique key. The same parameters to this function will // always generate the same value. This is how multiple // processes can connect to the same queue. key = ftok("/bin/ls", 'K'); // was the key allocation successful ? if(key < 0) { perror("ftok"); exit(1); } // connect to the message queue; fail if the // there is no message queue associated with // this key. This function returns the id of // the queue. msqid = msgget(key, MSG_Q_KEY_FLAG); // was the allocation a success ? if(msqid < 0) { perror("msgget"); exit(1); } // display info to the screen cout <<"\nSuccessfully connected to server id #"<<msqid<<" with " <<"the key "<<key <<"\n\nNow sending messages...."; // send 10 messages to the server for(int x = 0; x < 10; ++x) { // set the message type - this must match // the 4th parameter of msgrcv() in the server.cpp code msg.messageType = MSG_Q_CHANNEL; // place data into the message queue structure to send to the server msg.someNumber = x; strncpy(msg.buff, "Message queues are awesome!", sizeof(msg.buff)); // this is where we send messages: // @param: msqid - the id of the message queue // @param: msg - the message structure which stores the // message to send // @param: sizeof(msg) - sizeof(long) - size of the message // excluding the required first member (messageType) which is // required. // @param: 0 - flag values (not useful for this example). if(msgsnd(msqid, &msg, sizeof(msg) - sizeof(long), 0) < 0) { perror("msgsnd"); exit(1); } } cout << "Sending complete!\n"; return 0; }// http://programmingnotes.org/ |
QUICK NOTES:
The highlighted lines are sections of interest to look out for.
You can view all allocated message queues using the ipcs command. You can delete a message queue from command line using ipcrm -q [KEY SHOWN BY IPCS]. Message queues are a finite resource. If something goes wrong during the execution of your program, you must manually delete all your queues.
The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.
The following is sample output.
SERVER OUTPUT:
The server has started!Waiting for someone to connect to server id #753664 with the key 1258295474
someNumber = 0 buff = Message queues are awesome!
someNumber = 1 buff = Message queues are awesome!
someNumber = 2 buff = Message queues are awesome!
someNumber = 3 buff = Message queues are awesome!
someNumber = 4 buff = Message queues are awesome!
someNumber = 5 buff = Message queues are awesome!
someNumber = 6 buff = Message queues are awesome!
someNumber = 7 buff = Message queues are awesome!
someNumber = 8 buff = Message queues are awesome!
someNumber = 9 buff = Message queues are awesome!Server is now shutting down!
CLIENT OUTPUT:
Successfully connected to server id #753664 with the key 1258295474Now sending messages....Sending complete!
C++ || Multi-Hash Interprocess Communication Using Fork, Popen, & Pipes
The following is another homework assignment which was presented in an Operating Systems Concepts class. Using two pipes, the following is a program which implements the computing of hash values on a file using the MD5, SHA1, SHA224, SHA256, SHA384, and SHA512 hashing algorithms provided on Unix based systems.
REQUIRED KNOWLEDGE FOR THIS PROGRAM
==== 1. OVERVIEW ====
Hash algorithms map large data sets of variable length (e.g. files), to data sets of a fixed length. For example, the contents of a 1GB file may be hashed into a single 128-bit integer. Many hash algorithms exhibit an important property called an avalanche effect – slight changes in the input data trigger significant changes in the hash value.
Hash algorithms are often used for verifying the integrity of files downloaded from the WEB. For example, websites hosting a file usually post the hash value of the file using the MD5 hash algorithm. By doing this, the user can then verify the integrity of the downloaded file by computing the MD5 algorithm on their own, and compare their hash value against the hash value posted on the website. The user will know if the download was valid only if the two hash values match.
==== 2. TECHNICAL DETAILS ====
The following implements a program for computing the hash value of a file using the MD5, SHA1, SHA224, SHA256, SHA384, and SHA512 hashing algorithms provided on Unix based systems.
This program takes the name of the target file being analyzed as a command line argument, and does the following:
1. Check to make sure the file exists.
2. Create two pipes.
3. Create a child process.
4. The parent transmits the name of the file to the child (over the first pipe).
5. The child receives the name of the file and computes the hash of the file using the MD5 algorithm (using Linux program md5sum).
6. The child transmits the computed hash to the parent (over the second pipe) and terminates.
7. The parent receives the hash, prints it, and calls wait().
8. Repeat the same process starting with step 3, but using algorithms SHA1...SHA512.
9. The parent terminates after all hashes have been computed.
The use of the popen function is used in order to launch the above programs and capture their output into a character array buffer.
This program also uses two pipes. The two pipes created are the following:
(1) Parent to child pipe: Used by the parent to transfer the name of the file to the child. The parent writes to this pipe and the child reads it.(2) Child to parent pipe: Used by the child to transfer the computed hashes to the parent. The child writes to this pipe and the parent reads it.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 |
// ============================================================================= // Author: K Perkins // Date: Aug 20, 2013 // Taken From: http://programmingnotes.org/ // File: Multihash.cpp // Description: Hash algorithms map large data sets of variable length (e.g. // files), to data sets of a fixed length. For example, contents of a // 1GB file may be hashed into a single 128-bit integer. Using 2 pipes, // this program implements the computing of hash values on a file using // the MD5, SHA1, SHA224, SHA256, SHA384, and SHA512 hashing algorithms. // // The two pipes created are the following: // (1) Parent to child pipe: Used by the parent to transfer the name of // the file to the child. The parent writes to this pipe and the // child reads it. // (2) Child to parent pipe: Used by the child to transfer the computed // hashes to the parent. The child writes to this pipe and the // parent reads it. // ============================================================================= #include <iostream> #include <fstream> #include <cstdlib> #include <cstring> #include <unistd.h> #include <sys/wait.h> using namespace std; // compile: g++ Multihash.cpp -o Multihash // run: ./Multihash <file name> // parentToChild = pipe for parent to child communication // childToParent = pipe for child to parent communication int parentToChild[2]; int childToParent[2]; // names of the hash programs const char hashAlgs[6][10] = {"md5sum", "sha1sum", "sha224sum", "sha256sum", "sha384sum", "sha512sum"}; // read end of pipe const int READ_END = 0; // write end of pipe const int WRITE_END = 1; // number of hash programs const int NUM_HASH_ALGS = sizeof(hashAlgs)/sizeof(hashAlgs[0]); // maximum length of hash value const int HASH_LENGTH = 1000; // maximum length of file name const int FILENAME_LENGTH = 100; // child function which computes the hash of a file and // returns the value back to the parent // @param 'hashAlg' == the name of the hash program void ComputeHash(const char hashAlg[]); int main(int argc, char* argv[]) { // declare variables ifstream infile; // used to see if file exists char hashValue[HASH_LENGTH]; // hash value which child passes to parent // check to see if theres enough commandline args if(argc < 2) { cerr<<"n** ERROR - NOT ENOUGH ARGUMENTS!n" <<"nUSAGE: "<<argv[0]<<" <FILE NAME>n"; exit(1); } // try to open file infile.open(argv[1]); // if file doesnt exist, then exit if(infile.fail()) { cerr<<"n** ERROR!n" <<"Cant find the file ""<<argv[1]<<""!nn"; exit(1); } infile.close(); // start the piping process for(int currentAlg = 0; currentAlg < NUM_HASH_ALGS; ++currentAlg) { // create pipes if((pipe(parentToChild) < 0) || (pipe(childToParent) < 0)) { cerr << "npipe failedn"; exit(1); } // create & fork a child pid_t pid = fork(); // make sure fork succeeded if(pid < 0) { cerr << "nfork failedn"; exit(1); } // child process else if(pid == 0) { ComputeHash(hashAlgs[currentAlg]); } // parent process else { // close selected pipes ends close(parentToChild[READ_END]); close(childToParent[WRITE_END]); // pass the filename to the child write(parentToChild[WRITE_END], argv[1], strlen(argv[1])+1); close(parentToChild[WRITE_END]); // read the incoming hash value from the child read(childToParent[READ_END], hashValue, sizeof(hashValue)); close(childToParent[READ_END]); // wait for child to complete wait(NULL); // display current hash to the screen cout<<"Hash Algorithm #"<<currentAlg+1<<":n"<<hashAlgs[currentAlg] <<" - HASH VALUE: "<<hashValue<<endl; // reset hash buffer memset(hashValue, (char)NULL, sizeof(hashValue)); } } cerr<<"The parent process is now exiting...n"; return 0; }// end of main void ComputeHash(const char hashAlg[]) { // declare variables char recievedFileName[FILENAME_LENGTH]; // saves recieved filename char cmdLine[FILENAME_LENGTH]; // saves command line for popen char hashOutput[HASH_LENGTH]; // saves final hash output FILE* popenOutput; // popen file pointer // close selected pipes ends close(parentToChild[WRITE_END]); close(childToParent[READ_END]); // get filename from parent read(parentToChild[READ_END], recievedFileName, sizeof(recievedFileName)); close(parentToChild[READ_END]); // construct command line argument to pass to popen strncpy(cmdLine, hashAlg, sizeof(cmdLine)); strncat(cmdLine, " ", sizeof(cmdLine)); strncat(cmdLine, recievedFileName, sizeof(cmdLine)); // get has value for current hash prog popenOutput = popen(cmdLine, "r"); // make sure that popen succeeded if(!popenOutput) { cerr<<"npopen failedn"; exit(1); } // set hash buffer to all NULLS memset(hashOutput, (char)NULL, sizeof(hashOutput)); // read program output into buffer fread(hashOutput, sizeof(char), sizeof(char)*sizeof(hashOutput), popenOutput); // close popen buffer pclose(popenOutput); // pass hash value back to parent write(childToParent[WRITE_END], hashOutput, strlen(hashOutput)+1); close(childToParent[WRITE_END]); // exit 'AKA' kill child exit(0); }// http://programmingnotes.org/ |
QUICK NOTES:
The highlighted lines are sections of interest to look out for.
The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.
Using the following example input file located here, the following is sample output:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
Hash Algorithm #1: md5sum - HASH VALUE: cd2f02cc5e50bd08d83cef630a32d7d6 INPUT_Dictionary_programmingnotes_freeweq_com.txt Hash Algorithm #2: sha1sum - HASH VALUE: 1e1d2fd77b331fc2b2f24822fdf2133f3678d662 INPUT_Dictionary_programmingnotes_freeweq_com.txt Hash Algorithm #3: sha224sum - HASH VALUE: bf69da0f9f9990cece8dd6800a25a9faca5381c0c62e0667c258e8d5 INPUT_Dictionary_programmingnotes_freeweq_com.txt Hash Algorithm #4: sha256sum - HASH VALUE: a8cd7a082ce571f7e66bc4d5eea8f71e7e455735922c90c060580b51d43593ec INPUT_Dictionary_programmingnotes_freeweq_com.txt Hash Algorithm #5: sha384sum - HASH VALUE: a564df501470a43eebb51c28f83ab07090e39c434fc40bf496622f536c525140301633d6513dec83ea512bcb38c4e2e6 INPUT_Dictionary_programmingnotes_freeweq_com.txt Hash Algorithm #6: sha512sum - HASH VALUE: e686a709b3025f8b21fd6d40ce92f741dd8644335e935baced86098ff3e7278b5443e28f02fa886e7c0cd391ce6e8aa91842a6de29535a40453d41862a0bc1c7 INPUT_Dictionary_programmingnotes_freeweq_com.txt The parent process is now exiting... |
C++ || Snippet – How To Use Popen & Save Results Into A Character Array
The following is sample code which demonstrates the use of the “popen” function call on Unix based systems.
The “popen” function call opens a process by creating a pipe, forking, and invoking the shell. It does this by executing the command specified by the incoming string function parameter. It creates a pipe between the calling program and the executed command, and returns a pointer to a stream that can be used to either read from or write to the pipe.
The following example demonstrates how to save the results of the popen command into a char array.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 |
// ============================================================================ // Author: Kenneth Perkins // Date: Aug 20, 2013 // Taken From: http://programmingnotes.org/ // File: PopenExample.cpp // Description: Demonstrate the use of the popen() command and demonstrate // how to save the results into a char array. // ============================================================================ #include <iostream> #include <cstdio> #include <cstring> #include <cstdlib> using namespace std; // the maximum output size const int MAX_OUTPUT_SIZE = 1000; int main(int argc, char* argv[]) { // declare variables char buffer[MAX_OUTPUT_SIZE]; // a char array to display the popen output FILE* progOutput; // a file pointer representing the popen output // launch the "md5sum" program to compute the MD5 hash of // the file "/bin/ls" and save it into the file pointer progOutput = popen("md5sum /bin/ls", "r"); // make sure that popen succeeded if (!progOutput) { perror("popen"); exit(1); } // reset buffer to all NULLS memset(buffer, (char)NULL, sizeof(buffer)); // read the popen output into the char array buffer if (fread(buffer, sizeof(char), sizeof(char) * sizeof(buffer), progOutput) < 0) { perror("fread"); exit(1); } // close the file pointer representing the popen output if (pclose(progOutput) < 0) { perror("pclose"); exit(1); } cout << "Program output: " << buffer << endl; return 0; }// http://programmingnotes.org/ |
QUICK NOTES:
The highlighted lines are sections of interest to look out for.
The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.
The following is sample output:
Program output: fa97c59cc414e42d4e0e853ddf5b4745 /bin/ls
C++ || Serial & Parallel Multi Process File Downloader Using Fork & Execlp
The following is another homework assignment which was presented in an Operating Systems Concepts class. The following are two multi-process programs using commandline arguments, which demonstrates more practice using the fork() and execlp() system calls on Unix based systems.
==== 1. OVERVIEW ====
File downloaders are programs used for downloading files from the Internet. The following programs listed on this page implement two distinct type of multi-process downloaders:
1. a serial file downloader which downloads files one by one.
2. a parallel file downloader which dowloads multiple files in parallel.
In both programs, the parent process first reads a file via the commandline. This file which is read is the file that contains the list of URLs of the files to be downloaded. The incoming url file that is read has the following format:
[URL1]
[URL2]
.
.
.
[URLN]
Where [URL] is an http internet link with a valid absolute file path extension.
(i.e: http://newsimg.ngfiles.com/270000/270173_0204618900-cc-asmbash.jpg)
After the url file is parsed, next the parent process forks a child process. Each created child process uses the execlp() system call to replace its executable image with that of the “wget” program. The use of the wget program performs the actual file downloading.
==== 2. SERIAL DOWNLOADER ====
The serial downloader downloads files one at a time. After the parent process has read and parsed the incoming url file from the commandline, the serial downloader proceeds as follows:
1. The parent forks off a child process.
2. The child uses execlp("/usr/bin/wget", "wget", [URL STRING1], NULL) system call in order to replace its program with wget program that will download the first file in urls.txt (i.e. the file at URL).
3. The parent executes a wait() system call until the child exits.
4. The parent forks off another child which downloads the next file specified in url.txt.
5. Repeat the same process until all files are downloaded.
The following is implemented below:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 |
// ============================================================================ // Author: Kenneth Perkins // Date: Aug 19, 2013 // Taken From: http://programmingnotes.org/ // File: Serial.cpp // Description: File downloaders are programs used for downloading files // from the Internet. Using the fork() & execlp("wget") command, the // following is a multi-process serial file downloader which reads an // input file containing url file download links as a commandline // argument and downloads the files located on the internet one by one. // ============================================================================ #include <iostream> #include <cstring> #include <fstream> #include <cstdlib> #include <unistd.h> #include <sys/wait.h> using namespace std; // compile & run // g++ Serial.cpp -o Serial // ./Serial urls.txt int main(int argc, char* argv[]) { // declare variables pid_t pid = -1; int urlNumber = 0; char urlName[256]; ifstream infile; // check if theres enough command line args if(argc < 2) { cout <<"\nERROR -- NOT ENOUGH ARGS!" <<"\n\nUSAGE: "<<argv[0]<<" <file containing url downloads>\n\n"; exit(1); } // try to open the file containing the download url links // exit if the url file is not found infile.open(argv[1]); if(infile.fail()) { cout <<"\nERROR -- "<<argv[1]<<" NOT FOUND!\n\n"; exit(1); } // get download url links from the file while(infile.getline(urlName, sizeof(urlName))) { ++urlNumber; // fork another process pid = fork(); if(pid < 0) { // ** error occurred perror("fork"); exit(1); } else if(pid == 0) { // ** child process cout <<endl<<"** URL #"<<urlNumber <<" is currently downloading... **\n\n"; execlp("/usr/bin/wget", "wget", urlName, NULL); } else { // ** parent process // parent will wait for the child to complete wait(NULL); cout <<endl<<"-- URL #"<<urlNumber<<" is complete! --\n"; } } infile.close(); cout <<endl<<"The parent process is now exiting...\n"; return 0; }// http://programmingnotes.org/ |
Since the serial downloader downloads files one at a time, that can become very slow. That is where the parallel downloader comes in handy!
==== 3. PARALLEL DOWNLOADER ====
The parallel downloader downloads files all at once and is implemented much like the serial downloader. The parallel downloader proceeds as follows:
1. The parent forks off n children, where n is the number of URLs in url.txt.
2. Each child executes execlp("/usr/bin/wget", "wget", [URL STRING], NULL) system call where eachis a distinct URL in url.txt.
3. The parent calls wait() (n times in a row) and waits for all children to terminate.
4. The parent exits.
The following is implemented below:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 |
// ============================================================================ // Author: Kenneth Perkins // Date: Aug 19, 2013 // Taken From: http://programmingnotes.org/ // File: Parallel.cpp // Description: File downloaders are programs used for downloading files // from the Internet. Using the fork() & execlp("wget") command, the // following is a multi-process parallel file downloader which reads an // input file containing url file download links as a commandline // argument and downloads the files located on the internet all at once. // ============================================================================ #include <iostream> #include <cstring> #include <fstream> #include <cstdlib> #include <unistd.h> #include <sys/wait.h> using namespace std; // compile & run // g++ Parallel.cpp -o Parallel // ./Parallel urls.txt int main(int argc, char* argv[]) { // declare variables pid_t pid = -1; int urlNumber = 0; char urlName[256]; ifstream infile; // check if theres enough command line args if(argc < 2) { cout <<"\nERROR -- NOT ENOUGH ARGS!" <<"\n\nUSAGE: "<<argv[0]<<" <file containing url downloads>\n\n"; exit(1); } // try to open the file containing the download url links // exit if the url file is not found infile.open(argv[1]); if(infile.fail()) { cout <<"\nERROR -- "<<argv[1]<<" NOT FOUND!\n\n"; exit(1); } // get download url links from the file while(infile.getline(urlName, sizeof(urlName))) { ++urlNumber; // fork another process pid = fork(); if(pid < 0) { // ** error occurred perror("fork"); exit(1); } else if(pid == 0) { // ** child process cout <<endl<<"** URL #"<<urlNumber <<" is currently downloading... **\n\n"; execlp("/usr/bin/wget", "wget", urlName, NULL); } } infile.close(); while(urlNumber > 0) { // ** parent process // parent will wait for the child to complete wait(NULL); cout <<endl<<"-- URL #"<<urlNumber<<" is complete! --\n"; --urlNumber; } cout <<endl<<"The parent process is now exiting...\n"; return 0; }// http://programmingnotes.org/ |
QUICK NOTES:
The highlighted lines are sections of interest to look out for.
The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.
Also note, while the parallel downloader executes, the outputs from different children may intermingle.
C++ || Snippet – How To Use Fork & Pipe For Interprocess Communication
The following is sample code which demonstrates the use of the fork, read, and write function calls for use with pipes on Unix based systems.
A pipe is a mechanism for interprocess communication. Data written to a pipe by one process can be read by another process. Creating a pipe is achieved by using the pipe function, which creates both the reading and writing ends of the pipe file descriptor.
In typical use, a parent process creates a pipe just before it forks one or more child processes. The pipe is then used for communication between either the parent or child processes, or between two sibling processes.
A real world example of this kind of communication can be seen in all operating system terminal shells. When you type a command in a shell, it will spawn the executable represented by that command with a call to fork. A pipe is opened to the new child process, and its output is read and printed by the terminal.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 |
// ============================================================================ // Author: Kenneth Perkins // Date: Aug 19, 2013 // Taken From: http://programmingnotes.org/ // File: ForkPipeExample.cpp // Description: Demonstrate the use of the fork() and pipe() command for // parent and child interprocess communication. // ============================================================================ #include <iostream> #include <cstdlib> #include <cstring> #include <sys/wait.h> #include <unistd.h> using namespace std; // global variables const int BUFFER_SIZE = 256; const int READ_END = 0; const int WRITE_END = 1; int main() { // declare variables int fd[2]; // pipe file descriptor pid_t pid = -1; char writeMsg[BUFFER_SIZE] = "Greetings From Your Parent!"; char readMsg[BUFFER_SIZE]; // create a pipe if(pipe(fd) < 0) { perror("pipe"); exit(1); } cout <<"\nParent is forking a child." << endl; // create a duplicate process of this current program pid = fork(); // exit if something went wrong with the fork if(pid < 0) { perror("fork"); exit(1); } // this code only gets executed by the child process else if(pid == 0) { cout <<"\nStarting the child process..\n\n"; // close the "write" end b/c we arent using it close(fd[WRITE_END]); // get data from the pipe using the read() function read(fd[READ_END], readMsg, sizeof(readMsg)); // close the "read" end close(fd[READ_END]); cout <<"Message from the parent via the pipe: "<<readMsg<<endl; } // the parent process executes here else { cout <<"\nParent is now waiting for child id #"<<pid<<" to complete..\n"; // close the "read" end b/c we arent using it close(fd[READ_END]); // write data on pipe write(fd[WRITE_END], writeMsg, strlen(writeMsg)+1); // close the "write" end close(fd[WRITE_END]); // use the "wait" function to wait for the child to complete wait(NULL); cout <<"\nThe child process is complete and has terminated!\n"; } // NOTE: this message gets displayed twice - Why?! cout <<"\nProgram is now exiting...\n"; return 0; }// http://programmingnotes.org/ |
QUICK NOTES:
The highlighted lines are sections of interest to look out for.
The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.
The following is sample output:
Parent is forking a child.
Parent is now waiting for child id #12776 to complete..Starting the child process..
Message from the parent via the pipe: Greetings From Your Parent!
Program is now exiting...
The child process is complete and has terminated!
Program is now exiting...
C++ || Snippet – How To Use Fork & Execlp For Interprocess Communication
The following is sample code which demonstrates the use of the “fork” and “execlp” function calls on Unix based systems.
The “fork” function call creates a new process by duplicating the calling process; or in more simpler terms, it creates a duplicate process (a child) of the calling (parent) process.
This new process, referred to as the child, is an exact duplicate of the calling process, referred to as the parent.
The “execlp” function call is part of a family of functions which replaces a current running process image with a new process image. That means by using the “execlp” function call, you are basically replacing the entire current running process with a new user defined program.
Though fork and execlp are not required to be used together, they are often used in conjunction with one another as a way of creating a new program running as a child of another process.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 |
// ============================================================================ // Author: Kenneth Perkins // Date: Aug 19, 2013 // Taken From: http://programmingnotes.org/ // File: ForkExample.cpp // Description: Demonstrate the use of the fork() command for parent and // child process manipulation. // ============================================================================ #include <iostream> #include <cstdlib> #include <sys/wait.h> #include <unistd.h> using namespace std; int main() { // declare variable pid_t pid = -1; cout <<"\nParent is forking a child." << endl; // create a duplicate process of this current program pid = fork(); // exit if something went wrong with the fork if(pid < 0) { perror("fork"); exit(1); } // this code only gets executed by the child process else if(pid == 0) { cout <<"\nStarting the child process..\n" << endl; // NOTE: the "execlp" function replaces the current // child process with the terminal "ls" command. // the child process finally dies after the "execlp" // function is complete execlp("/bin/ls", "ls", "-l", NULL); // "ls -l" command cout <<"\nNOTE: This never gets executed - Why?!"; } // the parent process executes here else { cout <<"\nParent is now waiting for child id #"<<pid<<" to complete.." << endl; // use the "wait" function to wait for the child to complete wait(NULL); cout <<"\nThe child process is complete and has terminated!\n"; } cout <<"\nParent is now exiting...\n"; return 0; }// http://programmingnotes.org/ |
QUICK NOTES:
The highlighted lines are sections of interest to look out for.
The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.
The following is sample output:
Parent is forking a child.
Parent is now waiting for child id #10872 to complete..Starting the child process..
total 1466
-rw-r--r-- 1 admin admin 468 Apr 27 2012 nautilus-computer.desktop
-rwxrwxr-x 1 admin admin 9190 Aug 19 15:17 ForkExample
-rw-rw-r-- 1 admin admin 1640 Aug 19 15:17 ForkExample.cppThe child process is complete and has terminated!
Parent is now exiting...
Python || How To Install PyPdf Using Python 3
“pyPdf” is a pure Python library built as a PDF toolkit. It is capable of:
• Extracting document information (title, author, ...),
• Splitting documents page by page,
• Merging documents page by page,
• Cropping pages,
• Merging multiple pages into a single page,
• Encrypting and decrypting PDF files.
By being pure Python, it should run on any Python platform without any dependencies on external libraries. It can also work entirely on StringIO objects rather than file streams, allowing for PDF manipulation in memory. It is therefore a useful tool for websites that manage or manipulate PDFs.
=== 1. INSTALLING PYPDF ===
A Windows installer for pyPdf is available here. At the time of this writing, the installer that was listed on the download page was titled “pyPdf-1.13.win32.exe.”
Note: If you are using Python 2 and want to install pyPdf, the Windows installer available on the download page should be all you need. No further installation instructions are necessary.
Next, follow the command prompts from the installer and wait!
When the pyPdf installer is completed, you should see the newly installed files which correspond to this module located in the following directory:
C:\Python32\Lib\site-packages\pyPdf
Note: Python 3.2 is installed on my system, and the above directory reflects that. The Python directory may be named something different on your individual system depending on the Python version you are using.
If everything installed correctly, proceed to the next step.
=== 2. UPDATE EXISTING PYPDF FILES ===
pyPdf was originally written for Python 2, but a Python 3 compatible branch has since been made available. The updated files can be found here, and enable pyPdf to be integrated with Python 3.
To update these new Python 3 files with the old Python 2 files, locate the following directory on your system:
C:\Python32\Lib\site-packages\pyPdf
Here is a sample screenshot demonstrating the files which resides in the above directory: (click to enlarge)
Next, update all of the old Python 2 files that’s listed in the above directory with the new Python 3 files that’s listed on this page.
One way to update the files that’s currently stored on your computer is to locate the file on your system with the exact same file name as listed here, and copy/paste the contents of the new file into the contents of the old file that’s currently stored on your computer.
Or you can download all of the files at once here, and move/replace these new files with the existing files that’s currently located on your system.
=== 3. UBUNTU USERS ===
If you are an Ubuntu Linux user and want to install pypdf, open the terminal and run the following command:
NOTE: Replace “python3.2” with whatever version of python that’s installed on your system.
sudo mkdir /usr/local/lib/python3.2/dist-packages/pyPdf
Next, copy and paste the files located in the above download link into the following directory:
/usr/local/lib/python3.2/dist-packages/pyPdf
Once the following steps are completed, you should now be ready to use pyPdf with Python 3 programs!
Hi, I'm a software engineer. I write articles about programming and design in my spare time.
