C++ Read in File Line by Line Function
Solarian Programmer
My programming ramblings
C Programming - read a file line by line with fgets and getline, implement a portable getline version
Posted on Apr 3, 2019 by Paul
In this article, I volition show you how to read a text file line by line in C using the standard C function fgets and the POSIX getline role. At the end of the article, I will write a portable implementation of the getline function that can be used with any standard C compiler.
Reading a file line past line is a trivial trouble in many programming languages, simply not in C. The standard manner of reading a line of text in C is to utilize the fgets function, which is fine if yous know in advance how long a line of text could be.
You tin can observe all the lawmaking examples and the input file at the GitHub repo for this article.
Let's offset with a unproblematic example of using fgets to read chunks from a text file. :
1 #include <stdio.h> two #include <stdlib.h> iii four int main ( void ) { 5 FILE * fp = fopen ( "lorem.txt" , "r" ); 6 if ( fp == Nix ) { 7 perror ( "Unable to open file!" ); 8 exit ( 1 ); ix } 10 11 char clamper [ 128 ]; 12 thirteen while ( fgets ( chunk , sizeof ( chunk ), fp ) != NULL ) { 14 fputs ( clamper , stdout ); xv fputs ( "|* \n " , stdout ); // mark string used to testify where the content of the clamper array has ended 16 } 17 eighteen fclose ( fp ); 19 } For testing the lawmaking I've used a elementary dummy file, lorem.txt. This is a slice from the output of the above program on my machine:
ane ~ $ clang -std=c17 -Wall -Wextra -pedantic t0.c -o t0 2 ~ $ ./t0 3 Lorem ipsum dolor sit amet, consectetur adipiscing elit. 4 |* v Fusce dignissim facilisis ligula consectetur hendrerit. Vestibulum porttitor aliquam luctus. Nam pharetra lorem vel ornare cond|* 6 imentum. 7 |* 8 Praesent et nunc at libero vulputate convallis. Cras egestas nunc vitae eros vehicula hendrerit. Pellentesque in est et sapien |* 9 dignissim molestie. x |* The code prints the content of the chunk array, as filled later on every call to fgets, and a marker cord.
If you lookout carefully, by scrolling the in a higher place text snippet to the right, you can see that the output was truncated to 127 characters per line of text. This was expected because our code can shop an entire line from the original text file only if the line can fit inside our chunk assortment.
What if you need to have the entire line of text available for further processing and not a piece of line ? A possible solution is to copy or concatenate chunks of text in a split up line buffer until we observe the cease of line character.
Let's start by creating a line buffer that will store the chunks of text, initially this will have the same length as the chunk array:
1 #include <stdio.h> 2 #include <stdlib.h> 3 #include <cord.h> four v int chief ( void ) { 6 FILE * fp = fopen ( "lorem.txt" , "r" ); 7 // ... eight 9 char chunk [ 128 ]; 10 11 // Store the chunks of text into a line buffer 12 size_t len = sizeof ( clamper ); 13 char * line = malloc ( len ); fourteen if ( line == NULL ) { 15 perror ( "Unable to allocate memory for the line buffer." ); 16 exit ( 1 ); 17 } 18 19 // "Empty" the cord 20 line [ 0 ] = '\0' ; 21 22 // ... 23 24 } Adjacent, we are going to append the content of the clamper array to the terminate of the line string, until we find the end of line graphic symbol. If necessary, nosotros'll resize the line buffer:
ane #include <stdio.h> two #include <stdlib.h> 3 #include <string.h> 4 v int principal ( void ) { six // ... 7 viii // "Empty" the cord 9 line [ 0 ] = '\0' ; ten xi while ( fgets ( clamper , sizeof ( clamper ), fp ) != Zilch ) { 12 // Resize the line buffer if necessary xiii size_t len_used = strlen ( line ); 14 size_t chunk_used = strlen ( chunk ); 15 sixteen if ( len - len_used < chunk_used ) { 17 len *= ii ; 18 if (( line = realloc ( line , len )) == Goose egg ) { 19 perror ( "Unable to reallocate retentiveness for the line buffer." ); 20 free ( line ); 21 exit ( 1 ); 22 } 23 } 24 25 // Copy the chunk to the end of the line buffer 26 strncpy ( line + len_used , chunk , len - len_used ); 27 len_used += chunk_used ; 28 29 // Bank check if line contains '\northward', if yep process the line of text xxx if ( line [ len_used - 1 ] == '\n' ) { 31 fputs ( line , stdout ); 32 fputs ( "|* \n " , stdout ); 33 // "Empty" the line buffer 34 line [ 0 ] = '\0' ; 35 } 36 } 37 38 fclose ( fp ); 39 free ( line ); 40 41 printf ( " \n\n Max line size: %zd \north " , len ); 42 } Please note, that in the above code, every time the line buffer needs to be resized its capacity is doubled.
This is the result of running the to a higher place code on my auto. For brevity, I kept merely the offset lines of output:
1 ~ $ clang -std=c17 -Wall -Wextra -pedantic t1.c -o t1 2 ~ $ ./t1 iii Lorem ipsum dolor sit amet, consectetur adipiscing elit. 4 |* 5 Fusce dignissim facilisis ligula consectetur hendrerit. Vestibulum porttitor aliquam luctus. Nam pharetra lorem vel ornare condimentum. half dozen |* 7 Praesent et nunc at libero vulputate convallis. Cras egestas nunc vitae eros vehicula hendrerit. Pellentesque in est et sapien dignissim molestie. viii |* 9 Aliquam erat volutpat. Mauris dignissim augue ac purus placerat scelerisque. Donec eleifend ut nibh european union elementum. 10 |* You tin see that, this time, we can impress total lines of text and not stock-still length chunks like in the initial approach.
Permit'south modify the higher up lawmaking in order to print the line length instead of the actual text:
1 // ... two three int primary ( void ) { 4 // ... 5 half dozen while ( fgets ( chunk , sizeof ( chunk ), fp ) != Zippo ) { 7 eight // ... 9 10 // Check if line contains '\n', if yes process the line of text 11 if ( line [ len_used - 1 ] == '\n' ) { 12 printf ( "line length: %zd \n " , len_used ); thirteen // "Empty" the line buffer 14 line [ 0 ] = '\0' ; fifteen } 16 } 17 18 fclose ( fp ); 19 costless ( line ); 20 21 printf ( " \due north\due north Max line size: %zd \n " , len ); 22 } This is the consequence of running the modified code on my machine:
1 ~ $ clang -std=c17 -Wall -Wextra -pedantic t1.c -o t1 two ~ $ ./t1 3 line length: 57 4 line length: 136 5 line length: 147 6 line length: 114 7 line length: 112 8 line length: 95 ix line length: 62 10 line length: 1 11 line length: 428 12 line length: 1 xiii line length: 460 14 line length: 1 fifteen line length: 834 16 line length: 1 17 line length: 821 xviii 19 20 Max line size: 1024 In the side by side example, I volition show you how to use the getline role available on POSIX systems like Linux, Unix and macOS. Microsoft Visual Studio doesn't take an equivalent office, so you won't be able to easily test this example on a Windows system. However, you lot should be able to test it if you are using Cygwin or Windows Subsystem for Linux.
1 #include <stdio.h> 2 #include <stdlib.h> 3 #include <cord.h> iv five int main ( void ) { 6 FILE * fp = fopen ( "lorem.txt" , "r" ); 7 if ( fp == NULL ) { 8 perror ( "Unable to open file!" ); 9 get out ( 1 ); x } 11 12 // Read lines using POSIX part getline 13 // This lawmaking won't work on Windows fourteen char * line = NULL ; 15 size_t len = 0 ; 16 17 while ( getline ( & line , & len , fp ) != - 1 ) { 18 printf ( "line length: %zd \n " , strlen ( line )); xix } 20 21 printf ( " \n\northward Max line size: %zd \n " , len ); 22 23 fclose ( fp ); 24 free ( line ); // getline will resize the input buffer as necessary 25 // the user needs to free the memory when not needed! 26 } Please notation, how simple is to use POSIX's getline versus manually buffering chunks of line like in my previous instance. It is unfortunate that the standard C library doesn't include an equivalent role.
When you use getline, don't forget to free the line buffer when y'all don't need information technology anymore. Besides, calling getline more than once will overwrite the line buffer, make a copy of the line content if you need to keep it for further processing.
This is the outcome of running the in a higher place getline case on a Linux car:
1 ~ $ clang -std=gnu17 -Wall -Wextra -pedantic t2.c -o t2 2 ~ $ ./t2 iii line length: 57 4 line length: 136 five line length: 147 6 line length: 114 7 line length: 112 viii line length: 95 9 line length: 62 x line length: ane 11 line length: 428 12 line length: ane 13 line length: 460 14 line length: 1 fifteen line length: 834 16 line length: 1 17 line length: 821 18 19 20 Max line size: 960 Information technology is interesting to note, that for this particular instance the getline function on Linux resizes the line buffer to a max of 960 bytes. If you run the aforementioned code on macOS the line buffer is resized to 1024 bytes. This is due to the different ways in which getline is implemented on different Unix like systems.
As mentioned before, getline is not present in the C standard library. Information technology could be an interesting practice to implement a portable version of this office. The idea here is not to implement the near performant version of getline, only rather to implement a simple replacement for not POSIX systems.
Nosotros are going to accept the above example and replace the POSIX's getline version with our ain implementation, say my_getline. Plainly, if you are on a POSIX system, you should apply the version provided by the operating system, which was tested past countless users and tuned for optimal performance.
The POSIX getline office has this signature:
i ssize_t getline ( char ** restrict lineptr , size_t * restrict northward , FILE * restrict stream ); Since ssize_t is also a POSIX defined type, usually a 64 bits signed integer, this is how we are going to declare our version:
1 int64_t my_getline ( char ** restrict line , size_t * restrict len , FILE * restrict fp ); In principle we are going to implement the part using the same approach every bit in one of the higher up examples, where I've defined a line buffer and kept copying chunks of text in the buffer until we constitute the end of line character:
1 // This will only accept effect on Windows with MSVC 2 #ifdef _MSC_VER 3 #define _CRT_SECURE_NO_WARNINGS 1 4 #define restrict __restrict five
