Reading a file in Python without the three common mistakes

Reading a file is one of the first genuinely useful things you do in Python. The mechanics take one line. The bugs take longer, and they are nearly always one of the same three.

The modern way to open a file

The line to learn is the with open(...) form:

with open("data.txt") as f:

Everything indented under that line has the file available as f, and the moment the block ends Python closes the file for you, even if something goes wrong inside. That automatic close is the reason this form has quietly replaced the older open and close pair.

Mistake one: forgetting to close, the old way

If you write f = open("data.txt") on its own, you now own that open file, and you have to remember to close it. Forget, and on a long-running program the open files pile up until the operating system refuses more. The with form removes the problem entirely by closing for you. There is almost no reason to open a file any other way.

Mistake two: reading the whole file when you wanted lines

f.read() gives you the entire file as one long string. That is occasionally what you want and usually not. If you meant to work line by line, loop over the file directly:

for line in f:

This reads one line at a time and does not load a huge file into memory all at once. On a small file it makes no difference. On a file too big to fit in memory it is the difference between working and crashing.

Mistake three: the invisible newline

Each line you read still carries the newline character from the end of the line in the file. Compare a line straight from the file against the text "quit" and it will not match, because the line is really "quit\n". Strip it first with .strip(), which removes whitespace and the trailing newline from both ends.

A safe shape to copy

Put together, a reliable read looks like this in plain terms: open the file with with, loop over it one line at a time, strip each line, and then do your work. That shape handles the close, the memory and the newline in one go.

Turning a line into fields

Once you have a clean line, you usually want to split it into pieces. line.split(",") breaks a comma line into a list of values. This is the seam between reading a file and actually using it.

Where to go next

File handling is the point where your programs start touching the real world instead of only made-up values. Reading Files covers the patterns above in order with examples you can run, and Saving Your Work (File Handling) takes it further into writing files back out safely.